Partek Flow Documentation

Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

We also provide live assistance via GoTo meeting or TeamViewer if you are uncomfortable with us accessing your EC2 instance directly. Before contacting us, please run ./partek_flow/flowstatus.sh to send us logs and other information that will assist us with your support request.

 

EC2 costs and our General recommendations:

With newer EC2 instance types, it is possible to change the instance type of an already deployed Flow EC2 server. We recommend doing several rounds of benchmarks with production-sized workloads and evaluate if the resources allocated to your Flow server are sufficient. You may find that reducing resources allocated to the Flow server may come with significant cost savings, but may cause UI responsiveness and job run-times to reach unacceptable levels. Once you have found an instance type that works, you may wish to use "reserved instance" pricing which is significantly cheaper than dynamic instance pricing. Reserved instances come with 1 or 3 year usage terms. Please see the EC2 Reserved Instance Marketplace to sell or purchase existing reserved instances at reduced rates. 

The network performance of the EC2 instance type becomes an important factor if your pr

Want HVM

No EBS optimized surcharge

Want placement group support

No T class servers as don’t want to slow responsiveness

We don’t use instance store since all data is lost after instance stop. Too risky.

 

EBS-only options:

primary usage of Flow is for alignment. For this use case, one will have to move copious amounts of data back (input fastq files) and forth (output bam files) between the Flow server and the end users, thus it is important to have as what AWS refers to as "High network performance" which for most cases is around 1 Gb/s. If focus is primarialy on downstream analysis and visualization (e.g. the primary input files are ADAT) then network performance is less of a concern.

We recommend HVM vitualization as we have not seen any performance impact from using them and non-HVM instance types can come with significant deployment barriers.

Make sure your instance is "EBS optimized" by default and you are not charged a surcharge for EBS optimization.

"T-class" servers although cheap may slow responsiveness for the Flow server and generally do not provide sufficient resources.

We do not recommend placing any data on "instance store" volumes since all data is lost on that volume after an instance stops. This is too risky as their are cases where user tasks can take up unexpected amounts of memory forcing a server stop/reboot.

 

Latest (April 2017) EC2 costs:

Pricing and resource table:

http://www.ec2instances.info/

TypeMemCoresEBS throughput (netw rate)Monthly cost
m4.large 8.0 GB2 vCPUs56.25 MB/s  M$78.840
r4.large15.25 GB2 vCPUs 50 MB/s  H(10G int) $97.09
*m4.xlarge16.0 GB4 vCPUs93.75 MB/s  H$156.950
r4.xlarge30.5 GB4 vCPUs100 MB/s     H$194.180
*m4.2xlarge32.0 GB8 vCPUs 125 MB/s     H $314.630
r4.2xlarge61.0 GB8 vCPUs200 MB/s     H(10G int)$388.360

...

Multi-node: less-beefy head node

 

Other Notes:

 

Pricing and resource table:

http://www.ec2instances.info/

 

ECU Vs. vCPU:

each vCPU is a hyperthread of an Intel Xeon core

...