Partek Flow Documentation

Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Install required packages for Flow:

$ sudo apt-get update

$ sudo apt-get install  python python perl make gcc g++ zlib1g libbz2-1.0 libstdc++6 libgcc1 libncurses5 libsqlite3-0 libfreetype6 libpng12-0 zip unzip libgomp1 libxrender1 libxtst6 libxi6 debconf

Now Install Flow: (see generic install instructions as well)

Note: Make sure you are running as the ubuntu user.

$ cd (we will install Flow to ubuntu's home directory)

$ wget --content-disposition packages.partek.com/linux/flow-release

$ unzip PartekFlow*.zip

$ ./partek_flow/start_flow.sh

Flow has finished loading when you see

...

INFO: Server startup in xxxxxxx ms

...

in the partek_flow/logs/catalina.out log file. This takes ~30 seconds.

Open Flow with a web browser: http://elastic.ip:8080/

...

Leave the library file directory at its default location and check that the free space listed for this directory matches is consistent with what was allocated for the ST1 EBS volume.

Done! You are Flow is ready to use Flow.

 

Anchor
aws-support
aws-support
Partek AWS Support

After the EC2 instance is provisioned, we are happy to assist with setting up Partek Flow or other issues you encounter with the usage of Partek Flow. The quickest way to receive help is to allow us remote access to your server by sending us Flowus Flow-Key.pem and amending the SSH rule for Flow-SG to include access from IP 97.84.41.194 (Partek HQ). We recommend sending us Flowus Flow-Key.pem via secure means. The easiest way to do this is with the following command:

$ curl -F "file=@FlowKey.pem" https://installfeedback.partek.com/fupload

We also provide live assistance via GoTo meeting or TeamViewer if you are uncomfortable with us accessing your EC2 instance directly. Before contacting us, please run $ ./partek_flow/flowstatus.sh to send us logs and other information that will assist us with your support request.

 

General recommendations

...

With newer EC2 instance types, it is possible to change the instance type of an already deployed Partek Flow EC2 server. We recommend doing several rounds of benchmarks with production-sized workloads and evaluate if the resources allocated to your Partek Flow server are sufficient. You may find that reducing resources allocated to the Partek Flow server may come with significant cost savings, but can cause UI responsiveness and job run-times to reach unacceptable levels. Once you have found an instance type that works, you may wish to use "reserved instance" pricing which is significantly cheaper than dynamic on demand instance pricing. Reserved instances come with 1 or 3 year usage terms. Please see the EC2 Reserved Instance Marketplace to sell or purchase existing reserved instances at reduced rates. 

The network performance of the EC2 instance type becomes an important factor if your the primary usage of Partek Flow is for alignment. For this use case, one will have to move copious amounts of data back (input fastq files) and forth (output bam files) between the Partek Flow server and the end users, thus it is important to have as what AWS refers to as "High high network performance" which for most cases is around 1 Gb/s. If focus is primarily on downstream analysis and visualization (e.g. the primary input files are ADAT) then network performance is less of a concern.

...

Make sure your instance is "EBS optimized" by default and you are not charged a surcharge for EBS optimization.

"T-class " servers, although cheap, may slow responsiveness for the Flow server and generally do not provide sufficient resources.

We do not recommend placing any data on "instance store" volumes since all data is lost on those volumes after an instance stops. This is too risky as there are cases where user tasks can take up unexpected amounts of memory forcing a server stop/reboot.

...

Network performance values for US-EAST-1 correspond to: Low ~ 50Mb/s, Medium ~ 300Mb/s, High ~ 1Gb/s.

 

Anchor
ebs-volumes
ebs-volumes
EBS Volumes

Volume typeChoice of a volume type and size:

This is dependent on the type of workload. For must users, the Partek Flow server tasks will be are alignment-heavy so we recommend a throughput optimized HDD (ST1) EBS volume since most aligner operations are sequential in nature. For workloads that focus primarily on downstream analysis, a general purpose SSD volume will suffice but the costs are greater. For those who focus on alignment or host several users the storage requirements can be high. ST1 EBS volumes have the following characteristics:

...

Note that EBS volumes can be grown or performance characteristics changed. To minimize costs, start with a smaller EBS volume allocation of 0.5 - 2 TB as most mature Flow installations generate roughly this amount of data. When necessary, the EBS volume and the underlying file system can be grown on-line (making ext4 a good choice). Shrinking is also possible, but may require the Flow server to be off-line.

...