Flow Documentation

Partek Flow Documentation

Page tree
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 41 Next »

Partek® Flow® is a web-based application for genomic data analysis and visualization. It can be installed on a desktop computer, computer cluster or cloud. Users can then access Partek Flow from any browser-enabled device, such as a personal computer, tablet or smartphone. 

This guide covers the following topics:

Minimum System Requirements

Partek Flow requires the following for successful installation on a Linux-based system:

  • Linux Ubuntu® 12.04, Redhat® 6, CentOS® 6, or newer
  • 64-bit 2GHz quad-core processor
  • 32GB of onboard memory
  • 1TB of storage available for data
  • /opt – 100GB of storage available for installation

Partek Flow can also be installed on Macintosh computers with comparable hardware specifications. Please refer to the the Partek Flow Mac OS X installation guide for detailed installation instructions.

Note that some analyses have higher requirements, e.g. to run the STAR aligner on a reference genome of size ~3 GB (such as human, mouse or rat), 16 cores with 38 GB of RAM are required. Input sample file size can also impact memory usage, which is particularly the case for TopHat alignments.

Increasing hardware resources (cores, RAM, disk space, and speed) will allow for faster processing of more samples.

Installing Partek Flow

This section describes the most common type of installation of Partek Flow using the Linux package managers. Note that they are not applicable to the following cases:

  • If you are installing on a computer cluster or on the cloud (see the section Installation on a Computer Cluster for contact information)
  • If you had installed older versions of Partek Flow using a zip file (see the System Administrator Guide section at the end of this document to verify the installation method and to switch to a package manager)

By default, Partek Flow is installed under /opt/partek_flow and temporary files are housed in /opt/partek_flow/temp.

The installation procedure varies per Linux distribution. To check your distribution, open a terminal and run:
$ cat /etc/issue

Installation on Debian/Ubuntu Distributions

  1. Add the public key for the Partek package repository*
    $ sudo apt-key adv --keyserver keys.gnupg.net --recv-keys C82B61BF

  2. Add the Partek package list to your repository*
    $ sudo wget -P /etc/apt/sources.list.d/{+}http://packages.partek.com/debian/partek-flow.list+http://packages.partek.com/debian/partek-flow.list
    *Steps 1 and 2 only need to be performed once prior to the first installation. Re-installation and updates do not require this step.

  3. Update the list of available packages
    $ sudo apt-get update

  4. Install Partek Flow
    $ sudo apt-get install partekflow
    When asked to continue, type the letter Y and press Enter.
    During the installation, you will be prompted for the Flow server port (Figure 1). Unless necessary, accept the default HTTP port: 8080 by pressing Enter.

    Figure 1. Configuring HTTP port for Partek Flow during installation

  5. If additional configuration is needed, use the reconfigure command below. This can be run any time after Partek Flow is installed. For details regarding each setting, contact the Partek Licensing Department. In most cases, this step can be skipped.
    $ sudo dpkg-reconfigure partekflow

  6. (Optional) To manually configure Partek Flow or to set additional advanced options or environment variables, edit the following configuration file:
    /etc/partekflow.conf
    Note that all changes made by the reconfigure command in step 5 are also stored in this configuration file.

  7. Start the Partek Flow server
    $ sudo service partekflowd start
    A message should indicate that Partek Flow is now running:Starting Partek Flow server: OK
    Step 7 needs to be performed only once after installation. Partek Flow will start automatically whenever the server restarts.

Installation on RedHat/CentOS Distributions

  1. Retrieve the Partek yum repo configuration
    $ sudo wget -P /etc/yum.repos.d
    {+}http://packages.partek.com/redhat/stable/partekflow.repo+http://packages.partek.com/redhat/stable/partekflow.repo
    Step 1 needs to be performed only once prior to the first installation. Re-installation and updates do not require this step.

  2. Install Partek Flow
    $ sudo yum install partekflowXXX

  3. When asked to continue, type the letter Y and press Enter

  4. (Optional) To manually configure Partek Flow or to set additional advanced options or environment variables, edit the file located at:
    /etc/partekflow.conf

  5. Start the Partek Flow server
    $ sudo service partekflowd restart
    The following message indicates that Partek Flow is now running:Starting Partek Flow server: OK
    Step 5 needs to be performed only once after installation. Partek Flow will start automatically whenever a server restarts.

Installation on a Computer Cluster

The installation procedure on a computer cluster is highly dependent on the type of computer cluster and the environment it is located. We can to support a large array of Linux distributions and configurations. Please consult with Partek Licensing Support (licensing@partek.com) for additional information.

End User Tools

A user can access Partek Flow using a web browser on any browser-enabled device, such as a personal computer, tablet, smartphone etc. We recommend using Google Chrome™ or, alternatively, Mozilla Firefox™. The screen resolution should be set to 1024 × 768 pixels or higher. This is particularly important for the use of visualization tools such as Chromosome Viewer.

Setting up the License Manager (for floating licenses)

This section is required for users who purchased a Partek Flow floating license. If you have purchased a node locked license, skip this section and proceed to Launching Partek Flow for the first time.
FlexLM is a software license manager required to provide floating licenses to multiple end users of Partek Flow. The instructions below are intended for customers planning on running the license server on a Linux host.

Prerequisites

FlexLM may require the linux standard base utilities to run on Ubuntu. If you receive the message "No such file or directory", then run the following commands as root:
$ apt-get update
$ apt-get install lsb-core

Linux FlexLM server installation instructions

  1. Open a Linux terminal as a non-root user in that user's home directory.

  2. Download the Partek FlexNet 11.12 distribution. Versions of FlexNet downloaded from other locations will not work as they do not contain the Partek vendor daemon (parteklm).
    $ cd
    *_$ wget http://www.partek.com/~devel/FlexNet/FlexNet11.12.zip_*
    $ unzip FlexNet11.12.zip

  3. Obtain the license key from Partek. Save it in the home directory as license.dat.
    If you have not received the license.dat file, contact your Account Manager or email licensing@partek.com.

  4. Determine if your linux workstation is 32 or 64-bit.

  5. Start the license server.
    If you are on a 64-bit workstation (most-likely):
    $ ~/FlexNet/linux64/lmgrd -c ~/license.dat -l ~/flexlog.txt
    If you are on a 32-bit workstation:
    $ ~/FlexNet/linux32/lmgrd -c ~/license.dat -l ~/flexlog.txt

  6. Verify the licensing server is running and offering features: flow_base, flow_hpc_nodes, and flow_hpc_cores:
    $ ~/FlexNet/linux64/lmutil lmstat -a (64-bit)
    $ ~/FlexNet/linux32/lmutil lmstat -a (32-bit)

The default configuration for the license server is complete. For additional details or advanced use cases, consult the FlexLM manual located in the FlexNet 11.12 distribution (FlexNet folder) named FLEXnet-v11.12-License_Administration_Guide.pdf.

Launching Partek Flow for the first time

Access Partek Flow on a web browser

Once Partek Flow server has been started, access the interface using a web browser.

  • If you are on a computer other than the Partek Flow server computer, localhost should be replaced with the IP address of the Partek Flow server computer

Enter the License key

When Partek Flow is launched for the first time, the user is prompted to provide a license key (Figure 2).

  • For users with Node locked licenses, copy and paste the license key received from Partek Licensing Support in the License key box. If you have not received the license.dat file, contact your Account Manager or email licensing@partek.com.
  • For users with Floating licenses, make sure that the FlexLM software is running (see section on Setting up the License Manager ). In the License key box, enter:
    SERVER license-server-hostname (Change to your license server name or ip)
    USE_SERVER

Figure 2. Setting up the Partek Flow license during installation

Setup the Administrator account

Partek Flow supports multiple users, each of which can either be classified as administrator or regular user, based on access privileges. First you must set-up an administrator account. Additional users may be added after installation.

To set up the Partek Flow administrator account. Specify the username ('admin'), password, and email (Figure 3) and click Next. 

 

Figure 3. Setting up the Partek Flow 'admin' account during installation

Download Genome Library Files

Partek-distributed library files and aligner indices can be pre-downloaded, to save time in the future (Figure 4). Unnecessary genome builds may be skipped by clicking the button before clicking Next.
All Partek Flow users share library files and the size of the library folder can grow significantly. At least 100GB of free space should be allocated for library files. For more information on library file management setup, see our Library file management user guide.

 

Figure 4. Downloading Partek-distributed library files during installation

After scheduling downloads, select the Finish button (Figure 5).

 

Figure 5. Completing Partek Flow installation

The installation will complete the browser will display the Partek Flow Homepage (Figure 6). Since there are no projects available, links for importing the tutorial data and to the user manual are displayed.

 

Figure 6. The Partek Flow homepage

Backing up the Partek Flow database

To backup the database, open a Linux terminal and enter the following commands.

  1. Change to the user account that runs Partek Flow. Suppose it is user account flow, then:
    $ su flow

  2. Then the default home directory should be/home/flow. Run the following command to make the backup database in home/flow directory, the archived file name is flowdbbackup.tar.gz:
    $ tar -czvf flowdbbackup.tar.gz .partekflow/

  3. Log out of the user account that runs Flow:
    Ctrl+D

Updating Partek Flow

Before performing updates, we recommend to backup the database as shown above.
Updates are applied using the Linux package manager. The update process will restart the Partek Flow server and the running tasks will be stopped and restarted. Therefore it is best to plan updates during periods of low activity on the Partek Flow server. To update Partek Flow, open a terminal window and enter the following command.

For Debian/Unbuntu, enter:
$ sudo apt-get install partekflow

For Redhat/Fedora/CentOS, enter:
$ sudo yum update partekflow

For the YUM package manager, if updating Partek Flow fails with a message claiming "package not signed," enter:
$ yum –nogpgcheck –y update partekflow
Note that our packages are signed and the message above is erroneous.

Uninstalling Partek Flow

To uninstall Partek Flow, open a terminal window and enter the following command.

Debian/Ubuntu:
$ sudo apt-get remove partekflow

RedHat/Fedora/CentOS:
$ sudo yum remove partekflow

The uninstall removes binaries only (/opt/partek_flow).
The logs, database (partek_db) and files in the home/flow/.partekflow folder will remain unaffected.

System Administrator Guide

This section provides additional tools that may be useful for system administrators who maintain the Partek Flow server.

Verifying that Partek Flow is Running on the Server

At anytime, you wish to know the status of Partek Flow:
$ service partekflowd status
Possible outputs are RUNNING or STOPPED.

Changing the Temporary Folder Location

By default, temporary files resulting from genomic data uploads to Flow are stored in /opt/partek_flow/temp and are removed upon upload completion. If Flow is installed on a small root partition (<20GB), exceedingly large uploads may fill the root partition and necessitate moving this temporary directory to a larger partition. In order to select a new Flow temp folder, complete the following logged in as root:

  1. Shutdown Flow
    # service partekflowd stop

  2. For this example we will use the new temporary folder location of /home/flow/partek_flow/temp. Adjust this path to meet your needs.

  3. Open the configuration file /etc/partekflow.conf and append the following line to the end of the file:
    CATALINA_TMPDIR=/home/flow/partek_flow/temp

  4. Ensure the new temporary directory exists and is writeable by Flow. If you use a different Linux user account to run Flow, make sure this folder can be modified by this user.
    # mkdir -p /home/flow/partek_flow/temp
    # chown flow:flowuser /home/flow/partek_flow/temp

  5. Start Flow
    # service partekflowd start

Diagnosing and Reporting Installation Issues with flowstatus.sh

Partek Flow comes with a standalone diagnostic script that reports how Flow is installed and detects common installation problems. This script can be run independently of Flow as installation issues or crashes can prevent Partek Flow from starting. This utility gathers Flow log files and server information which, upon customer approval, will be sent to Partek so our support team has all requisite information to address the issue. Some examples of when this script should be run include:

  • Support personnel need additional information and will request this script be run
  • Flow crashes or is otherwise inaccessible
  • Flow is unable to see input or output directories or projects are suddenly missing
  • Unexpected behavior after a Flow or system update
  • Tasks fail to run due to missing files or directory permission issues

When a task fails, the first course of action is to enter its task's details page (Figure 7), then click on the button labeled Send logs to Partek. This creates a support ticket and you will be contacted. In some cases the task failure logs sent when clicking on this button do not contain adequate information. In this case, Partek Technical Support will request that you run this script. Whenever possible, please run this script as the root user to ensure that system log information is collected.

If you are unable to install Partek Flow, this script will not be available. Please contact Partek Technical Support if you cannot install Partek Flow.

 

Figure 7. Task details page of a failed task. Warning information section is optional, the content of the Error message section depends on the type of error. The Send logs to Partek button sends the log files to Partek Technical Support

Running flowstatus.sh via the command line

  1. Locate the Flow installation directory. This is defined as the FLOWhome variable in the file /etc/partekflow.conf.
    $ grep FLOWhome /etc/partekflowd.conf
    For this example, we assume the Flow install directory is /opt/partek_flow. Replace this with the directory found in step 1.

  2. Run the script
    $ bash /opt/partek_flow/flowstatus.sh
    After the script is run, a report will appear on the screen, then you will be asked if you wish to upload this report to Partek. If this report is uploaded, you will be contacted by support personnel who will assist with your issue. 

    If you are unable to find the flowstatus.sh script on your system or you want to run the most recent version of this script, then execute the following:
    *_$ wget https://customer.partek.com/flowstatus.sh_*
    $ bash flowstatus.sh

Interpreting flowstatus.sh reports

When running the flowstatus.sh script, you will see a report similar to Figure 8.

 

Figure 8. An example of a flowstatus.sh report

The relevant details of the report are:

Script running as Linux user: The user account the flowstatus.sh script was run under

Flow status: Is the Partek Flow server running or not?

Flow HTTP port: To use Partek Flow, most users will access the URL http://localhost:8080. The number associated with this URL is the HTTP port which defaults to 8080. Sometimes this port will be changed to another value. For example, if the port was changed to 8081, you will need to access Partek Flow by visiting the URL http://localhost:8081.

Flow is running as Linux user: The user account under which the Partek Flow server runs. This defaults to 'flow', however, this could have been changed to ameliorate permission issues by running Partek Flow under the same user that is the primary user of this server (i.e. the user that logs into and uses the desktop on this server).

Flow installation method: For all default installs, Partek Flow is installed with the package manager. If this is not your installation method, you are advised to contact Partek support in order to maintain your Partek Flow installation or assist with installation issues. The conversion steps are described in the next section.

Flow install directory: By default, this should be /opt/partek_flow. If this is not the case, the upgrade process for Partek Flow becomes more involved.

Flow database directory: This is a relatively small directory that stores all Partek Flow configuration and information about analysis and projects generated by Partek Flow. It is crucial that this directory be backed up regularly. If it is removed or corrupted, ALL projects in Partek Flow disappear. The actual raw input and output files for all projects are not lost, however.

After displaying Partek Flow configuration information, several installation checks are performed. This covers common issues that can break a Partek Flow installation such as full disks or running Partek Flow under the wrong user account.

Converting a Zip Installation to Use the Package Manager

If you have used a .zip file to install a previous build of Partek Flow and you wish to convert your installation to a package manager, we recommend that you contact the Partek Licensing Support (licensing@partek.com) for assistance in this process. Briefly, we describe the conversion steps below.

  1. Log in to the existing Linux user account used to run Partek Flow. Next, stop the Partek Flow server.
    $ cd
    $ ~/partek_flow/stop_flow.sh

  2. Ensure Partek Flow is no longer running. If the output contains only "grep bin/flow" this requirement is met.
    $ ps aux | grep bin/flow

  3. If Partek Flow is running and repeating step 1 above does not shut down the Partek Flow server, then use the following command where PID is the process ID of Partek Flow. The PID is found from the output of step 2 above, column two
    $ kill -9 PID

  4. Backup the existing Partek Flow database and installation directories. Substitute the example paths below with those specific to the Partek Flow installation.
    $ cp -r ~/.partekflow ~/.partekflow_backup
    $ mv ~/partek_flow ~/partek_flow_backup

  5. Follow the Installation steps relevant to the Linux distribution on the Partek Flow server.

    For Debian/Ubuntu:
    Upon reaching Configure Partek Flow installation settings enter
    $ sudo dpkg-reconfigure partekflow

    For Redhat/Fedora/Centos:
    Edit the following file: /etc/partekflow.conf 

    These prompts set the existing Linux account name and home directory used to run the previous Partek Flow server installation. 

Transferring Partek Flow to a new machine

Contact your Account Manager or email licensing@partek.com to request for transfer and to obtain a new license.dat file based on the Host ID of your new machine. Follow the steps below to move the Partek Flow license and database:

On OLD MACHINE
  1. Shutdown existing Partek Flow installation:
    $ sudo service partekflowd stop

  2. Backup Partek Flow database:
    $ sudo su - flow
    $ tar cvfz partekflowdb.bkup.tgz /home/flow/.partekflow

  3. Copy partekflowdb.bkup.tgz to new machine

  4. Remove existing Partek Flow installation:
    Debian/Ubuntu:
    $ sudo apt-get remove partekflow partekflow-bin

    RedHat/Fedora/CentOS
    $ sudo yum remove partekflow partekflow-bin 
On NEW MACHINE
  1. Install Partek Flow as described earlier in this document.
    When prompted for license, paste the license generated for the new machine. 

  2. Shutdown Partek Flow to install previous database:
    $ sudo service partekflowd stop

  3. Unpack partekflowdb.bkup.tgz:
    $ sudo su - flow
    $ tar xvfoz partekflowdb.bkup.tgz
    $ exit

  4. Restart Partek Flow
    $ sudo service partekflowd start 

 

Your Rating: Results: 1 Star2 Star3 Star4 Star5 Star 23 rates

  • No labels