The User added tasks feature in Partek® Flow® allows administrators to add their own executables and scripts to the set of analysis tools available on a given installation. These tasks can then be run through the Partek Flow graphical user interface and can be included as part of analysis pipelines.
The process of adding a task is only available to users with administrative privileges. However, once the task has been added to an instance of Partek Flow, all other users, including regular users, can use the task.
The instructions below assume that you are logged in as an admin.
Licensing
The User added tasks feature is a part of the Partek Flow Enterprise offerings and requires a separate license. Having this license allows the addition of new user added tasks and allows existing user added tasks to be run individually or within pipelines. To check if your installation is licensed, go to Settings > Licensing and make sure that the User added tasks is licensed (Figure 1).
Creation
Prior to beginning, all source code and executable(s) for the analysis tool need to be downloaded and installed in a directory Partek Flow has access to.(Need to ask Adam on preferred location - .partekflow/user_tasks? /opt? anywhere under home/flow? The executable (or a symbolic link to it) must be installed in this directory: FLOWhome/user_tasks, where FLOWhome is the location of the database directory (cross ref, flowstatus.sh, Fig2).This directory and any binaries within it must be available to all workers in a cluster install environment.
To create a new user added task, go to Settings > Task management. Select the Add task button and choose Create new task in the pop-up dialog (Figure 2).
To create a new user added task, go to Settings > Task management. Select the Add task button and choose Create new task in the pop-up dialog (Figure 2).
The creation of a new user added task has four steps: Task details, Input, Output, and Options. A task creation wizard will guide you with each step.
Task details
In the Task details step (Figure 3), provide the following:
- Task name - a unique name for the task
- Description (optional) - a more thorough description of the task
- Version - a unique version number for the given task name
- Section - choose a section in the context-sensitive Task Menu in which this task will be grouped, or define a new section by editing the text field
The combination of Task name + Version must be unique (i.e., no other task already installed can have the same combination)
For the Executable, provide a full path to the executable that this task will run (Figure 4) by clicking on the field of the Browse button. The executable is the main binary or script to call when the user added task is run. This executable (or a symbolic link to it) must be installed in this directory: FLOW_HOME/user_tasks. In the example in Figure 3, the full path is /home/flow/.partekflow/user_tasks. This directory and any binaries within it must be available to all workers in a cluster install environment. The dialog (Figure 4 inset) lists all the available executables stored in the directory. Select the file and click Continue to proceed.
SECURITY WARNING - Since binaries in this folder will be accessible through the Partek Flow web UI, this directory should be highly restricted from modification by non-administrator users.
For the run format, select whether the task Runs on each sample or all samples (Figure 5).
- Each sample - runs the executable once per individual sample (i.e., runs the samples in parallel)
- All samples - runs the binary once with all the samples provided in a single command line (e.g, merges all samples into a single run)
Click Next to proceed.
Input
In the Input step, specify what type of data the task can run on, and how to pass that data on the command line (Figure 6).
Partek Flow requires User added tasks to be performed on either one of the three main Input data types: Unaligned reads, Aligned reads, and Variants. Additional inputs can be specified in the Options step. The type of Input data selected from the dropdown menu will determine the additional fields that need to be defined:
Input from - choose between Parameter or Last option as the placement of the input on the command line
Accepts paired-end - choose between Yes, No, Exclusively as the behavior of this task's input with regard to paired-end reads
Single-end parameter - enter a parameter to add before single-end reads
1st pair parameter - enter a parameter to add before the first file (_1) in a paired-end reads set
2nd pair parameter - enter a parameter to add before the first file (_2) in a paired-end reads set
You can optionally include Sample information as an argument to the command (Figure 8). Select Include sample info and enter the Sample info parameter. The parameter is used to specify the sample information file on the command line.
When running the command on All samples, the text file will have all sample files as rows. When running the command as Each sample then there will be one info file per command and it will only contain the rows for the files in that sample.
Output
The expected outputs of the task must be defined and configured. To add a new output, select the Add output button.
Define and configure the output.
- Type - choose the type of data produced by the task from
- The set of output types is restricted to those currently supported by downstream Partek Flow tasks: Unaligned reads, Aligned reads, Assembled reads, or Variants
If you want to output a data type that is not part of this set, select Custom and this will produce an output that can be downloaded
- Name - enter a name for the output data node (this will be the data node that will display in the Analyses tab)
- Exposed - if checked, this output will be exposed as a node on the Analyses tab, otherwise this output will only be accessible through the task result page
Ouput files as - (for Custom types) choose between a Single file output, All files with prefix, and a list of files in a Directory
- Output prefix - to restrict only output files with the given prefix
Output to - placement of output on the command line, which can be: Last Option, Parameter, Standard Out
Output parameter - parameter to place on the command line before this output
After configuring the dialog field, click Save. The Output will appear in the Table (Figure 13). Any existing output can be edited or deleted using the action buttons on the far right of the table.
Options
Additional input options to the command line can be configured.
To add a new input, select the Add option button, then define and configure the dialog (Figure 15).
- Name - a label that concisely describes the option, keeping this short ensures that the option label is well displayed and easy to find
Type - the type of value this option can assume, which determines the interface used to choose the value during task setup
Number - spinner component to select number values
- Yes/No - check box to select boolean values
- List - select one or more item from a list of string values
- Text - input box to select arbitrary string values
- Fixed - input box to select a fixed string value (will not show up on the configuration UI of the user task)
- File - file selector for a single file
- Directory - file selector for a directory
- Library file - selector for supported, built in library files (e.g. reference fasta, reference VCF)
Parameter - the parameter used to identify this option on the command line. For example: -a [value], where '-a' is the parameter
- Description - additional information about the option will be available in an 'info' icon (tooltip) when configuring the option
- Advanced - options that are set to advanced will be moved to the advanced options dialog, and can be saved to option sets
- Required - options that are set to required must be filled with a value when configured, otherwise the option is something that can be left off the command line
- Only Integers - if checked, this option will not allow fractional numbers, such as 74.42
- Values range - enter two values to define a range of values
- Default value - enter a default value
- Allow multiple - if checked, allows one or more of the list items to be selected
- Add Value - (for List types) enter one or more strings
- Library type - (for Library file types) can be a Reference sequence or Variant database
After configuring the dialog field, click Save. The Option will appear in the Table as in Figure 16. Any existing option can be edited or deleted using the action buttons on the far right of the table.
Complete the addition of the User added task by clicking the Finish button from the Options step.
Management
The new task will show up as an entry in the Task management table.
Like native Partek Flow tasks this task can have its system wide version set, and can be globally enabled or disabled. Unlike native Partek Flow tasks, there are additional actions that can be taken on user added tasks:
- Download: Downloads the task definition as a file that can be imported into other Partek Flow installations.
- Edit: Opens the add task wizard with all fields pre-filled. This is a clone operation, so a new task name or version must be selected.
- Delete: Removes the task
Running a Task
User added tasks appear in the context sensitive Task Menu when a selected data node matches the Input data type.
If there are additional options that need to be defined, a task wizard will appear.
The Name of the User added task will appear as a task node in the Analyses tab.
Integration Scripts
User added tasks have some format restrictions for passing arguments to the command line.
For example, a binary that has multiple positional arguments cannot be directly wrapped. To execute these binaries through Partek Flow’s User added tasks, one must first wrap the binary in a script that can, for example, parse parameterized arguments into positional arguments. This can be done using any scripting language executable by the Partek Flow server.
User added tasks also require a single binary per task. So if one wishes to execute multiple steps and have that represented as a single task in Partek Flow, one must first wrap those steps in a script that can be imported into Partek Flow.
Additional Assistance
If you need additional assistance, please visit our support page to submit a help ticket or find phone numbers for regional support.
Your Rating: | Results: | 2 | rates |