Page History
...
Using a web browser, log in to Partek Flow. From the Home page click the New Project button; enter a project name (Figure 1) and then click Create project. Figure 1:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
The Project name is the basis of the default name of the output directory for this project. Project names are unique, thus a new project cannot have the same name as an existing project within the same Partek Flow server.
...
For a new project, where no samples have yet been added, the Data tab will automatically prompt you to add samples (Figure 2). To add samples to the project, click Import data. Four options will be displayed. Figure 2:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Automatically create samples from files
...
Select the Automatically create samples from files button. The next screen will feature a file browser that will show any folders you have access to in the Partek Flow server (Figure 3). Select a folder by clicking the folder name. Files in the selected folder that have file formats that can be imported by Partek Flow will be displayed and tick-marked on the right panel (Figure 3). You can exclude some files from the folder by unselecting the check mark on the left side of the filename. When you have made your selections, click the Create sample button. Figure 3:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Alternatively, files can also be uploaded and imported into the project from the user's local computer. Select the My computer radio button (Figure 4) and the options of selecting the local file and the upload (destination) directory will appear. Only one file at a time can be imported to a project using this method. Figure 4:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Multiple data files can be compressed a single .zip file before uploading. Partek Flow will automatically unzip the files and put them in the upload directory.
...
The columns in the expanded view show the files associated with each sample. Files are organized by file type. Any filename extensions that indicate compression (such as .gz) are not shown. Figure 5:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Once a sample is created in a project, the files associated with it can be modified. In the expanded view, mouse over the +/- column of a sample. The highlighted icons will correspond to the options for the sample on that row.
...
Click the Project dropdown menu to select an existing project. In the Samples box, you will find a list of samples from the selected project. Choose the samples you wish to add to your new project by highlighting them using your mouse. If needed, use the keyboard Ctrl- or Shift- key to select more than one sample. Once you make your selection, click the Add button. These samples will now appear in the Sample management table in the Data tab. Figure 6:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Create a new blank sample
...
Mouse over the +/- column and click the green icon ( ) to associate a file(s) to the sample. Perform the process for every sample in your project. Figure 7:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Adding Files to an existing project
Additional samples can be added to any existing project simply by opening the project, going to the Data tab, and clicking the Import data button (Figure 8). Three options to add samples will be revealed. Figure 8:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Importing feature counts
Alternatively, if you have a matrix of data, such as raw read count data in text format, select Import feature counts. This will bring up the Input options page (Figure 9).
...
If the read counts are based on a compatible annotation file in Partek Flow, you can specify that annotation file under File format. Select the appropriate genome build and annotation model for your count data.
Figure 9:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Otherwise, manually specify the orientation of the data matrix by changing the Input format drop-down menu (Figure 10). If the data has been log transformed, specify the base under Counts format.
Figure 10:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Project output directory
The project output directory is the folder within the Partek Flow server where all output files produced during analysis will be stored.
...
Attributes can be managed or created within a project. Under the Data tab, click the button to open the Manage attributes page (Figure 11). Figure 11:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
To prepare for later data analysis using statistical tools, attributes can either be categorical or numeric (i.e., continuous).
...
To add a categorical attribute in the Manage attributes page, click the Add new attribute (Figure 12). In the dialog box, type a Name for the attribute, select the Categorical radio button next to Attribute type, select the visibility of the attribute and then click the Add button. Figure 12:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Individual categories for the attribute must then be entered. Enter a name of the New category and click Add. The Name of the new category will show up in the table. The category can also be edited by clicking or deleted by clicking . Additionally, sub-categories can be added by clicking in the Options column. After all the categories have been added to this attribute, click Close to proceed.
Repeat the process for additional attributes of the samples in your study. When done, click Back to sample management table. Categorical attributes will default to Project-specific visibility.
An additional feature to facilitate the controlled vocabulary in Partek Flow is the integration of terms used by SNOMED CT, a healthcare terminology used for electronic health records. They can be added easily by managing the categories of an attribute and selecting Import terms from SNOMED CT shown in Figure 13. Available terms include those for body structure, specimen, clinical findings, or organism. Select multiple terms from SNOMED CT by holding Ctrl key on the keyboard while clicking the mouse to select. These terms will become new top level categories in your attribute. When terms are imported from SNOMED CT, the Term depth indicates the number of sublevels that will also be imported as subcategories in your attribute. Figure 13:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Adding a numeric attribute
To add a numeric attribute in the Manage attributes page, click the Add new attribute. In the dialog box (Figure 14), type a Name for the attribute, select the Numeric radio button next to Attribute type, and then click the Add button. Some optional parameters for numeric attributes include the Minimum value, Maximum value, and Units. When done, click Add to return to the Manage attributes page. Repeat the process add more numeric attributes. When done, click Back to sample management table. Figure 14:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Adding a system-wide attribute
...
For an System-wide categorical attribute, the different categories are listed and you have the option of pre-filling the columns with N/A (or any other category within the attribute). Click Add column and you will return to the Data Tab.
Figure 15:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Assigning categories or values to attributes
After adding all the desired attributes to a project, the sample management table will show a new column for each attribute (Figure 16). The columns will initially as "N/A", as the samples have not yet been categorized or assigned a value. To edit the table, click . Assign the sample attributes by using a drop down for categorical attributes (controlled vocabulary) or typing with a keyboard for numeric attributes. Figure 16:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
When all the attributes have been entered, click Apply changes and the sample management table will be updated. After editing the sample table, make sure there are no fields with blank or N/A values before proceeding. To rename or delete attributes, click Manage attributes from the Data tab to access the Manage attributes page. Note that you cannot delete an attribute if a sample is assigned to it.
...
Using a text editor, prepare a table containing the attributes. An example is shown in Figure 17. There should only be one tab between columns with no extra tabs after the last column. In this particular example, the first column contains the filename and the text file is saved as Sampleinfo.txt. Figure 17:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
The first row of the table in the text file contains the attributes (as headers). The first column of the table in the text file, regardless of the header of the first column, should contain either the sample names or the file names of the samples already added in Partek Flow. The first column is the unique identifier that will match the samples to the correct values or categories.
...
Select the attributes you want to import by clicking the Import check box. Imported attributes that do not currently exist in the project will create new project-specific attributes.
Figure 18:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
You can change the name of a specific attribute by editing the Attribute name text box. Columns containing letter characters are automatically selected as categorical attributes. Columns containing numbers are suggested to be numeric attributes and can be changed to categorical using the drop down menu under Attribute type.
Guidelines for preparing the sample annotation text file
...
In the Data tab, each sample can be renamed or deleted from the project by clicking the gear icon next to the sample name. The gear icon is readily visible upon mouse over (Figure 19). Deleting a sample from a project does not delete the associated files, which will remain on the disk.
Figure 19:
Numbered figure captions | ||
---|---|---|
|
...
| ||
You can download your completed Sample management table by clicking the Download link at the lower corner of the table. This will export a tab-delimited text file with contents of the table.
...
After samples have been added and associated with valid data files, a data node will appear in the Analyses tab (Figure 20). The Analyses tab is where different analysis tools and the corresponding reports are accessed.
Figure 20:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Data and Task nodes
The Analyses tab contains two elements: data nodes (circles) and task nodes (rounded rectangles) connected by lines and arrows. Collectively, they represent a data analysis pipeline.
Data nodes (Figure 21) may represent a file imported into the project, or a file generated by Partek Flow as an output of a task (e.g., alignment of FASTQ files generates BAM files).
Figure 21:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Task nodes (Figure 22) represent the analysis steps performed on the data within a project. For details on the tasks available in Partek Flow, see the specific chapters of this user manual dedicated to the different tasks. Figure 22:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
The context sensitive menu
Clicking on a node reveals the context sensitive menu, on the right side of the screen. Figure 23:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Only the tasks that are available for the selected data node will be listed in the menu. For data nodes, actions that can be performed on that specific data type will appear.
...
In Figure 24, alignment was performed on the unaligned reads. Two additional nodes were added: a task node for Align reads and an output data node containing the Aligned reads. Figure 24:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Running a task
To run a task, select a data node and then locate the task you wish to perform from the task pane. Mouse over to see a description of the action to be performed. Click the specific task, set the additional parameters (Figure 25), and click Finish. The task will be scheduled, the display will refresh, and the screen will return to the project's Analyses tab.
In Figure 25, the STAR aligner was selected and the choices for the aligner index and additional alignment options appeared. Figure 25:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Tasks that are currently running (or scheduled in the queue) appear as translucent nodes. The progress of the task is indicated by the progress bar within the task node. Hovering the mouse pointer over the node will highlight the related nodes (with a thin black outline) and display the status of the task (Figure 26).
If a task is expected to generate data nodes, expected nodes appear even before the task is completed. They will have a lighter shade of color to indicate that they have not yet been generated as the task is still being performed. Once all tasks are done, all nodes would appear in the same shade. Figure 26:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Canceling and deleting tasks
Tasks can only be cancelled or deleted by the user that started the task. Running or pending tasks can be canceled by clicking the right mouse button on the task node and then selecting Cancel (Figure 27). Alternatively, the task node may be selected and the Cancel task selected from the task pane. Figure 27:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Cancelled or failed tasks are flagged by small red circles with exclamation points on the tasks nodes. Data nodes connected to incomplete tasks are also incomplete as no output can be generated (Figure 28). For failure due to errors, see the Task details. Figure 28:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
To delete tasks from the project click the right mouse button on the task node and then select Delete (Figure 29). Alternatively, click the task node and select Delete task from the task pane. The nodes downstream of this task will be deleted. However, deleting the output files is optional (Figure 28, inset). Figure 29:
Numbered figure captions | ||
---|---|---|
|
In addition to using the right mouse button and the task pane, tasks can also be deleted and cancelled in the Log tab.
| ||
Task Results and Task Actions
Selecting a task node will reveal a menu pane with two sections: Task results and Task actions (Figure 30). Figure 30:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Items from the Task results section inform on the action performed in that node. Certain tasks generate a Task report (Figure 31), which include any tables or charts that the task may have produced.
Figure 31:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
The Task details shows detailed diagnostic information about the task (Figure 32). It includes the duration and parameters of the task, lists of input and output files, and the actual commands (in the command line) that were run. Figure 32:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Additionally, the Task details page would contain the error logs of unsuccessful runs. The user can download the logs or send them directly to Partek. This page plays an important role in diagnosing and troubleshooting issues related to task.
...
Another action available for a task node is Add task description (Figure 33), which is a way to add notes to the project. The user can enter a description, which will be displayed when the mouse pointer is hovered over the task node.
Figure 33:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Layers
It is common for next-generation sequencing data analysis to examine different task parameters for optimization. Users may want to modify an upstream step (e.g. alignment parameters) and inspect its effect on downstream results (e.g. percent aligned reads).
The implementation of Layers in Partek Flow makes optimizations easy and organized. Instead of creating separate nodes in a pipeline, another set of nodes with a different color is stacked on top of previous analyses (Figure 34). To see the parameters that were changed between runs, hover the mouse icon over the set of stacked task nodes and a pop-up balloon will display them. The text color signifies the layer corresponding to a specific parameter. Figure 34:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Layers are formed when the same task is performed on the same data node more than once. They are also formed when a task node is selected and the Re-run it w/new parameters is selected in the task pane. This will allow the users to change the options only for the selected task. The user may choose to re-run the task to which the changes have been made, as well as all the downstream tasks until the analysis is completed. To do so, select Re-run w/new parameters, downstream tasks from the task pane.
To select a different layer, use the left mouse button to click on any node of the desired layer. All the nodes associated with the selected layer have the same color and when clicked will be displayed on the top of the stack.
...
Data associated with any data node can be downloaded using the Download data link in the task pane (Figure 35). Compressed files will be downloaded to the local computer where the user is accessing the Partek Flow server. Note that bigger files (such as unaligned reads) would take longer to download. For guidance, a file size estimate is provided for each data node. These zipped files can easily be imported by the Partek® Genomics Suite® software.
Figure 35:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
The Log Tab
The Log tab contains a table of the tasks that are running, scheduled, or those that have been completed completed within the Partek Flow project (Figure 36). It provides an overview of the task progress, enables task management, and links to detailed reports for each task. Figure 36:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Each row of the table corresponds to a task node in the Analyses tab. The list can be sorted according to a specific column using the sort icon .
The Task column lists the name of the tasks. On the left of the task name is a colored circle indicating the layer of this task. The column is searchable by task name. Clicking the task name will open the Task report page. If the task did not generate a report, the link will go to the Task details page.
...
To add a collaborator, type a username in the Add member box and click the button. Previous collaborators may be selected using the dropdown box. To delete a collaborator, select the next to their username. Figure 37:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Project Management
Project Archiving and Deletion
A project may be archived or deleted using the button on the upper right side of the Project View page (Figure 38). Figure 38:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
When deleting a project, the project will be deleted from the Partek Flow instance. In addition, the user is also given the choice to delete the output files produced by the project.
...
Options include the raw input files, the annotation files as well at the output files associated with the project. Select the files you would like to zip up and click Archive. Figure 39:
Numbered figure captions | ||||
---|---|---|---|---|
| ||||
Additional assistance |
---|
|
Rate Macro | ||
---|---|---|
|