View Source

Introduction

The following instructions detail the use of SFTP (Secured File Transfer Protocol) to transfer data to and from your Partek® Flow® instance. SFTP offers significant performance and security enhancements over FTP for file transfers. It also enables the use of robust file syncing utilities, e.g. RSYNC, and is compatible with common file transfer programs such as FileZilla and WinSCP.

To transfer files with SFTP, you will need to have your Partek Flow:

Server Name. Example: myname.partek.com
Username. Example: flowloginname
Private authentication key

This information should have been e-mailed to you from the Partek licensing team. If you lose this information, contact Partek support and we will resend your authentication key to you.

SFTP with FileZilla

FileZilla is a graphical file transfer tool that runs on Windows, OSX, and Linux. It is great when needing to do bulk transfers as all transfers are added to a queue and processed in the background. It is possible to browse your files on the Partek Flow server while transfers are active. This is also the best solution when you are not on a computer with command line access or you are uncomfortable with command line operations.

Downloading FileZilla

We recommend downloading the FileZilla install packages from us. They are also available from download aggregator sites (e.g. CNET, download.com, sourceforge) but these sites have been known to bundle adware and other unwanted software products into the downloads they provide, so avoid them.

Mac OSX:

http://packages.partek.com/bin/filezilla/fz-osx.app.tar.bz2

Windows 32-bit:

http://packages.partek.com/bin/filezilla/fz-win32.exe

Windows 64-bit:

http://packages.partek.com/bin/filezilla/fz-win64.exe

Linux (Please use your distribution's package manager to install Filezilla):

Ubuntu:

$ sudo apt-get update

$ sudo apt-get install filezilla

RedHat, see the following guide:

http://juventusitprofessional.blogspot.com/2013/09/linux-install-filezilla-on-centos-or.html

OpenSuse, see:

https://software.opensuse.org/package/filezilla

Connecting to your Partek server with FileZilla

After starting FileZilla, click on the Site Manager icon located at the top left corner of the FileZilla window.

Flow Documentation > SFTP File Transfer Instructions > sftp_filezilla_quick_connect.png

Click on the New Site button on the left of the popup dialog.

Flow Documentation > SFTP File Transfer Instructions > sftp_filezilla_site_manager.png

Type in a name for the connection. Example: “Partek SFTP”.

Flow Documentation > SFTP File Transfer Instructions > sftp_filezilla_new_site.png

The connection details to the right need to be changed to reflect the information you received via email. The default settings will NOT work.

Flow Documentation > SFTP File Transfer Instructions > sftp_filezilla_logon_type.png

Set Host: to your partek server name
Leave Port: blank
Change Protocol: to SFTP - SSH File Transfer Protocol
Change User: to your Partek Flow login name
Change Logon Type: to Key File and select the key file received via email.

Flow Documentation > SFTP File Transfer Instructions > sftp_file_id_rsa_key.png

When selecting your key file, change the file selection from its default of PPK files to All files. Otherwise you key file will not be visible in the file browser.

After selecting your key file, click the Connect button.

Flow Documentation > SFTP File Transfer Instructions > sftp_filezilla_unknown_host.png

Click the checkbox to always trust this host and click OK. Once connected, you can begin to browse and transfer files. The files and folders to the left are on your computer, the ones on the right are on the Flow server.

Flow Documentation > SFTP File Transfer Instructions > sftp_filezilla_partek_sftp.png

When you need to reconnect to your server, run FileZilla and click the down arrow next to the Site Manager icon, then select Partek SFTP.

SFTP command line usage

Importing your private authentication key

You will receive a file called id_rsa via email. Download this file, note where you downloaded it to, then use ssh-add to import the key. If you logout or reboot your computer, you will need to re-run the commands below. After key import, you will not be asked a password when transferring files to your Partek Flow server.

$ cd directory/with/key

$ chmod 600 id_rsa

$ eval $(ssh-agent)

$ ssh-add id_rsa

Copying files and folders between your Partek Flow server and local computer

Uploading data to us

$ scp -r local_folder flowloginname@myname.partek.com:~/remote_folder

Downloading data from the Partek Flow server

$ scp -r flowloginname@myname.partek.com:~/remote_folder local_folder

SCP is useful for sending one file or folder at a time. If copying a folder, the “-r” option tells SCP to copy all files and folders recursively in that folder.

RSYNC usage

RSYNC is useful when resuming a failed transfer. Instead of re-uploading or downloading what has already been transferred, RSYNC will copy only what it needs.

The command below will sync the folder "local_folder" with the "remote_folder" on Partek's servers. To transfer in the other direction, reverse the last two parameters.

$ rsync -avr --progress ./local_folder/ flowloginname@myname.partek.com:~/remote_folder/

With rsync, don't forget the trailing '/' on directory names.

Points of Caution

When you delete files from the Partek Flow server they are gone and can not be recovered.
Please use Partek Flow to delete projects and results. Manually removing data using SFTP could break your server.
Wait until ALL input data for a particular project has been transferred to the Partek Flow server before importing data via Partek Flow. If you try to import samples while the upload is occurring the import job will crash.
When upload raw data to Partek hosted Flow server, we recommend to a create subfolder for each experiment at the same level of "FlowData" folder or inside "FlowData" folder.