...

Note: This tool requires the Flow rest API license feature

Obtaining the

...

Allocator

$ wget customer.partek.com/flow_worker_allocator

$ chmod 755 flow_worker_allocator

Configuring the

...

Allocator

The configuration file flow.worker.allocator.json must exist in your the home directory. If this file is not present, an example configuration file will be written by running the allocator like so:

$ ./flow_worker_allocator

The allocator will exit immediately so you can edit the example configuration file can be edited. This configuration file is documented in the Configuration file formatFile Format section below.

Note: Your  The system’s temp folder (e.g. /tmp) must be writable and mounted with execute permissions in order for the allocator to run.

Authenticating the allocator to Flow

After flow.worker.allocator.json has been configured, run the allocator:

$ ./flow_worker_allocator

You will be asked for your Flow administrator username and password. After successful authentication, a token is stored in ~/flow.auth.json which is used for all following authentication attempts. If this token is removed, the allocator will again prompt for a Flow username and password. This token only allows access to the Flow rest API and if compromised can not be used to recover an account password.

Starting the allocator as a background process

nohup ./flow_worker_allocator 2>/dev/null &

...

The allocator takes no arguments. All configuration is stored in ~/flow.worker.allocator.json.

Stopping the allocator

killall flow_worker_allocator

...

If the allocator was run as a foreground process, CONTROL-C or SIGTERM will cause the process to exit.

Logging

The allocator writes daily rotated log files to ~/flow.worker.allocator.log. For verbose output, set DebugMode to 1 in the configuration file.

Allocation criteria and strategies

Once configured, the allocator will poll the Flow server every CheckIntervalSec seconds to ask if there is any pending work that would be able to start immediately if another worker was added. If this is true, WorkerCounterCmd is used to query the job scheduler to see how many Flow workers have been allocated. If this is below the WorkerResourceLimit : MaxWorkers limit, one worker is allocated using WorkerAllocatorCmd.

...

It is recommended that WorkerResourceLimit : IdleShutdownMin be relatively short so that allocations are elastic: large workloads are able to allocate all resources available to them and quickly return those resources to the job scheduler when Flow workers are idle.

Configuration file format

FlowAPITimeoutSec : integer

...