Adaptors¶
This section contains the adaptor documentation which is generated from the information provided by the adaptors themselves.
Contents
File System¶
Note
Supported property names should be prefixed with
"xenon.adaptors.filesystems"
. We’ve left this prefix out to improve
readability of the tables.
S3¶
The S3 adaptor uses Apache JClouds to talk to s3 and others. To authenticate use PasswordCredential with access key id as username and secret access key as password
field | value |
---|---|
supports_third_party_copy | False |
can_create_symboliclinks | False |
can_read_symboliclinks | False |
is_connectionless | True |
supported_credentials | PasswordCredential |
can_append | False |
supports_reading_posix_permissions | False |
supports_setting_posix_permissions | False |
supports_rename | False |
needs_size_beforehand | True |
- location string:
- http[s]://host[:port]/bucketname[/workdir]
- https://s3.region.amazonaws.com/bucketname[/workdir]
supported properties:
name | description | data_type | default |
---|---|---|---|
s3.bufferSize | The buffer size to use when copying files (in bytes). | size | 64K |
File¶
This is the local file adaptor that implements file functionality for local access.
field | value |
---|---|
supports_third_party_copy | False |
can_create_symboliclinks | True |
can_read_symboliclinks | True |
is_connectionless | True |
supported_credentials | DefaultCredential |
can_append | True |
supports_reading_posix_permissions | True |
supports_setting_posix_permissions | True |
supports_rename | True |
needs_size_beforehand | False |
- location string:
- (null)
- (empty string)
- [/workdir]
- driveletter:[/workdir]
supported properties:
name | description | data_type | default |
---|---|---|---|
file.bufferSize | The buffer size to use when copying files (in bytes). | size | 64K |
Sftp¶
The SFTP adaptor implements all file access functionality to remote SFTP servers
field | value |
---|---|
supports_third_party_copy | False |
can_create_symboliclinks | True |
can_read_symboliclinks | True |
is_connectionless | False |
supported_credentials | DefaultCredential, CertificateCredential, PasswordCredential, CredentialMap |
can_append | True |
supports_reading_posix_permissions | True |
supports_setting_posix_permissions | True |
supports_rename | True |
needs_size_beforehand | False |
- location string:
- host[:port][/workdir]
supported properties:
name | description | data_type | default |
---|---|---|---|
sftp.strictHostKeyChecking | Enable strict host key checking. | boolean | true |
sftp.loadKnownHosts | Load the standard known_hosts file. | boolean | true |
sftp.loadSshConfig | Load the OpenSSH config file. | boolean | true |
sftp.agent | Use a (local) ssh-agent. | boolean | false |
sftp.agentForwarding | Use ssh-agent forwarding when setting up a connection. | boolean | false |
sftp.connection.timeout | The timeout for creating and authenticating connections (in milliseconds). | natural | 10000 |
sftp.bufferSize | The buffer size to use when copying files (in bytes). | size | 64K |
Ftp¶
The FTP adaptor implements file access on remote ftp servers.
field | value |
---|---|
supports_third_party_copy | False |
can_create_symboliclinks | False |
can_read_symboliclinks | True |
is_connectionless | False |
supported_credentials | DefaultCredential, PasswordCredential |
can_append | True |
supports_reading_posix_permissions | True |
supports_setting_posix_permissions | False |
supports_rename | True |
needs_size_beforehand | False |
- location string:
- host[:port][/workdir]
supported properties:
name | description | data_type | default |
---|---|---|---|
ftp.bufferSize | The buffer size to use when copying files (in bytes). | size | 64K |
Webdav¶
The webdav file adaptor implements file access to remote webdav servers.
field | value |
---|---|
supports_third_party_copy | False |
can_create_symboliclinks | False |
can_read_symboliclinks | False |
is_connectionless | True |
supported_credentials | DefaultCredential, PasswordCredential |
can_append | False |
supports_reading_posix_permissions | False |
supports_setting_posix_permissions | False |
supports_rename | True |
needs_size_beforehand | False |
- location string:
- http://host[:port][/workdir]
- https://host[:port][/workdir]
supported properties:
name | description | data_type | default |
---|---|---|---|
webdav.bufferSize | The buffer size to use when copying files (in bytes). | size | 64K |
Scheduler¶
Note
Supported property names should be prefixed with
"xenon.adaptors.schedulers"
. We’ve left this prefix out to improve
readability of the tables.
Local¶
The local jobs adaptor implements all functionality by emulating a local queue.
field | value |
---|---|
is_embedded | True |
supports_interactive | True |
supports_batch | True |
uses_file_system | True |
supported_credentials | DefaultCredential |
- location string:
- [/workdir]
supported properties:
name | description | data_type | default |
---|---|---|---|
local.queue.pollingDelay | The polling delay for monitoring running jobs (in milliseconds). | long | 1000 |
local.queue.multi.maxConcurrentJobs | The maximum number of concurrent jobs in the multiq. | integer | 4 |
Ssh¶
The SSH job adaptor implements all functionality to start jobs on ssh servers.
field | value |
---|---|
is_embedded | True |
supports_interactive | True |
supports_batch | True |
uses_file_system | True |
supported_credentials | DefaultCredential, CertificateCredential, PasswordCredential, CredentialMap |
- location string:
- host[:port][/workdir][ via:otherhost[:port]]*
supported properties:
name | description | data_type | default |
---|---|---|---|
ssh.strictHostKeyChecking | Enable strict host key checking. | boolean | true |
ssh.loadKnownHosts | Load the standard known_hosts file. | boolean | true |
ssh.loadSshConfig | Load the OpenSSH config file. | boolean | true |
ssh.agent | Use a (local) ssh-agent. | boolean | false |
ssh.agentForwarding | Use ssh-agent forwarding | boolean | false |
ssh.timeout | The timeout for the connection setup and authetication (in milliseconds). | long | 10000 |
ssh.queue.pollingDelay | The polling delay for monitoring running jobs (in milliseconds). | long | 1000 |
ssh.queue.multi.maxConcurrentJobs | The maximum number of concurrent jobs in the multiq.. | integer | 4 |
At¶
The At Adaptor submits jobs to an at scheduler. This adaptor uses either the local or the ssh scheduler adaptor to run commands on the machine running at, and the file or the stfp filesystem adaptor to gain access to the filesystem of that machine.
field | value |
---|---|
is_embedded | False |
supports_interactive | False |
supports_batch | True |
uses_file_system | True |
supported_credentials | DefaultCredential, CertificateCredential, PasswordCredential, CredentialMap |
- location string:
- local://[/workdir]
- ssh://host[:port][/workdir][ via:otherhost[:port]]*
supported properties:
name | description | data_type | default |
---|---|---|---|
at.poll.delay | Number of milliseconds between polling the status of a job. | long | 1000 |
ssh.strictHostKeyChecking | Enable strict host key checking. | boolean | true |
ssh.loadKnownHosts | Load the standard known_hosts file. | boolean | true |
ssh.loadSshConfig | Load the OpenSSH config file. | boolean | true |
ssh.agent | Use a (local) ssh-agent. | boolean | false |
ssh.agentForwarding | Use ssh-agent forwarding | boolean | false |
ssh.timeout | The timeout for the connection setup and authetication (in milliseconds). | long | 10000 |
ssh.queue.pollingDelay | The polling delay for monitoring running jobs (in milliseconds). | long | 1000 |
ssh.queue.multi.maxConcurrentJobs | The maximum number of concurrent jobs in the multiq.. | integer | 4 |
local.queue.pollingDelay | The polling delay for monitoring running jobs (in milliseconds). | long | 1000 |
local.queue.multi.maxConcurrentJobs | The maximum number of concurrent jobs in the multiq. | integer | 4 |
Slurm¶
The Slurm Adaptor submits jobs to a Slurm scheduler. This adaptor uses either the local or the ssh scheduler adaptor to run commands on the machine running Slurm, and the file or the stfp filesystem adaptor to gain access to the filesystem of that machine.
field | value |
---|---|
is_embedded | False |
supports_interactive | True |
supports_batch | True |
uses_file_system | True |
supported_credentials | DefaultCredential, CertificateCredential, PasswordCredential, CredentialMap |
- location string:
- local://[/workdir]
- ssh://host[:port][/workdir][ via:otherhost[:port]]*
supported properties:
name | description | data_type | default |
---|---|---|---|
slurm.disable.accounting.usage | Do not use accounting info of slurm, even when available. Mostly for testing purposes | boolean | false |
slurm.poll.delay | Number of milliseconds between polling the status of a job. | long | 1000 |
ssh.strictHostKeyChecking | Enable strict host key checking. | boolean | true |
ssh.loadKnownHosts | Load the standard known_hosts file. | boolean | true |
ssh.loadSshConfig | Load the OpenSSH config file. | boolean | true |
ssh.agent | Use a (local) ssh-agent. | boolean | false |
ssh.agentForwarding | Use ssh-agent forwarding | boolean | false |
ssh.timeout | The timeout for the connection setup and authetication (in milliseconds). | long | 10000 |
ssh.queue.pollingDelay | The polling delay for monitoring running jobs (in milliseconds). | long | 1000 |
ssh.queue.multi.maxConcurrentJobs | The maximum number of concurrent jobs in the multiq.. | integer | 4 |
local.queue.pollingDelay | The polling delay for monitoring running jobs (in milliseconds). | long | 1000 |
local.queue.multi.maxConcurrentJobs | The maximum number of concurrent jobs in the multiq. | integer | 4 |
Gridengine¶
The SGE Adaptor submits jobs to a (Sun/Oracle/Univa) Grid Engine scheduler. This adaptor uses either the local or the ssh scheduler adaptor to run commands on the machine running Grid Engine, and the file or the stfp filesystem adaptor to gain access to the filesystem of that machine.
field | value |
---|---|
is_embedded | False |
supports_interactive | False |
supports_batch | True |
uses_file_system | True |
supported_credentials | DefaultCredential, CertificateCredential, PasswordCredential, CredentialMap |
- location string:
- local://[/workdir]
- ssh://host[:port][/workdir][ via:otherhost[:port]]*
supported properties:
name | description | data_type | default |
---|---|---|---|
gridengine.ignore.version | Skip version check is skipped when connecting to remote machines. WARNING: it is not recommended to use this setting in production environments! | boolean | false |
gridengine.accounting.grace.time | Number of milliseconds a job is allowed to take going from the queue to the qacct output. | long | 60000 |
gridengine.poll.delay | Number of milliseconds between polling the status of a job. | long | 1000 |
ssh.strictHostKeyChecking | Enable strict host key checking. | boolean | true |
ssh.loadKnownHosts | Load the standard known_hosts file. | boolean | true |
ssh.loadSshConfig | Load the OpenSSH config file. | boolean | true |
ssh.agent | Use a (local) ssh-agent. | boolean | false |
ssh.agentForwarding | Use ssh-agent forwarding | boolean | false |
ssh.timeout | The timeout for the connection setup and authetication (in milliseconds). | long | 10000 |
ssh.queue.pollingDelay | The polling delay for monitoring running jobs (in milliseconds). | long | 1000 |
ssh.queue.multi.maxConcurrentJobs | The maximum number of concurrent jobs in the multiq.. | integer | 4 |
local.queue.pollingDelay | The polling delay for monitoring running jobs (in milliseconds). | long | 1000 |
local.queue.multi.maxConcurrentJobs | The maximum number of concurrent jobs in the multiq. | integer | 4 |
Torque¶
The Torque Adaptor submits jobs to a TORQUE batch system. This adaptor uses either the local or the ssh scheduler adaptor to run commands on the machine running TORQUE, and the file or the stfp filesystem adaptor to gain access to the filesystem of that machine.
field | value |
---|---|
is_embedded | False |
supports_interactive | False |
supports_batch | True |
uses_file_system | True |
supported_credentials | DefaultCredential, CertificateCredential, PasswordCredential, CredentialMap |
- location string:
- local://[/workdir]
- ssh://host[:port][/workdir][ via:otherhost[:port]]*
supported properties:
name | description | data_type | default |
---|---|---|---|
torque.ignore.version | Skip version check is skipped when connecting to remote machines. WARNING: it is not recommended to use this setting in production environments! | boolean | false |
torque.accounting.grace.time | Number of milliseconds a job is allowed to take going from the queue to the accinfo output. | long | 60000 |
torque.poll.delay | Number of milliseconds between polling the status of a job. | long | 1000 |
ssh.strictHostKeyChecking | Enable strict host key checking. | boolean | true |
ssh.loadKnownHosts | Load the standard known_hosts file. | boolean | true |
ssh.loadSshConfig | Load the OpenSSH config file. | boolean | true |
ssh.agent | Use a (local) ssh-agent. | boolean | false |
ssh.agentForwarding | Use ssh-agent forwarding | boolean | false |
ssh.timeout | The timeout for the connection setup and authetication (in milliseconds). | long | 10000 |
ssh.queue.pollingDelay | The polling delay for monitoring running jobs (in milliseconds). | long | 1000 |
ssh.queue.multi.maxConcurrentJobs | The maximum number of concurrent jobs in the multiq.. | integer | 4 |
local.queue.pollingDelay | The polling delay for monitoring running jobs (in milliseconds). | long | 1000 |
local.queue.multi.maxConcurrentJobs | The maximum number of concurrent jobs in the multiq. | integer | 4 |