API

The Server

xenon.init(port=None, do_not_exit=False, disable_tls=False, log_level='WARNING')

Start the Xenon GRPC server on the specified port, or, if a service is already running on that port, connect to that.

If no port is given, a random port is selected. This means that, by default, every python instance will start its own instance of a xenon-grpc process.

Parameters:
  • port – the port number
  • do_not_exit – by default the GRPC server is shut down after Python exits (through the atexit module), setting this value to True will prevent that from happening.

File Systems

class xenon.FileSystem(service, wrapped)

The Xenon FileSystem subsystem.

Variables:id (string) – id
append_to_file(path, data_stream)

Open an existing file and return an OutputStream to append data to this file.

cancel(copy_operation=None)

Cancel a copy operation.

Parameters:copy_operation (xenon.CopyOperation) – copy_operation
close()

Close this filestem Any pending/running copy operations of this filestystem will be terminated Will also forget this filesystem

copy(source=None, destination_filesystem=None, destination=None, mode=None, recursive=None)

Asynchronously Copy an existing source path to a target path on a different file system.

Parameters:
  • source (xenon.Path) – source
  • destination_filesystem (xenon.FileSystem) – destination_filesystem
  • destination (xenon.Path) – destination
  • mode (xenon.CopyRequest.CopyMode) – mode
  • recursive (bool) – recursive
classmethod create(adaptor=None, location=None, properties=None, certificate_credential=None, password_credential=None, default_credential=None, credential_map=None, keytab_credential=None)

Create a new FileSystem using the adaptor that connects to a data store at location using the credentials to get access.

Parameters:
  • adaptor (string) – adaptor
  • location (string) – location
  • properties (map<string, string>) – properties
  • certificate_credential (xenon.CertificateCredential) – certificate_credential
  • password_credential (xenon.PasswordCredential) – password_credential
  • default_credential (xenon.DefaultCredential) – default_credential
  • credential_map (xenon.CredentialMap) – credential_map
  • keytab_credential (xenon.KeytabCredential) – keytab_credential
create_directories(path=None)

Creates a new directory, including parent directories, failing if the directory already exists.

Parameters:path (xenon.Path) – path
create_directory(path=None)

Creates a new directory, failing if the directory already exists.

Parameters:path (xenon.Path) – path
create_file(path=None)

Creates a new empty file, failing if the file already exists.

Parameters:path (xenon.Path) – path

Creates a new symbolic link, failing if the link already exists

Parameters:
delete(path=None, recursive=None)

Deletes an existing path.

Parameters:
  • path (xenon.Path) – path
  • recursive (bool) – recursive
exists(path=None)

Tests if a path exists.

Parameters:path (xenon.Path) – path
classmethod get_adaptor_description(name=None)

Gives the description of the adaptor with the given name.

Parameters:name (string) – name
classmethod get_adaptor_descriptions()

Gives a list of the descriptions of the available adaptors.

get_adaptor_name()

Get the name of the adaptor that created this FileSystem.

classmethod get_adaptor_names()

Gives a list names of the available adaptors.

get_attributes(path=None)

Get the PathAttributes of an existing path.

Parameters:path (xenon.Path) – path
get_path_separator()

Get the path separator used by this file system.

get_status(copy_operation=None)

Retrieve the status of an copy.

Parameters:copy_operation (xenon.CopyOperation) – copy_operation
get_working_directory()

Get the current working directory of this file system.

is_open()

Return if the connection to the FileSystem is open.

list(dir=None, recursive=None)

List all entries in the directory dir.

Parameters:
  • dir (xenon.Path) – dir
  • recursive (bool) – recursive
classmethod list_file_systems()

List the created filesystems Specific to grpc, not part of Xenon library

classmethod local_file_systems()

Returns filesystems for all local drives Not part of FileSystem class in Xenon library In Xenon library available as LocalFileSystemUtils.getLocalFileSystems()

read_from_file(path=None)

Open an existing file and return an InputStream to read from this file.

Parameters:path (xenon.Path) – path

Reads the target of a symbolic link

Parameters:path (xenon.Path) – path
rename(source=None, target=None)

Rename an existing source path to a non-existing target path

Parameters:
set_posix_file_permissions(path=None, permissions=None)

Sets the POSIX permissions of a path

Parameters:
set_working_directory(path=None)

Set the current working directory of this file system to directory.

Parameters:path (xenon.Path) – path
wait_until_done(copy_operation=None, timeout=None)

Wait until a copy operation is done or until a timeout expires.

Parameters:
  • copy_operation (xenon.CopyOperation) – copy_operation
  • timeout (uint64) – timeout
write_to_file(path, data_stream)

Open a file and return an OutputStream to write to this file. In Xenon library if request is missing size field then FileSystem.writeToFile(Path file) is used else FileSystem.writeToFile(Path path, long size) is used

class xenon.Path(path)

Wrapper around PurePosixPath form the pathlib module. This class reveals a string representation of the underlying path object to GRPC. You may use this class like a pathlib.PurePosixPath, including using it as an argument to open calls as it derives from os.PathLike (Python > 3.6). For more information see the Python documentation on pathlib.

is_hidden()

Checks if a file is hidden. Just compares the first character in the filename with ‘.’.

Message classes

class xenon.PosixFilePermission

An enumeration.

GROUP_EXECUTE = 6
GROUP_READ = 4
GROUP_WRITE = 5
NONE = 0
OTHERS_EXECUTE = 9
OTHERS_READ = 7
OTHERS_WRITE = 8
OWNER_EXECUTE = 3
OWNER_READ = 1
OWNER_WRITE = 2
class xenon.CopyMode

An enumeration.

CREATE = 0
IGNORE = 2
REPLACE = 1
class xenon.CopyStatus(service, wrapped)

Status of a copy operation.

Variables:
  • copy_operation (xenon.CopyOperation) – copy_operation
  • bytes_copied (uint64) – bytes_copied
  • bytes_to_copy (uint64) – bytes_to_copy
  • done (bool) – done
  • running (bool) – running
  • state (string) – state
  • error_message (string) – error_message
  • error_type (xenon.CopyStatus.ErrorType) – error_type
class ErrorType

An enumeration.

ALREADY_EXISTS = 3
CANCELLED = 2
NONE = 0
NOT_CONNECTED = 4
NOT_FOUND = 1
XENON = 5
error_type

Schedulers

class xenon.Scheduler(service, wrapped)

The Xenon Schedulers subsystem.

Variables:id (string) – id
cancel_job(job=None)

Cancel a job

Parameters:job (xenon.Job) – job
close()

Close this Scheduler. If scheduler is embedded then any pending/running jobs will be killed Will also forget this scheduler

classmethod create(adaptor=None, location=None, properties=None, certificate_credential=None, password_credential=None, default_credential=None, credential_map=None, keytab_credential=None)

Create a new Scheduler using the adaptor connecting to the location using credentials to get access.

Parameters:
  • adaptor (string) – adaptor
  • location (string) – location
  • properties (map<string, string>) – properties
  • certificate_credential (xenon.CertificateCredential) – certificate_credential
  • password_credential (xenon.PasswordCredential) – password_credential
  • default_credential (xenon.DefaultCredential) – default_credential
  • credential_map (xenon.CredentialMap) – credential_map
  • keytab_credential (xenon.KeytabCredential) – keytab_credential
classmethod get_adaptor_description(name=None)

Gives the description of the adaptor with the given name.

Parameters:name (string) – name
classmethod get_adaptor_descriptions()

Gives a list of the descriptions of the available adaptors.

get_adaptor_name()

Get the name of the adaptor that created this Scheduler.

classmethod get_adaptor_names()

Gives a list names of the available adaptors.

get_default_queue_name()

Get the name of the default queue.

get_file_system()

Retrieve the FileSystem used internally by this Scheduler.

get_job_status(job=None)

Get the status of a Job.

Parameters:job (xenon.Job) – job
get_job_statuses(jobs=None)

Get the status of all specified jobs.

Parameters:jobs (xenon.Job) – jobs
get_jobs(queues=None)

Get all job identifier of jobs currently in (one ore more) queues.

Parameters:queues (string) – queues
get_location()

Get the location that this Scheduler is connected to.

get_properties()

Get the properties used to create this Scheduler.

get_queue_names()

Get the queue names supported by this Scheduler.

get_queue_status(queue=None)

Get the status of the queue.

Parameters:queue (string) – queue
get_queue_statuses(queues=None)

Get the status of all queues.

Parameters:queues (string) – queues
is_open()

Test if the connection of this Scheduler is open.

classmethod list_schedulers()

List the created schedulers Specific to grpc, not part of Xenon library

classmethod local_scheduler()

Get scheduler on local filesystem with default location, credential and no properties Not part of Scheduler class in Xenon library In Xenon library available as Scheduler.create(“local”)

submit_batch_job(description=None)

Submit a batch job.

Parameters:description (xenon.JobDescription) – description
submit_interactive_job(description, stdin_stream)

Submit an interactive job The first response message in the response stream will contain the job identifier and empty stdout and stdout. Other response messages will also contain the job identifier and filled stdout and/or stderr.

wait_until_done(job=None, timeout=None)

Wait until a job is done or until a timeout expires.

Parameters:
  • job (xenon.Job) – job
  • timeout (uint64) – timeout
wait_until_running(job=None, timeout=None)

Wait until a job starts running, or until a timeout expires.

Parameters:
  • job (xenon.Job) – job
  • timeout (uint64) – timeout

Message classes

class xenon.Job(id_)

Job.

Variables:id (string) – the Xenon job identifyer.
class xenon.JobDescription(**kwargs)

This class describes a job to a Scheduler instance.

Variables:
  • executable (string) – executable
  • arguments (string) – arguments
  • working_directory (string) – working_directory
  • environment (map<string, string>) – environment
  • queue_name (string) – queue_name
  • max_runtime (uint32) – max_runtime
  • stderr (string) – stderr
  • stdin (string) – stdin
  • stdout (string) – stdout
  • name (string) – name
  • max_memory (uint32) – max_memory
  • scheduler_arguments (string) – scheduler_arguments
  • tasks (uint32) – tasks
  • cores_per_task (uint32) – cores_per_task
  • tasks_per_node (uint32) – tasks_per_node
  • start_per_task (bool) – start_per_task
  • start_time (string) – start_time
  • temp_space (uint32) – temp_space
class xenon.JobStatus(service, wrapped)

Status of a job.

Variables:
  • job (xenon.Job) – job
  • state (string) – state
  • running (bool) – running
  • done (bool) – done
  • scheduler_specific_information (map<string, string>) – scheduler_specific_information
  • exit_code (int32) – exit_code
  • error_message (string) – error_message
  • error_type (xenon.JobStatus.ErrorType) – error_type
  • name (string) – name
class ErrorType

An enumeration.

CANCELLED = 2
IO = 5
NONE = 0
NOT_CONNECTED = 3
NOT_FOUND = 1
OTHER = 6
XENON = 4
error_type
class xenon.QueueStatus(service, wrapped)

Status of a queue.

Variables:
  • name (string) – name
  • scheduler_specific_information (map<string, string>) – scheduler_specific_information
  • error_message (string) – error_message
  • error_type (xenon.QueueStatus.ErrorType) – error_type
class ErrorType

An enumeration.

IO = 4
NONE = 0
NOT_CONNECTED = 2
NOT_FOUND = 1
OTHER = 5
XENON = 3
error_type

Credentials

class xenon.CertificateCredential
certfile

Field xenon.CertificateCredential.certfile

passphrase

Field xenon.CertificateCredential.passphrase

username

Field xenon.CertificateCredential.username

class xenon.PasswordCredential
password

Field xenon.PasswordCredential.password

username

Field xenon.PasswordCredential.username

Exceptions

exception xenon.exceptions.AttributeNotSupportedException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.CopyCancelledException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.DirectoryNotEmptyException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.FileSystemClosedException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.IncompleteJobDescriptionException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.InvalidCredentialException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.InvalidJobDescriptionException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.InvalidLocationException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.InvalidOptionsException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.InvalidPathException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.InvalidPropertyException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.InvalidResumeTargetException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.NoSuchCopyException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.NoSuchJobException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.NoSuchPathException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.NoSuchQueueException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.PathAlreadyExistsException(method, exc_code, exc_msg)

Exception that is raised if FileSystem.create_directory() fails due to an existing path.

exception xenon.exceptions.PropertyTypeException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.UnknownAdaptorException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.UnknownPropertyException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.UnknownRpcException(method, exc_code, exc_msg)

Default exception if nothing is known.

exception xenon.exceptions.UnsupportedJobDescriptionException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.UnsupportedOperationException(method, exc_code, exc_msg)

TODO: add doc-string.

exception xenon.exceptions.XenonException(method, code, msg)

Xenon base exception.

exception xenon.exceptions.XenonRuntimeException(method, exc_code, exc_msg)

TODO: add doc-string.

xenon.exceptions.make_exception(method, e)

Creates an exception for a given method, and RpcError.