API¶
Contents
The Server¶
-
xenon.
init
(port=None, do_not_exit=False, disable_tls=False, log_level='WARNING')¶ Start the Xenon GRPC server on the specified port, or, if a service is already running on that port, connect to that.
If no port is given, a random port is selected. This means that, by default, every python instance will start its own instance of a xenon-grpc process.
Parameters: - port – the port number
- do_not_exit – by default the GRPC server is shut down after Python exits (through the atexit module), setting this value to True will prevent that from happening.
File Systems¶
-
class
xenon.
FileSystem
(service, wrapped)¶ The Xenon FileSystem subsystem.
Variables: id (string) – id -
append_to_file
(path, data_stream)¶ Open an existing file and return an OutputStream to append data to this file.
-
cancel
(copy_operation=None)¶ Cancel a copy operation.
Parameters: copy_operation (xenon.CopyOperation) – copy_operation
-
close
()¶ Close this filestem Any pending/running copy operations of this filestystem will be terminated Will also forget this filesystem
-
copy
(source=None, destination_filesystem=None, destination=None, mode=None, recursive=None)¶ Asynchronously Copy an existing source path to a target path on a different file system.
Parameters: - source (xenon.Path) – source
- destination_filesystem (xenon.FileSystem) – destination_filesystem
- destination (xenon.Path) – destination
- mode (xenon.CopyRequest.CopyMode) – mode
- recursive (bool) – recursive
-
classmethod
create
(adaptor=None, location=None, properties=None, certificate_credential=None, password_credential=None, default_credential=None, credential_map=None, keytab_credential=None)¶ Create a new FileSystem using the adaptor that connects to a data store at location using the credentials to get access.
Parameters: - adaptor (string) – adaptor
- location (string) – location
- properties (map<string, string>) – properties
- certificate_credential (xenon.CertificateCredential) – certificate_credential
- password_credential (xenon.PasswordCredential) – password_credential
- default_credential (xenon.DefaultCredential) – default_credential
- credential_map (xenon.CredentialMap) – credential_map
- keytab_credential (xenon.KeytabCredential) – keytab_credential
-
create_directories
(path=None)¶ Creates a new directory, including parent directories, failing if the directory already exists.
Parameters: path (xenon.Path) – path
-
create_directory
(path=None)¶ Creates a new directory, failing if the directory already exists.
Parameters: path (xenon.Path) – path
-
create_file
(path=None)¶ Creates a new empty file, failing if the file already exists.
Parameters: path (xenon.Path) – path
-
create_symbolic_link
(link=None, target=None)¶ Creates a new symbolic link, failing if the link already exists
Parameters: - link (xenon.Path) – link
- target (xenon.Path) – target
-
delete
(path=None, recursive=None)¶ Deletes an existing path.
Parameters: - path (xenon.Path) – path
- recursive (bool) – recursive
-
exists
(path=None)¶ Tests if a path exists.
Parameters: path (xenon.Path) – path
-
classmethod
get_adaptor_description
(name=None)¶ Gives the description of the adaptor with the given name.
Parameters: name (string) – name
-
classmethod
get_adaptor_descriptions
()¶ Gives a list of the descriptions of the available adaptors.
-
get_adaptor_name
()¶ Get the name of the adaptor that created this FileSystem.
-
classmethod
get_adaptor_names
()¶ Gives a list names of the available adaptors.
-
get_attributes
(path=None)¶ Get the PathAttributes of an existing path.
Parameters: path (xenon.Path) – path
-
get_path_separator
()¶ Get the path separator used by this file system.
-
get_status
(copy_operation=None)¶ Retrieve the status of an copy.
Parameters: copy_operation (xenon.CopyOperation) – copy_operation
-
get_working_directory
()¶ Get the current working directory of this file system.
-
is_open
()¶ Return if the connection to the FileSystem is open.
-
list
(dir=None, recursive=None)¶ List all entries in the directory dir.
Parameters: - dir (xenon.Path) – dir
- recursive (bool) – recursive
-
classmethod
list_file_systems
()¶ List the created filesystems Specific to grpc, not part of Xenon library
-
classmethod
local_file_systems
()¶ Returns filesystems for all local drives Not part of FileSystem class in Xenon library In Xenon library available as LocalFileSystemUtils.getLocalFileSystems()
-
read_from_file
(path=None)¶ Open an existing file and return an InputStream to read from this file.
Parameters: path (xenon.Path) – path
-
read_symbolic_link
(path=None)¶ Reads the target of a symbolic link
Parameters: path (xenon.Path) – path
-
rename
(source=None, target=None)¶ Rename an existing source path to a non-existing target path
Parameters: - source (xenon.Path) – source
- target (xenon.Path) – target
-
set_posix_file_permissions
(path=None, permissions=None)¶ Sets the POSIX permissions of a path
Parameters: - path (xenon.Path) – path
- permissions (xenon.PosixFilePermission) – permissions
-
set_working_directory
(path=None)¶ Set the current working directory of this file system to directory.
Parameters: path (xenon.Path) – path
-
wait_until_done
(copy_operation=None, timeout=None)¶ Wait until a copy operation is done or until a timeout expires.
Parameters: - copy_operation (xenon.CopyOperation) – copy_operation
- timeout (uint64) – timeout
-
write_to_file
(path, data_stream)¶ Open a file and return an OutputStream to write to this file. In Xenon library if request is missing size field then FileSystem.writeToFile(Path file) is used else FileSystem.writeToFile(Path path, long size) is used
-
-
class
xenon.
Path
(path)¶ Wrapper around
PurePosixPath
form thepathlib
module. This class reveals a string representation of the underlying path object to GRPC. You may use this class like a pathlib.PurePosixPath, including using it as an argument to open calls as it derives from os.PathLike (Python > 3.6). For more information see the Python documentation on pathlib.Checks if a file is hidden. Just compares the first character in the filename with ‘.’.
Message classes¶
-
class
xenon.
PosixFilePermission
¶ An enumeration.
-
GROUP_EXECUTE
= 6¶
-
GROUP_READ
= 4¶
-
GROUP_WRITE
= 5¶
-
NONE
= 0¶
-
OTHERS_EXECUTE
= 9¶
-
OTHERS_READ
= 7¶
-
OTHERS_WRITE
= 8¶
-
OWNER_EXECUTE
= 3¶
-
OWNER_READ
= 1¶
-
OWNER_WRITE
= 2¶
-
-
class
xenon.
CopyStatus
(service, wrapped)¶ Status of a copy operation.
Variables: - copy_operation (xenon.CopyOperation) – copy_operation
- bytes_copied (uint64) – bytes_copied
- bytes_to_copy (uint64) – bytes_to_copy
- done (bool) – done
- running (bool) – running
- state (string) – state
- error_message (string) – error_message
- error_type (xenon.CopyStatus.ErrorType) – error_type
-
class
ErrorType
¶ An enumeration.
-
ALREADY_EXISTS
= 3¶
-
CANCELLED
= 2¶
-
NONE
= 0¶
-
NOT_CONNECTED
= 4¶
-
NOT_FOUND
= 1¶
-
XENON
= 5¶
-
-
error_type
¶
Schedulers¶
-
class
xenon.
Scheduler
(service, wrapped)¶ The Xenon Schedulers subsystem.
Variables: id (string) – id -
close
()¶ Close this Scheduler. If scheduler is embedded then any pending/running jobs will be killed Will also forget this scheduler
-
classmethod
create
(adaptor=None, location=None, properties=None, certificate_credential=None, password_credential=None, default_credential=None, credential_map=None, keytab_credential=None)¶ Create a new Scheduler using the adaptor connecting to the location using credentials to get access.
Parameters: - adaptor (string) – adaptor
- location (string) – location
- properties (map<string, string>) – properties
- certificate_credential (xenon.CertificateCredential) – certificate_credential
- password_credential (xenon.PasswordCredential) – password_credential
- default_credential (xenon.DefaultCredential) – default_credential
- credential_map (xenon.CredentialMap) – credential_map
- keytab_credential (xenon.KeytabCredential) – keytab_credential
-
classmethod
get_adaptor_description
(name=None)¶ Gives the description of the adaptor with the given name.
Parameters: name (string) – name
-
classmethod
get_adaptor_descriptions
()¶ Gives a list of the descriptions of the available adaptors.
-
get_adaptor_name
()¶ Get the name of the adaptor that created this Scheduler.
-
classmethod
get_adaptor_names
()¶ Gives a list names of the available adaptors.
-
get_default_queue_name
()¶ Get the name of the default queue.
-
get_file_system
()¶ Retrieve the FileSystem used internally by this Scheduler.
-
get_job_statuses
(jobs=None)¶ Get the status of all specified jobs.
Parameters: jobs (xenon.Job) – jobs
-
get_jobs
(queues=None)¶ Get all job identifier of jobs currently in (one ore more) queues.
Parameters: queues (string) – queues
-
get_location
()¶ Get the location that this Scheduler is connected to.
-
get_properties
()¶ Get the properties used to create this Scheduler.
-
get_queue_names
()¶ Get the queue names supported by this Scheduler.
-
get_queue_status
(queue=None)¶ Get the status of the queue.
Parameters: queue (string) – queue
-
get_queue_statuses
(queues=None)¶ Get the status of all queues.
Parameters: queues (string) – queues
-
is_open
()¶ Test if the connection of this Scheduler is open.
-
classmethod
list_schedulers
()¶ List the created schedulers Specific to grpc, not part of Xenon library
-
classmethod
local_scheduler
()¶ Get scheduler on local filesystem with default location, credential and no properties Not part of Scheduler class in Xenon library In Xenon library available as Scheduler.create(“local”)
-
submit_batch_job
(description=None)¶ Submit a batch job.
Parameters: description (xenon.JobDescription) – description
-
submit_interactive_job
(description, stdin_stream)¶ Submit an interactive job The first response message in the response stream will contain the job identifier and empty stdout and stdout. Other response messages will also contain the job identifier and filled stdout and/or stderr.
-
Message classes¶
-
class
xenon.
Job
(id_)¶ Job.
Variables: id (string) – the Xenon job identifyer.
-
class
xenon.
JobDescription
(**kwargs)¶ This class describes a job to a Scheduler instance.
Variables: - executable (string) – executable
- arguments (string) – arguments
- working_directory (string) – working_directory
- environment (map<string, string>) – environment
- queue_name (string) – queue_name
- max_runtime (uint32) – max_runtime
- stderr (string) – stderr
- stdin (string) – stdin
- stdout (string) – stdout
- name (string) – name
- max_memory (uint32) – max_memory
- scheduler_arguments (string) – scheduler_arguments
- tasks (uint32) – tasks
- cores_per_task (uint32) – cores_per_task
- tasks_per_node (uint32) – tasks_per_node
- start_per_task (bool) – start_per_task
- start_time (string) – start_time
- temp_space (uint32) – temp_space
-
class
xenon.
JobStatus
(service, wrapped)¶ Status of a job.
Variables: - job (xenon.Job) – job
- state (string) – state
- running (bool) – running
- done (bool) – done
- scheduler_specific_information (map<string, string>) – scheduler_specific_information
- exit_code (int32) – exit_code
- error_message (string) – error_message
- error_type (xenon.JobStatus.ErrorType) – error_type
- name (string) – name
-
class
ErrorType
¶ An enumeration.
-
CANCELLED
= 2¶
-
IO
= 5¶
-
NONE
= 0¶
-
NOT_CONNECTED
= 3¶
-
NOT_FOUND
= 1¶
-
OTHER
= 6¶
-
XENON
= 4¶
-
-
error_type
¶
-
class
xenon.
QueueStatus
(service, wrapped)¶ Status of a queue.
Variables: - name (string) – name
- scheduler_specific_information (map<string, string>) – scheduler_specific_information
- error_message (string) – error_message
- error_type (xenon.QueueStatus.ErrorType) – error_type
-
class
ErrorType
¶ An enumeration.
-
IO
= 4¶
-
NONE
= 0¶
-
NOT_CONNECTED
= 2¶
-
NOT_FOUND
= 1¶
-
OTHER
= 5¶
-
XENON
= 3¶
-
-
error_type
¶
Credentials¶
Exceptions¶
-
exception
xenon.exceptions.
AttributeNotSupportedException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
CopyCancelledException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
DirectoryNotEmptyException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
FileSystemClosedException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
IncompleteJobDescriptionException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
InvalidCredentialException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
InvalidJobDescriptionException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
InvalidLocationException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
InvalidOptionsException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
InvalidPathException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
InvalidPropertyException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
InvalidResumeTargetException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
NoSuchCopyException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
NoSuchJobException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
NoSuchPathException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
NoSuchQueueException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
PathAlreadyExistsException
(method, exc_code, exc_msg)¶ Exception that is raised if
FileSystem.create_directory()
fails due to an existing path.
-
exception
xenon.exceptions.
PropertyTypeException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
UnknownAdaptorException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
UnknownPropertyException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
UnknownRpcException
(method, exc_code, exc_msg)¶ Default exception if nothing is known.
-
exception
xenon.exceptions.
UnsupportedJobDescriptionException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
UnsupportedOperationException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
exception
xenon.exceptions.
XenonException
(method, code, msg)¶ Xenon base exception.
-
exception
xenon.exceptions.
XenonRuntimeException
(method, exc_code, exc_msg)¶ TODO: add doc-string.
-
xenon.exceptions.
make_exception
(method, e)¶ Creates an exception for a given method, and RpcError.