API Documentation

multyvac.multyvac – Multyvac

class multyvac.Multyvac(api_key=None, api_secret_key=None, api_url=None)

Multyvac

The primary object for interacting with the Multyvac API. All Multyvac modules are exposed through this.

on_multyvac()

Returns True if this process is currently running on Multyvac.

send_log_to_support()

Sends this machine’s log file to Multyvac support.

exception multyvac.RequestError(http_status_code, code, message, hint=None, retry=False)

Exception class for errors when making web requests to the Multyvac API.

multyvac.job – Job

class multyvac.job.JobModule(*args, **kwargs)

Most of JobModule’s methods are exposed directly through multyvac. For example, multyvac.submit().

get(jid, fields=None)

Returns a Job with corresponding id.

Parameters:jid – Can be a job id or list of job ids.
Returns:A list if input was a list, otherwise returns an object. Also, can return None if a single jid is requested, and it cannot be found.
get_by_name(name, fields=None)

Get a job by name.

Parameters:name – The name of the job.
Returns:The most recent job with the matching name. Use list() to get a list of all matching jobs.
kill(jid)

Kills a job. If the job is queued, it will never run. If it’s processing, it will be abruptly stopped. If it’s already finished, nothing changes.

Parameters:jid – A job id or list of job ids to kill.
kill_all()

Kills all unfinished jobs.

list(jid=None, name=None, limit=50, before=None, after=None, status=None)

Query jobs.

Parameters:
  • jid – Can be a job id or list of job ids.
  • name – Can be a name or list of names.
  • limit – Maximum number of jobs to return.
  • before – Only jobs that were created before this jid are returned.
  • after – Only jobs that were created after this jid are returned.
  • status – Only jobs with this status are returned. Can be a list of statuses.

Use before and after to paginate through jobs efficiently.

Returns:A list of matching jobs.
queue_stats()

Returns a dict that shows the number of jobs that are queued and processing.

shell_submit(cmd, _name=None, _core='c1', _multicore=1, _layer=None, _vol=None, _env=None, _result_source='stdout', _result_type='binary', _max_runtime=None, _profile=False, _restartable=True, _tags=None, _depends_on=None, _stdin=None)

Submit a job to Multyvac.

Parameters:
  • cmd – The shell command to execute.
  • _name – The string name to give the job.
  • _core – The core to use.
  • _multicore – The number of cores to assign to the job.
  • _layer – The name of the layer to use.
  • _vol – The name or a list of names of volumes to mount.
  • _env – A dictionary of any environment variables that should be set in the shell running the job.
  • _result_source – The source of the result. By default it’s the standard output of the job. But it can also be set to ‘file:{path}’ where {path} should be replaced by the location of a file that will contain the job’s result.
  • _result_type – The format of the job’s result. By default, it is set to binary, which means Multyvac cannot interpret its contents.
  • _max_runtime – The maximum number of minutes this job should be allowed to run before it is forcibly killed.
  • _profile – Not implemented yet.
  • _restartable – If a server running the job fails unexpectedly, can this job be safely restarted?
  • _tags – A dict mapping keys to values of arbitrary data. Good for storing job metadata.
  • _depends_on – Not implemented.
  • _stdin – The standard input that should be piped into the job.
Returns:

Job id.

submit(f, *args, **kwargs)

Submit a Python function as a job to Multyvac.

Parameters:
  • args – The positional arguments to pass into f.
  • kwargs – The keywords arguments to pass info f. kwargs may also include special keys that are prefixed with ‘_’. See the arguments to shell_submit() for more information.

Set _ignore_module_dependencies=True as a keyword to prevent module dependencies from being automatically sync-ed. Do this only if you have setup a layer with all of your dependencies pre-installed.

Returns:Job id.
wait(jobs_or_jids, timeout=None)

An efficient way to get the results for a batch of jobs.

Blocks until all jobs are finished, and then returns a list of Jobs with output information (result, stdout, stderr, ...) already retrieved. Call each returned job’s respective get_result() function.

Parameters:
  • jobs_or_jids – A list of Job objects or jids.
  • timeout (float) – If the jobs have not finished by this many seconds, the functions return None.
Returns:

A list of jobs.

class multyvac.job.Job(jid, **kwargs)

Represents a Multyvac Job and its associated operations.

get_result(raise_on_error=True)

Better than using the result attribute directly.

First, it waits for the job to finish processing. Once finished, returns the result. If the job encountered an error (or stalled/killed) a JobError exception is raised.

Parameters:raise_on_error – If set to False, an exception is returned, rather than raised, in the event of an error.
kill()

Kills this job.

open_ssh_console()

Opens an SSH console to a running job. If a job is queued, waits indefinitely until it is processing.

run_command(cmd)

Runs the specified command over ssh. Blocks until the job has begun processing.

If successful, returns (stdout, stderr). Otherwise, returns False, which happens most notably if the job has already finished by the time this command is run.

update()

Updates this Job object with the latest version available of itself from Multyvac.

wait(status=['done', 'error', 'killed', 'stalled'], timeout=None)

Wait for the job to reach the specified status.

Parameters:status – Can be a status string or a list of strings.

Returns the status if the job reaches it, otherwise on a timeout returns False.

wait_for_open_port(port, timeout=None)

Blocks until it detects that port has been opened by the job. Returns a dict of the external address and port combination (will be different than the input port due to NAT).

If port is 22, also returns path to identity file, and username.

Parameters:timeout – The amount of time to wait for the job to start.

Once started, the job must open the port within 10 seconds. Otherwise, False is returned.

exception multyvac.job.JobError

Exception class for errors encountered by a job.

multyvac.volume – Volume

class multyvac.volume.Volume(name, **kwargs)

Represents a Multyvac Volume and its associated operations.

get_contents(path)

Returns a dict containing metadata and the contents of the file.

Parameters:path – The path to the file in this volume.
get_file(remote_path, local_path)

Copies the file from the volume to the local filesystem.

Parameters:
  • remote_path – Source path in volume.
  • local_path – Destination path in local filesystem.
ls(path)

Lists the contents of a directory.

Returns a list of dicts. Each dict specifies the path to an element, the mode, the size, and the type of element, file (f) or directory (d).

Parameters:path – Path to directory.
mkdir(path)

Creates a new directory.

Parameters:path (str) – The path to create the new directory at.
put_contents(contents, target_path, target_mode=None)

Creates a new file with the specified contents.

Parameters:
  • contents – A string of the contents of the new file.
  • target_path – The path in the volume to create the new file.
  • target_mode – The mode in octal notation of the new file. Ex. 0755.
put_file(local_path, remote_path, target_mode=None)

Copies a file from the local filesystem to the volume’s.

Parameters:
  • local_path – Source path in local filesystem.
  • remote_path – Destination path in volume.
rm(path)

Remove a file or directory.

Parameters:path – The relative path in the volume to zap.
sync_down(remote_path, local_path, max_runtime=3600)

Syncs data down from Multyvac.

Parameters:
  • remote_path – The relative path in the volume to sync from.
  • local_path – The local path to sync to.
  • max_runtime – Number of seconds to allow the syncing job to run for before it’s forcibly killed.
sync_up(local_path, remote_path, max_runtime=3600)

Syncs data up to Multyvac.

Parameters:
  • local_path – Can be a string or list of strings.
  • remote_path – The relative path in the volume to sync to.
  • max_runtime – Number of seconds to allow the syncing job to run for before it’s forcibly killed.
class multyvac.volume.VolumeModule(multyvac)

Top-level Volume module. Use this through multyvac.volume.

create(name, mount_path, mount_type=None, description=None)

Creates a new volume.

Parameters:
  • name – The name of the volume. Must not already exist.
  • mount_path – The path in the filesystem that job’s will see the volume mounted at.
  • mount_type – Currently only ‘bind’ is supported.
  • description – An optional description of the volume.
get(name)

Returns a Volume object.

Parameters:name – Name of volume.
list(name=None)

Returns a list of volume objects.

Parameters:name – A string or list of strings to filter results to only a set of volumes.
exception multyvac.volume.SyncError(exit_status, message)

Encapsulates errors when making rsync requests to Multyvac.

multyvac.layer – Layer

class multyvac.layer.LayerModule(multyvac)

Top-level Layer module. Use this through multyvac.layer.

create(name)

Creates a new layer.

Parameters:name – The name of the layer. Must not already exist.
get(name)

Returns the volume with :param name:.

list(name=None)

Returns a list of layer objects.

Parameters:name – A string or list of strings to filter results to only a set of layers.
class multyvac.layer.Layer(name, **kwargs)

Represents a Multyvac Layer and its associated operations.

get_contents(path)

Returns a dict containing metadata and the contents of the file.

Parameters:path – The path to the file in this layer.
get_file(remote_path, local_path)

Copies the file from the layer to the local filesystem.

Parameters:
  • remote_path – Source path to file in layer.
  • local_path – Destination path in local filesystem.
get_modify_layer_job(jid)

Use this if you’ve lost the ModifyLayerJob object returned by Layer.modify().

Parameters:jid – The job id of the original layer modification job.
Returns:ModifyLayerJob
ls(path)

Lists the contents of a directory.

Returns a list of dicts. Each dict specifies the path to an element, the mode, the size, and the type of element, file (f) or directory (d).

Parameters:path – Path to directory in layer.
mkdir(path)

Creates a new directory in the layer. :param path: Path to create new directory.

modify(vol=None, max_runtime=3600)

Creates a job that can be SSH-ed into. You can SSH into this job, and any modifications you make to the filesystem will become a permanent part of the layer.

Returns a ModifyLayerJob object, which is a subclass of a regular Job. You’ll want to use the Job.open_ssh_console() method to open an SSH console to do things like “apt-get”. You can also use sudo for root access.

Once you’re done, be sure to call ModifyLayerJob.snapshot() to save your layer and stop the running job. If you want to discard your changes, use ModifyLayerJob.abort().

Parameters:
  • vol – The names of any volumes you want to have mounted in the new job.
  • max_runtime – The number of seconds before the modification job is forcibly killed. This helps prevent runaway jobs.
put_contents(contents, target_path, target_mode=None)

Creates a new file with the specified contents. :param contents: A string of the contents of the new file. :param target_path: The path in the layer to create the new file. :param target_mode: The mode in octal notation of the new file.

Ex. 0755.
put_file(local_path, remote_path, target_mode=None)

Copies the file from the local filesystem to the layer.

Parameters:
  • local_path – Source path in local filesystem.
  • remote_path – Destination path in layer.
rm(path)

Remove file or directory from the layer

class multyvac.layer.ModifyLayerJob(jid, **kwargs)

A job with a few shortcut methods for modifying layers.

abort()

Aborts the changes made to the layer.

get_ssh_info()

Returns a dict containing username, address, port, and identity_file to be used to SSH into the job. If the job fails to start an SSH server for some reason, False is returned.

snapshot()

Saves the changes made to the layer

multyvac.api_key – ApiKey

class multyvac.api_key.ApiKeyModule(multyvac)

Top-level ApiKey module. Use this through multyvac.api_key.

get(id, username=None, password=None)

Returns an ApiKey.

Parameters:
  • id – The id of the ApiKey to return.
  • username – Web account credential optionally used in lieu of API credentials.
  • password – Web account credential optionally used in lieu of API credentials.
list(username=None, password=None)

Returns a list of ApiKeys.

Parameters:
  • username – Web account credential optionally used in lieu of API credentials.
  • password – Web account credential optionally used in lieu of API credentials.
class multyvac.api_key.ApiKey(id, **kwargs)

Represents a Multyvac ApiKey and its associated operations.

activate()

Activates the ApiKey.

deactivate()

Deactivates the ApiKey.

multyvac.config – Config

class multyvac.config.ConfigModule(multyvac, api_key=None, api_secret_key=None, api_url=None)

Top-level Config module. Use this through multyvac.config.

get_api_domain()

Returns the domain of the Multyvac API endpoint.

get_auth()

If a key is set, returns a tuple of (api key, api secret key). Otherwise, raises a ConfigError.

get_multyvac_path()

Returns the path on the filesystem to the multyvac configuration directory.

path_to_private_key()

Returns the path on the file system to the private key necessary to SSH into a job. Note that this private key is tied to the currently set api key.

save_to_disk()

Saves the current configuration to disk. The next time multyvac is imported, the current configurations will be available.

set_key(api_key, api_secret_key, api_url=None)

Sets the API credentials forcibly replacing any other credentials that have been set.

Parameters:
  • api_key – The ApiKey id.
  • api_secret_key – The ApiKey secret.
  • api_url – The API end point to use.
exception multyvac.config.ConfigError

Raised when Multyvac’s configuration is incomplete or incorrect.