Remote API package

You can connect to the HQueue server over the network and call functions to manipulate and query jobs and other information.

On this page

Overview
Connecting to the server
Connecting to clients
Client API
Server API
- Queue
- Clients
- Client groups
- Job management
- Job queries
- Meta
- Resources
- Configuration
- Notes
- Deprecated functions

Overview ¶

You can call functions on the HQueue server through an XML/RPC¹ client (from the same machine or over the network). You can use the server’s API to submit or query jobs.

XML/RPC is a simple language-agnostic standard for translating function calls and return values into HTTP requests and responses. Python comes with a built-in XML/RPC client library, so you don’t actually need to know anything about the protocol, it is simply the underlying technology that makes remote calls possible.

You can also connect and call functions on HQueue clients, however the API is quite limited.

Connecting to the server ¶

In Python 3 you can connect to the server using the xmlrpc.client library (In Python 2 this was called xmlrpclib). XML/RPC clients are available in other languages, however we will limit ourselves to showing Python on this page.

import xmlrpc.client

# Connect to the HQueue server
hq = xmlrpc.client.ServerProxy("http://hq_server_hostname:5000")

The returned object acts as a proxy for the remote API: you can call function attributes on the object as if you were calling functions inside the remote HQueue server. See the list of available functions below.

If the connection is not valid, trying to call a function on the proxy will raise an exception:

try:
    hq.ping()
except ConnectionRefusedError:
    print("HQueue server is not available")

Note

Check the XML/RPC client library documentation to see what kind of exceptions it will raise.

The following examples show how you call functions on the proxy object:

Submit a new job:

# Define a job that renders an image from an IFD using Mantra.
job_spec = {
    "name": "Render My Image",
    "shell": "bash",
    "command": (
        "cd $HQROOT/houdini_distros/hfs;"
        " source houdini_setup;"
        " mantra < $HQROOT/path/to/ifds/some_frame.ifd"
    )
}

job_ids = hq.newjob(job_spec)

You can use the job ID(s) returned by newjob() to query job progress using getJob():

job = hq_server.getJob(job_ids[0], ["progress", "status"])
status = job["status"]

print("The job status is", status)
if status in hq.getRunningJobStatusNames():
    print("{:.2f}% complete".format(job["progress"] * 100.0))

Connecting to clients ¶

HQueue clients also act as an XML/RPC API server. This allows you to call functions that force the client to send a heartbeat to the server, or cause the client to exit.

import xmlrpc.client

# Connect to the HQueue client.
hq_client = xmlrpc.client.ServerProxy("http://hq_client_hostname:5001")

Client API ¶

getVersion() → str

Returns the HQueue client version.

ping() → bool

Test whether the HQueue client is alive and active. Always returns True.

Test whether the HQueue client is alive and active.

wakeup()

Wake up the client if it is sleeping and force it to send a hearbeat to the server.

terminate()

Request the client to terminate. Terminating a client results in the following:

Any paused jobs on the client are canceled.
New jobs cannot be assigned to the client.
Any jobs currently running on the client continue executing until they are complete.
The client is removed from the list of clients in the HQueue server.
Once all jobs on the client are finished, the client process exits.

Server API ¶

Queue ¶

runcmd(cmd) → int

Executes the given command on the HQueue farm. This is done by creating a new anonymous job with its command property set to cmd. The job is assigned to a client machine and processed.

Returns the id of the anonymous job.

Executes the given command on the HQueue farm.

getNumWaitingJobs(wait_time=0, min_priority=-1, max_priority=-1) → int

Returns the number of jobs waiting for at least wait_time minutes.

A waiting job is a job waiting for a resource; for example, an available client machine.

If wait_time is 0 or negative, returns the total number of waiting jobs.

If min_priority is greater than or equal to 0, only jobs with priorities greater than or equal to min_priority are included in the returned number. If it is less than 0 then there is no minimum priority restriction.

If max_priority is greater than or equal to 0, only jobs with priorities less than or equal to max_priority are included in the returned number. If it is less than 0 then there is no maximum priority restriction.

Returns the number of jobs waiting for at least wait_time minutes.

getServerTime() → xmlrpc.client.DateTime

Returns the current time reported by the HQueue server. The time is in UTC.

Returns the current time reported by the HQueue server.

getServerErrors() → dict

Returns the errors reported by the HQueue server.

The returned dictionary contains two keys: total and errors. total is the number of server errors, and errors is the list of errors (each error is a dict).

Returns an empty dict if no error is reported.

Returns the errors reported by the HQueue server.

getVersion() → str

Returns the HQueue server version.

ping() → bool

Test whether the HQueue server is alive and active. Always returns True.

Test whether the HQueue server is alive and active.

Clients ¶

getClients(client_ids=None, attribs=None) → list of dict

Returns a list of dictionaries where each dictionary contains the attributes for a client specified in client_ids.

client_ids must be a list of client id numbers. If it is not specified or is set to None, then attributes for all registered clients are returned.

You can optionally set the attribs argument to a list of client attributes that will be returned. This can help reduce the function’s execution time and the amount of data returned.

Here is a list of client attributes:

available - whether the client is available to process jobs at the current time or not
availabilityRules - list of rules outlining the client’s availability on the farm
cpus - number of cpus
cpuSpeed - processor speed
description - the client’s description if set. An empty string is returned if no description is set for the client.
disabled - whether the client has been disabled on the farm or not
groups - list of group ids that the client is a member of
hostname - client machine name
id - client id
idleTime - number of minutes that the client has been idle (no activity from the logged-in user). Only applies to workstations. Is set to -1 if the idle time is unknown.
ip - client’s IP address
label - client’s display label. If the client has no name set, the label is the client’s hostname. Otherwise the label is hostname:name where hostname and name are the client’s hostname and name respectively.
lastHeartbeat - time when the client last contacted the server. The time is in UTC.
latestNote - the text of the latest note from the thread corresponding to the client. An empty string is returned if there is no corresponding thread.
load - client load
memory - amount of physical memory in kilobytes
memoryInUse - amount of physical memory in kilobytes currently in-use by processes
name - the name of the client if a name has been specified in the client’s hqnode.ini file. By default this is empty (i.e. "").
needsUpgrade - whether the client’s code is out-of-date and requires an update
platform - client’s operating system and architecture
runningJobs - ids for the jobs currently running on the client
threadId - the id of the note thread corresponding to the client. null is returned if there is no corresponding thread.
version - hash string representing the version of the HQueue code installed on the client.

Returns a list of dictionaries where each dictionary contains the attributes for a client specified in client_ids.

getClient(client_id, attribs=None) → dict

Convenience function for getClients(). Returns a dictionary of attributes for the specified client.

Convenience function for getClients().

getClientByName(clientname, attribs=None) → dict

Convenience function for getClients(). Returns a dictionary of attributes for the specified client.

clientname can be in the hostname format to specify a client by hostname or the hostname:name format to refer to a client with the specified hostname and name. By default, clients have an empty ("") name.

Convenience function for getClients().

getClientsByName(clientnames, attribs=None) → list of dict

Convenience function for getClients(). Returns a list of dictionaries where each dictionary contains the attributes for a client specified by a clientname in clientnames.

clientnames must be a list of clientnames. You can list clientnames in the hostname format to specify a client by hostname, or in the hostname:name format to refer to a client with the specified hostname and name.

Convenience function for getClients().

getClientErrors(client_id) → dict

Returns the errors reported by the a client specified in client_ids.

The returned dictionary contains two keys: total and errors. total is the number of client errors, and errors is the list of errors (each error is a dict).

Returns an empty dict if no error is reported.

Returns the errors reported by the a client specified in client_ids.

upgradeClients(client_ids)

Upgrades the clients associated with the list of client_ids.

Upgrades the clients associated with the list of client_ids.

Client groups ¶

createClientGroup(name) → int

Creates a new client group with name specified by name. Returns the id of the new group.

Creates a new client group with name specified by name.

getClientGroups(group_ids=None, attribs=None) → list of dict

Returns a list of dictionaries where each dictionary contains the attributes for a client group specified in group_ids.

group_ids must be a list of group id numbers. If it is not specified or is set to None, then attributes for all registered client groups are returned.

You can optionally set the attribs argument to a list of client group attributes that is returned. This can help reduce the function’s execution time and the amount of data returned.

Here is a list of client group attributes:

clients - list of ids for clients that are members of the group.
id - group id
name - group name

Returns a list of dictionaries where each dictionary contains the attributes for a client group specified in group_ids.

getClientGroup(group_id, attribs=None) → dict

Convenience function for getClientGroups(). Returns a dictionary of attributes for the specified client group.

Convenience function for getClientGroups().

getClientGroupByName(name, attribs=None) → dict

Convenience function for getClientGroups(). Returns a dictionary of attributes for the specified client group.

Convenience function for getClientGroups().

addClientToGroup(group_id, client_id)

Adds the client associated with the client_id to the group associated with the group_id.

Adds the client associated with the client_id to the group associated with the group_id.

removeClientFromGroup(group_id, client_id)

Removes the client associated with client_id from the group associated with the group_id.

Removes the client associated with client_id from the group associated with the group_id.

deleteClientGroup(group_id)

Deleted the client group associated with the group_id.

Deleted the client group associated with the group_id.

Job management ¶

newjob(jobspec,parent_id=None,child_ids=[]) → list of int

Submit a job to the server. jobspec can be either a JSON string, a dictionary or a list of dictionaries representing the job specification(s). See The Job Specification for details on job specifications.

The optional argument parent_id is an id for the job that the new job should be parented to.

The optional argument child_ids is a list of ids for existing jobs that should be parented to the new job.

Returns a list of ids for the new jobs.

Submit a job to the server.

getUnfinishedRootJobIds() → list of int

Returns a list of job ids for all unfinished root jobs. Root jobs have no parent.

Returns a list of job ids for all unfinished root jobs.

getFinishedRootJobIds() → list of int

Returns a list of job ids for all finished root jobs. Root jobs are jobs that have no parent.

Returns a list of job ids for all finished root jobs.

pauseJobs(job_ids)

Pauses all the jobs linked to each job in the list of job_ids.