Houdini 20.0 HQueue

Resources

You can specify what resources (such as licenses) are available on each client, so jobs can be scheduled on clients where they can run.

On this page

Overview

Resources are things certain jobs require, such as Houdini licenses and Mantra Render licenses. You can specify the resources available on clients so HQueue can properly schedule jobs that require certain resources.

HQueue only schedules jobs requiring a license if at least one unused license is available, or if another job can be run on a machine already using a license. For example, Houdini, Render and Karma licenses are a “per machine resource”. That means the licenses are consumed per machine rather than per job.

You can register custom resources of your own to guide the scheduling of jobs that require other types of external resources, such as third-party licenses, database locks, and so on.

How to

In a web browser, connect to the HQueue server (http://hostname:5000/). Hover over the menu icon, then click Resources.

The Resources page summarizes the available resources and resources in use.

To...Do this

Change the number of licenses available to HQueue

If you decrease the number of a licenses available, make sure the new amount is not less than the number of licenses currently in use.

  1. On the Resource page, click Manage Resources.

  2. Set the maximum number of Houdini, Render and Karma licenses HQueue can use, then click Save.

Custom Resources

Custom resources can be per machine or per job.

Per machine

A resource used by a single machine, and can be shared across all jobs running on that machine. For example, a Houdini license is used by a workstation, and then you can run multiple instances of Houdini on that workstation.

Per job

A resource that is consumed by a single job. For example, a database connection can only be used by one job at a time.

To...Do this

Create an a custom resource

  1. On the Resources page, click Manage Resources.

  2. Under Custom Resources, click Add.

  3. Set the resource name and the maximum number of resources. If the resource is per-machine, turn on Per machine resource.

  4. Click Save.

For example, you might have a set of jobs that connect to a database and want to limit the number of concurrent connections. Create a custom per job resource to represent the connections, name it db_connection, and give it a limit amount, say 5.

Then when you submit jobs requiring the connection, add a "resources": {"db_connection": 1} property in the job specification to tell HQueue that the job will consume one db_connection resource.

Edit a custom resource

  1. On the Resources page, click Manage Resources.

  2. Under custom resources, do one of the following:

    • Edit the resource name text box.

    • Change the maximum number of available resources. (Do not make this less than the number of resources of this type currently in use.)

    • Change whether the resource is per-machine. (None of the resources of this type may be in use.)

    • Click Delete to delete the resource.

  3. Click Save.

Submitting jobs that require custom resources

When you submit custom jobs using the XML/RPC interface, you can specify that the job requires custom resources.

In the job specification, add a resources key. The value is a dictionary mapping custom resource names to the number of that resource needed by the job.

job_spec = { 
    "name": "Hello World",
    "shell": "bash",
    "command": 'echo "Hello World"'
    "resources": {"db_connection": 1}
}

See job properties for more information about the job specification.

HQueue

Getting started

  • About HQueue

    HQueue is a general-purpose job scheduling system. You can use it to distribute renders, simulations, and other work to remote clients.

  • Installation

    How to set up a basic HQueue farm.

  • Configuration

    How to set configuration options for the HQueue server and clients.

  • How to submit jobs

    How to put work on the farm.

Managing the farm

  • Managing jobs

    How to view and manage jobs on the farm.

  • Managing clients

    How to use the web interface or local logins to add, remove, restart, and manage client machines.

  • Managing client groups

    How to use the web interface or local logins to create and manage groups of client machines.

  • Network Folders

    How to use the Network Folder management page.

  • Resources

    You can specify what resources (such as licenses) are available on each client, so jobs can be scheduled on clients where they can run.

  • Notes

    Each client and job can have informational notes attached.

Next steps

  • Logging

    HQueue stores separate logs for server errors and scheduling events, and each client also generates a log.

  • Uninstalling

    How to uninstall the HQueue server or client software.

  • FAQs

    Answers to frequently asked questions.

Guru level

  • Remote API

    You can connect to the HQueue server over the network and call functions to manipulate and query jobs and other information.

  • Jobs specification details

    More detail on the internals of a job specification for users who want to submit custom jobs.

  • Multiple clients, single machine

    A guide on how to run multiple HQueue clients on a single machine.