Houdini 20.0 HQueue

How to submit jobs

How to put work on the farm.

On this page

Submit from Houdini

  1. In Houdini, go to /out and create a HQueue Render node or HQueue Simulation node.

  2. Configure the HQueue node:

    • Set the HQueue Server parameter to HQueue server’s network address.

    • Point the HQueue node to the render node that will render (for example, a Mantra node) or simulation (for example, a Geometry ROP). You can do this either by connecting the render node into the HQueue node’s input or by setting the HQueue node’s Output Driver parameter.

    • Set the Target HFS parameter to the Houdini install directory to use on the farm.

  3. Click the Render button on the HQueue Render node or the Simulate button on the HQueue Simulation.

Submit Karma renders

  1. In Houdini, go to /out and create an HQueue Render node.

  2. Configure the HQueue node:

    • Set the HQueue Server parameter to HQueue server’s network address.

    • If you are using the Karma ROP or USD Render ROP to render with Karma, then set the Output Driver parameter to the ROP node or directly connect the ROP node to the HQueue Render node’s input.

    • If you are using the USD Render ROP LOP to render with Karma, then set the Output Driver parameter to the ROP node.

    • Set Target HFS to the Houdini install directory to use on the farm.

  3. Click the Render button on the HQueue Render node.

Submit a job from a script

Use the XML/RPC interface to script creating and managing jobs.

Submit sliced simulations

  1. Use the Distribute shelf tool to create and configure the HQueue Simulation render node.

    The shelf tool is on most dynamics shelf tabs such as Particle Fluids, Fluid Containers, Cloth, Wires, etc.

  2. For volume sliced simulations, set the Slice Divisions parameter on the HQueue Simulation ROP to break up the simulation into slices.

    The number of slices determine the number of client machines required to process the simulation. For example, if Slice Divisions is set to 2×2×2, then a total of 8 client machines are required on HQueue to simulate the scene.

    For particle sliced simulations, the Distribute shelf tool automatically inspects the number of slice planes in the simulation and sets the Number of Slices parameter accordingly.

  3. On the HQueue Simulation render node, click the Render button to submit the job.

Submit cluster simulations

There are 2 methods to setup your cluster simulation for submission to the HQueue farm.

If you created a cluster simulation using one of the shelf tools like Smoke Cluster or Pyro Cluster

Use the Distribute shelf tool, and choose “Clustering” as the distribution method when prompted. The Distribute shelf tool is on most dynamics shelf tabs such as Particle Fluids, Fluid Containers, Cloth, Wires, etc.

If you manually created a cluster simulation without using a shelf tool

You need to manually prepare your cluster simulation for distribution.

(The following procedure assumes that you have at least a Cluster Points SOP in your geometry network and that the clustered geometry is exported to DOPs for simulation.)

  1. Attach a ROP Output Driver SOP to the end your cluster simulation network. Specifically, the ROP Output Driver should be connected to the network chain that contains the Dop I/O SOP which imports the cluster simulation back into the SOP context.

  2. On the ROP Output Driver, set the Output File parameter so that it contains ${CLUSTER} in its name. This is required so the output of each cluster simulation is written to a different file. The name should also contain $F.

  3. Create a File Merge SOP in the same context as the ROP Output Driver. The merge node is used to load the cluster geometry files back into the scene.

  4. On the File Merge node, create a channel reference from the File parameter to the Geometry ROP’s Output File parameter. Set the first value of the Merge Range parameter to 0 and create a channel reference from the second value of the Merge Range parameter to the Cluster Settings->Clusters parameter on your Cluster Points SOP. Finally, set the Merge parameter to 'CLUSTER'.

  5. Navigate to /out and create an HQueue Simulation render node.

    Set the Output Driver to point to the ROP Output Driver that you created earlier. Set the Partitioning Type to “Clusters”. Finally, set the Cluster Node to your Cluster Points SOP.

  6. Configure the HQueue Simulation ROP to point to your HQueue server and the location of Houdini on the farm as usual.

  7. Click the Render button to submit the cluster simulation job.

Submit wedged simulations

  1. Navigate to /out and create an HQueue Simulation render node.

  2. Set the Output Driver parameter to point to your Wedge ROP.

  3. Set the Partitioning Type to Wedges.

  4. Configure the HQueue Simulation ROP to point to your HQueue server and the location of Houdini on the farm as usual.

  5. On the output driver node pointed by the Wedge ROP, set its output file parameter so that the filename contains ${WEDGENUM}. This is required so the output of each wedge is saved to a different file.

  6. Go back to the HQueue Simulation ROP and click the Render button to submit the job.

Submit PDG jobs

  1. In Houdini, go into your TOP network and create a HQueue Scheduler node.

  2. Configure the HQueue Scheduler node with the following information:

    • The address of the HQueue server.

    • Set Universal HFS to the Houdini install directory to use on the farm.

    • Click the Load Path Map button to load HQueue’s network folders configuration into the PDG job environment.

  3. Set the Default Scheduler on the TOP network to point to the HQueue Scheduler node.

  4. Cook the TOP network.

Submit from a non-network folder

When you submit jobs from Houdini, the .hip file does not need to be saved under a shared network folder. You can work on a file in a local directory and then automatically copy the file to a shared network folder and submit it.

  1. On the HQueue Render node or HQueue Simulation node, change the Render Current HIP File menu to “Copy Project Files to Shared Folder and Render”.

  2. Set the Project Path to where you would like to copy the .hip file to.

  3. When you click Render a file dependency dialog pops-up the following:

    Choose the files you want to copy to the shared network folder, then click OK.

Submit a job remotely

You can configure the HQueue Render node or HQueue Simulation node to submit a job for some other .hip file. This might be useful if you want to render a .hip file that’s already on a shared network folder, but you don’t have access to it.

  1. On the ROP, change the Render Current HIP File menu to “Render Target HIP File”.

  2. Set the Target HIP to the location of the .hip file on the shared network folder. You can use $HQROOT in the path to represent the root of the shared network folder.

  3. Click the Render button.

HQueue

Getting started

  • About HQueue

    HQueue is a general-purpose job scheduling system. You can use it to distribute renders, simulations, and other work to remote clients.

  • Installation

    How to set up a basic HQueue farm.

  • Configuration

    How to set configuration options for the HQueue server and clients.

  • How to submit jobs

    How to put work on the farm.

Managing the farm

  • Managing jobs

    How to view and manage jobs on the farm.

  • Managing clients

    How to use the web interface or local logins to add, remove, restart, and manage client machines.

  • Managing client groups

    How to use the web interface or local logins to create and manage groups of client machines.

  • Network Folders

    How to use the Network Folder management page.

  • Resources

    You can specify what resources (such as licenses) are available on each client, so jobs can be scheduled on clients where they can run.

  • Notes

    Each client and job can have informational notes attached.

Next steps

  • Logging

    HQueue stores separate logs for server errors and scheduling events, and each client also generates a log.

  • Uninstalling

    How to uninstall the HQueue server or client software.

  • FAQs

    Answers to frequently asked questions.

Guru level

  • Remote API

    You can connect to the HQueue server over the network and call functions to manipulate and query jobs and other information.

  • Jobs specification details

    More detail on the internals of a job specification for users who want to submit custom jobs.

  • Multiple clients, single machine

    A guide on how to run multiple HQueue clients on a single machine.