Installation

Before you begin ¶

Note

This guide assumes the server and client machines all share the same operating system (Linux, macOS, or Windows). It’s possible to set up an HQueue farm with mixed operating systems, but this guide doesn’t cover it.

The following items are required before setting up an HQueue farm:

Network connectivity	Machines must be able to “see” or ping each other by their domain (DNS) names or IP addresses. Client machines must be able to connect to the server machine through port `5000`. The server machine must be able to connect to client machines through port `5001`. This may require opening ports in any local firewalls.
Shared network folder	The server and clients must be able to access a shared network filesystem, for example an NFS mount or Samba (SMB) share. The shared folder must be mounted at the same location on every machine, for example /mnt/storage/ on Linux, or H:/ on Windows.
Houdini	Houdini must be installed on every client machine and in the same install directory. Mac On macOS, the default Houdini installation path is /Applications/Houdini/HoudiniX.Y.ZZZ. Windows On Windows, the default Houdini installation path is C:/Program Files/Side Effects Software/Houdini X.Y.ZZZ. Linux On Linux, the default Houdini installation path is /opt/hfsX.Y.ZZZ. Houdini licensing must also be set up on each client machine so the client software can launch licensed executables such as `hython`, `mantra` and `karma`.

Installing HQueue ¶

Setting up an HQueue farm involves installing the HQueue server, registering a network folder with HQueue and adding client machines to the farm.

Note

HQueue installation is currently not available in the Houdini Launcher. Use the Houdini installer instead to install HQueue.

Installing HQueue on Linux ¶

Install the HQueue server
1. Choose a machine, for example serverMachine, to run the HQueue server.
2. Download the Houdini installer tarball on the chosen machine.
3. Unpack the tarball and run the Houdini installer with sudo.
  
  For example:
```
tar zxf houdini-X.X.XXX-linux_x86_64_gccY.Y.tar.gz
cd houdini-X.X.XXX-linux_x86_64_gccY.Y
sudo ./houdini_install
```
4. In the installer:
  - Uncheck all components except for HQueue Server.
  - Proceed with the installation and when prompted, leave all HQueue server options at their default values.
5. After the installer finishes, the server installs at /opt/hqueue.
6. You can open a web browser and navigate to http://serverMachine:5000 to confirm that the HQueue server is running.
  
  Note
  
  If DNS is not configured for serverMachine then you can connect to HQueue using the server machine’s IP address.
  
  For example, http://X.X.X.X:5000.
  
  Note
  
  The HQueue server doesn’t automatically start up after a machine reboot. To configure the server to start up automatically, create a systemd script that points to /opt/hqueue/scripts/hqserverd.
Register the network folder with HQueue
1. Open a web browser and go to the HQueue web interface. For example, http://serverMachine:5000.
2. Hover the mouse over the “triple bars” icon in the top-right corner and then select Network Folders.
3. In the Network Folder 1 box, set the Linux Mount field to where the network folder is mounted (i.e. /Users/Shared/myShare). Leave all other fields at their default values.
4. Save the changes. References to $HQROOT in HQueue jobs and Houdini parameters resolve to the value of the Linux Mount field on Linux machines.
Add a client machine
1. Download the Houdini installer tarball on the client machine.
2. Unpack the tarball and run the Houdini installer without sudo.
  
  For example:
```
tar zxf houdini-X.X.XXX-linux_x86_64_gccY.Y.tar.gz
cd houdini-X.X.XXX-linux_x86_64_gccY.Y
./houdini_install
```
  Note
  
  The HQueue client will execute jobs with the user account that runs the ./houdini_install installer script.
3. In the installer:
  - When asked to re-run the installer as root, enter “no”.
  - Uncheck all components except for HQueue Client.
  - Proceed with the installation and when prompted for the HQueue server to connect to, set the URL to point to your farm. For example, http://serverMachine:5000.
    
    Note
    
    If DNS is not configured for serverMachine then you can alternatively use the server machine’s IP address in the URL.
    
    For example, http://X.X.X.X:5000.
  - Leave the rest of the HQueue client options at their default values.
4. After the installer finishes, the HQueue client installs in your home directory, $HOME/hqclient.
5. Go to the clients page in the HQueue web interface to confirm that the machine was successfully added to the farm.
Note

Repeat these steps for each additional client machine you want to add to the farm.

Installing HQueue on macOS ¶

Install the HQueue server
1. Choose a machine, for example serverMachine, to run the HQueue server.
2. Download and run the Houdini installer on the chosen machine.
3. In the installer, check HQueue Server in the components list and uncheck all other components. Proceed with the installation.
4. After the installer finishes, the server installs at /Library/HQueueServer and a new com.sidefx.hqserver.plist service file exists in /Library/LaunchDaemons. The service files ensures that the server automatically starts up after a machine reboot.
5. You can open a web browser and navigate to http://serverMachine:5000 to confirm that the HQueue server is running.
  
  Note
  
  If DNS is not configured for serverMachine then you can connect to HQueue using the server machine’s IP address.
  
  For example, http://X.X.X.X:5000.
Register the network folder with HQueue
1. Open a web browser and go to the HQueue web interface. For example, http://serverMachine:5000.
2. Hover the mouse over the “triple bars” icon in the top-right corner and then select Network Folders.
3. In the Network Folder 1 box, set the macOS Mount field to where the network folder is mounted (i.e. /Users/Shared/myShare). Leave all other fields at their default values.
4. Save the changes. References to $HQROOT in HQueue jobs and Houdini parameters resolve to the value of the macOS Mount field on macOS machines.
Add a client machine
1. Download and run the Houdini installer on the client machine.
2. In the installer, check HQueue Client (files only) in the components list and uncheck all other componentts. Proceed with the installation.
3. After the installer finishes, open a macOS Terminal app and in the terminal run sudo /Library/HQueueClient/install.sh to run the installer script that completes the client installation.
4. In the installer script:
  - When prompted, enter the user account to run the HQueue client. The user account is also used to run jobs assigned to the client. If you chose hquser as the user account, then the installer script creates the user account. See /hqueue/gettingstarted.html#hquser_account for more information.
  - When prompted, enter the server machine name and server port number. This is how the HQueue client locates and connect to the HQueue server.
5. After the installer script finishes, the client installs at /Library/HQueueClient and a new com.sidefx.hqclient.plist service file exists in /Library/LaunchDaemons. The service file ensures that the client automatically starts up after a machine reboot.
6. Go to the clients page in the HQueue web interface to confirm that the machine was successfully added to the farm.
Note

Repeat these steps for each additional client machine you want to add to the farm.

Installing HQueue on Windows ¶

Install the HQueue server
1. Choose a machine, for example serverMachine, to run the HQueue server.
2. Download and run the Houdini installer on the chosen machine.
3. In the installer, check HQueue Server in the components list and uncheck all other components.
4. Proceed with the installation and when prompted, leave all HQueue server options at their default values.
5. After installation, the server installs at C:/HQueueServer and a new HQueueServer service exists in Windows Services. The service ensures that the server automatically starts up after a machine reboot.
6. You can open a web browser and navigate to http://serverMachine:5000 to confirm that the HQueue server is running.
  
  Note
  
  If DNS is not configured for serverMachine then you can connect to HQueue using the server machine’s IP address.
  
  For example, http://X.X.X.X:5000.
Register the network folder with HQueue
1. Open a web browser and go to the HQueue web interface. For example, http://serverMachine:5000.
2. Hover the mouse over the “triple bars” icon in the top-right corner and then select Network Folders.
3. In the Network Folder 1 box, set the Windows Mount field to where the network folder is mounted. You can enter either a mapped network drive lettter (i.e. H:) or a UNC path (i.e. \\diskServer\shareName). Leave all other fields at their default values.
4. Save the changes. References to $HQROOT in HQueue jobs and Houdini parameters resolve to the value of the Windows Mount field on Windows machines.
Add a client machine
1. Download and run the Houdini installer on the client machine.
2. In the installer:
  - Check HQueue Client in the components list and uncheck all other components.
  - When prompted to enter the HQueue server information, set the Name field to the machine hosting the HQueue server. For example, serverMachine. Additionally, set the Port Number field to the HQueue server’s listen port (default is 5000). This is how the HQueue client locates and connect to the HQueue server.
    
    Note
    
    If DNS is not configured for serverMachine then you can enter the server machine’s IP address in the Name field.
  - Leave the rest of the HQueue client options at their default values.
3. After the installer finishes, the client installs at C:/HQueueClient and a new HQueueClient scheduled task is created to ensure that the client launches at startup on a machine reboot. The scheduled task can be found in the Windows Task Scheduler app.
4. Reboot the client machine after installation. Otherwise, the client may not have access to mapped network drives.
Note

Repeat these steps for each additional client machine you want to add to the farm.

Testing the Farm ¶

To test that the HQueue farm is configured correctly you can submit a simple Houdini render job:

Launch Houdini.
Create a simple scene. At a minimum, create some geometry, a light, a camera and a Mantra ROP for rendering.
Navigate to the /out node network and create an HQueue Render ROP.
In the Parameters pane, set the Output Driver parameter to your Mantra ROP.
Point the HQueue Server parameter to your HQueue farm.

For example, http://serverMachine:5000.

Note

If DNS is not configured for serverMachine then you can alternatively use the server machine’s IP address in the URL. For example, http://X.X.X.X:5000.
Then set Target HFS to where Houdini is installed on the client machines.
Mac
For macOS, set Target HFS to:
/Applications/Houdini/HoudiniX.Y.ZZZ/Frameworks/Houdini.framework/Versions/Current/Resources
Windows
For Windows, set Target HFS to:
C:/Program Files/Side Effects Software/Houdini X.Y.ZZZ
Linux
For Linux, set Target HFS to:
/opt/hfsX.Y.ZZZ
Save the .hip file in your (HQROOT) network folder. You can find the HQROOT network folder settings on the HQueue webs interface in the Network Folders web page.

Mac

On macOS, the network folder is defined by the macOS Mount field for the HQROOT network folder. For example, save the .hip file to /Volumes/myShare/test.hip.

Windows

On Windows, the network folder is defined by the Windows Mount field for the HQROOT network folder. For example, save the .hip file to H:/test.hip.

Linux

On Linux, the network folder is defined by the Linux Mount field for the HQROOT network folder. For example, save the .hip file to /mnt/myShare/test.hip.
At the top of the Parameters pane, click Submit Job.
- A new job appears in the HQueue web interface.
- You can click on the job’s id to view more details.

Troubleshooting ¶

When a job fails, you can troubleshoot it by clicking on the job’s id in the HQueue web interface. This takes you inside the job to view more details. If any of the job’s Child Jobs failed, then click on the child job’s id to further troubleshoot it. The Output Log contains an error message explaining why the job failed.

You can learn about common job failure errors and possible solutions in the Troubleshooting FAQs.

Scripted Installations ¶

You can perform the HQueue installation from the command-line allowing for scripted installations. This is beneficial if you want to automate the installation and configuration of an HQueue farm.

Scripted Installation on Linux ¶

Scripted installation on Linux begin with the Houdini installer tarball, houdini-X.X.XXX-linux_x86_64_gccY.Y.tar.gz. Download the tarball from the SideFX website.

Install the HQueue server

Run the following commands to install the HQueue server:

# Unpack the installer tarball.
tar zxf houdini-X.X.XXX-linux_x86_64_gccY.Y.tar.gz

# Locate the unpacked installer directory.
cd houdini-X.X.XXX-linux_x86_64_gccY.Y

# Run the installer script to install the server.
sudo ./houdini.install --auto-install --accept-EULA 20XX-XX-XX --no-install-houdini –-no-install-license --install-hqueue-server --hqueue-server-dir /opt/hqueue --upgrade-hqueue-server-if-exists yes

The HQueue server installs to /opt/hqueue and runs with the root account.

Install the HQueue client

Run the following commands to install the HQueue client:

# Unpack the installer tarball.
tar zxf houdini-X.X.XXX-linux_x86_64_gccY.Y.tar.gz

# Locate the unpacked installer directory.
cd houdini-X.X.XXX-linux_x86_64_gccY.Y

# Run the installer script to install the client.
sudo ./houdini.install --auto-install --accept-EULA 20XX-XX-XX --no-install-houdini –-no-install-license --install-hqueue-client --hqueue-client-dir /home/myUser/hqclient --hqueue-client-user myUser --hqueue-server-name myServer --hqueue-server-port 5000

The HQueue client installs to /home/myUser/hqclient and runs with the myUser account. The client connects to the HQueue server located at http://myServer:5000.

Scripted Installation on macOS ¶

Scripted installation on macOS begin with the Houdini installer image, houdini-X.X.XXX-macosx_YYY_clangZZ_ZZ.dmg. Download the image from the SideFX website.

(Prerequisite) Generate Installer Choice XML Files
- Scripted installation on macOS requires pre-generated XML files that tell the installer what components to install.
- Run the following commands to generate the choices XML file:
```
# Ensure that no previous Houdini installer image is mounted.
sudo hdiutil detach /Volumes/Houdini

# Mount the Houdini installer image.
sudo hdiutil attach houdini-X.X.XXX-macosx_YYY_clangZZ_ZZ.dmg

# Generate the choices XML file.
installer -showChoiceChangesXML -pkg /Volumes/Houdini/Houdini.pkg  -target / &> /tmp/install_choices.xml

# Cleanup.  Unmount the Houdini installer image.
sudo hdiutil detach /Volumes/Houdini
```
- Open /tmp/install_choices.xml in a text editor and deselect all component choices in the file by replacing lines that read <integer>1</integer> with <integer>0</integer>. Save the changes to the file and close the text editor.
- Make two copies of /tmp/install_choices.xml and name the copies /tmp/hqueue_server_install_choices.xml and /tmp/hqueue_client_install_choices.xml for the server and client installations respectively.
- For the server installation, open /tmp/hqueue_server_install_choices.xml in a text editor. Find the group of lines that select the HQueue server component. The lines look like this:
```
<dict>
        <key>attributeSetting</key>
        <integer>0</integer>
        <key>choiceAttribute</key>
        <string>selected</string>
        <key>choiceIdentifier</key>
        <string>hqserver</string>
</dict>
```
  Replace <integer>0</integer> with <integer>1</integer> in the group of lines to select the HQueue server component in the installation. Save the changes to the file and close the text editor.
- Similarly for the client installation, open /tmp/hqueue_client_install_choices.xml in a text editor. Find the group of lines that select the HQueue client component. The lines look like this:
```
<dict>
        <key>attributeSetting</key>
        <integer>0</integer>
        <key>choiceAttribute</key>
        <string>selected</string>
        <key>choiceIdentifier</key>
        <string>hqclient</string>
</dict>
```
  Replace <integer>0</integer> with <integer>1</integer> in the group of lines to select the HQueue client component in the installation. Save the changes to the file and close the text editor.
- The server and client choices XML files are now ready for scripted installations.
Note

You don’t have to re-generate the server and client choices XML files every time you install or upgrade the HQueue server and clients. You can re-use the choices XML files for subsequent installations. You only need to re-generate the choices XML files when the component options in the Houdini installer change.

Install the HQueue server

Run the following commands to install the HQueue server:

# Ensure that no previous Houdini installer image is mounted.
sudo hdiutil detach /Volumes/Houdini

# Mount the Houdini installer image.
sudo hdiutil attach houdini-X.X.XXX-macosx_YYY_clangZZ_ZZ.dmg

# Install the HQueue server.
sudo installer -pkg /Volumes/Houdini/Houdini.pkg -target / -applyChoiceChangesXML /tmp/hqueue_server_install_choices.xml

# Cleanup.  Unmount the Houdini installer image.
sudo hdiutil detach /Volumes/Houdini

The HQueue server installs to /Library/HQueueServer and runs with the root account. A new launchctl service, /Library/LaunchDaemons/com.sidefx.hqserver.plist, is also created that runs the server.

Install the HQueue client

Run the following commands to install the HQueue client:

# Ensure that no previous Houdini installer image is mounted.
sudo hdiutil detach /Volumes/Houdini

# Mount the Houdini installer image.
sudo hdiutil attach houdini-X.X.XXX-macosx_YYY_clangZZ_ZZ.dmg

# Install the HQueue client files.
sudo installer -pkg /Volumes/Houdini/Houdini.pkg -target / -applyChoiceChangesXML /tmp/hqueue_client_install_choices.xml

# Run the HQueue client installer script.
cd /Library/HQueueClient
sudo ./install.sh --user myUser --hq-server myServer --hq-server-port 5000

# Cleanup.  Unmount the HOudini installer image.
sudo hdiutil detach /Volumes/Houdini

The HQueue client installs to /Library/HQueueClient and runs with the myUser account. The client connects to the HQueue server located at http://myServer:5000.

Scripted Installation on Windows ¶

Scripted installation on Windows begin with the Houdini installer executable, houdini-X.X.XXX-win64-vcYYY.exe. Download the executable from the SideFX website.

Install the HQueue server

Run the following command to install the HQueue server:

houdini-X.X.XXX-win64-vcYYY.exe /S /AcceptEULA=20XX-XX-XX /LicenseServer=No /DesktopIcon=No /FileAssociations=No /HoudiniServer=No /SideFXLabs=No /HQueueServer=Yes /Action=install /HQServerInstallDir=C:\HQueueServer /ServiceName=HQueueServer

The HQueue server installs to C:\HQueueServer and a new HQueueServer service is created that runs the server.

Install the HQueue client

Run the following commands to install the HQueue client:

# Install the HQueue client.
houdini-X.X.XXX-win64-vcYYY.exe /S /AcceptEULA=20XX-XX-XX /LicenseServer=No /DesktopIcon=No /FileAssociations=No /HoudiniServer=No /SideFXLabs=No /HQueueClient=Yes /HQClientInstallDir=C:\HQueueClient /HQClientServerName=myServer /HQClientServerPort=5000

# Reboot the machine to complete installation.
shutdown /r

The HQueue client installs to C:\HQueueClient and runs with the user account that ran the installer executable. The client connects to the HQueue server located at http://myServer:5000.

Upgrading HQueue ¶

To upgrade the HQueue server, go through the regular server installation process. Depending on the operating system, the installer may prompt you to install or upgrade the existing HQueue server installation. Choosing to upgrade overwrites the server files but preserves the existing server database and configuration files. Choosing to install overwrites the server files, database, and configuration files. If the installer does not prompt you to install or upgrade, the installer performs an upgrade.

Once the HQueue server upgrades to a newer version, the HQueue clients automatically upgrade by pulling the new client code from the HQueue server. You may see “Upgrade <machine>” jobs appear in the HQueue website that indicate the clients auto-upgrading themselves. If auto-upgrading fails or does not occur, then you can manually request to upgrade the clients from the clients web page on the HQueue website (see Managing clients) or install the new client version using the Houdini installer. Installing via the Houdini installer will upgrade and preserve the client’s existing configuration.

Upgrading an old HQueue Python 2 Farm ¶

Upgrading an old HQueue Python 2 farm is similar to a regular HQueue upgrade.

Use the Houdini installer to upgrade the HQueue Server. The installer detects the old HQueue Python 2 server and backups the old server before proceeding with a new installation/upgrade.

The old server is backed up to

Linux

/opt/hqueue.backup.YYYY-MM-DD_HHMMSS

Mac

/Library/HQueueServer.YYYY-MM-DD_HHMMSS.bak

Windows

C:/HQueueServer_bk.MM-DD-YYYY.HHMMSS

The installer also converts the old server database and configuration files to the new formats expected by the new HQueue server.

After the installation and the server has been upgraded, the old Python 2 HQueue clients can be upgraded. You can only upgrade old Python 2 clients using the Houdini installer. Upgrading clients from the HQueue website is not possible.

The installer detects the old HQueue Python 2 client and backups the old client before proceeding with a new installation/upgrade.

The old client backs up to

Linux

/home/myUser/hqclient.backup.YYYY-MM-DD_HHMMSS

Mac

/Library/HQueueClient.YYYY-MM-DD_HHMMSS.bak

Windows

C:/HQueueClient_bk.MM-DD-YYYY.HHMMSS

The installer also converts the old configuration files to the new formats expected by the new HQueue client.

After the client installation, your new Python 3 HQueue farm is ready for use. It is up to you to remove the backup server and client folders if desired.

Optional: `hquser` account ¶

On client machines you can optionally set up an hquser user and hqgroup user group to run HQueue as. By convention the user and group ID are both 601, and the user’s password is hqpass, but you can change these.

Linux

Your Linux distribution may come with a user interface for creating user accounts, otherwise use the adduser command.

Mac

You can create users and groups in the Users and Groups panel of System Preferences.

Having consistent user accounts across clients has benefits:

Don’t have to keep track of different user accounts on different machines when you need to log into a particular client.
A single consistent user owns all files created by the client software.

Once you have created the user and group, you should start the HQueue server/client as hquser.

Installation

Before you begin ¶

Installing HQueue ¶

Installing HQueue on Linux ¶

Installing HQueue on macOS ¶

Installing HQueue on Windows ¶

Testing the Farm ¶

Troubleshooting ¶

Scripted Installations ¶

Scripted Installation on Linux ¶

Scripted Installation on macOS ¶

Scripted Installation on Windows ¶

Upgrading HQueue ¶

Upgrading an old HQueue Python 2 Farm ¶

Optional: `hquser` account ¶

HQueue

Getting started ¶

Managing the farm ¶

Next steps ¶

Guru level ¶

Before you begin ¶

Installing HQueue ¶

Installing HQueue on Linux ¶

Installing HQueue on macOS ¶

Installing HQueue on Windows ¶

Testing the Farm ¶

Troubleshooting ¶

Scripted Installations ¶

Scripted Installation on Linux ¶

Scripted Installation on macOS ¶

Scripted Installation on Windows ¶

Upgrading HQueue ¶

Upgrading an old HQueue Python 2 Farm ¶

Optional: hquser account ¶

HQueue

Getting started ¶

Managing the farm ¶

Next steps ¶

Guru level ¶

Optional: `hquser` account ¶