How to distribute particle fluid simulation using HQueue (advanced)

Use the Houdini installer on a Linux machine to install the HQueue server. The installer will direct you to the HQueue server’s online help, which will explain how to set up client machines.

The file hqserver.ini in the HQueue installation directory contains several useful configuration options, such as the port the HQueue HTTP server runs on.

The client machines on the HQueue network need to be able to load the Houdini file containing your simulation. Therefore, you need to save the .hip file and any external files it depends on to a network-accessible location, such as an NFS or Samba share.

Because different platforms (Windows, Linux, Mac OS X) handle paths and network shares differently, HQueue provides automatic path translation to allow clients on heterogeneous networks to all load the .hip file. This requires that you save the file to a specifically named network share, and have the same network share available at the required locations on all clients, so HQueue can substitute different path prefixes.

• The default network share names are H: on Windows, /mnt/hq on Linux, /Volumes/hq on Mac OS X.

• You can configure different prefixes for different platforms in the HQueue settings file (for example, if you are already using H: on your Windows clients). Go to the HQueue server installation directory and edit the settings in hqserver.ini.

See the help on the dynamics tools and the Particle Fluids shelf tab for how to set up a particle fluid network.

There are two tools on the _Particle fluids tab for slicing a particle fluid simulation into separate regions for distribution:

• The Slice along line tool. This is the simplest way to slice up space, by creating a specified number of equally spaces slices along a certain direction.

  

• The Slice tool. The tool lets you slice the space using a hierarchy of intersecting planes, allowing you to divide the space in arbitrarily complex ways.

  

Each region requires an individual machine on the HQueue client network, so only set up as many regions as you have machines. For example, using the Slice along line tool with Number of regions set to 8 would require 8 machines on the HQueue network.

1. Click Distribute on the Particle Fluids tab.

2. Select the particle fluid and press Enter .

   This creates a few nodes in various networks for distributing the up-res:

   • A distributedsim HQueue Simulation render node in /out to render the simulation out to HQueue.

     This render node sends the distribution job to the HQueue server.

   • A distribute_fluidname object in /obj to load the simulation files created by the HQueue clients back into Houdini.

   • A DISTRIBUTE_fluidname_CONTROLS Null node in the DOP network.

     This node is mostly a placeholder for values. The render node explicitly sets values on the “control” Null DOP such as the tracker address/port and slice divisions, and then multiple distribution-aware DOPs in the network reference the value on the control Null.

3. If you want to change the default path pattern for saved up-res geometry files ($HIP/geo/dist_${SLICE}_$F.bgeo.gz), go to the Scene level (/obj) and double-click the distribute_fluidname object to go inside. Select the loadslices File node and change the File parameter. See using expressions in filenames for more information.

   The path should be relative to $HIP.

4. In the main menu, choose Render > Edit render node > distributedsim. In the parameter editor:

   • Set the frame range you want to simulate.

   • Check the Slice divisions parameter. The product of the three components should equal the number of regions in your particle fluid simulation (for example, 3, 1, 1 for a simulation with 3 regions).

     This number is set automatically when the Distribute tool creates the render node. However, if you then go back and change the number of slices (for example, by changing the Number of regions parameter on a Slice Along Line node), you need to update this parameter on the render node.

   • Set the HQ Server to the hostname and port of the HQueue server, for example render.example.com:5000.

5. In the parameter editor, click Render to send the job to HQueue.

   The render node sends its information to the HQueue server. The HQueue server sends commands to its client machines to load the .hip file and simulate and save a slice of the particle fluid simulation per-frame. The individual client machines save their generated geometry cache files to the shared network directory.

Once the HQueue network has generated the up-res geometry, you should stop the original simulation network from cooking, since you now have the end-result cached.

Do one of the following:

• Turn off simulation in Houdini: right click (or pressing and holding on) the simulation menu in the bottom right corner of the main window and turn off Enable simulation.

• Start a fresh new .hip file that simply loads the generated particle fluid geometry files using a File surface node.

• Turn off the visibility of any scene-level objects that require the simulation to cook.

The Distribute tool on the on the Particle Fluids tab modifies the network to ready it for distribution. If you try to play back the simulation after these modifications, the network will only calculate a slice of the whole simulation, and several nodes will generate network traffic trying to talk to HQueue peers.

If you want to check the simulation on a single machine, you can use parameters on the “control” Null DOP created by the Distribute tool to temporarily switch the network back to non-distributed operation.