Houdini 18.0 Executing Tasks

Tips and tricks

Useful general information and best practices for working with TOPs.

On this page

Cooking

  • Test your network on small data sets, and make sure everything works, before running on big data sets.

  • Use Tasks ▸ Generate Static Work Items (See static vs. dynamic) to get a sense of the work to be done before cooking the network.

Attributes

  • Use "pull" attribute references. Use @attribute syntax in parameters to control what gets done (see pull and push references). Not only is it much easier than pushing attributes or writing custom code, PDG has an important feature that lets you resume a cook after being interrupted – but only if you used @attribute syntax so PDG can predict what output files would have been named.

  • In interactive Houdini, attribute references come from the currently selected item. When other networks (such as SOP networks for generating geometry and COP2 networks for compositing renders) in the HIP file have @attribute values, the "current" values of those references are driven by which work item is selected in the network interface.

    You can, for example, set a viewer pane to show the SOP network output, and then click around the work items in the TOP network, showing the different geometry output by the different items.

    You can also take advantage of this to quickly preview output geometry. The Show Output shelf tool creates a Geometry object, and inside create a File SOP, then sets the File node’s path to `@pdg_output` (the backticks are necessary to escape the expression since the path is a string parameter). When you view the network in a viewer, as you click work items around the TOP network, you can see their geometry output. You can then delete the object/network when you're done inspecting geometry output.

    See attribute references for more information.

Simulations

  • Do not generate simulation frames in parallel. When generating simulation output using the ROP Geometry Output node, go to the ROP Fetch tab and turn on All Frames in One Batch.

    If you leave this option off, the separate frames will be generated in parallel. However, since simulations must cook everything up to the frame to render, this leads to a situation where to generate frame 2, Houdini must cook frames 1 and 2, to generate frame 3 it must cook frames 1, 2, and 3, and so on. This is very inefficient.

Output files

See file paths for more information.

  • Try to separate temporary "work in progress" files from "final output" files in separate directories. For example, generate intermediate geometry files in __PDG_TEMP__/geo, and final geometry files in __PDG_DIR__/geo. This makes it easier to find the important output, and lets you clear out the temp files wholesale to force a rebuild.

  • You may want to avoid using destructive file operations (move/rename, remove). They can make files appear to be missing, triggering the dependency system to do extra work.

Executing Tasks

Basics

Next steps

  • Running external programs

    How to wrap external functionality in a TOP node.

  • File tags

    Work items track the "results" created by their work. Each result is tagged with a type.

  • Feedback loops

    You can use for-each blocks to process looping, sequential chains of operations on work items.

  • Command servers

    Command blocks let you start up remote processes (such as Houdini or Maya instances), send the server commands, and shut down the server.

  • TOP Service Manager

    The TOP Service Manager manages pools of persistent Houdini sessions that can be used to reduce work item cooking time

  • Integrating PDG and render farm software

    How to use different schedulers to schedule and execute work.

  • Visualizing work item performance

    How to visualize the relative cook times (or file output sizes) of work items in the network.

  • Event Handling

    You can register a Python function to handle events from a PDG node or graph

  • Tips and tricks

    Useful general information and best practices for working with TOPs.

Reference

  • All TOPs nodes

    TOP nodes define a workflow where data is fed into the network, turned into "work items" and manipulated by different nodes. Many nodes represent external processes that can be run on the local machine or a server farm.

  • Processor Node Callbacks

    Processor nodes generate work items that can be executed by a scheduler

  • Partitioner Node Callbacks

    Partitioner nodes group multiple upstream work items into single partition

  • Scheduler Node Callbacks

    Scheduler nodes execute work items

  • Python API

    The classes and functions in the Python pdg package for working with dependency graphs.