Houdini 18.5 Executing tasks with PDG/TOPs

PDG node network interface

How to use the unique network editor features of TOP networks.

On this page

See also how to cook the network.

TOP nodes

Work item dots

The distinguishing feature of TOPs nodes is the grid of dots representing the work items in each node. If a node hasn’t generated work items yet, it will not show any dots. You can pre-generate the "static" work items (those items known ahead of time) using Tasks ▸ Generate Static Work Items. Dynamic work items will appear as the network runs (see cooking the network).

To...Do this

Show information about an individual work item

  • Press on a work item dot to show a pop-up info window containing information about the work item, including its attribute values.

  • You can also ⌃ Ctrl + click a dot to open a persistent info window, or right-click the dot and choose Task info.

Hide the dots on one node

  • Click the up-pointing triangle in the bottom-left corner of the node to "collapse" the node, hiding the dots.

  • Click the down-pointing triangle on a collapsed node to show the dots again.

Hide the dots on all nodes

In the menus at the top of the Network Editor, turn off Tasks ▸ Show Task Graph.

This may be necessary if networks have huge numbers of visible work items. See also the controls in the display options for limiting the number of dots shown on each node.

Limit the number of dots shown on nodes

See the display options.

Show a table of all work item data

Click the table icon in the bottom-right corner of a TOP node to open the task graph table.

The color of the dots normally indicates the individual work item’s status (you can switch to show relative speed as color instead, see visualizing performance).

Dot

Color

Status

Description

Bright green

In progress (cooking)

The work item is currently running.

Medium green

Done (cooked)

The work item finished without any warning or error.

Dark blue-grey

Waiting

The work item has started but is waiting (to be scheduled or for upstream work to finish).

Grey

Uncooked/Not cooking

The work item has been generated but hasn’t run yet.

Orange

Warning

The work item generated a warning.

Red

Failed

The work item encountered an error (or the executable returned a non-zero status code).

Brown

Canceled

The user canceled the cook while the task item was running.

Work item pages and batches

When a TOP node’s work item count exceeds 200, work item pages will appear on the node. Each work item page contains 200 or less work items and each node can contain a maximum of 50 work item pages.

Work item pages provide a quick and easy way to access your numerous work items directly on their nodes. However, when a node has work items numbering in the thousands, it may be easier for you to access and manage your large number of work items through the Task Graph Table.

Work item pages

Tip

When navigating work items on a node with the [ and ] keys, you will automatically switch between work item pages.

When batch processing is enabled for a TOP node, you can display its work item dots as batch rings by setting Task Collapse Mode to By Batch in the Task Graph Display Options.

Work item batch rings

Status icon and counts

To the left of each TOP node, Houdini shows an icon indicating the overall status of the work items in the node, and a table of the number of work items with each status. When the node is cooking, a ring around the icon shows the progress of the work items. (If you hide work items with certain statuses using the display options, those items are not included in the progress.)

Icon

Status

Description

Not started

This node is not running.

Waiting

This node has started, but is waiting for its items to be scheduled or for upstream dynamic items to finish.

Cooking

This node is running, the work items inside are being scheduled on the machine/render farm.

Done

All work items in this node have finished running with no warnings/errors.

Warning

At least one work item in this node raised a warning as it ran.

Error

At least one work item in this node raised an error as it ran.

Tasks menu

This menu appears at the top of the Network Editor when viewing a TOP network.

Generate Static Work Items

Makes every node in the network generate any "static" work items it knows will be needed ahead of time, based on parameter values and/or incoming static items.

Cook Output Node

Starts cooking the network to produce the final output.

Cook Selected Node

Cooks the network down to the selected node, instead of down to the node with the output flag. This can be useful to test different parts of the network.

Dirty All

Marks all nodes in the network as needing to recook. This means the next time the network cooks, all work items will run.

Delete All Results From Disk

Deletes all result files tracked by work items in the graph.

Dirty Selected Node

Marks the selected node as needing to recook. The next time the network cooks, all work items in the node will run.

Dirty and Cook Selected Node

Forces all work items in the network to recook down to the selected node. This can be useful to test different parts of the network.

Dirty and Cook Output Node

Forces all work items in the network to recook down to the final output.

Delete Selected Node Results From Disk

Deletes all result files tracked by work items in the selected node.

Cancel Current Cook

Stops cooking the network and may cancel in-flight processes.

Pause Current Cook

Pauses cooking the network. Any in-flight processes will continue to run.

Show Task Graph

Whether to show dots representing work items inside the nodes.

Task Graph Table

Opens the Task Graph Table window. See the Task Graph Table section below.

Task Graph Display Options

Opens a window with TOP-specific display options

Show Task Graph Performance

When on, the Network Editor colors dots according to relative speed or output size, rather than by status. See visualizing performance for more information.

Performance Stat

When Show task graph performance is on, this controls whether to color work items by relative cook time or output size.

PDG Service Menu

Opens the PDG Service Menu.

PDG Path Map

Opens the PDG Path Map.

Task Graph Table

  • To show the Task Graph Table, click the table icon in the bottom-right corner of a TOP node or select Task Graph Table from the Tasks menu in the Network Editor.

    The Task Graph Table shows the metadata of all work items in the graph, or all work items in a particular node.

  • Use the Task Graph Table - Nodes options window and pin toggle to choose which work items appear in the Task Graph Table.

  • Use the Task Graph Table - Columns options window to choose which attribute or property columns appear in the Task Graph Table.

Task Graph Table - Nodes

Opens the Task Graph Table - Nodes options window.

Task Graph Table - Nodes options window

The Task Graph Table - Nodes options window lets you specify which nodes' work items currently appears in the Task Graph Table.

Node display mode

Show selected nodes

When on, the Task Graph Table is context-aware and only displays the work items of the nodes currently selected in the Network Editor.

Show custom list of nodes

When on, the Task Graph Table only displays the work items of the nodes currently selected in the Custom node list. The nodes that appears in the Custom node list change from topnet to topnet.

Custom node list

List nodes containing

Filters the list of TOP nodes that appears in the Task Graph Table - Nodes options window by path. This is useful when you only want to list nodes from a certain path, an individual subnet, or nodes with specific words in their names.

This filter field only accepts text string inputs. It does not accept operators or wildcards as inputs.

Filtering the node list

Select All

Selects all the nodes listed in the Custom node list.

Select None

Deselects all the nodes listed in the Custom node list.

Task Graph Table - Columns

Opens the Task Graph Table - Columns options window.

Task Graph Table - Columns options window

The Task Graph Table - Columns options window lets you specify what work item information currently appears as columns in the Task Graph Table.

Task Graph Table columns
Filter Attribute List

Show attributes containing

Filters the list of column options by attribute or property name. This is useful when you only want to list column options that contain specific text.

This filter field only accepts text string inputs. It does not accept operators or wildcards as inputs.

Filtering the list of column options

List all attributes

When on, all work item attributes and properties are listed as column options in the Filter Attribute List.

Show only work item properties

When on, only work item properties are listed as column options in the Filter Attribute List.

Show only work item attributes

When on, only work item attributes are listed as column options in the Filter Attribute List.

Select All

Selects all the attributes and/or properties listed in the Filter Attribute List.

Select None

Deselects all the attributes and/or properties listed in the Filter Attribute List.

Remote

Opens the Remote PDG Instance Options window.

The Remote PDG Instance Options window lets you connect the Task Graph Table to a remote instance of PDG.

Pin

You can pin and unpin the current contents of the Task Graph Table by clicking the pin icon at the top of the Task Graph Table window.

When the Task Graph Table pin is active, only the work items for the TOP nodes that were selected when you activated the pin are displayed in the Task Graph Table. If you navigate away from those nodes, like by selecting other nodes in the Network Editor or by switching contexts (for example, switching from task-level to object-level), the Task Graph Table will continue to display only the work items that belong to the pinned TOP nodes.

Also, if the pinned TOP nodes change in any way, then all the information for their work items is automatically updated in the Task Graph Table.

Task Graph Display Options

You can open the Task Graph Display Options window through the Tasks menu in the Network Editor or the Network View Display Options window (hotkey: D key).

Task Graph Display Options window

Show Tasks

Turns off the display of dynamic work items with certain states.

Note

These controls do not affect static work items.

Cooked

Show cooked work items in the nodes.

Cooking

Show work items that are cooking.

Dirty

Show work items that have changed since their last cook.

Canceled

Show work items that have been canceled.

Failed

Show work items that have failed to cook.

Sort By

Automatic

If the network is visualizing performance, sort by performance, otherwise sort by index. This is the default.

State

Sort by the work item state, for example "cooking", "cooked", "failed", and so on.

ID

Sort by each work item’s unique ID.

Index

Sort by the index attribute. This tends to keep child items in line with their parents.

None

Work items appear in no particular order.

Task Collapse Mode

Off

Do not auto-collapse nodes. This is the default and the recommended setting.

By Batch

If batching is enabled on a node, then all the work items on that node appear as batch rings instead of work item dots.

Isolate Highlight Tasks

When you select a work item, this hides all work items that aren’t ancestors or descendants of the selected item.

Max Number of Rows

The maximum number of rows of dots to show inside nodes. When extra rows are hidden, the visible rows will end with a "fade out" effect to show that more work items exist but are not shown.

Tasks Per Row

How many dots to show on each row inside nodes. The default is 10. Decreasing this number makes the dots bigger, increasing it makes the dots smaller.

Use Work Item Color Attribute

When on, you can use an attribute as your work item dot color instead of the work items' current states. This allows you customize the color of the work item dots on your nodes.

For example, if you have custom states for your work items, then you could write some custom python code to map the color attribute you specify with those states. You could also use an Attribute Create TOP node to create a custom color attribute and then map the color attribute to certain work items by placing the node in a specific location in your TOP network.

Example: A custom color attributecreate node placed between a wedge node and a ropgeometry node will assign each wedge’s downstream work item dots a different color.

To specify which attribute to use, type the name of the attribute in the Work Item Color Attribute field.

Work Item Color Attribute

Specifies which attribute to use for your custom work item dot color.

This field only accepts a single attribute name as input.

Executing tasks with PDG/TOPs

Basics

Beginner Tutorials

Next steps

  • Running external programs

    How to wrap external functionality in a TOP node.

  • File tags

    Work items track the "results" created by their work. Each result is tagged with a type.

  • PDG Path Map

    The PDG Path Map manages the mapping of paths between file systems.

  • Feedback loops

    You can use for-each blocks to process looping, sequential chains of operations on work items.

  • Command servers

    Command blocks let you start up remote processes (such as Houdini or Maya instances), send the server commands, and shut down the server.

  • PDG Service Manager

    The PDG Service Manager manages pools of persistent Houdini sessions that can be used to reduce work item cooking time

  • Integrating PDG with render farm schedulers

    How to use different schedulers to schedule and execute work.

  • Visualizing work item performance

    How to visualize the relative cook times (or file output sizes) of work items in the network.

  • Event handling

    You can register a Python function to handle events from a PDG node or graph

  • Tips and tricks

    Useful general information and best practices for working with TOPs.

  • Troubleshooting PDG scheduler issues on the farm

    Useful information to help you troubleshoot scheduling PDG work items on the farm.

  • PilotPDG

    Standalone application or limited license for working with PDG-specific workflows.

Reference

  • All TOPs nodes

    TOP nodes define a workflow where data is fed into the network, turned into "work items" and manipulated by different nodes. Many nodes represent external processes that can be run on the local machine or a server farm.

  • Processor Node Callbacks

    Processor nodes generate work items that can be executed by a scheduler

  • Partitioner Node Callbacks

    Partitioner nodes group multiple upstream work items into single partitions.

  • Scheduler Node Callbacks

    Scheduler nodes execute work items

  • Custom File Tags and Handlers

    PDG uses file tags to determine the type of an output file.

  • Python API

    The classes and functions in the Python pdg package for working with dependency graphs.

  • Job API

    Python API used by job scripts.