Houdini 18.0 Executing Tasks

Using attributes

TOP attributes are like point attributes, You can use them to set parameters in Houdini nodes called by the work items.

On this page

Overview

A work item contains attributes, named bits of information similar to attributes on a point in Houdini geometry. The script or executable associated with the work item can read the attributes to control the work they do. Attributes are passed down to work items created from "parent" work items, so you can use them to have the result of a work item affect the processing of its child items, group items together, and pass information down through the network.

Custom code can read attributes to control generation of work items, however the most common use of attributes is to reference them in parameters of TOP nodes and/or Houdini networks and nodes that are referenced by the TOP network.

  • For example, let’s say you want to wedge different render qualities. You can use the Wedge TOP to create work items that have a pixelsamples attribute set to different values. Then, in the ROP Mantra Render TOP, you can set Pixel samples using @pixelsamples to reference the attribute value.

  • You can also reference work item attributes in external assets/networks called by the TOP network. For example, the HDA Processor cooks a Houdini asset for each work item. You can use @attribute references in that asset’s parameters to pull values from the work item.

You can reference a component of a vector using @attribute.component, where component is a zero-based number, or x, y, or z (equivalent of 0, 1, 2). For example @pos.x or @pdg_output.0.

Attribute types

Work items support basic string, float and integer array attributes much like Houdini geometry. All primitive attribute types are arrays, and the array length can be arbitrarily long. Work items also have a File attribute type, which is similar to a string but stores additional metadata such as the file tag and the modification time of the file. File attributes are displayed as clickable links in the work item info panel, and can be used with the path handling mechanism in TOPs to make them usable on both a local machine and farm machine.

There are two additional types of attributes that can be created on work items for use cases that require complex structured data. The first of these is the Python Object attribute type, which can be assigned a single PyObject value. TOPs is only able to display basic information about the contents of the attribute using the Python Object’s string representation, and many nodes cannot operate directly on data stored in that type of attribute. Full access to the object is possible through the PDG Python API or in TOP nodes that explicitly make use of a Python Object attribute such as the Shotgun nodes. If the Python Object consists of nested array and dictionary objects, values in the object can be accessed using the @ operator in HScript. For example @obj.key.subkey.2.value, which would be equivalent to the Python code work_item['obj'].object['key']['subkey][2]['value]. @obj will always evaluate to the standard string representation of the Python Object. This is the same as doing repr(work_item['obj'].object) in Python.

The second complex attribute type is the Geometry attribute, which can store native Houdini geometry data. The geometry attribute type primarily used by the Geometry Import and Invoke TOP nodes, which are able to operate on geometry directly in the current Houdini process. Like the Python Object type, most nodes aren’t able to directly access or make use of the attribute, however the attribute contents can be inherited between work items even if the node creating the work items cannot use geometry. A TOP Geometry node in a SOP network is able to pass geometry attributes on output work items back to SOPs.

Attribute flags

Work item attributes can have various flags set that determine the way they're accessed and presented to other parts of the TOP network. Common flags include No Copy which indicates that an attribute should not be copied onto downstream work items, and Env Export which causes the attribute value to be exported as an environment variable when cooking out of process. A full list of supported flags is listed in the pdg.attribFlag documentation.

Using attributes in expressions (pull)

When nodes are "called" as part of a TOP network cooking (for example, when TOPs cooks an HDA Processor, or renders frames using a ROP Mantra Render), it fills in any @attribute references using the cooking work item’s attributes.

Tip

When nodes cook for display in the viewer, they fill in @attribute references using the attributes of the currently selected work item. This means you can click around work items and see the visible output of a network change to reflect the output of the selected item.

These are called "pull" references.

If you don’t want to or can’t put TOPs-specific expressions in certain parameters, you can instead use an alternate "push" mechanism where you specify in the TOP network parameters to overwrite at run time.

String parameters

An @attribute reference by itself is an Hscript expression, just like ch('../geo1/tx') * 2 is an expression. You can use @attribute references as part of HScript expressions to compute values in node parameters.

Remember that if you want to use an expression in a string parameter, you must escape it by enclosing it in backticks (`). For example, to reference the work item’s built-in output attribute in a file path parameter, you must enclose the reference in backticks:

`@pdg_output`

You can incorporate an expression within "normal" text in the parameter. For example, to use the work item’s frame number in a file path:

$HIP/output/$OS_`@pdg_frame`.exr

Note

If it’s ambiguous what an @attribute expression refers to (for example, a geometry point attribute and a TOP work item attribute have the same name), SOP attributes take precedence over TOP work item attributes.

Replacing parameter contents at run time (push)

When using the Wedge TOP to vary parameter values, if you don’t want to put TOP-specific expressions on other nodes (for example, if the ROP you want to render is shared with other pipelines or used manually), you can have the Wedge node replace parameter values with wedge attributes at run time.

See the help for the Wedge TOP for how to override parameters at render time.

Built-in attributes

Note

When referencing built-in attributes, use @pdg_name, for example, @pdg_index.

index

Used as a key for sorting work items within a node.

name

Makes it easier to distinguish individual work items in the interface.

frame

The frame number this item is working on. Of course, not all work items are doing rendering, however needing to store a frame number is common enough that we included it in the built-in attributes.

input

A list of filenames used as the input for this bit of work. For example, if the work item represents "draw text over a rendered image", this would be the path to the image to manipulate.

output

A list of filenames produced as the output of this bit of work. For example, if the work item represents "draw text over a rendered image", this would be the path to the output image with the text drawn into it.

Tip

As processor nodes generate new work items from incoming work items, the output attribute of the "parent" work item is automatic assigned as the input attribute of the "child" work item.

Attribute Pattern Syntax

When using nodes like the Attribute Delete or Attribute Copy it is often useful to specify a list of attributes using a pattern. PDG has a special attribute pattern syntax that can you can use to describe which attributes should be matched in nodes that can operate on multiple attributes. The pattern is applied from left to right and consists of a space-separated sequence of match conditions using the symbols described below.

It is also possible to use the pattern matching functionality in a Python Script or in your own custom nodes. You can construct an pdg.AttributePattern object from a string and use it with the pdg.WorkItem.attribMatch function to apply the pattern to attributes on a work item. Alternatively, you can call the pdg.AttributePattern.contains function directly to match against a specific attribute name and type.

Pattern

Meaning

*

Match all attribute names, of any type

name

Match a specific attribute name

prefix*

Match any attribute name with the prefix. for example file*

*suffix

Match any attribute name with the suffix, for example *size

^pattern

Exclude attributes matching the pattern from the results of the preceding patterns. For example, the pattern * ^size* will match all attributes except for ones that begin with the prefix "size".

:type

Suffix that indicates the pattern should only match attributes of the specified pdg.attribType. The type can be float, integer, string, file, geometry or pyobject. For example, size*:float *:integer ^sizesmall will match all float attributes with the prefix size, except for the one named sizesmall, and all integer attributes.

Executing Tasks

Basics

Next steps

  • Running external programs

    How to wrap external functionality in a TOP node.

  • File tags

    Work items track the "results" created by their work. Each result is tagged with a type.

  • Feedback loops

    You can use for-each blocks to process looping, sequential chains of operations on work items.

  • Command servers

    Command blocks let you start up remote processes (such as Houdini or Maya instances), send the server commands, and shut down the server.

  • TOP Service Manager

    The TOP Service Manager manages pools of persistent Houdini sessions that can be used to reduce work item cooking time

  • Integrating PDG and render farm software

    How to use different schedulers to schedule and execute work.

  • Visualizing work item performance

    How to visualize the relative cook times (or file output sizes) of work items in the network.

  • Event Handling

    You can register a Python function to handle events from a PDG node or graph

  • Tips and tricks

    Useful general information and best practices for working with TOPs.

Reference

  • All TOPs nodes

    TOP nodes define a workflow where data is fed into the network, turned into "work items" and manipulated by different nodes. Many nodes represent external processes that can be run on the local machine or a server farm.

  • Processor Node Callbacks

    Processor nodes generate work items that can be executed by a scheduler

  • Partitioner Node Callbacks

    Partitioner nodes group multiple upstream work items into single partition

  • Scheduler Node Callbacks

    Scheduler nodes execute work items

  • Python API

    The classes and functions in the Python pdg package for working with dependency graphs.