Houdini Engine 3.3
 All Classes Files Functions Variables Typedefs Enumerations Enumerator Macros Pages
PDG

Introduction to PDG

PDG is a procedural architecture designed to distribute and manage tasks. For more information about PDG, visit the PDG product page.

With Houdini Engine, the features of PDG can be made available easily in other applications. Most notably, PDG can be used to build complex asset generation tools comprised of networks of HDAs, to manage the dependencies and flow of data.

PDG networks can be embedded in a HDA using TOP nodes, and utilized in a Houdini Engine session. Houdini Engine provides PDG API to manage the PDG cooking process and acquire the output results of the TOP nodes.


Utilizing PDG via TOPs in HDAs

There are 2 ways to embed TOP nodes in HDAs:

TOP SOP

This is the simplest method to have PDG working out of the box in Houdini Engine. A TOP SOP contains TOP nodes, forming the PDG graph, and passes any output geometry to the next SOP node. Multiple outputs are automatically merged into a single detail with groups. The contained TOP node network is automatically cooked when the TOP SOP is cooked, and its output is available just as any other SOP node. Therefore this does not require the explicit use of PDG APIs to cook and dirty the graph.

HAPI_PDG_Topsop.png

TOP Network

A TOP Network is an Object node containing TOP nodes. The PDG graph must be explicitly cooked using provided HAPI PDG API. This allows the host application to provide the user with PDG cook controls such as dirtying and cooking. Results of each cooked work item can be queried, and file paths to generated outputs can be used to load the data into the host application.

HAPI_PDG_Topnetwork.png

Environment and Paths

It is important to make sure that the environment variables and non-literal paths used in the TOP nodes are valid in the Houdini Engine session.

A very important setting is the scheduler's working directory, which is the root directory for generating intermediate result files, and potentially for final result files as well. By default, all the TOP schedulers use the $HIP environment variable which is not valid in a Houdini Engine session, since there is no hip file. One of the following solutions should be used instead:

  • Specify a global environment value in the houdini.env file for the working directory path (e.g. MY_HAPI_PDG_DIR=C:/projects/hapipdg), then set it as the value for the scheduler's Working Directory parm.
  • Expose the scheduler's Working Directory parm on the HDA, and specify it in the host application. This allows to set the working directory for each HDA.

It is recommended to base all other paths, specially for result files, on a common root path, such as the working directory or another global environment path (e.g. $MY_ROOT_PATH/geometry/test1.bgeo). This allows to use a single variable to manage all paths used in PDG work items.


Schedulers

By default, the TOP SOP and TOP Networks use the Local Scheduler, which schedules PDG work items to be executed on the local machine. For distributing the work on other machines, farm schedulers can also be used by creating their respective TOP scheduler node and specifying them to be used in the TOP network. There is no special consideration or extra work required for using other schedulers in an HDA, but do make sure that any environment settings required by the schedulers are set in the Houdini Engine session. Usually this can be done via the houdini.env file when creating the Houdini Engine session.


Dirtying, Cooking, and Events

PDG allows to dirty and cook TOP networks, which means the display node, and implicitly its dependency graph, is dirtied and cooked. Similarly, individual TOP nodes, and their dependency graph, can be specifically dirtied and cooked. HAPI_DirtyPDGNode allows to supply a flag to also delete the previous results of the work items.

HAPI_CookPDG supports both blocking and non-blocking cooking. While cooking, PDG events (HAPI_PDG_EventType) are emitted for the generated TOP nodes and their work items in the dependency graph. These events provide real-time status of the cook, similar to the TOP graph node and work item status shown in Houdini. To query the events for a specific PDG graph cook, its PDG graph context must be specified. Since each TOP node is part of a PDG graph context, its id can be retrieved by HAPI_GetPDGGraphContextId.

The following HAPI_PDG_EventType events might be useful:

  • HAPI_PDG_EVENT_WORKITEM_ADD : When a work item is added to the graph.
  • HAPI_PDG_EVENT_WORKITEM_REMOVE : When a work item is removed from the graph.
  • HAPI_PDG_EVENT_WORKITEM_STATE_CHANGE : When a work item has changed state. Look at work item's current HAPI_PDG_WorkitemState.
  • HAPI_PDG_EVENT_COOK_WARNING : Warning message for a node.
  • HAPI_PDG_EVENT_COOK_ERROR : PDG cook has completed with an error.
  • HAPI_PDG_EVENT_COOK_COMPLETE: PDG cook has completed successfully.

Work items change state during a PDG cook. The following HAPI_PDG_WorkitemState might be useful:

  • HAPI_PDG_WORKITEM_DIRTY : Work item has been dirtied.
  • HAPI_PDG_WORKITEM_COOKING : Work item is cooking.
  • HAPI_PDG_WORKITEM_COOKED_SUCCESS : Work item has completed cooking successfully.
  • HAPI_PDG_WORKITEM_COOKED_CACHE : Work item has completed cooking, but using previously cached result.
  • HAPI_PDG_WORKITEM_COOKED_FAIL : Work item has failed to cook.
  • HAPI_PDG_WORKITEM_COOKED_CANCEL : Work item has been cancelled.

Generated results can be queried when work item states HAPI_PDG_WORKITEM_COOKED_SUCCESS and HAPI_PDG_WORKITEM_COOKED_CACHE are emitted. See PDG Cooking With Events for a code sample of cooking and listening for events.


PDG Code Samples

For code samples showing how to utilize PDG in HAPI, see PDG Cooking Samples.


PDG API

The following functions allow to manage and query the PDG graph within a HDA. See HAPI.h for more information.

Cooking

  • HAPI_CookPDG
  • HAPI_DirtyPDGNode
  • HAPI_PausePDGCook
  • HAPI_CancelPDGCook

Context, Cook Events, and State

  • HAPI_GetPDGGraphContexts
  • HAPI_GetPDGGraphContextId
  • HAPI_GetPDGEvents
  • HAPI_GetPDGState

Work Items

  • HAPI_CreateWorkitem
  • HAPI_GetWorkitemInfo
  • HAPI_SetWorkitemIntData
  • HAPI_SetWorkitemFloatData
  • HAPI_SetWorkitemStringData
  • HAPI_CommitWorkitems
  • HAPI_GetNumWorkitems
  • HAPI_GetWorkitems
  • HAPI_GetWorkitemDataLength
  • HAPI_GetWorkitemIntData
  • HAPI_GetWorkitemFloatData
  • HAPI_GetWorkitemStringData
  • HAPI_GetWorkitemResultInfo