Houdini 19.5 Executing tasks with PDG/TOPs

PDG Services

PDG services manages pools of persistent Houdini sessions that can be used to reduce work item cooking time.

On this page

Overview

A PDG service is a long-running process that acts as a server for cooking work items. When a node is configured to use a PDG service, a cooking work item will submit the job to a pool of services. The first service that is available will retrieve the work item and perform its work.

PDG services avoids the overhead associated with starting a new process. Services can also perform optimizations specific to their jobs (for example, the HDA Processor service can create the necessary nodes exactly once, reusing the same nodes for each job and avoiding the time associated with creating the network). As a result, PDG services can vastly increase the speed of your jobs.

Supported nodes

PDG Services panel

To open the PDG Services panel:

  • In the TOPs Tasks menu, select PDG Services.

  • Create a new Pane Tab and then from the panes menu select New Pane Tab Type ▸ TOPs ▸ PDG Services.

PDG Services list

Lists all available, registered services and whether or not they are currently running. You can select, add, or remove registered PDG services from this list.

Add Service

Opens the Add PDG Service window.

Delete Service

This RMB menu removes a registered service. See Delete a registered PDG service.

Start

Starts the selected PDG service.

Stop

Stops the selected PDG service.

Automatic

When on, the selected service is automatically started by the first PDG graph cook where the service is used (if the service is not already running).

Status

Shows whether the selected service is running or stopped.

Label

Specifies the service node type for the selected PDG service.

Name

Specifies the name of the selected PDG service.

Pool Size

Sets the number of instances that will be created for the service pool. More instances will result in faster cook times because more items will be able to be cooked in parallel.

Owner

Shows the owner of the service. Currently, a service can only be owned by the Houdini session.

Scheduler Type

Shows the name of the scheduler type that the service is associated with.

Scheduler

For some scheduler types, a service must be run with respect to a specific scheduler node. You can specify the scheduler node with this field.

MQ Server Log Level

Shows the logging level of the MQ server associated with the service.

MQ Server Log Directory

Shows the directory where the logging file for the MQ server will be stored.

Client Log Directory

Specifies the folder where service pool clients will write their log files.

Add PDG Service window

Type

The service type associated with the registered service.

Owner

Controls the owner of the service. Currently, the service can only be owned by the Houdini session. This means that the lifetime of the service is associated with the current Houdini session. If the service is still running when Houdini is exited, the service will be shut down automatically.

Automatically Start Service

When on, the service is automatically started (if it’s not already running) during the first PDG cook where the service is used.

Pool Size

The number of clients that are created by the service.

Port

The defined port configuration.

Automatic

The port that the service will use is determined automatically.

Custom

The port that the service will use is a custom port you need to specify.

MQ Server Log Level

Controls what type of information is written to the MQ server’s log file.

None

No information will be logged.

Errors

Only errors will be logged.

Warnings

Warnings and errors will be logged.

All

General status messages, warnings, and errors will be logged.

MQ Server Log Directory

Specifies the logging directory in which the MQ server will create its log file.

Client Logging Directory

Specifies the logging directory in which Clients within the service pool will create their log files.

Add

Registers the service.

Cancel

Cancels the service registration operation.

PDG Services nodes

In addition to creating and managing services with the PDG Services panel, you can also work with PDG services directly in your graphs with the following TOP nodes:

How-tos

These are the steps for using a PDG service with one of the supported node types.

Add a PDG service

  1. In the PDG Services window, click Add Service located in the PDG Services list.

    The Add PDG Service window appears.

  2. In the Add PDG Service window, specify all the service details you want for the new service including configuring any nodes that you want to use the service.

    For this step, please consult the node’s documentation for information on configuring the node to use the service.

  3. Click Add.

    The new registered service appears in the PDG Services list.

    Also now when any associated configured nodes cook, they will use the new PDG services pool to cook their work items.

Start a PDG service

  1. In the PDG Services window, select the service you want to start from the PDG Services list list.

  2. Click Start.

Stop a PDG service

  1. In the PDG Services window, select the service you want to stop from the PDG Services list list.

  2. Click Stop.

Remove a PDG service

  1. In the PDG Services window, RMB-click+hold on the name of the service you want to remove.

  2. Click the Remove Service RMB option that appears.

    The selected service is removed from the PDG Services list.

Tip

You can re-register a PDG service at any time.

Executing tasks with PDG/TOPs

Basics

Beginner Tutorials

Next steps

  • Running external programs

    How to wrap external functionality in a TOP node.

  • File tags

    Work items track the results created by their work. Each result is tagged with a type.

  • PDG Path Map

    The PDG Path Map manages the mapping of paths between file systems.

  • Feedback loops

    You can use for-each blocks to process looping, sequential chains of operations on work items.

  • Command servers

    Command blocks let you start up remote processes (such as Houdini or Maya instances), send the server commands, and shut down the server.

  • PDG Services

    PDG services manages pools of persistent Houdini sessions that can be used to reduce work item cooking time.

  • Integrating PDG with render farm schedulers

    How to use different schedulers to schedule and execute work.

  • Visualizing work item performance

    How to visualize the relative cook times (or file output sizes) of work items in the network.

  • Event handling

    You can register a Python function to handle events from a PDG node or graph

  • Tips and tricks

    Useful general information and best practices for working with TOPs.

  • Troubleshooting PDG scheduler issues on the farm

    Useful information to help you troubleshoot scheduling PDG work items on the farm.

  • PilotPDG

    Standalone application or limited license for working with PDG-specific workflows.

Reference

  • All TOPs nodes

    TOP nodes define a workflow where data is fed into the network, turned into work items and manipulated by different nodes. Many nodes represent external processes that can be run on the local machine or a server farm.

  • Processor Node Callbacks

    Processor nodes generate work items that can be executed by a scheduler

  • Partitioner Node Callbacks

    Partitioner nodes group multiple upstream work items into single partitions.

  • Scheduler Node Callbacks

    Scheduler nodes execute work items

  • Custom File Tags and Handlers

    PDG uses file tags to determine the type of an output file.

  • Python API

    The classes and functions in the Python pdg package for working with dependency graphs.

  • Job API

    Python API used by job scripts.

  • Utility API

    The classes and functions in the Python pdgutils package are intended for use both in PDG nodes and scripts as well as out-of-process job scripts.