Houdini 18.5 Executing tasks with PDG/TOPs

PDG Service Manager

The PDG Service Manager manages pools of persistent Houdini sessions that can be used to reduce work item cooking time

On this page

Overview

A "PDG service" is a long-running process that acts as a server for cooking work items. When a node is configured to use a PDG service, a cooking work item will submit the job to a pool of services. The first service that is available will retrieve the work item and perform its work.

PDG services avoid the overhead associated with starting a new process. Services can also perform optimizations specific to its job (for example, the HDA Processor service can create the necessary nodes exactly once, reusing the same nodes for each job and avoiding the time associated with creating the network). As a result, PDG services can result in a vast speed increase for jobs that are relatively quick.

Supported Nodes

PDG Service Menu

In the Tasks menu, select PDG Service Menu to open the PDG Service Menu:

Registered Service List

Service

Shows the name of the PDG service.

Status

Indicates whether the service is running or stopped.

Pool Size

The number of instances that will be created for the service pool. More instances will result in faster cook times because more items will be able to be cooked in parallel.

Automatically Start

Indicates whether the service will be automatically started by the first PDG graph cook where the service is used (if the service is not already running).

Scheduler Type

The name of the scheduler type that the service is associated with.

Scheduler

For some scheduler types, the service must be run with respect to a specific scheduler node. This field allows the scheduler node to be specified.

Start

Starts the service.

Stop

Stops the service.

Logging

Client Logging Directory

This field specifies the folder where service pool clients will write their log files.

Service Registration

Add Service

Remove Service

Opens a dialog for removing registered services.

Register PDG Service Dialog

Service Type

The service type associated with the registered service. Each service type can only be registered once.

Scheduler Type

The scheduler type associated with the registered service. Only scheduler types that support PDG services will appear in the list.

A service must be associated with a particular scheduler type because the scheduler type provides the mechanism for starting and stopping the service. The Local scheduler, for example, will create the service and its clients as new processes on the local system.

Pool Size

The number of clients that will be created by the service.

Port

The port configuration choice. The port that the service will use can either be determined automatically or a custom port can be provided.

Port Number

Specifies the port number when "Custom" port mode is chosen.

Client Logging Directory

Clients within the service pool will create logfiles within the specified logging directory.

Automatically Start Service On First PDG Cook

When this option is enabled, the service will be automatically started (if it’s not already running) during the first PDG cook where the service is used.

Register

Registers the service.

Cancel

Cancels the register operation.

Remove PDG Service Dialog

This dialog shows a list of registered services which can then be selected and removed. Services can be re-registered at any time.

How To

Described below are the steps for using a PDG service with one of the supported node types:

1. Open the service registration dialog to register the service with the desired configuration.

2. Press the Start button for the desired service.

3. Configure any nodes that you want to use the service. For this step, please consult the node’s documentation for information on configuring the node to use the service.

4. When any configured nodes are cooked, they will now use the PDG service pool to cook work items.

Executing tasks with PDG/TOPs

Basics

Beginner Tutorials

Next steps

  • Running external programs

    How to wrap external functionality in a TOP node.

  • File tags

    Work items track the "results" created by their work. Each result is tagged with a type.

  • PDG Path Map

    The PDG Path Map manages the mapping of paths between file systems.

  • Feedback loops

    You can use for-each blocks to process looping, sequential chains of operations on work items.

  • Command servers

    Command blocks let you start up remote processes (such as Houdini or Maya instances), send the server commands, and shut down the server.

  • PDG Service Manager

    The PDG Service Manager manages pools of persistent Houdini sessions that can be used to reduce work item cooking time

  • Integrating PDG with render farm schedulers

    How to use different schedulers to schedule and execute work.

  • Visualizing work item performance

    How to visualize the relative cook times (or file output sizes) of work items in the network.

  • Event handling

    You can register a Python function to handle events from a PDG node or graph

  • Tips and tricks

    Useful general information and best practices for working with TOPs.

  • Troubleshooting PDG scheduler issues on the farm

    Useful information to help you troubleshoot scheduling PDG work items on the farm.

  • PilotPDG

    Standalone application or limited license for working with PDG-specific workflows.

Reference

  • All TOPs nodes

    TOP nodes define a workflow where data is fed into the network, turned into "work items" and manipulated by different nodes. Many nodes represent external processes that can be run on the local machine or a server farm.

  • Processor Node Callbacks

    Processor nodes generate work items that can be executed by a scheduler

  • Partitioner Node Callbacks

    Partitioner nodes group multiple upstream work items into single partitions.

  • Scheduler Node Callbacks

    Scheduler nodes execute work items

  • Custom File Tags and Cache Handlers

    PDG uses file tags to determine the type of an output file.

  • Python API

    The classes and functions in the Python pdg package for working with dependency graphs.

  • Job API

    Python API used by job scripts.