Houdini 18.0 Executing Tasks

File paths

Best practices for input/output file paths in TOP networks.

On this page

Overview

Most TOP nodes that do work let you specify input and/or output file paths. In TOPs, each scheduler node can specify a working directory (this is because different render farm software may use different shared network filesystems). You should make all the file paths you output to relative to this PDG_DIR or $HIP directory.

How to

  • Set the base working directory on the scheduler node. This directory is available to work scripts as the PDG_DIR environment variable.

    • Use separate working directories for each HIP file. This is to avoid having two HIP files writing to the same PDG_DIR. Many of the default generated filenames used in parameter defaults are only unique within the HIP file.

    • For render farm schedulers, make sure the directory inside the network filesystem (for example, NFS mount of SMB share) shared with the render farm client machines.

  • When you specify filenames (for example, images to render or geometry files to generate), make sure the path is relative to $HIP or PDG_DIR.

  • Houdini-specific nodes such as ROP Geometry should continue to use $HIP-relative paths.

  • When you use PDG_DIR in filename parms for non-Houdini TOP nodes such as ffmpeg or imagemagick, use the form __PDG_DIR__ instead of ${PDG_DIR}. If you use ${PDG_DIR}, Houdini will try and fail to expand the variable itself before the dependency graph gets it. Houdini will ignore __PDG_DIR__ syntax, but schedulers will expand it to the correct path for the filesystem on the farm.

Environment variables

PDG_DIR

The TOP network’s working directory, as specified on the Scheduler node. In TOP parameters, this is an alias for __PDG_DIR__. This is set in the job environment to the local path to the cook working directory.

PDG_TEMP

A shared temporary file directory inside the working directory for the current session. The default is $PDG_DIR/pdgtemp/houdini_process_id. This is set in the job environment.

PDG_SCRIPTDIR

A shared script directory inside the temp directory. Script files are copied into this directory if they are listed as file dependencies. The default is $PDG_TEMP/scripts. This is set in the job environment.

Alternatively, you can simply put scripts in known locations in the shared network filesystem.

PDG_ITEM_NAME

The name of the work item being executed, which also corresponds to the name of the serialized work item file in the data directory. This is set in the job environment.

PDG_HYTHON

If set in Houdini, schedulers use this path instead of $HFS/bin/hython. Note that if this is a different version of Houdini being run by Local Scheduler, at least $HFS will need to be cleared in the work item environment to avoid library conflicts.

PDG_RESULT_SERVER

The hostname and port of the server that jobs send their status and results to. This is set in the job environment.

Executing Tasks

Basics

Beginner Tutorials

Next steps

Reference

  • All TOPs nodes

    TOP nodes define a workflow where data is fed into the network, turned into "work items" and manipulated by different nodes. Many nodes represent external processes that can be run on the local machine or a server farm.

  • Processor Node Callbacks

    Processor nodes generate work items that can be executed by a scheduler

  • Partitioner Node Callbacks

    Partitioner nodes group multiple upstream work items into single partitions.

  • Scheduler Node Callbacks

    Scheduler nodes execute work items

  • Custom File Tags and Cache Handlers

    PDG uses file tags to determine the type of an output file.

  • Python API

    The classes and functions in the Python pdg package for working with dependency graphs.