Python Scheduler TOP node

A Python-based programmable Scheduler for PDG.

This node provides an interface to write Python code for a custom PDG Scheduler and use it to schedule work items. The base Scheduler’s callbacks are exposed as Python text fields which should be filled in according to their function definition. Each callback field is then evaluated and executed during the general PDG work item schedule workflow.

The Python Scheduler node can be used as is in a TOP network, or it can be saved out to a Python file using the Save To Python button. The Save To Python functionality will write out a complete implementation of a custom Scheduler node with the filled in callback entries. The Python file should be placed in the PDG Python path (e.g. /home/username/houdini17.5/pdg/types/) so that it will be automatically picked up and registered with the Scheduler Factory. This happens on the launch of Houdini, which means changes to the custom Scheduler will require a relaunch of Houdini.

Once registered, the custom Scheduler node can be created by selecting it from the TAB menu under Schedulers. If it does not show up there, either the Python file was not found by Houdini at launch, or the Python file has errors.

Parameters

Save To Python

Save this node out to a Python file, along with its custom callback entries.

Working Directory

The relative directory where the work will be generating intermediate files and output. The intermediate files will be placed in a subdirectory. Typically $HIP is used. For running jobs on the farm, the working directory would typically be a mounted directory on all client machines, including the machine scheduling the work.

Scheduling

Schedule

onSchedule Callback

Called when a work item needs to scheduled. Returns True if the work item has been successfully scheduled.

Schedule Static

onScheduleStatic Callback

Called when the scheduler should process a static dependency graph.

Submit As Job

submitAsJob Callback

Called when the scheduler should cook the entire TOP Network as a standalone job. Returns the status URI for the submitted job.

Initialization and Cleanup

Start

onStart Callback

Called when the scheduler is registered with PDG. Should initialize or setup scheduler state.

Stop

onStop Callback

Called when the scheduler is unregistered with PDG. Should clean up scheduler state.

Start Cook

onStartCook Callback

Called when a cook starts, before any work items are scheduled. Returns True if started.

Stop Cook

onStopCook Callback

Called when cook completes or is canceled. Returns True if stopped.

Shared Server

End Shared Server

endSharedServer Callback

Closes an existing shared server instance. Returns True on success.

Result Server Address

workItemResultServerAddr Callback

Returns the results server address. This is the TCP port that listens for status being returned by jobs. It should be a string-based IP address: (e.g. '192.168.1.123:83000').

Transfer File

transferFile Callback

Called when a file should be copied from the local machine to a remote location (e.g. a shared network drive). These would be files that are necessary to do the wok, so they are copied to the working directory. Returns True if copied the file.

Logging

Log URI

getLogURI Callback

Returns the URI that points to the log of the given work item (e.g. file:///myfarm/tasklogs/jobid20.log).

Status URI

getStatusURI Callback

Returns the URI that points to the status of the given work item (e.g. http://myfarm/status/jobid20)