| Since | 17.5 |
This node provides an interface to write Python code for a custom PDG Scheduler and use it to schedule work items. The base Scheduler’s callbacks are exposed as Python text fields which should be filled in according to their function definition. Each callback field is then evaluated and executed during the general PDG work item schedule workflow. The Scheduler node API reference has a detailed explanation for each of the callback functions that you can implement.
The Python Scheduler node can be used as is in a TOP network, or it can be saved out to a Python file using the Save To Python button. The Save To Python functionality will write out a complete implementation of a custom Scheduler node with the filled in callback entries. The Python file should be placed in the PDG Python path (e.g. /home/username/houdini17.5/pdg/types/) so that it will be automatically picked up and registered with the Scheduler Factory. This happens on the launch of Houdini, which means changes to the custom Scheduler will require a relaunch of Houdini.
Once registered, the custom Scheduler node can be created by selecting it from the TAB menu under Schedulers. If it does not show up there, either the Python file was not found by Houdini at launch, or the Python file has errors.
Parameters
Save To Python
Save this node out to a Python file, along with its custom callback entries.
Working Directory
Specifies the relative directory where the work generates intermediate files and output. The intermediate files are placed in a subdirectory. For the Local Scheduler or HQueue, typically $HIP is used. For other schedulers, this should be a relative directory to Local Shared Root Path and Remote Shared Root Path; this path is then appended to these root paths.
Validate Outputs When Recooking
When enabled, PDG will check the output files of work items when the graph recooks, to see if the files still exist on disk. Work items that are missing output files will be dirtied and cook again.
Check Expected Outputs on Disk
When enabled, PDG will look on disk for any expected work items outputs that were not explicitly reported when the work item cooked. Expected outputs for a work item are checkd immediately after the scheduler marks the work item as cooked. Output files that were reported by the work item normally while cooking will not be checked.
Block on Failed Work Items
When this option is enabled the scheduler will block the cook from completing if there are any failed work items in that scheduler. This makes it possible to manually retry those work items, by preventing the PDG graph cook from ending before failed items can be retried. A cook that is blocked on failed work items can still be canceled using the ESC key, the cancel button in the TOP task bar, or the cancel API method.
Scheduling
Schedule
onSchedule Callback
Called when a work item needs to scheduled. Returns True if the work item has been successfully scheduled.
Schedule Static
onScheduleStatic Callback
Called when the scheduler should process a static dependency graph.
Submit As Job
submitAsJob Callback
Called when the scheduler should cook the entire TOP Network as a standalone job. Returns the status URI for the submitted job.
Initialization and Cleanup
Start
onStart Callback
Called when the scheduler is registered with PDG. Should initialize or setup scheduler state.
Stop
onStop Callback
Called when the scheduler is unregistered with PDG. Should clean up scheduler state.
Start Cook
onStartCook Callback
Called when a cook starts, before any work items are scheduled. Returns False to cancel the cook, True otherwise.
Stop Cook
onStopCook Callback
Called when cook completes or is canceled. Return value is ignored.
Shared Server
Logging
Log URI
getLogURI Callback
Returns the URI that points to the log of the given work item (e.g. file:///myfarm/tasklogs/jobid20.log).
Status URI
getStatusURI Callback
Returns the URI that points to the status of the given work item (e.g. http://myfarm/status/jobid20)
| See also |
Deadline Scheduler
