To actually do the work you specified when the network runs, something must take the commands that are ready to go, create an execution environment, and run them. In a TOP network, this is the scheduler node.

You can have more than one scheduler in a TOP network, however the scheduler that cooks the network is specified on the TOP network node.

Having multiple schedulers allows you to set up different "profiles" and switch between them (for example, a local scheduler for small-scale testing, and a render farm scheduler for full production runs).

You can override the scheduler used for certain nodes. This lets you run small jobs and filesystem modifications locally, instead of incurring the overhead of sending them out to a farm. (Some nodes even have an option to do work in the main process, rather than scheduling them at all.) A node can also override certain top-level job parameters.

The default scheduler node for new TOP networks schedules work using a process queue. Depending on the workload, this may actually be faster than using a render farm because of the lower overhead.

Out of the box, TOPs has built-in scheduler nodes for several render farm packages (see below).

TOPs only works with distributed setups where the controlling machine and all servers share a network filesystem.

You can use custom scheduler bindings to integrate other third party or in-house scheduling software.

You can change the limit on the number of compute resources available to the Local Scheduler node (the default limit is the number of cores in the local computer).

• Both HQueue and the Local Scheduler have a concept of how many "CPUs / slots" (an abstract unit) a job requires, and a maximum number of slots on a machine to use to run jobs simultaneously.

• For example, if a certain job is itself multi-processing or multi-threaded, using four cores, you can mark that job as requiring 4 slots. The Local Scheduler or HQueue Scheduler will then only schedule the job if the machine has at least 4 "free" slots below the maximum limit.

• To specify the number of slots the jobs generated by a TOP node require:

  1. Select the TOP node.

  2. In the parameter editor, open the Gear menu and choose Edit Rendering Parameters.

  3. Select the Node Properties tab in the pane on the left.

  4. In the Scheduler Properties tree folder are all the overridable properties for each type of scheduler, select Slots Per Work Item below the Local folder.

  5. Drag that block of properties into the Schedulers folder in the left pane of the window. This will install those properties onto the TOP node.

  6. Dismiss the window, then toggle on Slots Per Work Item and set the number of slots to assign.

  Note

  You can create an expression on this parameter if you want to vary it’s value based on work item attributes.

• The Local Scheduler’s Total Slots lets you specify the maximum number of slots the TOPs is allowed to use simultaneously while cooking.

• HQueue CPUs per Job lets you specify per-machine CPU limits.

(Future versions may allow limits on other resource types, such as databases, similar to HQueue.)

Starting operations in separate processes, scheduling work on the farm, and moving data across the network filesystem all add overhead to work you do in TOPs. Some nodes have options to do small jobs inside the process cooking the graph, and you can override small work items to run on the same maching instead of being scheduled on the farm.

However, even for an HDA Processor node running locally, if the asset cooks quickly, the overhead of running it in a separate process still kills overall performance. There are few things you can do to optimize or work around this:

• Use a ROP Geometry node to generate geometry instead of HDA Processor, and do your wedging using time/frame number.

  In the same HIP file as your TOP Network, instantiate your asset, and point a ROP Geometry node at the SOP network. On the ROP Geometry node, turn on All Frames in One Batch on the ROP Fetch tab. That will create a single process that cooks the full frame range one frame at a time, which is typically used for simulations but is often more efficient for light-weight geometry jobs.

  You can also play with a custom batch size. For example, instead of All Frames in One Batch, you could use 20 frames per batch.

• If possible, you can try using the Invoke node instead of HDA Processor. The Invoke node cooks a SOP chain in the process cooking the graph instead of in its own process. The limitation is that you can only use it on a compiled block. For small/quick operations this is much more efficient.

• You can also try increasing the number of processes on the local machine in the Local Scheduler. The default maximum is very low (number of cores / 4) to avoid slowing down the local machine, but if you're overriding small jobs to run locally you should probably increase this to the number of cores.