To actually do the work you specified when the network runs, something must take the commands that are ready to go, create an execution environment, and run them. In a TOP network, this is the scheduler node.

You can have more than one scheduler in a TOP network, however the scheduler that cooks the network is specified on the TOP network node.

Having multiple schedulers allows you to set up different "profiles" and switch between them (for example, a local scheduler for small-scale testing, and a render farm scheduler for full production runs).

You can override the scheduler used for certain nodes. This lets you run small jobs and filesystem modifications locally, instead of incurring the overhead of sending them out to a farm. (Some nodes even have an option to do work in the main process, rather than scheduling them at all.) A node can also override certain top-level job parameters.

The default scheduler node for new TOP networks schedules work using a process queue. Depending on the workload, this may actually be faster than using a render farm because of the lower overhead.

Out of the box, TOPs has built-in scheduler nodes for several render farm packages (see below).

TOPs only works with distributed setups where the controlling machine and all servers share a network filesystem.

You can use custom scheduler bindings to integrate other third party or in-house scheduling software.

Starting operations in separate processes, scheduling work on the farm, and moving data across the network filesystem all add overhead to work you do in TOPs. Some nodes have options to do small jobs inside the process cooking the graph, and you can override small work items to run on the same maching instead of being scheduled on the farm.

However, even for an HDA Processor node running locally, if the asset cooks quickly, the overhead of running it in a separate process still kills overall performance. There are few things you can do to optimize or work around this:

• Use a ROP Geometry node to generate geometry instead of HDA Processor, and do your wedging using time/frame number.

  In the same HIP file as your TOP Network, instantiate your asset, and point a ROP Geometry node at the SOP network. On the ROP Geometry node, turn on All Frames in One Batch on the ROP Fetch tab. That will create a single process that cooks the full frame range one frame at a time, which is typically used for simulations but is often more efficient for light-weight geometry jobs.

  You can also play with a custom batch size. For example, instead of All Frames in One Batch, you could use 20 frames per batch.

• If possible, you can try using the Invoke node instead of HDA Processor. The Invoke node cooks a SOP chain in the process cooking the graph instead of in its own process. The limitation is that you can only use it on a compiled block. For small/quick operations this is much more efficient.

• You can also try increasing the number of processes on the local machine in the Local Scheduler. The default maximum is very low (number of cores / 4) to avoid slowing down the local machine, but if you're overriding small jobs to run locally you should probably increase this to the number of cores.

• On the Local Scheduler, turn on HDAProcessor pool and set it to a number greater than 0. This creates a pool of HDAProcessor instances that will be re-used for HDAProcessor jobs, instead of each new HDAProcessor job running in a separate instance.

A future version of Houdini will probably add batching to the HDA Processor so it can cook multiple asset variations (wedges) in a single process, much like the ROP Geometry’s batching mode.