I'm trying to figure out how batching is intended to work on the HDA Processor. I can increase the number of iterations and things work much as I'd expect them to, but increasing Batch Size seems to give odd results. Moreover, when Batch Size is greater than 1, none of the work items in the HDA Processor will execute anymore and instead they are all set perpetually into the ‘Waiting’ state.

I haven't been able to find any useful documentation as to what the intended behaviour is for this setting, besides what the help files mention:

Batch Size: When enabled, indicates that this node should generate batch work items and sets the batch size to this parameter’s value.

More broadly however, I am hoping to figure out what is the best way to output multiple geometry files from a single HDA Processor.

For example, I have created a 1000x1000 terrain heightfield with noise on it. I then wish to split it up into 100 tiles by cropping the input heightfield, 100x100 each, run an operation on each tile and then recombine the full landscape. Using the HDA Processor, whether I increase the number of iterations or set up a wedge node to drive my tile generator parameters, the result is quite slow. It spins up separate processes for every single tile, which means it has to load up the full landscape 100 times and only then crop it.

By comparison, a ROP Geometry node can work much faster. If it's set to run sequentially over frames, you can batch multiple geometries to be output from a single session. I can define a frame range from 1 to 100, set the Frames Per Batch parameter to be 25, and I can get four simultaneous processes to crunch through 25 tiles each really quickly, without having to reload the landscape between exports.

It seems like the HDA Processor should be able to do this as well, and the batching controls suggest that kind of functionality. But I can't get it to output anything at all?

Here's a practical demonstration of what I am trying to do:

The batch options were removed from hdaprocessor in recent builds. The reason for this is that unlike ROPGeometry, with hdaprocessor you would want different parm values per iteration. Batches currently only have one shared set of parm values, and only the frame number changes per sub item. For this to work for you, we would need a way to serialize a different set of parm values per subitem and then cook the hda in a loop via Engine.

However it would be possible now to manually cook the node with new parm values with a Houdini command chain via HOM with Send Command node, this would be a sort of explicit batch processing.

Thanks, Chris!