I'm doing some wedged rendering with redshift and thought it would be a good opportunity to try TOPs for the first time. I was able to get it working on a basic level with a Wedge TOP (3 workitems) connected to a ROP Fetch TOP, referencing my redshift ROP (900 frames each, so 2700 workitems).

The problem is that each workitem seems to be a separate command, and therefore a separate houdini process. I'd like to batch each wedge together, into the one houdini process, so I don't suffer the houdini startup and scene cook time penalty each frame.

I'm new to TOPs and find it all a bit confusing but it seems like the ‘All Frames in One Batch’ option on the ROP Fetch could do what I need. When I generate the work items they seem to be doing what I need (the work item command represents a frame range, not individual frames), however when I cook the TOP, all my work items fail with this python error:

Loading .hip file C:/myfilepath/waveforms_v04.hiplc. Traceback (most recent call last): File "C:/myfilepath/pdgtemp/47396/scripts/rop.py", line 500, in cooker.cookBatchFrames(args) File "C:/myfilepath/pdgtemp/47396/scripts/rop.py", line 253, in cookBatchFrames """.format(args=args, index_expr=index_expr, parm_name=parm.name()) AttributeError: 'NoneType' object has no attribute 'name' [Redshift] Closing the RS instance. End of the plugin log system.

I've tried it with fetching a mantra ROP and it seems to work ok. I'm using Houdini 17.5.173.

thanks

Update: I just took a look in that python script (rop.py) that's erroring out and I think I've found the issue. It seems to be looking for an output file parameter on the ROP, but the script only has a hardcoded list of parameter names that obviously only includes relevant parameters found on built-in SESI ROPs.

Redshift's output parm name is RS_outputFileNamePrefix so clearly doesn't get found, and errors out. I did a hacky workaround by adding a spare parameter ‘vm_picture’ to the Redshift ROP, channel referencing redshift's output path value and now it all seems to be working.

Ideally:

a) In the short term there should be a better error message that says the ROP node you are fetching is unsupported, rather than just the python error.

or

b) There should probably be a well documented, better way of accessing this output path information from ROP nodes (maybe even a parameter on the ROP Fetch that you could channel reference to). Many vfx facilities will have their own collection of ROP hdas to do various things in the pipeline and that hardcoded python script won't have any knowledge of those parameter names either.

thanks

Hey, Matt!

Did you have to wrap your Redshift ROP Fetch in a For loop?

And are you rendering on a system with multiple GPUs?

One of the issues I've encountered with sending parallel TOPs renders to Redshift is that it swamps the GPU(s) and runs the memory up beyond the 90/10% limit, which ends up crashing Redshift (and sometimes other things). Fortunately, the Redshift logging is dynamic enough to let you see the process start screaming verbosely before it dies horribly…

Any chance you can share a scene with your TOPs setup? I'd like to both get Redshift running reliably with the current PDG iteration and then share out some examples to help the Houdini/Redshift community.

–
mC

No for loops, and just a single GPU and only one concurrent process ('single' tick box on the local scheduler). I'm just using TOPs since I wanted to give it a try, and since it seems like it might streamline the workflow a bit for me, not for any parallel wizadry.

Re. the original problem, come to think of it, is there any reason why output files are requirement of batching in the first place? I can imagine there could be situations where you want to run a ROP (maybe not sidefx built in ROPs, but custom pipeline tools) as a batch across multiple frames but not necessarily generate output files, or even an output file per frame.

Hey, Matt…looks like I was one step behind your issue.

Once I got RS reliably rendering in the TOPnet, I hit the ROPFetch output issue. Nasty little bug; totally nerfs caching and prevents any downstream manipulation of renders.

You vm_picture param add worked like a charm. I'd tried a few even more hacky things originally but this was definitely the win.

I brought this to the attention of the Redshift devs and the next official build (guessing it will be 2.6.43) will have an invisible vm_picture parameter that is the same value as RS_outputFileNamePrefix. I think that also means all the experimental 3.0.x builds after that will have it as well.

Love how fast the RS team is as turning around low-impact changes like this.

Now I just need to keep massaging settings until I can figure out the best/most efficient frame handling with GPUs. So excited about using PDG in my workflow; really looking forward to full GPU/3rd party support from SideFX.

Starting with Houdini 17.5.280, the ROP Fetch node has an “Output Parm Name” field. When that's enabled, the ROP Fetch node + the rop.py job script will use the value in that parm to figure out which ROP parameter contains the output path. This provides a user-driven way to force PDG to use the correct output parm name, when it can't figure it out automatically.

We should also be able to add the Redshift ROP's output parm name (and the ones from the Arnold ROP, etc) to the list that the ROP Fetch node uses internally so that it works correctly out of the box. I'm planning to look into that this week.

Awesome! Thanks, Taylor.

You may want to touch base with the Redshift devs, too. They've responded that they're also adding vm_picture to the next RS Houdini build. All sorts of dev love today.


–
mC

Thanks very much!

Just an update: as of Houdini 17.5.293 and Redshift 2.6.43. This now appears to be working. The VM_picture has been added to Redshift and the ROP Fetch seems to work without the custom Output Parm Name.

Fantastic! Thanks for the update!

–
mC