Hqueue for sops

   2773   6   3
User Avatar
Member
224 posts
Joined: June 2009
Offline
I have a large urban model with lots of buildings. For each building, I need to to do some slow operations using Sops that add some attributes to the geometry.

I have never used Hqueue before, but I am wondering: is there a way to parallelise this process using Hqueue. I would have to create one job for each building. Each building could then be processed by a separate client.

http://www.sidefx.com/docs/houdini15.0/ref/hqueue [sidefx.com]
This page says: “Technically HQueue can start any script on client machines, not just simulations and renders.Once you have an HQueue farm set up, you can use it to distribute other types of work.”

So it sounds like it is possible. Any suggestions on how to do this?
Patrick
User Avatar
Member
224 posts
Joined: June 2009
Offline
I have been looking at this in more detail. It seems I will have to create my own ROP since HQueue Render and HQueue Simulation will not work for me.

Here is what I am thinking. Am I on the right track? Or is there a better way?

From the HQueue API
http://www.sidefx.com/docs/hqueue15.0/help/api.html [sidefx.com]

The key method seems to be
newjob( jobspec, parent_id=None, child_ids= )

So for each building in my model, I start a new job to process that building.

The jobspec seems to be the key piece of data, which can include a script like this.
http://www.sidefx.com/docs/hqueue15.0/help/jobdetails [sidefx.com]
{
“name”: “Process building 1”,
“command” : “hython /path/to/myScript.py 1”,
}

So now for the script… it should load the hip file, set a few parameters, cook, and write a bgeo file to a shared drive.

Finally, I just need to merge all the bgeo files, which I think I should be able to define as a separate child job (so that it only executes when the others have finished).
Patrick
User Avatar
Staff
733 posts
Joined: Oct. 2012
Offline
I think the HQueue Simulation ROP should work with a Geometry ROP, and the Wedge ROP can let you create separate jobs for each building.
User Avatar
Member
224 posts
Joined: June 2009
Offline
Thanks, I did a simple test, but did not yet run it on Hqueue.

See H15 file attached. Does it seem about right?

Attachments:
test.hipnc (95.2 KB)

Patrick
User Avatar
Staff
4177 posts
Joined: Sept. 2007
Offline
I think your hip file suggests the right idea, but I wasn't sure what all the blend sops were for… but here is a working example.

Just make sure that your wedge rop is iterating over each floor (via switch sop, blast sop, etc…) and make sure you put $WEDGE in the bgeo. It should work.

Make sure to set the HQSim ROP to partition by wedges, and i the wedge rop, if you tell it to iterate over more than the number of floors, you'll get an error from the shell rop embedded in the wedge rop.

Does that help? You should be able to point that at your hqueue server, cook it, and then read in all 6 “floors” of geometry.

Attachments:
wedge_hqueue_example.hipnc (159.3 KB)

I'm o.d.d.
User Avatar
Staff
733 posts
Joined: Oct. 2012
Offline
goldleaf
I think your hip file suggests the right idea, but I wasn't sure what all the blend sops were for… but here is a working example.

Yeah, I also noticed that when opening the file and submitted a bug for it - the file opens correctly (without the errors on load and the blend sops) in the version the hip file was created with (15.0.244.16)
User Avatar
Member
224 posts
Joined: June 2009
Offline
Thanks goldleaf, cwhite - I will give it a try.
Patrick
  • Quick Links