I have a large urban model with lots of buildings. For each building, I need to to do some slow operations using Sops that add some attributes to the geometry.
I have never used Hqueue before, but I am wondering: is there a way to parallelise this process using Hqueue. I would have to create one job for each building. Each building could then be processed by a separate client.
http://www.sidefx.com/docs/houdini15.0/ref/hqueue [sidefx.com]
This page says: “Technically HQueue can start any script on client machines, not just simulations and renders.Once you have an HQueue farm set up, you can use it to distribute other types of work.”
So it sounds like it is possible. Any suggestions on how to do this?
Hqueue for sops
2773 6 3- phtj
- Member
- 224 posts
- Joined: June 2009
- Offline
- phtj
- Member
- 224 posts
- Joined: June 2009
- Offline
I have been looking at this in more detail. It seems I will have to create my own ROP since HQueue Render and HQueue Simulation will not work for me.
Here is what I am thinking. Am I on the right track? Or is there a better way?
From the HQueue API
http://www.sidefx.com/docs/hqueue15.0/help/api.html [sidefx.com]
The key method seems to be
newjob( jobspec, parent_id=None, child_ids= )
So for each building in my model, I start a new job to process that building.
The jobspec seems to be the key piece of data, which can include a script like this.
http://www.sidefx.com/docs/hqueue15.0/help/jobdetails [sidefx.com]
{
“name”: “Process building 1”,
“command” : “hython /path/to/myScript.py 1”,
}
So now for the script… it should load the hip file, set a few parameters, cook, and write a bgeo file to a shared drive.
Finally, I just need to merge all the bgeo files, which I think I should be able to define as a separate child job (so that it only executes when the others have finished).
Here is what I am thinking. Am I on the right track? Or is there a better way?
From the HQueue API
http://www.sidefx.com/docs/hqueue15.0/help/api.html [sidefx.com]
The key method seems to be
newjob( jobspec, parent_id=None, child_ids= )
So for each building in my model, I start a new job to process that building.
The jobspec seems to be the key piece of data, which can include a script like this.
http://www.sidefx.com/docs/hqueue15.0/help/jobdetails [sidefx.com]
{
“name”: “Process building 1”,
“command” : “hython /path/to/myScript.py 1”,
}
So now for the script… it should load the hip file, set a few parameters, cook, and write a bgeo file to a shared drive.
Finally, I just need to merge all the bgeo files, which I think I should be able to define as a separate child job (so that it only executes when the others have finished).
Patrick
- cwhite
- Staff
- 733 posts
- Joined: Oct. 2012
- Offline
- phtj
- Member
- 224 posts
- Joined: June 2009
- Offline
- goldleaf
- Staff
- 4177 posts
- Joined: Sept. 2007
- Offline
I think your hip file suggests the right idea, but I wasn't sure what all the blend sops were for… but here is a working example.
Just make sure that your wedge rop is iterating over each floor (via switch sop, blast sop, etc…) and make sure you put $WEDGE in the bgeo. It should work.
Make sure to set the HQSim ROP to partition by wedges, and i the wedge rop, if you tell it to iterate over more than the number of floors, you'll get an error from the shell rop embedded in the wedge rop.
Does that help? You should be able to point that at your hqueue server, cook it, and then read in all 6 “floors” of geometry.
Just make sure that your wedge rop is iterating over each floor (via switch sop, blast sop, etc…) and make sure you put $WEDGE in the bgeo. It should work.
Make sure to set the HQSim ROP to partition by wedges, and i the wedge rop, if you tell it to iterate over more than the number of floors, you'll get an error from the shell rop embedded in the wedge rop.
Does that help? You should be able to point that at your hqueue server, cook it, and then read in all 6 “floors” of geometry.
I'm o.d.d.
- cwhite
- Staff
- 733 posts
- Joined: Oct. 2012
- Offline
goldleaf
I think your hip file suggests the right idea, but I wasn't sure what all the blend sops were for… but here is a working example.
Yeah, I also noticed that when opening the file and submitted a bug for it - the file opens correctly (without the errors on load and the blend sops) in the version the hip file was created with (15.0.244.16)
- phtj
- Member
- 224 posts
- Joined: June 2009
- Offline
-
- Quick Links