Forums PDG/TOPs Pyro Clusters with PDG

Pyro Clusters with PDG

   3699   6   3
User Avatar
scarything
Member
5 posts
Joined: June 2017
Offline
 
Hey,

I have a situation where i need to simulate 50+ pyro clusters and I thought it might be a good idea to learn a bit about TOPs/PDG at the same time.

I've got the setup working as far as in being able to wedge the cluster filter to separate my clusters.
What i can't quite figure out is how i tell the system to only simulate one wedge at a time? Right now it's doing multiple clusters at a time, which isn't optimal as i'd like to be able to have it run through a range of clusters i choose whenever i have some spare CPU hours on my machine.

Perhaps a regular wedge ROP is more suited for this?
 Report
User Avatar
Tyler Britton2
Member
83 posts
Joined: April 2014
Offline
 
On a side note, I have noticed that PDG isnt using all of my processing power. I 28 core (56 threads CPU), and I am only getting about 30% CPU usage on my tasks. I am running 6 pyro sims that normally give me about 85% usage when running locally without tops. I have limited each work item to 8 CPUs (to avoid running out of RAM), but I also get low CPU when using no limits. I have Maximum CPUs to Use set to Use All CPUs Except One.

Is there a way I could get some more use out of my CPUs? I am running individual pyro sims, not clustering.
 Report
User Avatar
tpetrick
Staff
585 posts
Joined: May 2014
Offline
 
scarything
What i can't quite figure out is how i tell the system to only simulate one wedge at a time?

On the “Schedulers” tab of the ROP Geometry TOP you're using to run your sims, you enable the “Single” flag for the local scheduler. That indicates that only one job of that type can be running at the same time.
Edited by tpetrick - June 3, 2019 16:17:56
 Report
User Avatar
kenxu
Member
544 posts
Joined: Sept. 2012
Offline
 
Re. CPU utilization, try turning up the “Houdini Max Threads” setting on the local scheduler. You've already set the Maximum CPUs to use, so that's not the issue.
- Ken Xu
 Report
User Avatar
Alejandro Echeverry
Member
691 posts
Joined: June 2006
Offline
 
kenxu
Re. CPU utilization, try turning up the “Houdini Max Threads” setting on the local scheduler. You've already set the Maximum CPUs to use, so that's not the issue.

I have the same issue of PDG not using all my cores even forcing the Max threads parameter to the number of threads my pc has, and also changing the “Maximum CPUs to Use” to “Use all cpus except one”.

The weird thing is that if you create a file from scratch that uses a pyro or flip sim using PDG all of this will work normally (PDG will use all the threads if you force it), but after editing more the file or changing it, PDG will stop using all the machine threads even if you force it.




This behavior is happening only when you want to use “All frames in one batch”; I tested with a node that cooks each work item without dependencies (in this case “invoke”, in the pipeline is doing the meshing), and PDG uses all my threads without problem for each work item.

Edited by Alejandro Echeverry - June 21, 2019 12:31:50

Attachments:
PDG_maxThreads.png (183.6 KB)
PDG_threads_noSingle.png (190.2 KB)
PDG_therads_invoke.png (199.8 KB)

Feel The Knowledge, Kiss The Goat!!!
http://www.linkedin.com/in/alejandroecheverry [linkedin.com]
http://vimeo.com/lordpazuzu/videos [vimeo.com]
 Report
User Avatar
Alejandro Echeverry
Member
691 posts
Joined: June 2006
Offline
 
This is now even more strange, I created a new blank hip file and copy pasted all my nodes from the file were PDG wasn't using all my threads and now everything is working as it should.

Why is this behavior happening? I'll continue to debug!

Attachments:
PDG_threads_newHIP.png (225.0 KB)

Feel The Knowledge, Kiss The Goat!!!
http://www.linkedin.com/in/alejandroecheverry [linkedin.com]
http://vimeo.com/lordpazuzu/videos [vimeo.com]
 Report
User Avatar
Alejandro Echeverry
Member
691 posts
Joined: June 2006
Offline
 
I think I found a fix to this strange behavior. Its all about to delete the pdg_temp folder and voila, all back to normal!

Maybe there is some corruption on pdg_temp files sometimes (Black Magic).

Cheers!!

Edited by Alejandro Echeverry - June 21, 2019 13:57:45

Attachments:
PDG_all Threads.png (166.5 KB)

Feel The Knowledge, Kiss The Goat!!!
http://www.linkedin.com/in/alejandroecheverry [linkedin.com]
http://vimeo.com/lordpazuzu/videos [vimeo.com]
 Report
  • Quick Links