Hi everyone,
I'm having a bit of trouble working out the correct setup to utilize all threads on all machines through deadline for something like outputting from a ROPgeo to disk.
I can distribute a job across these machines, however the concurrent tasks are limited to 16 in deadline scheduler and this does not take advantage of all cores.
Does anyone have an example of a setup that would achieve this? I'm running H18.
Many thanks in advance!
Deadline - Use all threads on all machines
2792 7 3- starion83
- Member
- 30 posts
- Joined: July 2016
- Offline
- starion83
- Member
- 30 posts
- Joined: July 2016
- Offline
- chrisgreb
- Member
- 603 posts
- Joined: Sept. 2016
- Offline
- starion83
- Member
- 30 posts
- Joined: July 2016
- Offline
Hey Chris, thanks for your reply!
I think because this would mean I'd have to run a machine for every thread, it's not an ideal solution is I'd have to start up many clients. Ideally I'd like to just run each client with the same capacity as I would if I was running the PDG locally where it can max out the CPU.
I think because this would mean I'd have to run a machine for every thread, it's not an ideal solution is I'd have to start up many clients. Ideally I'd like to just run each client with the same capacity as I would if I was running the PDG locally where it can max out the CPU.
- chrisgreb
- Member
- 603 posts
- Joined: Sept. 2016
- Offline
- martinkindl83
- Member
- 260 posts
- Joined: Nov. 2014
- Offline
- chrisgreb
- Member
- 603 posts
- Joined: Sept. 2016
- Offline
I see, you would need 4 workers to saturate a 64 core machine. This sounds like something that would need to be taken up with Thinkbox.
One work around may be to break up the graph into multiple graphs which use Local Scheduler. Then a top-level graph would use Top Fetch nodes with Deadline scheduler to process those sub graphs in parallel, so each smaller graph would consume all the resources of the machine they were assigned to. However, this would no doubt obfuscate your graph.
One work around may be to break up the graph into multiple graphs which use Local Scheduler. Then a top-level graph would use Top Fetch nodes with Deadline scheduler to process those sub graphs in parallel, so each smaller graph would consume all the resources of the machine they were assigned to. However, this would no doubt obfuscate your graph.
- starion83
- Member
- 30 posts
- Joined: July 2016
- Offline
chrisgreb
I see, you would need 4 workers to saturate a 64 core machine. This sounds like something that would need to be taken up with Thinkbox.
One work around may be to break up the graph into multiple graphs which use Local Scheduler. Then a top-level graph would use Top Fetch nodes with Deadline scheduler to process those sub graphs in parallel, so each smaller graph would consume all the resources of the machine they were assigned to. However, this would no doubt obfuscate your graph.
Thanks for the reply, and sorry for the delay with mine!
Once I get a chance I'll look into this again.
-
- Quick Links