Found 424 posts.
Search results Show results as topic list.
PDG/TOPs » PDG to Deadline Connection
- chrisgreb
- 603 posts
- Offline
With that path mapping, try setting the Python Job Parm on Deadline Scheduler TOP to \$PYTHON instead.
Edited by chrisgreb - Dec. 15, 2020 17:38:27
PDG/TOPs » Many Work Items overloading PDG
- chrisgreb
- 603 posts
- Offline
PDG itself can scale to orders of magnitude larger graphs, however there could be something in Houdini's UI or in the jobs themselves that are causing such lag.
Is this only happening during the cook? If so maybe Local Scheduler is running too many concurrent processes for your system to handle?
Is it possible to attach the hip file here or on a support ticket?
Is this only happening during the cook? If so maybe Local Scheduler is running too many concurrent processes for your system to handle?
Is it possible to attach the hip file here or on a support ticket?
PDG/TOPs » Using command chain with existing xmlrpc server
- chrisgreb
- 603 posts
- Offline
There are particular RPC functions that 'sendcommand' expects to be available to execute the script. You should base your server on the implementation in $HHP/pdgjob/genericrpc.py, which is what is used by the other command chains.
For example you can run
To start a python command chain server that can be connected to.
For example you can run
python.exe $HHP/pdgjob/genericrpc.py --start --host 0.0.0.0 --port 59902 --log c:/temp/myserver.log
To start a python command chain server that can be connected to.
Edited by chrisgreb - Dec. 13, 2020 10:32:35
PDG/TOPs » Scheduler "Total Slots" for Single CPU - Does it Matter?
- chrisgreb
- 603 posts
- Offline
“CPU” here refers to logical core, not physical socket. So for your machine “1/4 of Total CPU Count” works out to limiting the concurrent jobs to 12.
PDG/TOPs » Deadline - Use all threads on all machines
- chrisgreb
- 603 posts
- Offline
I see, you would need 4 workers to saturate a 64 core machine. This sounds like something that would need to be taken up with Thinkbox.
One work around may be to break up the graph into multiple graphs which use Local Scheduler. Then a top-level graph would use Top Fetch nodes with Deadline scheduler to process those sub graphs in parallel, so each smaller graph would consume all the resources of the machine they were assigned to. However, this would no doubt obfuscate your graph.
One work around may be to break up the graph into multiple graphs which use Local Scheduler. Then a top-level graph would use Top Fetch nodes with Deadline scheduler to process those sub graphs in parallel, so each smaller graph would consume all the resources of the machine they were assigned to. However, this would no doubt obfuscate your graph.
PDG/TOPs » Deadline - Use all threads on all machines
- chrisgreb
- 603 posts
- Offline
Did you try changing the Job Options/ Concurrent Tasks parm on the Deadline Scheduler TOP?
PDG/TOPs » Open another application with tops
- chrisgreb
- 603 posts
- Offline
There's no built-in node to do that, but you may be able to execute a Unity command line with Generic Generator TOP to focus it.
PDG/TOPs » Deadline - Use all threads on all machines
- chrisgreb
- 603 posts
- Offline
You could try running more clients on each machine?
Edited by chrisgreb - Nov. 30, 2020 09:20:59
PDG/TOPs » renderman render with tractor
- chrisgreb
- 603 posts
- Offline
Yes, that should work, do other sorts of jobs work?
Here's some general farm scheduling tips:
https://www.sidefx.com/docs/houdini/tops/farm_troubleshooting.html [www.sidefx.com]
Here's some general farm scheduling tips:
https://www.sidefx.com/docs/houdini/tops/farm_troubleshooting.html [www.sidefx.com]
PDG/TOPs » Q: HOM: Equivalent of RMB "Cook selected Node"?
- chrisgreb
- 603 posts
- Offline
Yes, that's right. The PDG node(s) for the TOP node you RMB on are passed to the graph context cook function. The pdg API has the lower level `pdg.GraphContext.cook` to do the same.
PDG/TOPs » export variations of HDA to multiple FBX files
- chrisgreb
- 603 posts
- Offline
Are the bgeo files saved by the hdaprocessor correct?
In the ropfetch case /obj/pdg_fbx_test/radx is not being set. There are two ways to do it. You can set an expression on it @radiusX , or use the “Overwrite Target Parameter..” on wedge1.
In the ropfetch case /obj/pdg_fbx_test/radx is not being set. There are two ways to do it. You can set an expression on it @radiusX , or use the “Overwrite Target Parameter..” on wedge1.
PDG/TOPs » Hda processor in UE4
- chrisgreb
- 603 posts
- Offline
PDG/TOPs » Hda processor in UE4
- chrisgreb
- 603 posts
- Offline
If you mean the houdini.env file, that should also work with the UE4 plugin. For example you can define a PROJ_HDAS var in your houdini.env and use it in the hdaprocessor parm as $PROJ_HDAS/asset.hda
PDG/TOPs » Deserialize a full graph
- chrisgreb
- 603 posts
- Offline
OstapOk, if this is something you're interested in, please submit an RFE.
Option #2 sounds cool!
PDG/TOPs » Deserialize a full graph
- chrisgreb
- 603 posts
- Offline
Ostap
Scheduler creates a state server - do you think it is the right way? Maybe better to use some Event Handler for this purpose if it's possible?
I think it would depend on the particular workflow you have in mind. Currently we only create a state server optionally based on the scheduler toggle parm.
…really slow. To deserialize a PDG graph you have to open the hip file first and sometimes the hip file could be huge. It can take minutes to load all those things.
There's two ways this could be avoided:
1. Keep hython running when the TOP cook finishes, therefore the state server will be available until the process is explicitly killed.
2. Extend the default state server to use a persistent back-end which can be used to bootstrap a remotegraph TOP in PilotPDG. I am guessing that this would be faster because you don't need to load a hip file.
Also, it is not convenient to change a visualisation workflow which depends on a job state (running, done or failed).
Any idea for a better/faster solution?
#2 could be a single workflow if there was some kind of central server to connect the client PilotPDG to the correct back end data. However, we haven't implemented a persistent back end, so it's theoretical at this point
PDG/TOPs » Deserialize a full graph
- chrisgreb
- 603 posts
- Offline
Ostap
Workitem dependencies visualization if the third person (wrangler) is looking at the farm job of interactive PDG submission?
The data layer server can serve multiple clients. They will just need to connect from PilotPDG or some custom client.
It should be possible to attach a 3rd
Workitem dependencies visualization if the graph has been submitted as a job (with Data Server) and the job is not alive anymore (completed, killed ..)?
At the end of the cook the script serializes the task graph state to a python file to disk. This could be used to load the state of the graph as it was when the cook ended.
Is it possible to start automatically Start Server for any PDG session?Yes, it can be started through the pdgd API.
PDG/TOPs » PDG best way to check JOB/work items times
- chrisgreb
- 603 posts
- Offline
That information is not currently stored on the work item, but we do have an RFE to start doing that. However you may want to take a look at the “Performance Monitor” for the TOP cook if you haven't already.
PDG/TOPs » Wedge simulation across multiple machines
- chrisgreb
- 603 posts
- Offline
Yes, there are two options. You can add a ‘Tag’ to the job so a machine can only run one of them, or you can increase the ‘CPUs per Job’ so that each job consumes more ‘CPU’ slots, thus running fewer jobs at once. Both of these are offered as Job Parms:
https://www.sidefx.com/docs/houdini/nodes/top/hqueuescheduler.html#job-parms [www.sidefx.com]
Which can be set on the ROP Geometry node by installing the corresponding properties:
https://www.sidefx.com/docs/houdini/tops/schedulers.html#jobparms [www.sidefx.com]
https://www.sidefx.com/docs/houdini/nodes/top/hqueuescheduler.html#job-parms [www.sidefx.com]
Which can be set on the ROP Geometry node by installing the corresponding properties:
https://www.sidefx.com/docs/houdini/tops/schedulers.html#jobparms [www.sidefx.com]
PDG/TOPs » TOP Deadline updates in Houdini 18.0.399 (new MQ)
- chrisgreb
- 603 posts
- Offline
pbowmar
Exact same error with PDG_USE_PDGNET set to 0 on all machines…
It shouldn't be the exact same error, I would expect the RPC to timeout, but the message would not mention PDGNetRPCMessage, because that is specific to the ‘PDGNET’ mode.
In any case, for submit-as-job you need to ensure that the Task Callback Port is opened between farm machines, so that the RPCs can reach PDG.
https://www.sidefx.com/docs/houdini/tops/farm_troubleshooting.html#work-items-fail-to-report-results-due-to-connection-refused-or-time-out [www.sidefx.com]
PDG/TOPs » TOP Deadline updates in Houdini 18.0.399 (new MQ)
- chrisgreb
- 603 posts
- Offline
pbowmar
Hi, found this super useful thread, hitting some issues that maybe unrelated but thought I'd give it a try.
I'm using 18.0.460 Win 10, I have 3 machines, 1x coord and 2x workers, shared Q: drive on all three, Deadline set up and working fine with Python test scripts on all machines.
I'm using Submit Graph As Job as I don't expect or really want interactivity, just execution of the static TOPs graph. Current test is a single ROP Geometry Output. This works fine with localscheduler.
I've set the PDG_USE_PDGNET env var to 1 on coord and client machines just to be safe.
I've tried both Local and Farm modes in the Message Queue section.
When I submit the job (Submit Graph As Job) it shows up correctly in Deadline, but errors on Workers with:2020-10-06 17:51:12: 0: STDOUT: Running Houdini 18.0.460 with PID 4980
2020-10-06 17:51:12: 0: STDOUT: PDG: Loading .hip file q://aws_testGrid18.0.460py27_v01.hip.
2020-10-06 17:51:13: 0: STDOUT: PDG: Setting batch sub index to 0
2020-10-06 17:51:13: 0: STDOUT: PDG_START: noisegrid_ropfetch10;0
2020-10-06 17:51:34: 0: STDOUT: ERROR: The attempted operation failed.
2020-10-06 17:51:34: 0: STDOUT: Error: Python error: Traceback (most recent call last):
2020-10-06 17:51:34: 0: STDOUT: File "<stdin>", line 44, in <module>
2020-10-06 17:51:34: 0: STDOUT: File "q:\pdgtemp\4348\scripts\pdgcmd.py", line 409, in execStartCook
2020-10-06 17:51:34: 0: STDOUT: _invokeRpc(s, "start_cook_batch", item_name, subindex, theJobid)
2020-10-06 17:51:34: 0: STDOUT: File "q:\pdgtemp\4348\scripts\pdgcmd.py", line 258, in _invokeRpc
2020-10-06 17:51:34: 0: STDOUT: return _invokeRpcFn(fn, fn_name, *args)
2020-10-06 17:51:34: 0: STDOUT: File "q:\pdgtemp\4348\scripts\pdgcmd.py", line 262, in _invokeRpcFn
2020-10-06 17:51:34: 0: STDOUT: return _invokePDGNetRpcFn(fn, fn_name, *args)
2020-10-06 17:51:34: 0: STDOUT: File "q:\pdgtemp\4348\scripts\pdgcmd.py", line 276, in _invokePDGNetRpcFn
2020-10-06 17:51:34: 0: STDOUT: if fn.lasterror == 5 or fn.lasterror == 6:
2020-10-06 17:51:34: 0: STDOUT: AttributeError: 'PDGNetRPCMessage' object has no attribute 'lasterror'
Any thoughts?
Cheers,
Peter B
Could you tried setting PDG_USE_PDGNET=0 in your houdini environment? If it's a general networking issue the error should show up in that mode as well.
-
- Quick Links