Hello ,
I can search in the doc but I can't find it or I have to search wrong, but how do you launch a Karma rendering in Hqueue, it may not be possible yet?
Render farm and karma
5519 12 6- bgood
- Member
- 141 posts
- Joined: Sept. 2013
- Offline
- mzigaib
- Member
- 948 posts
- Joined: April 2008
- Offline
- BradThompson
- Member
- 64 posts
- Joined: March 2017
- Offline
In H19, I've gotten it to work using the technique in this video: https://www.youtube.com/watch?v=kNrW9SYuxkk [www.youtube.com] . In summary, drop a fetch node in /OUT context, point the fetch to the render rop inside the /STAGE karma render node, then connect the fetch node to an Hqueue Render node.
This works on machines with full Houdini licenses, but not on the rest of the farm that only has renderer licenses. I believe it's because this setup calls hython to generate the USD file for Husk and that consumes a license. I'm assuming that what we really need is something similar to IFD workflow, where we use a USD render rop on a licensed machine to generate the USD files, then ask husk/hqueue to render the USD directly.
Am I on the right track? Is there a way to do this without having to roll my own submission tool?
This works on machines with full Houdini licenses, but not on the rest of the farm that only has renderer licenses. I believe it's because this setup calls hython to generate the USD file for Husk and that consumes a license. I'm assuming that what we really need is something similar to IFD workflow, where we use a USD render rop on a licensed machine to generate the USD files, then ask husk/hqueue to render the USD directly.
Am I on the right track? Is there a way to do this without having to roll my own submission tool?
- OneBigTree
- Member
- 378 posts
- Joined: Nov. 2010
- Offline
Is there any new information on this?
I have tried using the Fetch in 19.5 but it fails immediately in HQueue.
Same with the new Karma ROP in the Out context. Mantra works fine. Some of the output log below:
.........................
Traceback (most recent call last):
File "/stage/usdrender_rop1/outputprocessors", line 2, in expression
File "D:\SIDEEF~1/HOUDIN~1.303/houdini/python3.9libs\loputils.py", line 11, in <module>
from pxr import Gf, Vt, Tf, Kind, Sdf, Sdr, Pcp
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Gf\__init__.py", line 29, in <module>
from pxr import Tf
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 163, in <module>
PreparePythonModule()
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 87, in PreparePythonModule
with WindowsImportWrapper():
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\contextlib.py", line 119, in __enter__
return next(self.gen)
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 47, in WindowsImportWrapper
dirs.append(os.add_dll_directory(path))
File "D:\SIDEEF~1/HOUDIN~1.303/python39\lib\os.py", line 1111, in add_dll_directory
cookie = nt._add_dll_directory(path)
OSError: The parameter is incorrect: '.\\python39'
PROGRESS: 0%
Generating Output File: R:/nettest_karma2/nettest_karma2.karma1.0016.exr
Error running Python menu script in parameter,
Output Processors (/out/karma1/lopnet/rop_usdrender/outputprocessors):
Traceback (most recent call last):
File "/out/karma1/lopnet/rop_usdrender/outputprocessors", line 2, in expression
File "D:\SIDEEF~1/HOUDIN~1.303/houdini/python3.9libs\loputils.py", line 11, in <module>
from pxr import Gf, Vt, Tf, Kind, Sdf, Sdr, Pcp
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Gf\__init__.py", line 29, in <module>
from pxr import Tf
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 163, in <module>
PreparePythonModule()
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 87, in PreparePythonModule
with WindowsImportWrapper():
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\contextlib.py", line 119, in __enter__
return next(self.gen)
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 47, in WindowsImportWrapper
dirs.append(os.add_dll_directory(path))
File "D:\SIDEEF~1/HOUDIN~1.303/python39\lib\os.py", line 1111, in add_dll_directory
cookie = nt._add_dll_directory(path)
OSError: The parameter is incorrect: '.\\python39'
------------------------------------------------------------------
I have tried using the Fetch in 19.5 but it fails immediately in HQueue.
Same with the new Karma ROP in the Out context. Mantra works fine. Some of the output log below:
.........................
Traceback (most recent call last):
File "/stage/usdrender_rop1/outputprocessors", line 2, in expression
File "D:\SIDEEF~1/HOUDIN~1.303/houdini/python3.9libs\loputils.py", line 11, in <module>
from pxr import Gf, Vt, Tf, Kind, Sdf, Sdr, Pcp
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Gf\__init__.py", line 29, in <module>
from pxr import Tf
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 163, in <module>
PreparePythonModule()
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 87, in PreparePythonModule
with WindowsImportWrapper():
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\contextlib.py", line 119, in __enter__
return next(self.gen)
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 47, in WindowsImportWrapper
dirs.append(os.add_dll_directory(path))
File "D:\SIDEEF~1/HOUDIN~1.303/python39\lib\os.py", line 1111, in add_dll_directory
cookie = nt._add_dll_directory(path)
OSError: The parameter is incorrect: '.\\python39'
PROGRESS: 0%
Generating Output File: R:/nettest_karma2/nettest_karma2.karma1.0016.exr
Error running Python menu script in parameter,
Output Processors (/out/karma1/lopnet/rop_usdrender/outputprocessors):
Traceback (most recent call last):
File "/out/karma1/lopnet/rop_usdrender/outputprocessors", line 2, in expression
File "D:\SIDEEF~1/HOUDIN~1.303/houdini/python3.9libs\loputils.py", line 11, in <module>
from pxr import Gf, Vt, Tf, Kind, Sdf, Sdr, Pcp
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Gf\__init__.py", line 29, in <module>
from pxr import Tf
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 163, in <module>
PreparePythonModule()
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 87, in PreparePythonModule
with WindowsImportWrapper():
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\contextlib.py", line 119, in __enter__
return next(self.gen)
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 47, in WindowsImportWrapper
dirs.append(os.add_dll_directory(path))
File "D:\SIDEEF~1/HOUDIN~1.303/python39\lib\os.py", line 1111, in add_dll_directory
cookie = nt._add_dll_directory(path)
OSError: The parameter is incorrect: '.\\python39'
------------------------------------------------------------------
- BradThompson
- Member
- 64 posts
- Joined: March 2017
- Offline
- OneBigTree
- Member
- 378 posts
- Joined: Nov. 2010
- Offline
BradThompsonIt's the full install and it is not just one node failing, it is the whole job, It doesn't even get assigned to any nodes after the job prep.
Does the render node that's failing have a full houdini or houdini engine license? Not just a karma license?
Did you get it to work with 19.5 (303)?
Edited by OneBigTree - Aug. 31, 2022 09:59:56
- BradThompson
- Member
- 64 posts
- Joined: March 2017
- Offline
OneBigTree
Did you get it to work with 19.5 (303)?
I ended up going another direction and wrote a tool that submits USD render jobs to Husk through hqueue. That way, only karma licenses are needed on the farm. I posted an early version of it here: https://www.sidefx.com/forum/topic/82771/?page=1#post-355883 [www.sidefx.com] . I'd love to post my latest version but I no longer have access to it. I didn't try the fetch node approach in 19.5.
- OneBigTree
- Member
- 378 posts
- Joined: Nov. 2010
- Offline
BradThompsonI see, thanks for the link. I hope SideFX adds this or something similar to the arsenal so people who do not want or can not build their own tools can use Karma with HQueue more easily, now that it is officially production ready.OneBigTree
Did you get it to work with 19.5 (303)?
I ended up going another direction and wrote a tool that submits USD render jobs to Husk through hqueue. That way, only karma licenses are needed on the farm. I posted an early version of it here: https://www.sidefx.com/forum/topic/82771/?page=1#post-355883 [www.sidefx.com] . I'd love to post my latest version but I no longer have access to it. I didn't try the fetch node approach in 19.5.
- BradThompson
- Member
- 64 posts
- Joined: March 2017
- Offline
- OneBigTree
- Member
- 378 posts
- Joined: Nov. 2010
- Offline
BradThompsonAnd here I was so happy that I got HQueue to work for the first time because they replaced the client service with a startup command....
I agree 100% and I was surprised that a husk submission tool was not included. I haven't tried it myself, but Deadline Render Manager from Thinkbox/Amazon is now free and has a Houdini integration. If I were tackling this problem today, I would investigate that approach.
- 0chumpy
- Member
- 11 posts
- Joined: May 2014
- Offline
OneBigTreesame here, I don't have a clue on how to fix this.
Is there any new information on this?
I have tried using the Fetch in 19.5 but it fails immediately in HQueue.
Same with the new Karma ROP in the Out context. Mantra works fine. Some of the output log below:
.........................
Traceback (most recent call last):
File "/stage/usdrender_rop1/outputprocessors", line 2, in expression
File "D:\SIDEEF~1/HOUDIN~1.303/houdini/python3.9libs\loputils.py", line 11, in <module>
from pxr import Gf, Vt, Tf, Kind, Sdf, Sdr, Pcp
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Gf\__init__.py", line 29, in <module>
from pxr import Tf
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 163, in <module>
PreparePythonModule()
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 87, in PreparePythonModule
with WindowsImportWrapper():
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\contextlib.py", line 119, in __enter__
return next(self.gen)
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 47, in WindowsImportWrapper
dirs.append(os.add_dll_directory(path))
File "D:\SIDEEF~1/HOUDIN~1.303/python39\lib\os.py", line 1111, in add_dll_directory
cookie = nt._add_dll_directory(path)
OSError: The parameter is incorrect: '.\\python39'
PROGRESS: 0%
Generating Output File: R:/nettest_karma2/nettest_karma2.karma1.0016.exr
Error running Python menu script in parameter,
Output Processors (/out/karma1/lopnet/rop_usdrender/outputprocessors):
Traceback (most recent call last):
File "/out/karma1/lopnet/rop_usdrender/outputprocessors", line 2, in expression
File "D:\SIDEEF~1/HOUDIN~1.303/houdini/python3.9libs\loputils.py", line 11, in <module>
from pxr import Gf, Vt, Tf, Kind, Sdf, Sdr, Pcp
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Gf\__init__.py", line 29, in <module>
from pxr import Tf
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 163, in <module>
PreparePythonModule()
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 87, in PreparePythonModule
with WindowsImportWrapper():
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\contextlib.py", line 119, in __enter__
return next(self.gen)
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 47, in WindowsImportWrapper
dirs.append(os.add_dll_directory(path))
File "D:\SIDEEF~1/HOUDIN~1.303/python39\lib\os.py", line 1111, in add_dll_directory
cookie = nt._add_dll_directory(path)
OSError: The parameter is incorrect: '.\\python39'
------------------------------------------------------------------
- NMVHS
- Member
- 21 posts
- Joined: May 2015
- Offline
OneBigTreeI got same errors, how did you fix this?
Is there any new information on this?
I have tried using the Fetch in 19.5 but it fails immediately in HQueue.
Same with the new Karma ROP in the Out context. Mantra works fine. Some of the output log below:
.........................
Traceback (most recent call last):
File "/stage/usdrender_rop1/outputprocessors", line 2, in expression
File "D:\SIDEEF~1/HOUDIN~1.303/houdini/python3.9libs\loputils.py", line 11, in <module>
from pxr import Gf, Vt, Tf, Kind, Sdf, Sdr, Pcp
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Gf\__init__.py", line 29, in <module>
from pxr import Tf
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 163, in <module>
PreparePythonModule()
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 87, in PreparePythonModule
with WindowsImportWrapper():
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\contextlib.py", line 119, in __enter__
return next(self.gen)
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 47, in WindowsImportWrapper
dirs.append(os.add_dll_directory(path))
File "D:\SIDEEF~1/HOUDIN~1.303/python39\lib\os.py", line 1111, in add_dll_directory
cookie = nt._add_dll_directory(path)
OSError: The parameter is incorrect: '.\\python39'
PROGRESS: 0%
Generating Output File: R:/nettest_karma2/nettest_karma2.karma1.0016.exr
Error running Python menu script in parameter,
Output Processors (/out/karma1/lopnet/rop_usdrender/outputprocessors):
Traceback (most recent call last):
File "/out/karma1/lopnet/rop_usdrender/outputprocessors", line 2, in expression
File "D:\SIDEEF~1/HOUDIN~1.303/houdini/python3.9libs\loputils.py", line 11, in <module>
from pxr import Gf, Vt, Tf, Kind, Sdf, Sdr, Pcp
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Gf\__init__.py", line 29, in <module>
from pxr import Tf
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 163, in <module>
PreparePythonModule()
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 87, in PreparePythonModule
with WindowsImportWrapper():
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\contextlib.py", line 119, in __enter__
return next(self.gen)
File "D:\SIDEEF~1\HOUDIN~1.303\python39\lib\site-packages\pxr\Tf\__init__.py", line 47, in WindowsImportWrapper
dirs.append(os.add_dll_directory(path))
File "D:\SIDEEF~1/HOUDIN~1.303/python39\lib\os.py", line 1111, in add_dll_directory
cookie = nt._add_dll_directory(path)
OSError: The parameter is incorrect: '.\\python39'
------------------------------------------------------------------
- OneBigTree
- Member
- 378 posts
- Joined: Nov. 2010
- Offline
-
- Quick Links