Custom Scheduler built in 17.5 not appearing in 18 [RoyalRender]

   2295   8   2
User Avatar
Member
30 posts
Joined: 5月 2017
Offline
Hello,

I am struggling to get a setup running built by a colleague to make interaction possible between TOPs and RoyalRender. As he got busy with other stuff he has abandoned support for it and I can't share his work either. It seemed to be working decently (with some unresolved theoretical issues on RR-side) under H17.5 where it was developed but unfortunately I can't get it to run again under H18.

His install instructions are the following:

1) copy royalrender.py into user\Documents\houdini17.5\pdg\types
(In our case, we put it on our network drive and directed $HOUDINI_PDG_PATH to it.
2) include path to package royalr in your PYTHONPATH


Both of that was working simply with edits to our launcher-Bat previously, but replicating the same setup with H18 did not turn up the royalrender-PDG-scheduler that was availible as a node in H17.5.

EDIT: Changed Title for clarity.

Now my suspicion is that the HOUDINI_PDG_PATH changed of some sorts and thus wouldn't read the file anymore. Is there some simple fix that would allow us to re-use this asset from before?

Thanks for your help,
Martin
Edited by martin-egger - 2020年7月10日 06:26:17
- Martin Egger
Student at Filmakademie BW
& Freelance FX TD
User Avatar
Member
603 posts
Joined: 9月 2016
Offline
That looks correct. Try setting $HOUDINI_PDG_TYPE_ERROR to 2 or 1 to get messages about type loading.
Edited by chrisgreb - 2020年7月10日 13:41:51
User Avatar
Member
30 posts
Joined: 5月 2017
Offline
Thank you Chris! I will try that and post my results as soon as I get to it next week.
- Martin Egger
Student at Filmakademie BW
& Freelance FX TD
User Avatar
Member
30 posts
Joined: 5月 2017
Offline
Alright, here is the log from my console, doesn't even list royalrender.py in any way unfortunately. But some other weird stuff, maybe that tells you something.

PDG Type Registry: Loading types from ‘mapper.py’
PDG Type Registry: Found module at ‘C:/PROGRA~1/SIDEEF~1/HOUDIN~1.523/houdini/pdg/types\mapper.py’
PDG Type Registry: Loading types from ‘partitioner.py’
PDG Type Registry: Found module at ‘C:/PROGRA~1/SIDEEF~1/HOUDIN~1.523/houdini/pdg/types\partitioner.py’
PDG Type Registry: Loading types from ‘processor.py’
PDG Type Registry: Found module at ‘C:/PROGRA~1/SIDEEF~1/HOUDIN~1.523/houdini/pdg/types\processor.py’
PDG Type Registry: Loading types from ‘houdini’
PDG Type Registry: Found module at ‘C:/PROGRA~1/SIDEEF~1/HOUDIN~1.523/houdini/pdg/types\houdini’
PDG Type Registry: Loading types from ‘partitioners’
PDG Type Registry: Found module at ‘C:/PROGRA~1/SIDEEF~1/HOUDIN~1.523/houdini/pdg/types\partitioners’
PDG Type Registry: Loading types from ‘schedulers’
PDG Type Registry: Found module at ‘C:/PROGRA~1/SIDEEF~1/HOUDIN~1.523/houdini/pdg/types\schedulers’
PDG Type Registry: Loading types from ‘utils’
PDG Type Registry: Found module at ‘C:/PROGRA~1/SIDEEF~1/HOUDIN~1.523/houdini/pdg/types\utils’
PDG Type Registry: Failed to import duplicate module ‘houdini’ which was previously imported from ‘C:/PROGRA~1/SIDEEF~1/HOUDIN~1.523/houdini/pdg/types\houdini’. Module will be skipped.
PDG Type Registry: Failed to import duplicate module ‘partitioners’ which was previously imported from ‘C:/PROGRA~1/SIDEEF~1/HOUDIN~1.523/houdini/pdg/types\partitioners’. Module will be skipped.
PDG Type Registry: Failed to import duplicate module ‘schedulers’ which was previously imported from ‘C:/PROGRA~1/SIDEEF~1/HOUDIN~1.523/houdini/pdg/types\schedulers’. Module will be skipped.
PDG Type Registry: Failed to import duplicate module ‘utils’ which was previously imported from ‘C:/PROGRA~1/SIDEEF~1/HOUDIN~1.523/houdini/pdg/types\utils’. Module will be skipped.
Could not initialize the help server:
Traceback (most recent call last):
File “<stdin>”, line 1, in <module>
File “C:/PROGRA~1/SIDEEF~1/HOUDIN~1.523/houdini/python2.7libs\houdinihelp\api.py”, line 136, in initialize
setup_asset_indexing()
File “C:/PROGRA~1/SIDEEF~1/HOUDIN~1.523/houdini/python2.7libs\houdinihelp\api.py”, line 526, in setup_asset_indexing
shutil.rmtree(dynamic_dir)
File “C:\PROGRA~1\SIDEEF~1\HOUDIN~1.523\python27\lib\shutil.py”, line 266, in rmtree
onerror(os.remove, fullname, sys.exc_info())
File “C:\PROGRA~1\SIDEEF~1\HOUDIN~1.523\python27\lib\shutil.py”, line 264, in rmtree
os.remove(fullname)
WindowsError: The process cannot access the file because it is being used by another process: ‘C:\\Users\\MARTIN~1\\DOCUME~1\\houdini18.0\\config\\Help\\index2.7\\MAIN_5wb9htv6b4hokjzf.seg’
- Martin Egger
Student at Filmakademie BW
& Freelance FX TD
User Avatar
Member
603 posts
Joined: 9月 2016
Offline
> doesn't even list royalrender.py in any way unfortunately.

Double check that you have “pdg/types/royalrender.py” on your HOUDINI_PATH. You can show it in houdini with python:

print(hou.houdiniPath())
print(hou.findDirectories('pdg/types'))

Looks like you have duplicate entries in your HOUDINI_PATH, which is causing the “Failed to import duplicate..” message.

> WindowsError: The process cannot access the file because it is being used by another process:

This is a spurious error you can ignore.
User Avatar
Member
30 posts
Joined: 5月 2017
Offline
>>> print “\n”.join(hou.houdiniPath())
P:/00_pipeline/03_various/02_user_dir/martin
P:/00_pipeline/03_various/01_plugins/SideFXLabs
P:/00_pipeline/03_various/01_plugins/rrSubmitter
P:/00_pipeline/02_render/02_redshift/18.0.460/Redshift/Plugins/Houdini/18.0.460
P:/00_pipeline/03_various/01_plugins/qLib
P:/00_pipeline/03_various/01_plugins/pdg-mutagen-toolset
P:/00_pipeline/03_various/01_plugins/mops
P:/00_pipeline/03_various/01_plugins/ffmpeg
P:/05_library/04_megascans/support/plugins/houdini/4.2/MSLiveLink
P:/00_pipeline/03_various/01_plugins/lbt/pylib/pyside
P:/00_pipeline/03_various/01_plugins/lbt/pylib/pynput
P:/00_pipeline/03_various/01_plugins/lbt
P:/00_pipeline/03_various/01_plugins/jdb_toolkit
P:/00_pipeline/03_various/01_plugins/dmnkTools
P:/00_pipeline/03_various/01_plugins/bg_render
P:/05_library/04_megascans/support/plugins/houdini/4.4/MSLiveLink
P:/00_pipeline/01_houdini
C:/MNP_nextcloud/00_pipeline/01_houdini
C:/Users/MARTIN~1/DOCUME~1/houdini18.0
C:/PROGRA~1/SIDEEF~1/HOUDIN~1.460/houdini
C:/PROGRA~1/SIDEEF~1/HOUDIN~1.460/bin
C:/MNP_nextcloud/00_pipeline/01_houdini
C:/Users/MARTIN~1/DOCUME~1/houdini18.0
C:/PROGRA~1/SIDEEF~1/HOUDIN~1.460/houdini
C:/PROGRA~1/SIDEEF~1/HOUDIN~1.460/bin


>>> print(“\n”.join(hou.findDirectories('pdg/types')))
C:/Users/MARTIN~1/DOCUME~1/houdini18.0/pdg/types - - - - - - - (WHERE I PUT IT)
C:/PROGRA~1/SIDEEF~1/HOUDIN~1.460/houdini/pdg/types
C:/Users/MARTIN~1/DOCUME~1/houdini18.0/pdg/types
C:/PROGRA~1/SIDEEF~1/HOUDIN~1.460/houdini/pdg/types

(Also, apparently Houdini read the type description as a royalrender.pyc was created next to the royalrender.py in my preferences folder)

It's true that we have duplicate entries in our path, since our pipeline setup is a bit sloppy, but I doubt this would cause something to not work apart from throwing aforementioned warnings.
In my case, I have
Also, this shows that the pythonpath is correctly being loaded with the “royalr” module in it.
>>> print(hou.hscript(“$PYTHONPATH”))
('', ‘Unknown command: ;P:/00_pipeline/03_various/01_plugins/pdg_rr_scheduler_python;&\n\r’)
>>> import royalr
>>>
(import worked fine)

Anything that might have changed in how Houdini reads the types? Or in the requirements for such to be registered? I just know my colleague took the other schedulers (Deadline / Hqueue) as starting points, but from 17.5.

I also just tried on a clean launch of Houdini (Without all the other stuff in our pipeline) but just the RR-Scheduler installed to my local houdini (like intended by the author) and it still doesn't show up in H18.0.460 or H18.0.523.
Edited by martin-egger - 2020年7月14日 07:41:57
- Martin Egger
Student at Filmakademie BW
& Freelance FX TD
User Avatar
スタッフ
585 posts
Joined: 5月 2014
Offline
Is there code in your custom scheduler's .py file so that it registers itself with PDG? It needs to have a registerTypes function in it that registers the scheduler. For example, for the built-in local scheduler in $HFS/houdini/pdg/types/schedulers/__init__.py:

import pdg
from . import local

...

def registerTypes(type_registry):
    ...
    type_registry.registerScheduler(local.LocalScheduler,
        label="Local Scheduler")
    ...

That's the same as it was in 17.5, though.
User Avatar
スタッフ
585 posts
Joined: 5月 2014
Offline
In ~/houdini18.0/pdg/types, trying creating a Python file with the following contents:

from pdg import TypeRegistry
from pdg.scheduler import PyScheduler

class EmptyScheduler(PyScheduler):
    def __init__(self, scheduler, name):
        PyScheduler.__init__(self, scheduler, name)

def registerTypes(type_registry):
    type_registry.registerScheduler(EmptyScheduler, label="Empty Scheduler")

Then, launch hython and run the following:

>>> import pdg
>>> pdg.TypeRegistry.types().typeNames(pdg.registeredType.Scheduler)
[u'inprocessscheduler', u'localscheduler', u'hqueuescheduler', u'pythonscheduler', u'emptyscheduler']

You should see the “emptyscheduler” in the list if the custom scheduler is being loaded properly.
User Avatar
Member
30 posts
Joined: 5月 2017
Offline
Hello Taylor, thank you so much for taking this on.

Interestingly this seemed to be the first issue. While $HOUDINI_PDG_PATH evaluated to the directory where royalrender.py (and the test-emptyscheduler) lied, the emptyscheduler did not show up. Only when I moved it to my preferences folder - then the function returned exactly what you printed out.

However, even when moving the royalrender.py to my preferences folder, it still wouldn't show up. And after investigating the code for the type registry function, it seems like it matches the emptyscheduler and that it _should_ work, yet it doesn't appear among the listed.

After checking with my colleague, he gave me the okay to send you the code over. I thought that's probably easier if you just had a quick scroll through it. So I'll send you a private message on the forums right now, I hope that's okay.

Thanks and kind regards,
Martin
- Martin Egger
Student at Filmakademie BW
& Freelance FX TD
  • Quick Links