Projects workflow (and python module resolution paths)

   674   2   0
User Avatar
Member
151 posts
Joined: 6月 2019
オフライン
I want to organize bunch of files (hips, resources and python scripts) as a project, but currently not quite understand how to do that, what is the current workflow.

Mainly I was struggling with python modules, I want just import mymodulefor project related modules.
I have a feeling that it used to work with $HIP and $JOB variables, essentially anything in $HIP|$JOB/python3.11libs was discoverable by interpreter, but right now it's not.

Right now I see basically two ways:
  1. Built-in project management with $JOB. For some reason it's not giving me python modules resolution. Not sure if it should, but it's something that I expect for "project". Also I completely lost with $JOB management and how and where it's stored
  2. Creating a package which I really like, but setting up packages for every project and load them is cumbersome

Both ways are not ideal, so probably I'm missing something
I even came up with my own poor man project management solution that I'll try to explain next
User Avatar
Member
151 posts
Joined: 6月 2019
オフライン
So currently what works for me is setting up project based on packages.
In theory I think it's how it's supposed to work and $JOB workflow should be deprecated in favor of packages, but it needs to be supported somehow by houdini itself.

The whole idea is based on relatively small script in ready.py
import pathlib

import hdefereval
import hou

current_hproject: pathlib.Path | None = None
project_was_loaded: bool = False


def find_hpackage(path: pathlib.Path) -> pathlib.Path | None:
    path = path.resolve()

    for current in [path] + list(path.parents):
        hpackage = current / "hpackage.json"
        if hpackage.is_file():
            return hpackage
    return None


def unload_hproject() -> None:
    global current_hproject

    if current_hproject:
        hou.logging.log(hou.logging.LogEntry(f"Unload package: {current_hproject} for project"))
        hou.ui.unloadPackage(current_hproject.as_posix())
    current_hproject = None
    hou.unsetenv("HPROJECT_ROOT")


def try_load_hproject(hip_path: pathlib.Path) -> None:
    global current_hproject
    global project_was_loaded
    hpackage = find_hpackage(hip_path)

    if hpackage:
        project_was_loaded = True
        hpackage_file = hpackage.resolve()
        current_hproject = hpackage_file
        hou.ui.loadPackage(hpackage_file.as_posix())
        hou.logging.log(hou.logging.LogEntry(f"Load package: {current_hproject} for project"))
    else:
        unload_hproject()


def hipfile_callback(event: hou.hipFileEventType):
    global current_hproject
    global project_was_loaded

    match event:
        case hou.hipFileEventType.BeforeLoad | hou.hipFileEventType.BeforeSave:
            hip_path = pathlib.Path(hou.hipFile.path())
            try_load_hproject(hip_path)

        case hou.hipFileEventType.AfterLoad | hou.hipFileEventType.AfterSave:
            project_was_loaded = False
            hip_path = pathlib.Path(hou.hipFile.path())
            if current_hproject:
                hdefereval.executeDeferred(
                    lambda: hou.qt.mainWindow().setWindowTitle(
                        f"[PROJECT: {current_hproject}]: {hip_path}"
                    )
                )

        case hou.hipFileEventType.BeforeClear:
            if not project_was_loaded:
                unload_hproject()

        case _:
            return


if hou.isUIAvailable():
    hou.hipFile.addEventCallback(hipfile_callback)

How it works:

There is a file hpackage.json (not to be conflicted with nodejs, unity or whatever) that should be placed in project root directory.

The package actually could be identical for any project
{
  "hpath": "$HOUDINI_PACKAGE_PATH",
  "env": [
    {
      "HPROJECT_ROOT": "$HOUDINI_PACKAGE_PATH"
    }
  ]
}

When I load a hip file I search for hpackage.json in any parent directory and if I find one I load the package dynamically with hou.ui.loadPackage. I'm not resolving if there are more than one hpackage, currently I just work with the simplest way possible without trying to implement anything like workspaces or monorepo.

I also load package on saving and unload it on new file or I couldn't find one in parent directories. Additionally I customize window title so I can see if I'm working "inside" project.

This is working great for me so far and even though it's mostly for UI session it's really easy to load the same "project" in hython because I rely on packages API.

However it's still clunky. Dynamic loading and especially unloading packages is not working as expected. For example once environment variables are set they are set for the whole session. Unloading packages are not unsetting variables, so I just came up with HPROJECT_ROOT that I unset manually.

I think ideally something like this should be provided and supported by houdini. I'd argue even something like "Project Explorer", a pane with current project files, essentially just a file explorer inside project root, could be really useful. Houdini is already like an IDE, it just lacks good ux with grouping resources as a project on disk.
User Avatar
Member
151 posts
Joined: 6月 2019
オフライン
Apparently if package environment variable set via append or prepend when package unloads it also removes what was added. That makes sense and actually great, it allows to have per-project PYTHONPATH or PATH so I'm not polluting houdini envs

I guess in addition to append, prepend and replace there should be set as a mark of unique package variable that safe to unset on unloading


I think dynamic packages are worth investing to. Per project shelves, desktops, anything, ideally should be possible with dynamic loading and unloading. The setup above already supports per project vex, opencl and basically any settings from HOUDINI_PATH but many are missing due to how packages load after startup. With some effort I think even renderers could be set via project configuration, not by scripts.
  • Quick Links