SOHO: Scripted Output of Houdini Objects

The back-end in Houdini is composed of several different components

SOHO: The API

SOHO provides a very small set of API functions. These functions sit between Houdini and Python scripts and provide a simple object model of the Houdini scene.

The SOHO API consists of 25 functions. These are documented in the HDK_SOHO_API namespace, though they are currently only available in a Python API.

These simple API functions have been wrapped in pure Python modules: soho.py and sohog.py, which can be found in $HFS/houdini/soho. It is the soho.py and sohog.py modules which are typically used.

SOHO Output Drivers

SOHO is invoked through a SOHO output driver. A SOHO output driver is an OTL with some specific parameters created to control the behaviour of the output driver.

A new output driver can be created by starting Houdini, then choosing "File -> New Operator Type" then choosing the "Output Driver Type".

What Happens When The Render Button Is Hit?

When the user hits the render button, the output driver evaluates some parameters (see HDK_SOHO_ROP_Parameters) to determine what should be done.

The string soho_program parameter is used to specify a Python program which gets run.

Before the Python program is invoked, the output driver may open files.

The output driver evaluates int soho_outputmode. This parameter can have three possible values:

In addition, the output driver will open a temporary file to trap errors from the Python program. This is passed through HDK_SOHO_API::getFile(2).

The process to invoke the Python program is given as follows:

  1. Open required file descriptors
  2. Push Python's sys.stdout and sys.stderr objects
  3. Add $HOUDINI_PATH/soho to the Python path
  4. Call program in string soho_program
  5. Restore sys.stdout and sys.stderr objects

SOHO Output Driver Parameters

The output driver looks for certain parameters which it uses to determine how the output driver should operate. As with all operators, the parameters do not have to be visible to the user, they just have to exist.

The parameters are all defined in $HFS/houdini/soho/parameters/SohoIntrinsics.ds, but are not visible in the Houdini interface by default. There are instructions in that file to make the parameters visible.

The easiest way to enable these properties is to set the environment variable HOUDINI_SOHO_DEVELOPER before starting Houdini. For example: 

    % setenv HOUDINI_SOHO_DEVELOPER 1; houdini -foreground
    # export HOUDINI_SOHO_DEVELOPER=1; houdini -foreground

This traverses all nodes in the scene and invokes specific methods on certain node types.

See also:
SOP_Node::resetSimulation(), DOP_Parent::setDOPTime(), DOP_Parent::resimulate(), POP_ContextData::reset()
In addition, these optional parameters can be used to control the behaviour of the SOHO API functions:

SHOP Clerk Evaluation

SHOP_Node is a sub-class of OP_Node which can be thought of as a parameter collection. The SHOP consists of a set of parameters which it knows nothing about.

SHOP_Clerk is a class which packages the parameters of a SHOP into a form usable by a particular renderer. Clerks can be written in C++ (SHOP_Clerk) or Python (see $HFS/scripts/python/shopclerks).

To be used by a renderer, the parameters have to be gathered and processed into a form that the renderer can use. For example, for mantra, the parameters need to be built into a single string. The class which does this processing is the SHOP_Clerk class.

The clerk is called in several places

SOHO Wrangling

When designing the interface for an Object OTL (i.e. an light or camera), there are two approaches to take. Firstly, you can embed the actual light or camera object inside a subnetwork, then build channel references between the contained object and the subnetwork container. This approach is easiest and works well if you don't want users to customize the contained object.

On the other hand, if users need to add a rendering property to the camera or light object, then, they have to unlock the asset and manually dive in to modify the contents.

Wrangling is a process which allows a Python script to sit between the soho_program and the Houdini object interface. The wrangling process is implemented solely in Python and has been implemented for IFD.py (mantra), RIB.py (RenderMan) and MI.py (mentalray).

When processing a light or camera object, these scripts will look for a parameter named light_wrangler or camera_wrangler. These specify Python scripts which are called to evaluate parameters (rather than calling SOHO to evaluate the parameters directly. The wrangler is free to evaluate any parameters it chooses to evaluate any parameters on the renderer.

For example, a light wrangler might have a parameter for a shadow map which gets evaluated during shadow map generation, but also when evaluating the light shader string.

By using a wrangler, you can design the interface the way you want, but let the user make modifications easily (without having to unlock assets).

SOHO and IPR

As mentioned in HDK_SOHO_API::evaluate(), there are a set of intrinsic parameters. One such parameter is the state:previewmode parameter. The value of this parameter is controlled by the IPR viewer, and may be one of

As well, the parameters objlist:deletedinstance, objlist:deletedlight, objlist:deletedspace and objlist:deletedfog will list all objects which have been deleted from the scene.

Note:
It is only safe to evaluate the object's name if it has been deleted.

RenderMan Specifics

Houdini has some helpful tools to help integrating RenderMan style renderers. These tools are mostly stand-alone programs which are invoked from different places in Houdini.

RenderMan: Creating SHOPs from a RenderMan shader

In the Houdini bin directory, you can find slo2otl.py. This parses the output of sloinfo to query the parameters defined on a prman shader object. The program then invokes the rmands.py module to generate a SHOP OTL for the shader.

This process is fairly trivial, and hinges on the stand-alone tool hotl (see http://www.sidefx.com/docs/current/assets/advanced_otl)

The rmands module builds a dialog script and uses hotl to package it up into the appropriate form.

RenderMan: Compiling A Shader

When building RSL shaders with VOPS, the compiler needs to generate the object code in a specific place, log the errors and output in another place. Each RenderMan implementation also has a different shader compiler.

VOPs are set up to use a wrapper program to compile shader code (hrmanshader). The source code for this application can be found in $HFS/houdini/public/hrmanshader.

SOHO Overrides

A SOHO render can be started in one of several places in Houdini. The user may have hit the render button. But alternatively, the user may have chosen to call the output driver from the viewport menu, or from the viewport render state, or the IPR viewer, or even invoking a render to generate a thumb-nail for a material preview.

Each of these different calling contexts has an override mechanism similar to the HDK_SOHO_API::pushOverrides() method.

There are several files in $HFS/houdini/soho/overrides which define parameters and their override values. The different contexts pass information to the overrides through a set of variables which are specific to the context.

For example, the ViewportRender override file is called when the user chooses to render from the render menu in the viewport. The viewport passes the viewing camera transform in the $cam_xform variable and the file overrides the space:world parameter (which is an intrinsic SOHO parameter) with the value of that transform.

These files are used to force rendering to be routed to flipbooks, to force the soho_outputmode etc.

See also:
HDK_SOHO_API::pushOverrides(), SOHO Wrangling

Monkey Patching SOHO

Sometimes it's necessary to modify the factory SOHO programs. For example, if an output driver doesn't work exactly the way you need it to, you may want to modify the Python code. The most obvious way of doing this is to copy the Python code to a location earlier in the HOUDINI_PATH. Your module will be picked up before the factory version and so you're free to make any changes you want.

The problem with this approach is that occasionally new features or bugs will be fixed by Side Effects Software in the factory module. That means that when you start using a new version of Houdini, you need to verify that your modified version gets updated with the correct changes.

Monkey Patching (http://en.wikipedia.org/wiki/Monkey_patch) is a method of modifying runtime code of a module without altering the original code. Using Monkey Patching allows you to use the factory code, but to make surgical changes to the code by replacing single functions in a module.

An example of Monkey Patching would be changing the sin() function in the math module. 

    def mySin(x):
        # Use Taylor Series Approximation
        return x - x**3/6 + x**5/120 - x**7/5040

    # Import the standard math module
    import math

    # Change the sin() function in math to point to my function
    def patchMath():
        math.sin = mySin
This will change the math.sin() function to call the mySin() function instead of its original function. If you call patchMath(), then, all other code that uses math.sin() will now call your function instead.

Here's an example which replaces the LightSource function in the RIBapi module. 

    import os, RIBapi

    factory_LightSource = RIBapi.LightSource
    def LightSource(shader, handle, parmlist):
        # Process the shader string
        shader = processShaderString(shader)
        # Call the factory LightSource function with the modified shader
        handle = factory_LightSource(shader, handle, parmlist)

    # Change the LightSource function in RIBapi module
    RIBapi.LightSource = LightSource

    execfile('%s/houdini/soho/RIB.py' % os.getenv('HFS'))

    # Restore the method in the RIBapi module
    RIBapi.LightSource = factory_LightSource
If this file were placed in $HOME/houdini$VERSION/soho/RIB.py, then, it will be called instead of $HFS/houdini/soho/RIB.api.

One way of thinking about Monkey Patching (philosophically) is that you

RIBhooks/IFDhooks

An alternative to monkey patching is to use the IFDhooks or RIBhooks modules.

The RIB/IFD modules (i.e. RIB.py RIBframe.py) have callbacks defined through the hook modules. By placing a file named RIBuserhooks.py (or IFDuserhooks.py) in the Python path (including $HOUDINI_PATH/soho), your hooks will be picked up and called during the RIB generation.

Please note that if your hook functions need to access other parts of SOHO (i.e. importing RIBsettings.py), you may run into recursion errors. That is, RIBsettings will import RIBhooks, which then imports RIBsettings, etc. Please look at RIBhooks.py for possible ways around this issue.

A simple example which uses this would be something like: 

'''
    RIBuserhooks.py
    A simple example to illustrate the RIBhooks features.
'''
import traceback
import RIBapi
Ri = RIBapi

def pre_lockObjects(parmlist, objparms, now, camera):
    ''' Called before SOHO locks objects.  This allows you to modify the
        objects that are visible to SOHO '''
    Ri.Comment('RIB Hook - rendering from camera: %s' % repr(camera))
    return True

def pre_render(camlist, now, objlist, lightlist, spacelist, foglist,
                fromlight, cubemap):
    ''' Called before RiFrameBegin() for every frame '''
    Ri.Comment('RIB Hook - Render at time: %g %s' % (now, repr(camlist)))
    return False

''' List of hooks in this file '''
_HOOKS = {
    'pre_lockObjects'   : pre_lockObjects,
    'pre_render'        : pre_render,
}

def call(name='', *args, **kwargs):
    ''' Hook callback function '''
    method = _HOOKS.get(name, None)
    if method:
        try:
            if method(*args, **kwargs):
                return True
            else:
                return False
        except Exception, err:
            Ri.Comment('Hook Error[%s]: %s %s' % (name, __file__, str(err)))
            Ri.Comment('Traceback:\n# %s\n' %
                        '\n#'.join(traceback.format_exc().split('\n')))
    return False
Generated on Mon Jan 28 00:27:59 2013 for HDK by  doxygen 1.5.9