HDK
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
SOHO: Scripted Output of Houdini Objects

The back-end in Houdini is composed of several different components

SOHO: The API

SOHO provides a very small set of API functions. These functions sit between Houdini and Python scripts and provide a simple object model of the Houdini scene.

The SOHO API consists of 25 functions. These are documented in the HDK_SOHO_API namespace, though they are currently only available in a Python API.

These simple API functions have been wrapped in pure Python modules: soho.py and sohog.py, which can be found in $HFS/houdini/soho. It is the soho.py and sohog.py modules which are typically used.

SOHO Output Drivers

SOHO is invoked through a SOHO output driver. A SOHO output driver is an OTL with some specific parameters created to control the behavior of the output driver.

A new output driver can be created by starting Houdini, then choosing "File -> New Operator Type" then choosing the "Output Driver Type".

What Happens When The Render Button Is Hit?

When the user hits the render button, the output driver evaluates some parameters (see HDK_SOHO_ROP_Parameters) to determine what should be done.

The string soho_program parameter is used to specify a Python program which gets run.

Before the Python program is invoked, the output driver may open files.

The output driver evaluates int soho_outputmode. This parameter can have three possible values:

  • 0: When the value is 0, the command specified in the string soho_pipecmd parameter is opened as a pipe. The file descriptor of this pipe is passed to the SOHO program (see HDK_SOHO_API::getFile())
  • 1: When the value is 1, the filename specified in the string soho_diskfile parameter is opened for writing. The file descriptor for this file is passed to the SOHO program (see HDK_SOHO_API::getFile()).
  • 2: When the value is 2, no files are opened automatically.

In addition, the output driver will open a temporary file to trap errors from the Python program. This is passed through HDK_SOHO_API::getFile(2).

The process to invoke the Python program is given as follows:

  1. Open required file descriptors
  2. Push Python's sys.stdout and sys.stderr objects
  3. Add $HOUDINI_PATH/soho to the Python path
  4. Call program in string soho_program
  5. Restore sys.stdout and sys.stderr objects
Note
Since SOHO traps both sys.stdout and sys.stderr (after soho.init() ), you may have to use sys.__stderr__.write(msg) to see the output in the textport. You can, of course, call soho.warning() or soho.message() to add text to the output driver.

SOHO Output Driver Errors

There are different ways of reporting errors to the ROP from within your Python code. The easiest is to write to the stderr which was set by HDK_SOHO_API::getFile(2).

Alternately, you can call one of:

  • soho.message(msg)
    Add the msg to the ROP as information text
  • soho.warning(msg)
    Add the msg to the ROP as a warning
  • soho.error(msg)
    Add the msg to the ROP as error text

SOHO Output Driver Parameters

The output driver looks for certain parameters which it uses to determine how the output driver should operate. As with all operators, the parameters do not have to be visible to the user, they just have to exist.

The parameters are all defined in $HFS/houdini/soho/parameters/SohoIntrinsics.ds, but are not visible in the Houdini interface by default. There are instructions in that file to make the parameters visible.

The easiest way to enable these properties is to set the environment variable HOUDINI_SOHO_DEVELOPER before starting Houdini. For example:

% setenv HOUDINI_SOHO_DEVELOPER 1; houdini -foreground
# export HOUDINI_SOHO_DEVELOPER=1; houdini -foreground
  • string soho_program (mandatory)
    This is the name of the Python program which is run when the user hits the render button.
  • int soho_outputmode
    Determine what file descriptor is passed to HDK_SOHO_API::getFile(1)
  • string soho_diskfile
    When soho_outputmode evaluates to 1, this parameter specifies the disk file which is opened and passed to HDK_SOHO_API::getFile(1)
  • int soho_mkpath
    When soho_outputmode evaluates to 1, this parameter specifies that the intermediate directories of output files will be created (during script creation).
  • string soho_pipecmd
    When soho_outputmode evaluates to 0, this parameter specifies the command which is opened as a pipe and passed to HDK_SOHO_API::getFile(1)
  • int soho_foreground
    When running soho_pipecmd, this determines whether the SOHO output driver will block and wait for the command to finish. The rps and rkill hscript commands can be used to manage background renders.
  • int soho_multiframe
    If this parameter returns non-zero, the soho_program will only be invoked one time, regardless of the frame range of the ROP.
  • int soho_initsim
    If this parameter exists and evaluates to a non-zero value, the output driver will initialize all simulation OPs before invoking the soho_program.

    This traverses all nodes in the scene and invokes specific methods on certain node types.

    See Also
    SOP_Node::resetSimulation(), DOP_Parent::setDOPTime(), DOP_Parent::resimulate()

    In addition, these optional parameters can be used to control the behavior of the SOHO API functions:

  • string soho_shopstyle
    This parameter specifies which SHOP Clerk should be used to evaluate shaders. See $HH/pythonX.Y/shopclerks.
    The value of this parameter is also used to select which override is used. For example, the RIB output driver has this parameter set to "RIB". Which will select the appropriate override files in $HH/soho/overrides (i.e. it won't use the VMantra overrides).
    See Also
    SHOP Clerk Evaluation
  • int soho_precision
    Specify the precision for floating point numbers (defaults to 12). This value is clamped between 1 and 32.
    See Also
    HDK_SOHO_API::printArray() HDK_SOHO_API::arrayToString()
  • int soho_indentstep
    Specify the indent step for HDK_SOHO_API::indent()
  • float soho_almostzero
    Values which have abs(value) < soho_almostzero will be output as 0 exactly.
    See Also
    HDK_SOHO_API::printArray() HDK_SOHO_API::arrayToString()
  • int soho_safename
    Force objects to have "safe" names. A safe name follows the normal variable naming rules, that is, characters must be from the set of alpha-numeric characters and the underscore ('_').
    See Also
    HDK_SOHO_API::addCandidates()
  • int soho_autoheadlight Add a light source at the camera's location if there are no other light sources in the scene.
    See Also
    HDK_SOHO_API::addCandidates()
  • int soho_ipr_support
    Allow this output driver to enable IPR mode in the IPR viewer. Be very cautious with this parameter since not all renderers support IPR mode.
  • int soho_viewport_menu
    This parameter is used to determine the visibility of the output driver in the viewport render menu.

SHOP Clerk Evaluation

SHOP_Node is a sub-class of OP_Node which can be thought of as a parameter collection. The SHOP consists of a set of parameters which it knows nothing about.

SHOP_Clerk is a class which packages the parameters of a SHOP into a form usable by a particular renderer. Clerks can be written in C++ (SHOP_Clerk) or Python (see $HFS/scripts/python/shopclerks).

To be used by a renderer, the parameters have to be gathered and processed into a form that the renderer can use. For example, for mantra, the parameters need to be built into a single string. The class which does this processing is the SHOP_Clerk class.

The clerk is called in several places

  • The hscript shopstring(string shop_path, string render_type) expression function. The render_type determines which clerk is used to process the SHOP string.
  • The corresponding HOM function: hou.ShopNode.shaderString(render_type). Again, render_type is used to determine which SHOP_Clerk is invoked.
  • The HDK_SOHO_API::evaluate() method, when the type is 'shader' (or 'bounds'). The clerk type is determined by the soho_shopstyle parameter defined on the output driver (see HDK_SOHO_ROP_Parameters).

SOHO Wrangling

When designing the interface for an Object OTL (i.e. an light or camera), there are two approaches to take. Firstly, you can embed the actual light or camera object inside a subnetwork, then build channel references between the contained object and the subnetwork container. This approach is easiest and works well if you don't want users to customize the contained object.

On the other hand, if users need to add a rendering property to the camera or light object, then, they have to unlock the asset and manually dive in to modify the contents.

Wrangling is a process which allows a Python script to sit between the soho_program and the Houdini object interface. The wrangling process is implemented solely in Python and has been implemented for IFD.py (mantra) and RIB.py (RenderMan).

When processing a light or camera object, these scripts will look for a parameter named light_wrangler or camera_wrangler. These specify Python scripts which are called to evaluate parameters (rather than calling SOHO to evaluate the parameters directly. The wrangler is free to evaluate any parameters it chooses to evaluate any parameters on the renderer.

For example, a light wrangler might have a parameter for a shadow map which gets evaluated during shadow map generation, but also when evaluating the light shader string.

By using a wrangler, you can design the interface the way you want, but let the user make modifications easily (without having to unlock assets).

SOHO and IPR

As mentioned in HDK_SOHO_API::evaluate(), there are a set of intrinsic parameters. One such parameter is the state:previewmode parameter. The value of this parameter is controlled by the IPR viewer, and may be one of

  • default: Standard rendering mode
  • generate: Generation phase of IPR rendering
    In generate mode, SOHO will keep the pipe (soho_pipecmd) command open between invocations of the soho_program.
  • update: Send updated changes from previous generation
    In this rendering mode, the special object list parameters: objlist:dirtyinstance, objlist:dirtylight, objlist:dirtyspace and objlist:dirtyfog will contain the list of all objects modified since the last render (whether a generate or update).

    As well, the parameters objlist:deletedinstance, objlist:deletedlight, objlist:deletedspace and objlist:deletedfog will list all objects which have been deleted from the scene.

    Note
    It is only safe to evaluate the object's name if it has been deleted.

RenderMan Specifics

Houdini has some helpful tools to help integrating RenderMan style renderers. These tools are mostly stand-alone programs which are invoked from different places in Houdini.

RenderMan: Creating SHOPs from a RenderMan shader

In the Houdini bin directory, you can find slo2otl.py. This parses the output of sloinfo to query the parameters defined on a prman shader object. The program then invokes the rmands.py module to generate a SHOP OTL for the shader.

This process is fairly trivial, and hinges on the stand-alone tool hotl (see http://www.sidefx.com/docs/current/assets/advanced_otl)

The rmands module builds a dialog script and uses hotl to package it up into the appropriate form.

RenderMan: Compiling A Shader

When building RSL shaders with VOPS, the compiler needs to generate the object code in a specific place, log the errors and output in another place. Each RenderMan implementation also has a different shader compiler.

VOPs are set up to use a wrapper program to compile shader code (hrmanshader). The source code for this application can be found in $HFS/houdini/public/hrmanshader.

SOHO Overrides

A SOHO render can be started in one of several places in Houdini. The user may have hit the render button. But alternatively, the user may have chosen to call the output driver from the viewport menu, or from the viewport render state, or the IPR viewer, or even invoking a render to generate a thumb-nail for a material preview.

Each of these different calling contexts has an override mechanism similar to the HDK_SOHO_API::pushOverrides() method.

There are several files in $HFS/houdini/soho/overrides which define parameters and their override values. The different contexts pass information to the overrides through a set of variables which are specific to the context.

For example, the ViewportRender override file is called when the user chooses to render from the render menu in the viewport. The viewport passes the viewing camera transform in the $cam_xform variable and the file overrides the space:world parameter (which is an intrinsic SOHO parameter) with the value of that transform.

These files are used to force rendering to be routed to flipbooks, to force the soho_outputmode etc.

See Also
HDK_SOHO_API::pushOverrides(), SOHO Wrangling

Monkey Patching SOHO

Sometimes it's necessary to modify the factory SOHO programs. For example, if an output driver doesn't work exactly the way you need it to, you may want to modify the Python code. The most obvious way of doing this is to copy the Python code to a location earlier in the HOUDINI_PATH. Your module will be picked up before the factory version and so you're free to make any changes you want.

The problem with this approach is that occasionally new features or bugs will be fixed by SideFX in the factory module. That means that when you start using a new version of Houdini, you need to verify that your modified version gets updated with the correct changes.

Monkey Patching (http://en.wikipedia.org/wiki/Monkey_patch) is a method of modifying runtime code of a module without altering the original code. Using Monkey Patching allows you to use the factory code, but to make surgical changes to the code by replacing single functions in a module.

An example of Monkey Patching would be changing the sin() function in the math module.

def mySin(x):
# Use Taylor Series Approximation
return x - x**3/6 + x**5/120 - x**7/5040
# Import the standard math module
import math
# Change the sin() function in math to point to my function
def patchMath():
math.sin = mySin

This will change the math.sin() function to call the mySin() function instead of its original function. If you call patchMath(), then, all other code that uses math.sin() will now call your function instead.

Here's an example which replaces the LightSource function in the RIBapi module.

import os, RIBapi
factory_LightSource = RIBapi.LightSource
def LightSource(shader, handle, parmlist):
shader = processShaderString(shader)
# Call the factory LightSource function with the modified shader
handle = factory_LightSource(shader, handle, parmlist)
# Change the LightSource function in RIBapi module
RIBapi.LightSource = LightSource
execfile('%s/houdini/soho/RIB.py' % os.getenv('HFS'))
# Restore the method in the RIBapi module
RIBapi.LightSource = factory_LightSource

If this file were placed in $HOME/houdini$VERSION/soho/RIB.py, then, it will be called instead of $HFS/houdini/soho/RIB.api.

One way of thinking about Monkey Patching (philosophically) is that you

  • Create a sub-class of the module which you patch
  • Inherit all functions of the baseclass
  • Override some functions (through Monkey Patching)
  • Replace all instances of the module with your new module

RIBhooks/IFDhooks

An alternative to monkey patching is to use the IFDhooks or RIBhooks modules.

The RIB/IFD modules (i.e. RIB.py RIBframe.py) have callbacks defined through the hook modules. By placing a file named RIBuserhooks.py (or IFDuserhooks.py) in the Python path (including $HOUDINI_PATH/soho), your hooks will be picked up and called during the RIB generation.

Please note that if your hook functions need to access other parts of SOHO (i.e. importing RIBsettings.py), you may run into recursion errors. That is, RIBsettings will import RIBhooks, which then imports RIBsettings, etc. Please look at RIBhooks.py for possible ways around this issue.

A simple example which uses this would be something like:

'''
RIBuserhooks.py
A simple example to illustrate the RIBhooks features.
'''
import traceback
import RIBapi
Ri = RIBapi
def pre_lockObjects(parmlist, objparms, now, camera):
''' Called before SOHO locks objects. This allows you to modify the
objects that are visible to SOHO '''
Ri.Comment('RIB Hook - rendering from camera: %s' % repr(camera))
return True
def pre_render(camlist, now, objlist, lightlist, spacelist, foglist,
fromlight, cubemap):
''' Called before RiFrameBegin() for every frame '''
Ri.Comment('RIB Hook - Render at time: %g %s' % (now, repr(camlist)))
return False
''' List of hooks in this file '''
_HOOKS = {
'pre_lockObjects' : pre_lockObjects,
'pre_render' : pre_render,
}
def call(name='', *args, **kwargs):
''' Hook callback function '''
method = _HOOKS.get(name, None)
if method:
try:
if method(*args, **kwargs):
return True
else:
return False
except Exception, err:
Ri.Comment('Hook Error[%s]: %s %s' % (name, __file__, str(err)))
Ri.Comment('Traceback:\n# %s\n' %
'\n#'.join(traceback.format_exc().split('\n')))
return False