|
HDK
|
Like most other operator type categories in Houdini, LOP nodes take one or more inputs, manipulate the incoming data, and output data in a form that can act as the input to another LOP node. The data passed between LOP nodes is a USD stage. LOPs behave as if they each own their own USD stage, though for performance reasons this is not the case. If a LOP node follows the ground rules, it is possible to write a LOP node as if this really were the case. However it is best to know the underlying truth to understand the best practices and get the best possible performance.
USD stages in LOPs are always wrapped inside an HUSD_DataHandle object. An HUSD_DataHandle can be thought of as a shared pointer wrapper around an XUSD_Data object. The XUSD_Data object holds the pointer to the USD stage. However this USD stage can be shared between many XUSD_Data objects. In this way, a single USD stage can be shared by many LOP nodes.
In order for this stage sharing to work, access to the stage through the data handle involves locking the data handle. This can take one of three forms:
SdfLayer. This layer may or may not be active on the stage, depending on how the lock is constructed. If constructed directly from the HUSD_DataHandleA layer lock can be created from a write lock, in which case the SdfLayer that can be modified is on the USD stage. This means each edit of the layer may issue a USD change notification and trigger some amount of recomposition. To prevent this, the layer lock constructor has an argument that causes the lock to create an SdfChangeBlock, and destroy it when the layer lock is destroyed. You can also create your own SdfChangeBlock objects. But be aware that as long as a change block exists, queries to the USD stage through the write lock may return incorrect results.
If the layer lock is created directly from the data handle, there is always an SdfChangeBlock created, but it doesn't really matter because the layer being modified isn't on the USD stage, so there will be no change notices and no need to recompose. But the read-only USD stage available through the layer lock will never reflect any edits made on the layer. So algorithms that take a layer lock object must always read information it needs from the stage before applying any edits to the active layer.
The read lock is generally used when reading stage data from an input LOP node. The write locks are used only on the LOP node's own data handle (accessed through LOP_Node::editableDataHandle(), which returns a non-const reference to the node's data handle). Write locks should only be created in the cook method of a LOP node.
The act of locking the data handle transforms the shared USD stage (using other data stored in the node-specific XUSD_Data object) to match a particular node's idea of what the stage should be. Once a stage is locked (for reading or writing) to a particular node's data handle, further attempts to lock the stage will still succeed, but at a steep cost. The LOP network will recook as many nodes as it needs to find the first LOP that isn't sharing a stage with the node being locked. It will recook all those nodes with a new stage. Then when both of these locks are released, one of the stages will simply be discarded. It is far better to structure your code to avoid this situation. See Maximizing LOP Performance for information on how to accomplish this.
Note that this expensive double locking will occur if a write lock and a layer lock are both created directly from the data handle at the same time. This is why the layer lock can be created from a write lock, in spite of the complications this introduces. Some algorithms require a layer lock and some algorithms require a write lock. To be able to use both types of algorithms in a single LOP node, it must be possible to have both types of lock existing at the same time. But any number of write and layer locks can be created and destroyed serially in a single cook method. Often this is the most efficient approach. For example a LOP node that may need to create prims and then edit them may choose to create a layer lock for use with an HUSD_CreatePrims object. Then the layer lock is destroyed and a write lock is used to apply further edits to the new prims using the more expressive USD APIs.
The USD stage gnerated by a LOP node network always consists of a root layer that is empty except for the small set of root prim metadata that USD requires to be on the root layer. Aside from this metadata, the root layer only contains a list of sublayers. These sublayers are, of course, added by LOP nodes. A LOP can add a sublayer from a file on disk (as the Sublayer LOP does), or it can create a new anonymous sublayer (sometimes referred to as a "LOP Layer"), or it can edit the same LOP Layer edited by the node connected to its primary input. In the latter two cases, the anonymous LOP Layer that can be manipulated by the LOP node is called the Active Layer. A LOP node can actually create multiple LOP Layers (and add multiple sublayers from disk), but at any given time, only the strongest LOP Layer (the Active Layer at that moment) can be the Edit Target and receive authored USD content.
The above description is, admittedly, largely a matter of convention. Because LOPs exposes a raw USD stage, and the full USD API is available to manipulate that stage, there is no way for Houdini or LOPs to prevent manipulations of the stage or individual layers that fall outside the above description. But doing so will invariably result in unexpected behaviors. Changes from downstream nodes may appear on the stage in upstream nodes. Changes may be discarded (until the next time a node cooks) when cooking nodes further up or downstream. And no doubt other bad behaviors may result.
LOPs tries to make it as easy as possible to conform to the rules. The USD edit target is set automatically when write locking a stage. Layers that are not the edit target have SdfLayer::SetPermissionToEdit(false) called on them (including layers loaded from disk). But both C++ and python code can circumvent these protections. Please do not do this.
Like other Houdini operator types, the bulk of the work for implementing a new LOP node lies in writing the cook method, cookMyLop. The basic structure of this method generally consists of:
The first step of initializing this node's data handle is done using one of the following methods:
editableDataHandle().editableDataHandle().createNewData() to create a new empty stageOne very common operation in a LOP node involves evaluating a Primitive Pattern to return a set of USD primitives. This is done using the HUSD_FindPrims object. HUSD_FindProps can find specific properties on a set of primitives. These objects are so common that their use has been wrapped into some utility functions, LOP_Node::getSimplifiedCollection. These methods take care of evaluating the primitive pattern, handling errors, and dealing with the fact that evaluating a primitive pattern can introduce a time dependency on the node (if the patter evaluation depends on a time varying attribute on the stage).
There are really no limits to the types of changes that can be made to the active layer. The HUSD library contains a large number of helper classes for making common edits using either the USD or SDF APIs:
Many of these classes use the HUSD_Path and HUSD_PathSet classes. These classes are wrappers arouns SdfPath and SdfPathSet respectively. HUSD_PathSet in particular provides some useful helper methods for making certain kinds of common queries and edits to the set (such as eliminating all paths from the set that have any ancestors also in the set). These wrappers add very little overhead and provide direct access to the underlying Sdf objects when needed. So it is recommended to use these objects whenever possible, rather than translating or copying to the Sdf classes. It should go without saying that using strings to store and manipulate USD paths should be avoided except as a last resort.
Some more complicated operations have also been wrapped into useful utility calsses in HUSD. In particular, the code for combining stages can be tricky, and should always be done using the following classes:
There are also some HUSD classes that perform edits that are not normally permitted, such as modifying the stage root layer. These classes are the only safe way to make these kinds of edits:
The HUSD classes are used to share functionality between LOP nodes, and also to isolte USD API calls to a single library in Houdini (which allows the USD library that ships with Houdini to be replaced). But custom LOP nodes are also free to call USD and SDF APIs directly (or through a custom wrapper library), as long as the rules for permitted edits are followed.
It is best to use the HUSD helper classes to create new layers and compose them onto the USD stage. These helper classes ensure anonymous layers are created using standard mechanisms, and are tracked properly to avoid nasty surprises. However sometimes this is not possible. Custom libraries that perform USD manipulation may not be able to use Houdini specific functionality because they need to operate within multiple DCCs. In these cases it is important to know a few rules to ensure these layers behave as expected in the LOP context.
When creating or composing layers that exist on disk, no special considerations are required. We recommend not altering layers on disk while they are in use by a LOP network, because LOP nodes may be affected by these changes to the underlying data, but Huodini cannot track such modifications. So if you do alter files on disk while Houdini is running, it is your responsibility to dirty any nodes that may be affected.
When creating anonymous layers, there are two concerns. One is to ensure that the USD ROP will save the layer to disk at a desired location. To force the USD ROP to save an anonymous layer to disk, two conditions must be met. First, the anonymous layer must be identified as a "LOP authored layer", which requires that the anonymous layer's "tag" start with LOP. Second, to specify the location on disk where the layer should be saved, there must exist a defined root prim on the layer at the path returned by HUSD_Constants::getHoudiniLayerInfoPrimPath (/HoudiniLayerInfo), with a prim type of HUSD_Constants::getHoudiniLayerInfoPrimType (HoudiniLayerInfo). On this prim there should be Custom Data with the name HoudiniSavePath which is a string set to the full path to the location on disk where the layer should be saved. In addition, the HoudiniSavePathIsTimeDependent Custom Data metadata can be set to true to indicate that the save path will change from frame to frame. This is mostly a hint to Houdini's save mechanism to help report to users when the LOP network is doing something potentially problematic (such as trying to sublayer a different layer at each frame). But there are cases where a time varying save path makes sense, such as when authoring a value clip.
The other concern is ensuring the layer doesn't get deleted when the USD stage is morphed in a way that removes the layer from the stage. This is done by calling HUSD_AutoWriteLock::addHeldLayer or HUSD_AutoLayerLock::addHeldLayer. These methods simply add a shared pointer to the layer to an array inside the LOP node's HUSD_DataHandle, which will prevent USD from deleting the layer.
All of these instructions for creating new anonymous layers in your LOP cook method also apply when writing code to do these things in a Python LOP. The Python LOP even allows you to call hou.pwd().addHeldLayer to add an anonymous layer to the array of held layers in the node's data handle.
Another common operation in LOPs is composing the output of one LOP node onto the stage of another LOP node. Examples of this are the multi-inputs of the Sublayer LOP and the Reference LOP. In order to do this safely, the output of the source LOP node must be copied, processed, and held in a registry (the HUSD_LockedStageRegistry). This registry maps the OP_Context used to cook the LOP node with the processed staged generated by the cook. It ensures that the stage will not be deleted as long as it is being referenced by another stage, and also ensures that if the stage is composed onto another stage in multiple places (the same LOP node is referenced multiple times) that the resulting USD saved to disk will point both references to the same file on disk.
Using the locked stage registry involves calling HUSD_LockedStageRegistry::getInstance to get the global registry object, then calling HUSD_LockedStageRegistry::getLockedStage on the returned object. This returns an HUSD_LockedStagePtr (a shared pointer to an HUSD_LockedStage). From this object, call HUSDLockedStage::getRootLayerIdentifier to get the identifier for the cooked stage's root layer. This will be an anonymous LOP layer, which can be added to the sublayer or reference metadata of the active stage. This can be done using USD APIs, or the HUSD helper classes. The returned HUSD_LockedStagePtr should be kept in member data of the LOP node until the next time the node is cooked. This ensures that the existing locked stage entry will be reused if another LOP node requests it.
As with evaluating parameters, be sure to avoid having any lock on any other LOP node's data handles when fetching a locked stage from the registry, as doing so may require cooking that LOP node, or reconfiguring a shared stage.
Because multiple LOP nodes can share a stage, cooking a chain of LOP nodes can cause this shared stage to be manipulated many, many times. The intermediate states of the stage may be vastly different from the final stage described by the LOP node at the end of the chain. But after the full recook, the final state of the stage may have only changed a few attributes on a single layer. Because of this churn on the shared stage, the LOPs Scene Viewer owns its own HUSD_DataHandle, with an XUSD_Data object that does not share a stage with any other XUSD_Data. When the display LOP recooks, the viewport "mirrors" this new stage state. This mirroring process is very efficient, being built on the SdfLayer::TransferContent() method, which only edits the desination if it differs from the source.
By performing viewport updates this way, the number of edits that must be processed by hydra and the render delegate is minimized. It also allows the viewport to choose to ignore changes occuring in the LOP Network (if the user presses the Pause Updates button). Finally, it allows the viewport stage to keep viewport overrides applied to its stage without requiring the viewport overrides to ever be applied to the LOP stage. Adding and removing the viewport overrides can be expensive, both in terms of composition, and its effect on hydra render delegate syncing.
Viewport overrides are stored in the session layer of the stage (which we have so far been ignoring). In LOPs, this session layer is reserved for viewport overrides, which are generally set by interacting with the scene graph tree, but can also be manipulated using the hou.LopNetwork.viewportOverrides() method. Like the root layer, the session layer simply contains a set of sublayers:
Like other USD stage edits, Houdini can't stop code from editing these layers directly, but doing so outside the confines of the standard APIs is likely to lead to unexpected behavior and should be avoided.
1.8.6