Houdini 22.0 Nodes Geometry nodes

Agent Add ML Deformer geometry node

Adds an agent shape, which is deformed using an ML model.

On this page
Since 22.0

This node adds shapes to an agent and configures them to use a trained ML model, which deforms the skin of the character. Internally, the ML model predicts weights used to combine the residual blend shapes. These combined blend shapes are then used to apply a correction to the rest skin geometry.

Note

Agent Add ML Deformer assumes the model was trained on a data set that was created using the ML Pose Serialize SOP and its default settings - Mode set to Subskeleton and Include World Rotation turned on. The Agent Add ML Deformer SOP’s Transform Group parameter also needs to match the joint group used in the ML Pose Serialize SOP when creating the training data set. These requirements are met if the ML Train Deformer recipe was used for training.

See the Machine Learning documentation for more information.

Parameters

Group

The agents to modify.

Layer Name

The name of the agent layer to create. The layer contains shape bindings to the new shapes.

Channel Name Prefix

The prefix for the blend shape channel names used by the deformer. Any missing blend shape channels are added to the agent’s rig.

If an agent is assigned multiple shapes that use the ML deformer, the channel names used by each shape must be unique. Channel names can be reused across shape variations when only one variant is assigned to each agent instance.

Model File

The trained model in ONNX format that takes as input a serialized pose and outputs a tuple of weights for the residual blend poses.

Execution Provider

Determines which ONNX execution provider to use for inference. By default, the node attempts to pick the best available provider and prefers to use GPU acceleration.

Automatic

Chooses the best provider for the current system. This option prioritizes CUDA if installed, DirectML/CoreML as a fallback depending on the platform, and CPU inference if no GPU provider is available.

CPU

Perform inference using the CPU.

CUDA

Perform inference using CUDA/cuDNN. CUDA and cuDNN must be installed using the packages provided by NVIDIA.

DirectML

Only available on Windows. Performs inference using the Windows Direct Machine Learning library.

CoreML

Only available on macOS. Performs inference using Apple’s Core ML library.

Device ID

The GPU to use for inference when multiple GPUs are available. It has no effect when CPU inference is used.

Transform Group

The transform group whose pose to serialize for inclusion into the data set on which Model File was trained.

Tip

The joints in the transform group should match the skeleton joint group used when training the model. The Agent from Rig SOP’s Point Groups parameter can be used to import joint groups when creating an agent from skeleton geometry.

Blend Shapes Source

The source of the residual blend shapes used by the ML deformer.

Packed Residual

The Residual Blend Shapes input is expected to consist of packed primitives, and each packed primitive is expected to contain a residual blend shape, which is a blend shape that has the capture shape subtracted from it. This is the format generated by the ML Train Deformer recipe.

Shape Geometry

Use blend shapes that already exist on the Shape Geometry input. The blend shapes are expected to have the same format as the Character Blend Shapes SOP’s Rest Geometry input.

Inputs

Agents

The agent primitives to modify.

Shape Geometry

The shapes to import. The name primitive attribute is used to partition the input geometry into separate agent shapes.

Residual Blend Shapes

The packed primitives that contain the residual blend shapes. The first residual blend shape always adds a weight of 1. The remaining residual blend shapes are scaled by the weights predicted by the ML model before they are applied.

Outputs

Agents

The modified agent primitives, with shapes using the ML deformer.

See also

Geometry nodes