The initial applications of ML in Houdini were enabled by nodes in the Basic Utilities category. Most nodes in this category are general-purpose and can also be used outside of ML. However, the category’s creation was largely motivated by ML use cases. In the context of Houdini and ML, Basic Utilities is a label applied to this diverse group of low-level nodes to make the ML area easier to understand.

The basic utilities allow you to perform a variety of ML-related tasks including outputting raw files, importing raw files, performing inference using ONNX, and training using external PyTorch scripts. These nodes are powerful but they exist at a relatively low level of abstraction. Using them often involves a lot of setup work. See Basic utilities for more detailed information.

Important nodes in this category are:

• ONNX Inference SOP

• Python Virtual Environment TOP

• Python Script TOP

• ROP Geometry Raw Output SOP

• Raw Import SOP

ONNX Inference SOP allows you to use existing ONNX models in the SOP context. To do this, you need to understand how the ONNX model represents its inputs and outputs as tensors. Basically, you need to understand how point/detail attributes and volumes translate to tensors and vice versa.

On top of existing models, the basic utilities allow you to train your own models. To run a training script within Houdini, you can use the combination of Python Virtual Environment TOP and Python Script TOP. You can either create your own training script or adapt an existing training script. To use these training features, you need a deep learning framework such as PyTorch.

Other Basic Utilities such as ROP Geometry Raw Output SOP allows you to convert geometry generated in a SOP network and convert it to a raw file format. This allows you to write out a data set generated in Houdini in a format that can be read by your training script.

Besides helping training, Python Virtual Environment TOP and Python Script TOP also allows you to perform inference. This can be useful when you have a model that is not in ONNX format. For example, you could run a native PyTorch model that was created using Python. In that case, ROP Geometry Raw Output SOP and Raw Import SOP can get the inputs from SOPs and end the outputs back to SOPs.

Using the Basic Utilities effectively for ML usually requires a fair amount of technical ML expertise. When you create training setups with the Basic Utilities in Houdini, you may need to do a significant amount of boilerplate work. Because this work is often similar across projects, you’re likely to start with an existing training setup and modify it for a new purpose. This process may involve repeated work in both your custom scripts and Houdini node networks.

Certain types of ML such as supervised learning, can be easier to set up using the Building Blocks family of ML nodes. These nodes automate the repeated work, providing a faster, higher-level way to set up your ML pipeline.

Building Blocks can help create a custom ML solution based on your procedural pipeline without tensors and PyTorch code. If you do need Pytorch, it’s usually a short snippet to create a model. For example, you can generate and preprocess your data set by using Houdini’s procedural networks. ML Example SOP and ML Example Output SOP can be inserted in your procedural networks to help with this. After the data set has been created and written out, you can apply training nodes such as ML Train Regression TOP, and ML Regression Kernel TOP to obtain a trained model. After that, ML Regression Inference SOP allows you to run inference with your trained model.

Building Blocks are specifically suited for discriminative machine learning tasks such as regression. You may have a chain of geometry operations in SOPs that takes an input geometry and produces an output geometry. For example, your procedural chain is fairly stable and deterministic but very expensive to cook. In that case, you can train a model that approximates closely what this procedural chain of nodes does.

Houdini includes ML solutions built on top of Building Blocks. These include the ML Train Deformer recipe and ML Train Volume Upres recipe. The ML Deformer recipe helps you approximate an expensive flesh simulation in real time. ML Volume Upres recipe allows you to train a model that up-resses pyro simulation (the full 3D volumes, not just 2D images).

Basic Utilities provide a low-level way to create or integrate ML solutions into Houdini. Building Blocks exist at a higher level of abstraction, allowing common supervised training and inferencing setups to be created without repetitive boiler-plate work. However, for highly specialized ML tasks, ready made high-level nodes exist in Houdini. These fall into two further categories: Trainable ML Solutions and Neural Nodes.

Trainable ML Solutions helps with various specialized tasks including terrain generation, denoising, realistic character deformation, volume upressing, object recognition, Gsplat creation, and texture synthesis. Some of these tools are built on top of Basic Utilities and/or Building Blocks. Trainable ML Solutions provide you with a convenient, specialized interface for training and inferencing that is directly relevant for your task.

Trainable ML Solutions include training nodes like:

• ML Train Computer Vision TOP

• ML Train GSplats TOP

• ML Train OIDN TOP

• ML Train Style Transfer TOP

Some of these nodes may have a corresponding preprocess node or a corresponding inference node.

Neural Nodes are self contained solutions based on models integrated into Houdini using the Basic Utilities. These self contained ML solutions usually have names starting with the word Neural (Neural Cellular Automata are an exception to this rule). Some Neural Nodes integrate existing third-party models. For example:

• Neural Layer to Mask (SAM2) COP - Built on top of the SAM2 image-segmentation model.

• Neural Layer to Depth (MoGe-2) COP - Based on the MoGe-2 monocular geometry estimation model.

• Neural Point Surface - Integrates a variety of models trained using Houdini.

Models in the ONNX format could in theory be run directly using the Basic Utilities, but this requires a lot of setup and translation work. To save time, there are canned solutions and interfaces tailored to the specific, high-level task you want to perform.