| On this page |
General improvements ¶
-
ML Train Regression now supports training models exceeding 2GB. -
ONNX Inference SOP and ONNX Inference COP have extended and improved support for ONNX models:-
Supports models greater than 2GB using an external data file.
-
Support for 7 additional, commonly used tensor element data types (other than float).
-
Support for empty tensor shapes (scalar tensors).
-
Support for picking the device used for inferencing.
-
Proper multi-threading when in CPU mode, using the same number of threads as Houdini.
-
SOP only - now supports content-dependent dynamic outputs. This means the output size may change based on what’s in the input (not on the input size).
-
Better information is provided for errors and warnings.
-
-
ML Inference nodes improvements (includes
ML Regression Inference SOP, ML Volume Tile Inference, ONNX Inference SOP , ONNX Inference COP ). -
Better memory management for storing ONNX models in Houdini using a new ONNX Cache implementation.
ML Building blocks ¶
-
ML Regression Kernel SOP and ML Regression Kernel TOP have improved performance from better SOP caching.
-
ML Pose Deserialize SOP now ensures that the linear parts of transforms are proper rotations. -
ML Regression Inference SOP has improved performance due to the elimination of redundant temporary storage buffers.
Image-specific ML tools ¶
-
New
Neural Layer to Depth (MoGe-2) COP allows you to input an image and estimates a depth, normal, and position map as outputs. -
New
Neural Layer to Mask (SAM2) COP creates image segmentation masks based on prompts using Meta’s SAM2 model. Prompts can be positive and negative clicks as well as an optional bounding box. -
New
Neural Cellular Automata Core COP runs a Neural Cellular Automata (NCA) model on an input cell state, evolving it over a number of iterations. -
New
Neural Cellular Automata Decode COP takes the cell state output from a Neural Cellular Automata Core COP node and decodes it into a visible pattern using a decoder neural network. -
New
ML Preprocess GSplats TOP preprocesses rendered EXR images, cameras, and a point cloud into a COLMAP-like dataset for Gaussian Splat training. -
New
ML Train GSplats TOP trains a 3D Gaussian Splatting model from a set of posed images. -
New
ML Train Neural Cellular Automata TOP trains a Neural Cellular Automata model to synthesize tileable pattern-based textures from a single target image.
-
New
ML Preprocess Computer Vision TOP prepares image datasets for computer vision training tasks. -
New
ML Train Computer Vision TOP trains an object detection, masking, or keypoint tracking model using a dataset prepared by the ML Preprocess Computer Vision TOP.
-
New ML Computer Vision workflow covers the full computer vision process within Houdini using a provided example file. This process allows you to take a video, isolate an object of interest, train a model on that dataset, and then run the model on the video to isolate that object throughout the video.
-
New
ML Computer Vision Inference COP applies an ONNX model trained using the ML Train Computer Vision TOP to detect and mask objects in an input image.
Terrain-specific ML tools ¶
-
New
Neural Terrain Generate SOP allows you to run inference custom or downloadable ML models. It contains a recipe to train custom ONNX models.
Character-specific ML tools ¶
-
New Agent Add ML Deformer configures agent shapes to be deformed using a model trained with the ML Train Deformer recipe for higher-quality skin deformation. This supports real-time deformation in the viewport for large crowds.
ML recipes ¶
-
New
ML Train Neural Cellular Automata COP recipe provides and easy to use entry point for training NCA models directly inside of COPs on COP input layers. -
New
ML Train GSplats from Karma LOP recipe automates the data generation and training process to create synthetic Gaussian splats. -
New Neural Cellular Automata recipe drops down the NCA Core and Decode COP nodes to provide a convenient starting point.
-
New Neural Cellular Automata Block recipe configures the NCA Core and Decode COPs to be used inside of a COP block to simulate NCA updates over time.