Houdini 22.0 Machine Learning Trainable ML Solutions

ML Train Neural Cellular Automata

On this page

Overview

There are several machine learning recipes available through the tab menu. These are similar to shelf tools that put down networks of nodes for learning purposes or plug-and-play workflows.

The ML Train Neural Cellular Automata recipe integrates the training process of neural cellular automata (NCA) models directly into COPs.

See Neural Cellular Automata for a high-level overview about COPs inference, recipes, and TOPs-based training.

Structure

ML_Train_Neural_Cellular_Automata

This subnetwork is a high-level wrapper around a TOPs training setup (the ML Train Neural Cellular Automata). It lets you configure several hyperparameters that influence the training.

Important nodes

ML Train Neural Cellular Automata

Train a neural cellular automata model to synthesize tileable pattern-based textures from a single target image.

Getting started

After creating the recipe, you need a pattern-based target image (the input_pattern) to bring an NCA into COPs. The viewport then displays a helper layout that checks for common issues when selecting target images. This example uses a close-up of a pair of jeans.

ML Train Neural Cellular Automata COPs recipe

Note

Click Download VGG Checkpoint on the ML_Train_Neural_Cellular_Automata subnetwork after a fresh Houdini installation.

Choosing a target image

The helper layout displays your input texture, a low-resolution version that reflects the detail level of the cells, a visualization of potential noise, and crops of the target image. Ideally, the target image is cropped to a square aspect ratio with a resolution of 1024×1024.

The most important factor to pay attention to is scale. The general pattern must still be visible at the 128×128 resolution that the NCA trains on. This is closely tied to long-range dependencies: a single feature of the pattern should not be larger than roughly 10-15 pixels in the low-resolution state. The more cells an element covers, the harder it is for the NCA to propagate the information since the update range is only one neighbor per iteration.

The following are guidelines for selecting a target image:

  • Minimize noise: The pattern should contain as little noise as possible.

  • Stay pattern-based: The crops shown in the helper layout should display similar content to one another.

  • Avoid fine details: Details that shrink down to single pixels on the NCA grid cannot be reliably reproduced.

Note

Your target texture does not have to be tileable.

Training an NCA model

Start the training using the Cook Output Node button on the ML_Train_Neural_Cellular_Automata subnetwork. Go to the PDG ML Training Monitor to follow the training progress and inspect results on the test page.

Results

After the training concludes, you can add the Neural Cellular Automata COP recipe to create the Neural Cellular Automata Core and Neural Cellular Automata Decode COPs. See Neural Cellular Automata for more information about inference workflows in COPs.

Note

Set the Model parameter on the Neural Cellular Automata Core COP to the exported .onnx file to generate new tileable textures.

Using this tool

To...Do this

Train your own NCA model

  1. Create the ML Train Neural Cellular Automata recipe.

  2. Change the input_pattern node to another RGB layer. This is ideally square with a 1024×1024 resolution.

  3. Check the helper layout to avoid issues with the uploaded target image.

  4. (Only the first time) On the ML_Train_Neural_Cellular_Automata subnetwork, click Download VGG Checkpoint to download the pretrained classification model that computes the style loss.

  5. Click Cook Output Node to initiate the training process.

  6. Open the PDG ML Training Monitor and select the ML_Train_Neural_Cellular_Automata subnetwork. This links the monitor to the training TOP node.

  7. Monitor the training progress through the loss plot. Cycle to other plots to visualize individual loss components over time.

  8. Switch to the test results page to inspect test renders visualizing the current state of the NCA model.

Starting from a checkpoint

  1. In the ML_Train_Neural_Cellular_Automata subnetwork’s Network tab, set NCA Initialization to from Checkpoint.

  2. Set NCA Checkpoint File to a .pt checkpoint on disk.

  3. Set Decoder Initialization to from Checkpoint.

  4. Set Decoder Checkpoint File to a .pt checkpoint on disk.

  5. Click Cook Output Node to initiate the training process.

  6. Open the PDG ML Training Monitor and select the ML_Train_Neural_Cellular_Automata subnetwork. This links the monitor to the training TOP node.

  7. Monitor the training progress through the loss plot. Cycle to other plots to visualize individual loss components over time.

  8. Switch to the test results page to inspect test renders visualizing the current state of the NCA model.

Trainable ML Solutions

ML recipes

ML workflows

ML nodes in other areas