| On this page | |
| Since | 22.0 |
Overview ¶
This node trains a 
Neural Cellular Automata model from a single target texture. The model learns local rules to synthesize tileable pattern-based variants at arbitrary resolutions and aspect
ratios.
This TOP is the training backbone of the COPs recipe and can be a
standalone utility to build other NCA training workflows or automations. See 
ML Train Neural Cellular Automata COP recipe for the simplest way to train an NCA.
Workflow ¶
-
Choose a target texture, either an image file on disk or the output of a COP node. Then, connect the output to this node’s input. See Data set
-
Set up the NCA and decoder models as well as loss, scheduling, and training hyperparameters (see [NCA model|#nca, Decoder model, and Training).
-
Cook the TOP node to launch the training process.
-
Monitor progress with the PDG ML Training Monitor panel and inspect test results (see Monitoring).
-
Load the resulting models back into Houdini and use it in COPs to synthesize tileable textures.
Note
To use this node, click the Download Checkpoint button located under Model > Loss after a fresh Houdini installation. This installs a pretrained VGG-16 checkpoint to compute the style loss.
Tip
Place the virtual environment into a global folder that you can reuse across projects. This way you can avoid reinstalling all the dependencies every time.
Data set ¶
An NCA is trained from a single target image. The image can come from a file on disk or from the output of a COP node (see Input Source). It should ideally be cropped to a square aspect ratio at a resolution of 1024×1024.
The main factor to consider when choosing a target is scale. The NCA evolves cells at a low resolution (128×128), and each cell propagates information to only one neighbour per iteration.
The target does not have to be tileable. The NCA produces tileable results regardless, by wrapping the border cells around to see the opposite side as their neighbours.
See the ML Train Neural Cellular Automata recipe for guidance on selecting a suitable target texture.
NCA model ¶
The NCA model’s purpose is learning all the structural information and rules required to grow a given pattern. It operates at a fairly coarse resolution of 128×128 during training. Its output is a 16 channel latent cell state describing the pattern. It can sometimes look similar to a normal or position pass, but the values don’t have a direct geometric meaning. They are what the model creates to make sense of the patterns' proportions and relationships. An analogy is a coordinate system that helps each cell understand where it is in the larger pattern, because it can only see its direct neighbours.
The single most impactful choice is whether to train from Scratch or from
Checkpoint (see NCA Initialization). Starting from a pretrained .pt checkpoint is almost always the better option. The model already knows how to grow and maintain a stable pattern, so learning a new pattern on top of it is
faster and more stable than starting from random weights. The trade-off is that
you cannot set up a custom network configuration when resuming a checkpoint, as
the architecture is fixed to match the one stored in the file. When training
from scratch, Hidden Channels sets the capacity of the update network. The
full set of NCA parameters is documented under Network in the
parameters reference below.
Decoder model ¶
The decoder model is a separate neural network, which takes in the cell state the NCA produces. It upreses and translates them to the correct RGB values. During training, both models are optimized simultaneously.
Note
During inference, they are loaded into separate COP nodes. This decouples the more expensive evolution of the cell state performed by the
Neural Cellular Automata Core COP
from the rendering of finer details done by the 
Neural Cellular Automata Decode COP.
Like the NCA, the decoder can be initialized from Scratch or from
Checkpoint (see Decoder Initialization). When resuming from an NCA checkpoint, you should load the matching decoder .pt checkpoint alongside
it. As with the NCA, a checkpoint fixes the network architecture so custom
configurations are only available when training from scratch. In that case, the
main decision is the Network Type. SIREN suits smooth targets, while WIRE
handles noisier targets and sharper edges/contrast better. Together with
Omega, which controls how much high-frequency detail the decoder can
represent, these are the parameters worth tuning first. All decoder parameters
are documented under Network in the parameters reference below.
Pool ¶
Growing a pattern with an NCA is an iterative process. The same local update rule is applied over many steps, starting from a blank seed and gradually evolving into the target pattern. Rather than restarting this evolution from scratch on every training step, the node keeps a pool of in-progress simulations alive in memory.
At each training step, a small batch (see Batch Size) is:
-
Drawn from the pool.
-
Advanced forward by a random number of Rollout Steps.
-
Evaluated against the target.
-
And written back with its updated cell state.
Periodically (see Reset Every) one of the sampled states is reset to a blank seed so that freshly seeded simulations keep re-entering the pool.
This ensures that over time the pool consists of a diverse set of differently progressed simulations. This is important to allow the NCA to learn rules that work across the entire range of states from just starting on an empty grid to maintaining fully formed patterns.
Note
In practice this plays out as follows:
Assume the poolsize is N=512, the batch size is B=2, the rollout
steps are a random number R between 96-128 and the pool is reset
every E=32.
The training keeps N=512 NCA simulations alive in memory, while always
picking B=2 and advancing them forward for on average R=~112 iterations
during each training step. Every E=32nd training state is reset back to
a blank seed state.
Partial Training ¶
By default, the node trains to completion in a single cook. With Enable Partial Training on, each cook advances training by at most Max Steps (Single Cook) and then saves its full training state to disk. Subsequent cooks pick up exactly where the previous one left off, as long as the State Base Name matches, until Total Steps is reached across all cooks. This is useful for spreading a long run across a farm or for interleaving arbitrary Houdini processing between training segments when the node is placed inside a TOPs 
feedback loop.
Partial training on this node follows the same pattern as other ML Train TOPs.
See the Partial Training section of the
ML Train Regression node for a fuller explanation
of the workflow.
Monitoring ¶
Once training has started, the PDG ML Training Monitor can see the progression of the total loss and its individual components (overflow, optimal transport style loss, and autocorrelation) over time. Loss plotting is controlled by Plot Loss under the Monitoring tab.
Setting the dropdown menu in the top right to Test Results displays periodic
test textures synthesized during training (see Test Model). Each test is generated with a different number of rollout steps, so the panel shows how the pattern grows and stabilizes over successive NCA iterations. This is a good way to judge whether the model has learned to reach and maintain the target pattern rather than drifting or collapsing over time.
Parameters ¶
Data ¶
Base Directory
The base path that contains all of the inputs, outputs, and virtual environment used during the training process. Outputs are produced in specific subdirectories depending on the type of output file.
Base Name
The name of the subdirectory that contains all outputs produced by this node.
Run Name
The name of the training run. If this node is being wedged out to perform different training runs with different settings, this parameter can create a per-wedge output directory.
Output Directory
The path to the directory that contains all ML models, tests, and plots for the current training run. By default, this parameter is derived from the Base Directory, Base Name, and Run Name parameters, but can be modified to a custom directory path.
Target Size
Resolution of the target texture used for training.
512 x 512
Train at 512×512 resolution. Uses two scale levels (2x, 4x) for the style loss. Requires less GPU memory and trains faster at the cost of upressing performance.
1024 x 1024
Train at 1024×1024 resolution. Uses three scale levels (2x, 4x, 8x) for the style loss.
Input Source
Determines where the target image comes from.
Image File
Load the target image from a file on disk specified by Target Image.
COP Node
Use the output of a COP node as the target image.
Source COP
Path to the COP node whose output is used as the target image. Only available when Input Source is set to COP Node.
Override Frame
When on, overrides the frame at which the COP node is evaluated. Only available when Input Source is set to COP Node.
Override Frame
The frame at which to evaluate the COP node. Only available when Override Frame is on.
Dataset Folder
Directory where the image exported from the COP node is stored. Only available when Input Source is set to COP Node.
Image Base Name
Filename prefix for the image exported from the COP node. Only available when Input Source is set to COP Node.
Target Image
Path to the target texture image on disk. Only available when Input Source is set to Image File.
Model Folder
Directory where the trained ONNX and PyTorch model files are written.
Model Base Name
Filename prefix for exported model files.
Network ¶
NCA Initialization
How the NCA model weights are initialized.
from Scratch
Initialize the NCA with random weights.
from Checkpoint
Load pre-trained weights from a checkpoint file. The network architecture is automatically configured to match the checkpoint.
Tip
You should mostly start from a pretrained checkpoint. The only limitation is you cannot set up custom network configurations, but are forced to use the one in the checkpoint. Once you have a few of those pretrained checkpoints, starting from it improves training speed and stability because the model has already learned to stay stable over time and produce a pattern. The specific pattern is not important as it’s much easier to train a new pattern on top of an existing checkpoint, than it is to start from scratch.
NCA Checkpoint File
Path to a PyTorch .pt file containing pre-trained NCA weights. Only
available when NCA Initialization is set to from Checkpoint.
Offset Initial Step
Only available when NCA Initialization is set to from Checkpoint. Step number to start counting from when resuming from pre-trained weights. This advances the learning rate schedulers by the specified number of steps so that training continues with the appropriate learning rate.
Note
Those steps will be counted as already finished when starting the training. To train for another 1000 steps after adding an offset of 1000., the Total Steps parameter has to be set to 2000.
Pool Warmup Steps
Only available when NCA Initialization is set to from Checkpoint. Number of NCA forward steps to run on each pool element before training begins. This populates the pool with evolved states when starting from pre-trained weights without a full training state checkpoint.
Decoder Initialization
How the decoder model weights are initialized.
from Scratch
Initialize the decoder with random weights.
from Checkpoint
Load pre-trained weights from a checkpoint file. The network architecture is automatically configured to match the checkpoint.
Decoder Checkpoint File
Path to a PyTorch .pt file containing pre-trained decoder weights. Only
available when Decoder Initialization is set to from Checkpoint.
Note
While not mandatory, you should specify the decoder checkpoint that belongs to the NCA checkpoint specified under NCA Checkpoint File.
Network Type
The type of neural network used for the decoder.
SIREN
Uses sinusoidal activation functions (sin(omega * x)). The best
choice for smooth targets without per pixel noise. Controlled by Omega and Hidden Channels.
WIRE
Uses Gabor wavelet activations that combine a sinusoidal component with a Gaussian envelope. WIRE is less sensitive to noise in the target image and can represent sharper edges/contrast better. Controlled by Omega, Gaussian Scale, and Hidden Channels.
Omega
Only available when Decoder Initialization is set to from Scratch and Network Type is SIREN. Frequency parameter for the SIREN sinusoidal activation function. Higher values allow the network to represent higher-frequency details.
Omega
Only available when Decoder Initialization is set to from Scratch and Network Type is WIRE. Frequency parameter for the WIRE Gabor wavelet activation. Higher values allow the network to represent higher-frequency details.
Gaussian Scale
Only available when Decoder Initialization is set to from Scratch and Network Type is WIRE. Scale parameter for the Gaussian envelope in the WIRE activation.
Pool Size
Number of NCA state tensors maintained in the training pool. Each training step samples a random batch from this pool, evolves it, and writes the result back. Larger pools increase diversity but use more memory.
Note
During each training step, a number of states (Batch Size) is advanced forward by a number of Rollout Steps. This is like running an already started simulation for a few more steps. Occasionally some of those simulations are reset depending on the Reset Every parameter. This ensures that over time a pool with a whole range of differently progressed simulations is created. This is important so the NCA learns rules that work across the entire range of states from just starting on an empty lattice to fully formed patterns.
Value Limit
Maximum magnitude for NCA state values. When Use Value Limit is on, values beyond this threshold are soft-clamped using a tanh compression to prevent numerical instability.
Download Checkpoint
Downloads the VGG16 checkpoint file required for the style loss.
VGG Checkpoint
Path to the pre-trained VGG16 weights file used by the style loss. The style loss extracts features from multiple VGG layers and compares them between the generated texture and the target using optimal transport.
Overflow Loss Weight
Weight of the overflow loss term, which penalizes NCA state values outside the -1, 1 range. Increase this value if training becomes unstable.
Tip
When running into numerical instabilities or NANs, try increasing this weight.
Auto Correlation Weight
Weight of the autocorrelation regularization term. When set above zero, an FFT-based circular autocorrelation loss is computed at the coarsest scale to encourage texture coherence.
Random Seed
Seed for the random number generator used when initializing model weights. Using the same seed produces identical initial weights, making training runs reproducible from the start.
Training ¶
Enable Partial Training
When on, each cook advances training by at most Max Steps (Single Cook) rather than running to completion. Subsequent cooks resume from the saved state, picking up exactly where the previous cook left off (as long as State Base Name matches). This is useful when training on a farm, where it limits the work done by a single cook while allowing training to resume later.
Note
Partial training is not required to periodically save or export partially trained models. The controls under the Checkpointing tab lets you write PyTorch and ONNX checkpoints at a regular interval within a single cook without partial training.
Tip
This can run the training in segments in a TOPs for loopallowing arbitrary Houdini processing of the data in between separate cooks. The training will pick up the state of the previous cook. An example is alternating the target texture every N steps to produce a shared NCA latent space for multiple textures. This is useful for blending between two NCA models smoothly in COPs.
Max Steps (Single Cook)
Maximum number of steps to run in a single cook when Enable Partial Training is on. The next cook resumes at the following step and continues until Total Steps is reached over all cooks.
Note
The node only trains from scratch if its output files (model, state, and
log) are not found on disk; otherwise it resumes from them. To force
re-training, 
the node and select Delete this Node’s Output Files from Disk. For a single work item, Delete File Outputs from
Disk.
Total Steps
Total number of training steps. Once this number of steps is reached, either by partial training, or in a single cook the training concludes. To start a new run with the same output directory the output files have to be removed first to prevent overwriting them.
NCA Learning Rate
Learning rate for the NCA model optimizer. Larger steps mean larger parameter updates. Smaller rates may take longer but help avoid skipping locally optimal solutions.
Decoder Learning Rate
Learning rate for the decoder model optimizer. Larger steps mean larger parameter updates. Smaller rates may take longer but help avoid skipping locally optimal solutions.
Reset Every
Every N steps, reset one pool element to a blank seed state. This prevents the pool from drifting too far from initialization and ensures the model learns to generate textures from scratch at all stages.
Rollout Steps
The minimum and maximum number of NCA forward iterations per training step. A random value is chosen from this range each step. Varying the rollout length trains the model to produce stable textures at different evolution stages.
Step Size
Number of training steps between each learning rate decay. The learning rate is multiplied by Step Gamma every Step Size steps.
Step Gamma
Multiplicative factor applied to the learning rate at each decay step. Values less than 1.0 reduce the learning rate over time.
Max Batch Size
Number of pool elements processed per training step. Larger batches produce smoother gradients but use more GPU memory.
Random Seed
Seed for the random number generator used during training. Setting this to a fixed value makes training steps deterministic, allowing runs to be reproduced.
Monitoring ¶
Test Model
When on, generate test texture images at regular intervals during training.
Tests Folder
Directory for test output images.
Test Base Name
Filename prefix for test files.
Test Interval
Generate test textures every N training steps.
Test Count
Number of test images to generate at each test interval. Each test image uses a different number of rollout steps to show how the texture evolves over successive NCA iterations.
Rollout Steps Increment
Step increment between successive test images. The first test uses this many rollout steps, the second uses twice as many, and so on.
Log to Standard Output
Echo log messages to standard output in addition to the log file.
Logs Folder
Directory for training log files.
Log Base Name
Filename prefix for log files.
Log Interval
Write a log entry every N training steps.
Plot Loss
When on, write loss data to a CSV file for visualization. This tracks the total loss as well as individual components (overflow, optimal transport style loss, autocorrelation).
Plots Folder
Directory for training plots.
Plot Base Name
Filename prefix for plot files.
Checkpointing ¶
Checkpoints Folder
Directory for model checkpoint files.
Checkpoint Base Name
Filename prefix for checkpoint files.
Checkpoint Model
When on, export PyTorch .pt checkpoint files at the interval specified by
Model Save Interval.
Model Save Interval
Export a PyTorch .pt checkpoint for both the NCA and decoder models every
N steps.
Checkpoint ONNX
When on, export ONNX checkpoint files at the interval specified by ONNX Export Interval.
ONNX Export Interval
Export an ONNX checkpoint for both the NCA and decoder models every N steps.
States Folder
Directory for training state checkpoints. State checkpoints contain the full training state including model weights, optimizer state, learning rate scheduler state, the pool, and loss history, allowing training to resume exactly where it left off.
State Base Name
Filename prefix for state checkpoint files.
Execution ¶
Cache Mode
Determines how the processor node handles work items that report expected file results.
Automatic
If the expected result file exists on disk, the work item is marked as cooked without being scheduled. If the file does not exist on disk, the work item is scheduled as normal. If upstream work item dependencies write out new files during a cook, the cache files on work items in this node will also be marked as out-of-date.
Automatic (Ignore Upstream)
The same as Automatic, except upstream file writes do not invalidate cache files on work items in this node and this node will only check output files for its own work items.
Read Files
If the expected result file exists on disk, the work item is marked as cooked without being scheduled. Otherwise the work item is marked as failed.
Write Files
Work items are always scheduled and the expected result file is ignored even if it exists on disk.
Environment Path
The path to the virtual environment on disk.
Use Symlinks
Determines whether the virtual environment should be created with symlinks to the source Python binaries. This is recommended on Linux and Mac when creating a virtual environment using Houdini’s Python interpreter.
Never
Always fully copy the Python interpreter to the virtual environment, instead of creating a symlink.
Linux and Mac Only
Create symlinks on Linux and Mac, but copy the interpreter executable on Windows.
Always
Create symlinks on all platforms
Use Pip Cache
When on, pip will attempt to use cached packages on the local system instead of downloading them every time. This can improve the installation times when repeatedly installing the same Python package in different virtual environments.
Override Device
Pin training to a specific device instead of using automatic detection. When not overridden, the device is selected automatically: CUDA if available, then MPS (Apple Silicon), then CPU.
Set CPU Threads
Limit the number of CPU threads used by PyTorch when training on the CPU.
Schedulers ¶
TOP Scheduler Override
Overrides the TOP scheduler for this node.
Schedule When
When on, specifies an expression that determines which work items from the node should be scheduled. If the expression returns zero for a given work item, that work item will immediately be marked as cooked instead of being queued with a scheduler. If the expression returns a non-zero value, the work item is scheduled normally.
Work Item Label
Determines how the node should label its work items. This parameter allows you to assign non-unique label strings to your work items which are then used to identify the work items in the attribute panel, task bar, and scheduler job names.
Use Default Label
The work items in this node will use the default label from the TOP network, or have no label if the default is unset.
Inherit From Upstream Item
The work items inherit their labels from their parent work items.
Custom Expression
The work item label is set to the Label Expression custom expression which is evaluated for each item.
Node Defines Label
The work item label is defined in the node’s internal logic.
Label Expression
When on, specifies a custom label for work items created by this node. The parameter can be an expression that includes references to work item attributes or built-in properties. For example, $OS: @pdg_frame will set the label of each work item based on its frame value.
Work Item Priority
Determines how the current scheduler prioritizes the work items in this node.
Inherit From Upstream Item
The work items inherit their priority from their parent items. If a work item has no parent, its priority is set to 0.
Custom Expression
The work item priority is set to the value of Priority Expression.
Node Defines Priority
The work item priority is set based on the node’s own internal priority calculations.
This option is only available on the 
Python Processor TOP, 
ROP Fetch TOP, and ROP Output TOP nodes. These nodes define their own prioritization schemes that are implemented in their node logic.
Priority Expression
Specifies an expression for work item priority. The expression is evaluated for each work item in the node.
This parameter is only available when Work Item Priority is set to Custom Expression.
| See also |
