| On this page |
Overview ¶
Neural cellular automata (NCA) are a form of cellular automata algorithms. In these algorithms, each cell in a grid runs its own program to determine that cell’s behavior based on the state of its neighbors. Conway’s Game of Life and the Reaction-Diffusion solver are examples of those algorithms.
In neural cellular automata however, each cell in a grid runs its own neural network (instead of a hand authored program) to determine that cell’s behavior based on the state of its neighbors. The network is trained to discover rules suited to the target use case, and learns the local rules that result in a larger pattern. See Self-Organising Textures for a detailed explanation of the architecture.
Note
Zebra stripes are an example of this principle in nature, where each cell in a zebra’s skin seemingly knows what color of hair to grow in order to form the characteristic stripe pattern, though no individual cell can perceive the entire animal. There are also no pixel-perfect copies of the pattern, since no two zebras are identical.
Using NCA in Copernicus ¶
COP nodes ¶
NCA in COPs results in two nodes that work together:
-
Neural Cellular Automata Core: Performs the inference logic and outputs the cell states. -
Neural Cellular Automata Decode: Takes this low-resolution cell information and decodes and upscales it simultaneously.
The use of two nodes improves performance because the rendering of fine detail is separated from the evolution of the cell state, which means the cells operate at a lower resolution before they're decoded into a higher resolution.
Tip
In some workflows you may want two Neural Cellular Automata Decode COPs. One with an Upres Factor of 1x and the other set to the desired output scale. This quickly iterates on a proxy resolution output that already has the correct colors. This way you only have to pay the cost of running the more expensive high resolution decode after finalizing your simulation setup.
Recipes ¶
The following recipes are available in the Copernicus network through the tab menu:
-
Neural Cellular Automata:
Creates Neural Cellular Automata Core and Neural Cellular Automata Decode COPs. The cell state is evolved in place to produce a static texture from the model iterations. This is a good starting point for textures and general image processing workflows.
-
Neural Cellular Automata Block:
Creates Neural Cellular Automata Core and Neural Cellular Automata Decode COPs with a block. This evolves the cells over time. The block wraps only the Neural Cellular Automata Core COP, with the Neural Cellular Automata Decode COP outside the block. This decouples the expensive evolution from the final rendering and runs the cells in a lower resolution before upressing afterwards.
Tip
When growing textures on meshes with adjacency information, use this recipe with the Neural Cellular Automata Core COP’s Model Iterations parameter set to a low value. Make use of the block’s own iterations or simulate over time to make sure the adjacency information is used frequently to update the solve between model iterations.
Controlling NCAs ¶
Beyond simply running a model for a number of iterations, the Neural Cellular Automata Core COP exposes a few controls that let you determine where and how the cells evolve. Each of these can be driven either globally with a parameter or locally with an optional input map for per-cell control.
Update mask ¶
The update mask controls which cells can update on each iteration. Wire a mask into the updatemask input to restrict evolution to specific regions. Only cells in the masked region continue to evolve. This is useful for growing a pattern into a particular area or for locally resetting and regrowing part of an image.
The Update Mask Type parameter sets how the mask is interpreted:
-
Amplitude-Weightedscales the strength of the update by the 0-1 mask value, which produces a smoother, more gradual evolution. -
Probabilisticalways applies a full update, but uses the mask value as the probability that a cell updates at all. This produces noisier results that emphasize the underlying cell structure.
The Update Rate and Seed parameters work alongside the mask. Update Rate sets the fraction of cells that update each step, while the Seed varies which random cells are chosen. This lets you easily generate variations of the same pattern.
Rotation ¶
The rotation input applies a per-cell rotation to the perception kernel, which
effectively rotates the direction in which each cell perceives its neighbors. This
locally rotates the resulting pattern, and lets the texture flow and align along
arbitrary directions across the image. The Cell Rotation parameter adds a
global rotation on top of the input, measured in turns, where a value of 1 is
a full 360° rotation.
Scale ¶
The scale input applies a per-cell scale to the perception gradient, which locally controls the effective feature size of the pattern. Larger scales produce coarser features and smaller scales produce finer detail. The Cell Scale parameter multiplies with the input map.
Adjacency ¶
You can run NCA models on geometry by leveraging the Adjacency workflows to let cells see their neighbors across UV seams. Combine adjacency with the previously mentioned masking, rotation, and scale controls to build intricate effects.
Tip
The default NCA resolution may be too coarse to resolve detail along the UV seams. For best results, you may need to use higher resolutions and only run the model a single iteration per block iteration.
Training an NCA model ¶
The integrated COPs recipe is the easiest starting point to train an NCA model.
Recipe ¶
After creating the recipe, all you need is a pattern-based 1024×1024 RGB target texture. See the 
ML Train Neural Cellular Automata recipe to learn how to use this recipe and train a custom model.
TOP node ¶
Use the ML Train Neural Cellular Automata TOP for automation. This TOP is the core of the COP recipe, but lets you create more complex training setups or configurations.