Houdini 22.0 Nodes TOP nodes

ML Train GSplats TOP node

Trains a 3D Gaussian Splatting model from a set of posed images.

On this page
Since 22.0

Overview

This node trains a 3D Gaussian Splatting model from a set of posed images.

See ML Train GSplats from Karma recipe to get started.

This TOP is the training backbone of the LOPs recipe and can be used as a standalone utility to build other Gaussian splat training workflows or automations.

Workflow

  1. Obtain a posed image data set, either a Houdini (EXR) data set produced by the ML Preprocess GSplats TOP or an external SfM (COLMAP) data set, and point the node at it (see Data set).

  2. Choose how the initial point cloud is seeded and set the densification strategy, loss, and training hyperparameters (see Initialization and Hyperparameters).

  3. Cook the TOP node to launch the training process.

  4. Monitor progress with the PDG ML Training Monitor panel and the live web viewer to track the loss and inspect test renders (see Monitor).

  5. Load the resulting gsplats back into Houdini: sublayer the .usd file in LOPs, or bring the .ply into SOPs and convert it to attributes with Bake GSplats.

Tip

Place the virtual environment into a global folder that you can reuse across projects. This way you can avoid reinstalling all the dependencies every time.

Data set

The ML Train GSplats node supports training on synthetic and captured data. Before starting, you need to obtain and then specify one of two data set types depending on the source.

Classical COLMAP or a COLMAP-like Houdini variant produced by the ML Preprocess GSplats. Fundamentally those share the same structure

  • images/ - the source images.

  • sparse/0/ - the binary COLMAP scene description (cameras.bin, images.bin, points3D.bin).

  • pointcloud/points3d.ply - the initial Gaussian splat point cloud built in Houdini (Only available in data sets exported using the ML Preprocess GSplats TOP).

Houdini’s export extends the standard with a separate initial Gaussian splats .ply file used to initiate the training process. Setting the Data set Type parameter turns on or off a few features that are only suitable for synthetic CG based training or vice versa.

Initialization

Before starting a run, its important to decide how the training is seeded and scaled, as these choices have the biggest impact on the result. The Initialization parameter controls how the initial Gaussian point cloud is created: reuse the Structure-from-Motion point cloud shipped with the data set, resume from a previously saved checkpoint .ply, or scatter a random cloud. When starting from a Houdini data set that includes a points3d.ply file, the Auto Detect Data Set PLY toggle lets the training pick it up as a more accurate starting point than the raw SfM data.

By default, Normalize World Space recenters and rescales the scene based on the training cameras. This improves numerical stability and keeps the hyperparameters meaningful across data sets of different physical sizes. It’s usually best leave it on and only adjust through Global Scale when the default scaling does not suit the scene. For example, outward-facing environment captures.

Densification is the other key decision. With Enable Densification on, the training adaptively adds and removes Gaussians as it optimizes. The Strategy menu chooses between the classic Default scheme and Markov Chain Monte Carlo (MCMC), where Cap Max GS lets you impose a hard maximum on the number of Gaussians. This is especially useful for hitting a specific or low point budget.

Note

The cap only prevents new Gaussians from being added. It can’t remove points that already existed at initialization.

The remaining initialization, densification, and regularization parameters are documented under GSplats in the parameters reference below.

Hyperparameters

The overall length of a run is set by Total Steps. Rather than re-tuning every schedule-driven parameter when you want a shorter or longer run, use Steps Scaler. It multiplies all step-based schedules (densification windows, evaluation, checkpointing, and PLY export) at once, so you can proportionally stretch or shrink a run while keeping its relative timing intact. This makes it easy to do a quick low-step preview and then rerun at full length without touching the individual intervals.

Test Every determines how the data set is split between training and validation. Every Nth image is held out as a test view and the rest are used for training. Because these held-out images are never trained on, this parameter directly trades reconstruction quality against the strength of your validation signal. A smaller value gives more test coverage but leaves less data to train on. While a larger value keeps more images for training at the cost of a sparser test set.

The full set of loss, learning rate, scheduling, and termination parameters is documented under Training in the parameters reference below.

Monitoring

Once the training has started, the PDG ML Training Monitor can show the progression of the total loss as well as separate loss components and the number of Gaussians over time.

Training progress visualized in the TOPs ML Training Monitor

Setting the dropdown menu in the top right to Test Results displays periodic test renders produced from held out perspectives the training has never seen. The panel displays the ground truth input render, the current Gaussian splats rasterized as the training would see them, and a loss heatmap visualizing the differences. This provides a great way of monitoring whether the camera coverage is enough to produce a reconstruction with the desired quality.

Training test results displayed in the PDG ML Training Monitor

Alongside the monitor panel, the node can serve a live web viewer built on viser that lets you inspect the current Gaussian splats interactively in a browser while the training runs. Because the viewer is served from the machine running the training, you can also point the Host parameter at a farm machine and inspect a remote run from your workstation.

Live training viewer utility

The viewer also exposes pause and stop controls for the training. Pausing temporarily halts optimization so you can inspect the current state without the splats changing underneath you. The stop control is useful because it ends the training gracefully and still writes out the current result, which does not happen when you cancel the cook in TOPs. Use it whenever the splats have converged to a satisfactory quality and you want to finish early without losing the output.

Tip

Generally, it’s advisable to produce a data set with more medium resolution images and denser coverage than a sparser data set with high resolution images.

3D Gaussian Splatting is relatively sensitive to big jumps between the viewing angle of two data points.

Parameters

Data

Directory Structure

Base Directory

The base path that contains all of the inputs, outputs, and virtual environment used during the training process. Outputs are produced in specific subdirectories depending on the type of output file.

Base Name

The name of the subdirectory that contains all outputs produced by this node.

Run Name

The name of the training run. If this node is being wedged out to perform different training runs with different settings, this parameter can create a per-wedge output directory.

Output Directory

The path to the directory that contains all ML models, tests, and plots for the current training run. By default, this parameter is derived from the Base Directory, Base Name, and Run Name parameters, but can be modified to a custom directory path.

Input Data Source

data set Folder

Parent directory containing the training data set as a named subfolder.

data set Name

Name of the data set subdirectory inside Data set Folder.

Note

The data set name is kept as its own parameter, separate from Data set Folder, even though it too resolves to a subfolder on disk. Splitting it out this way matches the other ML Train nodes and makes it easy to wedge across different data sets while sharing a common parent folder.

Data Set Type

The format of the input data set.

Houdini (EXRs)

A data set produced by the ML Preprocess GSplats TOP, consisting of EXR images with associated camera and Structure-from-Motion point data in a COLMAP-like format and an optional points3d.ply file containing initial Gaussian splats.

SfM (COLMAP)

A COLMAP data set produced by a Structure-from-Motion pipeline storing images, the reconstruction of the camera poses and an initial point cloud.

Tip

A Structure from Motion COLMAP data set can be created from a set of photographs using several external paid and free tools. Epic’s Reality Scan, which is commonly used for photogrammetry, provides a straightforward workflow for estimating the posed cameras and exporting a COLMAP data set.

Data Downscale Factor

Integer factor used to downscale data set images before training. Higher values reduce memory usage and training time at the cost of detail.

Note

When starting from high-resolution camera images downscaling is recommended to increase performance. It also helps reduce per-pixel noise, which can lead to artifacts.

Tip

When starting the training from scratch, you should first start training with lower resolution images to ensure the training is focused on optimizing the main structure of the Gaussian splats without wasting resources on fine details and data loading. The initial gsplats can be refined in a second pass by reusing the first training run’s output as a starting point and fine tuning it using the high-res images. See Initialization below to learn how to start from a .ply checkpoint.

Extra Features

Multi parameter list of auxiliary AOVs from the Houdini data set (e.g. normals or albedo) to include in training alongside RGB color. Each entry adds an extra per-Gaussian feature channel that is optimized against the corresponding AOV images.

AOV Name

The name of the AOV layer in the multi-layer EXR data set images.

Note

Most AOVs can be trained as extra features into a splat, but there are a few limitations to keep in mind

  • Avoid view dependent AOVs (e.g. specular). The spherical harmonics are reserved for the RGB values.

  • Avoid extremely high dynamic range (e.g. P depending on the scene scale). It makes sense to scale such values before training on them.

  • Avoid contrast rich textures that display different details than the RGB. See more in Geometry Weight.

Signature

Number of channels in the feature: Mono (1), UV (2), or RGB (3).

Loss Weight

Weight for this feature in the loss relative to the RGB term.

Tip

Use the PDG ML Training Monitor to monitor the different loss magnitudes and fine tune the loss weight.

Geometry Weight

How strongly this feature’s gradient influences splat geometry (positions, scales, orients). A value of 0 trains the feature as appearance only, without affecting geometry. This means only the RGBA values of the target EXR image steer the optimization of the geometric components of the Gaussian splat. This leads to the best possible result for the overall appearance, but chokes the optimization of the extra features. Higher values can lead to artifacts in the RGB appearance of the Gaussian splats.

Tip

This setting depends highly on the actual data being fed to the training. It is always a trade-off between RGB quality and extra feature quality. The balance becomes increasingly more difficult with more extra features.

Note

Training for very contrast rich AOVs that contain texture information contrary to the RGB plane’s texture information can lead to artifacts. This happens because the training optimizes the Gaussian splats' appearance by moving around the separate splats to represent a given texture. When the RGB texture provides one alignment signal, but an additional feature such as a checkerboard texture provides a conflicting signal, the splats may align with the checkerboard edges. As a result, the checkerboard pattern can appear in the geometry of the entire model. In these cases, a fine balance between the Geometry Weight and other loss related hyperparameters is required. It also makes sense to aim for a higher point count to allow for more splats in areas that require those different detail levels.

GSplats trained with RGB and an extra feature storing a checkerboard texture leading to artifacts
Input Color Space

Apply Color Space Conversion to C (RGB)

When on, the RGB input is converted through OpenColorIO before being used as a training target. Extra features remain in their source colorspace.

Note

Most viewers expect incoming Gaussian splats to be in sRGB colorspace. Furthermore, the training parameters are calibrated for such values. Turning off this color conversion and training Gaussian splats in other colorspaces should be considered experimental.

Config File

Path to the OCIO configuration file used for the color transform.

Input Space

The colorspace of the data set images.

Training Space

The colorspace the model is trained against.

Output

GSplats Folder

Directory where trained PLY results are written.

GSplats Base Name

Filename prefix for exported gsplats files.

Export USD

Turn on an additional USD export of the trained gsplats.

File Type

USD encoding: binary .usd or ASCII .usda.

GSplats

Scene

Normalize World Space

When on, the scene is recentered and rescaled based on the training cameras before training. This improves numerical stability and keeps hyperparameters such as learning rates consistent across data sets of different physical sizes.

Global Scale

A global scaler that applies to all scene-size-related parameters.

Tip

The normalization applies scaling based on the camera positions in the data set. The default global scale value works well for scenes where the cameras surround an object. However, when the cameras are in the center and point outwards, they produce a larger scene than the normalization scheme captures. This happens often when capturing environments. Those cases may require adjusting the global scale to lower values.

Initialization

Initialization

How the initial Gaussian point cloud is seeded.

Point Cloud (SfM)

Use the Structure-from-Motion point cloud from the data set.

from Checkpoint PLY

Resume training from a previously saved checkpoint PLY file.

Random

Generate a random cloud inside the initial extent.

Auto Detect Data Set PLY

When on, a /pointcloud/points3d.ply file found in the data set folder is used as the initial point cloud instead of the SfM data. This allows feeding more accurate initialization data to the training that already includes more Gaussian splat properties beyond color and position in the regular SfM data set. The color is expected to be in the same colorspace as the source EXR images.

Checkpoint File

Path to the checkpoint PLY file to resume training from.

Offset Initial Step

Step number to start counting from when resuming. This affects schedule- driven parameters such as learning rate decay and densification windows. It makes sense to set this to the previous training’s final step count.

Note

Those steps will be counted as already finished when starting the training. To train for another 1000 steps after adding an offset of 1000 the Total Steps parameter has to be set to 2000 for example.

Initial Number of Points

Number of Gaussians created for random initialization.

Initial Extent

Size of the cube, in world units, used to scatter random initial points.

Initial Opacity

Starting opacity for each Gaussian when using the Default strategy.

Initial Scale

Starting scale for each Gaussian when using the Default strategy, expressed as a multiplier relative to the local point density.

Initial Opacity

Starting opacity for each Gaussian when using the MCMC strategy.

Initial Scale

Starting scale for each Gaussian when using the MCMC strategy.

Random Seed

Random seed for parameter initialization (quaternions, scales, features, random points).

Densification

Enable Densification

When on, adaptive densification and pruning are performed during training. When off, only the parameters of the initial Gaussians are optimized. No points are added or removed.

Tip

This can retrain already existing Gaussian splats with new lighting, different textures or slight deformation, while maintaining a constant point count.

Strategy

Densification algorithm.

Default

The densification strategy from the original 3D Gaussian Splatting paper: duplicate under-reconstructed Gaussians, split over-large ones, and prune low-opacity or oversized Gaussians, with periodic opacity resets.

Markov Chain Monte Carlo

The MCMC strategy, which treats densification as sampling from a stationary distribution: relocate dead Gaussians, add new Gaussians drawn from the opacity distribution, and inject positional noise. This provides a way to cap the number of Gaussians to a fixed maximum. This is useful when trying to create gsplats with a specific or low point budget.

Note

The cap only limits whether new Gaussians are added. It cannot remove already existing ones if you start out with a higher count than your target maximum.

Prune Opacity

Gaussians with opacity below this threshold are pruned (Default strategy).

Cap Max GS

Maximum number of Gaussians allowed during training (MCMC strategy). No new points will be added beyond this count. However, points that already existed during initialization will not be removed to reach that point count. To target a specific count the initial input cannot have more than the specified max value.

Min Opacity

Opacity threshold below which Gaussians are considered dead and get relocated (MCMC strategy).

Noise Learning Rate

Amplitude of the positional noise injected by the MCMC strategy at each step (MCMC strategy).

Advanced

Grow Grad 2D

Screen-space gradient threshold above which a Gaussian is considered under-reconstructed and densified (Default strategy).

Grow Scale 3D

World-space scale threshold separating duplication from splitting: Gaussians smaller than this are duplicated, larger are split (Default strategy).

Grow Scale 2D

Screen-space scale threshold used to also trigger splitting for Gaussians that project to large areas in the image (Default strategy).

Prune Scale 3D

Gaussians whose world-space scale exceeds this fraction of the scene extent are pruned (Default strategy).

Prune Scale 2D

Gaussians whose projected screen-space scale exceeds this fraction are pruned (Default strategy).

Use Absolute Gradients

Use absolute screen-space gradients (AbsGS) instead of signed gradients when evaluating the growth criterion. Typically requires a larger Grow Grad 2D value (Default strategy).

Revised Opacity

Use the revised opacity heuristic when splitting/duplicating so the combined opacity of new Gaussians matches the original (Default strategy).

Loss

SSIM Lambda

Blend factor between L1 and SSIM in the photometric loss. The total loss is (1 - lambda) * L1 + lambda * SSIM.

Alpha Loss

Controls how image alpha is used in the loss.

Off

Alpha is ignored.

Direct: Supervision in Loss Term

An extra loss term directly supervises the rendered alpha against the data set alpha.

Indirect: Random Background Composite

Each iteration composites the data set image over a random background color using its alpha, and the rendered image is composited the same way. This supervises alpha implicitly through the RGB loss and is computationally less expensive, while delivering on par quality in most cases.

Alpha Lambda

Weight of the direct alpha loss term.

Boost Edge Loss

Multiplier applied to the loss near alpha edges when alpha supervision is active. Higher values push the model to resolve silhouettes more accurately.

Regularizer

Opacity Regularization

L1 regularization weight applied to Gaussian opacities (Default strategy).

Scale Regularization

L1 regularization weight applied to Gaussian scales (Default strategy).

Opacity Regularization

L1 regularization weight applied to Gaussian opacities (MCMC strategy).

Scale Regularization

L1 regularization weight applied to Gaussian scales (MCMC strategy).

Training

Data Preparation

Random Seed

Random seed for the data set sampling order (image selection on each step).

Validation

Enable Testing

Run test evaluations on held-out views during training.

Test Every

Every Nth image in the data set is held out as a test image. The remaining images are used for training.

Termination

Total Steps

Total number of training steps.

Steps Scaler

Multiplier applied to all step-based schedules (densification windows, evaluation, checkpointing, PLY export). Useful for proportionally lengthening or shortening a run without re-tuning every individual interval.

Optimization
Overrides

Tip

Similar to turning off the densification and through that maintaining a constant point count the following toggles can constrain certain Gaussian splat properties to their initial values. For example, the position to ensure points come back exactly like they were passed into the training. It’s best to use these overrides with densification turned off.

Disable Position Optimization

Freeze Gaussian positions.

Disable Color Optimization

Freeze Gaussian color and spherical harmonics coefficients.

Disable Opacity Optimization

Freeze Gaussian opacities.

Disable Scale Optimization

Freeze Gaussian scales.

Disable Rotation Optimization

Freeze Gaussian rotations.

Disable Feature Optimization

Freeze extra feature channels.

Position Learning Rate

Learning rate for Gaussian positions. This rate is scaled by the scene extent when Normalize World Space is on.

Color Learning Rate

Learning rate for the diffuse RGB component (band 0 of the spherical harmonics coefficients).

SPH Learning Rate

Learning rate for higher-order spherical harmonics coefficients.

Opacity Learning Rate

Learning rate for Gaussian opacities.

Scale Learning Rate

Learning rate for Gaussian scales.

Rotation Learning Rate

Learning rate for Gaussian rotations.

Feature Learning Rate

Learning rate for extra features.

Scheduling

SPH Degree Interval

Every N steps, increase the active spherical harmonics degree by one up to the maximum. Lower degrees are trained first to stabilize color before higher frequency directional effects are introduced.

Densification Schedule

Refine Start Iter

Step at which densification begins (Default strategy).

Refine Stop Iter

Step at which densification ends (Default strategy). The remaining steps only refine the existing Gaussians.

Reset Every

Every N steps, reset Gaussian opacities to a small value so that unused Gaussians can be pruned (Default strategy).

Note

This feature is most useful when training on real world images to clear floaters. When training on “perfect” CG data it can be turned off, by setting the iteration value higher than the Total Steps.

Refine Every

Perform a densification pass every N steps (Default strategy).

Refine Scale 2D Stop Iter

Step at which 2D-scale-based pruning stops. 0 disables this criterion (Default strategy).

Pause Refine after Reset

Number of steps to skip densification after an opacity reset, allowing the optimizer to stabilize. The default value of 0 means no pause at all, but one might want to set this parameter to the number of images in the training set (total images in the data set - test images) to avoid pruning Gaussians before they could recover after a reset. This can occur with very low prune opacity thresholds (Default strategy).

Refine Start Iter

Step at which densification begins (MCMC strategy).

Refine Stop Iter

Step at which densification ends (MCMC strategy).

Refine Every

Perform an MCMC refinement step every N steps.

Training Granularity

Max Batch Size

Number of images processed per training step. Larger batches produce smoother gradients but use more GPU memory.

Randomization Source

Random Seed

Random seed for per-step training randomness (random backgrounds, strategy split/noise).

Monitoring

Viewer

Enable Viewer

Start the interactive gsplat web viewer during training so the current model can be inspected live in a web browser.

Open Viewer

Open the viewer URL in the default web browser.

Note

If the viewer doesn’t load correctly in your browser make sure to disable any extensions that could interfere with it. It is recommended to use Google Chrome.

Automatically Open Viewer

When on, automatically open the viewer URL in the default browser when training starts.

Host

Host name the open viewer button targets. The host is always the machine running the training. This allows you to specify the IP of the training computer and open the viewer directly. This can be useful when running the training on a farm machine, but inspecting it from a work station.

Port

TCP port the viewer attempts to serve on.

Note

If the port isn’t available the port number will be incremented until a free port is found. The new port will be printed in the training log on the work item.

Test GSplats

Save Renders

Save rendered validation images at each test step.

Save Video

Encode a flythrough video from the test renders.

Tests Folder

Directory for test outputs (metrics and rendered views).

Test Base Name

Filename prefix for test files.

Test Interval

Run test evaluation every N training steps.

Log

Log to Standard Output

Echo log messages to standard output in addition to the log file.

Verbose Densification Logging

Log detailed information for each densification operation.

Logs Folder

Directory for training log files.

Log Base Name

Filename prefix for log files.

Log Interval

Write a log entry every N training steps.

Plot Loss

Plots Folder

Directory for training plots.

Plot Base Name

Filename prefix for plot files.

Checkpointing

Checkpoints

Checkpoints Folder

Directory for PLY and USD checkpoint files.

Checkpoint Base Name

Filename prefix for checkpoint files.

Checkpoint PLY

When on, export PLY checkpoint files at the interval specified by PLY Export Interval.

PLY Export Interval

Export a PLY checkpoint every N steps.

Checkpoint USD

When on, export USD checkpoint files at the interval specified by USD Export Interval.

USD Export Interval

Export a USD checkpoint every N steps.

State

States Folder

Directory for training state checkpoints (model + optimizer state).

State Base Name

Filename prefix for state checkpoint files.

Execution

Virtual Environment

Cache Mode

Determines how the processor node handles work items that report expected file results.

Automatic

If the expected result file exists on disk, the work item is marked as cooked without being scheduled. If the file does not exist on disk, the work item is scheduled as normal. If upstream work item dependencies write out new files during a cook, the cache files on work items in this node will also be marked as out-of-date.

Automatic (Ignore Upstream)

The same as Automatic, except upstream file writes do not invalidate cache files on work items in this node and this node will only check output files for its own work items.

Read Files

If the expected result file exists on disk, the work item is marked as cooked without being scheduled. Otherwise the work item is marked as failed.

Write Files

Work items are always scheduled and the expected result file is ignored even if it exists on disk.

Environment Path

The path to the virtual environment on disk.

Use Pip Cache

When on, pip will attempt to use cached packages on the local system instead of downloading them every time. This can improve the installation times when repeatedly installing the same Python package in different virtual environments.

Data Loading

Cache Images to VRAM

Keep decoded training images resident on the GPU. Significantly speeds up training at the cost of VRAM.

Convert EXRs to NPZ

Convert data set EXRs to compressed NumPy .npz files once up front for faster subsequent loads. Only applies to Houdini data sets.

Number of Workers

Strategy for choosing the number of data-loading worker processes.

Equal to 1/4 of Total CPU Count

Use one quarter of the available CPU cores.

Equal to CPU Count Less One

Use all CPU cores minus one.

Custom Worker Count

Use the value from Number of Workers.

Number of Workers

Explicit worker count when Number of Workers is set to Custom Worker Count.

Training Device

Override GPU

Pin training to a specific GPU instead of using the default device.

GPU

Index of the GPU to use when Override GPU is on.

Schedulers

TOP Scheduler Override

Overrides the TOP scheduler for this node.

Schedule When

When on, specifies an expression that determines which work items from the node should be scheduled. If the expression returns zero for a given work item, that work item will immediately be marked as cooked instead of being queued with a scheduler. If the expression returns a non-zero value, the work item is scheduled normally.

Work Item Label

Determines how the node should label its work items. This parameter allows you to assign non-unique label strings to your work items which are then used to identify the work items in the attribute panel, task bar, and scheduler job names.

Use Default Label

The work items in this node will use the default label from the TOP network, or have no label if the default is unset.

Inherit From Upstream Item

The work items inherit their labels from their parent work items.

Custom Expression

The work item label is set to the Label Expression custom expression which is evaluated for each item.

Node Defines Label

The work item label is defined in the node’s internal logic.

Label Expression

When on, specifies a custom label for work items created by this node. The parameter can be an expression that includes references to work item attributes or built-in properties. For example, $OS: @pdg_frame will set the label of each work item based on its frame value.

Work Item Priority

Determines how the current scheduler prioritizes the work items in this node.

Inherit From Upstream Item

The work items inherit their priority from their parent items. If a work item has no parent, its priority is set to 0.

Custom Expression

The work item priority is set to the value of Priority Expression.

Node Defines Priority

The work item priority is set based on the node’s own internal priority calculations.

This option is only available on the Python Processor TOP, ROP Fetch TOP, and ROP Output TOP nodes. These nodes define their own prioritization schemes that are implemented in their node logic.

Priority Expression

Specifies an expression for work item priority. The expression is evaluated for each work item in the node.

This parameter is only available when Work Item Priority is set to Custom Expression.

See also

TOP nodes