| On this page | |
| Since | 22.0 |
Overview ¶
This node trains a 3D Gaussian Splatting model from a set of posed images.
See ML Train GSplats from Karma recipe to get started.
This TOP is the training backbone of the LOPs recipe and can be used as a standalone utility to build other Gaussian splat training workflows or automations.
Workflow ¶
-
Obtain a posed image data set, either a Houdini (EXR) data set produced by the
ML Preprocess GSplats TOP or an external SfM (COLMAP) data set, and point the node at it (see Data set).
-
Choose how the initial point cloud is seeded and set the densification strategy, loss, and training hyperparameters (see Initialization and Hyperparameters).
-
Cook the TOP node to launch the training process.
-
Monitor progress with the PDG ML Training Monitor panel and the live web viewer to track the loss and inspect test renders (see Monitor).
-
Load the resulting gsplats back into Houdini: sublayer the
.usdfile in LOPs, or bring the.plyinto SOPs and convert it to attributes withBake GSplats.
Tip
Place the virtual environment into a global folder that you can reuse across projects. This way you can avoid reinstalling all the dependencies every time.
Data set ¶
The ML Train GSplats node supports training on synthetic and captured data. Before starting, you need to obtain and then specify one of two data set types depending on the source.
Houdini’s export extends the standard with a separate initial Gaussian splats .ply file used to initiate the training process. Setting the Data set Type
parameter turns on or off a few features that are only suitable for synthetic CG based training or vice versa.
Initialization ¶
Before starting a run, its important to decide how the training is seeded and scaled, as these choices have the biggest impact on the result. The Initialization
parameter controls how the initial Gaussian point cloud is created: reuse the Structure-from-Motion point cloud shipped with the data set, resume from a previously saved checkpoint .ply, or scatter a random cloud. When starting from a Houdini data set that includes a points3d.ply file, the Auto Detect Data Set PLY toggle lets the training pick it up as a more accurate starting point than the raw SfM data.
By default, Normalize World Space recenters and rescales the scene based on the training cameras. This improves numerical stability and keeps the hyperparameters meaningful across data sets of different physical sizes. It’s usually best leave it on and only adjust through Global Scale when the default scaling does not suit the scene. For example, outward-facing environment captures.
Densification is the other key decision. With Enable Densification on, the training adaptively adds and removes Gaussians as it optimizes. The Strategy menu chooses between the classic Default scheme and Markov Chain Monte Carlo (MCMC), where Cap Max GS lets you impose a hard maximum on the number of Gaussians. This is especially useful for hitting a specific or low point budget.
Note
The cap only prevents new Gaussians from being added. It can’t remove points that already existed at initialization.
The remaining initialization, densification, and regularization parameters are documented under GSplats in the parameters reference below.
Hyperparameters ¶
The overall length of a run is set by Total Steps. Rather than re-tuning every schedule-driven parameter when you want a shorter or longer run, use Steps Scaler. It multiplies all step-based schedules (densification windows, evaluation, checkpointing, and PLY export) at once, so you can proportionally stretch or shrink a run while keeping its relative timing intact. This makes it easy to do a quick low-step preview and then rerun at full length without touching the individual intervals.
Test Every determines how the data set is split between training and validation. Every Nth image is held out as a test view and the rest are used for training. Because these held-out images are never trained on, this parameter directly trades reconstruction quality against the strength of your validation signal. A smaller value gives more test coverage but leaves less data to train on. While a larger value keeps more images for training at the cost of a sparser test set.
The full set of loss, learning rate, scheduling, and termination parameters is documented under Training in the parameters reference below.
Monitoring ¶
Once the training has started, the PDG ML Training Monitor can show the progression of the total loss as well as separate loss components and the number of Gaussians over time.
Setting the dropdown menu in the top right to Test Results displays periodic test renders produced from held out perspectives the training has never seen. The panel displays the ground truth input render, the current Gaussian splats rasterized as the training would see them, and a loss heatmap visualizing the differences. This provides a great way of monitoring whether the camera coverage is enough to produce a reconstruction with the desired quality.
Alongside the monitor panel, the node can serve a live web viewer built on viser that lets you inspect the current Gaussian splats interactively in a browser while the training runs. Because the viewer is served from the machine running the training, you can also point the Host parameter at a farm machine and inspect a remote run from your workstation.
The viewer also exposes pause and stop controls for the training. Pausing temporarily halts optimization so you can inspect the current state without the splats changing underneath you. The stop control is useful because it ends the training gracefully and still writes out the current result, which does not happen when you cancel the cook in TOPs. Use it whenever the splats have converged to a satisfactory quality and you want to finish early without losing the output.
Tip
Generally, it’s advisable to produce a data set with more medium resolution images and denser coverage than a sparser data set with high resolution images.
3D Gaussian Splatting is relatively sensitive to big jumps between the viewing angle of two data points.
Parameters ¶
Data ¶
Base Directory
The base path that contains all of the inputs, outputs, and virtual environment used during the training process. Outputs are produced in specific subdirectories depending on the type of output file.
Base Name
The name of the subdirectory that contains all outputs produced by this node.
Run Name
The name of the training run. If this node is being wedged out to perform different training runs with different settings, this parameter can create a per-wedge output directory.
Output Directory
The path to the directory that contains all ML models, tests, and plots for the current training run. By default, this parameter is derived from the Base Directory, Base Name, and Run Name parameters, but can be modified to a custom directory path.
data set Folder
Parent directory containing the training data set as a named subfolder.
data set Name
Name of the data set subdirectory inside Data set Folder.
Note
The data set name is kept as its own parameter, separate from Data set Folder, even though it too resolves to a subfolder on disk. Splitting it out this way matches the other ML Train nodes and makes it easy to wedge across different data sets while sharing a common parent folder.
Data Set Type
The format of the input data set.
Houdini (EXRs)
A data set produced by the ML Preprocess GSplats
TOP, consisting of EXR images with associated camera and
Structure-from-Motion point data in a COLMAP-like format and an optional
points3d.ply file containing initial Gaussian splats.
SfM (COLMAP)
A COLMAP data set produced by a Structure-from-Motion pipeline storing images, the reconstruction of the camera poses and an initial point cloud.
Tip
A Structure from Motion COLMAP data set can be created from a set of photographs using several external paid and free tools. Epic’s Reality Scan, which is commonly used for photogrammetry, provides a straightforward workflow for estimating the posed cameras and exporting a COLMAP data set.
Data Downscale Factor
Integer factor used to downscale data set images before training. Higher values reduce memory usage and training time at the cost of detail.
Note
When starting from high-resolution camera images downscaling is recommended to increase performance. It also helps reduce per-pixel noise, which can lead to artifacts.
Tip
When starting the training from scratch, you should first start
training with lower resolution images to ensure the training is focused
on optimizing the main structure of the Gaussian splats without wasting
resources on fine details and data loading. The initial gsplats can be
refined in a second pass by reusing the first training run’s output as a
starting point and fine tuning it using the high-res images. See
Initialization below to learn how to start from a .ply checkpoint.
Extra Features
Multi parameter list of auxiliary AOVs from the Houdini data set (e.g. normals or albedo) to include in training alongside RGB color. Each entry adds an extra per-Gaussian feature channel that is optimized against the corresponding AOV images.
AOV Name
The name of the AOV layer in the multi-layer EXR data set images.
Note
Most AOVs can be trained as extra features into a splat, but there are a few limitations to keep in mind
-
Avoid view dependent AOVs (e.g.
specular). The spherical harmonics are reserved for the RGB values. -
Avoid extremely high dynamic range (e.g.
Pdepending on the scene scale). It makes sense to scale such values before training on them. -
Avoid contrast rich textures that display different details than the RGB. See more in Geometry Weight.
Signature
Number of channels in the feature: Mono (1), UV (2), or RGB (3).
Loss Weight
Weight for this feature in the loss relative to the RGB term.
Tip
Use the PDG ML Training Monitor to monitor the different loss magnitudes and fine tune the loss weight.
Geometry Weight
How strongly this feature’s gradient influences splat geometry (positions,
scales, orients). A value of 0 trains the feature as appearance only,
without affecting geometry. This means only the RGBA values of the
target EXR image steer the optimization of the geometric components of
the Gaussian splat. This leads to the best possible result for the overall
appearance, but chokes the optimization of the extra features. Higher
values can lead to artifacts in the RGB appearance of the Gaussian splats.
Tip
This setting depends highly on the actual data being fed to the training. It is always a trade-off between RGB quality and extra feature quality. The balance becomes increasingly more difficult with more extra features.
Note
Training for very contrast rich AOVs that contain texture information
contrary to the RGB plane’s texture information can lead to artifacts.
This happens because the training optimizes the Gaussian splats'
appearance by moving around the separate splats to represent a given
texture. When the RGB texture provides one alignment signal, but an additional feature such as a checkerboard texture provides a conflicting signal, the splats may align with the checkerboard edges. As a result, the checkerboard pattern can appear in the geometry of the entire model. In these cases, a fine balance between the
Geometry Weight and other loss related hyperparameters is required.
It also makes sense to aim for a higher point count to allow for more
splats in areas that require those different detail levels.
Apply Color Space Conversion to C (RGB)
When on, the RGB input is converted through OpenColorIO before being used as a training target. Extra features remain in their source colorspace.
Note
Most viewers expect incoming Gaussian splats to be in sRGB colorspace. Furthermore, the training parameters are calibrated for such values. Turning off this color conversion and training Gaussian splats in other colorspaces should be considered experimental.
Config File
Path to the OCIO configuration file used for the color transform.
Input Space
The colorspace of the data set images.
Training Space
The colorspace the model is trained against.
GSplats Folder
Directory where trained PLY results are written.
GSplats Base Name
Filename prefix for exported gsplats files.
Export USD
Turn on an additional USD export of the trained gsplats.
File Type
USD encoding: binary .usd or ASCII .usda.
GSplats ¶
Normalize World Space
When on, the scene is recentered and rescaled based on the training cameras before training. This improves numerical stability and keeps hyperparameters such as learning rates consistent across data sets of different physical sizes.
Global Scale
A global scaler that applies to all scene-size-related parameters.
Tip
The normalization applies scaling based on the camera positions in the data set. The default global scale value works well for scenes where the cameras surround an object. However, when the cameras are in the center and point outwards, they produce a larger scene than the normalization scheme captures. This happens often when capturing environments. Those cases may require adjusting the global scale to lower values.
Initialization
How the initial Gaussian point cloud is seeded.
Point Cloud (SfM)
Use the Structure-from-Motion point cloud from the data set.
from Checkpoint PLY
Resume training from a previously saved checkpoint PLY file.
Random
Generate a random cloud inside the initial extent.
Auto Detect Data Set PLY
When on, a /pointcloud/points3d.ply file found in the data set folder is
used as the initial point cloud instead of the SfM data. This allows feeding
more accurate initialization data to the training that already includes more
Gaussian splat properties beyond color and position in the regular SfM
data set. The color is expected to be in the same colorspace as the source
EXR images.
Checkpoint File
Path to the checkpoint PLY file to resume training from.
Offset Initial Step
Step number to start counting from when resuming. This affects schedule- driven parameters such as learning rate decay and densification windows. It makes sense to set this to the previous training’s final step count.
Note
Those steps will be counted as already finished when starting the training. To train for another 1000 steps after adding an offset of 1000 the Total Steps parameter has to be set to 2000 for example.
Initial Number of Points
Number of Gaussians created for random initialization.
Initial Extent
Size of the cube, in world units, used to scatter random initial points.
Initial Opacity
Starting opacity for each Gaussian when using the Default strategy.
Initial Scale
Starting scale for each Gaussian when using the Default strategy, expressed as a multiplier relative to the local point density.
Initial Opacity
Starting opacity for each Gaussian when using the MCMC strategy.
Initial Scale
Starting scale for each Gaussian when using the MCMC strategy.
Random Seed
Random seed for parameter initialization (quaternions, scales, features, random points).
Enable Densification
When on, adaptive densification and pruning are performed during training. When off, only the parameters of the initial Gaussians are optimized. No points are added or removed.
Tip
This can retrain already existing Gaussian splats with new lighting, different textures or slight deformation, while maintaining a constant point count.
Strategy
Densification algorithm.
Default
The densification strategy from the original 3D Gaussian Splatting paper: duplicate under-reconstructed Gaussians, split over-large ones, and prune low-opacity or oversized Gaussians, with periodic opacity resets.
Markov Chain Monte Carlo
The MCMC strategy, which treats densification as sampling from a stationary distribution: relocate dead Gaussians, add new Gaussians drawn from the opacity distribution, and inject positional noise. This provides a way to cap the number of Gaussians to a fixed maximum. This is useful when trying to create gsplats with a specific or low point budget.
Note
The cap only limits whether new Gaussians are added. It cannot remove already existing ones if you start out with a higher count than your target maximum.
Prune Opacity
Gaussians with opacity below this threshold are pruned (Default strategy).
Cap Max GS
Maximum number of Gaussians allowed during training (MCMC strategy). No new points will be added beyond this count. However, points that already existed during initialization will not be removed to reach that point count. To target a specific count the initial input cannot have more than the specified max value.
Min Opacity
Opacity threshold below which Gaussians are considered dead and get relocated (MCMC strategy).
Noise Learning Rate
Amplitude of the positional noise injected by the MCMC strategy at each step (MCMC strategy).
Grow Grad 2D
Screen-space gradient threshold above which a Gaussian is considered under-reconstructed and densified (Default strategy).
Grow Scale 3D
World-space scale threshold separating duplication from splitting: Gaussians smaller than this are duplicated, larger are split (Default strategy).
Grow Scale 2D
Screen-space scale threshold used to also trigger splitting for Gaussians that project to large areas in the image (Default strategy).
Prune Scale 3D
Gaussians whose world-space scale exceeds this fraction of the scene extent are pruned (Default strategy).
Prune Scale 2D
Gaussians whose projected screen-space scale exceeds this fraction are pruned (Default strategy).
Use Absolute Gradients
Use absolute screen-space gradients (AbsGS) instead of signed gradients when evaluating the growth criterion. Typically requires a larger Grow Grad 2D value (Default strategy).
Revised Opacity
Use the revised opacity heuristic when splitting/duplicating so the combined opacity of new Gaussians matches the original (Default strategy).
SSIM Lambda
Blend factor between L1 and SSIM in the photometric loss. The total loss
is (1 - lambda) * L1 + lambda * SSIM.
Alpha Loss
Controls how image alpha is used in the loss.
Off
Alpha is ignored.
Direct: Supervision in Loss Term
An extra loss term directly supervises the rendered alpha against the data set alpha.
Indirect: Random Background Composite
Each iteration composites the data set image over a random background color using its alpha, and the rendered image is composited the same way. This supervises alpha implicitly through the RGB loss and is computationally less expensive, while delivering on par quality in most cases.
Alpha Lambda
Weight of the direct alpha loss term.
Boost Edge Loss
Multiplier applied to the loss near alpha edges when alpha supervision is active. Higher values push the model to resolve silhouettes more accurately.
Opacity Regularization
L1 regularization weight applied to Gaussian opacities (Default strategy).
Scale Regularization
L1 regularization weight applied to Gaussian scales (Default strategy).
Opacity Regularization
L1 regularization weight applied to Gaussian opacities (MCMC strategy).
Scale Regularization
L1 regularization weight applied to Gaussian scales (MCMC strategy).
Training ¶
Random Seed
Random seed for the data set sampling order (image selection on each step).
Enable Testing
Run test evaluations on held-out views during training.
Test Every
Every Nth image in the data set is held out as a test image. The remaining images are used for training.
Total Steps
Total number of training steps.
Steps Scaler
Multiplier applied to all step-based schedules (densification windows, evaluation, checkpointing, PLY export). Useful for proportionally lengthening or shortening a run without re-tuning every individual interval.
Tip
Similar to turning off the densification and through that maintaining a constant point count the following toggles can constrain certain Gaussian splat properties to their initial values. For example, the position to ensure points come back exactly like they were passed into the training. It’s best to use these overrides with densification turned off.
Disable Position Optimization
Freeze Gaussian positions.
Disable Color Optimization
Freeze Gaussian color and spherical harmonics coefficients.
Disable Opacity Optimization
Freeze Gaussian opacities.
Disable Scale Optimization
Freeze Gaussian scales.
Disable Rotation Optimization
Freeze Gaussian rotations.
Disable Feature Optimization
Freeze extra feature channels.
Position Learning Rate
Learning rate for Gaussian positions. This rate is scaled by the scene extent when Normalize World Space is on.
Color Learning Rate
Learning rate for the diffuse RGB component (band 0 of the spherical harmonics coefficients).
SPH Learning Rate
Learning rate for higher-order spherical harmonics coefficients.
Opacity Learning Rate
Learning rate for Gaussian opacities.
Scale Learning Rate
Learning rate for Gaussian scales.
Rotation Learning Rate
Learning rate for Gaussian rotations.
Feature Learning Rate
Learning rate for extra features.
SPH Degree Interval
Every N steps, increase the active spherical harmonics degree by one up to the maximum. Lower degrees are trained first to stabilize color before higher frequency directional effects are introduced.
Refine Start Iter
Step at which densification begins (Default strategy).
Refine Stop Iter
Step at which densification ends (Default strategy). The remaining steps only refine the existing Gaussians.
Reset Every
Every N steps, reset Gaussian opacities to a small value so that unused Gaussians can be pruned (Default strategy).
Note
This feature is most useful when training on real world images to clear floaters. When training on “perfect” CG data it can be turned off, by setting the iteration value higher than the Total Steps.
Refine Every
Perform a densification pass every N steps (Default strategy).
Refine Scale 2D Stop Iter
Step at which 2D-scale-based pruning stops. 0 disables this criterion
(Default strategy).
Pause Refine after Reset
Number of steps to skip densification after an opacity reset, allowing the optimizer to stabilize. The default value of 0 means no pause at all, but one might want to set this parameter to the number of images in the training set (total images in the data set - test images) to avoid pruning Gaussians before they could recover after a reset. This can occur with very low prune opacity thresholds (Default strategy).
Refine Start Iter
Step at which densification begins (MCMC strategy).
Refine Stop Iter
Step at which densification ends (MCMC strategy).
Refine Every
Perform an MCMC refinement step every N steps.
Max Batch Size
Number of images processed per training step. Larger batches produce smoother gradients but use more GPU memory.
Random Seed
Random seed for per-step training randomness (random backgrounds, strategy split/noise).
Monitoring ¶
Enable Viewer
Start the interactive gsplat web viewer during training so the current model can be inspected live in a web browser.
Open Viewer
Open the viewer URL in the default web browser.
Note
If the viewer doesn’t load correctly in your browser make sure to disable any extensions that could interfere with it. It is recommended to use Google Chrome.
Automatically Open Viewer
When on, automatically open the viewer URL in the default browser when training starts.
Host
Host name the open viewer button targets. The host is always the machine running the training. This allows you to specify the IP of the training computer and open the viewer directly. This can be useful when running the training on a farm machine, but inspecting it from a work station.
Port
TCP port the viewer attempts to serve on.
Note
If the port isn’t available the port number will be incremented until a free port is found. The new port will be printed in the training log on the work item.
Save Renders
Save rendered validation images at each test step.
Save Video
Encode a flythrough video from the test renders.
Tests Folder
Directory for test outputs (metrics and rendered views).
Test Base Name
Filename prefix for test files.
Test Interval
Run test evaluation every N training steps.
Log to Standard Output
Echo log messages to standard output in addition to the log file.
Verbose Densification Logging
Log detailed information for each densification operation.
Logs Folder
Directory for training log files.
Log Base Name
Filename prefix for log files.
Log Interval
Write a log entry every N training steps.
Plots Folder
Directory for training plots.
Plot Base Name
Filename prefix for plot files.
Checkpointing ¶
Checkpoints Folder
Directory for PLY and USD checkpoint files.
Checkpoint Base Name
Filename prefix for checkpoint files.
Checkpoint PLY
When on, export PLY checkpoint files at the interval specified by PLY Export Interval.
PLY Export Interval
Export a PLY checkpoint every N steps.
Checkpoint USD
When on, export USD checkpoint files at the interval specified by USD Export Interval.
USD Export Interval
Export a USD checkpoint every N steps.
States Folder
Directory for training state checkpoints (model + optimizer state).
State Base Name
Filename prefix for state checkpoint files.
Execution ¶
Cache Mode
Determines how the processor node handles work items that report expected file results.
Automatic
If the expected result file exists on disk, the work item is marked as cooked without being scheduled. If the file does not exist on disk, the work item is scheduled as normal. If upstream work item dependencies write out new files during a cook, the cache files on work items in this node will also be marked as out-of-date.
Automatic (Ignore Upstream)
The same as Automatic, except upstream file writes do not invalidate cache files on work items in this node and this node will only check output files for its own work items.
Read Files
If the expected result file exists on disk, the work item is marked as cooked without being scheduled. Otherwise the work item is marked as failed.
Write Files
Work items are always scheduled and the expected result file is ignored even if it exists on disk.
Environment Path
The path to the virtual environment on disk.
Use Symlinks
Determines whether the virtual environment should be created with symlinks to the source Python binaries. This is recommended on Linux and Mac when creating a virtual environment using Houdini’s Python interpreter.
Never
Always fully copy the Python interpreter to the virtual environment, instead of creating a symlink.
Linux and Mac Only
Create symlinks on Linux and Mac, but copy the interpreter executable on Windows.
Always
Create symlinks on all platforms
Use Pip Cache
When on, pip will attempt to use cached packages on the local system instead of downloading them every time. This can improve the installation times when repeatedly installing the same Python package in different virtual environments.
Cache Images to VRAM
Keep decoded training images resident on the GPU. Significantly speeds up training at the cost of VRAM.
Convert EXRs to NPZ
Convert data set EXRs to compressed NumPy .npz files once up front for
faster subsequent loads. Only applies to Houdini data sets.
Number of Workers
Explicit worker count when Number of Workers is set to Custom Worker Count.
Override GPU
Pin training to a specific GPU instead of using the default device.
GPU
Index of the GPU to use when Override GPU is on.
Schedulers ¶
TOP Scheduler Override
Overrides the TOP scheduler for this node.
Schedule When
When on, specifies an expression that determines which work items from the node should be scheduled. If the expression returns zero for a given work item, that work item will immediately be marked as cooked instead of being queued with a scheduler. If the expression returns a non-zero value, the work item is scheduled normally.
Work Item Label
Determines how the node should label its work items. This parameter allows you to assign non-unique label strings to your work items which are then used to identify the work items in the attribute panel, task bar, and scheduler job names.
Use Default Label
The work items in this node will use the default label from the TOP network, or have no label if the default is unset.
Inherit From Upstream Item
The work items inherit their labels from their parent work items.
Custom Expression
The work item label is set to the Label Expression custom expression which is evaluated for each item.
Node Defines Label
The work item label is defined in the node’s internal logic.
Label Expression
When on, specifies a custom label for work items created by this node. The parameter can be an expression that includes references to work item attributes or built-in properties. For example, $OS: @pdg_frame will set the label of each work item based on its frame value.
Work Item Priority
Determines how the current scheduler prioritizes the work items in this node.
Inherit From Upstream Item
The work items inherit their priority from their parent items. If a work item has no parent, its priority is set to 0.
Custom Expression
The work item priority is set to the value of Priority Expression.
Node Defines Priority
The work item priority is set based on the node’s own internal priority calculations.
This option is only available on the
Python Processor TOP,
ROP Fetch TOP, and ROP Output TOP nodes. These nodes define their own prioritization schemes that are implemented in their node logic.
Priority Expression
Specifies an expression for work item priority. The expression is evaluated for each work item in the node.
This parameter is only available when Work Item Priority is set to Custom Expression.
| See also |