| On this page | |
| Since | 22.0 |
Overview ¶
This node preprocesses rendered EXR images, cameras, and a point cloud into a COLMAP-like dataset for Gaussian Splat training.
COLMAP is a Structure from Motion pipeline used to estimate camera positions from multiple views. This step is usually required to create datasets suitable for Gaussian splatting or NeRF training. If you're starting with clean CG data, you can replace this step with a simple conversion and export it into the right format.
The 
ML Train GSplats TOP takes in COLMAP datasets
describing the scene and image/view relationships. To leverage this pre-existing
functionality, the ML Preprocess GSplats TOP creates a dataset matching the
COLMAP standard.
See ML Train GSplats from Karma recipe to get started.
Houdini’s export extends this standard with a separate points3d.ply file, which stores initial Gaussian splats to initialize the training process.
This file stores other gsplats related attributes beyond color, which provide a
more accurate point cloud, avoiding random initialization of any properties
beyond color.
Workflow ¶
The simplest way to produce a Gaussian splatting dataset is from a set of EXR images with camera metadata and a depth AOV.
When both camera metadata and the depth AOV are available, the initial point cloud is built by back-projecting the depth AOV of the source EXR images from the camera in world space. This is the recommended and most convenient option.
This point cloud is then colored using the EXR’s RGB plane and any extra features are initialized from their respective AOVs.
The output is a dataset folder containing
-
images/- the source EXR images. -
sparse/0/- the binary COLMAP scene description (cameras.bin,images.bin,points3D.bin). -
pointcloud/points3d.ply- the initial Gaussian splat point cloud built in Houdini.
Note
All input cameras are currently assumed to share the same intrinsics (resolution, focal length, and aperture). The node exports a single shared PINHOLE camera model in the COLMAP reconstruction.
Note
The training pipeline assumes perfect pinhole cameras and undistorted target images. Input images rendered with camera effects such as lens distortion or depth of field are not supported and will cause severe artifacts during training.
Parameters ¶
Preprocessing ¶
Base Directory
The base path that contains all of the inputs, outputs, and virtual environment used during the training process. Outputs are produced in specific subdirectories depending on the type of output file.
Base Name
The name of the subdirectory that contains all outputs produced by this node.
Dataset Name
The name of the subdirectory that contains datasets produced by this node.
Output Directory
The directory to output preprocessed results. By default, this is relative to the Base Directory, Base Name and Dataset Name.
Generate When
Determines when this node will generate work items. You should generally leave this set to “Automatic” unless you know the node requires a specific generation mode, or that the work items need to be generated dynamically.
All Upstream Items are Generated
This node will generate work items once all of the input nodes have generated their work items.
All Upstream Items are Cooked
This node will generate work items once all of the input nodes have cooked their work items.
Each Upstream Item is Cooked
This node will generate work items each time a work item in an input node is cooked.
Automatic
The generation mode is selected based on the generation mode of the input nodes. If any of the input nodes are generating work items when their inputs cook, this node will be set to Each Upstream Item is Cooked. Otherwise, it will be set to All Upstream Items are Generated.
Cache Mode
Determines how the processor node handles work items that report expected file results.
Automatic
If the expected result file exists on disk, the work item is marked as cooked without being scheduled. If the file does not exist on disk, the work item is scheduled as normal. If upstream work item dependencies write out new files during a cook, the cache files on work items in this node will also be marked as out-of-date.
Automatic (Ignore Upstream)
The same as Automatic, except upstream file writes do not invalidate cache files on work items in this node and this node will only check output files for its own work items.
Read Files
If the expected result file exists on disk, the work item is marked as cooked without being scheduled. Otherwise the work item is marked as failed.
Write Files
Work items are always scheduled and the expected result file is ignored even if it exists on disk.
EXR Image Directory
The directory containing the multi-AOV EXR images to use as training views. Each image corresponds to a camera and is expected to contain the color plane along with any AOVs (depth, extra features) referenced below.
Camera Source
Where the camera parameters and transforms for each view are read from.
When the cameras are not read from EXR metadata, each camera is matched
to its source EXR by name. A camera named cam_0001 is paired with
cam_0001.exr in the EXR Image Directory.
EXR Metadata
Extract camera parameters directly from the metadata stored in the source EXR files. Karma stores this metadata by default.
Note
To test whether an EXR file has camera metadata, you can load a file COP in Copernicus and toggle on the Load Camera parameter.
File
Load cameras from a .bgeo geometry file on disk.
SOP Node
Read cameras from a SOP node in the current .hip file.
LOP Node
Read cameras from a LOP node in the current .hip file.
Cameras
The geometry file containing the camera primitives.
SOP Path
The SOP node containing the camera primitives.
LOP Path
The LOP node providing the camera primitives.
Primitives
The USD primitive pattern identifying the camera primitives on the selected LOP.
Static Import Frame
The frame at which the LOP cameras are sampled. Cameras are imported as a static snapshot at this frame. It has to match the frame used when rendering the EXR images.
Point Cloud Source
Where the initial point cloud used to initialize the splat training comes from.
When using any option other than EXR Depth + Metadata, the points will be passed to the training dataset without any processing.
Tip
You can color and assign other gsplats attributes to your point cloud
to pre build gaussian splats, which will initialize the
training. The full list of supported attributes includes v@Cd, v@scales,p@orient, and f@GS_Alpha. The color is expected to be in the same color space as the EXR.
EXR Depth + Metadata
Build a point cloud by back-projecting the depth AOV of the source EXR images using the camera metadata. This is the recommended and most convenient option.
File
Load the point cloud from a geometry file on disk.
SOP Node
Read the point cloud from a SOP node.
LOP Node
Read the point cloud from a LOP node.
Depth AOV Name
The name of the depth plane in the source EXRs, used to back-project pixels into 3D.
Number of Points
The target number of points to generate for the initial point cloud when back-projecting from depth.
Projection Resolution
The fraction of the image resolution to use when back-projecting pixels. Half and Quarter reduce the number of candidate pixels and speed up the projection step.
Full
Use the full image resolution.
Half
Use half the image resolution.
Quarter
Use quarter the image resolution.
Use Max Projections
When on, limits the number of cameras used for depth back-projection to the value set by Limit Cameras to.
Limit Cameras to
The maximum number of cameras considered during depth back-projection. When dealing with a high number of input EXRs it makes sense to limit the number of cameras to a subset of the whole dataset to speed up the preprocessing. Cameras are filtered with even spacing from a larger dataset to ensure the data distribution doesn’t get skewed.
Initialize Extra Features
A list of extra per point features to initialize from AOVs in the source EXR images in addition to color. Each entry adds a feature channel that is sampled from the specified AOV.
AOV Name
The name of the AOV in the source EXR images to sample the extra feature from.
Signature
The number of channels in the feature.
Mono
Single-channel (scalar) feature.
UV
Two-channel feature.
RGB
Three-channel feature.
Smoothing Iterations
The number of smoothing iterations applied to the back-projected point cloud to reduce noise. This blurs the point colors reducing any unwanted hot spots.
Point Cloud
The geometry file containing the initial point cloud.
SOP Path
The SOP providing the initial point cloud.
LOP Path
The LOP providing the initial point cloud.
Primitives
The USD primitive pattern identifying the point cloud primitives on the selected LOP.
Static Import Frame
The frame at which the LOP point cloud is sampled. It has to match the frame used when rendering the EXR images.
Execution ¶
Environment Path
The path to the virtual environment on disk.
Use Symlinks
Determines whether the virtual environment should be created with symlinks to the source Python binaries. This is recommended on Linux and Mac when creating a virtual environment using Houdini’s Python interpreter.
Never
Always fully copy the Python interpreter to the virtual environment, instead of creating a symlink.
Linux and Mac Only
Create symlinks on Linux and Mac, but copy the interpreter executable on Windows.
Always
Create symlinks on all platforms
Use Pip Cache
When on, pip will attempt to use cached packages on the local system instead of downloading them every time. This can improve the installation times when repeatedly installing the same Python package in different virtual environments.
Schedulers ¶
TOP Scheduler Override
Overrides the TOP scheduler for this node.
Schedule When
When on, specifies an expression that determines which work items from the node should be scheduled. If the expression returns zero for a given work item, that work item will immediately be marked as cooked instead of being queued with a scheduler. If the expression returns a non-zero value, the work item is scheduled normally.
Work Item Label
Determines how the node should label its work items. This parameter allows you to assign non-unique label strings to your work items which are then used to identify the work items in the attribute panel, task bar, and scheduler job names.
Use Default Label
The work items in this node will use the default label from the TOP network, or have no label if the default is unset.
Inherit From Upstream Item
The work items inherit their labels from their parent work items.
Custom Expression
The work item label is set to the Label Expression custom expression which is evaluated for each item.
Node Defines Label
The work item label is defined in the node’s internal logic.
Label Expression
When on, specifies a custom label for work items created by this node. The parameter can be an expression that includes references to work item attributes or built-in properties. For example, $OS: @pdg_frame will set the label of each work item based on its frame value.
Work Item Priority
Determines how the current scheduler prioritizes the work items in this node.
Inherit From Upstream Item
The work items inherit their priority from their parent items. If a work item has no parent, its priority is set to 0.
Custom Expression
The work item priority is set to the value of Priority Expression.
Node Defines Priority
The work item priority is set based on the node’s own internal priority calculations.
This option is only available on the 
Python Processor TOP, 
ROP Fetch TOP, and ROP Output TOP nodes. These nodes define their own prioritization schemes that are implemented in their node logic.
Priority Expression
Specifies an expression for work item priority. The expression is evaluated for each work item in the node.
This parameter is only available when Work Item Priority is set to Custom Expression.
| See also |
