Houdini 22.0 Nodes Copernicus nodes

Neural Layer to Depth (MoGe-2) Copernicus node

Estimates metric-scale depth, normal, and position maps from an input image.

On this page
Since 22.0

This node estimates scene information from an input image using Microsoft’s MoGe-2 model. The model approximates a metric-scale depth map, normal map, and position map, along with a sky mask and a 3D point cloud of the visible image surfaces.

Note

To use this node, click Download Models after a fresh Houdini installation.

Use this node to quickly and easily get depth, normal, and position maps, a sky mask, and a 3D point cloud from a 2D image. This lets you create slap comps at a fast rate, which is useful when you want to do the following:

Set Pre-Process parameters that resample the input or apply a color transformation before the model estimates the input. Set Post-Process parameters that adjust the output normals (N), points, and skymask after the model estimates the input. Turn on Create GSplat Points to output the points as Gaussian splats.

The following are the different versions of this node that you can add to your scene. These are the same node except they differ in what’s displaced.

Parameters

Model

The MoGe-2 model to use.

MoGe-2 ViT-L Normal

Use a MoGe-2 model that has a large Vision Transformer and normal map estimation.

Custom

Use your own MoGe-2 ONNX Model File.

Download Models

Downloads the MoGe-2 ONNX model to the SHFS model location.

Reload Model

Forces a reload of the .onnx model file.

Model File

The custom MoGe-2 .onnx model file to load.

Number of Tokens

The total number of visual patches (tokens) that the model processes during inference, which controls inference resolution. Higher values preserve finer detail but increase runtime and memory cost. This affects the model’s internal working resolution, not the final output image size.

Pre Processing

Resample Size

The dimensions (width by height in meters) to which the node resamples the input image. When off, the node uses the input image’s size.

Color Transform

The color transformation to apply to the input image before processing.

OCIO

Use OpenColorIO to transform colors from the Original Space to the target space (To Space). See Color management in Houdini for more information.

None (Raw)

Don’t apply any color transformation.

Original Space

The space that the input color is in.

To Space

The output color space to which the input color transforms.

Post Processing

Attempt Aspect Correct

Attempts to estimate the camera’s field of view from the output points and applies the result to the 3D point cloud.

Scale by Metric

Expresses the depth and position (P) outputs in real-world units rather than relative or affine values.

Apply Sky Mask

Masks out regions classified as sky in the skymask output. This prevents unstable point estimations in areas of the image that lack reconstructible surfaces.

Sky Depth Shift

Applies a depth offset (in meters) to sky regions when Apply Sky Mask is on.

Remove Edges

Removes points that are near depth discontinuities, which are sudden changes in depth disparity. Configure Max Depth Change to set the maximum depth change that the points can surround until they're removed.

Max Depth Change

The maximum change in depth disparity (in meters) the node allows until it removes points around the depth discontinuities. Lower values remove more points while higher values preserve more detail (but may retain noisy points).

Normal Type

The method used to store the normals (see Normals for more information).

Signed (-1 to 1)

Output signed normals, compatible with geometry attributes.

Offset (0 to 1)

Output offset normals, compatible with normal maps.

GSplat

Create GSplat Points

Visualizes the 3D point cloud as a Gaussian splat .

GSplat Scale

The size (in meters) of the Gaussian splat points.

Camera

Use Solved Camera

Matches output layers to a camera solved with pixel-GSplat point correspondences.

RANSAC Seed

The seed used to randomly select candidate point correspondences used to solve for a camera.

RANSAC Iterations

The number of RANSAC loop iterations executed to solve for a camera. Increasing this can improve the accuracy of the solution.

RANSAC Tolerance

The reprojection error tolerance used to determine if a camera solution is acceptable.

Advanced

Execution Provider

Determines which ONNX execution provider to use for inference. By default, the node attempts to pick the best available provider and prefers to use GPU acceleration.

Mac

You must set the Execution Provider to CPU to prevent initialization errors on macOS.

Automatic

Chooses the best provider for the current system. This option prioritizes CUDA if installed, DirectML/CoreML as a fallback depending on the platform, and CPU inference if no GPU provider is available.

CPU

Perform inference using on the CPU.

CUDA

Perform inference using CUDA/cuDNN. CUDA and cuDNN must be installed using the packages provided by NVIDIA.

DirectML

Only available on Windows. Performs inference using the Windows Direct Machine Learning library.

CoreML

Only available on macOS. Performs inference using Apple’s Core ML library.

Device ID

This parameter can be used to specify which GPU to use for inference when multiple GPUs are available. It has no effect when using CPU inference.

Inputs

source

The source image to estimate.

Note

If you wire in a non-RGB input, it converts to RGB.

Outputs

depth

The estimated depth map.

normal

The estimated surface normal map.

pos

The estimated world-space position map.

points

The 3D point cloud reconstructed from the source image. When Create GSplat Points is on, the node visualizes the point cloud as a Gaussian splat.

height

The heightfield representation that’s derived from depth data.

skymask

The mask of sky regions when Apply Sky Mask is on.

Copernicus nodes