Locate the $HH/help/files/pdg_examples/top_computervision folder. You will find a top_computervision.hip file for Houdini and an input.mp4 as the video source.

If you want to use your own sources, you need:

• A video source or image sequence that has a target object of interest.

• A method to create a mask for selected frames. This can be done within Houdini or in another DCC program.

The first step is to mask out your object of interest from the video. The goal is to create enough examples for the model to learn from the object without hand masking the full video.

• Before selecting your training frames, you may want to clean up the source footage. For example, crop out black borders or other unwanted image artifacts so the model trains on the actual content you want to detect.

• Select a number of frames relative to your video length. In the example, the video is 10 seconds at 30fps and has 300 frames. A rough recommendation is to mask out at least 10 of these frames by hand to create a better and more accurate dataset.

• Choose frames that represent a good range of cases the training model will see in the next stage. Good examples include frames where the object appears from different angles, close to the edge of the frame, or slightly cutoff.

• The masks do not need to be a perfect one-to-one match to the object of interest. In the example, you can see the logo is roughly masked out using a brush stroke.

  

• To prepare your images, you can use a node like Paint 3D COP or a DCC software.

In the example file, open the training_data geometry node, then the copnet1 COP Network.

• The input.mp4 file is connected to a batch of Time Shift COP node node to select the desired frames for the training stage.

  • These frames are then masked out using a Paint 3D COP node at the position of the object of interest.

The ML Preprocess Computer Vision TOP prepares image datasets for the computer vision training tasks. It prepares a .json file that uses the same format as the COCO dataset to represent the images, sizes, and masks. The generated dataset folder that contains the source images, mask images, and COCO .json. These can then be trained using the ML Train Computer Vision TOP.

To generate a dataset:

1. In the example file, locate the topnet TOP network. Then the Preprocess Computer Vision TOP node.

2. Set the Object Classes. The example uses Logo as the class for the blurred out logo.

   • The Class Color is an optional setting and is more for visualization purposes.

3. Choose how to Input Images. There are multiple methods to input images such as dictionary attributes, multi-aov EXRs, and COP nodes. See Input Format for more information.

   • The example uses the COP Nodes as the Input Source. The Source COP and Mask COP parameters point to the selected frames and masked out frames.

4. Set your Evaluating Using option. This example uses a Custom Range which allows for more control to set a single frame, frame range, or skip certain frames.

   • This can be useful to skip frames where your object of interest pans out of the camera in certain frames.

5. Cook the node.

6. Review the generated dataset folder.

   • It should contain the source images, mask images, and COCO format .json file. You will then use the generated dataset directory as the input dataset for the ML Train Computer Vision TOP node.

The ML Train Computer Vision TOP trains an object detection, masking, or keypoint detection model using a dataset prepared by the ML Preprocess Computer Vision TOP. This means the node uses the generated dataset folder including the COCO .json, masks, and source images to train.

There are various parameters that change how your dataset is trained. You can read descriptions for each parameter in the node’s parameters section. Some of the tabs such as Checkpointing, Execution, and Schedulers are similar to other TOPs and ML nodes. Here’s a brief description of each tab:

1. In the example file, locate the ML Train Computer Vision TOP node in the TOP network.

2. In the Input Data Source tab, you input your generated dataset from the previous stage here.

   • To get started, it’s recommended to use a pre-trained weight in the Network tab instead of setting things manually.

3. Cook the node.

   • Make sure you have your pre-trained weights downloaded or it may cause an error.

The results will be located in the folder specified in the Output Directory parameter. This folder contains the following:

config - Stores training configuration files and logs. Use it to inspect or reproduce the settings used for the run.

models - Stores model checkpoints and exported model files. PyTorch checkpoints can be used to resume or preserve training states. ONNX files are exported models that can be used for inference.

plots - Stores training-progress data used by the Training Monitor. These plots show values such as loss/error and validation metrics. This helps you judge whether training is improving.

tests - Stores visual test images generated during training. These images show what the model detected at different checkpoints making it easier to inspect false positives, missed objects, and overall model quality.

Validation outputs may also be written as CSV files. These store numerical validation metrics and are automatically detected by the Training Monitor.

The example files provides two methods to use your trained data.

1. Use the ROP Image Output TOP and FFmpeg Encode Video TOP to render the inference result to disk.

2. Use the ML Computer Vision Inference COP interactively in Houdini.

These two methods use the same trained model and inference COP network. The difference is mainly where and how the result is viewed or written.

In the example TOP network, the _ROP Image Output TOP cooks the same inference COP network. The difference is that it runs the COP result as a PDG background task and writes the resulting frames to disk.

Then, the FFmpeg Encode Video TOP assembles those rendered frames into a final movie file, such as an .mp4.