Description

We're happy to announce that the Reality Capture plugin is ready for testing. It’s still under the beta label for the time being, but we would love to hear your feedback. 

The plugin allows you to run Reality Capture processes from within Houdini in order to streamline your pipeline. It is designed to take multiple images as input, and generate high resolutions models as the output, which can then be ran through Houdini's mesh pipeline. It is not designed to be a complete replacement of every feature in RC, but the ones that fit well in a Houdini pipeline. 

Reality Capture is a state-of-the-art all-in-one photogrammetry software solution which automatically extracts beautiful and accurate 3D models from a set of ordinary images and / or laser-scans

Update 2 - 12/05/2018

The Reality Capture plugin is now a part of the Game Development Toolset


Release Notes 

  • New RC SDK has been integrated, giving a much longer hard expiration date


Update 1 - 9/18/2018

Version 0.6 has been uploaded and the download link has been updated. Apologies for the interruption on the licensing side, we're still working through some of the kinks. 


V0.6 Release Notes 

  • Nodes now have a "Manual Mode" on by default where the cooking will need to be explicitly invoked. This will help with runaway processes
  • Licensing is now done through a Get License button in each node, this will launch the system browser. 
  • For Win7 users, there is an environment variable to use a Qt Browser instead of the system browser for licensing. It is RC_USE_QT_BROWSER
  • Texture Model was broken out from Create Model and into it's own node, which is combined with the Load Texture Model (which is now gone) 
  • Caching is now done on both the alignment and the model, allowing for files to be saved with references to the models and opened quickly
  • Max Display Polycount added to the Create Model node, this allows for simplifying the model before rendering it. There is still an internal representation that is full res
  • Create Model now has a export raw model option where it'll dump out the full res ply object
  • Create Model nodes can now be chained together in other to generate lower resolution models for texturing purposes, while still maintaining a high original 
  • Align Images Node will now display the number of cameras aligned in the used component (support for multiple components is planned) 


Quick Demo

Installation Instructions 

  1. Install the Game Development Toolset
  2. Run Houdini and Verify the nodes are being loaded properly

You should have 5 nodes under the Reality Capture Tab Submenu in the Geometry Context


 Pro Tip

If you’re only seeing 2 nodes go to the troubleshooting section

First Time Usage

To activate the plugin, drop an RC Align Images node and click on the Get License Button.         

This will launch a web browser and entering your credentials to generate a license.txt file, which will then reside in your $HOME directory (usually C:/Users/USERNAME/), and should only happen the first time you run, or when you need to renew your licenses. 

Note: It is possible that your browser opened and you didn't realize that there is a new tab open, if you don't see anything happen, double check your tabs.


Note: If your browser isn't opening, try adding the variable RC_USE_QT_BROWSER=1 to your houdini.env file

Licensing Info

The Plugin requires a valid Reality Capture license to run. All license types are compatible with the plugin, except the Steam version or Demo Version of Reality Capture. The Promo license will work fine. 

Command line usage will only work with a CLI license for the time being

Usage Instructions

Reality Capture Nodes 

In the current version there are 5 Nodes, that can be found in the geometry context under Reality Capture. 


RC Register Images is where you tell RC which images to process 

RC Align Images takes said images and converts them into a point cloud

RC Create Model will generate a high res model

RC Texture Model will generate a textured model 

RC Extract Cameras is an utility node to verify the camera positions

Recommended Flow

The intended node graph will look like the image on the right. 


One additional option is to have an axis aligned bounding box as the second input for the Create Model node, this will allow you to limit the processing area of the model generation. 



Note

For better analysis of your model, you might want to change some of the default display settings. This can by done by hitting D while on the viewport. As well as add an Ambient Light to your scene, cranked to white. 


Node Documentation

RC Register Images

This node is the start of the chain, and where you add your images. Simply click Add Images and a dialog will launch in order for you to select your chosen images.


Pro Tip: If you feel like a file format is missing from the filter let us know, if you know that RC can process that data feel free to type *.* in the filename dialog to be able to select any file format.


The image list is then stored as a detail attribute and will be passed on to the next nodes. 




Advanced Tip

 If you'd like to automate this process or maybe bypass it altogether, all you need is an string array detail attribute named "images", you can generate this with python with the following code: 

my_image_list = []
geo.addAttrib(hou.attribType.Global, "images", my_image_list, False)
geo.setGlobalAttribValue("images", my_image_list)

RC Align Images

RC Align Images processes the images and returns a point cloud to Houdini.


We’ve exposed a few parameters from Reality Capture, if you feel like you are missing a parameter please us know at support@sidefx.com 






Parameters

Image Downscale Factor - scale down your images in order to speed up processing times. 1 will not scale, 2 will half the image, 3 will quarter it, so on and so forth

Max Features Per Image - defines the absolute limit to features used per image, More features mean lower speed but may result in fewer components.
Image Overlap -  defines how much of the image space is covered with the same part of the object, when talking about neighboring images
Preselector Features -  This is the number of features that will be used in alignment from the detected ones. Optimally, set it to 1/4-1/2 of the detected features.
Detector Sensitivity - The natural feature detector focuses on places in an image which can be easily noticed and tend to be stable under various image and geometry transformations - e.g. changing the view point or changing lighting conditions. With this setting you can control how the detector looks at the stability. Setting it to ultra will detect more feature points in places with weak texture. This will also add points, which might have texture due to noise in the image. Setting this value to low will, on the other hand, throw away feature points which could be used for image registration.
Draft Mode - Runs the alignment with hardcoded lower settings
Cache File - Location to store RC progress file. These files will be passed from node to node. 

RC Create Model

RC Create Model takes the Alignment Data and converts it to a high resolution model

The first input should be the output of a RC Align Images Node and the second input is optional but it is an axis aligned box to limit the computational area. 

Note: This is actually processing a cached file on disk, not the points that are incoming. That file is stored as a detail attribute on the RC Align Images node. So attempts to manipulate the point cloud will be in vain.


Parameters

Model Options

Detail Decimation Factor - The bigger the value, the lower the detail and the lower the number of triangles in the final model.
Max PolyCount - Simplify the model down to a target resolution. 
Colorize Model - Generate a vertex colors on your model based on the images. 
Generate Triangles -  Also generate triangles (not only a point cloud) 

Advanced Options

Use Input's Model Data-  Don't recompute the model data, but use the input's data. This is useful to chain Create Model nodes together in order to do different simplification passes. 
Max Display Polycount - Optionally reduce the model that is getting displayed by the viewport, this is separate from the RC internal model data but can help with keeping the viewport from loading 100+ million meshes. 

Export Raw RC Model - Export the internal RC model data (full res) to a ply file 
Cache File- Location to store RC progress files. These files will be passed from node to node. 

RC Texture Model

RC Texture Model is a helper node in order to load your Textured Model 

If connected to a Create Model node, it will inspect the texture model file path, load the geometry and setup a material with textures so it is ready for baking once you have a lower res model.


Parameters 

Model Path - Path to the exported textured object file. Textures will be automatically created based on RC's criteria

Max Texture Resolution - RC will split up the textures as UDIMs, this will determine the size of them

Style - Visibility based style is fast and sharp, photoconsistency based style is slower with more complex results


Note

Per Reality Capture Developers, it is better to generate a 10% reduced version of your model, texture that and then do a traditional texture bake to a low res model instead of doing the texturing process on the low res model. The low res model is usually too far from the original model and can result on bad and blurry projections.

Since we already have the RC model loaded in memory we can take the opportunity to also generate textures for it.

This will output a separate (slightly reduced model) with UDIM textures, which means it should have the highest possible texture quality since it will keep making more textures to ensure a good texel resolution.

RC Extract Cameras

RC Extract Cameras will generate points for each of the cameras that were aligned properly. Currently only positional data is recovered but there are plans of bringing more parameters.


Currently we only support camera positions. Camera rotations is something we are investigating. 









Technical Information

Licensing Info

The Plugin requires a valid Reality Capture license to run. All license types are compatible with the plugin, except the Steam version of Reality Capture. The Promo license will work fine. 

Command line usage will only work with a CLI license for the time being

Compatibility Info 

The Plugin is currently only available on Windows. 

FAQ

Can I set Control Points in Houdini?

Unfortunately not at this time. This is something that is done on the image, and we currently don’t have any interfaces for manipulating those.

We are looking at options to export control points from RC and bring them into Houdini though.

Can I use the RC Plugin to also process LIDAR data in Houdini?

Yes, you have to convert your e57 files to lsp files using Reality Capture and then add them as images. We are working with the RC team on exposing the e57>lsp conversion so you can just add the e57 directly in Houdini. 

Known Issues

  • Reality Capture will pop up a Captcha every few models you process, in order to make sure you’re not running it command line.
  • The plugin has a hard expiration date set to the relatively near future, it is displayed on the console when RC authenticates
  • Error reporting is relatively light right now

Trouble Shooting

Please send support questions to support@sidefx.com

Environment Variables

For the plugin to work there are 2 variables that need to be set

HOUDINI_PATH - needs to contain the root directory of the plugin files
PATH - needs to contain the bin directory of the plugin files

As of version .2 the installer patches your Docs/houdini16.5/houdini.env file with the following lines

# Reality Capture Plugin for Houdini
HOUDINI_PATH=C:\Program Files\Side Effects Software\Reality Capture Plugin for Houdini 16.5;&
PATH=C:\Program Files\Side Effects Software\Reality Capture Plugin for Houdini 16.5\bin;$PATH&;

DLL Loading Errors

If you are seeing the RC Register Images and RC Load Texture Model nodes but not the other 3. That means that the HOUDINI_PATH is set properly. 
But for some reason the plugin itself (dso) isn’t loading.

The most common reason is binary differences with Houdini itself. So make sure you have the latest daily build.

The other reason is missing Windows dependencies. There are several C++ Redistributable Versions that exist and we’re working on tracking down which version is actually needed, but we’ve had customers report success by installing them. Here is are some links to them

This is the most likely culprit

Microsoft Visual C++ 2015 Update 3 Redistributable (x86 and x64)

But here are others in case it still not working

Microsoft Visual C++ 2005 SP1 Redistributable (x86)
Microsoft Visual C++ 2005 SP1 Redistributable (x64)
Microsoft Visual C++ 2008 SP1 Redistributable (x86)
Microsoft Visual C++ 2008 SP1 Redistributable (x64)
Microsoft Visual C++ 2010 SP1 Redistributable (x86)
Microsoft Visual C++ 2010 SP1 Redistributable (x64)
Microsoft Visual C++ 2012 Update 4 Redistributable (x86 and x64)
Microsoft Visual C++ 2013 Redistributable (x86 and x64)

Related Links

COMMENTS

  • pbowmar 3 months, 3 weeks ago  | 

    Linux version please! This is useful for VFX and film too!

    • lkruel 3 months, 3 weeks ago  | 

      Hey Peter,

      Unfortunately the RC SDK doesn't support linux at this point

  • Rod TD 3 months, 2 weeks ago  | 

    Linux is essential!!! Please call RC devs to a linux version!

  • belegdae 3 months, 1 week ago  | 

    +1 for Linux version!

  • Snappy_Darko 3 months ago  | 

    I'm not getting the nodes RC Register Images and RC Load Texture Model...the other 3 show (so the inverse problem as the one reported above). The link to 'likely culprit' (Microsoft Visual C++ 2015 Update 2 Redistributable (x86 and x64)) is dead too.

  • lo_res 3 months ago  | 

    When I click the display button on the align images node, it fails to open up the licence activation web page. I've tried having firefox, chrome and edge open when doing this, but have had no joy. I have a valid trial copy of reality capture installed.

    • ffaccounts 3 months ago  | 

      Same here unfortunately. I see the nodes, but once I connect the RC_registerimages to the RC_alignimages it gives me an error: Failed to load License. I have 6 registered payed licences here (one of them on this PC), reality Capture works fine, but the plugin fails to connect. I'm so eager to try this. If someone has a solution it would be very much appreciated.

      • ffaccounts 3 months ago  | 

        I checked if there is a license.txt in the $HOME directory (usually C:/Users/USERNAME/) and there was not. If there is a way to generate that manually, that would maybe solve the issue.

  • lkruel 3 months ago  | 

    Hey everyone, unfortunately the RC dlls have a hard expiration date that they just hit. I'll work on uploading a new version asap. Unfortunately I am out of the country until Monday, but I'll see what I can do

    • Pisut Wisessing 3 months ago  | 

      Thanks for letting us know. Was planning to test some stuff this weekend.

    • lkruel 2 months, 4 weeks ago  | 

      New build is up. Sorry for the inconvenience!

  • RickToxik 2 months ago  | 

    Yeah we would like Linux very much too : )

  • ciello 1 month, 2 weeks ago  | 

    This seems really promising, and the recent webinar showing off these abilities was very exciting! The licensing issue is a giant red flag though. It's too delicate :( Maybe remove the button->webpage->license.txt approach and allow us to plug in a license from RC directly? That would at least remove some of the breaking points for windows machines. The self-destructing dll is also not something I can pitch to my team without being laughed out of the room.

  • fhamel 1 month, 1 week ago  | 

    Hello Luis! Can we use this plug-in with a Houdini Core license, or does this require FX in any way? (I can't think of a reason off-hand why this would require FX but better safe than sorry.) Thanks!

    • lkruel 1 month, 1 week ago  | 

      Hey there, Core will work just fine :D

  • Barrett Meeker 2 weeks, 5 days ago  | 

    I can't seem to get this to work in Houdini 17.0.352 Indie with a Promo license of Reality Capture. I got the nodes showing up but the rc_alignimages node doesn't seem to ever output anything. I tried the sample content and it doesn't seem to work for me either.

    I tried the standalone Reality Capture software and it does work on my machine so it seems my license is working. Any tips? Would really love to do this workflow in Houdini.

    • lkruel 2 weeks, 5 days ago  | 

      Hey Barret.

      Can you try using the game development toolset, as opposed to the installer?

      I'll update the tutorial instructions to reflect the changes.

Please log in to leave a comment.