REALITY CAPTURE PLUGIN - OPEN BETA
We're happy to announce that the Reality Capture plugin is ready for testing. It’s still under the beta label for the time being, but we would love to hear your feedback.
The plugin allows you to run Reality Capture processes from within Houdini in order to streamline your pipeline. It is designed to take multiple images as input, and generate high resolutions models as the output, which can then be ran through Houdini's mesh pipeline. It is not designed to be a complete replacement of every feature in RC, but the ones that fit well in a Houdini pipeline.
Reality Capture is a state-of-the-art all-in-one photogrammetry software solution which automatically extracts beautiful and accurate 3D models from a set of ordinary images and / or laser-scans
Update 2 - 12/05/2018
The Reality Capture plugin is now a part of the Game Development Toolset.
- New RC SDK has been integrated, giving a much longer hard expiration date
Update 1 - 9/18/2018
Version 0.6 has been uploaded and the download link has been updated. Apologies for the interruption on the licensing side, we're still working through some of the kinks.
V0.6 Release Notes
- Nodes now have a "Manual Mode" on by default where the cooking will need to be explicitly invoked. This will help with runaway processes
- Licensing is now done through a Get License button in each node, this will launch the system browser.
- For Win7 users, there is an environment variable to use a Qt Browser instead of the system browser for licensing. It is RC_USE_QT_BROWSER
- Texture Model was broken out from Create Model and into it's own node, which is combined with the Load Texture Model (which is now gone)
- Caching is now done on both the alignment and the model, allowing for files to be saved with references to the models and opened quickly
- Max Display Polycount added to the Create Model node, this allows for simplifying the model before rendering it. There is still an internal representation that is full res
- Create Model now has a export raw model option where it'll dump out the full res ply object
- Create Model nodes can now be chained together in other to generate lower resolution models for texturing purposes, while still maintaining a high original
- Align Images Node will now display the number of cameras aligned in the used component (support for multiple components is planned)
- Install the Game Development Toolset
- Run Houdini and Verify the nodes are being loaded properly
You should have 5 nodes under the Reality Capture Tab Submenu in the Geometry Context
If you’re only seeing 2 nodes go to the troubleshooting section
First Time Usage
To activate the plugin, drop an RC Align Images node and click on the Get License Button.
This will launch a web browser and entering your credentials to generate a license.txt file, which will then reside in your $HOME directory (usually C:/Users/USERNAME/), and should only happen the first time you run, or when you need to renew your licenses.
Note: It is possible that your browser opened and you didn't realize that there is a new tab open, if you don't see anything happen, double check your tabs.
Note: If your browser isn't opening, try adding the variable RC_USE_QT_BROWSER=1 to your houdini.env file
Command line usage will only work with a CLI license for the time being
In the current version there are 5 Nodes, that can be found in the geometry context under Reality Capture.
RC Register Images is where you tell RC which images to process
RC Align Images takes said images and converts them into a point cloud
RC Create Model will generate a high res model
RC Texture Model will generate a textured model
RC Extract Cameras is an utility node to verify the camera positions
The intended node graph will look like the image on the right.
One additional option is to have an axis aligned bounding box as the second input for the Create Model node, this will allow you to limit the processing area of the model generation.
For better analysis of your model, you might want to change some of the default display settings. This can by done by hitting D while on the viewport. As well as add an Ambient Light to your scene, cranked to white.
RC Register Images
This node is the start of the chain, and where you add your images. Simply click Add Images and a dialog will launch in order for you to select your chosen images.
The image list is then stored as a detail attribute and will be passed on to the next nodes.
If you'd like to automate this process or maybe bypass it altogether, all you need is an string array detail attribute named "images", you can generate this with python with the following code:
my_image_list = 
geo.addAttrib(hou.attribType.Global, "images", my_image_list, False)
RC Align Images
RC Align Images processes the images and returns a point cloud to Houdini.
We’ve exposed a few parameters from Reality Capture, if you feel like you are missing a parameter please us know at firstname.lastname@example.org
Image Downscale Factor - scale down your images in order to speed up processing times. 1 will not scale, 2 will half the image, 3 will quarter it, so on and so forth
Max Features Per Image - defines the absolute limit to features used per image, More features mean lower speed but may result in fewer components.
Image Overlap - defines how much of the image space is covered with the same part of the object, when talking about neighboring images
Preselector Features - This is the number of features that will be used in alignment from the detected ones. Optimally, set it to 1/4-1/2 of the detected features.
Detector Sensitivity - The natural feature detector focuses on places in an image which can be easily noticed and tend to be stable under various image and geometry transformations - e.g. changing the view point or changing lighting conditions. With this setting you can control how the detector looks at the stability. Setting it to ultra will detect more feature points in places with weak texture. This will also add points, which might have texture due to noise in the image. Setting this value to low will, on the other hand, throw away feature points which could be used for image registration.
Draft Mode - Runs the alignment with hardcoded lower settings
Cache File - Location to store RC progress file. These files will be passed from node to node.
RC Create Model
RC Create Model takes the Alignment Data and converts it to a high resolution model
The first input should be the output of a RC Align Images Node and the second input is optional but it is an axis aligned box to limit the computational area.
Note: This is actually processing a cached file on disk, not the points that are incoming. That file is stored as a detail attribute on the RC Align Images node. So attempts to manipulate the point cloud will be in vain.
Detail Decimation Factor - The bigger the value, the lower the detail and the lower the number of triangles in the final model.
Max PolyCount - Simplify the model down to a target resolution.
Colorize Model - Generate a vertex colors on your model based on the images.
Generate Triangles - Also generate triangles (not only a point cloud)
Max Display Polycount - Optionally reduce the model that is getting displayed by the viewport, this is separate from the RC internal model data but can help with keeping the viewport from loading 100+ million meshes.
Export Raw RC Model - Export the internal RC model data (full res) to a ply file
Cache File- Location to store RC progress files. These files will be passed from node to node.
RC Texture Model
RC Texture Model is a helper node in order to load your Textured Model
If connected to a Create Model node, it will inspect the texture model file path, load the geometry and setup a material with textures so it is ready for baking once you have a lower res model.
Model Path - Path to the exported textured object file. Textures will be automatically created based on RC's criteria
Max Texture Resolution - RC will split up the textures as UDIMs, this will determine the size of them
Style - Visibility based style is fast and sharp, photoconsistency based style is slower with more complex results
Per Reality Capture Developers, it is better to generate a 10% reduced version of your model, texture that and then do a traditional texture bake to a low res model instead of doing the texturing process on the low res model. The low res model is usually too far from the original model and can result on bad and blurry projections.
Since we already have the RC model loaded in memory we can take the opportunity to also generate textures for it.
This will output a separate (slightly reduced model) with UDIM textures, which means it should have the highest possible texture quality since it will keep making more textures to ensure a good texel resolution.
RC Extract Cameras
RC Extract Cameras will generate points for each of the cameras that were aligned properly. Currently only positional data is recovered but there are plans of bringing more parameters.
Currently we only support camera positions. Camera rotations is something we are investigating.
The Plugin requires a valid Reality Capture license to run. All license types are compatible with the plugin, except the Steam version of Reality Capture. The Promo license will work fine.
Command line usage will only work with a CLI license for the time being
The Plugin is currently only available on Windows.
Can I set Control Points in Houdini?
Unfortunately not at this time. This is something that is done on the image, and we currently don’t have any interfaces for manipulating those.
We are looking at options to export control points from RC and bring them into Houdini though.
Can I use the RC Plugin to also process LIDAR data in Houdini?
Yes, you have to convert your e57 files to lsp files using Reality Capture and then add them as images. We are working with the RC team on exposing the e57>lsp conversion so you can just add the e57 directly in Houdini.
- Reality Capture will pop up a Captcha every few models you process, in order to make sure you’re not running it command line.
- The plugin has a hard expiration date set to the relatively near future, it is displayed on the console when RC authenticates
- Error reporting is relatively light right now
Please send support questions to email@example.com.
For the plugin to work there are 2 variables that need to be set
HOUDINI_PATH - needs to contain the root directory of the plugin files
PATH - needs to contain the bin directory of the plugin files
As of version .2 the installer patches your Docs/houdini16.5/houdini.env file with the following lines
# Reality Capture Plugin for Houdini
HOUDINI_PATH=C:\Program Files\Side Effects Software\Reality Capture Plugin for Houdini 16.5;&
PATH=C:\Program Files\Side Effects Software\Reality Capture Plugin for Houdini 16.5\bin;$PATH&;
DLL Loading Errors
If you are seeing the RC Register Images and RC Load Texture Model nodes but not the other 3. That means that the HOUDINI_PATH is set properly.
But for some reason the plugin itself (dso) isn’t loading.
The most common reason is binary differences with Houdini itself. So make sure you have the latest daily build.
The other reason is missing Windows dependencies. There are several C++ Redistributable Versions that exist and we’re working on tracking down which version is actually needed, but we’ve had customers report success by installing them. Here is are some links to them
This is the most likely culpritMicrosoft Visual C++ 2015 Update 3 Redistributable (x86 and x64)
But here are others in case it still not workingMicrosoft Visual C++ 2005 SP1 Redistributable (x86)
Microsoft Visual C++ 2005 SP1 Redistributable (x64)
Microsoft Visual C++ 2008 SP1 Redistributable (x86)
Microsoft Visual C++ 2008 SP1 Redistributable (x64)
Microsoft Visual C++ 2010 SP1 Redistributable (x86)
Microsoft Visual C++ 2010 SP1 Redistributable (x64)
Microsoft Visual C++ 2012 Update 4 Redistributable (x86 and x64)
Microsoft Visual C++ 2013 Redistributable (x86 and x64)