Reality Capture Plugin - Open Beta

   1555   11   3
User Avatar
Staff
182 posts
Joined: Nov. 2012
Offline
Happy to announce that the Reality Capture plugin for Houdini just entered its Open Beta phase. Feel free to download it and let us know what you think of it. It does require a RC license, more info at the link below

https://www.sidefx.com/tutorials/reality-capture-plugin-open-beta/ [www.sidefx.com]
Luiz Kruel
Senior Technical Artist
SideFX
User Avatar
Member
2 posts
Joined: Jan. 2018
Offline
Hi
is it possible to install this on apprentice? i actually have a reality capture license and would love to try that out.
I have started Houdini, closed it, installed the plugin, and my license doesn't work anymore now.
User Avatar
Member
475 posts
Joined: Oct. 2016
Online
Hi, Luiz,

third attempt - the forum loves to kick me out while I am typing a comment.

I hope it is OK to start an open discussion about the open beta. We talked about a few issues that I consider important for any pipeline-integration of RC through Houdini. Control points clearly are one of those points - since if a model is broken up into components (due to poor capturing for example) you NEED to set control points to “glue it together”.

According to this comment on CR's support forum [support.capturingreality.com] you can both import 3d CPD and 2d (image) measurement data. If this works on a separate file (e.g. an XML per image, I haven't tested yet), it should be possible to import (through filesystem, but EVERYTHING is filesystem in RC anyway) control point data even from Houdini.
I will have a go and see if this works as advertised - if so, I can only hope that the SDK exposes the command as well, so it *should* be straight forward to integrate control port import into the plugin (as the current help file says, importing control point data is possible through the UI as well, so it should work. Should.)

Which would be “yeehaw” big time :-)

Marc
Edited by malbrecht - Aug. 23, 2018 03:54:31

Software Sewers Service (aka Pipeline Problems Solver)
https://vimeo.com/marcalbrecht [vimeo.com] - https://www.marc-albrecht.de [www.marc-albrecht.de]
User Avatar
Staff
182 posts
Joined: Nov. 2012
Offline
Hey Marc!

Yeah, please do, I'd love to hear what other people have to say as well.

Control Points is something really missing from the plugin right now, the main reason for that is on having to build a Control Point interface in Qt wouldn't be trivial, it's possible, but a significant time investment. It's definitely something we can add down the line.

In the meantime, exporting Control Points and Camera positions from RC and a way to send components back and forth is definitely something we want to work on.

But the heart of the plugin is to make a repeatable pipeline, so you can have multiple sets of files and not pay the cost of running through the process every time. This assumes that the pictures are clean and automatable, so not a lot of manual cleanup/control points needed. But having a way to interject some control is desired, so we'll keep that option on the table
Luiz Kruel
Senior Technical Artist
SideFX
User Avatar
Member
475 posts
Joined: Oct. 2016
Online
Hi, Luiz,

agreed … to some degree

Control points aren't only necessary for fixing alignment problems, but also for any kind of post-alignment (read: world space scaling and positioning). As soon as you are dealing with multi-part-scans (buildings, roads, environments or even high-detail objects) you need to be able to get the photogrammetry output into the same “reality” (positionwise) and that's where GCP come into play.

I wouldn't even go as far as expecting an interface inside Houdini. Because, USUALLY you will have information about where on an image control points are to be expected - either by a shape recognition pipeline or because you simply define your images to be perfect (I am kidding here, am I not).
Read: Even if the plugin only allows for attributes to be set on the images defining control point positions plus their respective IDs (which is how you import them into RC, I assume the SDK “ticks” the same way), that would be a lot of help already.

Marc

Software Sewers Service (aka Pipeline Problems Solver)
https://vimeo.com/marcalbrecht [vimeo.com] - https://www.marc-albrecht.de [www.marc-albrecht.de]
User Avatar
Member
428 posts
Joined: Oct. 2011
Offline
It would be great if the lower left status bar update could be a little more active. Some of the operations (align images and create model for example) can take a really long time and its hard to tell if there is any progress happening.

-b
http://www.racecar.no [www.racecar.no]
User Avatar
Staff
182 posts
Joined: Nov. 2012
Offline
MirkoBinyan seems like there was some weirdness, just got your post.

Yes you should be able to use this with apprentice. If you're getting errors take a look at the troubleshooting section.

I'm investigating a bug where the houdini.env file isn't generated properly, but the troubleshooting section has info on how to set yourself up properly
Luiz Kruel
Senior Technical Artist
SideFX
User Avatar
Member
2 posts
Joined: Jan. 2018
Offline
so it's not because of the plugin. Thanks, will try to solve that
User Avatar
Member
475 posts
Joined: Oct. 2016
Online
Over this last weekend, I had freed up the time to work on our multi-scale project in Houdini using the RC plugin. Thanks to the hardcoded license (not a fault of SideFX) this was impossible (license expired). I was able to create a new license.txt file using the SDK, but since the plugin uses its own “linker library”, the new SDK DLLs obviously didn't work …

My perspective for the time being is: The plugin opens up a lot of *possible* workflow improvements to my clients, effectively removing the need to write their own RC-replacement (using the SDK) and instead leveraging the power of Houdini. It is *the* foot-in-the-door for Houdini for their workflows. This is not only for photogrammetry, but for laser scanning AND MORE.
With the current licensing system (RC-side) this has now been put to rest. It is a kill-switch-feature: The need to go online for RC WHILE IN PRODUCTION is already a major PITA - the additional danger of having to wait for new hardcoded time-of-death DLLs has set a stop to this branch of the project. There have been talks about a change in the licensing system for RC, but this is muddy water from almost two years ago - and we can only base decisions on facts available at the moment. For the time being, the RC/Houdini solution is too fragile (again, this is *only* and *exclusively* because of the RC licensing problem).

Privately, I will continue to “play around” with the RC-Houdini-bridged power-solution. For my clients it's back to using CLI licenses for RC - dozens of them.

Software Sewers Service (aka Pipeline Problems Solver)
https://vimeo.com/marcalbrecht [vimeo.com] - https://www.marc-albrecht.de [www.marc-albrecht.de]
User Avatar
Member
475 posts
Joined: Oct. 2016
Online
A few thoughts - if I may - from my attempts to replace the CLI version with the Houdini plugin (by the way: The new compilation works!)

Wish for: Warning “no GPU found”
- for whatever reason on my Surface Book 2 the NVidia GPU got switched off while using Houdini/RC-plugin, which led to the crazy situation of the model having been built (on NVidia, because RC is using GPU only) but created display glitches (on the Intel GPU). After that the NVidia was “gone” and could not be used any more (until I unplugged/replugged the tablet part).
It would be helpful if the plugin could spit out a warning “no GPU found, RC only works with GPU”, because it was extremely hard to figure out what was causing the “nothing is happening” problems!

Wish for: Better progress indication
- we talked about this, I am just putting it down as a note: ANY better progress report would be helpful, and be it in a way that says “no, Houdini is NOT CRASHED, it is merely waiting for RC”. This could be a separate thread keeping Houdini alive (nurturing idle tasks) to avoid the screen going blind.

Desperately needed: Camera exports
- we tackled this before: It would be VERY helpful if the plugin could fire off an XMP export. Currently, we can only extract camera positions, which does help to some degree - but just like with flight logs the position information at times isn't precise enough. If the plugin could simply send a command “export XMP using the saved preference settings” (using a pre-made XML), the XMP stored in the images directory could be read out and their data used. That is how I do it in the CLI-based workflow.
Alternatively extracting (usable) rotation data would help, too

Wish for: Control Points
- again just noting this down: The ability to send in 2d position data to an image plus a Vec3 per point indicated in order to create either ground control points (to match up meshes with imports from different sources using the same world space) or to set reference points between images (to avoid camera-drop-outs, creating more than one component in RC) is somewhat “crucial” for any shot not being taken properly (which at production times can happen).

I hope this is helpful. I would welcome any exchange of thoughts with other users of this bridge solution - be it here openly on the forum or on some private channel.

Software Sewers Service (aka Pipeline Problems Solver)
https://vimeo.com/marcalbrecht [vimeo.com] - https://www.marc-albrecht.de [www.marc-albrecht.de]
User Avatar
Member
475 posts
Joined: Oct. 2016
Online
Moin,

I wonder if there is any progress on the plug-in. My current R&D project involves RC (again) and I stumbled over at least one critical bug in RC (the principal point for the lens undistortion coefficients seems to be calculated wrong) and one serious issue (undistorted images are exported with arbitrary crops, whereas other exports link original camera files, so there is no reproducible pixel-to-pixel relation between original images and undistorted exports, which makes any reprojection impossible).

Most of these issues could be worked around more easily if I could access RC parameters through Houdini (since I am trying to use Houdini for the R&D part). Also, the ground-point and reference-point setting mentioned before has proven to be most significant for any cooperation with “the outside world”.

Marc

Software Sewers Service (aka Pipeline Problems Solver)
https://vimeo.com/marcalbrecht [vimeo.com] - https://www.marc-albrecht.de [www.marc-albrecht.de]
User Avatar
Staff
182 posts
Joined: Nov. 2012
Offline
Hey Marc!

Good to hear from you.

The only recent work that was done was with regards to caching the files properly and a few quality of life improvements. We haven't done anything with the control points or other deeper points of integrations yet.
Luiz Kruel
Senior Technical Artist
SideFX
  • Quick Links