Happy to announce that the Reality Capture plugin for Houdini just entered its Open Beta phase. Feel free to download it and let us know what you think of it. It does require a RC license, more info at the link below
https://www.sidefx.com/tutorials/reality-capture-plugin-open-beta/ [www.sidefx.com]
Full Version: Reality Capture Plugin - Open Beta
Hi
is it possible to install this on apprentice? i actually have a reality capture license and would love to try that out.
I have started Houdini, closed it, installed the plugin, and my license doesn't work anymore now.
is it possible to install this on apprentice? i actually have a reality capture license and would love to try that out.
I have started Houdini, closed it, installed the plugin, and my license doesn't work anymore now.
Hi, Luiz,
third attempt - the forum loves to kick me out while I am typing a comment.
I hope it is OK to start an open discussion about the open beta. We talked about a few issues that I consider important for any pipeline-integration of RC through Houdini. Control points clearly are one of those points - since if a model is broken up into components (due to poor capturing for example) you NEED to set control points to “glue it together”.
According to this comment on CR's support forum [support.capturingreality.com] you can both import 3d CPD and 2d (image) measurement data. If this works on a separate file (e.g. an XML per image, I haven't tested yet), it should be possible to import (through filesystem, but EVERYTHING is filesystem in RC anyway) control point data even from Houdini.
I will have a go and see if this works as advertised - if so, I can only hope that the SDK exposes the command as well, so it *should* be straight forward to integrate control port import into the plugin (as the current help file says, importing control point data is possible through the UI as well, so it should work. Should.)
Which would be “yeehaw” big time :-)
Marc
third attempt - the forum loves to kick me out while I am typing a comment.
I hope it is OK to start an open discussion about the open beta. We talked about a few issues that I consider important for any pipeline-integration of RC through Houdini. Control points clearly are one of those points - since if a model is broken up into components (due to poor capturing for example) you NEED to set control points to “glue it together”.
According to this comment on CR's support forum [support.capturingreality.com] you can both import 3d CPD and 2d (image) measurement data. If this works on a separate file (e.g. an XML per image, I haven't tested yet), it should be possible to import (through filesystem, but EVERYTHING is filesystem in RC anyway) control point data even from Houdini.
I will have a go and see if this works as advertised - if so, I can only hope that the SDK exposes the command as well, so it *should* be straight forward to integrate control port import into the plugin (as the current help file says, importing control point data is possible through the UI as well, so it should work. Should.)
Which would be “yeehaw” big time :-)
Marc
Hey Marc!
Yeah, please do, I'd love to hear what other people have to say as well.
Control Points is something really missing from the plugin right now, the main reason for that is on having to build a Control Point interface in Qt wouldn't be trivial, it's possible, but a significant time investment. It's definitely something we can add down the line.
In the meantime, exporting Control Points and Camera positions from RC and a way to send components back and forth is definitely something we want to work on.
But the heart of the plugin is to make a repeatable pipeline, so you can have multiple sets of files and not pay the cost of running through the process every time. This assumes that the pictures are clean and automatable, so not a lot of manual cleanup/control points needed. But having a way to interject some control is desired, so we'll keep that option on the table
Yeah, please do, I'd love to hear what other people have to say as well.
Control Points is something really missing from the plugin right now, the main reason for that is on having to build a Control Point interface in Qt wouldn't be trivial, it's possible, but a significant time investment. It's definitely something we can add down the line.
In the meantime, exporting Control Points and Camera positions from RC and a way to send components back and forth is definitely something we want to work on.
But the heart of the plugin is to make a repeatable pipeline, so you can have multiple sets of files and not pay the cost of running through the process every time. This assumes that the pictures are clean and automatable, so not a lot of manual cleanup/control points needed. But having a way to interject some control is desired, so we'll keep that option on the table
Hi, Luiz,
agreed … to some degree
Control points aren't only necessary for fixing alignment problems, but also for any kind of post-alignment (read: world space scaling and positioning). As soon as you are dealing with multi-part-scans (buildings, roads, environments or even high-detail objects) you need to be able to get the photogrammetry output into the same “reality” (positionwise) and that's where GCP come into play.
I wouldn't even go as far as expecting an interface inside Houdini. Because, USUALLY you will have information about where on an image control points are to be expected - either by a shape recognition pipeline or because you simply define your images to be perfect (I am kidding here, am I not).
Read: Even if the plugin only allows for attributes to be set on the images defining control point positions plus their respective IDs (which is how you import them into RC, I assume the SDK “ticks” the same way), that would be a lot of help already.
Marc
agreed … to some degree
Control points aren't only necessary for fixing alignment problems, but also for any kind of post-alignment (read: world space scaling and positioning). As soon as you are dealing with multi-part-scans (buildings, roads, environments or even high-detail objects) you need to be able to get the photogrammetry output into the same “reality” (positionwise) and that's where GCP come into play.
I wouldn't even go as far as expecting an interface inside Houdini. Because, USUALLY you will have information about where on an image control points are to be expected - either by a shape recognition pipeline or because you simply define your images to be perfect (I am kidding here, am I not).
Read: Even if the plugin only allows for attributes to be set on the images defining control point positions plus their respective IDs (which is how you import them into RC, I assume the SDK “ticks” the same way), that would be a lot of help already.
Marc
It would be great if the lower left status bar update could be a little more active. Some of the operations (align images and create model for example) can take a really long time and its hard to tell if there is any progress happening.
-b
-b
MirkoBinyan seems like there was some weirdness, just got your post.
Yes you should be able to use this with apprentice. If you're getting errors take a look at the troubleshooting section.
I'm investigating a bug where the houdini.env file isn't generated properly, but the troubleshooting section has info on how to set yourself up properly
Yes you should be able to use this with apprentice. If you're getting errors take a look at the troubleshooting section.
I'm investigating a bug where the houdini.env file isn't generated properly, but the troubleshooting section has info on how to set yourself up properly
so it's not because of the plugin. Thanks, will try to solve that
Over this last weekend, I had freed up the time to work on our multi-scale project in Houdini using the RC plugin. Thanks to the hardcoded license (not a fault of SideFX) this was impossible (license expired). I was able to create a new license.txt file using the SDK, but since the plugin uses its own “linker library”, the new SDK DLLs obviously didn't work …
My perspective for the time being is: The plugin opens up a lot of *possible* workflow improvements to my clients, effectively removing the need to write their own RC-replacement (using the SDK) and instead leveraging the power of Houdini. It is *the* foot-in-the-door for Houdini for their workflows. This is not only for photogrammetry, but for laser scanning AND MORE.
With the current licensing system (RC-side) this has now been put to rest. It is a kill-switch-feature: The need to go online for RC WHILE IN PRODUCTION is already a major PITA - the additional danger of having to wait for new hardcoded time-of-death DLLs has set a stop to this branch of the project. There have been talks about a change in the licensing system for RC, but this is muddy water from almost two years ago - and we can only base decisions on facts available at the moment. For the time being, the RC/Houdini solution is too fragile (again, this is *only* and *exclusively* because of the RC licensing problem).
Privately, I will continue to “play around” with the RC-Houdini-bridged power-solution. For my clients it's back to using CLI licenses for RC - dozens of them.
My perspective for the time being is: The plugin opens up a lot of *possible* workflow improvements to my clients, effectively removing the need to write their own RC-replacement (using the SDK) and instead leveraging the power of Houdini. It is *the* foot-in-the-door for Houdini for their workflows. This is not only for photogrammetry, but for laser scanning AND MORE.
With the current licensing system (RC-side) this has now been put to rest. It is a kill-switch-feature: The need to go online for RC WHILE IN PRODUCTION is already a major PITA - the additional danger of having to wait for new hardcoded time-of-death DLLs has set a stop to this branch of the project. There have been talks about a change in the licensing system for RC, but this is muddy water from almost two years ago - and we can only base decisions on facts available at the moment. For the time being, the RC/Houdini solution is too fragile (again, this is *only* and *exclusively* because of the RC licensing problem).
Privately, I will continue to “play around” with the RC-Houdini-bridged power-solution. For my clients it's back to using CLI licenses for RC - dozens of them.
A few thoughts - if I may - from my attempts to replace the CLI version with the Houdini plugin (by the way: The new compilation works!)
Wish for: Warning “no GPU found”
- for whatever reason on my Surface Book 2 the NVidia GPU got switched off while using Houdini/RC-plugin, which led to the crazy situation of the model having been built (on NVidia, because RC is using GPU only) but created display glitches (on the Intel GPU). After that the NVidia was “gone” and could not be used any more (until I unplugged/replugged the tablet part).
It would be helpful if the plugin could spit out a warning “no GPU found, RC only works with GPU”, because it was extremely hard to figure out what was causing the “nothing is happening” problems!
Wish for: Better progress indication
- we talked about this, I am just putting it down as a note: ANY better progress report would be helpful, and be it in a way that says “no, Houdini is NOT CRASHED, it is merely waiting for RC”. This could be a separate thread keeping Houdini alive (nurturing idle tasks) to avoid the screen going blind.
Desperately needed: Camera exports
- we tackled this before: It would be VERY helpful if the plugin could fire off an XMP export. Currently, we can only extract camera positions, which does help to some degree - but just like with flight logs the position information at times isn't precise enough. If the plugin could simply send a command “export XMP using the saved preference settings” (using a pre-made XML), the XMP stored in the images directory could be read out and their data used. That is how I do it in the CLI-based workflow.
Alternatively extracting (usable) rotation data would help, too
Wish for: Control Points
- again just noting this down: The ability to send in 2d position data to an image plus a Vec3 per point indicated in order to create either ground control points (to match up meshes with imports from different sources using the same world space) or to set reference points between images (to avoid camera-drop-outs, creating more than one component in RC) is somewhat “crucial” for any shot not being taken properly (which at production times can happen).
I hope this is helpful. I would welcome any exchange of thoughts with other users of this bridge solution - be it here openly on the forum or on some private channel.
Wish for: Warning “no GPU found”
- for whatever reason on my Surface Book 2 the NVidia GPU got switched off while using Houdini/RC-plugin, which led to the crazy situation of the model having been built (on NVidia, because RC is using GPU only) but created display glitches (on the Intel GPU). After that the NVidia was “gone” and could not be used any more (until I unplugged/replugged the tablet part).
It would be helpful if the plugin could spit out a warning “no GPU found, RC only works with GPU”, because it was extremely hard to figure out what was causing the “nothing is happening” problems!
Wish for: Better progress indication
- we talked about this, I am just putting it down as a note: ANY better progress report would be helpful, and be it in a way that says “no, Houdini is NOT CRASHED, it is merely waiting for RC”. This could be a separate thread keeping Houdini alive (nurturing idle tasks) to avoid the screen going blind.
Desperately needed: Camera exports
- we tackled this before: It would be VERY helpful if the plugin could fire off an XMP export. Currently, we can only extract camera positions, which does help to some degree - but just like with flight logs the position information at times isn't precise enough. If the plugin could simply send a command “export XMP using the saved preference settings” (using a pre-made XML), the XMP stored in the images directory could be read out and their data used. That is how I do it in the CLI-based workflow.
Alternatively extracting (usable) rotation data would help, too
Wish for: Control Points
- again just noting this down: The ability to send in 2d position data to an image plus a Vec3 per point indicated in order to create either ground control points (to match up meshes with imports from different sources using the same world space) or to set reference points between images (to avoid camera-drop-outs, creating more than one component in RC) is somewhat “crucial” for any shot not being taken properly (which at production times can happen).
I hope this is helpful. I would welcome any exchange of thoughts with other users of this bridge solution - be it here openly on the forum or on some private channel.
Moin,
I wonder if there is any progress on the plug-in. My current R&D project involves RC (again) and I stumbled over at least one critical bug in RC (the principal point for the lens undistortion coefficients seems to be calculated wrong) and one serious issue (undistorted images are exported with arbitrary crops, whereas other exports link original camera files, so there is no reproducible pixel-to-pixel relation between original images and undistorted exports, which makes any reprojection impossible).
Most of these issues could be worked around more easily if I could access RC parameters through Houdini (since I am trying to use Houdini for the R&D part). Also, the ground-point and reference-point setting mentioned before has proven to be most significant for any cooperation with “the outside world”.
Marc
I wonder if there is any progress on the plug-in. My current R&D project involves RC (again) and I stumbled over at least one critical bug in RC (the principal point for the lens undistortion coefficients seems to be calculated wrong) and one serious issue (undistorted images are exported with arbitrary crops, whereas other exports link original camera files, so there is no reproducible pixel-to-pixel relation between original images and undistorted exports, which makes any reprojection impossible).
Most of these issues could be worked around more easily if I could access RC parameters through Houdini (since I am trying to use Houdini for the R&D part). Also, the ground-point and reference-point setting mentioned before has proven to be most significant for any cooperation with “the outside world”.
Marc
Hey Marc!
Good to hear from you.
The only recent work that was done was with regards to caching the files properly and a few quality of life improvements. We haven't done anything with the control points or other deeper points of integrations yet.
Good to hear from you.
The only recent work that was done was with regards to caching the files properly and a few quality of life improvements. We haven't done anything with the control points or other deeper points of integrations yet.
Hi
Has the plugin expired by any chance or is something else going on. I used the RC nodes to align photos of a object. That rc file is now cached but it shows up every time i create a new rc_align node, even when the images are completely different. Once i press the align images again it performs a calculation and turns no result in the view port.
I am a bit stumped by this. Not sure if its something to do with RC licensing or something I am doing wrong.
Has the plugin expired by any chance or is something else going on. I used the RC nodes to align photos of a object. That rc file is now cached but it shows up every time i create a new rc_align node, even when the images are completely different. Once i press the align images again it performs a calculation and turns no result in the view port.
I am a bit stumped by this. Not sure if its something to do with RC licensing or something I am doing wrong.
Not claiming to know the answer, I can tell that the latest release of RealityCapture has broken a lot of pipelines for us. I could imagine the API having been changed as well with zero documentation about the differences, so it is quite possible that the plugin would have to be “re-checked” feature by feature.
I'm looking into it right now. The caching is a know bug if your scene isn't saved, we store the cache based on the filename, but if you have 2 files with the same name (like untitled) it will tend to pull up an old cache.
The recent release hasn't affected us yet since there isn't a new API for the PPI features as of yet.
The recent release hasn't affected us yet since there isn't a new API for the PPI features as of yet.
Ok. In the meantime I'll try to see if i get the error on another computer. In the meantime is there a way to bypass this issue ?
Unfortunately no. I'm in contact with the Reality Capture team right now and will report back as soon as I have a resolution. Hopefully today.
Sorry for the inconvenience
Sorry for the inconvenience
Awesome. Thank you for the help.
Cheers
Cheers
Just uploaded a new plugin that is on the latest SDK. Should be working now but do let me know if there are any weirdness.
I'm planning to spend some time in the near future to do another pass and take advantage of PPI and some of the other new features.
It'll be on Tomorrow's daily GameDev build
Sorry for the inconvenience.
Luiz
I'm planning to spend some time in the near future to do another pass and take advantage of PPI and some of the other new features.
It'll be on Tomorrow's daily GameDev build
Sorry for the inconvenience.
Luiz
I've been using it until now it was fine, now the rc_align images no longer functions. ( license and everything else is fine, no errors) Pressing the align images seems to run something then completely stops showing no result in the viewport. I also tried different versions of Houdini/gamedev build and recreating the project from scratch in a different location. No luck so far.
Is it again the caching issue ? ( using rc align in a new project will give me a random point cloud unrelated to the images i input)
Thank you in advance
Is it again the caching issue ? ( using rc align in a new project will give me a random point cloud unrelated to the images i input)
Thank you in advance