Building Octree: Octrees before rendering?

   12400   13   2
User Avatar
Member
37 posts
Joined: 6月 2008
Offline
Hi guys,

Building octrees takes a long time for heavy or complex geometry. I don't know much about octrees, are they in world/object space or camera based? If they are in some kind of world/object space it would be really cool to have a kind of delayed load octree which could be baked out before rendering allowing mantra to skip the octree creation step and start rendering more quickly.

I THINK that's the idea with vrmeshes in VRay. They can't be displaced or altered after they're made but renders start immediately.

Would love to hear other peoples thoughts and please correct me if I'm wrong in understanding this.
User Avatar
Member
7046 posts
Joined: 7月 2005
Offline
Yeah I'd love for this to be faster. I think it got a little bit threaded in H12 but it still sucks up a lot of time before you see an image…
Cheers,

Peter Bowmar
____________
Houdini 20.5.262 Win 10 Py 3.11
User Avatar
Member
334 posts
Joined: 7月 2007
Offline
Yeah would be greate to have the ability to store the occtree with a delayed load. +1!

Maybe submit an rfe?
www.gimpville.no
User Avatar
Member
379 posts
Joined: 12月 2006
Offline
Definitely +1 for this.
User Avatar
Member
4271 posts
Joined: 7月 2005
Offline
Before the wave of +1's…..

I guess some questions and caveats. I suspect the Octree's positional data is currently stored in the camera's space and not world. So that would have to be adjusted.

Also if you are saving out the octree its possible that you'll have limitations like not being able to change certain attributes like Subdivision surfaces.

Maybe instead of saving out Mantra's internal octree structure, support was added for a random access geometry format. Like extend the TBF/point cloud files to support full geometry sets. That way when tracing against geo archives you don't need to load the full mesh in, only the relevant section. (This is how a vrmesh (vray proxy) works).

http://www.spot3d.com/vray/help/150SP1/vrayproxy_params.htm#vrmesh [spot3d.com]
if(coffees<2,round(float),float)
User Avatar
Member
1743 posts
Joined: 3月 2012
Offline
Hi everyone!

I've just committed an attempt at parallelizing the construction of k-D trees (what's currently used in place of octrees), so I'd be really interested to see how it works for ya. Tomorrow's daily build should have the change, so long as that build works.

Caveats: For some renders, it temporarily uses a bit more memory. Also, on renders where the total amount of memory used approaches or exceeds the machine's physical memory, the performance may be as poor as before. That one's kind of expected, since there's not much that can be done with just a few weeks work to reduce paging out to disk.
Writing code for fun and profit since... 2005? Wow, I'm getting old.
https://www.youtube.com/channel/UC_HFmdvpe9U2G3OMNViKMEQ [www.youtube.com]
User Avatar
Member
7046 posts
Joined: 7月 2005
Offline
Cool! Let's hope it's not Donut Day
Cheers,

Peter Bowmar
____________
Houdini 20.5.262 Win 10 Py 3.11
User Avatar
Member
1743 posts
Joined: 3月 2012
Offline
pbowmar
Cool! Let's hope it's not Donut Day
I'll put Tim Horton's on speed dial. :wink:
Writing code for fun and profit since... 2005? Wow, I'm getting old.
https://www.youtube.com/channel/UC_HFmdvpe9U2G3OMNViKMEQ [www.youtube.com]
User Avatar
Member
334 posts
Joined: 7月 2007
Offline
Wow cant wait to try it out!

Does this mean full threading of octree genetration? Or more threading? Hoping for the first!
www.gimpville.no
User Avatar
Member
37 posts
Joined: 6月 2008
Offline
ndickson
Hi everyone!

I've just committed an attempt at parallelizing the construction of k-D trees (what's currently used in place of octrees), so I'd be really interested to see how it works for ya. Tomorrow's daily build should have the change, so long as that build works.

Caveats: For some renders, it temporarily uses a bit more memory. Also, on renders where the total amount of memory used approaches or exceeds the machine's physical memory, the performance may be as poor as before. That one's kind of expected, since there's not much that can be done with just a few weeks work to reduce paging out to disk.

Awesome can't wait to try it out!
User Avatar
Member
4271 posts
Joined: 7月 2005
Offline
ndickson
That one's kind of expected, since there's not much that can be done with just a few weeks work to reduce paging out to disk.

KD-Tree stuff aside, these efforts are equally welcomed.
if(coffees<2,round(float),float)
User Avatar
Member
1743 posts
Joined: 3月 2012
Offline
Korhon
Does this mean full threading of octree genetration? Or more threading? Hoping for the first!
It's just the building of the geometry k-D tree structure used by mantra (presumably, that's most of what it's doing when it says “Building octree…”). This building was all serial before. Specifically, it:
  1. Builds a temporary k-D tree structure (that can be built in parallel), recursing in parallel
  2. Estimates the amount of memory used, recursing in serial, (fairly fast, so parallelization might not make a difference)
  3. Builds the usual k-D tree structure from the temporary structure, recursing in parallel
  4. Deletes the tempoary structure, recursing in parallel
  5. Reorders the bounding boxes pointed to by the k-D tree, in serial (because parallelization would get messy and might not help)

    Of course, all bets are off if memory pages to disk, but when using 12 cores, building the temporary k-D structure seems to take 65-75% of the time and reordering the boxes takes maybe 20% of the time.
Writing code for fun and profit since... 2005? Wow, I'm getting old.
https://www.youtube.com/channel/UC_HFmdvpe9U2G3OMNViKMEQ [www.youtube.com]
User Avatar
Member
12602 posts
Joined: 7月 2005
Offline
This is great, thank you very much.

Can we expect this work to accelerate building pointcloud KD-Trees and those Houdini operations that use KD-Trees too? (eg VolumeFromAttrib?)
Jason Iversen, Technology Supervisor & FX Pipeline/R+D Lead @ Weta FX
also, http://www.odforce.net [www.odforce.net]
User Avatar
Member
1743 posts
Joined: 3月 2012
Offline
jason_iversen
Can we expect this work to accelerate building pointcloud KD-Trees and those Houdini operations that use KD-Trees too? (eg VolumeFromAttrib?)
Once I've cleaned up a few things, I may be working on that next, (unless I'm confusing it with something else). It sounds like it should be easier than parallelizing the k-D tree building in mantra, but I haven't looked at it myself.

Wolfwood
ndickson
That one's kind of expected, since there's not much that can be done with just a few weeks work to reduce paging out to disk.
KD-Tree stuff aside, these efforts are equally welcomed.

I hear ya. Paging to disk can bring a program to its knees. Reducing it usually involves a massive redesign/reorganization of how memory is used, as well as assumptions on how the program is commonly used, and sometimes even a complete change of how the program behaves.
Writing code for fun and profit since... 2005? Wow, I'm getting old.
https://www.youtube.com/channel/UC_HFmdvpe9U2G3OMNViKMEQ [www.youtube.com]
  • Quick Links