Render 50-240 mil polygons bgeo file???

   13512   20   6
User Avatar
Member
217 posts
Joined: 7月 2007
Offline
Hi,

I want to ask for some workflow, if you want to render
50, 140 or 240 mil. polygons bgeo file?

With 50mil poly Mantra use 11GB of memory, with 140 mantra killl my workstation( 24GB).

With instance everything is great 9 trees * 50 000 instance
4500mil polygons - PBR 9min per frame, this is amazing.

But with one big bgeo file…

Igor
WEB
http://www.igorfx.com/ [igorfx.com]

Vimeo
https://vimeo.com/igorzanic [vimeo.com]
User Avatar
Member
379 posts
Joined: 12月 2006
Offline
Maybe it is possible to separate that big bgeo file into several chunks?
Using Micropolygon ?
Maybe Delayed Load will help?
User Avatar
Member
217 posts
Joined: 7月 2007
Offline
Hi SreckoM,

I am using Delayed Load, with BoundingBox set, micro, PBR, no shaders, no light. Just simple geo. Mantra use 11GB just to start.

This geo is one big ocean surface with 50mil polygons.

I ask few HoudiniTD's, but answer is same, this is big problem for Houdini and Mantra.
WEB
http://www.igorfx.com/ [igorfx.com]

Vimeo
https://vimeo.com/igorzanic [vimeo.com]
User Avatar
Member
665 posts
Joined: 7月 2005
Offline
Hi Ceegee,

I feel your pain. Rendering such a large Object (both in space, and complexity) can be a nightmare.

Make sure there are no attributes on there you don't want. Just P, possibly N, and v. Consider breaking the Grid into several different grids so that Mantra only has to load one at a time.

Most importantly though, I'd recommend using Displacement maps. Most approaches I've seen in the past (including my own) Generate a low res grid, then apply the high detail at render time.

Good luck, and curious to hear how you solve it!
-j
User Avatar
Member
217 posts
Joined: 7月 2007
Offline
Hi Jacob,

Yes my geometry is very clean, but sometime i need to use 50-100mil mesh geometry, i can split, but this can also be nightmare.

Yes displacment is one option, i have some crazy splashes.

You can see on image, and this is low res sim.

This is for my personal project.

Attachments:
render.jpg (195.3 KB)

WEB
http://www.igorfx.com/ [igorfx.com]

Vimeo
https://vimeo.com/igorzanic [vimeo.com]
User Avatar
Member
665 posts
Joined: 7月 2005
Offline
Huh, with out seeing that picture, I would say why are you splashes in the same geo as the surface.

With seeing the picture, I now say, why don't I render them together?!

I agree that this is a memory management scenario. I've never had much luck brute forcing these scenarios in the past. Remember you're not making 2 different renders, just multiple pieces of geo in the same render to load one at a time. Splitting things up is lame, but I am always keen to learn of new techniques I have missed!

Looks good, hope to see it rendered one day!

-j
User Avatar
スタッフ
6367 posts
Joined: 7月 2005
Offline
Very nice looking picture there!

Our back-of-the-envelope estimates suggest your memory usages are to be expected. Which makes us sad, as we'd like to be able to render that sort of thing!

Could you please very kindly submit the high res file for our analysis? Hopefully we can use it to guide our optimization efforts. Or just brute-force it on a machine with lots of RAM.

If you could send it to: ftp.sidefx.com and let us know the file name, it would be quite appreciated. If it is the reproducible output from a sim, we can re-sim at as well if that cuts bandwidth.
User Avatar
Member
217 posts
Joined: 7月 2007
Offline
Hi Jacob,

I am already optimizing as much as i can, but sometime you need to render big amount of polygons, for example Vray render 35mil poly vrmesh with 2GB of ram. So i hope we can do same thing with Houdini.

Hi jlait,

thanks a lot i really appreciate it.

And this is first render, everything is low res.

http://www.igorfx.com/houdini/igor_water_RND_v001.mov [igorfx.com]
WEB
http://www.igorfx.com/ [igorfx.com]

Vimeo
https://vimeo.com/igorzanic [vimeo.com]
User Avatar
Member
134 posts
Joined: 3月 2009
Offline
I can second this memory problem with mantra (which is a big problem, were constantly fighting this issue).

Were doing heavy fluid sims right now for a feature production and when having highres fluid meshes of 100+ mil polys there is no way to render on a 24GB farm node, even 48GB blades is having trouble sometimes when having a single big bgeo. We have tested same geo in Renderman and it just eats millions of polys for breakfast while mantra eats the ram instead hehe.
Magnus Pettersson
Lead Effects TD @ Storm studios
User Avatar
Member
217 posts
Joined: 7月 2007
Offline
Hi jlait,

I don't have access to FTP, so I have upload files to my website i hope this is ok.

Mantra use 11-12GB
http://www.igorfx.com/houdini/ocean_50mil.rar [igorfx.com]

Mantra use 24GB+
http://www.igorfx.com/houdini/ocean_140mil.rar [igorfx.com]

Thanks a lot, i realy want to render this with houdini.

Igor
WEB
http://www.igorfx.com/ [igorfx.com]

Vimeo
https://vimeo.com/igorzanic [vimeo.com]
User Avatar
Member
1002 posts
Joined: 7月 2005
Offline
Try reducing the KD-Tree memory factor to 0.5 or 0.25. You might pay for this with a small loss in rendering performance but the memory use for the ray tracing KD-Tree will be reduced.

Andrew
User Avatar
Member
217 posts
Joined: 7月 2007
Offline
Hi Andrew

Still same problem kd-tree 0.5, 0.25, 0.1, 0.01 same problem
When i press render mplay using 5GB then on creating geometry mantra use another 5GB and render start.

If i use small patch of ocean 1mil ( bgeo seq) and then in houdini make 50 geo nodes and use this seq as delayed load, but move geo nodes to make one big surface mantra use 0.5GB of ram
- whale_rnd.jpg

With point instancing everything great 5 trees 90 000 per tree * 50 000
4500mil,i know instance use less memory.

Attachments:
whale_rnd.jpg (437.1 KB)
tree_4500mil.jpg (252.1 KB)
tree_4500mil.mov (4.3 MB)

WEB
http://www.igorfx.com/ [igorfx.com]

Vimeo
https://vimeo.com/igorzanic [vimeo.com]
User Avatar
スタッフ
6367 posts
Joined: 7月 2005
Offline
I got your files, thank you.

They really highlight the inefficiencies of our polygon soup format. This is something Andrew knows about and was pushing to get fixed with a poly-patch style primitive, but it did not make it for Houdini 12.

For example, by tristripping the 145,590,048 poly file I end up with 10,208 tristrips - a rather significant reduction! Vertex count also drops from 436 million to 145 million.

ginfo -M -T values (which match what mantra takes) report

11.19GB - poly
1.36GB - tristrip

Unfortunately, the tristrip sop took > 160 GB of RAM, so it isn't something to try at home.
User Avatar
Member
258 posts
Joined:
Offline
All I want to say is you got my props for even going down this road. That whale shot is sick as is all the work I have seen from you.
User Avatar
Member
2199 posts
Joined: 7月 2005
Online
One thing we do/did, in H8 days and before, is use the camera settings to render in blocks, cutting the image up into smaller pieces. I don't know if that technique will work with one massive piece of geometry but it works great when you split stuff into lots of models.
You would think internally mantra was doing the same optimisation but that never seemed to be the case. It was the only way we could render massive files inside the now really old limit of 1Gb (huh remember those good old days fondly)
The trick is finding just the right hammer for every screw
User Avatar
Member
334 posts
Joined: 7月 2007
Offline
Having same issue when rendering lots of flip sims. Mantra sure loves my ram! I think this will be a more common issue for many, since its now possible to simulate and mesh very hires fluidsims.
www.gimpville.no
User Avatar
Member
217 posts
Joined: 7月 2007
Offline
Thanks sl0throp.

Hi Simon, yes i try that also but i think that problem is that mantra load full geo in ram, and then use that for render.

As Korhon, this can be major problem for rendering big and detail splashes.

imagine that you need to render 78-80mil poly river

This is one simple example with 20mil and all layers.

My old naiad sim and h12 PBR render.

http://www.igorfx.com/houdini/flood_PBR.mov [igorfx.com]

Igor
WEB
http://www.igorfx.com/ [igorfx.com]

Vimeo
https://vimeo.com/igorzanic [vimeo.com]
User Avatar
Member
37 posts
Joined: 6月 2008
Offline
Hi CeeGee,

I hope this isn't too off topic. Can you talk about the shading techniques you're using?

Are you using uniform volumes to get the light attenuation in your refractions? The whale render is working really nicely for that.

Also are you using vorticity or velocity to drive the shading on the particles or the core mesh?

Keep up the great work!
User Avatar
Member
334 posts
Joined: 7月 2007
Offline
Another offtopic! 8)

Have to done any comparisons between naiad and h12 flip Igor? I find the new flipsolver to be very fast, but i dont got any naiad experience to compare with.

Ive made a flip foam tool that works like naiads splash paricles. Im guessing naiad can create foam particles much faster than houdini pops, since they dont thread.
www.gimpville.no
User Avatar
Member
217 posts
Joined: 7月 2007
Offline
Hi burgess3d,

There is nothing special in my render, everything is standard mantra surface, with one sunlight and envlight.

Velocity and vorticity in this example is just particle render with color from this attributes and use later in comp.
And for splash in naiad i generate normal from fluid and transfer to splash particles and on geo node i use render as point and use normal for render.

Igor
WEB
http://www.igorfx.com/ [igorfx.com]

Vimeo
https://vimeo.com/igorzanic [vimeo.com]
  • Quick Links