Large sim files

   7781   13   2
User Avatar
Member
61 posts
Joined: Feb. 2011
Offline
I've been trying cache out my .sim files, because my Houdini keeps crashing during rendering to cache out the geometry particles of the simulation.
So that I can continue the simulation from the last frame it had cached out.
Which, by the way, also gives me issues.

I've done it with a different simulation, with similar amounts of particles, and the files there didn't get larger than about 250MB. But with the current simulation, where I generate particles for only 5 frames, they go up to 5.3GB, getting larger every frame and it's not even half way the simulation.
Is there a reason why the .sim files keep getting bigger?

Also, I thought that when you check on “Allow Caching to Disk” and “Save Checkpoints” that Houdini doesn't use your RAM anymore to store the data in.
But currently it's still filling up the RAM, which is causing my Houdini to crash.
How can I keep it from using my RAM?

Then for the other issue of picking up the simulation from the last chached .sim file.
How exactly does that work?
Because activating “Playback Simulation” seems to only play the frames it has and then simulate nothing after.
Also, having it play without “Playback Simulation” on gives me the same result.

Thanks in advance!

Attachments:
ZombieHEad_files.zip (1.0 MB)

- A little nonsense now and then, is relished by the wisest men.
User Avatar
Member
443 posts
Joined: Sept. 2012
Offline
I've done it with a different simulation, with similar amounts of particles, and the files there didn't get larger than about 250MB. But with the current simulation, where I generate particles for only 5 frames, they go up to 5.3GB, getting larger every frame and it's not even half way the simulation.
Is there a reason why the .sim files keep getting bigger?

.sim files store simulation data to continue simulation. That's why these files are larger than .bgeo and tak longer to load up.

This is not ideal way of working. Find out why Houdini keeps on crashing and write out .bgeo.
User Avatar
Member
443 posts
Joined: Sept. 2012
Offline
Then for the other issue of picking up the simulation from the last chached .sim file.
How exactly does that work?
Because activating “Playback Simulation” seems to only play the frames it has and then simulate nothing after.
Also, having it play without “Playback Simulation” on gives me the same result.

You need to read .sim files either in file dop or file sop. Preferably file sop.
User Avatar
Member
61 posts
Joined: Feb. 2011
Offline
I realize .sim files are larger than .bgeo.
The question was rather why they were going over 5GB while in a different yet similar simulation the .sim files didn't even reach 300MB.
Note that there are similar amounts of particles and attributes used in the simulations I'm comparing.

Find out why Houdini keeps on crashing and write out .bgeo
Can DOPs continue with a simulation from .bgeo files?

And I think the main reason why Houdini crashes is because it runs out of memory to use.
That's why I'm eager to know why Houdini keeps filling up my RAM when I tell it to write to disk.
- A little nonsense now and then, is relished by the wisest men.
User Avatar
Member
443 posts
Joined: Sept. 2012
Offline
What platform are you on?
Houdini using 27.8GB RAM on my CentOS.
User Avatar
Member
443 posts
Joined: Sept. 2012
Offline
And I think the main reason why Houdini crashes is because it runs out of memory to use.
That's why I'm eager to know why Houdini keeps filling up my RAM when I tell it to write to disk.


Houdini works better on real world units (SI). Your head geometry is way to big. It's 20 x 30 x 20 = 12000 cubic meter that means it's huge water tank.

I scaled it down to real world units and it works fine, very fast and taking only 1GB of RAM.
Edited by - March 2, 2015 11:24:36
User Avatar
Member
443 posts
Joined: Sept. 2012
Offline
Can DOPs continue with a simulation from .bgeo files?

Absolutely not but I don't know what you exactly mean by that.
User Avatar
Member
61 posts
Joined: Feb. 2011
Offline
Hmm, the size tip might actually help.
Thanks for that ^^

What I mean with continuing with the simulation is if Houdini crashes while it is still sim'ing out, then you could open Houdini and continue from the last frame it had cached out and wrote to disk.
So that if you have a simulation of 100 frames and it crashed after sim'ing frame 50, you can continue from frame 50 without having to re-sim the first 49 frames.
So I'm currently under the assumption that Houdini needs .sim files to do that and that .bgeo won't work because it doesn't have all that simulation data. I guess?

I found where I read that.
It's in the info you get when hovering over “Save Checkpoints” on a DOP Network.

I'll let you know how it went after making everything smaller.

Thanks!
- A little nonsense now and then, is relished by the wisest men.
User Avatar
Member
443 posts
Joined: Sept. 2012
Offline
I want to suggest you some things to enhance your workflow.
1. Setting up viscosity everywhere doesn't make any sense. It really painful to change settings on so many places. If you want to read viscosity by attribute from sop then Fluid Source SOP have that option no need to add extra attributecreate and no need to set it on Flip Object DOP. Any way it's going to overwritten.

2. You can avoid color transfer from sphere by using paint sop.

3. CHOPs become very slow on large amount of points so avoid those on large number of points.

4. Avoid point SOP since you already used to AttribWrangle and AttribVop.

5. Avoid messing up with gravity. It results in unstable simulation.
User Avatar
Member
61 posts
Joined: Feb. 2011
Offline
Thanks for the advice.

This is kind of off topic now and I think my issue has been fixed by making everything smaller, but I just want to put it out there.

The reason I put viscosity in SOPs and bring it in DOPs is because I'm giving it custom values.
When I set it on the Fluid Source SOP, it gives all the points the same viscosity.
My intention is to give each point a different viscosity value at birth and have them keep that value for the entire simulation.
This seems to work only like this, as far as I have found.
I've searched for it on this forum and I saw people suggesting the method I'm using now.
Everything is working and adjustable the way I would like it.
So YAY! I guess ^^

For the gravity, I had put it twice as strong because it seemed to be too weak.
That was most likely cause because I was working in a large scale.
So now that I made everything smaller I'm using the normal gravity again.

Thanks again for the help!
- A little nonsense now and then, is relished by the wisest men.
User Avatar
Member
443 posts
Joined: Sept. 2012
Offline
I mean no need to use attribcreat to add viscosity. Set it on fluidsounce then modify down the chain. Points I figured out are based on my personal experience since 2008. Anyway work the way you feel comfortable.
User Avatar
Member
61 posts
Joined: Feb. 2011
Offline
Ooh like that.
Yeah, that's a good point
I'll remember that. Thanks!
- A little nonsense now and then, is relished by the wisest men.
User Avatar
Member
11 posts
Joined: Sept. 2013
Offline
PradeepBarua
Can DOPs continue with a simulation from .bgeo files?

Absolutely not but I don't know what you exactly mean by that.

Hi Pradeep, resume simulation from bgeo sequence is possible, Bgeo sequence store the necessary fields to resume.
Senior Houdini Effects Technical Director @ Weta Digital
User Avatar
Member
258 posts
Joined: July 2006
Online
Ivan Pulido Suarez
PradeepBarua
Can DOPs continue with a simulation from .bgeo files?

Absolutely not but I don't know what you exactly mean by that.

Hi Pradeep, resume simulation from bgeo sequence is possible, Bgeo sequence store the necessary fields to resume.

I agree with Ivan
Head of FX @ MPC
CG Supervisor/ Sr. FX TD /
https://gumroad.com/timvfx [gumroad.com]
www.timucinozger.com
  • Quick Links