Howdy people,

We seem to have hit a huge problems with a bunch of our scenes at work. I really cant figure out how to solve this or what is causing it.

I have some large production hip files that normaly sit at 100mb. I have been working with these files for a good bit of time, and had no problem until a few days ago where my hip files where causing our renderfarm hbatch to choke. When saving, my .hip file bloats to over 600 mb, then resaving that file cause the hip file to bloat to 1 gb.

If i merge into a new scene i get a 1.6gb hip file. Nothing seems to work at all to save these scenes from becoming corrupted. When we compare the decent sized file to the news one there looks to be huge amounts of binary data flooding it.

Has anyone ever had problem saving hip files? It must be due to an external file as it effects all the old files also. The wierd thing is even if i delete all contents for the scene it and save it, it still sits at 600mb???????

I tried calling support but I think its canada day.

Thanks
Nick

If it is the problem I'm thinking it is I haven't encountered it for some time.
The problem may be an endless list of empty line entries that bloat the file.
The solution is to expand the hip file and go in and delete the empty lines.

Yeah we spotted some of these also. I dont know the source of the problem but i do know how it is spreading to other scenes. Seems like the encountered bad data/memory stays with houdini across new scenes/newly loaded scenes.

I.e If i load a “bad” scene into houdini, the corrupted data sticks around for the whole session and gets saved to what ever scene it encounters. Like a virus i suppose ;¬) then doubles on every save.

I got around it by restarting houdini and loading my good scene into a new the new session and its been saving fine since. I still have bad scenes that i cant fix tho, so will have to rebuild those from a good source.

Thanks
Nick

Hi,

Before I go on, are they lots of locked SOPs in the file? If you lock the SOPs, the data is saved with the hip file thus causing the bloat. Though it doesn't explain going from 600MB to 1.6GB. :shock:

BTW, what version are you using? Also, was the hip file created in a previous version of Houdini and moved to a recent one?

Now, onto a possible solution which I never really tried on Windows so I suggest testing this on a Linux machine if you have.

There is a command, hexpand, which will… erm… expand a hip file into a directory structure. You can use du to check where the bloat is. I remember it's a directory or file which is very big & often safe to delete.

After you delete the directory or file, you can use hcollapse to get your hip file back without the bloat.

Again, please make a backup of your files before testing.

Good luck and keep us posted.

Cheers!
steven

Yeah there is some locked nodes in the scene but even i remove them it doesnt make a diffrence.

hexpand sounds like an idea that could work. Ive just tried to get hexpand to work on a linux box but no luck. Not sure how im supposed todo this? Anyone got some pointers?
Any help would be great
Thanks
Nick

Its ok, someone pointed me to my error :¬D I was passing a full path to my hip file when using hexpand. Just needed a simple cd to the files directory so im guessing i was still in my houdini filestructure which makes sense .

doh!!!
cd $filefolder

hexpand $filename

This does the job as expected, nothing complicated there. If i want todo this on windows do I still need cygwin installed to run hexpand?

Anyway if this allows me to fix my file then I will follow this up.

Thanks for the help guys.
nick

Ok i found the source of the problem and the reason why this problem is shared across the houdini scenes.

The corrupted data is from the userDisplayOptions.h file in the hip file. Its full of “junk” data. I can delete this and hcollapse the scenes hopefully this will work.

Does anyone know how to fix this from within houdini? And why this could happen?

Thanks
Nick

If you still have a copy of the bloated hip file, I would suggest sending it to SESI support so they can hopefully identify the problem.

Good to hear the hexpand & hcollapse went okay.

Good luck!

Cheers!
steven

Already have :¬)

Strangely one of our hip files that was bloated to 530mb compressed down to 600KB Sounds like a alot of empty lines that probbins mentioned.

Lets hope theres a nice solution to this.
Cheers
Nick

The problem has been identified and fixed in both 9.5 and 9.1.

Ondrej
The problem has been identified and fixed in both 9.5 and 9.1.

Didn't Houdini have a similar bug somewhere around version 6 or 7? I recall having to hexpand the files, nuke the offending, bloated file, and recollapsing them.

– Antoine

yeah…it's a classic
from 5.x IIRC

Antoine Durr
Didn't Houdini have a similar bug somewhere around version 6 or 7? I recall having to hexpand the files, nuke the offending, bloated file, and recollapsing them.

Similar symptom, totally different bug.

Ondrej

Antoine Durr
Didn't Houdini have a similar bug somewhere around version 6 or 7? I recall having to hexpand the files, nuke the offending, bloated file, and recollapsing them.

Similar symptom, totally different bug.

In other words, the virus has mutated. ;-)

– Antoine

arctor
yeah…it's a classic
from 5.x IIRC

FWIW, this is the first time in long time that I've heard of expanding hip file sizes.

Glad this has been sorted now, have to say the new build saved our butts :¬) meant redoing a little work rather then fresh builds of scenes.
I hope to never see this again.
:wink:
Cheers
Nick