cache loading very heavy

   12371   20   1
User Avatar
Member
102 posts
Joined: March 2013
Offline
hello everyone, I just have a problem need help.
loading sequence cache(*.bgeo) is very heavy. for example, I have a grid has about 1 million primitives:

play it without cache I can run about 2 fps.
play with cache it slow down to 0.25 fps…

it's very strange. in my memory bgeo is speedy.maybe there is something wrong in my setting?
hope we have a solution, thank you!

Attachments:
cacheProblem.hip (66.9 KB)

User Avatar
Member
102 posts
Joined: March 2013
Offline
still don't know the reason. but I just found a solution.
use add sop to delete mesh and only cache pc. then transfer all attribute back to the static mesh. this time it's very fast
User Avatar
Member
258 posts
Joined: July 2006
Offline
you probaly had fields you did not need, maybe some volume attributes etc, and resulting bgeo files were probably big.
Head of CG @ MPC
CG Supervisor/ Sr. FX TD /
https://gumroad.com/timvfx [gumroad.com]
www.timucinozger.com
User Avatar
Member
1391 posts
Joined: Dec. 2010
Offline
Yes , delete unused attributes make a little faster reading time for cache files.
However that's obvious reading data from RAM is more faster than HDD !

I usually using SSD drive for reading and writing large cache files .

Also reading and writing data from EX4 partitions (on Linux) is faster than NTFS (on Win) , Of course in my experience !
https://www.youtube.com/c/sadjadrabiee [www.youtube.com]
Rabiee.Sadjad@Gmail.Com
User Avatar
Member
258 posts
Joined: July 2006
Offline
Joker386
Yes , delete unused attributes make a little faster reading time for cache files.
However that's obvious reading data from RAM is more faster than HDD !

I usually using SSD drive for reading and writing large cache files .

Also reading and writing data from EX4 partitions (on Linux) is faster than NTFS (on Win) , Of course in my experience !

Lol I can buy a HP workstation with the cost of massive SDDs I would need inorder to be able to work.
Head of CG @ MPC
CG Supervisor/ Sr. FX TD /
https://gumroad.com/timvfx [gumroad.com]
www.timucinozger.com
User Avatar
Member
1391 posts
Joined: Dec. 2010
Offline
tricecold
Joker386
Yes , delete unused attributes make a little faster reading time for cache files.
However that's obvious reading data from RAM is more faster than HDD !

I usually using SSD drive for reading and writing large cache files .

Also reading and writing data from EX4 partitions (on Linux) is faster than NTFS (on Win) , Of course in my experience !

Lol I can buy a HP workstation with the cost of massive SDDs I would need inorder to be able to work.

:? :? :? :!:
https://www.youtube.com/c/sadjadrabiee [www.youtube.com]
Rabiee.Sadjad@Gmail.Com
User Avatar
Member
4189 posts
Joined: June 2012
Offline
That brings up the question, what is the current price of SSDs?

How much disk cache do you need Tricecold? What do you use Joker386?
User Avatar
Member
1391 posts
Joined: Dec. 2010
Offline
I just use 2 SSD drive (2 * 250GB) , So I have to move my cache files to another partition after finishing it's project !!!
But SSD in EX4 is really cool and fast for reading and writing huge cache files 8)
https://www.youtube.com/c/sadjadrabiee [www.youtube.com]
Rabiee.Sadjad@Gmail.Com
User Avatar
Member
48 posts
Joined: June 2011
Offline
Loading it is one thing, but displaying it is also a concern. If you're loading fresh geometry into the viewport every frame, and you've got a million+ polygons per frame, it can depend a on your GPU just how quickly it can be loaded onto the card.

Still, with a reasonable card, it's rather more likely that disk speed will be the bottleneck.
User Avatar
Member
1391 posts
Joined: Dec. 2010
Offline
Yes , VGA is very important too.
Of course sometimes we don't need to have very perfect VGA , like working with Pyro Simulations :wink:

About Flip Simulations , we can use only particles in the viewport , Not Fluid Mesh.

Or maybe we can display some particles in the viewport ,(not all of them ) to achieve proxy , then render full particles in the rendering , something like Fur guides .

But about reading and writing huge cache files , Hmmmm , That's not too many options
:?
https://www.youtube.com/c/sadjadrabiee [www.youtube.com]
Rabiee.Sadjad@Gmail.Com
User Avatar
Member
4189 posts
Joined: June 2012
Offline
For smoke sims you can always use Volume Compress SOP to drop the bit rate 16 bit - half the size, and you can compress the bgeo with .gz
User Avatar
Member
1391 posts
Joined: Dec. 2010
Offline
MartybNz
For smoke sims you can always use Volume Compress SOP to drop the bit rate 16 bit - half the size, and you can compress the bgeo with .gz

“Volume Compress SOP” is great tip , really thanks MartybNz. :shock:
But I usually didn't use bgeo.gz for huge simulations files, sometimes I need to write and read large cache files very fast , But .gz files are so slower than uncompressed files , I just use them for archiving my cache files for future :wink:
https://www.youtube.com/c/sadjadrabiee [www.youtube.com]
Rabiee.Sadjad@Gmail.Com
User Avatar
Member
29 posts
Joined: May 2013
Offline
HI iGou,

Did you use alembic? You may get improve performance by use alembic.

Here is my playback result:

1. Mountain SOP 1.5 fps
2. File SOP(w bgeo) 44 fpm
3. Alembic SOP( Load Houdini Geometry) 1.9 fps
4. Alembic SOP(Alembic Delayed Load Primitives) 6 fps
User Avatar
Member
1391 posts
Joined: Dec. 2010
Offline
Hideki Suzuki
HI iGou,

Did you use alembic? You may get improve performance by use alembic.

Here is my playback result:

1. Mountain SOP 1.5 fps
2. File SOP(w bgeo) 44 fpm
3. Alembic SOP( Load Houdini Geometry) 1.9 fps
4. Alembic SOP(Alembic Delayed Load Primitives) 6 fps


Yes , I used alembic before too , That's so fast , even faster than bgeo files !!!
I think the only thing is ,It can't save Volumetric objects like PyroFX
https://www.youtube.com/c/sadjadrabiee [www.youtube.com]
Rabiee.Sadjad@Gmail.Com
User Avatar
Member
102 posts
Joined: March 2013
Offline
danwood82
Loading it is one thing, but displaying it is also a concern. If you're loading fresh geometry into the viewport every frame, and you've got a million+ polygons per frame, it can depend a on your GPU just how quickly it can be loaded onto the card.

Still, with a reasonable card, it's rather more likely that disk speed will be the bottleneck.
thank you my friend, I guess u are right. I loaded from partition0(SSD*2) but my card is a poor game card.
Hideki Suzuki
Did you use alembic? You may get improve performance by use alembic.
No I didnt try alembic before because I always think bgeo is the best :wink:
I should try later, thanks Suzuki-san.
User Avatar
Member
48 posts
Joined: June 2011
Offline
igou
thank you my friend, I guess u are right. I loaded from partition0(SSD*2) but my card is a poor game card.

There isn't necessarily anything wrong with a “game card”… you'll get a lot more bang-for-buck getting a GeForce than a Quadro of the same price. Quadros can cost anything up to 6x more for essentially the same piece of hardware, but with “professional” driver support for stability. A $300 gaming card would probably work fantastically… a $300 workstation card would be about the slowest you could get.

What card do you have?
User Avatar
Member
102 posts
Joined: March 2013
Offline
gtx760 4GB. I just want more memory for pyro so choose the cheaper one(have 4GB).
btw, could you give me some points that I can make sense where is the different Geforce from Quadro? the only I know is for maya quadro will faster in viewport camera orbit. and my friends say the different only appear in CAT software.
User Avatar
Member
258 posts
Joined: July 2006
Offline
MartybNz
That brings up the question, what is the current price of SSDs?

How much disk cache do you need Tricecold? What do you use Joker386?

How much? As much as possible. Well think about it. In a studio environment for a shot a few buildings are burning, camera flying by, would easily take more than several terabytes of data. So I beleive SSD are a bad investment for Houdini TDs. Of course you can have it as your main partition or storing softwares and etc but not for Storage.

When I say investing i am not talking for few hundred GBs of course
Head of CG @ MPC
CG Supervisor/ Sr. FX TD /
https://gumroad.com/timvfx [gumroad.com]
www.timucinozger.com
User Avatar
Member
48 posts
Joined: June 2011
Offline
igou
gtx760 4GB. I just want more memory for pyro so choose the cheaper one(have 4GB).
btw, could you give me some points that I can make sense where is the different Geforce from Quadro? the only I know is for maya quadro will faster in viewport camera orbit. and my friends say the different only appear in CAT software.

I really don't know the deep-down technical difference - I think there are sometimes cases of GeForce cards having their compute capabilities intentionally limited compared to their Quadro counterparts, but that's only potentially relevant if you're using OpenCL/CUDA capabilities, and even then, I get the feeling it only tends to apply to double-precision calculations, which I don't think Houdini or any other VFX software uses. I could be wrong, but I think double precision tends to be geared more towards scientific computing uses.

In my experience, a GTX 760 is going to fly, and I think any perceived difference in viewport performance is dubious. I've used quite a few of both, and the only times I've ever had slowdown or memory issues has been because of workstations that get packaged with a Quadro 600, which basically runs out the RAM and crashes the moment you attempt to playblast a FLIP sim :-P


The real “professional” benefits of Quadro cards come from managing a large studio, where you want to be able to roll out a driver update to 500 machines, and be reasonably confident that it won't break anything in your pipeline. It's not so much about any actual performance advantage. If you've only got one or two machines, you're not really going to save yourself much money in administration costs.
User Avatar
Member
4189 posts
Joined: June 2012
Offline
Lots of ‘consumer’ cards supported in the next Houdini:

http://www.sidefx.com/index.php?option=com_content&task=view&id=2680&Itemid=390 [sidefx.com]
  • Quick Links