What are the best ways to diagnose Houdini hip file load times? Or does anybody have any tricks they've learned for speeding Houdini up?
I'm especially interested in actual hip file load times. We know the times it takes for our pythonrc.py and 123/456 startup scripts to run. But what about loading an actual .hip file (the time after the pythonrc.py and pre the 123.py)? How can we best determine what's making that go slow?
We're seeing hip files take 5~ minutes to load that are a mere 19 MB that when playing around in the network editor don't appear to be all that heavy.
Faster launch times?
10349 21 5-
- Brian Walters
- Member
- 155 posts
- Joined: Sept. 2015
- Offline
-
- Olaf Finkbeiner
- Member
- 323 posts
- Joined: Jan. 2015
- Offline
-
- Brian Walters
- Member
- 155 posts
- Joined: Sept. 2015
- Offline
-
- Olaf Finkbeiner
- Member
- 323 posts
- Joined: Jan. 2015
- Offline
-
- Brian Walters
- Member
- 155 posts
- Joined: Sept. 2015
- Offline
-
- Olaf Finkbeiner
- Member
- 323 posts
- Joined: Jan. 2015
- Offline
-
- Brian Walters
- Member
- 155 posts
- Joined: Sept. 2015
- Offline
-
- Olaf Finkbeiner
- Member
- 323 posts
- Joined: Jan. 2015
- Offline
-
- pezetko
- Member
- 392 posts
- Joined: Nov. 2008
- Offline
You have to split application load time and cooking time for you hip file.
Application load time is should be same/very similar to the case when you are opening houdini without any scene loaded.
You can test that by running:That way there will be no cooking (manual update mode) after file is loaded.
To determine cooking time there is Performance Monitor that will tell you how much time was spent on each node.
Run the Performance Monitor sampling then turn on scene update (right bottom corner, change update mode to Auto or refresh) and analyze results from Performance monitor to see where it spend most of the time while cooking your scene.
Application load time is should be same/very similar to the case when you are opening houdini without any scene loaded.
You can test that by running:
houdini -n path/to/your/file.hip
To determine cooking time there is Performance Monitor that will tell you how much time was spent on each node.
Run the Performance Monitor sampling then turn on scene update (right bottom corner, change update mode to Auto or refresh) and analyze results from Performance monitor to see where it spend most of the time while cooking your scene.
-
- goldleaf
- Staff
- 4208 posts
- Joined: Sept. 2007
- Online
Hey Brian! Here are a few ideas to try:
- Load with a desktop that doesn't have a Scene Viewer (but keep a Geometry Spreadsheet so everything still cooks)
- Double-check that any disk-loading HDA's aren't doing something strange, that may have been necessary a long time ago (or made certain assumptions which are slow to check now)
- Try bypassing File SOPs, to see if certain network drives are making things load slower; if you know certain groups of File SOPs are all mounted in the same network drives, try disabling/enabling in groups
- Other idea for checking network drives, try saving the hip file + all dependencies to local workstation's /tmp dir, and opening it from there (may not be possible if loading lots of crazy stuff)
- See if there are nodes copying cache-cruft somehow by saving nodes to disk via opscript, then loading them back into a fresh Houdini session
- Try loading versions from the backup folder, see if any load faster and find out what's different if you do
Also, I know hip file load times are the question, but in Studios/environments which use a wrapper I try these:
- Try launching vanilla houdini without the wrapper, but still from the network-installed location (i.e.
- Check on network affects, by installing to the local machine's temp directory, and launching it vanilla style like above (i.e.
Either of those are nice to narrow down crashes or other bugs that might be environment-related.
Hope that helps!
- Load with a desktop that doesn't have a Scene Viewer (but keep a Geometry Spreadsheet so everything still cooks)
- Double-check that any disk-loading HDA's aren't doing something strange, that may have been necessary a long time ago (or made certain assumptions which are slow to check now)
- Try bypassing File SOPs, to see if certain network drives are making things load slower; if you know certain groups of File SOPs are all mounted in the same network drives, try disabling/enabling in groups
- Other idea for checking network drives, try saving the hip file + all dependencies to local workstation's /tmp dir, and opening it from there (may not be possible if loading lots of crazy stuff)
- See if there are nodes copying cache-cruft somehow by saving nodes to disk via opscript, then loading them back into a fresh Houdini session
- Try loading versions from the backup folder, see if any load faster and find out what's different if you do
Also, I know hip file load times are the question, but in Studios/environments which use a wrapper I try these:
- Try launching vanilla houdini without the wrapper, but still from the network-installed location (i.e.
cd /location/on/network/hfs16.5.476;source houdini_setup;cd -; houdinifx -foreground
)- Check on network affects, by installing to the local machine's temp directory, and launching it vanilla style like above (i.e.
cd /tmp/houdinitest/hfs16.5.476;source houdini_setup;cd /tmp; houdinifx -foreground
)Either of those are nice to narrow down crashes or other bugs that might be environment-related.
Hope that helps!
I'm o.d.d.
-
- lloydwood
- Member
- 107 posts
- Joined: Nov. 2007
- Offline
So this is an old thread but seems relevant to an issue I am having. We are finding Houdini takes significantly longer to start up when it has packages (qLib, SideFXLabs, etc) located centrally on the server rather than locally on a User's machine.
With the local setup (Houdini and external OTL libraries locally installed), it takes about 40s to start.
With a network setup (Houdini local and external OTL libraries in a network location), it takes up to 5mins to start.
Could network lag be responsible for all of this difference or could something else be going on?
With the local setup (Houdini and external OTL libraries locally installed), it takes about 40s to start.
With a network setup (Houdini local and external OTL libraries in a network location), it takes up to 5mins to start.
Could network lag be responsible for all of this difference or could something else be going on?
-
- matt_barker
- Member
- 14 posts
- Joined: April 2019
- Offline
I have also noticed a big slowdown similar to above post, after moving my otls and packages to the server. In the morning i just launch a few sessions so i don't have to wait for them when i need them.
Launching scenes is fine. Just launching the session to an empty scene is slow.
H18.5.633
Launching scenes is fine. Just launching the session to an empty scene is slow.
H18.5.633
Edited by matt_barker - Oct. 27, 2021 19:37:41
-
- doctorbob
- Member
- 30 posts
- Joined:
- Offline
if you happen to have Arnold installed, it can **massively** slow down your startup depending on your startup directory. the autodesk licensing stuff will attempt to load every dll in the current working directory as it starts. quality stuff as always from autodesk.
i modified our launcher system to set $CWD to a blank directory and startup times were more or less normal (for loading a bunch of otls off a network share - that still hurts).
cheers,
chrisg
i modified our launcher system to set $CWD to a blank directory and startup times were more or less normal (for loading a bunch of otls off a network share - that still hurts).
cheers,
chrisg
-
- matt_barker
- Member
- 14 posts
- Joined: April 2019
- Offline
-
- wlvl_gv
- Member
- 11 posts
- Joined: April 2022
- Offline
Picking this thread back up.
I have our studio's python scripts and otls.
I have qLib and SideFX labs
The above files are on server in a centralized location and they are called using separeted packages json.
Loading times ( empty scene ) are:
Vanilla Houdini ( local )
12s
Hsite + our python_pipe
19s
Hsite + our python_pipe + our OTLS package
27s
Hsite + our python_pipe + our OTLS package
+qLib package
50s
Hsite + our python_pipe + our OTLS package
+SideFXLabs
50s
Hsite + our python_pipe + our OTLS package
+SideFXLabs +qLib
300s (5min!)
Using Houdini 20.0.547, Windows machines.
In my experience 50 seconds to load a session with all the packages is pretty normal ( this was the case for our H19.5 build ).
In H20 seems like when loading both SideFXLabs AND qLib, launching time explodes.
1. Anybody body else sharing this issue?
I am also getting this warning.
I am wondering if it signals that something is not properly installed?
2. Is there a way to print out each step houdini is executing during lauch?
I have our studio's python scripts and otls.
I have qLib and SideFX labs
The above files are on server in a centralized location and they are called using separeted packages json.
Loading times ( empty scene ) are:
Vanilla Houdini ( local )
12s
Hsite + our python_pipe
19s
Hsite + our python_pipe + our OTLS package
27s
Hsite + our python_pipe + our OTLS package
+qLib package
50s
Hsite + our python_pipe + our OTLS package
+SideFXLabs
50s
Hsite + our python_pipe + our OTLS package
+SideFXLabs +qLib
300s (5min!)
Using Houdini 20.0.547, Windows machines.
In my experience 50 seconds to load a session with all the packages is pretty normal ( this was the case for our H19.5 build ).
In H20 seems like when loading both SideFXLabs AND qLib, launching time explodes.
1. Anybody body else sharing this issue?
I am also getting this warning.
opalias: The alias 'qLib::camera_frustrum_ql::3' has the same name as an existing operator.
2. Is there a way to print out each step houdini is executing during lauch?
-
- animatrix_
- Member
- 4812 posts
- Joined: Feb. 2012
- Offline
Hi,
You can use the -profile-startup option when you launch Houdini:
https://www.sidefx.com/docs/houdini/ref/commandline.html [www.sidefx.com]
You can use the -profile-startup option when you launch Houdini:
https://www.sidefx.com/docs/houdini/ref/commandline.html [www.sidefx.com]
Senior FX TD @ Industrial Light & Magic
Get to the NEXT level in Houdini & VEX with Pragmatic VEX! [www.pragmatic-vfx.com] https://lnk.bio/animatrix [lnk.bio]
Get to the NEXT level in Houdini & VEX with Pragmatic VEX! [www.pragmatic-vfx.com] https://lnk.bio/animatrix [lnk.bio]

-
- BabaJ
- Member
- 2137 posts
- Joined: Sept. 2015
- Offline
wlvl_gv
Vanilla Houdini ( local )
12s
Interesting. On a fresh OS boot(Win 10), using launcher for 19.5 vanilla - It takes my system 35 secs.
It doesn't matter to me and is not an issue but since I've started using Houdini since around H14 the start up times have slowly creeped longer and longer.
Can't remember how fast H14 was but if I remember correctly, it was 'snappy' - maybe 3 seconds, 5 at most.
Edited by BabaJ - Dec. 20, 2023 14:46:38
-
- edward
- Member
- 7934 posts
- Joined: July 2005
- Offline
-
- ObeidaZakzak
- Member
- 123 posts
- Joined: Dec. 2019
- Offline
wlvl_gv
2. Is there a way to print out each step houdini is executing during lauch?
When loading your plugins from a server, you can use Wireshark to monitor the file requests sent by Houdini.
When loading plugins on the same machine, you can use ProcessMonitor (I know this one for Windows at least).
I had (still having ^^) a similar issue with launch time for Houdini in general from a NAS/server containing plugins, and got help from IT/SysAdmins to analyze data provided with these tools, and noticed that Houdini requests many files (like about 80k file requests) that do not exist.
Houdini Pipeline Supervisor @ TAT Studio
-
- wlvl_gv
- Member
- 11 posts
- Joined: April 2022
- Offline
Thanks a lot for all the inputs!
The -profile-startup option helps a lot to see where is taking longer to load. thanks @animatrix_
@edward
thanks!
I downloaded SideFX Labs from GitHub and that has all expanded folder OTLs.
Installing SideFX Labs using the Launcher and then coping that folder to the centralized location fixed the issue, since the otls are inside one single .hda packed file.
Launch time for:
Hsite + our python_pipe + our OTLS package + SideFXLabs20 + qLib
is 34sec now.
Using Houdini 20.0.590, Windows machines.
The -profile-startup option helps a lot to see where is taking longer to load. thanks @animatrix_
@edward
thanks!
expanded OTLswere the culprit!
I downloaded SideFX Labs from GitHub and that has all expanded folder OTLs.
Installing SideFX Labs using the Launcher and then coping that folder to the centralized location fixed the issue, since the otls are inside one single .hda packed file.
Launch time for:
Hsite + our python_pipe + our OTLS package + SideFXLabs20 + qLib
is 34sec now.
Using Houdini 20.0.590, Windows machines.
Edited by wlvl_gv - Feb. 8, 2024 14:18:02
-
- Quick Links