What are the best ways to diagnose Houdini hip file load times? Or does anybody have any tricks they've learned for speeding Houdini up?

I'm especially interested in actual hip file load times. We know the times it takes for our pythonrc.py and 123/456 startup scripts to run. But what about loading an actual .hip file (the time after the pythonrc.py and pre the 123.py)? How can we best determine what's making that go slow?

We're seeing hip files take 5~ minutes to load that are a mere 19 MB that when playing around in the network editor don't appear to be all that heavy.

are you talking pure file load times or time for file to cook after load?

I'm talking pure load times. Time to cook is obviously dependent on what the user is doing.

Where and/or how do you measure this?

The old stop-watch on my phone. Start as soon as the splash screen pops-up, and stop as soon as the GUI is visible.

so you are not just loading a scene but also starting a new instance of houdini.

If you start houdini first and then onlöy load the scene does it still load this slow? Maybe turn of auto update for this test.

Can you post a slow loading scene example?

Hey Olaf,

Unforunately, no, I cannot post an example hip file. That's why I was asking for general tips and tricks in debugging slow launch times.

Launching a new houdini session and then opening a “heavy” hip file (with Manual cook mode enabled) is still quite slow. :-|

Hey Brian,
i looked at my files and dont see any crazy anomalies. Like a 19mb file loading this slow.
Maybe you can create a dummy file that does in general do what the slow file is doing.
So you dont need to share confidential stuff.
What is heavy about it? Loads of nodes?

greetings

Olaf

You have to split application load time and cooking time for you hip file.

Application load time is should be same/very similar to the case when you are opening houdini without any scene loaded.
You can test that by running: That way there will be no cooking (manual update mode) after file is loaded.

To determine cooking time there is Performance Monitor that will tell you how much time was spent on each node.
Run the Performance Monitor sampling then turn on scene update (right bottom corner, change update mode to Auto or refresh) and analyze results from Performance monitor to see where it spend most of the time while cooking your scene.

Hey Brian! Here are a few ideas to try:

- Load with a desktop that doesn't have a Scene Viewer (but keep a Geometry Spreadsheet so everything still cooks)
- Double-check that any disk-loading HDA's aren't doing something strange, that may have been necessary a long time ago (or made certain assumptions which are slow to check now)
- Try bypassing File SOPs, to see if certain network drives are making things load slower; if you know certain groups of File SOPs are all mounted in the same network drives, try disabling/enabling in groups
- Other idea for checking network drives, try saving the hip file + all dependencies to local workstation's /tmp dir, and opening it from there (may not be possible if loading lots of crazy stuff)
- See if there are nodes copying cache-cruft somehow by saving nodes to disk via opscript, then loading them back into a fresh Houdini session
- Try loading versions from the backup folder, see if any load faster and find out what's different if you do

Also, I know hip file load times are the question, but in Studios/environments which use a wrapper I try these:

- Try launching vanilla houdini without the wrapper, but still from the network-installed location (i.e. cd /location/on/network/hfs16.5.476;source houdini_setup;cd -; houdinifx -foreground)
- Check on network affects, by installing to the local machine's temp directory, and launching it vanilla style like above (i.e. cd /tmp/houdinitest/hfs16.5.476;source houdini_setup;cd /tmp; houdinifx -foreground)

Either of those are nice to narrow down crashes or other bugs that might be environment-related.

Hope that helps!

So this is an old thread but seems relevant to an issue I am having. We are finding Houdini takes significantly longer to start up when it has packages (qLib, SideFXLabs, etc) located centrally on the server rather than locally on a User's machine.

With the local setup (Houdini and external OTL libraries locally installed), it takes about 40s to start.

With a network setup (Houdini local and external OTL libraries in a network location), it takes up to 5mins to start.

Could network lag be responsible for all of this difference or could something else be going on?

I have also noticed a big slowdown similar to above post, after moving my otls and packages to the server. In the morning i just launch a few sessions so i don't have to wait for them when i need them.

Launching scenes is fine. Just launching the session to an empty scene is slow.

H18.5.633

if you happen to have Arnold installed, it can **massively** slow down your startup depending on your startup directory. the autodesk licensing stuff will attempt to load every dll in the current working directory as it starts. quality stuff as always from autodesk.

i modified our launcher system to set $CWD to a blank directory and startup times were more or less normal (for loading a bunch of otls off a network share - that still hurts).

cheers,
chrisg

Thanks Drbob,

I have the "Built-in" arnold installed with maya on the same machine, but it isnt installed for Houdini, we're mostly Redshift here. I'll try uninstalling that anywhoo.

I'll also try setting $CWD as you did, having trouble finding info on the variable though.

Picking this thread back up.

I have our studio's python scripts and otls.
I have qLib and SideFX labs
The above files are on server in a centralized location and they are called using separeted packages json.

Loading times ( empty scene ) are:

Vanilla Houdini ( local )
12s

Hsite + our python_pipe
19s

Hsite + our python_pipe + our OTLS package
27s

Hsite + our python_pipe + our OTLS package
+qLib package
50s

Hsite + our python_pipe + our OTLS package
+SideFXLabs
50s

Hsite + our python_pipe + our OTLS package
+SideFXLabs +qLib
300s (5min!)


Using Houdini 20.0.547, Windows machines.


In my experience 50 seconds to load a session with all the packages is pretty normal ( this was the case for our H19.5 build ).
In H20 seems like when loading both SideFXLabs AND qLib, launching time explodes.
1. Anybody body else sharing this issue?

I am also getting this warning.
I am wondering if it signals that something is not properly installed?


2. Is there a way to print out each step houdini is executing during lauch?

Hi,

You can use the -profile-startup option when you launch Houdini:
https://www.sidefx.com/docs/houdini/ref/commandline.html [www.sidefx.com]

wlvl_gv
Vanilla Houdini ( local )
12s

Interesting. On a fresh OS boot(Win 10), using launcher for 19.5 vanilla - It takes my system 35 secs.

It doesn't matter to me and is not an issue but since I've started using Houdini since around H14 the start up times have slowly creeped longer and longer.

Can't remember how fast H14 was but if I remember correctly, it was 'snappy' - maybe 3 seconds, 5 at most.

Make sure that you don't have any expanded OTLs (ie. that all the hda/otl files are actually files, not directories), those are killer.

wlvl_gv
2. Is there a way to print out each step houdini is executing during lauch?

When loading your plugins from a server, you can use Wireshark to monitor the file requests sent by Houdini.

When loading plugins on the same machine, you can use ProcessMonitor (I know this one for Windows at least).

I had (still having ^^) a similar issue with launch time for Houdini in general from a NAS/server containing plugins, and got help from IT/SysAdmins to analyze data provided with these tools, and noticed that Houdini requests many files (like about 80k file requests) that do not exist.

Thanks a lot for all the inputs!
The -profile-startup option helps a lot to see where is taking longer to load. thanks @animatrix_

@edward
thanks!

expanded OTLs

were the culprit!
I downloaded SideFX Labs from GitHub and that has all expanded folder OTLs.

Installing SideFX Labs using the Launcher and then coping that folder to the centralized location fixed the issue, since the otls are inside one single .hda packed file.

Launch time for:
Hsite + our python_pipe + our OTLS package + SideFXLabs20 + qLib
is 34sec now.

Using Houdini 20.0.590, Windows machines.