Hey guys, so at my work place we've been pushing more jobs utilise TOPS both locally and in our local farm (deadline) and so far it's been great however we noticed some problems when doing heavier jobs.

If we have pre-cached files done locally using and if we submit the entire top network to deadline for rendering, the current Houdini session will freeze and be rendered non-usable even tho we can still see the top network being cooked. Even after the deadline cook is done, the current Houdini session still freezes. We also tried setting the PDG Fetch(points to our rop geometry)which caches already existed explicitly to READ FILES ONLY b and submit to Deadline will still results in the current Houdini session to freeze.

Jobs are running alright without issue but would be great to solve the freezing Houdini session.

Thanks

Houdini ver: 18.0.416
Deadline Ver 10.1.9.2

Does the freezing issue happen with both the local scheduler and the deadline scheduler, or just when running the graph with deadline?

It may be easier if you file a bug report so that you can attach a .hip file and Houdini info.txt for us to look at.

tpetrick
Does the freezing issue happen with both the local scheduler and the deadline scheduler, or just when running the graph with deadline?

It may be easier if you file a bug report so that you can attach a .hip file and Houdini info.txt for us to look at.

I can file a bug report but not sure if I can attach the hip file due to NDA. Let me see what other ways I can do to make it easier for your side to debug. Cheers!

tpetrick
Does the freezing issue happen with both the local scheduler and the deadline scheduler, or just when running the graph with deadline?

It may be easier if you file a bug report so that you can attach a .hip file and Houdini info.txt for us to look at.

Hi there, so we submitted a support email to sesi few days ago and was told to try and roll our current Hou version to 18.0.658 and seemingly, the Houdini freezing issues doesn't seem to appear that frequently anymore.

However, I noticed that it will still occasionally freezes or crash our current Houdini session when we're trying to cook jobs locally or via deadline and it seems like the overhead from the amount of frames is causing the issue?

In the picture provided, we have 5 wedges and running each wedge to two separate infection solver and lastly feeding it to a particle sim. If there's no files in our job dir, the Houdini session doesn't freeze but if there are existing caches in our job dir, could it be the reading the files from our existing caches causes the freeze?

In the rop fetch help card:

Batching all frames into one job is typically used for simulations so that the simulation data can be kept in memory rather than be cached to disk. Smaller batch sizes, like a 4-frames-per job, can also help avoid overhead loading the .hip file when cooking a ROP that cooks quickly.

Since our wedges are simulations, batching all frames into one job is preferred but lets say the two infection solvers are done and no need for re-sim, just un-check batch all frames and set smaller batch sizes to 4-6 to reduce overhead from reading the caches?

Looking forward to hearing back from sesi <3

Cheers

Thanks to the guys at Sesi for helping out in solving the issue. Updated to the newest Houdini version fixed the freezing.