Following on from a now-closed H22 beta post regarding XPU shader pre-compile issues:

It has been pointed out that the H22 XPU shader pre-compile feature is running extra slowly than H21. 22.0.398 contains a fix that would have affected multi-GPU users. For anyone affected by the H22 issue with multi-gpu, it would be good to get feedback on 22.0.398 to verify its now running much faster.

But in general (even in H21) its been noted that the shader-precompile feature can run extra slowly on some machines, exhibiting single-thread-like behavior. I'm curious what types of machines this happens on. Its definitly multi-threaded in code, and runs fast on our Linux machines.

Is it possible for anyone to give specific feedback on this?
eg
- is it slow only on Intel? (in which case it might be an e-cores issue. does disabling the e-cores in the bias make it multi-threaded again?) or is it slow on AMD too?
- is it slow only on Windows? Or does Linux also experience this issue?

thanks lots

I‘ve put this in an RFE (because of the locked forum), but I‘ll post this here again:

I am on an amd 64core threadripper, two identical gpus, win11. When starting the caching, multiple (i guess 64) karma_cc.exe start up plus the houdini.exe also has cpu load (total cpu load is near 90%). After about a minute, most of the karma.exe are gone with only 1-3 remaining (cpu load is down to single digits). Not sure if the multithreading should be achieved through having multiple karma.exe running or if the processes running in houdini.exe should be more multithreaded.

This makes me wonder if this also affects the compilation time while in interactive mode (for stuff that has not been compiled yet).

ronald_a
When starting the caching, multiple (i guess 64) karma_cc.exe start up plus the houdini.exe

This is strange, it should only be starting 1 karma.exe (and does so on my windows machine)
Do you have some KARMA_XPU_... environment variables set by chance?

brians

ronald_a
When starting the caching, multiple (i guess 64) karma_cc.exe start up plus the houdini.exe

This is strange, it should only be starting 1 karma.exe (and does so on my windows machine)
Do you have some KARMA_XPU_... environment variables set by chance?

I do have KARMA_XPU variables set - most notebly KARMA_XPU_THREAD_COMPILE = 1. I disabled all the KARMA_XPU variables for now and now the karma_cc.exes are gone. only one karma.exe remains which runs at a steady 2-4% cpu load.

brians
Is it possible for anyone to give specific feedback on this?
eg
- is it slow only on Intel? (in which case it might be an e-cores issue. does disabling the e-cores in the bias make it multi-threaded again?) or is it slow on AMD too?
- is it slow only on Windows? Or does Linux also experience this issue?

I ran precompile on Windows 11 with the following configuration: Intel Core Ultra 7 265k (8P 12E), Nvidia RTX 5080 with 610.88 driver using Houdini 22.0.401
Total time taken was 4 hours 20 minutes 9 seconds. Average core usage was 20%. Usage was almost entirely P-cores and occasionally spiked to all P cores, but was only using 2~4 most of the time. Compiling completed successfully without error.

I don't think the threading is only the issue, it wouldn't be 100x faster if it were using all threads.

I deleted any existing cache for H22 and did a fresh run. It took about 7.5 hours (with cpu usage around 2-3%)! Interestingly enough, before doing that, I continued on an existing cache (which was just 7 or eight steps in). After an initial very long step 9, cpu usage increased to about 11% with a remaining time of about 2 hours about 40 steps in. I have seen this behavior before where the cpu usage is somewhat unpredictable. It‘s really weird.

ronald_a
I do have KARMA_XPU variables set - most notebly KARMA_XPU_THREAD_COMPILE = 1. I disabled all the KARMA_XPU variables for now and now the karma_cc.exes are gone. only one karma.exe remains which runs at a steady 2-4% cpu load.

cool
at least the karma_cc.exes are gone

jsmack
I ran precompile on Windows 11 with the following configuration: Intel Core Ultra 7 265k (8P 12E), Nvidia RTX 5080 with 610.88 driver using Houdini 22.0.401
Total time taken was 4 hours 20 minutes 9 seconds. Average core usage was 20%. Usage was almost entirely P-cores and occasionally spiked to all P cores, but was only using 2~4 most of the time.

I'm starting to suspect some kind of file-locking issue within the nvidia cache.

ronald_a
I deleted any existing cache for H22 and did a fresh run. It took about 7.5 hours (with cpu usage around 2-3%)!

Good to know

ronald_a
I have seen this behavior before where the cpu usage is somewhat unpredictable. It‘s really weird.

Yes it is weird :/
Hopefully I'll be getting an e/p-cores machine soon so I can debug more closely. Will report back findings here.

thanks all