I'm having the same problem with Houdini 19.5.303 Karma XPU not using the GPU's on my Debian 11 system.
Have been installing the (never current) Debian nVidia drivers (usually from backports) for years, and running Houdini/RedShift (since RS version 2.6.30) without any problems.
Because I'd like to make Karma XPU my main renderer, I took the plunge (June 23rd) and followed the instructions to install the 515.48.07 nvidia driver from here...
https://www.linuxcapable.com/install-510-nvidia-drivers-on-debian-11-bullseye [
www.linuxcapable.com]
It was successful, and everything has continued to run fine since.
In tests with Houdini 19.0.???, Karma XPU was running both my cards (RTX 2070 Super & GTX 1080) at around 100%.
In tests with Houdini 19.5.303, Karma XPU doesn't appear to use the cards at all. And, I'm also getting the "Unable to create optix context for device..." errors.
(Not sure why optix wasn't included with that 515.48.07 install.)
However, when I render with Redshift 3.0 or 3.5 versions, and Houdini 19.0 or 19.5 versions, the cards are both running at 100%.
Yes, I get "The denoiser will be disabled" error from Redshift, but it still renders with the GPUs near 100%.
Anyway, main point is, Redshift is running fine on my Debian 11 system with Houdini 19.5. Karma XPU is not.
P.S. Monitored GPU usage with "nvidia-smi dmon", "psensor", & "NVIDIA Settings". All were in agreement.