I installed the correct versions of the Cuda toolkit and the CUDNN libraries and set the path environment in windows to the correct folders, but I can't get it to work.
I also tried some different folder structures and copies the CUDNN libraries to different locations, but no luck.

Does anyone has some idea how to correctly install the libraries?

As far as I know, it's impossible. I worked with the developer and they could not help get it to work for me. Maybe it works on Linux?

Thanks for the quick answer. That is too bad, hopefully they will fix it soon in an update.
Using things like anydepth2 and SAM2 is really handy, but right now on cpu it is really slow.

@jsmack - I just succeeded!

After installing Cuda 12.2 and CUDNN 8.9.2 (version for 12.2) the trick was to track down zlib.dll (who knew??)

After a wild goose chase I found it already on my system in "Program Files\NVIDIA Corporation\Nsight Systems 2023.2.3\host-windows-x64"

It needs to be copied to the bin folder of your CUDNN installation, and renamed zlibwapi.dll

ONNX is working super fast with my 4090 now

I've been trying to get SAM/SAM2 running in Houdini using the ONNX nodes with no success. After much fiddling around with the encoder parameters Houdini stopped complaining and started crashing with segmentation faults instead. Could either of you provide some guidance (or better yet, a HIP)? I'd really appreciate it!

mrboni
After installing Cuda 12.2 and CUDNN 8.9.2 (version for 12.2) the trick was to track down zlib.dll (who knew??)

Has this been fixed or obviated in 20.5.613 or 21.0.440? CUDA/CUDNN has much newer versions as well so im curious about which to install.

I've got this working. I'm on W11 Pro, Houdini 21.0.440, with a 3090 (0) and 4090 (1).
Install the Nvidia CUDA Toolkit 12.8 per the Houdini installation docs [www.sidefx.com]. I installed cuDNN 9.15. The installer does not add the directory to PATH, which is C:\Program Files\NVIDIA\CUDNN\v9.15\bin\12.9. I did not need to copy over zlib.dll.
Even though the 3090 is the primary GPU, when set to CUDA it uses the 4090 (conveniently ideal for my use-case). Setting CUDA_VISIBLE_DEVICES does nothing. When using DirectML, it uses the 3090.
Because of this, the following benchmarks aren't apples to apples.
240f Large Neural Surface Test:

• CUDA (4090) - 159.7s
  
• DirectML (3090) - 272.5s
  

60f Small Neural Surface Test:

• CUDA (4090) - 11.8s
  
• DirectML (3090) - 17.9s
  
• CPU (9950x) - 42.6s
  

Thank @cdenny. This did the trick.

Leaving this for others:
Just added to PATH. As of today I had to use the cudNN 9.7, not the latest avail cudnn 9.16.
Cuda is cuda_12.8.0_571.96 - Had to disable Windows Antivirus for the Download and Install process from Nvidia though, as otherwise the Installer would fail with some 7zip data error.