Jeff Lait

jsmack
How is memory managed on platforms with unified memory? With no main memory to evict to, is that setting used at all? Is there any unnecessary copying of layer data when moving from cpu to gpu compute? I'm finding even with 128GB of unified memory, it can be pretty constraining, I can't imagine what it's like with only 16GB of VRAM.

On these platforms you'll want to leave the VRAM percentage to 100 as you don't want to ever "swap" to main memory. As you note, that will just unnecessarily copy it to a new location leaving you no farther ahead. Worse yet, if it is a read-only layer; next time it is needed for the GPU it will be copied to the GPU (and hence to normal RAM) and "helpfully" keep the CPU copy around. The same argument applies if you run in HOUDINI_OCL_DEVICETYPE CPU mode.

With 16gb of VRAM you'd likley have a similar experience, just that you'll be hitting the swapping to main memory - so you'd basically just have an "extra" 16gb of RAM to work with.

elovikov
I wonder now does it also mean that putting nodes inside compile block helps to save memory?

Yes, it does.

When compiled we do a feed-forward cook of the graph so can minimize the use of layers. We also can avoid cooking "dead" outputs on nodes, so you can get a performance boost there too.

It is, however, not very tenable to keep wrapping stuff in compiled blocks when playing around :>

There are some gotcha's, because as you note the parmaeters have to be all cooked while compiling the compiled block; not just in time while executing.

If you want to cap VRAM consumption you can use Edit::Copernicus Settings::Video Ram. By default we will use all of your VRAM before we start evicting stuff to RAM. For compositing you may get away with a low value for this - the high value is to avoid things like Pyro from swapping.

We don't convert to RGBA in memory - it may look like that because the nodes all operate on RGBA, but they are doing just-in-time widening; so the memory use of a Mono channel is just one mono value. However, we DO operate on 32bit float by default, so that is 4x the memory of a 8-bit image.

An important optimization we don't do yet is to throw away the results that are no longer needed. Currently every COP node keeps its output, which adds up quickly at 4k. This means you can walk the network without recooking; but that is cold comfort when you've run out of memory.