How PDG decides if a file on disk is dirty or valid

   2652   2   1
User Avatar
Member
65 posts
Joined: May 2009
Offline
Hi,

I'm wondering if there is a place in the documentation, that describes the exact logic how PDG/TOPs decide if a file on disk is a valid finished work item, or a dirty leftover from a previous run?

With my - pretty limited - hands on experience I couldn't come up with a working mental model for this. It seems it's not a `delete to dirty` type primitive approach, nor a checksum based `see if upstream deps changed` type one.

What do I miss here?

Thanks in advance,
Mate
User Avatar
Staff
585 posts
Joined: May 2014
Offline
PDG only cares about existing files on disk if the node supports caching, i.e. it has the Cache Mode parameter. If the Cache Mode is set to Write Files the work items will always recook, even if the output files are found on disk. If the parm is set to Read Files then PDG will check for an existing file, and if its found the work item will be marked as cooked. If the file is missing the work item is treated as failed. PDG has no way to generally validate that file is correct – it simply checks if the path exists.

When the Cache Mode is set to Automatic the work item will write to disk if the file is missing, or use the existing file if its found on disk. Additionally, if an upstream dependency has a missing cache file, that also invalidates the cache of any downstream work items. For example if the first node in your graph is set to Write Files, causing those work items to always write to disk, then all downstream work items would also be forced to write to disk because of that.

It's possible to install a custom cache handler, which is described here: https://www.sidefx.com/docs/houdini/tops/custom_tags.html#custom-cache-handlers [www.sidefx.com]
User Avatar
Member
65 posts
Joined: May 2009
Offline
Thanks a lot!
I was missing the Cache Mode parameter. Getting there..
Best regards,
Mate
  • Quick Links