Temporary USD files

   6911   16   2
User Avatar
Member
326 posts
Joined: Jan. 2013
Offline
Hi there,

The parameter to the USD Render node says that
"The directory where any required USD files will be saved. If this directory is in the $HOUDINI_TEMP_DIR, it will be deleted automatically when the render is complete."

But at the end of the render, they are still not deleted, but remain along the same path that was specified in the parameter.
I can't figure out if this is a bug or do I need to do something extra?

So far, the temporary solution I've found is to use Husk Post Frame which will delete the temp directory after each frame is rendered, is there a better solution?

And is it possible to specify directly from the interface which USD file to render without using temporary files?

Debian GNU/Linux 11 (bullseye)
Houdini 19.0.601

Thanks!
User Avatar
Staff
593 posts
Joined: June 2020
Offline
That doesn't sound right. Can you share either a hip file or at least a screengrab of your USD Render node's parms? It definitely should be cleaning up after itself if the path is a subdirectory of $HOUDINI_TEMP_DIR.

Attachments:
cleanup.mp4 (278.2 KB)

User Avatar
Member
326 posts
Joined: Jan. 2013
Offline
robp_sidefx
That doesn't sound right. Can you share either a hip file or at least a screengrab of your USD Render node's parms? It definitely should be cleaning up after itself if the path is a subdirectory of $HOUDINI_TEMP_DIR.

Thanks for the reply Rob.

Ah, I get it. The directory is cleared only if a path is specified in $HOUDINI_TEMP_DIR, but not automatically cleared if another path is given.

We had a problem with this because USD Render can potentially create large temporary files that the user may forget to delete or if they are located on a render farm where the artist usually does not have access and it turns out that the only option is to delete this after each frame is rendered by the script .

Or maybe there is a better option?
User Avatar
Staff
593 posts
Joined: June 2020
Offline
alexwheezy
Ah, I get it. The directory is cleared only if a path is specified in $HOUDINI_TEMP_DIR, but not automatically cleared if another path is given.

Exactly this.

I'll try to dig into the design history of the current behaviour and will reply again after that.

It would be useful to hear a bit more detail about your farm-rendering scenario (when do which steps of the process get executed on which machines, etc).
Edited by robp_sidefx - May 3, 2022 07:27:18
User Avatar
Member
326 posts
Joined: Jan. 2013
Offline
The task is sent to the render server which distributes the task itself to a free node and if we affect the use of USD in this pipeline then when creating temporary USD files, they will be saved on the local disk of the node itself namely in HOUDINI_TEMP_DIR which can potentially be located not on the system disk but on a system with limited resources, for example, has insufficient disk space, and in this situation, the inability to delete temporary files automatically by means of the node itself leads to the fact that we use Husk Post Frame to delete them.

It's just that my uncertainty is whether this is the expected behavior or not? Should we take care of deleting the temporary files that USD Render creates?
User Avatar
Staff
593 posts
Joined: June 2020
Offline
Sorry for the delay in replying. I'm still trying to tease apart the exact workflow here. You mentioned "they will be saved ... in HOUDINI_TEMP_DIR". But they're still not being automatically cleaned up?
User Avatar
Member
2 posts
Joined: Sept. 2013
Offline
Hey Rob! Long time now see!

This issue has been driving me nuts. Especially when doing crazy pyro clustering whith lots of vdb combining/merging. It was filling up temp drives on our farm and crashing systems when a student got over zealous. I made a python script to work around the issue if there's any interest in it.

1. select the usdrender_rop
2. in the Husk Post-Render field: $HOUDINI_TEMP_DIR/usd_renders/$RENDERID/cleanup.py - (this script will be generated by the next script)
3. In the Post-Frame Script use this python:

import hou
import os

# Get the currently selected USD_RENDER_ROP
selected_node = hou.selectedNodes()[0]

# Get the directory path from the selected USD_RENDER_ROP's 'Save To Directory' parameter
dir_path = selected_node.parm('savetodirectory_directory').eval()

# Create the cleanup.py script in the directory path
cleanup_file_path = os.path.join(dir_path, 'cleanup.py')
cleanup_script = 'import os\nfor file in os.listdir("{}"):\n\tif file != "cleanup.py":\n\t\tpath = os.path.join("{}", file)\n\t\tos.remove(path)\n'.format(dir_path, dir_path)
with open(cleanup_file_path, 'w') as cleanup_file:
    cleanup_file.write(cleanup_script)

# Save the directory path to a text file in the same directory
path_file_path = os.path.join(dir_path, 'path.txt')
with open(path_file_path, 'w') as path_file:
    path_file.write(dir_path)

I can't wait to implement this in the studio. It has been driving me crazy!

Cheers!
Kyle
User Avatar
Staff
593 posts
Joined: June 2020
Offline
Hey Kyle!

If you have a scene you can share where $HOUDINI_TEMP_DIR/.../myUsdFiles isn't being cleaned up, please do share it so I can dig a bit deeper.

- Rob
User Avatar
Member
2 posts
Joined: Sept. 2013
Offline
Hey Rob,

Yes happy to share it. Although I've only gotten this working on the usdrender_rop, I still need to sort out how to implement in the top context as well as in a ropnet using a fetch.

Kyle

Attachments:
usd_volume_bug_v006.hipnc (1.3 MB)

User Avatar
Staff
593 posts
Joined: June 2020
Offline
Thanks Kyle. I can reproduce that the .vdb files aren't being cleaned up, though the .usd files are.
User Avatar
Member
1 posts
Joined: Jan. 2021
Offline
Hi Guys,

I'm still experiencing this problem when using topnet and rop fetch, it'd fill up my storage really fast when doing some heavy sims. All the temp vdb files are stored in the path: C:\Users\sdqdj\AppData\Local\Temp\houdini_temp\usd_renders.

Could someone provide a solution for this? I would greatly appreciate it.

Yeqin
User Avatar
Staff
593 posts
Joined: June 2020
Offline
This should have been resolved with Houdini 20. Are you testing with that, or 19.x?
User Avatar
Member
7 posts
Joined: Oct. 2019
Offline
Hi,

We seem to be running into this issue too on our renderfarm. The folder appdata\local\temp\houdini_temp\usd_renders is not being cleaned up after render, filling our drives rapidly.

We are running 19.5.605 and are unable to change versions.
User Avatar
Staff
593 posts
Joined: June 2020
Offline
This has been backported to 19.5.829
User Avatar
Member
7 posts
Joined: Oct. 2019
Offline
Thanks for the update, unfortunately we are locked in at 19.5.605.

We managed to get around it by running a pre frame and post task script in Deadline that clears the folder before and after the job finishes just to make sure it stays empty.
User Avatar
Member
1 posts
Joined: Feb. 2017
Online
This is still a issue for us on Houdini 20 635 using deadline.

Woke up to the whole farm down due to filled up root drives .
User Avatar
Staff
593 posts
Joined: June 2020
Offline
Do you have any way of telling whether the files were from successful or failed renders? I believe we're doing a fairly good job of tidying up after ourselves when Houdini shuts down cleanly, so I'm curious to hear more about your situation.
  • Quick Links