Forums Solaris and Karma Animation sequence rendering practices - speed loss once sent to Deadline

Animation sequence rendering practices - speed loss once sent to Deadline

   760   7   2
User Avatar
Mirko Jankovic
Member
98 posts
Joined: 8月 2015
Offline
 
I was wondering if there are some better practices when rendering animations.
What is clear from even earlier test is that rendering in GUI provides faster results.
When shot is submitted to farm (using Deadline in my case) time per frames increase 2-3 times at least.
I understand that if rendering frame by frame in Deadline it will reload scene every time over and over again, unlike maya's batch rendering where first frames is slower including time to load but the rest of frames are just frame render time.
One option is of course to render bigger frame batches but that does bring other issues at hand.
Finally before Solaris with redshift ROP I had best time's when rendering with hscript.
Ie in houidni env load scene and start rendering with hscript and that keeps scene loaded and with skip rendered frames in redshift options even multiple machines on farm can render full speed. But have to run script manually on each machine and also monitoring overall progress is a bit messier.

So all in all I was wondering to see what practices when rendering animated sequences with Karma XPU from Solaris are others using?
Any tips and tricks?
It is rather annoying that I can see render times rather close to Redshift speed wise in my scene, but when sent to farm it gets extra hit in speed.
Edited by Mirko Jankovic - 2024年1月23日 13:54:09
 Report
User Avatar
jsmack
Member
7771 posts
Joined: 9月 2011
Offline
 
I don't know anything about deadline, but husk can render chunks of frames from a usd file without having to start from scratch each frame. If my experience with other render farm software holds with deadline, you pretty much ignore all the plugins they ship with and write your own integration to get the best performance.
 Report
User Avatar
Mirko Jankovic
Member
98 posts
Joined: 8月 2015
Offline
 
I'm afraid that writing custom plugins is a bit out of my scope
I did run into people trying to solve that and sharing on deadline forum asking for official deadline Solaris husk plugin but in the mean time they are trying something:
https://forums.thinkboxsoftware.com/t/official-houdini-solaris-husk-submitter/30214/88 [forums.thinkboxsoftware.com]

I'm still trying to figure out if this one form the forum can do the job.
 Report
User Avatar
Jonathan de Blok
Member
253 posts
Joined: 7月 2013
Offline
 
For deadline jobs you can set the chunk_size to something bigger then 1 (the amount of frames per task). You don't have to render frame by frame. Depending on the scene I usually set it between 25 .. 100 or something like that.
Edited by Jonathan de Blok - 2024年1月24日 04:59:30
More code, less clicks.
 Report
User Avatar
Mirko Jankovic
Member
98 posts
Joined: 8月 2015
Offline
 
Yea I'm aware of that one and mentioned it, but it can be a problem when you are rendering something more complex that takes like 10-15 minutes per frame, and chunk of 10 of those can take hours. If any of the workers fails for whatever reason it will render whole bunch from start. And with missing skip existing frames in Solaris that is time wasted even more. So yea it all comes to balancing but still feels like time wasted to loading scene between each task IF there is a way to make it work more similar to mayabatch ie load once and then just keep pumping frames
 Report
User Avatar
Jonathan de Blok
Member
253 posts
Joined: 7月 2013
Offline
 
Btw, there is this annoying deadline issue that trips up all the caches, not sure of it happens in Solaris as sell. It's in deadline's houdini script where there are a few lines that trigger refreshes on all parm/nodes that have a fileReference. Even if they are upstream of cached output etc.

This caused massive delays for me a while back compared to straight UI renders..

I'll grab some more details when I get back to my workstation.
Edited by Jonathan de Blok - 2024年1月24日 13:00:22
More code, less clicks.
 Report
User Avatar
Jonathan de Blok
Member
253 posts
Joined: 7月 2013
Offline
 
It's this bit..

(You can copy anything to the 'custom' folder and deadline will use that instead of the original files. This keeps things relatively safe during updates etc. So copy this:
\DeadlineRepository10\plugins\Houdini
to: 
\DeadlineRepository10\custom\plugins\Houdini
)

The issue is with the path remapping code, as a quick fix add the 'if False' bit from the screenshot, or just delete it.



def perform_pathmapping(tempdir):
    """
    Perform pathmapping on all input parameters and other select parameters
    :param tempdir: the temporary location that a text file will be written to to aid pathmapping.
    :return: None
    """
    if not tempdir:
        print("Temporary Directory has not been set. Skipping Pathmapping.")
        return

    print("Begin Path Mapping")
    # gather a list of all parameters that need to be pathmapped
    if False:
        parms = gather_parms_to_map()
        if parms:
            pathmap_parms(tempdir, parms)

        envs = gather_envs_to_map()
        if envs:
            pathmap_envs(tempdir, envs)
    else:
        print("Skipped path mapping: line 367 custom/plugins/Houdini/hrender_dl.py")         
    print("End Path Mapping")

Attachments:
Screenshot 2024-01-24 192034.jpg (202.1 KB)

More code, less clicks.
 Report
User Avatar
Mirko Jankovic
Member
98 posts
Joined: 8月 2015
Offline
 
Ah yes I actually forgot about that path mapping issue and did run into it with fairly big env scene where path mapping was taking longer then render it self. While looking at log it was stuck at path mapping for looong time then just goes through rendering.
Will test how it goes with your fix. Thanks!
 Report
  • Quick Links