I need to limit render time on batch renders. vm_timelimit?

   1646   2   1
User Avatar
Member
3 posts
Joined: 8月 2013
Offline
Relatively new to Houdini, please be understanding. I'm trying to put an arbitrary time limit on Mantra renders on our render farm.

vm_timelimit works perfectly in the render view with "Preview" (Enable gradual refinement) turn on by the button at the top of the tab window. And maybe checking "Enable Preview" in the Mantra nodes Preview tab will do the same thing on a batch (in this case Deadline specifically) renders? But that doesn't turn on Preview in the render view with Preview turned off there? Maybe I just have to batch render to test that? If I leave the time limit enabled in a render view render, but turn off preview, it just leaves any unrendered tiles black when the time limit is reached. I need gradual refinement on batch renders like I get in the Render View.

But I'm unclear about the parameters related to Preview and I don't see a detailed description? Can anyone add insights into these parameters, particularly in the context of batch renders, not the Render View? Can vm_previewtime be used with vm_timelimit? Previewtime limiting max tile time, timelimit limiting the total overall render time? How do previewadaptive and in particular previewadaptivefactor work? previewadaptivefactor set to 0 would have no effect? 1 full effect, but I don't understand the inbetween values?

Preview
Enable preview

Houdini render property vm_preview
IFD property renderer:preview
Enable IPR rendering


Preview time

Houdini render property vm_previewtime
IFD property renderer:previewtime
Target time to spend on a tile in IPR.


Enable adaptive preview

Houdini render property vm_previewadaptive
IFD property renderer:previewadaptive
Adaptively update tile sampling.


Adaptive preview factor

Houdini render property vm_previewadaptivefactor
IFD property renderer:previewadaptivefactor
Controls how much of an effect the adaptive sampling contributes per tile.


Thanks in advance!
User Avatar
Member
3 posts
Joined: 8月 2013
Offline
I ran some test frames on Deadline and the time limit was in effect, but a complete and properly formatted exr was not output and Mantra didn't give a happy exit code. The time limit was on, "Preview", "Preview Time", "Enable adaptive preview" and "Adaptive preview factor" were all enabled/set. Though I am unclear what units "Preview Time" is using/expecting, and I put "Adaptive preview factor" at 1, unsure what that meant beyond fully applied.

I don't know what to make of this output? Here's some relevant seeming snippets from a log.

The end of the log starting where the time limit is imposed:
"2021-07-28 15:30:32: 0: STDOUT: mantra: Render time limit exceeded (0:05:00)
2021-07-28 15:30:33: 0: STDOUT: Threading Efficiency: 6389.08%
2021-07-28 15:30:33: 0: STDOUT: Ray Counts:
2021-07-28 15:30:33: 0: STDOUT: 22,074,624 primary rays 2.50 rays/pixel
2021-07-28 15:30:33: 0: STDOUT: 255,141,269 occlusion rays 28.84 rays/pixel
2021-07-28 15:30:33: 0: STDOUT: 397,683,996 opacity rays 44.95 rays/pixel
2021-07-28 15:30:33: 0: STDOUT: 263,901,937 shading rays 29.83 rays/pixel
2021-07-28 15:30:33: 0: STDOUT: 938,801,826 total rays 106.11 rays/pixel
2021-07-28 15:30:33: 0: STDOUT: Unified Cache: 63.97 GB of 63.97 GB used
2021-07-28 15:30:33: 0: STDOUT: In Cache Faults Generated Item Size
2021-07-28 15:30:33: 0: STDOUT: Geometry: 63.84 GB 5524202 114.53 GB 21.74 KB
2021-07-28 15:30:33: 0: STDOUT: TBF Texture: 132.65 MB 20015 143.77 MB 7.36 KB
2021-07-28 15:30:33: 0: STDOUT: TBF File Cache: 2860 file opens (20.82 MB read)
2021-07-28 15:30:33: 0: STDOUT: OpenImageIO Texture statistics
2021-07-28 15:30:33: 0: STDOUT: Options: gray_to_rgb=0 flip_t=0 max_tile_channels=6
2021-07-28 15:30:33: 0: STDOUT: Queries/batches :
2021-07-28 15:30:33: 0: STDOUT: texture : 0 queries in 0 batches
2021-07-28 15:30:33: 0: STDOUT: texture 3d : 0 queries in 0 batches
2021-07-28 15:30:33: 0: STDOUT: shadow : 0 queries in 0 batches
2021-07-28 15:30:33: 0: STDOUT: environment : 354397970 queries in 354397970 batches
2021-07-28 15:30:33: 0: STDOUT: Interpolations :
2021-07-28 15:30:33: 0: STDOUT: closest : 0
2021-07-28 15:30:33: 0: STDOUT: bilinear : 0
2021-07-28 15:30:33: 0: STDOUT: bicubic : 354397970
2021-07-28 15:30:33: 0: STDOUT: Average anisotropic probes : 1
2021-07-28 15:30:33: 0: STDOUT: Max anisotropy in the wild : 1
2021-07-28 15:30:33: 0: STDOUT: OpenImageIO ImageCache statistics (shared) ver 2.0.10
2021-07-28 15:30:33: 0: STDOUT: Options: max_memory_MB=32751.0 max_open_files=100 autotile=0
2021-07-28 15:30:33: 0: STDOUT: autoscanline=0 automip=0 forcefloat=0 accept_untiled=1
2021-07-28 15:30:33: 0: STDOUT: accept_unmipped=1 deduplicate=1 unassociatedalpha=0
2021-07-28 15:30:33: 0: STDOUT: failure_retries=0
2021-07-28 15:30:33: 0: STDOUT: Images : 2 unique
2021-07-28 15:30:33: 0: STDOUT: ImageInputs : 1 created, 0 current, 1 peak
2021-07-28 15:30:33: 0: STDOUT: Total pixel data size of all images referenced : 48.0 MB
2021-07-28 15:30:33: 0: STDOUT: Total actual file size of all images referenced : 16.8 MB
2021-07-28 15:30:33: 0: STDOUT: Pixel data read : 48.0 MB
2021-07-28 15:30:33: 0: STDOUT: File I/O time : 0.1s (0.0s average per thread, for 66 threads)
2021-07-28 15:30:33: 0: STDOUT: File open time only : 0.0s
2021-07-28 15:30:33: 0: STDOUT: Tiles: 1 created, 1 current, 1 peak
2021-07-28 15:30:33: 0: STDOUT: total tile requests : 356000604
2021-07-28 15:30:33: 0: STDOUT: micro-cache misses : 65 (1.82584e-05%)
2021-07-28 15:30:33: 0: STDOUT: main cache misses : 1 (2.80898e-07%)
2021-07-28 15:30:33: 0: STDOUT: redundant reads: 0 tiles, 0 B
2021-07-28 15:30:33: 0: STDOUT: Peak cache memory : 48.0 MB
2021-07-28 15:30:33: 0: STDOUT: Image file statistics:
2021-07-28 15:30:33: 0: STDOUT: opens tiles MB read --redundant-- I/O time res File
2021-07-28 15:30:33: 0: STDOUT: BROKEN
2021-07-28 15:30:33: 0: STDOUT: 2 1 1 48.0 0.1s 4096x2048x3.f16 S:/PATH_REMOVED_FOR_PUBLIC_POSTING.exr UNTILED
2021-07-28 15:30:33: 0: STDOUT: Tot: 1 1 48.0 ( 0 0.0) 0.1s
2021-07-28 15:30:33: 0: STDOUT: 1 not tiled, 1 not MIP-mapped
2021-07-28 15:30:33: 0: STDOUT: 1 was constant-valued in all pixels
2021-07-28 15:30:33: 0: STDOUT: Broken or invalid files: 1
2021-07-28 15:30:33: 0: STDOUT: 1
2021-07-28 15:30:33: 0: STDOUT: Object A procedural of /obj/NAME_REMOVED_FOR_PUBLIC_POSTING displaced 1.61633e-05 (bounds set to 0.005)
2021-07-28 15:30:33: 0: STDOUT: Object A procedural of /obj/NAME_REMOVED_FOR_PUBLIC_POSTING_001 displaced 4.95911e-05 (bounds set to 0.005)
2021-07-28 15:30:33: 0: STDOUT: Frame Wall Clock Time: 0:05:15.96
2021-07-28 15:30:33: 0: STDOUT: Total Wall Clock Time: 0:05:16.79
2021-07-28 15:30:33: 0: STDOUT: Total CPU Time: 0:01:13.61
2021-07-28 15:30:33: 0: STDOUT: System CPU Time Only: 0:00:08.65
2021-07-28 15:30:33: 0: STDOUT: Peak Memory Usage: 167.59 GB
2021-07-28 15:30:35: 0: STDOUT: 0x00007FFD58A13E49 (0x00007FFD3854DA10 0x00000000043F09A8 0x0000000000000100 0x0000000000000000), RaiseException() + 0x69 bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x00007FFD559D3361 (0x0000000000000000 0x00000000043F1AF0 0x0000000000000000 0x0000000000000000), _is_exception_typeof() + 0x1081 bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x00007FFD5ACEF476 (0x000000000000024F 0x0000000000000000 0x000000000000024F 0x0000000000000250), RtlCaptureContext2() + 0x4A6 bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x00007FFD38528D5C (0x00000001DC212D88 0x00000001DC2132E0 0x00000000B7888280 0x00000000B7888280), ?readPixelSampleCounts@DeepScanLineInputFile@Imf_2_4_sidefx@@QEAAXHH@Z() + 0x18C bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x00000000526665E7 (0x00000000B78881E0 0x0000000000000004 0x0000000000000005 0x00000001C93F9710), newIMGFormat() + 0x2E57 bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x000000005265DFFD (0x00000001C987EE80 0x00000001C987F0E8 0x00000000B78881E0 0x00000001C987EE80)
2021-07-28 15:30:35: 0: STDOUT: 0x000000005265FC27 (0x0000000100000002 0x0000000100001000 0x0000000100000004 0x00000001CB637600)
2021-07-28 15:30:35: 0: STDOUT: 0x0000000025CB1C03 (0x0000000052673DC0 0x00000000043F47D0 0x00000001CADB1ED0 0x00000000043F4738), ?openFile@IMG_File@@MEAAHPEBD@Z() + 0x123 bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x0000000025CB1726 (0x00000000043F52F0 0x000000007EC40480 0x0000000000000000 0x00007FFD5848F0B0), ?open@IMG_File@@SAPEAV1@PEBDPEBVIMG_FileParms@@PEBVIMG_Format@@@Z() + 0x346 bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x0000000025C8D188 (0x000000007F9F2E80 0x0000000000000000 0x000000007EC40480 0x000000001A3CC7EE), ?IMGfindAndSetDataWindow@@YA_NPEBD0N_J_NH220N2PEAV?$basic_ostream@DU?$char_traits@D@std@@@std@@32PEAVUT_Options@@@Z() + 0x398 bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x000000005266055D (0x000000007EC40480 0x000000007EC40480 0x00000000043F5B80 0x0000000000000000)
2021-07-28 15:30:35: 0: STDOUT: 0x0000000025C9C752 (0x000000006F91C0C0 0x0000000000000001 0x000000002F82BFA0 0x000000001A3CE382), ?close@IMG_File@@QEAAHXZ() + 0x112 bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x0000000025CE7CC4 (0x000000007F9E6800 0x0000002A6ED02040 0x000000006F91EC60 0x000000001A3CC7EE), ?closeFile@IMG_FileFilter@@MEAAHXZ() + 0x24 bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x0000000025C9C677 (0x000000006F91C0C0 0x0000000000000001 0x0000002A510B3D60 0x0000000017D555E9), ?close@IMG_File@@QEAAHXZ() + 0x37 bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x0000000025CE7CC4 (0x000000007FB6D7C0 0x000000001A3CC7EE 0x000000006A27D0C0 0x0000000017D55508), ?closeFile@IMG_FileFilter@@MEAAHXZ() + 0x24 bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x0000000025C9C677 (0x000000006F91C0C0 0x0000000000000001 0x0000000000000000 0x0000000000000000), ?close@IMG_File@@QEAAHXZ() + 0x37 bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x0000000025CD6950 (0x0000000000000001 0x00000000043F5B80 0x000000006F91C0C0 0x000000006F91C0C0), ?close@IMG_DeepShadow@@QEAA_NXZ() + 0x20 bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x0000000017D31B54 (0x0000002A41A121B0 0x000000006A27D0C0 0x000000006F91C0C0 0x00000000043F5C60), ?close@RAY_DeepResolver@@QEAA_NXZ() + 0x84 bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x0000000017D55287 (0x00000001C8D5F0E0 0x00007FFD4DD85B42 0x000000002F053220 0x00007FFD58A185F6), ?close@RAY_Imager@@UEAA_NXZ() + 0x27 bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x0000000017DE7899 (0x00000001C8D5F0E0 0x000000006F91C0C0 0x000000007FB67280 0x00007FFD559D0000), ?finishRender@RAY_Scene@@AEAAXXZ() + 0x59 bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x0000000017DEC5F0 (0x000000006F91C0C0 0x0000000000000000 0x0000000000000000 0x4073BF5C27E95315), ?render@RAY_Scene@@QEAAHXZ() + 0x9A0 bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x0000000017E25E05 (0x000000007ECB6860 0x0000000000000001 0x0000000000000001 0x00000000241CDF20), ?checkIPRForced@RAY_Scene@@AEAA_NXZ() + 0x8DC5 bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x000000001843EC8A (0x0000000000000000 0x00000000043FCC7C 0x0000000000000000 0x0000000000000001), ?runCommand@CMD_Manager@@AEAAXPEAD@Z() + 0xB0A bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x000000001842B1E5 (0x00000000311E1380 0x0000000000000007 0x000000002F7D8180 0x00000000043FDC80), ?internalExecute@CMD_Manager@@AEAAXPEBD@Z() + 0x745 bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x000000001843D4FA (0x0000000000000000 0x000000002F7D8180 0x00000000043FBD30 0x000000002F7D8180), ?processInput@CMD_Manager@@AEAAXPEAVCMD_Source@@PEBD@Z() + 0xBA bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x000000001843BF90 (0x000000002F770100 0x00000000311E1380 0x0000000000000000 0x000000002F0AE801), ?internalSendInput@CMD_Manager@@AEAAXPEBD_N@Z() + 0x3F0 bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x000000001843F20D (0x0000000019A4DC78 0x00000000043FDDA0 0x0000000000000004 0x00000000043FE560), ?sendInputNoLock@CMD_Manager@@QEAAXPEBD_N@Z() + 0x4D bytes(s)
2021-07-28 15:30:35: 0: STDOUT: 0x0000000017E29E8A (0x000000006F91C0C0 0x0000000000000000 0x000000004F46A900 0x00000000311E132360:
2021-07-28 15:30:35: 0: STDOUT: -- TRACEBACK BEGIN --
2021-07-28 15:30:35: 0: STDOUT: Traceback from mantra 18.5.499 (Compiled on windows-x86_64-cl19.16):
2021-07-28 15:30:35: 0: STDOUT: SymInitialize returned error: 87
2021-07-28 15:30:35: 0: STDOUT: -- TRACEBACK END --
2021-07-28 15:30:36: 0: INFO: Process exit code: 150
2021-07-28 15:30:36: 0: Done executing plugin command of type 'Render Task'
2021-07-28 15:30:36: 0: An exception occurred: Error: Renderer returned non-zero error code, 150. Check the log for more information.
2021-07-28 15:30:36: at Deadline.Plugins.PluginWrapper.RenderTasks(String taskId, Int32 startFrame, Int32 endFrame, String& outMessage, AbortLevel& abortLevel) (Deadline.Plugins.RenderPluginException""





I am not trying to report a bug here, just understand what's happening. If indeed these results are seeming buggy, rather than user error or issue outside Mantra, like a Deadline issue, etc., I'll make a bug report in the proper way.
Edited by dylanear - 2021年7月29日 15:03:27
User Avatar
Member
3 posts
Joined: 8月 2013
Offline
No one has any clues or comments regarding enabling a render time limit on batch renders?
  • Quick Links