Search - User list
Full Version: Mantra startup time + PDG - [was Mantra geometry preparation]
Root » PDG/TOPs » Mantra startup time + PDG - [was Mantra geometry preparation]
anon_user_37409885
Is there any way to use PDG to improve the bottleneck in preparing geometry for rendering in Mantra?

Currently a very light-weight scene, i.e. a particle system or simple geo, spends most of the ‘render time’ in geometry preparation.

Thx!
anon_user_37409885
@Daryl Thanks for the question.

If anyone can help out with an answer to the original question I'd appreciate that.
anon_user_37409885
To clarify the question for anyone whom may wish to help; this is not a SOPs, DOPs, or ROPs question but a Mantra + PDG question. The geometry setup internally with Mantra is a huge delay when doing lightweight renders, if PDG can help that part I'm all ears.
jason_iversen
goat
To clarify the question for anyone whom may wish to help; this is not a SOPs, DOPs, or ROPs question but a Mantra + PDG question. The geometry setup internally with Mantra is a huge delay when doing lightweight renders, if PDG can help that part I'm all ears.

Do you feel that delay with rendering with IPR?
anon_user_37409885
jason_iversen
Do you feel that delay with rendering with IPR?


IPR is perfect! No delay and I was hoping that PDG would allow us to access the magic IPR magic in a render to disk/flipbook
jason_iversen
goat
jason_iversen
Do you feel that delay with rendering with IPR?
IPR is perfect! No delay and I was hoping that PDG would allow us to access the magic IPR magic in a render to disk/flipbook

Then I suspect you're not suffering from a computational bottleneck at all but rather relatively slow launch of mantra and mplay. Are you on Windows? Linux?
anon_user_37409885
sure - it's the startup and initial setup of mantra. It occurs on Linux, windows and mac os. Do we have a solution for this slowness?
kenxu
If it's within Mantra itself, I'm afraid it won't be able to take advantage of PDG, at least for now. Any change to make Mantra faster would need code changes within Mantra, and to this point we have not done so in the case of PDG. I'll pass along the query though to our Mantra team, so they may take this into consideration.
mark
goat
sure - it's the startup and initial setup of mantra. It occurs on Linux, windows and mac os. Do we have a solution for this slowness?
Depending on your studio's setup, mantra's startup time can be affected by engine mode. What times do you get when you run:
% time mantra -e none < /dev/null
% time mantra -e basic < /dev/null
% time mantra -e full < /dev/null
On my machine, the startup time difference is 0.5s vs. 0.9s. You can also control this using the
MANTRA_ENGINE_PROCEDURAL variable.
anon_user_37409885
Testing those I get similar times, but running a flipbook on a torus it takes ~3 sec between frames. If launch times are fast then what is this disparity and can it be reduced to 0.59 sec instead of 3 sec? thanks!

mantra -e none
real 0m0.599s

mantra -e basic
real 0m0.726s

mantra -e full
real 0m0.722s
jason_iversen
It would be nice if you could use IPR mode for a flipbook, but alas you can't - not unless you write a script to step through the timeline yourself and rip the image from hou.IPRViewer and append it to a flipbook at timed intervals, checking for a time limit or not hou.IPRViewer.isRendering().

… which brings up a little point: the Mantra “Render Time Limit” feature seems to exit with an error code when it shouldn't.
anon_user_37409885
Thanks Jason - I guess in trying to see if PDG can help with this issue we need to understand what is causing the 3 sec delay between frames. If it's not geometry preparation, as IPR is instant, and it's not launch times of Mantra, as that is only 0.5 sec, do we know what is causing the missing 2.5 sec?
jason_iversen
Make friends with vtune or strace is what I do.

Often it's loading of libraries and stuff.

Are you on Linux, Mac or Windows? Are you running in a deeply stacked production environment?
anon_user_37409885
With more testing it looks like the culprit has been found.

The ‘missing’ 2.5 secs appears to be not geometry or launch times but material setup. Putting a simple shader like constant, specular or a texture removes the delay, down from ~4 sec a frame to 1 sec. The rendering time part is the same, just the delay between frames is removed.

Without any geometry in the scene, frames launch straight away one after each other, whilst adding a torus will make each frame take a total of ~4 sec. We assume that it's using the default plastic shader. Appling constantVop material reduces the total rendering time for a frame to 1 sec.

My assumption is that LOPs could fix this and there is probably very little PDG can do to improve material setup time.
jason_iversen
Ah yes, okay. Grubbing around the back of my memory, I thought there was a VEX (disk) cache that was persistent and able to be reused render to render… something which - at least for a while - was only enabled on Linux? Or perhaps it invalidates that cache frame to frame? SideFX might have to jump in here and jog my memory.

Usually settings for these caches turn up in the environment variables (http://www.sidefx.com/docs/houdini/ref/env.html) - like HOUDINI_DISABLE_VEX_INSTANCE_CACHE
anon_user_37409885
oh cool! That's great to hear.

I am getting some permission errors in the 17.5, i.e.

Error:       Saving to stream failed (ref: String Save <Permission denied> ).

but it more sounds like a bug that VEX shaders aren't cached. I couldn't see any env variable that was relevant unfortunately.

I'll submit a bug! bug #95480
morganv123
I have run into this too. Using a ropmantra node in TOPS to render 100 frames of a very simple Font SOP text node take around 48 seconds a frame - even with “All frames in batch” turned on. Whereas a standard Mantra node in Output context takes 12 seconds per frame.
anon_user_37409885
morganv123
I have run into this too. Using a ropmantra node in TOPS to render 100 frames of a very simple Font SOP text node take around 48 seconds a frame - even with “All frames in batch” turned on. Whereas a standard Mantra node in Output context takes 12 seconds per frame.

This sounds more to do with the number of CPUs in the local scheduler being set to 0, that then means the logical CPU are divide by 4. @kenxu do we have any enlightenment on why the 1/4 was used?


@morgav123 Could you try setting the CPUs to something like -1 or 128. Also try to putting 1 in the Frames per Batch.
kenxu
The 1/4 was used as a heuristic to avoid overloading the machine by default. We had cases where the user tried to run way too much on a single machine: 80 sims each with 250 frames with each frame baking out 1 gig of data. To top it off the sims used opencl, which drained the graphics memory - drivers do not behave well under those conditions. To prevent situations like that, we opted for some very conservative settings. Feel free to up the number of CPUs used - just please keep in mind what you're asking the machine to do
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Powered by DjangoBB