PDG Success

   2865   5   2
User Avatar
Staff
92 posts
Joined: Dec. 2015
Offline
Hey all, as we all know it's still early days for PDG, with lots of wrinkles to iron out - but we're looking for any successes (big and small) in using PDG.

Have you run a proof of concept and found that it saved time, and/or made you more efficient?

If so send me a PM here or email chebert@sidefx.com.

Doesn't have to be publicly publishable, we can keep a secret if needed

Cheers!
Chris
User Avatar
Member
691 posts
Joined: June 2006
Offline
Hey Chris!

I'm really enjoin PDG in my daily work. it speeds so much my workflow and more important I'm using my workstations in a more efficient way also. I just found that some workflows over-complicate things a little bit, that's why I'm trying to not over-use the hda processor approach; Not always you want to create assets for very simple things, instead I'm using heavily the geometry import and rop geometry output top nodes to import and manipulate geometry; I think both need more options to have a seamlessly experience with the SOP context but in the current state they are still very useful.

I hope that this tech continues to evolve, and more important I really hope it gets even more integrated into Houdini regards methodology.

Cheers!

Thanks,

Alejandro
Edited by Alejandro Echeverry - July 10, 2019 11:11:41
Feel The Knowledge, Kiss The Goat!!!
http://www.linkedin.com/in/alejandroecheverry [linkedin.com]
http://vimeo.com/lordpazuzu/videos [vimeo.com]
User Avatar
Staff
92 posts
Joined: Dec. 2015
Offline
Great to hear Alejandro, thanks for sharing…
User Avatar
Member
387 posts
Joined: Nov. 2008
Offline
I'm just scratching the surface with PDG possibilities. As other guys having a hard time to comprehend them (looking forward to some Masterclasses).

But my everyday use of the most simple TOPs with parallel File Cache Sop, parallel IFD baking (TOPs replaced older batch/python tools) and simple wedges with attributes make me hunger for more cores and ram to keep the production velocity. They are awesome!

Although I am still trying to find proper balance on using TOPs outside of Houdini to replace older proprietary (mostly python) tools at the probably more expensive (license cost) run on PDG core.
User Avatar
Staff
92 posts
Joined: Dec. 2015
Offline
Masterclasses coming soon… and thanks!
User Avatar
Member
150 posts
Joined: Feb. 2009
Offline
I've used PDG in production on my first two shows since March (working remotely), and am very close to being able to use PDG in production for hybrid cloud infrastructure with my community focused project openfirehawk.com [openfirehawk.com].

I love this workflow so much, if the next studio I were to work for onsite didn't provide me with the craziest machine possible I will provide my own. Dependencies have historically been a drag wherever I've gone- always with something missing, its time the solutions got democratized for artists. I'm very happy with the use I've put PDG towards in a short time.

Even as we all work through some of the challenges to solve some its workflow and understanding, I'm all in. It may take a little longer to converge upon a unified solution for all users and big studio schedulers, but the paradigm offered is a game changer for me.

I've built dependency systems in the past for studios that enabled some of the benefits of PDG, to achieve even a fraction of the functionality starting up would probably take me 1-2 months and long hours just to have some of the basic functionality, and it would only be limited to a single scheduler, so that code would lock me in to something I don't want to be deeply locked in to at this stage (a scheduler).

There's no other way for me to have achieved such high efficiency on the last two shows without what would have required a huge amount of work if I had to build it alone. In time PDG is also a solid blueprint for handling truly hybrid cloud infrastructure as well. There is no other existing blueprint that will be as flexible for hybrid infrastructure because of its dataflow concept for all kinds of problems that can be solved with it. The cloud is going to get very smooth because of this.

When I first heard PDG was coming, it changed my decision on the type of system to purchase for FX because I knew my ram and core needs would be a lot higher (with concurrent tasks), since PDG would enable management of high core counts locally with less work and faster iterations. So I bought a system with a spare empty proc slot to scale in the years to come. I prefer to have less systems to manage so that is a plus. With these per frame dependency abilities, a big studio might have some of what pdg does, but for a startup operation, its a tremendous advantage to hit the ground running, and in time handling multiple shots from a single hip file.

In any case, none of the studios I have worked for had such extensive dependency abilities to match PDG, perhaps the dev is a hard sell because studios don't have the metrics to understand the true cost of dependencies not being the best they can be. So for that reason also, I'm super grateful as an artist that Side FX decided to do this, and that dependencies are finally going to get sexy and democratized like they should be. I want these abilities wherever I go.

PDG allowed me to handle high iteration rates on a single workstation over different shots with big flip (80,000,000 particles) and Bullet simulations (85,000 objects) with per frame dependency and very low latency (something most schedulers struggle with). I was running multiple wedges over assets for parallelism, and also multiple wedges of parameter spaces to get sim settings right when I needed to. At times I had 7+ simulations+flipbooks simultaneously to zero down on the right artist choices, doing all of it on a single machine with preview rendered output as sims progressed as well. overlaying parameter settings in flipbooks with automatic mpegs on completion was also very helpful.

I am very happy that the amount of pipe I had to build to have such shiny features was reduced by a factor of 10 to get started, and is more capable than any other dependency system I have used in 10 years with Houdini on a local workstation.

If I was in a big studio again, I think it's amazing to see that we have a pathway to the first widespread workflow that allows artists to do this and be less hindered by a farm during crunches, while iterating over way more tasks locally than possible before. Sure, we still need to render, and some sim needs wont fit this use case, but when it comes to drafts and iteration the farm is less of a bottle neck because of what this tech will do provided companies leverage PDG to operate in the farm in tandem with the local scheduler which is blazing fast.

I think when studios realise the optimal workflow for their artists, it will change the types of systems that should be purchased if they want to remove the farm bottle neck from artist progress. PDG offers this for most FX tasks when people work the right way - fast local iterations before upping the quality for final submission. It also incentivises artists to keep setups fast since significant latency can be removed with the local scheduler, we have rapid visual feedback over tasks and more ability to speed things up.

Big thanks to your team for creating this for us!
https://openfirehawk.com/ [openfirehawk.com]
Support Open Firehawk - An open source cloud rendering project for Houdini on Patreon.
This project's goal is to provide an open source framework for cloud computing for heavy FX based workflows and allows end users to pay the lowest possible price for cloud resources.
  • Quick Links