Dear SideFX,

I don't want to bother you and the PDG dev team, but in a nutshell here are my experiences with the first iteration of TOPs:
Downloaded the first production release (which was ~1 month old after the announcement), the daily build wasn't available, as usual (why?).
Inspired by the presentation, I thought it's a great idea if my next tutorial for 3D World mag would be about the basics of PDG, and this kind of node based automation would be also very useful for my scientific/data visualization research, ie great opportunities with CSV import etc. So I jumped in and started the learning with the official tuts(which are great btw) and do experiments, understand it's guts…

Sadly I quickly realised that this isn't a finished product, tons of bugs and missing basic features, chaotic properties etc.
I checked the Changelog and there were a lot of fixes/changes. Ok, lets put away, and download the daily build, when it's available. I did so, but again, clearly, this isn't a finished product…. so I decided to “ground” TOPs/PDG until it gets the acceptable quality level, which is higher with other modules of Houdini, even first releases, like Vellum in 17 (almost grounded then though…). Not just the frequent crashes with TOPs, but also the the feel of an experimental software. The modifications in the daily releases which I read in the Changelog and just running through the topic titles here underline my experiences.

Ok this isn't the airliner industry, but how much these glitches affect the work and the joy of work of artists/TDs is an another story…

PLEASE, PLEASE can you fix at least the main issues before the release next time? The Oscar acceptance speech of Mark Elendt crossed my mind:
(is should start at the relevant part, 2:50) EDIT: no, it doesn't work so scrub to 2:50


Btw I'm afraid, that the Python programming language is somehow guilty… So guys I guess it's a production nightmare to bring together a working PDG system, I have no dubt, but what Tesla does in example: they postpone the sale until they reach a certain quality level (which doesn't mean perfectionism, rather practical thinking in long terms), and this is the mandatory, not the original release date. When I “filed divorce” from Maya and Clarisse, the main reason was the unacceptable amount of bugs, some really serious, I should say ridiculous. It's easy to fall down in terms of code quality for the sake of wow factor, as we learned with so many softwares.

Again my intention is to express my feelings, not to bother you, and I'm also interested in other users overall experiences and opinions with PDG/TOPs.

Cheers,
Greg

I don't think MCAS is using PDG, if that's what you're implying.

Luckily not. Python shouldn't be allowed/certified for flight critical systems, if it would, I would never get on a plane, seriously. The requirements there are totally different so the quality level is much higher than in our industry, they use special programming languages and hardware. The worst thing with the MCAS system isn't the software itself (however using non-redundant single input for such flight control commands is against the rules) but the reuse of a more than half a century old aircraft design with this sofware “patch” (fourth generation) to get certified with roughly two times more powerful jet engines than the original to carry even more pax. It's good for the airlines for extra profit and for Boeing to keep the race with Airbus. I mean in short terms. This is a shame story but it would be offtopic to explain here (if you are interested beyond the media hype:
https://theaircurrent.com/aviation-safety/vestigal-design-issue-clouds-737-max-crash-investigations/ [theaircurrent.com]
Less tech geek, but more personal: https://www.seattletimes.com/business/boeing-aerospace/for-victims-loved-ones-latest-boeing-737-max-tragedy-leaves-anguish-anger-and-lots-of-questions/ [www.seattletimes.com] )

I used the word “ground” because I should actually rule myself to not play with the PDG I already loose considerable amount of time to figure out strange things which they modified/enhanced in the next few daily build iterations, TOP SOP is still crashy etc.

re: csv work, you can use SOPs and vex for this.
Insanely fast and you get a spreadsheet… plus vex/VOPs flexibility.

My take on TOPs so far is that its more of a ROPs augmentation than an ETL tool. Kinda feels like a backwashed studio project; very specific and un-Houdini-like.

Hi Greg,

Thank you for your honest feedback, and we are sorry your first experience with PDG is not as smooth as we would like. Ahead of release, we were most focused on the FX and wedging workflows - we realize there are some features especially around the periphery of PDG that are in need of some love and polish, and we are hard at work closing the gap around those. PDG is a project of enormous scope - the use cases are so varied that it was difficult to get the full range of feedback ahead of release. This is not to make excuses, we acknowledge our shortcomings here and we will fix them. Please help us concentrate our efforts on the area(s) causing the most pain. You have identified the CSV nodes and TOPSOP - those are known to us - we're going to make significant modifications and improvements to those, and will roll them out in a future production build. If there are other specific areas causing pain, please point them out. We'll post updates on the CSV stuff as we have them.

Specifically around the TOPSOP - can you supply us with a reproducible case for the crash? That would help us a lot to track down the problem there.

Hi Ken,

Thank you for your feedback! I really appreciate your presence here in the forum, it's great when a developer doesn't hide behind the curtain, what some other CG sw devs do…

I think there would be an easy way to cause less pain but with minimal extra effort:
Sell the new version with the new such features as experimental/preview etc. module.
So in this case the same dates, same reveal show, just with the title like:
Houdini 17.5 with PDG Preview.
… and also do the same in the documentation/release notes, a clear statement that this is a preview/experimental module, and the user can shape the final product, lets discuss on the forum etc.
So if Solaris will be similarly “liquid”: Solaris Preview would make the sense.

I think Houdini users (and potential ones in the future) would appreciate more this approach, than what some other companies do. Learning the first release, then learning again the modified one is much harder and disappointing than doing the job with this in mind: this is a preview, so everything can change in the future (which is the case). I think CG people tolerate and prefer this honest way much more than the other one (Maya&etc story).

Communication matters a lot.
There is a trend in the sw industry that companies sell betas/experimentals as final product, I hope that SideFX will avoid this in the future.

As cpb mentioned, TOPs feels un-Houdini-like, I feel the same.
A TOP VOP would be great I think, less Python but more opportunity to customize/tweak the pipeline graphically without any coding. It might sound a bit pervert at the first time, but ie Point VOP runs the node chain on every point. So in TOPs, the points would be the work items.
Is there a chance for a somewhat VOP/VEX, C++, OpenCL based PDG toolset in the future?

I've seen the CSV additions in the changelog, it's nice to see the instant improvements! I'll try them later when the PDG isn't at preview/constantly changing state.

I'll send crash logs in private regarding the TOPSOP.

Cheers,
Greg

Thank you for your understanding Greg. I agree with your point about messaging, but “Houdini 17.5 with PDG Preview” doesn't reflect the true state of the product either. If the whole thing was really in that state, I'd have a few more grey hairs and wrinkles added to my face by this point .

My honest assessment of the situation is this - the core of the PDG graph/tech is rock solid, the wedging/FX workflows are in good shape and we're seeing good results here, schedulers - local scheduler is solid, HQueue works well but has usability issues in setup (not a new thing), particularly with mounts / UNC pathing issues. Tractor/Deadline each has some polish issues but mostly solved now, but with the notable problem on Tractor with the way it creates too many jobs that we're still working on (farm setup unfortunately is always an advanced topic and never fun). The peripheral nodes - all the partitioners are solid, the mappers are for advanced users only at this point, imagemagick and ffmpeg are in good shape, but JSON was missing key features and CSV nodes were, frankly, not quite there. That said, we have already fixed many of the most serious problems, and with continued focus I think we can close the remaining gaps soon.

A large part of the problem is with testing - try as we might, our resources are limited internally and so are the breadth of use cases that we can cover, especially for a project of this scope, which is for Houdini but which is also aimed so much at areas outside of Houdini. During alpha/beta, most our testers are from super big shops - and they are super technical and mostly write their own stuff anyway. So they'll tend to push the framework/core of the tech to the limit, but then will tend to leave the more “out of the box” nodes less tested. We had some other unforseeen issues outside of our control coming into release as well, like losing key team members at a really bad time, that made the last stretch of this thing “fun”. With 20/20 hind sight, the most optimal thing we could have done is to hold a few of these peripheral nodes like CSVs back and add them in a later production build in a more polished state, or label those particular nodes as “beta”. Alas, we aren't perfect, especially me , and that's not what we did. Thanks though for your patience and understanding, and for giving us feedback and helping to track down the issues. We'll keep at it - there are definitely some more wrinkles to iron out, but we're not too far from getting the most serious ones tucked away.

In terms of TOP VOP, GPU etc, yeah there is a ton of forward development on PDG. But as you said, we have to make sure what is there is well polished first.

Hi Ken, thanks for your explanation.
It seems - as usual - I ran into the weakest part of a new system. Additionally what I need is the opposite of the big shops, but might be similar to other indies. That's why I asked for the opinion of others in the first message.

Thus the “PDG Preview” wasn't appropriate for the whole system, sorry, it was a quick idea, and yes it woud be better to flag things at node level, like you mentioned.
I think it would be great in the documentation some sort of a state of nodes page, like a table with all the nodes, and in the first column the solid ones would be, in the second the betas and in the third the ones like CSV, which are changing, with the note like “Visit our PDG section in the forum for any feedback, suggestion etc.”. I guess some users don't know that there is this dedicated PDG forum. I just accidentally run into it, and would send this in the lounge originally.

Perhaps extending the Tab menu settings in the prefs would make also the sense, like this:
|x| Don't show experimental nodes
|x| Don't show beta nodes

Yes, a few days ago a couple of us were lamenting the fact that if the CSV nodes were someone's first impression of PDG, it wasn't going to go well. With better testing, we should have identified those specific nodes and held them back or labeled as such, as you suggest. We've already gotten much better with them, and we'll make a posting once we feel we've reached the next level with these.

The scene with TOPSOP which I mentioned earlier doesn't crash now, and it's much convenient to create wedge based permutations in SOPs, than use a separate TOP container.

As I mentioned in my H17 review [www.creativebloq.com], developers at SideFX really consider bug reports, it seems PDG isn't an exception I'm looking forward for the further enhancements.

Before PDG, working in Houdini likes:
Working -> Working -> Working -> Oops, crashed -> Working -> Working -> Working…
After PDG:
Working -> Crashed -> Worki -> Houdini freezed -> Wor -> Closed instantly -> Wo -> Crash again … The crash/restart/cook time is more than my working time.
After digging in PDG for several months, I really don't want to say that PDG is such an incomplete tool and is so much pain to use.
The HDA Processor is soooo SLOW, and hard to debug, without any real-time feedback.
The cache mode is confused me at first, but the “Auto” mode is just a file existing check which is disappointing.
Anyway, Oops, My Houdini crashed again when I cook PDG Node.

Still learning, but my impression of TOPS is a bit different, but I guess my expectations are also quite different.
I see it more like a scheduler on steroids, that greatly increases You throughput on farm. To have all this fine controlled dependencies is just amazing.

Just some examples:

- Clustered and sliced simulations, in addition with wedging in a custom scheduler environment was doable, but You had to use environment variables, that communicate the wedge/slice/cluster information to the batch process. It is working, but would break much easier and was kind of tricky in developing state. PDG on the other side was just working and with a huge artistic usability.

- All kind of repetitive workflows are now much easier to set up, without the need to code a lot. You can plug Your workflow together and modify it on the fly. You can create Digital Assets from this workflows and create higher levels of abstraction for this, so that one node does in fact do several things. Coding and maintaining this is of course doable, but can get quite difficult as soon as You have multiple deep dependencies, that should be parallelized on farm, to use all the resources.

- But even without farm computers, we see now cpus with up to 64 cores in a single cpu setup, You have to use that resources as well.

For example, You process all the collision geometry preparation in the background, while already working on some other parts of Your scene, or even let PDG completely build all Your hips to that point, that all necessary assets are loaded, and the mandatory steps are also done in background, as well. You open a hip where everything is done to that point where You and Your artistic skills are really needed.
After this PDG takes care of all the other stuff that is necessary in a production. It can free up a lot of Your time for the stuff that no machine can do. This means You are faster and more productive. A lot of work is still very repetitive and if You can shift it to PDG, great.

I might miss a lot here, but that is my initial impression of PDG. Probably it is much more, but step by step

Hi Eric,

We are sorry to hear that your experience with PDG has not been smooth. Since launch, working with our clients, we have had promising successes but also cases where it's not quite where we want it to be. We have made an effort to be as transparent as possible with its current state, and freely acknowledge where we are still short, with a commitment to continuously strive to close the gaps where they exist.

With that said, while we are aware of some problems with PDG, instability and crashing all over the place is not one of them. We have a quite a number of clients actively testing and using the tech as this point, some pushing it quite hard, so we'd be quite surprised if there was some systematic instability with it that none of them informed us of. The main problems with PDG currently, as far as we are aware, are:

- UI / UX

https://www.sidefx.com/forum/topic/67311/ [www.sidefx.com]

- Peripheral stock nodes not having enough functionality (we have improved some, especially around CSV, but still probably not enough)

- Deployment issues around default farm bindings to Tractor and Deadline, with many issues already fixed such as the submitting too many jobs issue and firewall complications.

- Training. Still a relative lack of training material, especially around key core concepts to work well with the system. Master class(es) coming up.


I wonder whether some of what you're experiencing has to do with issues like this raised, where PDG was used in a way that's not really supported:

https://www.sidefx.com/forum/topic/67118/?page=1#post-285971 [www.sidefx.com]

That's still our fault, by the way, because we still haven't gotten the training material out on these “not supported” gotchas. All of this is not to say that your problem isn't real, nor is it an attempt to be defensive about this - it's entirely possible that you have found cases where PDG is not behaving well. If you could please provide us a test scene (perhaps in a different thread, so we can focus on just the technicals of that), it would help us track it down. We can also take up the HDA Processor slowness there as well, there are some features to mitigate this.