Found 132 posts.
Search results Show results as topic list.
PDG/TOPs » Work Item Indices
- kenxu
- 544 posts
- Offline
We did talk about this internally - we're not seeing anything obvious that would account for that - waiting for your files to see what's actually happening.
PDG/TOPs » Collecting TOPs
- kenxu
- 544 posts
- Offline
I'm a bit confused I'm afraid. We can already TopFetch and RopFetch from the TopSop. Are you talking about the need to have some geometry import nodes from sops to forms of output that a topnet could produce? Even if it's that, there are already csv importers etc in SOPs. Maybe elaborate a bit on the workflow you're expecting?
PDG/TOPs » Unreal Engine not supported?
- kenxu
- 544 posts
- Offline
I'm afraid not. We are quite busy with everything we have to do, I honestly can't see us getting to it in the next quarter at the very least.
PDG/TOPs » Why we don't have a dedicated TOP context?
- kenxu
- 544 posts
- Offline
That choice was made under time pressure and while it is a sensible choice for v1.0 - I would not read too much into it at this point. Putting a TOP context at the root level probably makes sense, but beyond that we're open to suggestions.
Edited by kenxu - April 29, 2019 10:50:11
PDG/TOPs » Collecting TOPs
- kenxu
- 544 posts
- Offline
We do think it makes sense to add a TOP network at the root level, like all the other contexts, but beyond that I'm not sure what more we could do. Things like the TOPSOP by definition goes where it's needed in other networks. We're open to suggestions on how best to handle this though.
PDG/TOPs » Offline Job Visualization
- kenxu
- 544 posts
- Offline
We're working on this feature - it's the major addition we are planning on. The end goal of this push would be the ability to attach a session of PilotPDG or Houdini to any running session of PDG and be able to visualize its state. Constructs will also be put in place to allow state changes in PDG to be broadcast and/or recorded.
PDG/TOPs » PDG/TOPs grounded - Open letter for SideFX
- kenxu
- 544 posts
- Offline
Yes, a few days ago a couple of us were lamenting the fact that if the CSV nodes were someone's first impression of PDG, it wasn't going to go well. With better testing, we should have identified those specific nodes and held them back or labeled as such, as you suggest. We've already gotten much better with them, and we'll make a posting once we feel we've reached the next level with these.
Edited by kenxu - April 25, 2019 09:50:58
PDG/TOPs » how to get an overview of threads per task?
- kenxu
- 544 posts
- Offline
It's on the roadmap but not soon I'm afraid. Still lots of higher priority things to address. If however folks feel this should be really up there more, please up vote.
PDG/TOPs » PDG/TOPs grounded - Open letter for SideFX
- kenxu
- 544 posts
- Offline
Thank you for your understanding Greg. I agree with your point about messaging, but “Houdini 17.5 with PDG Preview” doesn't reflect the true state of the product either. If the whole thing was really in that state, I'd have a few more grey hairs and wrinkles added to my face by this point .
My honest assessment of the situation is this - the core of the PDG graph/tech is rock solid, the wedging/FX workflows are in good shape and we're seeing good results here, schedulers - local scheduler is solid, HQueue works well but has usability issues in setup (not a new thing), particularly with mounts / UNC pathing issues. Tractor/Deadline each has some polish issues but mostly solved now, but with the notable problem on Tractor with the way it creates too many jobs that we're still working on (farm setup unfortunately is always an advanced topic and never fun). The peripheral nodes - all the partitioners are solid, the mappers are for advanced users only at this point, imagemagick and ffmpeg are in good shape, but JSON was missing key features and CSV nodes were, frankly, not quite there. That said, we have already fixed many of the most serious problems, and with continued focus I think we can close the remaining gaps soon.
A large part of the problem is with testing - try as we might, our resources are limited internally and so are the breadth of use cases that we can cover, especially for a project of this scope, which is for Houdini but which is also aimed so much at areas outside of Houdini. During alpha/beta, most our testers are from super big shops - and they are super technical and mostly write their own stuff anyway. So they'll tend to push the framework/core of the tech to the limit, but then will tend to leave the more “out of the box” nodes less tested. We had some other unforseeen issues outside of our control coming into release as well, like losing key team members at a really bad time, that made the last stretch of this thing “fun”. With 20/20 hind sight, the most optimal thing we could have done is to hold a few of these peripheral nodes like CSVs back and add them in a later production build in a more polished state, or label those particular nodes as “beta”. Alas, we aren't perfect, especially me , and that's not what we did. Thanks though for your patience and understanding, and for giving us feedback and helping to track down the issues. We'll keep at it - there are definitely some more wrinkles to iron out, but we're not too far from getting the most serious ones tucked away.
In terms of TOP VOP, GPU etc, yeah there is a ton of forward development on PDG. But as you said, we have to make sure what is there is well polished first.
My honest assessment of the situation is this - the core of the PDG graph/tech is rock solid, the wedging/FX workflows are in good shape and we're seeing good results here, schedulers - local scheduler is solid, HQueue works well but has usability issues in setup (not a new thing), particularly with mounts / UNC pathing issues. Tractor/Deadline each has some polish issues but mostly solved now, but with the notable problem on Tractor with the way it creates too many jobs that we're still working on (farm setup unfortunately is always an advanced topic and never fun). The peripheral nodes - all the partitioners are solid, the mappers are for advanced users only at this point, imagemagick and ffmpeg are in good shape, but JSON was missing key features and CSV nodes were, frankly, not quite there. That said, we have already fixed many of the most serious problems, and with continued focus I think we can close the remaining gaps soon.
A large part of the problem is with testing - try as we might, our resources are limited internally and so are the breadth of use cases that we can cover, especially for a project of this scope, which is for Houdini but which is also aimed so much at areas outside of Houdini. During alpha/beta, most our testers are from super big shops - and they are super technical and mostly write their own stuff anyway. So they'll tend to push the framework/core of the tech to the limit, but then will tend to leave the more “out of the box” nodes less tested. We had some other unforseeen issues outside of our control coming into release as well, like losing key team members at a really bad time, that made the last stretch of this thing “fun”. With 20/20 hind sight, the most optimal thing we could have done is to hold a few of these peripheral nodes like CSVs back and add them in a later production build in a more polished state, or label those particular nodes as “beta”. Alas, we aren't perfect, especially me , and that's not what we did. Thanks though for your patience and understanding, and for giving us feedback and helping to track down the issues. We'll keep at it - there are definitely some more wrinkles to iron out, but we're not too far from getting the most serious ones tucked away.
In terms of TOP VOP, GPU etc, yeah there is a ton of forward development on PDG. But as you said, we have to make sure what is there is well polished first.
Edited by kenxu - April 25, 2019 09:34:46
PDG/TOPs » Work Item Indices
- kenxu
- 544 posts
- Offline
there are some general issues with the JSON/CSV nodes that kept popping up for different use cases, so having those ironed out while TOPs is still in early development would be very nice. Also, for standalone PDG, I expect users will start to pipe in a lot of different kinds of data, so having a basic set of flexible data retrieval nodes would likely become even more important.
Could not agree more. It's a repeated exercise at this point of looking at use cases that we may still not be addressing well, ironing out the wrinkles there, rinse and repeat. If you're open to it, we'd be up for a periodic call to see where the remaining issues are for you and see what we could do about it. Please message me in case you're interested.
PDG/TOPs » Work Item Indices
- kenxu
- 544 posts
- Offline
Hi gyltefors, firstly we acknowledge that some of these peripheral nodes have not received quite the level of production testing as they need, as we were more focused on the core FX workflows that would impact many more people out of the gate. However, we are working to close the gap as quickly as possible. For the first production build, both the CSV nodes and JSON nodes have received a significant amount of attention. If you take a look at the change log for csv or json, you'll see that a significant number of issues have already been resolved:
https://www.sidefx.com/changelog/?journal=17.5&categories=54&body=&version=17.5&build_0=173&build_1=234&show_versions=on&show_compatibility=on&items_per_page= [www.sidefx.com]
While that is part of the problem, the flip side of the coin is that your specific use case is not at all a simple one . WRT the JSON node, we have taken a detailed look at your use case. Part of the issue there at least is that the json file in that case is a hierarchy that is being flattened. Entries are heterogeneous, with ids that point to each other to reconstruct the hierarchy. Even if one were to write code to solve the problem, it would not be trivial. That said, we are doing all we can to help. In addition to the hierarchical array retrieval we have already added, we are planning to add:
1) The resolve path. So if you make a query like “carparks/*/address”, we'll attach for each workitem a resolved path, so that it will read carparks/1/address and carparks/2/address etc. This will help you put things back together.
2) Supporting sub-trees for queries, where the result of the query is itself a json blob representing a sub-portion of the original json. This would allow you to hierarchically pick apart a json file.
So these would fall more into the RFE rather than the BUG bucket, and should help you get further. However, we also recommend that the form of the data be restructured on your end to make it a little easier to digest. Finally, WRT to CSV, one of the issues that was blocking you - that the table sop was not supporting csv files written by csvoutput, has been solved and made the production build.
If there are any further specific problems we can help you with, please let us know.
https://www.sidefx.com/changelog/?journal=17.5&categories=54&body=&version=17.5&build_0=173&build_1=234&show_versions=on&show_compatibility=on&items_per_page= [www.sidefx.com]
While that is part of the problem, the flip side of the coin is that your specific use case is not at all a simple one . WRT the JSON node, we have taken a detailed look at your use case. Part of the issue there at least is that the json file in that case is a hierarchy that is being flattened. Entries are heterogeneous, with ids that point to each other to reconstruct the hierarchy. Even if one were to write code to solve the problem, it would not be trivial. That said, we are doing all we can to help. In addition to the hierarchical array retrieval we have already added, we are planning to add:
1) The resolve path. So if you make a query like “carparks/*/address”, we'll attach for each workitem a resolved path, so that it will read carparks/1/address and carparks/2/address etc. This will help you put things back together.
2) Supporting sub-trees for queries, where the result of the query is itself a json blob representing a sub-portion of the original json. This would allow you to hierarchically pick apart a json file.
So these would fall more into the RFE rather than the BUG bucket, and should help you get further. However, we also recommend that the form of the data be restructured on your end to make it a little easier to digest. Finally, WRT to CSV, one of the issues that was blocking you - that the table sop was not supporting csv files written by csvoutput, has been solved and made the production build.
If there are any further specific problems we can help you with, please let us know.
Edited by kenxu - April 22, 2019 17:23:32
PDG/TOPs » Unreal Engine not supported?
- kenxu
- 544 posts
- Offline
We will address the PDG link to the UE4 plugin after we address some of the basic (and urgent) issues around the UE4 plugin first.
PDG/TOPs » PDG feature proposal
- kenxu
- 544 posts
- Offline
Thanks all for the great suggestions. Indeed there was talk earlier in the alpha / beta periods of PDG to eventually develop a side scrolling view like the one described. It's a bigger thing though, so we'll definitely keep this mind as we look to our future roadmap.
PDG/TOPs » DOCS: RFE: special attributes and attribute access in all TOP node help cards
- kenxu
- 544 posts
- Offline
PDG/TOPs » Scrolling frame range with TOP in SOP context is laggy.
- kenxu
- 544 posts
- Offline
In the context of SOPs the TOPNet in that context is specialized to be able to be inlined with SOPs - you'll notice the input and output on it that is not on other “pure” TOPNets. We used to call it the TOPSOP, but some have felt it more consistent to name it just Top Network. So it does have some specialization to work with SOPs, and we're putting some more work on this node. The above mentioned is some of that.
PDG/TOPs » PDG/TOPs grounded - Open letter for SideFX
- kenxu
- 544 posts
- Offline
Hi Greg,
Thank you for your honest feedback, and we are sorry your first experience with PDG is not as smooth as we would like. Ahead of release, we were most focused on the FX and wedging workflows - we realize there are some features especially around the periphery of PDG that are in need of some love and polish, and we are hard at work closing the gap around those. PDG is a project of enormous scope - the use cases are so varied that it was difficult to get the full range of feedback ahead of release. This is not to make excuses, we acknowledge our shortcomings here and we will fix them. Please help us concentrate our efforts on the area(s) causing the most pain. You have identified the CSV nodes and TOPSOP - those are known to us - we're going to make significant modifications and improvements to those, and will roll them out in a future production build. If there are other specific areas causing pain, please point them out. We'll post updates on the CSV stuff as we have them.
Specifically around the TOPSOP - can you supply us with a reproducible case for the crash? That would help us a lot to track down the problem there.
Thank you for your honest feedback, and we are sorry your first experience with PDG is not as smooth as we would like. Ahead of release, we were most focused on the FX and wedging workflows - we realize there are some features especially around the periphery of PDG that are in need of some love and polish, and we are hard at work closing the gap around those. PDG is a project of enormous scope - the use cases are so varied that it was difficult to get the full range of feedback ahead of release. This is not to make excuses, we acknowledge our shortcomings here and we will fix them. Please help us concentrate our efforts on the area(s) causing the most pain. You have identified the CSV nodes and TOPSOP - those are known to us - we're going to make significant modifications and improvements to those, and will roll them out in a future production build. If there are other specific areas causing pain, please point them out. We'll post updates on the CSV stuff as we have them.
Specifically around the TOPSOP - can you supply us with a reproducible case for the crash? That would help us a lot to track down the problem there.
Edited by kenxu - April 10, 2019 10:37:07
PDG/TOPs » PDG feature proposal
- kenxu
- 544 posts
- Offline
Thanks for the suggestions Nick! What would be the advantage of having a LR graph over a TB one in your view?
PDG/TOPs » Is there a way to pick up a cooked result of PDG?
- kenxu
- 544 posts
- Offline
So let's say you have an output string parameter, and some attribute like fooAttr you can put in something like:
path_to_my_file/my_file_`@fooAttr`_rest_of_filename.bgeo
With that expression, PDG can predict what the file on disk should be, and if it finds the file there, then the instant cooking will work. By contrast, if you exported fooAttr to an environment variable $FOOATTR, then this will work when outputting the files, but NOT work for the instant cook feature:
path_to_my_file/my_file_$FOOATTR_rest_of_filename.bgeo
path_to_my_file/my_file_`@fooAttr`_rest_of_filename.bgeo
With that expression, PDG can predict what the file on disk should be, and if it finds the file there, then the instant cooking will work. By contrast, if you exported fooAttr to an environment variable $FOOATTR, then this will work when outputting the files, but NOT work for the instant cook feature:
path_to_my_file/my_file_$FOOATTR_rest_of_filename.bgeo
PDG/TOPs » Context View suggestion for PDG
- kenxu
- 544 posts
- Offline
Agreed that whatever preview scheme we come up with should be holistic and apply potentially to Houdini as well.
PDG/TOPs » Why we don't have a dedicated TOP context?
- kenxu
- 544 posts
- Offline
Because we ran out of time before the release
Perhaps a better answer is that the top network has all kinds of options on it. We have to have sensible defaults and/or ways for you to control those. Having this context is convenient, but it doesn’t allow you to do something you can’t now do, which is also why it got dropped before the release.
It’s coming though
Perhaps a better answer is that the top network has all kinds of options on it. We have to have sensible defaults and/or ways for you to control those. Having this context is convenient, but it doesn’t allow you to do something you can’t now do, which is also why it got dropped before the release.
It’s coming though
-
- Quick Links