Hi,

Is it possible to deserialize a full graph?
For example, I have a python serialized file but I don't have the original hip file and I would like to deserialize (recreate) all scene nodes.

Thanks
Ostap

No, you can create a TOP graph procedurally with HOM, and you can create a PDG graph procedurally with the the PDG python API. However there's no way currently to dump a PDG graph to disk from a Houdini session.

you create a TOP graph procedurally with HOM

But is it make sense if I can't create dump of my TOP graph?

Ostap

you create a TOP graph procedurally with HOM

But is it make sense if I can't create dump of my TOP graph?

The TOP ‘dump’ is the hip file. It might make sense to create a TOP graph with HOM if it was part of some higher level automated system.

What's the actual workflow you are looking for? Maybe the `hou.hipFile.merge` function would be useful?

I'm looking for something much more lightweight for restoring PDG session. For example for farm wranglers purpose.
The opening hip file is also the option but it is an additional step and time to review a scene.

Ostap
For example for farm wranglers purpose.

It sounds like you mean building a custom UI to view the node graph instead of using Houdini / PilotPDG ?

One thing to note is that the PDG graph is not a 1-1 mapping to the TOP graph. For example there are no subnets, switches or nulls in PDG, in addition without the hip file all the graph layout and node parm formatting is lost.

It sounds like you mean building a custom UI to view the node graph instead of using Houdini / PilotPDG ?

Yes, but also understanding the accessibility of tools for today.

In the beginning, I thought that PilotPDG will satisfy all our requirements (provide full information about PDG graph and be light enough) but looks like the relation between the original Houdini is really tight.
As we already talking about that, can you please share any roadmap/future improvements regarding that?

Could you elaborate on why PilotPDG is not light enough? Are you talking about startup speed or licensing or UX ?

We don't have any current plans regarding this, but the startup speed of PilotPDG has come up before as a pain point.

I mean startup speed and licensing but you already replied above.

Thanks

To continue this conversation.
How do you see:
Workitem dependencies visualization if the third person (wrangler) is looking at the farm job of interactive PDG submission?
Workitem dependencies visualization if the graph has been submitted as a job (with Data Server) and the job is not alive anymore (completed, killed ..)?

Is it possible to start automatically Start Server for any PDG session?
Maybe dump Start Server data to some file could be an option to visualize what is going even after the job is done?

Ostap
Workitem dependencies visualization if the third person (wrangler) is looking at the farm job of interactive PDG submission?

The data layer server can serve multiple clients. They will just need to connect from PilotPDG or some custom client.

It should be possible to attach a 3rd
Workitem dependencies visualization if the graph has been submitted as a job (with Data Server) and the job is not alive anymore (completed, killed ..)?

At the end of the cook the script serializes the task graph state to a python file to disk. This could be used to load the state of the graph as it was when the cook ended.

Is it possible to start automatically Start Server for any PDG session?

Yes, it can be started through the pdgd API.

Yes, it can be started through the pdgd API.

Scheduler creates a state server - do you think it is the right way? Maybe better to use some Event Handler for this purpose if it's possible?

At the end of the cook the script serializes the task graph state to a python file to disk. This could be used to load the state of the graph as it was when the cook ended.

That what we are doing right now but it is really slow. To deserialize a PDG graph you have to open the hip file first and sometimes the hip file could be huge. It can take minutes to load all those things.
Also, it is not convenient to change a visualisation workflow which depends on a job state (running, done or failed).
Any idea for a better/faster solution?

Ostap
Scheduler creates a state server - do you think it is the right way? Maybe better to use some Event Handler for this purpose if it's possible?

I think it would depend on the particular workflow you have in mind. Currently we only create a state server optionally based on the scheduler toggle parm.

…really slow. To deserialize a PDG graph you have to open the hip file first and sometimes the hip file could be huge. It can take minutes to load all those things.

There's two ways this could be avoided:
1. Keep hython running when the TOP cook finishes, therefore the state server will be available until the process is explicitly killed.
2. Extend the default state server to use a persistent back-end which can be used to bootstrap a remotegraph TOP in PilotPDG. I am guessing that this would be faster because you don't need to load a hip file.

Also, it is not convenient to change a visualisation workflow which depends on a job state (running, done or failed).
Any idea for a better/faster solution?

#2 could be a single workflow if there was some kind of central server to connect the client PilotPDG to the correct back end data. However, we haven't implemented a persistent back end, so it's theoretical at this point

In case of option #1 we will full up our render machines really fast : )

Option #2 sounds cool!
Is it possible to dump data to the file from a PDG State Server? So if we can't find an alive server then we fall back to predefined directory with dump. Also would be nice to have the option of dump verbosity to prevent really huge files (huge files as it is with Serialized Graph right now).

Ostap
Option #2 sounds cool!

Ok, if this is something you're interested in, please submit an RFE.

Already did it.
Thank you.