It will be really helpful to expose this functionality.
Thanks
Ostap
Found 180 posts.
Search results Show results as topic list.
PDG/TOPs » Workitem inProcess one thread
- Ostap
- 209 posts
- Offline
PDG/TOPs » Workitem inProcess one thread
- Ostap
- 209 posts
- Offline
Is it possible to implement "Run on Main Thread" option in a custom scheduler for "inProccess" workitems?
Do you have publicly available source for inprocess scheduler?
Thanks
Do you have publicly available source for inprocess scheduler?
Thanks
Edited by Ostap - Jan. 7, 2021 18:26:43
PDG/TOPs » Workitem inProcess one thread
- Ostap
- 209 posts
- Offline
Is it possible somehow to cook "inProccess" in the same thread (or maybe reuse the main one)?
What we are trying to solve:
We have not thread-safe call and probably make workitem dep. (in the same row) is the first think which is coming to mind but it can transform into a limitation because we don't need a predefined order. We just need a random serial execution.
What we are trying to solve:
We have not thread-safe call and probably make workitem dep. (in the same row) is the first think which is coming to mind but it can transform into a limitation because we don't need a predefined order. We just need a random serial execution.
PDG/TOPs » Deserialize a full graph
- Ostap
- 209 posts
- Offline
PDG/TOPs » Deserialize a full graph
- Ostap
- 209 posts
- Offline
In case of option #1 we will full up our render machines really fast : )
Option #2 sounds cool!
Is it possible to dump data to the file from a PDG State Server? So if we can't find an alive server then we fall back to predefined directory with dump. Also would be nice to have the option of dump verbosity to prevent really huge files (huge files as it is with Serialized Graph right now).
Option #2 sounds cool!
Is it possible to dump data to the file from a PDG State Server? So if we can't find an alive server then we fall back to predefined directory with dump. Also would be nice to have the option of dump verbosity to prevent really huge files (huge files as it is with Serialized Graph right now).
PDG/TOPs » Deserialize a full graph
- Ostap
- 209 posts
- Offline
Yes, it can be started through the pdgd API.
Scheduler creates a state server - do you think it is the right way? Maybe better to use some Event Handler for this purpose if it's possible?
At the end of the cook the script serializes the task graph state to a python file to disk. This could be used to load the state of the graph as it was when the cook ended.
That what we are doing right now but it is really slow. To deserialize a PDG graph you have to open the hip file first and sometimes the hip file could be huge. It can take minutes to load all those things.
Also, it is not convenient to change a visualisation workflow which depends on a job state (running, done or failed).
Any idea for a better/faster solution?
PDG/TOPs » Deserialize a full graph
- Ostap
- 209 posts
- Offline
Is it possible to start automatically Start Server for any PDG session?
Maybe dump Start Server data to some file could be an option to visualize what is going even after the job is done?
Maybe dump Start Server data to some file could be an option to visualize what is going even after the job is done?
PDG/TOPs » Deserialize a full graph
- Ostap
- 209 posts
- Offline
To continue this conversation.
How do you see:
Workitem dependencies visualization if the third person (wrangler) is looking at the farm job of interactive PDG submission?
Workitem dependencies visualization if the graph has been submitted as a job (with Data Server) and the job is not alive anymore (completed, killed ..)?
How do you see:
Workitem dependencies visualization if the third person (wrangler) is looking at the farm job of interactive PDG submission?
Workitem dependencies visualization if the graph has been submitted as a job (with Data Server) and the job is not alive anymore (completed, killed ..)?
PDG/TOPs » Overload "Delete Results From Disk"
- Ostap
- 209 posts
- Offline
PDG/TOPs » Overload "Delete Results From Disk"
- Ostap
- 209 posts
- Offline
PDG/TOPs » Overload "Delete Results From Disk"
- Ostap
- 209 posts
- Offline
Is it possible to prevent deletion from your side?
We would like to use our own methods to delete files.
Maybe something like that:
We would like to use our own methods to delete files.
Maybe something like that:
type_registry.registerDirtyHandler( simple_handler, handler_type=pdg.dirtyHandlerType.Files, skip_deletion=True)
PDG/TOPs » Cook only failed tasks
- Ostap
- 209 posts
- Offline
Is there any way to re-cook only failed tasks?
If the result is a file on the disk then reasonable to use a caching mechanism (Automatic Mode) but how to re-cook failed tasks if the result is not a file?
If the result is a file on the disk then reasonable to use a caching mechanism (Automatic Mode) but how to re-cook failed tasks if the result is not a file?
PDG/TOPs » Overload "Delete Results From Disk"
- Ostap
- 209 posts
- Offline
PDG/TOPs » Overload "Delete Results From Disk"
- Ostap
- 209 posts
- Offline
PDG/TOPs » Overload "Delete Results From Disk"
- Ostap
- 209 posts
- Offline
It is a pretty important improvement for us.
Is there any chance to get some time frame when it could be done?
Is there any chance to get some time frame when it could be done?
PDG/TOPs » Partition Split by Attribute: Improvement
- Ostap
- 209 posts
- Offline
That is amazing! Really useful and it works as expected.
Thank you!
Maybe in this implementation parm should be renamed into “Split by AttributeS”?
Thank you!
Maybe in this implementation parm should be renamed into “Split by AttributeS”?
PDG/TOPs » Overload "Delete Results From Disk"
- Ostap
- 209 posts
- Offline
Hi,
Is it possible to overload/make callback for “Delete Results From Disk” call from workitem context menu?
For some cases, it is not so easy just delete files from the disk if we are using wrappers or need an additional callback.
Thanks
Ostap
Is it possible to overload/make callback for “Delete Results From Disk” call from workitem context menu?
For some cases, it is not so easy just delete files from the disk if we are using wrappers or need an additional callback.
Thanks
Ostap
PDG/TOPs » Partition Split by Attribute: Improvement
- Ostap
- 209 posts
- Offline
PDG/TOPs » Dynamic to Static
- Ostap
- 209 posts
- Offline
It will be useful!
Thanks!
because right it is still not possible to wait for an attribute or result data from the dynamic workitem (as I understand from your example) if I need static in the end
Thanks!
because right it is still not possible to wait for an attribute or result data from the dynamic workitem (as I understand from your example) if I need static in the end
Edited by Ostap - Aug. 11, 2020 03:05:22
PDG/TOPs » Dynamic to Static
- Ostap
- 209 posts
- Offline
Is there any way to convert workitem execution from dynamic to static state?
I'm using a static generation than dynamic and I would like to convert execution to static back again.
I'm using a static generation than dynamic and I would like to convert execution to static back again.
-
- Quick Links