DjangoBB LoFi version

Full Version: Ability to set a PDG node as cooked via python.

Root » PDG/TOPs » Ability to set a PDG node as cooked via python.

madjestic

June 26, 2019 11:07:54

Hey SESI,

Is there a way to use python to set some (or current) TOP node as cooked?

Thanks,
Vlad

kenxu

June 26, 2019 17:14:31

You can trigger a cook with the cook method on pdg.GraphContext or the cook method on pdg.Node. However, I don't think you can just change the state through the API without actually cooking it…why do you need this?

madjestic

June 27, 2019 04:41:44

I have a use-case when I start a 3rd party software process on the farm (/foo/bar/baz.EXE), using Pyton TOP, which produces the right output and quits, but, apparently, does not terminate gracefully, so the Python TOP stays forever in “cooking” state. We created a hacky workaround it by checking for a specific file (END.txt) script and terminating the process:

process = subprocess.Popen(command)
while not os.path.isfile(outPath + '\END.txt'):
    time.sleep(5)
process.terminate()

I understand that the process should terminate gracefully and hence there should be no need for the above, however we failed to trace the problem. It may be due to our farm configurations, file/processes permission issues, python configuration or a python or PDG bug… so we sort of gave up for now and use this workaround.

Cheers,
Vlad

chrisgreb

June 28, 2019 08:52:15

Which scheduler are you using?

It sounds like you're saying the process baz.EXE never terminates, but ends up hanging for some unknown reason.

If that's the case, I think what you're doing is the best workaround - if the process won't terminate then naturally some other process has to kill it. There is an RFE to have work item timeouts, which would kill a hanging job after awhile, but in that case they would be marked as failed, which is not what you want.