Search - User list
Full Version: Tractor Scheduler floods farm with separate jobs per work item
Root » PDG/TOPs » Tractor Scheduler floods farm with separate jobs per work item
vochsel
Hi there,

We've begun using PDG/TOP Nets for some tractor scheduling and have run into an issue where submitting work items via the built in tractor scheduler causes our tractor to be flooded with separate jobs when run.

Is there a reason that each PDG work item gets submitted as its own job? It would seem to make more sense to group all work items under the one job as tasks. Perhaps there is PDG concept I'm not familiar with yet. Just wondering if there was anyone else experiencing this.

Thanks,
Ben
seelan
Yes, we are aware of this, and have a pending RFE to fix it.
vochsel
Awesome, great to hear.

Thanks so much seelan.
mestela
We're finding a few other unexpected behaviours with pdg and tractor, would you rather them mentioned here, or in a support email, or both?
seelan
If they are similar or related issues, then you can post here. Otherwise support email might be best.
protozoan
Has there been any progress made on this front?
(I pulled the plug on PDG on Tractor during the alpha because of the job flooding, but since there's a major hard- and software overhaul here I would like to bring it back asap).
chrisgreb
No progress yet, but it's still on the radar. Some background:

The problem with fixing this is that Tractor doesn't allow child tasks to be attached to running parent jobs through the API, instead the parent job must write new task specs to it's stdout. This implies that the PDG scheduler itself or some proxy process must be running on the farm for the duration of the cook so that PDG's dynamic workitem generation will work. This is a fairly major change.
protozoan
chrisgreb
This is a fairly major change.

Thanks for the info, we'll keep an eye out for it and as soon as you put something in a build, we shall test it.
louis dunlevy
Any update on this Tractor issue?

thanks guys
L
chrisgreb
louis dunlevy
Any update on this Tractor issue?

thanks guys
L
No, it's on the top of the list, but probably still a few weeks off.
GraemePhillips
On the TOP of the list… I see what you did there. Looking forward to having it all working together harmoniously.
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Powered by DjangoBB