Q: More granular control of work item assignment in Deadline

   2963   3   2
User Avatar
Member
7025 posts
Joined: 7月 2005
Offline
Hi,

Am I missing more granular control of task/work item assignment in Deadline?

Simple example, I want to assign certain work items (tasks in DL) to a specific Deadline group, let's say because it has a lot of RAM, and other work items to a group that is high-CPU.

I see Job File key-values and Plugin File Key-Values, though I don't yet 100% understand how those are consumed. Are there attributes I can add to work items that will get ultimately interpreted by Deadline?

I've read through the PDGDeadline.py plugin with maybe 60% understanding (at the moment) fwiw

Still learning Deadline too so happy to be nudged at looking at something in Deadline docs.

18.5.418 Win 10 Py 3

Cheers,

Peter B
User Avatar
Member
603 posts
Joined: 9月 2016
Offline
We've found Deadline is a bit less flexible than other farm schedulers for things like this, so the UX is a bit convoluted. The pool and other such attributes live only at the Job level, so we need to create separate jobs for each pool that is used in the cook.

The way to accomplish this is to create a Deadline Scheduler TOP for each pool that you need. For example a 'deadlinescheduler_highmem' in addition to 'deadlinescheduler1'. Then you can use the TOP Scheduler override on the TOP node to do the assignment.
Edited by chrisgreb - 2021年1月21日 09:18:44
User Avatar
Member
7025 posts
Joined: 7月 2005
Offline
Interesting thanks for that pointer Chris. I'll take a stab at it!
User Avatar
Member
7025 posts
Joined: 7月 2005
Offline
This seems to work well, thanks Chris!

For anyone reading this, you may need to look into modifying the Generate When parm to "When All Inputs Cooked" if you want to use concurrent tasks.

Multiple workers seems to work fine regardless.

Cheers,

Peter B
  • Quick Links