I'm trying to find existed partitioner where you can make dependencies around a primary node and all other workitems (like a zip function). Do you have some partition with similar behavior?
Example hip with custom partitioner in the attachment or just python snippet.
node_map = {} for work_item in work_items: if not work_item.node.name in node_map: node_map[work_item.node.name] = [] node_map[work_item.node.name].append(work_item) # genericgenerator1 - could be defined as a primary input # genericgenerator# - all other nodes for work_item in node_map['genericgenerator1']: partition_holder.addItemToPartition(work_item, work_item.index) for i in node_map['genericgenerator2']: partition_holder.addItemToPartition(i, work_item.index)
Thanks
Ostap