Let's say I have two nodes, one with 5 work items (A B C D E) and another with 3 work items (A B C), how do I partition the work items of my first node with random indexes of work items from the second node? I can do a Partition by Attribute, but how would I find the maximum number of work items in the 2nd node from within the first node's pipe?

I think you can get a similar randomization pattern with Partition by Combination.

alexwheezy
I think you can get a similar randomization pattern with Partition by Combination.

Hmmm I tried that, however it does not always match partitions between the two nodes. Also the partitioning is very linear, and not random.

Partition by comparison can do this, a bit cumbersome to setup if you don't know the work item counts up front. But seems to work well enough

tyler.britton

alexwheezy
I think you can get a similar randomization pattern with Partition by Combination.

Hmmm I tried that, however it does not always match partitions between the two nodes. Also the partitioning is very linear, and not random.

You could try using a Python Partitioner with a custom index that will have a random value for other items.