Running a set number of steps in parallel at a time #3296

amn · 10 years ago

Typically, we either run steps in sequence or parallel across a set of hosts. If we have multiple hosts, and instead want the steps to run against a defined number of hosts at a time, is there a good way to do this?

The default would be run the steps against 1 host at a time, but that could be changed to run 2, 3, etc... hosts in parallel at a time until the steps have been run against all hosts.

replies 3
views 2408
stars 0

robinshen ADMIN · 10 years ago

I am not quite understand the question. Would you please elaborate the usage scenario?

amn · 10 years ago

I have 8 hosts that the same steps will be run on. Instead of running them all sequentially, I would like to be able to designate running them in groups in parallel.

repeat parameters: host1, host2, host3, host4, host5, host6, host7, host8

run in groups of 2

steps run on host1 and host2 in parallel
-then-
steps run on host3 and host4 in parallel
-then-
steps run on host5 and host6 in parallel
-then-
steps run on host7 and host8 in parallel

We want to not have to run one the steps against one host at a time, but also don't want them all to be run in parallel together. Being able to choose how many run together at once (in the example above 2), would be ideal.

robinshen ADMIN · 10 years ago

Thanks for the clarification. To do this, you may define a parallel container step "group", and define its repeat parameter as:
param name: group
param value: host1+host2, host3+host4, host5+host6, host7+host8

Then inside this parallel container step, define the build step you want to execute on host and define its repeat param as:
param name: host
param value: ${params.get("group").replace('+', ',')}

Then define node selection setting of your build step to use the host param:
on node with specifed script evaluating to true:
node.hostName==params.get("host")

Every time you want to adjust the group, just change the group param above.