Your browser was unable to load all of the resources. They may have been blocked by your firewall, proxy or browser configuration.
Press Ctrl+F5 or Ctrl+Shift+R to have your browser try again.

cannot modify the same variable in parallel steps [QB 8.0.0, 2018-02-26] #3999

OnQuickBuild ·

Hello Robin

I've encountered a problem, where I try to append certain information from parallel steps into one variable, so that at the end I can use one variable to display all that I collected.

I have the step structure as follows:

[parallel 1] [parallel 2] [parallel 3]
sequential sequential sequential


Three parallel groups run on three different agents.

Inside the sequential steps, they consist of pretty much the same set of steps (that kind that you modify one of them and it will reflect on all three sequential steps)

And one of the steps in sequential is publish JUnit report, in "post action scripts" I have:

groovy:
// append failed names to Email body if there are failures:
if (current.errors + current.failures > 0) {
  def failed_pkg = params.get('release_name')
  logger.warn(' ********* add ' + failed_pkg + ' to Email body **********')

  if (vars.getValue('test_failed_packages') == 'null') {
    logger.warn('variable is still null')
    vars.get('test_failed_packages').setValue(failed_pkg)
  }
  else
    vars.get('test_failed_packages').setValue(vars.getValue('test_failed_packages') + '\n' + failed_pkg)
}

As you can see I added a line to output if the variable is still null (I set the variable to null in the beginning of the build)
And where there are failures in all three agents, the log will all say "variable is still null". And the final value will be overridden by the last parallel that executed the step.

I wonder if I used the wrong way to achieve what I intended. How could you modify the same variables and save the values on the go?

Thank you!!

  • solved #8
  • replies 7
  • views 77
  • stars 0
robinshen ADMIN ·

Updating same variable concurrently will not work. You may update different variables, and then collect result from different variables after all concurrent steps finished.

OnQuickBuild ·

Thanks for your reply.

Unfortunately I can't use this scheme, because the number of parallel "chunks" is determined by the number of certain resources:
the repeat parameter is using "node addresses within certain criteria", and the node selection is corresponding to "node with selected resources".

And I don't know how to create dynamic variables for the entire build, as I remembered each step is executed within separate "loading" classes through Groovy.

How else can I do this?

Thanks!

OnQuickBuild ·

Hi Robin

Could you help with another question?

When using the step "checkout" in a workspace that doesn't clean workspace before execution, what does the checkout step do, if I rebuild the same revision?
For example, we use Subversion. Does it perform a "svn update"?

The reason I ask is that there are external links to other repo, and if the external link has updates, it doesn't effect repo's revision. But in my local laptop, if I just do a "svn update" it will just pick up the external changes.

Thank you!

robinshen ADMIN ·

Thanks for your reply.

Unfortunately I can't use this scheme, because the number of parallel "chunks" is determined by the number of certain resources:
the repeat parameter is using "node addresses within certain criteria", and the node selection is corresponding to "node with selected resources".

And I don't know how to create dynamic variables for the entire build, as I remembered each step is executed within separate "loading" classes through Groovy.

How else can I do this?

Thanks!

Or you may store the result into build.reports like below

build.reports["test_failed_packages." + step.path.toString()] = failed_pkg;

And then after all concurrent step finished, iterate over build.reports (it is a synchronized hash map used internally by QB) to find out all values with key starting with "test_failed_packages" and then aggregate them.

robinshen ADMIN ·

Hi Robin

Could you help with another question?

When using the step "checkout" in a workspace that doesn't clean workspace before execution, what does the checkout step do, if I rebuild the same revision?
For example, we use Subversion. Does it perform a "svn update"?

The reason I ask is that there are external links to other repo, and if the external link has updates, it doesn't effect repo's revision. But in my local laptop, if I just do a "svn update" it will just pick up the external changes.

Thank you!

It simply runs another "svn checkout" using the same revision, and external links will not be updated. This makes sure all sources are not changed for a "rebuild".

OnQuickBuild ·

Thank you for your answer.

But I have something to disagree on. (I'm guessing svn checkout at the same revision is equivalent to svn update)

Let's say I have the repo to be built: main
And "main" has external links to the head revision of "dependency"

So when "dependency" changed, "main" won't get triggered since there's no direct changes/commits on "main".

Then in "main" QB config, I added a separate repo specifically for the external to "dependency", and used a repo step to "record changes".
This case "main" will be triggered because the repo it referenced to has updates.

Then I compared the newest file from "dependency" svn source, with the same file from "main" workspace, they are the same.

Anyway I got around to it. Probably a svn checkout on existing working copy is the same as svn update.

Thanks!

robinshen ADMIN ·

I was thinking you are actually "rebuilding the same revision" which is the case when you are doing promotion or specifying the revision explicitly. In case a checkout using latest version (even if workspace is not cleaned up) without revision being specified, it will grab the head revision including any updates to externals. However it is a limitation that changes in externals can not trigger QB build automatically (QB can not get external changes by running subversion log command). What you did now is the way to do: add the dependency repository simply for sake of detecting changes and trigger the build.

PS: According to my test, subversion checkout has the same effect as update if operate on an existing repository.