Your browser was unable to load all of the resources. They may have been blocked by your firewall, proxy or browser configuration.
Press Ctrl+F5 or Ctrl+Shift+R to have your browser try again.

QuickBuild

run bash script on remote agent #2707

#1
limor ·
Hi
I need to run Shell commands of agent node (ubuntu machine), and get \ set build variables.
In QB 4 there is no option to write bash script, in - steps - misc - execute script - does not support bash.
in steps - build - Shell/Bash command - I tried to run existing script but how can I get and set build variables?
Is there a convenient way to run bash script in QB?
Thanks from advanced
Limor
  • replies 5
  • views 4166
  • stars 0
#2
robinshen ADMIN ·
This can be done by executing the command in a groovy script and then set variable value to use the command output. To do it, create a step of type "Misc/Execute Script" and set script as below:
groovy:

def command = "cmd /c echo hello";
vars.get("var1").setValue(command.execute().text);
logger.info(vars.getValue("var1"));
#3
limor ·
I need to run linux commands, as I wrote I created groovy script that failed to run the commands:

groovy:

def fail_list;
change_repos_list = vars.get("change_repos_list");


for ( repo_dir in change_repos_list )
{
def workingDir = new File(configuration.getWorkspaceDir().getAbsolutePath() +"/"+ repo_dir);
def workingDirBase = new File(configuration.getWorkspaceDir().getAbsolutePath());
logger.info("workingDir:" + workingDir);

def skip_commans=false;

def parentConf = configuration.getParent().getParent().getParent();
logger.info("parentConf=" + parentConf );
def branch = parentConf.getVariables().find {vari -> vari.name == 'branch_name'};
logger.info("branch='" + branch + "'");


if (workingDir.exists())
{
cmd1="virtualenv env";
cmd2="source ../env/bin/activate";
cmd3="python setup.py install";
cmd4="pip install flake8";
cmd5="pip install node";
cmd6="flake8 .";
cmd7="nosetests . --nologcapture --nocapture";
cmd8="echo \$?";
cmd9="rm -rf env"

cmd_array = [cmd1,cmd2,cmd3,cmd4,cmd5,cmd6,cmd7]

i=0;
for ( cmd in cmd_array )
{
i=i+1;
logger.info( "skip_commans: " + skip_commans);

logger.info( "Running command: " + cmd);
if(i==1 || i==9)
{
cmdreturn = cmd.execute(null, workingDirBase);
}
else
{
def cmdreturn = cmd.execute(null, workingDir);
}

cmdreturn.waitFor();
logger.info( "return code: "+cmdreturn.exitValue());
logger.info( "stderr: "+cmdreturn.err.text);
logger.info( "stdout: "+cmdreturn.in.text);
if (i==7 && cmdreturn.exitValue()!=0)
{
if(fail_list == null)
{
fail_list=repo_dir;
}
else
{
repos_list=fail_list+","+repo_dir;
}
vars.get("cosmo_unit_tests_fail").setValue(repos_list);
}
}

}
else
{
logger.info(" "+workingDir + " folder does not exist");
}

}

Errors:

7:49:56,518 [master@build64A:8810] INFO - Executing pre-execute action...
17:49:56,518 [master@build64A:8810] INFO - Running step...
17:49:56,520 [master>cosmo_unit_tests_remote@build64A:8810] INFO - Checking step execute condition...
17:49:56,521 [master>cosmo_unit_tests_remote@build64A:8810] INFO - Step execute condition satisfied, executing...
04:13:56,051 [master>cosmo_unit_tests_remote@ubuntu:8811] INFO - Executing pre-execute action...
04:13:56,052 [master>cosmo_unit_tests_remote@ubuntu:8811] INFO - Running step...
04:13:56,058 [master>cosmo_unit_tests_remote>get_repos_change_list@ubuntu:8811] INFO - Checking step execute condition...
04:13:56,058 [master>cosmo_unit_tests_remote>get_repos_change_list@ubuntu:8811] INFO - Step execute condition satisfied, executing...
04:13:57,077 [master>cosmo_unit_tests_remote>get_repos_change_list@ubuntu:8811] INFO - Executing pre-execute action...
04:13:57,078 [master>cosmo_unit_tests_remote>get_repos_change_list@ubuntu:8811] INFO - Running step...
04:13:57,078 [master>cosmo_unit_tests_remote>get_repos_change_list@ubuntu:8811] INFO - Executing post-execute action...
04:13:57,090 [master>cosmo_unit_tests_remote>cosmo_unit_tests@ubuntu:8811] INFO - Checking step execute condition...
04:13:57,090 [master>cosmo_unit_tests_remote>cosmo_unit_tests@ubuntu:8811] INFO - Step execute condition satisfied, executing...
04:13:58,108 [master>cosmo_unit_tests_remote>cosmo_unit_tests@ubuntu:8811] INFO - Executing pre-execute action...
04:13:58,108 [master>cosmo_unit_tests_remote>cosmo_unit_tests@ubuntu:8811] INFO - Running step...
04:13:58,160 [master>cosmo_unit_tests_remote>cosmo_unit_tests@ubuntu:8811] INFO - workingDir:/opt/install/buildagent/workspace/root/cosmo/trunk/builds/ContinuousBuild/cosmo-cli
04:13:58,160 [master>cosmo_unit_tests_remote>cosmo_unit_tests@ubuntu:8811] INFO - parentConf=root/cosmo
04:13:58,161 [master>cosmo_unit_tests_remote>cosmo_unit_tests@ubuntu:8811] INFO - branch='develop'
04:13:58,161 [master>cosmo_unit_tests_remote>cosmo_unit_tests@ubuntu:8811] INFO - skip_commans: false
04:13:58,161 [master>cosmo_unit_tests_remote>cosmo_unit_tests@ubuntu:8811] INFO - Running command: virtualenv env
04:13:59,809 [master>cosmo_unit_tests_remote>cosmo_unit_tests@ubuntu:8811] INFO - return code: 0
04:13:59,810 [master>cosmo_unit_tests_remote>cosmo_unit_tests@ubuntu:8811] INFO - stderr:
04:13:59,810 [master>cosmo_unit_tests_remote>cosmo_unit_tests@ubuntu:8811] INFO - stdout: New python executable in env/bin/python
Installing setuptools, pip...done.
04:13:59,810 [master>cosmo_unit_tests_remote>cosmo_unit_tests@ubuntu:8811] INFO - skip_commans: false
04:13:59,810 [master>cosmo_unit_tests_remote>cosmo_unit_tests@ubuntu:8811] INFO - Running command: python setup.py install
04:13:59,810 [master>cosmo_unit_tests_remote>cosmo_unit_tests@ubuntu:8811] INFO - return code: 0
04:13:59,811 [master>cosmo_unit_tests_remote>cosmo_unit_tests@ubuntu:8811] INFO - Executing post-execute action...
04:13:59,812 [master>cosmo_unit_tests_remote>cosmo_unit_tests@ubuntu:8811] ERROR - Step 'master>cosmo_unit_tests_remote>cosmo_unit_tests' is failed.
java.lang.RuntimeException: Failed to evaluate below expression:
groovy:

def fail_list;
change_repos_list = vars.get("change_repos_list");


for ( repo_dir in change_repos_list )
{
def workingDir = new File(configuration.getWorkspaceDir().getAbsolutePath() +"/"+ repo_dir);
def workingDirBase = new File(configuration.getWorkspaceDir().getAbsolutePath());
logger.info("workingDir:" + workingDir);

def skip_commans=false;
/*logger.info( "delMainDir: " + delMainDir); */
def parentConf = configuration.getParent().getParent().getParent();
logger.info("parentConf=" + parentConf );
def branch = parentConf.getVariables().find {vari -> vari.name == 'branch_name'};
logger.info("branch='" + branch + "'");

/*File gitFile = new File(workingDir,".git");*/

if (workingDir.exists())
{
cmd1="virtualenv env";
cmd2="source ../env/bin/activate";
cmd3="python setup.py install";
cmd4="pip install flake8";
cmd5="pip install node";
cmd6="flake8 .";
cmd7="nosetests . --nologcapture --nocapture";
cmd8="echo \$?";
cmd9="rm -rf env"
cmd_array = [cmd1,cmd3,cmd4,cmd5,cmd6,cmd7]
i=0;
for ( cmd in cmd_array )
{
i=i+1;
logger.info( "skip_commans: " + skip_commans);
/* if(i<11 && skip_commans)
{
logger.info( "skip commands due to a failure in previous command");
break;
}*/

logger.info( "Running command: " + cmd);
if(i==1 || i==9)
{
cmdreturn = cmd.execute(null, workingDirBase);
}
else
{
def cmdreturn = cmd.execute(null, workingDir);
}

cmdreturn.waitFor();
logger.info( "return code: "+cmdreturn.exitValue());
logger.info( "stderr: "+cmdreturn.err.text);
logger.info( "stdout: "+cmdreturn.in.text);
if (i==7 && cmdreturn.exitValue()!=0)
{
/* break;
skip_commans=true; */
if(fail_list == null)
{
fail_list=repo_dir;
}
else
{
repos_list=fail_list+","+repo_dir;
}
vars.get("cosmo_unit_tests_fail").setValue(repos_list);
}
}

}
else
{
logger.info(" "+workingDir + " folder does not exist");
}
}
at com.pmease.quickbuild.util.ExceptionUtils.wrapException(ExceptionUtils.java:87)
at com.pmease.quickbuild.DefaultScriptEngine.evaluate(DefaultScriptEngine.java:93)
at com.pmease.quickbuild.plugin.basis.ScriptStep.run(ScriptStep.java:47)
at com.pmease.quickbuild.plugin.basis.ScriptStep$$EnhancerByCGLIB$$31539075.CGLIB$run$1(<generated>)
at com.pmease.quickbuild.plugin.basis.ScriptStep$$EnhancerByCGLIB$$31539075$$FastClassByCGLIB$$3046344c.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:215)
at com.pmease.quickbuild.DefaultScriptEngine$Interpolator.intercept(DefaultScriptEngine.java:270)
at com.pmease.quickbuild.plugin.basis.ScriptStep$$EnhancerByCGLIB$$31539075.run(<generated>)
at com.pmease.quickbuild.stepsupport.Step.execute(Step.java:479)
at com.pmease.quickbuild.stepsupport.StepExecutionJob.executeStepAwareJob(StepExecutionJob.java:29)
at com.pmease.quickbuild.stepsupport.StepAwareJob.executeBuildAwareJob(StepAwareJob.java:47)
at com.pmease.quickbuild.BuildAwareJob.execute(BuildAwareJob.java:61)
at com.pmease.quickbuild.grid.GridJob.run(GridJob.java:78)
at java.lang.Thread.run(Thread.java:744)
Caused by: com.pmease.quickbuild.QuickbuildException: java.io.IOException: Stream closed
at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:162)
at java.io.BufferedInputStream.read(BufferedInputStream.java:325)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at java.io.BufferedReader.read1(BufferedReader.java:203)
at java.io.BufferedReader.read(BufferedReader.java:279)
at java.io.Reader.read(Reader.java:140)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.getText(DefaultGroovyMethods.java:12153)
at org.codehaus.groovy.runtime.DefaultGroovyMethods.getText(DefaultGroovyMethods.java:12104)
at org.codehaus.groovy.runtime.dgm$303.doMethodInvoke(Unknown Source)
at org.codehaus.groovy.reflection.GeneratedMetaMethod$Proxy.doMethodInvoke(GeneratedMetaMethod.java:70)
at groovy.lang.MetaClassImpl$GetBeanMethodMetaProperty.getProperty(MetaClassImpl.java:3468)
at org.codehaus.groovy.runtime.callsite.GetEffectivePojoPropertySite.getProperty(GetEffectivePojoPropertySite.java:61)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGetProperty(AbstractCallSite.java:235)
at script1393294438108161778831.run(script1393294438108161778831.groovy:61)
at com.pmease.quickbuild.plugin.basis.BasisPlugin$18.evaluate(BasisPlugin.java:261)
at com.pmease.quickbuild.DefaultScriptEngine.evaluate(DefaultScriptEngine.java:80)
at com.pmease.quickbuild.plugin.basis.ScriptStep.run(ScriptStep.java:47)
at com.pmease.quickbuild.plugin.basis.ScriptStep$$EnhancerByCGLIB$$31539075.CGLIB$run$1(<generated>)
at com.pmease.quickbuild.plugin.basis.ScriptStep$$EnhancerByCGLIB$$31539075$$FastClassByCGLIB$$3046344c.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:215)
at com.pmease.quickbuild.DefaultScriptEngine$Interpolator.intercept(DefaultScriptEngine.java:270)
at com.pmease.quickbuild.plugin.basis.ScriptStep$$EnhancerByCGLIB$$31539075.run(<generated>)
at com.pmease.quickbuild.stepsupport.Step.execute(Step.java:479)
at com.pmease.quickbuild.stepsupport.StepExecutionJob.executeStepAwareJob(StepExecutionJob.java:29)
at com.pmease.quickbuild.stepsupport.StepAwareJob.executeBuildAwareJob(StepAwareJob.java:47)
at com.pmease.quickbuild.BuildAwareJob.execute(BuildAwareJob.java:61)
at com.pmease.quickbuild.grid.GridJob.run(GridJob.java:78)
at java.lang.Thread.run(Thread.java:744)
at com.pmease.quickbuild.plugin.basis.BasisPlugin$18.evaluate(BasisPlugin.java:285)
at com.pmease.quickbuild.DefaultScriptEngine.evaluate(DefaultScriptEngine.java:80)
... 12 more
04:13:59,827 [master>cosmo_unit_tests_remote@ubuntu:8811] INFO - Executing post-execute action...
04:13:59,827 [master>cosmo_unit_tests_remote@ubuntu:8811] ERROR - Step 'master>cosmo_unit_tests_remote' is failed.
com.pmease.quickbuild.QuickbuildException: Composite step 'cosmo_unit_tests_remote' failed due to unsatisfication of success condition.
at com.pmease.quickbuild.stepsupport.CompositeStep.run(CompositeStep.java:100)
at com.pmease.quickbuild.stepsupport.Step.execute(Step.java:479)
at com.pmease.quickbuild.stepsupport.StepExecutionJob.executeStepAwareJob(StepExecutionJob.java:29)
at com.pmease.quickbuild.stepsupport.StepAwareJob.executeBuildAwareJob(StepAwareJob.java:47)
at com.pmease.quickbuild.BuildAwareJob.execute(BuildAwareJob.java:61)
at com.pmease.quickbuild.grid.GridJob.run(GridJob.java:78)
at java.lang.Thread.run(Thread.java:744)
17:50:01,346 [master@build64A:8810] INFO - Executing post-execute action...
17:50:01,346 [master@build64A:8810] ERROR - Step 'master' is failed.
com.pmease.quickbuild.QuickbuildException: Composite step 'master' failed due to unsatisfication of success condition.
at com.pmease.quickbuild.stepsupport.CompositeStep.run(CompositeStep.java:100)
at com.pmease.quickbuild.stepsupport.Step.execute(Step.java:479)
at com.pmease.quickbuild.stepsupport.StepExecutionJob.executeStepAwareJob(StepExecutionJob.java:29)
at com.pmease.quickbuild.stepsupport.StepAwareJob.executeBuildAwareJob(StepAwareJob.java:47)
at com.pmease.quickbuild.BuildAwareJob.execute(BuildAwareJob.java:61)
at com.pmease.quickbuild.grid.GridJob.run(GridJob.java:78)
at java.lang.Thread.run(Thread.java:662)
#4
limor ·
Can I get environment variables from groovy?
#5
robinshen ADMIN ·
I am afraid not, as environments altered by forked command will not change environments of current process. How about writing the output to an external file, and then read back that file in groovy to set variable?
#6
limor ·
If there is no other choice I will use an external file.
Thank you
Log in to reply