QB2B3: Changed files/sets not correct #546

scastria · 2 decades ago

I have two successive builds in QB2, #19 and #20. #19 used svn rev 4127 of my repo and #20 used svn rev 4129 of my repo. However, no changed files are listed in the svn changes section of the build report for #20. I can see the different svn rev numbers in the log. The build was kicked off by the scheduler so it was able to detect the change in the svn repo. I have this problem in QB1 all the time, so I have learned not to trust the Revision Log report in QB1. Can QB2 be fixed to make the changed files/sets reliable AND can we put the SVN repo revisions in a easy to find location so I don't have to check the build log to see what svn repo revision was checked out?

replies 9
views 3545
stars 0

robinshen ADMIN · 2 decades ago

The revision log should be accurate. Please follow below steps to further investigate the problem:
1. Edit configuration setting to have it run in debug mode.
2. Delete build #20, and manually trigger another build in this configuration.
3. If the changes are not accurate for this build, please check the build log and the full Subversion log command will be there. QuickBuild uses this command to get changes for the build.

If this problem only applies for scheduled builds, please turn on debug level for system log (by editing the file conf/log4j.properties, you do not need to restart the server since the file will be monitored). And the subversion log command can be found there if there is a new build generated.

Thanks
Robin

scastria · 2 decades ago

I don't see any log commands in the build log. I have TRACE turned on which I assume is more than DEBUG. Here is the full log for a build that DOES have a valid changeset report. It just shows a checkout command, but no log command. I do see something related to calculating changes since previous build in the system log which is also attached at the bottom.

10:18:44,0 [master@gx2xp:8810] DEBUG - Acquiring workspace mutex...
10:18:44,0 [master@gx2xp:8810] INFO - Trigger step 'master' on node 'gx2xp:8810'...
10:18:44,0 [master@gx2xp:8810] INFO - Checking execute condition of step 'master'...
10:18:44,0 [master@gx2xp:8810] INFO - Execute condition satisfied, running step 'master'...
10:18:44,1 [checkout-svn@gx2xp:8810] DEBUG - Acquiring workspace mutex...
10:18:44,1 [checkout-svn@gx2xp:8810] INFO - Trigger step 'checkout-svn' on node 'gx2xp:8810'...
10:18:44,1 [checkout-svn@gx2xp:8810] INFO - Checking execute condition of step 'checkout-svn'...
10:18:44,1 [checkout-svn@gx2xp:8810] INFO - Execute condition satisfied, running step 'checkout-svn'...
10:18:44,1 [checkout-svn@gx2xp:8810] INFO - Checking out revision '1238' of repository 'svn'...
10:18:44,1 [checkout-svn@gx2xp:8810] INFO - Checking out (url: svn://machine.domain.com/repos/lgcbuild/trunk, to: C:\QuickBuild2\workspace\root\lgcbuilds\lgcbuild\lgcbuild, revision: 1238)
10:18:44,1 [checkout-svn@gx2xp:8810] DEBUG - Executing command: C:\svn-win32-1.5.5\bin\svn.exe checkout svn://machine.domain.com/repos/lgcbuild/trunk C:\QuickBuild2\workspace\root\lgcbuilds\lgcbuild\lgcbuild --non-interactive --username renobuild --password ****** -r 1238
10:18:44,5 [checkout-svn@gx2xp:8810] INFO - U C:\QuickBuild2\workspace\root\lgcbuilds\lgcbuild\lgcbuild\ivy\release_specs\ivy-DS_5000_4_0_0_QC4.xml
10:18:45,0 [checkout-svn@gx2xp:8810] INFO - Checked out revision 1238.
10:18:46,1 [checkout-svn@gx2xp:8810] INFO - Step 'checkout-svn' is successful.
10:18:46,1 [checkout-svn@gx2xp:8810] DEBUG - Workspace mutex released.
10:18:46,1 [master@gx2xp:8810] INFO - Step 'master' is successful.
10:18:46,1 [master@gx2xp:8810] DEBUG - Workspace mutex released.

2009-07-13 15:34:09,2 [Thread-1887] INFO com.pmease.quickbuild.plugin.scm.svn.SvnRepository - Taking snapshot of repository 'svn'...
2009-07-13 15:34:09,2 [Thread-1887] INFO com.pmease.quickbuild.plugin.scm.svn.SvnRepository - Determining head revision for repository: svn
2009-07-13 15:34:09,3 [Thread-1888] INFO com.pmease.quickbuild.DefaultBuildEngine - Evaluating build condition for configuration 'root/lgcbuilds/lgcbuild'...
2009-07-13 15:34:09,4 [Thread-1888] INFO com.pmease.quickbuild.plugin.scm.svn.SvnRepository - Getting changes since build 'lgcbuild.43'...

scastria · 2 decades ago

This question makes me wonder where do I specify how I want the changeset calculated. I might want the changeset report to show the commits since last SUCCESSFUL build or I might want the changeset report to show the commits since last ANY build. In QB1, the repository definition had "Revision log base build version". What is the equivalent in QB2? Is that what the "Snapshot Taking Script" is for in the advanced tab? By the name of it, it seems this is just responsible for taking the snapshot, but not choosing what to compare against when calculating the changeset report.

scastria · 2 decades ago

Here is some results of testing this changeset report with QB2B5:

1. I created a dummy build configuration that just contains a single checkout
2. I triggered a first build where I hardcoded the svn rev number to 1000 to get a baseline
3. changeset report was empty as expected since it no previous build to compare to
4. I then changed the svn rev number hardcoding to 1010 and ran the build WITHOUT cleaning up the workspace
5. changeset report shows all 10 commits, <img src="{SMILIES_PATH}/icon_biggrin.gif" alt=":D" title="Very Happy" />
6. build log DOES show the svn log command with -r 1000:1010, <img src="{SMILIES_PATH}/icon_biggrin.gif" alt=":D" title="Very Happy" />
7. I then changed the svn rev number hardcoding to 1020 and ran the build CLEAN by cleaning up the workspace as part of master
8. changeset report shows all 10 commits, <img src="{SMILIES_PATH}/icon_biggrin.gif" alt=":D" title="Very Happy" />
9. build log DOES show the svn log command with -r 1010:1020, <img src="{SMILIES_PATH}/icon_biggrin.gif" alt=":D" title="Very Happy" />
10. I then changed the svn rev number hardcoding to 1030 and let the scheduler trigger the build WITHOUT cleaning up the workspace
11. changeset report shows all 10 commits, <img src="{SMILIES_PATH}/icon_biggrin.gif" alt=":D" title="Very Happy" />
12. system log DOES show the svn log command with -r 1020:1030, <img src="{SMILIES_PATH}/icon_biggrin.gif" alt=":D" title="Very Happy" />
13. I then changed the svn rev number hardcoding to 1040 and let the scheduler trigger the build CLEAN by cleaning up the workspace as part of master
14. changeset report shows all 10 commits, <img src="{SMILIES_PATH}/icon_biggrin.gif" alt=":D" title="Very Happy" />
15. system log DOES show the svn log command with -r 1030:1040, <img src="{SMILIES_PATH}/icon_biggrin.gif" alt=":D" title="Very Happy" />
16. I then removed the svn rev number hardcoding and let the scheduler trigger the build WITHOUT cleaning up the workspace
17. changeset report shows all remaining commits, <img src="{SMILIES_PATH}/icon_biggrin.gif" alt=":D" title="Very Happy" />
18. system log DOES show the svn log command with -r 1040:1239, <img src="{SMILIES_PATH}/icon_biggrin.gif" alt=":D" title="Very Happy" />
19. I then deleted that build, restored my workspace back to 1040, and let the scheduler trigger the build CLEAN by cleaning up the workspace as part of master
20. changeset report shows all remaining commits, <img src="{SMILIES_PATH}/icon_biggrin.gif" alt=":D" title="Very Happy" />
21. system log DOES show the svn log command with -r 1040:1239, <img src="{SMILIES_PATH}/icon_biggrin.gif" alt=":D" title="Very Happy" />

So everything is working in these tests against QB2B5. Maybe it was broken in QB2B3 when I originally posted this. I will see if I can find a broken Changeset against QB2B5.

robinshen ADMIN · 2 decades ago

In case of a scheduled build, the log command will come into system log (since build is not generated or may not necessarily to be generated at this time), and you will need to tune system log setting to debug level by editing entry "com.pmease.quickbuild" in file "conf/log4j.properties", as explained in my previous post.

When calculating changes, 2.0 always uses previous build as base build, regardless it is failed or succeed. However, you may compare a particular build with other build in the same configuration to get the exact changes between them in the GUI.

scastria · 2 decades ago

Sorry, I guess I didn't read all of your post. Thanks, I will try that.

In the meantime, I am working on finishing up my testing which I will post to this discussion thread.

scastria · 2 decades ago

The QB2 changeset report I think looks accurate. However, since I am still using QB1 in production AND I continue to get unreliable changeset reports from it (as mentioned at the top of this post), I thought I would investigate based on this information in this post.

I think one bug in QB1 that is causing unreliable changeset reports is the quiet period. I performed a checkin to a repo and then triggered a build almost immediately. I can see the build log showing this:

svn log
'-r'
'{2009-08-02T18:20:17Z}:{2009-08-02T18:25:18Z}'

2009-08-02 13:25:18,493 [Thread-196469] INFO - Repository not quiet in recent 300 seconds, sleeping a while before checking again...

So it obviously detected my change so it should be available in the revision log. However, because it was within the quiet period, QB1 waiting a while and reattempted the log. Using this command:

svn log
'-r'
'{2009-08-02T18:25:18Z}:{2009-08-02T18:30:18Z}'

2009-08-02 13:30:18,777 [Thread-196469] INFO - Repository "svn-branch" quiet in recent 300 seconds, continue to check out artifacts...

Notice that the second log command continued where it left off from the first log command and did NOT detect any more changes. Therefore, it was safe to continue the build since the quiet period had elapsed. However, I am thinking that the bug in QB1 is that the results of this second log command is being used for the revision log. That would incorrect. It needs to combine the changed files from BOTH log commands, otherwise, it loses my checkin and the revision log is empty.

That is exactly what I got when this build finished, an empty revision log.

I don't know if all QB1 revision log bugs are related to quiet period, but we can at least fix this one to make it better.

robinshen ADMIN · 2 decades ago

The change sets of current build is calculated by running svn log command between revision of base build (normally the previous build) and current build. And it does not rely on results of the quiet period detection. To verify, please make another checkin, and run the build immediately, you will find another svn log command after below log entry:
"Get change list since build"

If the changesets are detected incorrectly, it is most probably that the "revision log base build version" is set incorrectly. The default value is ${lastBuild==null?'':lastBuild.version}, which means calculating changes since last build.

#10

scastria · 2 decades ago

Unfortunately, my build retention has caused this particular build to be removed. You are correct that the date filters is purely to test the quiet period and the revision svn log command is used to get changes since last build. I have my svn repo set to:

${lastSuccessBuild==null?blank:lastSuccessBuild.version}

so I get changes since last successful build. AAAHHHHHAAAAA! I just figured out what is wrong. My build retention only keeps the last 7 builds. This particular build had been failing for a long time causing all 7 builds to be failures. Therefore, there was no successful build available to compare against for generating the revision log. I think I will change back to last build instead of last successful build. Sorry, for thinking QB1 had a bug in this area, but your feedback finally helped me realize why I felt I couldn't trust QB1's revision log all this time. Now I can.