I''ve been trying to figure out how the automated regression testing
stuff works, partly because I''m interested in whether the patches
I submitted broke anything and what I should be looking for.
So this came in on 5th April (latest example of a regression
message for qemu-upstream-4.2-testing). I would take this to
mean the changesets applied (the two listed below) caused
a regression in the test guest-localmigrate/x10 (whatever
that is).
However, the two commits did get pushed from staging to the
main repos, with two subsequent ''tolerable FAIL - pushed''
messages.
I''m not the author of the two commits in question, but if I
was I''d want to know what test actually failed and why. The logs
weren''t vastly illuminating without knowing what the test does.
I''d also be keen to know whether I should be investigating this
or whether waiting a couple of days might result in a retest
that appears to pass the code.
Is there a newbie guide for this?
Alex
---------- Forwarded Message ----------
Date: 5 April 2013 23:03:10 +0100
From: "xen.org" <ian.jackson@eu.citrix.com>
To: xen-devel@lists.xensource.com
CC: ian.jackson@eu.citrix.com
Subject: [Xen-devel] [qemu-upstream-4.2-testing test] 17485: regressions -
FAIL
flight 17485 qemu-upstream-4.2-testing real [real]
http://www.chiark.greenend.org.uk/~xensrcts/logs/17485/
Regressions :-(
Tests which did not succeed and are blocking,
including tests which could not be run:
test-amd64-amd64-xl-qemuu-winxpsp3 12 guest-localmigrate/x10 fail REGR.
vs. 17143
Tests which did not succeed, but are not blocking:
build-armhf 4 xen-build fail never
pass test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 13 guest-stop fail
never pass test-amd64-amd64-xl-qemuu-win7-amd64 13 guest-stop
fail never pass test-amd64-i386-xl-qemuu-win7-amd64 13 guest-stop
fail never pass test-i386-i386-xl-qemuu-winxpsp3 13 guest-stop
fail never pass test-amd64-i386-xend-qemuu-winxpsp3 16 leak-check/check
fail never pass
version targeted for testing:
qemuu bf8e2e756e3e98b8388068173e24d99d1e09d2dd
baseline version:
qemuu 351f94ff4bf3a7795ca5b282305aa610e598eec0
------------------------------------------------------------
People who touched revisions under test:
Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Weidong Han <hanweidong@huawei.com>
Zhenguo Wang <wangzhenguo@huawei.com>
------------------------------------------------------------
jobs:
build-amd64 pass
build-armhf fail
build-i386 pass
build-amd64-oldkern pass
build-i386-oldkern pass
build-amd64-pvops pass
build-i386-pvops pass
test-amd64-i386-qemuu-rhel6hvm-amd pass
test-amd64-amd64-xl-qemuu-win7-amd64 fail
test-amd64-i386-xl-qemuu-win7-amd64 fail
test-amd64-i386-qemuu-rhel6hvm-intel pass
test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 fail
test-amd64-i386-xend-qemuu-winxpsp3 fail
test-amd64-amd64-xl-qemuu-winxpsp3 fail
test-i386-i386-xl-qemuu-winxpsp3 fail
------------------------------------------------------------
sg-report-flight on woking.cam.xci-test.com
logs: /home/xc_osstest/logs
images: /home/xc_osstest/images
Logs, config files, etc. are available at
http://www.chiark.greenend.org.uk/~xensrcts/logs
Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary
Not pushing.
------------------------------------------------------------
commit bf8e2e756e3e98b8388068173e24d99d1e09d2dd
Author: Hanweidong <hanweidong@huawei.com>
Date: Tue Apr 2 13:22:41 2013 +0000
xen-mapcache: pass the right size argument to test_bits
Compute the correct size for test_bits().
qemu_get_ram_ptr() and qemu_safe_ram_ptr() will call xen_map_cache()
with size is 0 if the requested address is in the RAM. Then
xen_map_cache() will pass the size 0 to test_bits() for checking if the
corresponding pfn was mapped in cache. But test_bits() will always
return 1 when size is 0 without any bit testing. Actually, for this
case, test_bits should check one bit. So this patch introduced a
__test_bit_size which is greater than 0 and a multiple of XC_PAGE_SIZE,
then test_bits can work correctly with __test_bit_size
>> XC_PAGE_SHIFT as its size.
upstream-commit-id: 044d4e1aae539bd4214175bd9591b3de7986cf18
Signed-off-by: Zhenguo Wang <wangzhenguo@huawei.com>
Signed-off-by: Weidong Han <hanweidong@huawei.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
commit 0718150131d1d68436ba92b209fa50430198e6f4
Author: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Date: Tue Apr 2 13:23:40 2013 +0000
xen-mapcache: replace last_address_index with a last_entry pointer
Replace last_address_index and last_address_vaddr with a single pointer
to the last MapCacheEntry used.
upstream-commit-id: e2deee3ea6136b6189e8cfd26379420b9a398d96
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
---------- End Forwarded Message ----------
--
Alex Bligh
Alex Bligh writes ("[Xen-devel] How does the regression testing stuff
work?"):> So this came in on 5th April (latest example of a regression
> message for qemu-upstream-4.2-testing). I would take this to
> mean the changesets applied (the two listed below) caused
> a regression in the test guest-localmigrate/x10 (whatever
> that is).
Yes. Although the apparent regression appeared once in this one test,
so if the tree is unreliable (sadly it seems to be) it could be a
heisenbug.
> However, the two commits did get pushed from staging to the
> main repos, with two subsequent ''tolerable FAIL - pushed''
messages.
Right.
> I''m not the author of the two commits in question, but if I
> was I''d want to know what test actually failed and why. The logs
> weren''t vastly illuminating without knowing what the test does.
> I''d also be keen to know whether I should be investigating this
> or whether waiting a couple of days might result in a retest
> that appears to pass the code.
If you visit this url:
> flight 17485 qemu-upstream-4.2-testing real [real]
> http://www.chiark.greenend.org.uk/~xensrcts/logs/17485/
You''ll see a big grid.
> Tests which did not succeed and are blocking,
> including tests which could not be run:
> test-amd64-amd64-xl-qemuu-winxpsp3 12 guest-localmigrate/x10 fail REGR.
> vs. 17143
Look at the column for the failure. You can see it because of the
red/pink REGR entry in the grid, or you can look at the column
heading.
Click on the column heading.
This will show you a lot of logs. The one which is most relevant is
the test script log, which is linked to from "fail" in the Steps table
there.
http://www.chiark.greenend.org.uk/~xensrcts/logs/17485/test-amd64-amd64-xl-qemuu-winxpsp3/12.ts-guest-localmigrate.log
So you can see there that the tester ran the 9th repetition of
ssh ... root@10.80.250.26 xl migrate win.guest.osstest localhost
and that appeared to work in the sense that it produced sane-looking
output, but it didn''t exit. So 400s later the tester says this:
ssh -o StrictHostKeyChecking=no -o BatchMode=yes -o
ConnectTimeout=100 -o ServerAliveInterval=100 -o
PasswordAuthentication=no -o ChallengeResponseAuthentication=no -o
UserKnownHostsFile=tmp/t.known_hosts_17485.test-amd64-amd64-xl-qemuu-winxpsp3
root@10.80.250.26 xl migrate win.guest.osstest localhost
status (timed out) at Osstest/TestSupport.pm line 375.
I don''t know why this didn''t work, but if it passed later it
must be a
race of some kind. It therefore may be the case that the changes
being tested were not in fact responsible.
Ian.
Alex Bligh writes ("Re: [Xen-devel] How does the regression testing stuff
work?"):> Thanks. I''m sort of confused how this can have been being tested
at all
> though given this is qemu-upstream-4.2-testing and:
> a) WinXP implies HVM and the logs also say HVM
Yes.
> b) Qemu-upstream implies Qemu-upstream-DM
Yes.
> c) Live migrate of HVM on Qemu-upstream-DM did not work at all in
> 4.2 until very recently (hence why I backported the patches)
Maybe it passed by fluke ?
Or maybe there was something wrong with the test in 17143 which meant
that it didn''t use qemu-upstream-dm at all. It''s difficult to
tell
now I''m afraid.
> If I click on the link to 17143 (after ''REGR'') I get a
404 error.
I''m afraid the logs have been expired.
Ian.
--On 8 April 2013 16:35:27 +0100 Ian Jackson <Ian.Jackson@eu.citrix.com> wrote:>> So this came in on 5th April (latest example of a regression >> message for qemu-upstream-4.2-testing). I would take this to >> mean the changesets applied (the two listed below) caused >> a regression in the test guest-localmigrate/x10 (whatever >> that is). > > Yes. Although the apparent regression appeared once in this one test, > so if the tree is unreliable (sadly it seems to be) it could be a > heisenbug.Thanks. I''m sort of confused how this can have been being tested at all though given this is qemu-upstream-4.2-testing and: a) WinXP implies HVM and the logs also say HVM b) Qemu-upstream implies Qemu-upstream-DM c) Live migrate of HVM on Qemu-upstream-DM did not work at all in 4.2 until very recently (hence why I backported the patches) If I click on the link to 17143 (after ''REGR'') I get a 404 error. -- Alex Bligh