flight 10135 xen-unstable real [real]
http://www.chiark.greenend.org.uk/~xensrcts/logs/10135/
Regressions :-(
Tests which did not succeed and are blocking:
test-amd64-i386-rhel6hvm-intel 7 redhat-install fail REGR. vs. 9956
test-amd64-i386-rhel6hvm-amd 7 redhat-install fail REGR. vs. 9956
test-amd64-amd64-xl-win 7 windows-install fail REGR. vs. 9956
test-amd64-i386-xl-win-vcpus1 7 windows-install fail REGR. vs. 9956
test-i386-i386-xl-win 7 windows-install fail REGR. vs. 9956
Tests which did not succeed, but are not blocking,
including regressions (tests previously passed) regarded as allowable:
test-amd64-amd64-xl-pcipt-intel 9 guest-start fail never pass
test-amd64-i386-win-vcpus1 16 leak-check/check fail never pass
test-amd64-i386-win 16 leak-check/check fail never pass
test-i386-i386-win 16 leak-check/check fail never pass
test-amd64-amd64-win 16 leak-check/check fail never pass
version targeted for testing:
xen 95d4e2e0bed3
baseline version:
xen 0a0c02a61676
------------------------------------------------------------
People who touched revisions under test:
Adin Scannell <adin@scannell.ca>
Andres Lagar-Cavilla <andres@lagarcavilla.org>
Anil Madhavapeddy <anil@recoil.org>
Daniel De Graaf <dgdegra@tycho.nsa.gov>
George Dunlap <george.dunlap@eu.citrix.com>
Ian Campbell <ian.campbell@citrix.com>
Ian Jackson <ian.jackson.citrix.com>
Ian Jackson <ian.jackson@eu.citrix.com>
Jan Beulich <jbeulich@suse.com>
Jean Guyader <jean.guyader@eu.citrix.com>
Keir Fraser <keir@xen.org>
Olaf Hering <olaf@aepfle.de>
Paul Durrant <paul.durrant@citrix.com>
Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Tim Deegan <tim@xen.org>
Wei Wang <wei.wang2@amd.com>
------------------------------------------------------------
jobs:
build-amd64 pass
build-i386 pass
build-amd64-oldkern pass
build-i386-oldkern pass
build-amd64-pvops pass
build-i386-pvops pass
test-amd64-amd64-xl pass
test-amd64-i386-xl pass
test-i386-i386-xl pass
test-amd64-i386-rhel6hvm-amd fail
test-amd64-i386-xl-credit2 pass
test-amd64-amd64-xl-pcipt-intel fail
test-amd64-i386-rhel6hvm-intel fail
test-amd64-i386-xl-multivcpu pass
test-amd64-amd64-pair pass
test-amd64-i386-pair pass
test-i386-i386-pair pass
test-amd64-amd64-pv pass
test-amd64-i386-pv pass
test-i386-i386-pv pass
test-amd64-amd64-xl-sedf pass
test-amd64-i386-win-vcpus1 fail
test-amd64-i386-xl-win-vcpus1 fail
test-amd64-amd64-win fail
test-amd64-i386-win fail
test-i386-i386-win fail
test-amd64-amd64-xl-win fail
test-i386-i386-xl-win fail
------------------------------------------------------------
sg-report-flight on woking.cam.xci-test.com
logs: /home/xc_osstest/logs
images: /home/xc_osstest/images
Logs, config files, etc. are available at
http://www.chiark.greenend.org.uk/~xensrcts/logs
Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary
Not pushing.
(No revision log; it would be 906 lines long.)
xen.org writes ("[xen-unstable test] 10135: regressions -
FAIL"):> flight 10135 xen-unstable real [real]
> http://www.chiark.greenend.org.uk/~xensrcts/logs/10135/
>
> Regressions :-(
>
> Tests which did not succeed and are blocking:
> test-amd64-i386-rhel6hvm-intel 7 redhat-install fail REGR. vs. 9956
My bisector wasn''t able to finger an exact changeset due to a blocking
bug in the vicinity, but:
pass 24183:53bec838bb08 Merge
blocked 24184:4ecd3615e726 tools: use system installed libaio by default.
blocked 24185:f88c745575bb docs: remove some fatally out of date ...
fail 24186:7aa5838499d1 tools: use system libaio for blktap1 as well.
Here the "system installed libaio" is that from Debian squeeze.
So for now I have applied the changeset below in the hope of getting a
pass and push.
Ian.
# HG changeset patch
# User Ian Jackson <Ian.Jackson@eu.citrix.com>
# Date 1322481443 0
# Node ID a9c67c2daf4b0181ef2581471ea920eecb495616
# Parent 95d4e2e0bed374602b5a78ee004b057ad8715d65
tools: Revert to our built-in aio
These two changesets:
24184:4ecd3615e726 tools: use system installed libaio by default.
24186:7aa5838499d1 tools: use system libaio for blktap1 as well.
cause HVM guest installs (both Windows and Redhat) to fail on Debian
squeeze with xl. So change the default for now.
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
diff -r 95d4e2e0bed3 -r a9c67c2daf4b Config.mk
--- a/Config.mk Fri Nov 25 20:32:05 2011 +0000
+++ b/Config.mk Mon Nov 28 11:57:23 2011 +0000
@@ -232,7 +232,7 @@ PYTHON_TOOLS ?= y
OCAML_TOOLS ?= y
CONFIG_MINITERM ?= n
CONFIG_LOMOUNT ?= n
-CONFIG_SYSTEM_LIBAIO ?= y
+CONFIG_SYSTEM_LIBAIO ?= n
ifeq ($(OCAML_TOOLS),y)
OCAML_TOOLS := $(shell ocamlopt -v > /dev/null 2>&1 && echo
"y" || echo "n")
On Mon, 2011-11-28 at 11:58 +0000, Ian Jackson wrote:> xen.org writes ("[xen-unstable test] 10135: regressions - FAIL"): > > flight 10135 xen-unstable real [real] > > http://www.chiark.greenend.org.uk/~xensrcts/logs/10135/ > > > > Regressions :-( > > > > Tests which did not succeed and are blocking: > > test-amd64-i386-rhel6hvm-intel 7 redhat-install fail REGR. vs. 9956 > > My bisector wasn''t able to finger an exact changeset due to a blocking > bug in the vicinity, but: > > pass 24183:53bec838bb08 Merge > blocked 24184:4ecd3615e726 tools: use system installed libaio by default. > blocked 24185:f88c745575bb docs: remove some fatally out of date ... > fail 24186:7aa5838499d1 tools: use system libaio for blktap1 as well. > > Here the "system installed libaio" is that from Debian squeeze.This is really weird since nothing in the failings tests even exercises this stuff. Only blktap (1 and 2) uses aio and the the tests don''t use it and neither does qemu-dm which is the only other thing I can think of. I''m setting up a repro to see if I can figure out what is going wrong.> So for now I have applied the changeset below in the hope of getting a > pass and push.I think that''s best for now. Ian.> > Ian. > > # HG changeset patch > # User Ian Jackson <Ian.Jackson@eu.citrix.com> > # Date 1322481443 0 > # Node ID a9c67c2daf4b0181ef2581471ea920eecb495616 > # Parent 95d4e2e0bed374602b5a78ee004b057ad8715d65 > tools: Revert to our built-in aio > > These two changesets: > 24184:4ecd3615e726 tools: use system installed libaio by default. > 24186:7aa5838499d1 tools: use system libaio for blktap1 as well. > cause HVM guest installs (both Windows and Redhat) to fail on Debian > squeeze with xl. So change the default for now. > > Committed-by: Ian Jackson <ian.jackson@eu.citrix.com> > > diff -r 95d4e2e0bed3 -r a9c67c2daf4b Config.mk > --- a/Config.mk Fri Nov 25 20:32:05 2011 +0000 > +++ b/Config.mk Mon Nov 28 11:57:23 2011 +0000 > @@ -232,7 +232,7 @@ PYTHON_TOOLS ?= y > OCAML_TOOLS ?= y > CONFIG_MINITERM ?= n > CONFIG_LOMOUNT ?= n > -CONFIG_SYSTEM_LIBAIO ?= y > +CONFIG_SYSTEM_LIBAIO ?= n > > ifeq ($(OCAML_TOOLS),y) > OCAML_TOOLS := $(shell ocamlopt -v > /dev/null 2>&1 && echo "y" || echo "n") > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel
On Mon, 2011-11-28 at 13:38 +0000, Ian Campbell wrote:> On Mon, 2011-11-28 at 11:58 +0000, Ian Jackson wrote: > > xen.org writes ("[xen-unstable test] 10135: regressions - FAIL"): > > > flight 10135 xen-unstable real [real] > > > http://www.chiark.greenend.org.uk/~xensrcts/logs/10135/ > > > > > > Regressions :-( > > > > > > Tests which did not succeed and are blocking: > > > test-amd64-i386-rhel6hvm-intel 7 redhat-install fail REGR. vs. 9956 > > > > My bisector wasn''t able to finger an exact changeset due to a blocking > > bug in the vicinity, but: > > > > pass 24183:53bec838bb08 Merge > > blocked 24184:4ecd3615e726 tools: use system installed libaio by default. > > blocked 24185:f88c745575bb docs: remove some fatally out of date ... > > fail 24186:7aa5838499d1 tools: use system libaio for blktap1 as well. > > > > Here the "system installed libaio" is that from Debian squeeze. > > This is really weird since nothing in the failings tests even exercises > this stuff. Only blktap (1 and 2) uses aio and the the tests don''t use > it and neither does qemu-dm which is the only other thing I can think > of.Turns out the test system is using the 2.6.32 kernel from xen.git and hence is using tapdisk2 for the guest cdrom drive.> I''m setting up a repro to see if I can figure out what is going > wrong.I didn''t get as far a reproing but I realised that the issue was that libaio was not installed on the target and so tapdisk was failing to run but error handling was broken so we didn''t notice apart from some messages deep in the logs. The following fixes the error handling: 8>------------------------------------------------------------- # HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1322493402 0 # Node ID 35883c0b285c8dfce341bdd04deda53ab6db03cf # Parent f688f1c8205f90bbfbd6900ccd410382c676b47f libxl: propagate error from tap_ctl_spawn. Failure here means that a disk will not be correctly setup. I briefly scanned tools/blktap2/control.c for other goto constructs which did not set their err variable but didn''t see any others. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r f688f1c8205f -r 35883c0b285c tools/blktap2/control/tap-ctl-create.c --- a/tools/blktap2/control/tap-ctl-create.c Mon Nov 28 12:59:15 2011 +0000 +++ b/tools/blktap2/control/tap-ctl-create.c Mon Nov 28 15:16:42 2011 +0000 @@ -44,8 +44,10 @@ tap_ctl_create(const char *params, char return err; id = tap_ctl_spawn(); - if (id < 0) + if (id < 0) { + err = id; goto destroy; + } err = tap_ctl_attach(id, minor); if (err)
Ian Campbell writes ("Re: [Xen-devel] [xen-unstable test] 10135:
regressions - FAIL"):> On Mon, 2011-11-28 at 13:38 +0000, Ian Campbell wrote:
> > I''m setting up a repro to see if I can figure out what is
going
> > wrong.
>
> I didn''t get as far a reproing but I realised that the issue was
that
> libaio was not installed on the target and so tapdisk was failing to run
> but error handling was broken so we didn''t notice apart from some
> messages deep in the logs. The following fixes the error handling:
Thanks, applied. (I have also fixed the tester.)
Ian.
On Mon, 2011-11-28 at 11:58 +0000, Ian Jackson wrote:> # HG changeset patch > # User Ian Jackson <Ian.Jackson@eu.citrix.com> > # Date 1322481443 0 > # Node ID a9c67c2daf4b0181ef2581471ea920eecb495616 > # Parent 95d4e2e0bed374602b5a78ee004b057ad8715d65 > tools: Revert to our built-in aio > > These two changesets: > 24184:4ecd3615e726 tools: use system installed libaio by default. > 24186:7aa5838499d1 tools: use system libaio for blktap1 as well. > cause HVM guest installs (both Windows and Redhat) to fail on Debian > squeeze with xl. So change the default for now. > > Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>AIUI the tester is now installs libaio1 so we can revert this one? According to http://www.chiark.greenend.org.uk/~xensrcts/results/bisect.xen-unstable.test-amd64-i386-rhel6hvm-intel.redhat-install.html changeset 24227:1027e7d13d02 was tested and failed. That would have included 24206:05dd94652d8d tools/check: Add files missing from 24205:5c88358164cc 24205:5c88358164cc tools: check for libaio unless user has configured CONFIG_SYSTEM_LIBAIO=n Now I don''t think the bisector would ever have given us reason to look at this commits but if they had gone it at the same time as the new dependency this could have saved a lot of trouble tracking down the problem. In http://www.chiark.greenend.org.uk/~xensrcts/logs/10032/build-i386/4.ts-xen-build.log (one of the runs finger by the bisector which included the above commits) I can see the CHECK-BUILD runes getting run but not the CHECK-INSTALL ones. The call to ./chk install in the install target seems to have been deliberately removed in 16906:f4ee7e5793cf for reasons I don''t quite understand (I suspect because of the install-into-a-staging-dir property of install). Since you install on a different host to the build host it''s possible that the tester might need to jump through some extra hoops to cause this stuff to run there anyway. Perhaps the tester could copy tools/check over and execute "./chk install" or "make check-install" itself? The top-level install.sh does something like this, but I''m not sure you are (or should be) using it. Ian.> > diff -r 95d4e2e0bed3 -r a9c67c2daf4b Config.mk > --- a/Config.mk Fri Nov 25 20:32:05 2011 +0000 > +++ b/Config.mk Mon Nov 28 11:57:23 2011 +0000 > @@ -232,7 +232,7 @@ PYTHON_TOOLS ?= y > OCAML_TOOLS ?= y > CONFIG_MINITERM ?= n > CONFIG_LOMOUNT ?= n > -CONFIG_SYSTEM_LIBAIO ?= y > +CONFIG_SYSTEM_LIBAIO ?= n > > ifeq ($(OCAML_TOOLS),y) > OCAML_TOOLS := $(shell ocamlopt -v > /dev/null 2>&1 && echo "y" || echo "n") > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel
~Ian Campbell writes ("Re: [Xen-devel] [xen-unstable test] 10135:
regressions - FAIL"):>> According to
>
http://www.chiark.greenend.org.uk/~xensrcts/results/bisect.xen-unstable.test-amd64-i386-rhel6hvm-intel.redhat-install.html
> changeset 24227:1027e7d13d02 was tested and failed. That would have
> included
> 24206:05dd94652d8d tools/check: Add files missing from 24205:5c88358164cc
> 24205:5c88358164cc tools: check for libaio unless user has configured
CONFIG_SYSTEM_LIBAIO=n
>
> Now I don''t think the bisector would ever have given us reason to
look
> at this commits but if they had gone it at the same time as the new
> dependency this could have saved a lot of trouble tracking down the
> problem.
"gone in" I guess. I spent a while staring that trying to make sense
of it assuming it was meant to say "done it".
But, no, because the check would have passed because it''s run on the
build host and the build host has the runtime lib installed because
it''s a dependency of the dev lib.
> In
>
http://www.chiark.greenend.org.uk/~xensrcts/logs/10032/build-i386/4.ts-xen-build.log
(one of the runs finger by the bisector which included the above commits) I can
see the CHECK-BUILD runes getting run but not the CHECK-INSTALL ones.
>
> The call to ./chk install in the install target seems to have been
> deliberately removed in 16906:f4ee7e5793cf for reasons I don''t
quite
> understand (I suspect because of the install-into-a-staging-dir property
> of install). Since you install on a different host to the build host
> it''s possible that the tester might need to jump through some
extra
> hoops to cause this stuff to run there anyway. Perhaps the tester could
> copy tools/check over and execute "./chk install" or "make
> check-install" itself? The top-level install.sh does something like
> this, but I''m not sure you are (or should be) using it.
I could have the tester run ./chk install on the install host and
see. But I don''t think we promise that this will work.
Running ./chk install on the build host during "make install" is a
good idea but wouldn''t have found this problem more quickly.
Ian.
On Wed, 2011-11-30 at 14:35 +0000, Ian Jackson wrote:> ~Ian Campbell writes ("Re: [Xen-devel] [xen-unstable test] 10135: regressions - FAIL"): > >> According to > > http://www.chiark.greenend.org.uk/~xensrcts/results/bisect.xen-unstable.test-amd64-i386-rhel6hvm-intel.redhat-install.html > > changeset 24227:1027e7d13d02 was tested and failed. That would have > > included > > 24206:05dd94652d8d tools/check: Add files missing from 24205:5c88358164cc > > 24205:5c88358164cc tools: check for libaio unless user has configured CONFIG_SYSTEM_LIBAIO=n > > > > Now I don''t think the bisector would ever have given us reason to look > > at this commits but if they had gone it at the same time as the new > > dependency this could have saved a lot of trouble tracking down the > > problem. > > "gone in" I guess. I spent a while staring that trying to make sense > of it assuming it was meant to say "done it".Yes, sorry. "... if they had gone in at the ... "> But, no, because the check would have passed because it''s run on the > build host and the build host has the runtime lib installed because > it''s a dependency of the dev lib. > > > In > > http://www.chiark.greenend.org.uk/~xensrcts/logs/10032/build-i386/4.ts-xen-build.log (one of the runs finger by the bisector which included the above commits) I can see the CHECK-BUILD runes getting run but not the CHECK-INSTALL ones. > > > > The call to ./chk install in the install target seems to have been > > deliberately removed in 16906:f4ee7e5793cf for reasons I don''t quite > > understand (I suspect because of the install-into-a-staging-dir property > > of install). Since you install on a different host to the build host > > it''s possible that the tester might need to jump through some extra > > hoops to cause this stuff to run there anyway. Perhaps the tester could > > copy tools/check over and execute "./chk install" or "make > > check-install" itself? The top-level install.sh does something like > > this, but I''m not sure you are (or should be) using it. > > I could have the tester run ./chk install on the install host and > see. But I don''t think we promise that this will work.We could decide to promise it going forward if it is useful.> Running ./chk install on the build host during "make install" is a > good idea but wouldn''t have found this problem more quickly.Right, if they are running on the same host I don''t think ./chk install will find much which is not found by ./chk build in practice. Ian.
Ian Campbell writes ("Re: [Xen-devel] [xen-unstable test] 10135:
regressions - FAIL"):> AIUI the tester is now installs libaio1 so we can revert this one?
Yes, I''ve done so.
# HG changeset patch
# User Ian Jackson <Ian.Jackson@eu.citrix.com>
# Date 1322756338 0
# Node ID a07e45d2aa7f792674b6325b11a48d297fd4d075
# Parent 7b180380681db3382f7c2f38e096bc551aa301d2
tools: Switch to system libaio (again)
The test system problems which prompted 24233:a9c67c2daf4b (itself a
tiny partial revert of 24184:4ecd3615e726 / 24186:7aa5838499d1) have
now been resolved, we think. So revert 24233.
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
diff -r 7b180380681d -r a07e45d2aa7f Config.mk
--- a/Config.mk Thu Dec 01 16:05:51 2011 +0000
+++ b/Config.mk Thu Dec 01 16:18:58 2011 +0000
@@ -232,7 +232,7 @@ PYTHON_TOOLS ?= y
OCAML_TOOLS ?= y
CONFIG_MINITERM ?= n
CONFIG_LOMOUNT ?= n
-CONFIG_SYSTEM_LIBAIO ?= n
+CONFIG_SYSTEM_LIBAIO ?= y
ifeq ($(OCAML_TOOLS),y)
OCAML_TOOLS := $(shell ocamlopt -v > /dev/null 2>&1 && echo
"y" || echo "n")