flight 9805 xen-4.1-testing real [real] http://www.chiark.greenend.org.uk/~xensrcts/logs/9805/ Regressions :-( Tests which did not succeed and are blocking: test-i386-i386-pv 5 xen-boot fail REGR. vs. 9756 test-amd64-amd64-xl-sedf 5 xen-boot fail REGR. vs. 9756 test-amd64-amd64-pv 5 xen-boot fail REGR. vs. 9756 test-amd64-i386-xl-multivcpu 5 xen-boot fail REGR. vs. 9756 test-amd64-i386-rhel6hvm-intel 5 xen-boot fail REGR. vs. 9756 test-amd64-i386-pv 5 xen-boot fail REGR. vs. 9756 test-amd64-amd64-xl-pcipt-intel 5 xen-boot fail REGR. vs. 9756 test-i386-i386-xl 5 xen-boot fail REGR. vs. 9756 test-amd64-i386-xl-credit2 5 xen-boot fail REGR. vs. 9756 test-amd64-i386-rhel6hvm-amd 5 xen-boot fail REGR. vs. 9756 test-amd64-i386-xl 5 xen-boot fail REGR. vs. 9756 test-amd64-amd64-xl 5 xen-boot fail REGR. vs. 9756 test-amd64-i386-xl-win-vcpus1 5 xen-boot fail REGR. vs. 9756 test-amd64-amd64-win 5 xen-boot fail REGR. vs. 9756 test-i386-i386-xl-win 5 xen-boot fail REGR. vs. 9756 test-amd64-amd64-pair 8 xen-boot/dst_host fail REGR. vs. 9756 test-amd64-amd64-pair 7 xen-boot/src_host fail REGR. vs. 9756 test-i386-i386-pair 7 xen-boot/src_host fail REGR. vs. 9756 test-i386-i386-pair 8 xen-boot/dst_host fail REGR. vs. 9756 test-amd64-i386-pair 8 xen-boot/dst_host fail REGR. vs. 9756 test-amd64-i386-pair 7 xen-boot/src_host fail REGR. vs. 9756 test-amd64-i386-win-vcpus1 5 xen-boot fail REGR. vs. 9756 test-amd64-amd64-xl-win 5 xen-boot fail REGR. vs. 9756 test-amd64-i386-win 5 xen-boot fail REGR. vs. 9756 test-i386-i386-win 5 xen-boot fail REGR. vs. 9756 version targeted for testing: xen 51f58b210447 baseline version: xen 9702967e89dd ------------------------------------------------------------ People who touched revisions under test: Andrew Cooper <andrew.cooper3@citrix.com> Gianluca Guida <gianluca.guida@citrix.com> Ian Campbell <ian.campbell@citrix.com> Jan Beulich <jbeulich@suse.com> Keir Fraser <keir@xen.org> Stefano Stabellini <stefano.stabellini@eu.citrix.com> ------------------------------------------------------------ jobs: build-amd64 pass build-i386 pass build-amd64-oldkern pass build-i386-oldkern pass build-amd64-pvops pass build-i386-pvops pass test-amd64-amd64-xl fail test-amd64-i386-xl fail test-i386-i386-xl fail test-amd64-i386-rhel6hvm-amd fail test-amd64-i386-xl-credit2 fail test-amd64-amd64-xl-pcipt-intel fail test-amd64-i386-rhel6hvm-intel fail test-amd64-i386-xl-multivcpu fail test-amd64-amd64-pair fail test-amd64-i386-pair fail test-i386-i386-pair fail test-amd64-amd64-pv fail test-amd64-i386-pv fail test-i386-i386-pv fail test-amd64-amd64-xl-sedf fail test-amd64-i386-win-vcpus1 fail test-amd64-i386-xl-win-vcpus1 fail test-amd64-amd64-win fail test-amd64-i386-win fail test-i386-i386-win fail test-amd64-amd64-xl-win fail test-i386-i386-xl-win fail ------------------------------------------------------------ sg-report-flight on woking.cam.xci-test.com logs: /home/xc_osstest/logs images: /home/xc_osstest/images Logs, config files, etc. are available at http://www.chiark.greenend.org.uk/~xensrcts/logs Test harness code can be found at http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary Not pushing. ------------------------------------------------------------ changeset: 23186:51f58b210447 tag: tip user: Jan Beulich <jbeulich@suse.com> date: Wed Nov 16 16:39:02 2011 +0000 x86-64/test_x86_emulate: fix blowfish test Incorrect register usage in the _start() wrapper caused the 64-bit execution emulation to fail. Signed-off-by: Jan Beulich <jbeulich@suse.com> Committed-by: Keir Fraser <keir@xen.org> xen-unstable changeset: 24149:753b21aa4087 xen-unstable date: Wed Nov 16 15:20:25 2011 +0000 changeset: 23185:efd132eee40c user: Gianluca Guida <gianluca.guida@citrix.com> date: Wed Nov 16 16:38:27 2011 +0000 [shadow] Disable higher level pagetables early unshadow only when the "process dying" hypercall is used. This patch fixes a performance problem in fully virtualized guests. Signed-off-by: Gianluca Guida <gianluca.guida@citrix.com> Tested-by: Jan Beulich <jbeulich@suse.com> Committed-by: Keir Fraser <keir@xen.org> xen-unstable changeset: 24148:3ecc8fef4281 xen-unstable date: Wed Nov 16 15:19:33 2011 +0000 changeset: 23184:4a91cf045893 user: Ian Campbell <ian.campbell@citrix.com> date: Wed Nov 16 16:37:49 2011 +0000 xen: avoid crash enabling turbo mode On a system which has not had P-state information pushed down into the hypervisor running "xenpm enable-turbo-mode" will reliably crash the host. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Committed-by: Jan Beulich <jbeulich@suse.com> xen-unstable changeset: 24144:cd3ef25f207a xen-unstable date: Tue Nov 15 14:24:38 2011 +0100 Committed-by: Keir Fraser <keir@xen.org> changeset: 23183:98ba0aceaf30 user: Stefano Stabellini <stefano.stabellini@eu.citrix.com> date: Wed Nov 16 16:33:58 2011 +0000 x86: re-inject emulated level pirqs in PV on HVM guests if still asserted PV on HVM guests can loose level interrupts coming from emulated devices if they have been remapped onto event channels. The reason is that we are missing the code to inject a pirq again in the guest when the guest EOIs it, if it corresponds to an emulated level interrupt and the interrupt is still asserted. Fix this issue and also return error when the guest tries to get the irq_status of a non-existing pirq. Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> Committed-by: Keir Fraser <keir@xen.org> xen-unstable changeset: 24007:0526644ad2a6 xen-unstable date: Thu Oct 27 16:07:18 2011 +0100 changeset: 23182:9702967e89dd user: Andrew Cooper <andrew.cooper3@citrix.com> date: Sat Nov 12 16:14:02 2011 +0000 Revert c/s 23666:b96f8bdcaa15 KEXEC: disconnect all PCI devices from the PCI bus on crash It turns out that this causes all mannor of problems on certain motherboards (so far with no pattern I can discern) Problems include: * Hanging forever checking hlt instruction. * Panics when trying to change switch root device * Drivers hanging when trying to check for interrupts. From: Andrew Cooper <andrew.cooper3@citrix.com> Signed-off-by: Keir Fraser <keir@xen.org> Committed-by: Keir Fraser <keir@xen.org> xen-unstable changeset: 24137:0844b17df7a9 xen-unstable date: Fri Nov 11 18:14:35 2011 +0000 (qemu changes not included) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Beulich
2011-Nov-17 08:32 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
>>> On 16.11.11 at 23:23, xen.org <ian.jackson@eu.citrix.com> wrote: > flight 9805 xen-4.1-testing real [real] > http://www.chiark.greenend.org.uk/~xensrcts/logs/9805/ > > Regressions :-( > > Tests which did not succeed and are blocking: > test-i386-i386-pv 5 xen-boot fail REGR. vs. 9756 > test-amd64-amd64-xl-sedf 5 xen-boot fail REGR. vs. 9756 > test-amd64-amd64-pv 5 xen-boot fail REGR. vs. 9756 > test-amd64-i386-xl-multivcpu 5 xen-boot fail REGR. vs. 9756 > test-amd64-i386-rhel6hvm-intel 5 xen-boot fail REGR. vs. 9756 > test-amd64-i386-pv 5 xen-boot fail REGR. vs. 9756 > test-amd64-amd64-xl-pcipt-intel 5 xen-boot fail REGR. vs. 9756 > test-i386-i386-xl 5 xen-boot fail REGR. vs. 9756 > test-amd64-i386-xl-credit2 5 xen-boot fail REGR. vs. 9756 > test-amd64-i386-rhel6hvm-amd 5 xen-boot fail REGR. vs. 9756 > test-amd64-i386-xl 5 xen-boot fail REGR. vs. 9756 > test-amd64-amd64-xl 5 xen-boot fail REGR. vs. 9756 > test-amd64-i386-xl-win-vcpus1 5 xen-boot fail REGR. vs. 9756 > test-amd64-amd64-win 5 xen-boot fail REGR. vs. 9756 > test-i386-i386-xl-win 5 xen-boot fail REGR. vs. 9756 > test-amd64-amd64-pair 8 xen-boot/dst_host fail REGR. vs. 9756 > test-amd64-amd64-pair 7 xen-boot/src_host fail REGR. vs. 9756 > test-i386-i386-pair 7 xen-boot/src_host fail REGR. vs. 9756 > test-i386-i386-pair 8 xen-boot/dst_host fail REGR. vs. 9756 > test-amd64-i386-pair 8 xen-boot/dst_host fail REGR. vs. 9756 > test-amd64-i386-pair 7 xen-boot/src_host fail REGR. vs. 9756 > test-amd64-i386-win-vcpus1 5 xen-boot fail REGR. vs. 9756 > test-amd64-amd64-xl-win 5 xen-boot fail REGR. vs. 9756 > test-amd64-i386-win 5 xen-boot fail REGR. vs. 9756 > test-i386-i386-win 5 xen-boot fail REGR. vs. 9756This is due to a bad backport of c/s 24007:0526644ad2a6: In -unstable, evtchn_unmask() must be called with d->event_lock held, while in 4.1 the function acquires the lock (and now gets called with the lock already held from do_physdev_op()''s case PHYSDEVOP_eoi). The change dates back to 23573:584c2e5e03d9, which hardly is a candidate for backporting (but maybe the locking change needs to be pulled out of there). Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2011-Nov-17 09:06 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
On 17/11/2011 08:32, "Jan Beulich" <JBeulich@suse.com> wrote:> This is due to a bad backport of c/s 24007:0526644ad2a6: In -unstable, > evtchn_unmask() must be called with d->event_lock held, while in 4.1 > the function acquires the lock (and now gets called with the lock already > held from do_physdev_op()''s case PHYSDEVOP_eoi). The change dates > back to 23573:584c2e5e03d9, which hardly is a candidate for backporting > (but maybe the locking change needs to be pulled out of there).Interestingly, Ubuntu''s 4.1 fix has exactly the same problem. I''ll revert the patch until someone has time to actually implement and test a working patch against our vanilla 4.1-testing tree. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefan Bader
2011-Nov-17 09:57 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
On 17.11.2011 10:06, Keir Fraser wrote:> On 17/11/2011 08:32, "Jan Beulich" <JBeulich@suse.com> wrote: > >> This is due to a bad backport of c/s 24007:0526644ad2a6: In -unstable, >> evtchn_unmask() must be called with d->event_lock held, while in 4.1 >> the function acquires the lock (and now gets called with the lock already >> held from do_physdev_op()''s case PHYSDEVOP_eoi). The change dates >> back to 23573:584c2e5e03d9, which hardly is a candidate for backporting >> (but maybe the locking change needs to be pulled out of there). > > Interestingly, Ubuntu''s 4.1 fix has exactly the same problem. >Hm, yes we should. I am pretty sure I hit that code path often enough, Wonder why I never saw any dead lock there... --Stefan> I''ll revert the patch until someone has time to actually implement and test > a working patch against our vanilla 4.1-testing tree. > > -- Keir > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2011-Nov-17 10:13 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
On 17/11/2011 09:57, "Stefan Bader" <stefan.bader@canonical.com> wrote:>>> This is due to a bad backport of c/s 24007:0526644ad2a6: In -unstable, >>> evtchn_unmask() must be called with d->event_lock held, while in 4.1 >>> the function acquires the lock (and now gets called with the lock already >>> held from do_physdev_op()''s case PHYSDEVOP_eoi). The change dates >>> back to 23573:584c2e5e03d9, which hardly is a candidate for backporting >>> (but maybe the locking change needs to be pulled out of there). >> >> Interestingly, Ubuntu''s 4.1 fix has exactly the same problem. >> > > Hm, yes we should. I am pretty sure I hit that code path often enough, Wonder > why I never saw any dead lock there...Perhaps your dom0 kernel doesn''t register a pirq_eoi_map. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefan Bader
2011-Nov-17 10:28 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
On 17.11.2011 11:13, Keir Fraser wrote:> On 17/11/2011 09:57, "Stefan Bader" <stefan.bader@canonical.com> wrote: > >>>> This is due to a bad backport of c/s 24007:0526644ad2a6: In -unstable, >>>> evtchn_unmask() must be called with d->event_lock held, while in 4.1 >>>> the function acquires the lock (and now gets called with the lock already >>>> held from do_physdev_op()''s case PHYSDEVOP_eoi). The change dates >>>> back to 23573:584c2e5e03d9, which hardly is a candidate for backporting >>>> (but maybe the locking change needs to be pulled out of there). >>> >>> Interestingly, Ubuntu''s 4.1 fix has exactly the same problem. >>> >> >> Hm, yes we should. I am pretty sure I hit that code path often enough, Wonder >> why I never saw any dead lock there... > > Perhaps your dom0 kernel doesn''t register a pirq_eoi_map. >Would be the only explanation. And quite possible. Heck, I would need to know what that is used for anyway. :/ The kernel is 3.0 based the interrupt I was looking at just was a normal apic emulated through events one... -Stefan> -- Keir > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2011-Nov-17 10:32 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
On Thu, 17 Nov 2011, Stefan Bader wrote:> On 17.11.2011 11:13, Keir Fraser wrote: > > On 17/11/2011 09:57, "Stefan Bader" <stefan.bader@canonical.com> wrote: > > > >>>> This is due to a bad backport of c/s 24007:0526644ad2a6: In -unstable, > >>>> evtchn_unmask() must be called with d->event_lock held, while in 4.1 > >>>> the function acquires the lock (and now gets called with the lock already > >>>> held from do_physdev_op()''s case PHYSDEVOP_eoi). The change dates > >>>> back to 23573:584c2e5e03d9, which hardly is a candidate for backporting > >>>> (but maybe the locking change needs to be pulled out of there). > >>> > >>> Interestingly, Ubuntu''s 4.1 fix has exactly the same problem. > >>> > >> > >> Hm, yes we should. I am pretty sure I hit that code path often enough, Wonder > >> why I never saw any dead lock there... > > > > Perhaps your dom0 kernel doesn''t register a pirq_eoi_map. > > > Would be the only explanation. And quite possible. Heck, I would need to know > what that is used for anyway. :/ The kernel is 3.0 based the interrupt I was > looking at just was a normal apic emulated through events one...Same here, I don''t have any dom0 kernels that support it. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2011-Nov-17 10:37 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
On 17/11/2011 10:28, "Stefan Bader" <stefan.bader@canonical.com> wrote:>>> Hm, yes we should. I am pretty sure I hit that code path often enough, >>> Wonder >>> why I never saw any dead lock there... >> >> Perhaps your dom0 kernel doesn''t register a pirq_eoi_map. >> > Would be the only explanation. And quite possible. Heck, I would need to know > what that is used for anyway. :/ The kernel is 3.0 based the interrupt I was > looking at just was a normal apic emulated through events one...Our automated tests still use 2.6.32. It wouldn''t surprise me if upstream Linux 3 doesn''t have the pirq_eoi_map stuff; it''s an optimisation rather than core absolutely-required functionality. It''s the dom0 PV kernel that matters in this case by the way, not the kernel that you are running in HVM mode as a domU. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2011-Nov-17 10:43 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
On Thu, 17 Nov 2011, Keir Fraser wrote:> On 17/11/2011 10:28, "Stefan Bader" <stefan.bader@canonical.com> wrote: > > >>> Hm, yes we should. I am pretty sure I hit that code path often enough, > >>> Wonder > >>> why I never saw any dead lock there... > >> > >> Perhaps your dom0 kernel doesn''t register a pirq_eoi_map. > >> > > Would be the only explanation. And quite possible. Heck, I would need to know > > what that is used for anyway. :/ The kernel is 3.0 based the interrupt I was > > looking at just was a normal apic emulated through events one... > > Our automated tests still use 2.6.32. It wouldn''t surprise me if upstream > Linux 3 doesn''t have the pirq_eoi_map stuff; it''s an optimisation rather > than core absolutely-required functionality. > > It''s the dom0 PV kernel that matters in this case by the way, not the kernel > that you are running in HVM mode as a domU.In any case considering that the PHYSDEVOP_eoi case wasn''t protected by a lock before this patch, we could just add the spinlock around the new code only, to protect accesses to the pirq_emuirq array. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Sander Eikelenboom
2011-Nov-17 11:32 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
Hello Keir, Thursday, November 17, 2011, 11:37:01 AM, you wrote:> On 17/11/2011 10:28, "Stefan Bader" <stefan.bader@canonical.com> wrote:>>>> Hm, yes we should. I am pretty sure I hit that code path often enough, >>>> Wonder >>>> why I never saw any dead lock there... >>> >>> Perhaps your dom0 kernel doesn''t register a pirq_eoi_map. >>> >> Would be the only explanation. And quite possible. Heck, I would need to know >> what that is used for anyway. :/ The kernel is 3.0 based the interrupt I was >> looking at just was a normal apic emulated through events one...> Our automated tests still use 2.6.32. It wouldn''t surprise me if upstream > Linux 3 doesn''t have the pirq_eoi_map stuff; it''s an optimisation rather > than core absolutely-required functionality.Don''t know how much room (time wise) the machines that do the automated testing have, but additional tests based on latest 3.x and/or konrad''s linux-next branch should perhaps be considered ? Especially since they upstream path is getting more and more feature complete and usable. -- Sander> It''s the dom0 PV kernel that matters in this case by the way, not the kernel > that you are running in HVM mode as a domU.> -- Keir-- Best regards, Sander mailto:linux@eikelenboom.it _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Jackson
2011-Nov-17 12:42 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
Sander Eikelenboom writes ("Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL"):> Don''t know how much room (time wise) the machines that do the automated testing have, but additional tests based on latest 3.x and/or konrad''s linux-next branch should perhaps be considered ?Yes. This is right at the top of my todo list, except for all the other things that are even more urgent/important... Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Pasi Kärkkäinen
2011-Nov-17 20:03 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
On Thu, Nov 17, 2011 at 10:37:01AM +0000, Keir Fraser wrote:> On 17/11/2011 10:28, "Stefan Bader" <stefan.bader@canonical.com> wrote: > > >>> Hm, yes we should. I am pretty sure I hit that code path often enough, > >>> Wonder > >>> why I never saw any dead lock there... > >> > >> Perhaps your dom0 kernel doesn''t register a pirq_eoi_map. > >> > > Would be the only explanation. And quite possible. Heck, I would need to know > > what that is used for anyway. :/ The kernel is 3.0 based the interrupt I was > > looking at just was a normal apic emulated through events one... > > Our automated tests still use 2.6.32. It wouldn''t surprise me if upstream > Linux 3 doesn''t have the pirq_eoi_map stuff; it''s an optimisation rather > than core absolutely-required functionality. > > It''s the dom0 PV kernel that matters in this case by the way, not the kernel > that you are running in HVM mode as a domU. >Yep, I was testing with Linux 3.1 dom0 kernel, and there this patch worked OK without problems. -- Pasi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel