flight 9805 xen-4.1-testing real [real]
http://www.chiark.greenend.org.uk/~xensrcts/logs/9805/
Regressions :-(
Tests which did not succeed and are blocking:
 test-i386-i386-pv             5 xen-boot                   fail REGR. vs. 9756
 test-amd64-amd64-xl-sedf      5 xen-boot                   fail REGR. vs. 9756
 test-amd64-amd64-pv           5 xen-boot                   fail REGR. vs. 9756
 test-amd64-i386-xl-multivcpu  5 xen-boot                   fail REGR. vs. 9756
 test-amd64-i386-rhel6hvm-intel  5 xen-boot                 fail REGR. vs. 9756
 test-amd64-i386-pv            5 xen-boot                   fail REGR. vs. 9756
 test-amd64-amd64-xl-pcipt-intel  5 xen-boot                fail REGR. vs. 9756
 test-i386-i386-xl             5 xen-boot                   fail REGR. vs. 9756
 test-amd64-i386-xl-credit2    5 xen-boot                   fail REGR. vs. 9756
 test-amd64-i386-rhel6hvm-amd  5 xen-boot                   fail REGR. vs. 9756
 test-amd64-i386-xl            5 xen-boot                   fail REGR. vs. 9756
 test-amd64-amd64-xl           5 xen-boot                   fail REGR. vs. 9756
 test-amd64-i386-xl-win-vcpus1  5 xen-boot                  fail REGR. vs. 9756
 test-amd64-amd64-win          5 xen-boot                   fail REGR. vs. 9756
 test-i386-i386-xl-win         5 xen-boot                   fail REGR. vs. 9756
 test-amd64-amd64-pair         8 xen-boot/dst_host          fail REGR. vs. 9756
 test-amd64-amd64-pair         7 xen-boot/src_host          fail REGR. vs. 9756
 test-i386-i386-pair           7 xen-boot/src_host          fail REGR. vs. 9756
 test-i386-i386-pair           8 xen-boot/dst_host          fail REGR. vs. 9756
 test-amd64-i386-pair          8 xen-boot/dst_host          fail REGR. vs. 9756
 test-amd64-i386-pair          7 xen-boot/src_host          fail REGR. vs. 9756
 test-amd64-i386-win-vcpus1    5 xen-boot                   fail REGR. vs. 9756
 test-amd64-amd64-xl-win       5 xen-boot                   fail REGR. vs. 9756
 test-amd64-i386-win           5 xen-boot                   fail REGR. vs. 9756
 test-i386-i386-win            5 xen-boot                   fail REGR. vs. 9756
version targeted for testing:
 xen                  51f58b210447
baseline version:
 xen                  9702967e89dd
------------------------------------------------------------
People who touched revisions under test:
  Andrew Cooper <andrew.cooper3@citrix.com>
  Gianluca Guida <gianluca.guida@citrix.com>
  Ian Campbell <ian.campbell@citrix.com>
  Jan Beulich <jbeulich@suse.com>
  Keir Fraser <keir@xen.org>
  Stefano Stabellini <stefano.stabellini@eu.citrix.com>
------------------------------------------------------------
jobs:
 build-amd64                                                  pass    
 build-i386                                                   pass    
 build-amd64-oldkern                                          pass    
 build-i386-oldkern                                           pass    
 build-amd64-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          fail    
 test-amd64-i386-xl                                           fail    
 test-i386-i386-xl                                            fail    
 test-amd64-i386-rhel6hvm-amd                                 fail    
 test-amd64-i386-xl-credit2                                   fail    
 test-amd64-amd64-xl-pcipt-intel                              fail    
 test-amd64-i386-rhel6hvm-intel                               fail    
 test-amd64-i386-xl-multivcpu                                 fail    
 test-amd64-amd64-pair                                        fail    
 test-amd64-i386-pair                                         fail    
 test-i386-i386-pair                                          fail    
 test-amd64-amd64-pv                                          fail    
 test-amd64-i386-pv                                           fail    
 test-i386-i386-pv                                            fail    
 test-amd64-amd64-xl-sedf                                     fail    
 test-amd64-i386-win-vcpus1                                   fail    
 test-amd64-i386-xl-win-vcpus1                                fail    
 test-amd64-amd64-win                                         fail    
 test-amd64-i386-win                                          fail    
 test-i386-i386-win                                           fail    
 test-amd64-amd64-xl-win                                      fail    
 test-i386-i386-xl-win                                        fail    
------------------------------------------------------------
sg-report-flight on woking.cam.xci-test.com
logs: /home/xc_osstest/logs
images: /home/xc_osstest/images
Logs, config files, etc. are available at
    http://www.chiark.greenend.org.uk/~xensrcts/logs
Test harness code can be found at
    http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary
Not pushing.
------------------------------------------------------------
changeset:   23186:51f58b210447
tag:         tip
user:        Jan Beulich <jbeulich@suse.com>
date:        Wed Nov 16 16:39:02 2011 +0000
    
    x86-64/test_x86_emulate: fix blowfish test
    
    Incorrect register usage in the _start() wrapper caused the 64-bit
    execution emulation to fail.
    
    Signed-off-by: Jan Beulich <jbeulich@suse.com>
    Committed-by: Keir Fraser <keir@xen.org>
    xen-unstable changeset:   24149:753b21aa4087
    xen-unstable date:        Wed Nov 16 15:20:25 2011 +0000
    
    
changeset:   23185:efd132eee40c
user:        Gianluca Guida <gianluca.guida@citrix.com>
date:        Wed Nov 16 16:38:27 2011 +0000
    
    [shadow] Disable higher level pagetables early unshadow only when the
"process dying" hypercall is used.
    
    This patch fixes a performance problem in fully virtualized guests.
    
    Signed-off-by: Gianluca Guida <gianluca.guida@citrix.com>
    Tested-by: Jan Beulich <jbeulich@suse.com>
    Committed-by: Keir Fraser <keir@xen.org>
    xen-unstable changeset:   24148:3ecc8fef4281
    xen-unstable date:        Wed Nov 16 15:19:33 2011 +0000
    
    
changeset:   23184:4a91cf045893
user:        Ian Campbell <ian.campbell@citrix.com>
date:        Wed Nov 16 16:37:49 2011 +0000
    
    xen: avoid crash enabling turbo mode
    
    On a system which has not had P-state information pushed down into the
    hypervisor running "xenpm enable-turbo-mode" will reliably crash
the
    host.
    
    Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
    Committed-by: Jan Beulich <jbeulich@suse.com>
    xen-unstable changeset:   24144:cd3ef25f207a
    xen-unstable date:        Tue Nov 15 14:24:38 2011 +0100
    Committed-by: Keir Fraser <keir@xen.org>
    
    
changeset:   23183:98ba0aceaf30
user:        Stefano Stabellini <stefano.stabellini@eu.citrix.com>
date:        Wed Nov 16 16:33:58 2011 +0000
    
    x86: re-inject emulated level pirqs in PV on HVM guests if still asserted
    
    PV on HVM guests can loose level interrupts coming from emulated
    devices if they have been remapped onto event channels.  The reason is
    that we are missing the code to inject a pirq again in the guest when
    the guest EOIs it, if it corresponds to an emulated level interrupt
    and the interrupt is still asserted.
    
    Fix this issue and also return error when the guest tries to get the
    irq_status of a non-existing pirq.
    
    Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
    Committed-by: Keir Fraser <keir@xen.org>
    xen-unstable changeset:   24007:0526644ad2a6
    xen-unstable date:        Thu Oct 27 16:07:18 2011 +0100
    
    
changeset:   23182:9702967e89dd
user:        Andrew Cooper <andrew.cooper3@citrix.com>
date:        Sat Nov 12 16:14:02 2011 +0000
    
    Revert c/s 23666:b96f8bdcaa15 KEXEC: disconnect all PCI devices from the PCI
bus on crash
    
    It turns out that this causes all mannor of problems on certain
    motherboards (so far with no pattern I can discern)
    
    Problems include:
    * Hanging forever checking hlt instruction.
    * Panics when trying to change switch root device
    * Drivers hanging when trying to check for interrupts.
    
    From: Andrew Cooper <andrew.cooper3@citrix.com>
    Signed-off-by: Keir Fraser <keir@xen.org>
    Committed-by: Keir Fraser <keir@xen.org>
    xen-unstable changeset:   24137:0844b17df7a9
    xen-unstable date:        Fri Nov 11 18:14:35 2011 +0000
    
    
(qemu changes not included)
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Jan Beulich
2011-Nov-17  08:32 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
>>> On 16.11.11 at 23:23, xen.org <ian.jackson@eu.citrix.com> wrote: > flight 9805 xen-4.1-testing real [real] > http://www.chiark.greenend.org.uk/~xensrcts/logs/9805/ > > Regressions :-( > > Tests which did not succeed and are blocking: > test-i386-i386-pv 5 xen-boot fail REGR. vs. 9756 > test-amd64-amd64-xl-sedf 5 xen-boot fail REGR. vs. 9756 > test-amd64-amd64-pv 5 xen-boot fail REGR. vs. 9756 > test-amd64-i386-xl-multivcpu 5 xen-boot fail REGR. vs. 9756 > test-amd64-i386-rhel6hvm-intel 5 xen-boot fail REGR. vs. 9756 > test-amd64-i386-pv 5 xen-boot fail REGR. vs. 9756 > test-amd64-amd64-xl-pcipt-intel 5 xen-boot fail REGR. vs. 9756 > test-i386-i386-xl 5 xen-boot fail REGR. vs. 9756 > test-amd64-i386-xl-credit2 5 xen-boot fail REGR. vs. 9756 > test-amd64-i386-rhel6hvm-amd 5 xen-boot fail REGR. vs. 9756 > test-amd64-i386-xl 5 xen-boot fail REGR. vs. 9756 > test-amd64-amd64-xl 5 xen-boot fail REGR. vs. 9756 > test-amd64-i386-xl-win-vcpus1 5 xen-boot fail REGR. vs. 9756 > test-amd64-amd64-win 5 xen-boot fail REGR. vs. 9756 > test-i386-i386-xl-win 5 xen-boot fail REGR. vs. 9756 > test-amd64-amd64-pair 8 xen-boot/dst_host fail REGR. vs. 9756 > test-amd64-amd64-pair 7 xen-boot/src_host fail REGR. vs. 9756 > test-i386-i386-pair 7 xen-boot/src_host fail REGR. vs. 9756 > test-i386-i386-pair 8 xen-boot/dst_host fail REGR. vs. 9756 > test-amd64-i386-pair 8 xen-boot/dst_host fail REGR. vs. 9756 > test-amd64-i386-pair 7 xen-boot/src_host fail REGR. vs. 9756 > test-amd64-i386-win-vcpus1 5 xen-boot fail REGR. vs. 9756 > test-amd64-amd64-xl-win 5 xen-boot fail REGR. vs. 9756 > test-amd64-i386-win 5 xen-boot fail REGR. vs. 9756 > test-i386-i386-win 5 xen-boot fail REGR. vs. 9756This is due to a bad backport of c/s 24007:0526644ad2a6: In -unstable, evtchn_unmask() must be called with d->event_lock held, while in 4.1 the function acquires the lock (and now gets called with the lock already held from do_physdev_op()''s case PHYSDEVOP_eoi). The change dates back to 23573:584c2e5e03d9, which hardly is a candidate for backporting (but maybe the locking change needs to be pulled out of there). Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2011-Nov-17  09:06 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
On 17/11/2011 08:32, "Jan Beulich" <JBeulich@suse.com> wrote:> This is due to a bad backport of c/s 24007:0526644ad2a6: In -unstable, > evtchn_unmask() must be called with d->event_lock held, while in 4.1 > the function acquires the lock (and now gets called with the lock already > held from do_physdev_op()''s case PHYSDEVOP_eoi). The change dates > back to 23573:584c2e5e03d9, which hardly is a candidate for backporting > (but maybe the locking change needs to be pulled out of there).Interestingly, Ubuntu''s 4.1 fix has exactly the same problem. I''ll revert the patch until someone has time to actually implement and test a working patch against our vanilla 4.1-testing tree. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefan Bader
2011-Nov-17  09:57 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
On 17.11.2011 10:06, Keir Fraser wrote:> On 17/11/2011 08:32, "Jan Beulich" <JBeulich@suse.com> wrote: > >> This is due to a bad backport of c/s 24007:0526644ad2a6: In -unstable, >> evtchn_unmask() must be called with d->event_lock held, while in 4.1 >> the function acquires the lock (and now gets called with the lock already >> held from do_physdev_op()''s case PHYSDEVOP_eoi). The change dates >> back to 23573:584c2e5e03d9, which hardly is a candidate for backporting >> (but maybe the locking change needs to be pulled out of there). > > Interestingly, Ubuntu''s 4.1 fix has exactly the same problem. >Hm, yes we should. I am pretty sure I hit that code path often enough, Wonder why I never saw any dead lock there... --Stefan> I''ll revert the patch until someone has time to actually implement and test > a working patch against our vanilla 4.1-testing tree. > > -- Keir > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2011-Nov-17  10:13 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
On 17/11/2011 09:57, "Stefan Bader" <stefan.bader@canonical.com> wrote:>>> This is due to a bad backport of c/s 24007:0526644ad2a6: In -unstable, >>> evtchn_unmask() must be called with d->event_lock held, while in 4.1 >>> the function acquires the lock (and now gets called with the lock already >>> held from do_physdev_op()''s case PHYSDEVOP_eoi). The change dates >>> back to 23573:584c2e5e03d9, which hardly is a candidate for backporting >>> (but maybe the locking change needs to be pulled out of there). >> >> Interestingly, Ubuntu''s 4.1 fix has exactly the same problem. >> > > Hm, yes we should. I am pretty sure I hit that code path often enough, Wonder > why I never saw any dead lock there...Perhaps your dom0 kernel doesn''t register a pirq_eoi_map. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefan Bader
2011-Nov-17  10:28 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
On 17.11.2011 11:13, Keir Fraser wrote:> On 17/11/2011 09:57, "Stefan Bader" <stefan.bader@canonical.com> wrote: > >>>> This is due to a bad backport of c/s 24007:0526644ad2a6: In -unstable, >>>> evtchn_unmask() must be called with d->event_lock held, while in 4.1 >>>> the function acquires the lock (and now gets called with the lock already >>>> held from do_physdev_op()''s case PHYSDEVOP_eoi). The change dates >>>> back to 23573:584c2e5e03d9, which hardly is a candidate for backporting >>>> (but maybe the locking change needs to be pulled out of there). >>> >>> Interestingly, Ubuntu''s 4.1 fix has exactly the same problem. >>> >> >> Hm, yes we should. I am pretty sure I hit that code path often enough, Wonder >> why I never saw any dead lock there... > > Perhaps your dom0 kernel doesn''t register a pirq_eoi_map. >Would be the only explanation. And quite possible. Heck, I would need to know what that is used for anyway. :/ The kernel is 3.0 based the interrupt I was looking at just was a normal apic emulated through events one... -Stefan> -- Keir > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2011-Nov-17  10:32 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
On Thu, 17 Nov 2011, Stefan Bader wrote:> On 17.11.2011 11:13, Keir Fraser wrote: > > On 17/11/2011 09:57, "Stefan Bader" <stefan.bader@canonical.com> wrote: > > > >>>> This is due to a bad backport of c/s 24007:0526644ad2a6: In -unstable, > >>>> evtchn_unmask() must be called with d->event_lock held, while in 4.1 > >>>> the function acquires the lock (and now gets called with the lock already > >>>> held from do_physdev_op()''s case PHYSDEVOP_eoi). The change dates > >>>> back to 23573:584c2e5e03d9, which hardly is a candidate for backporting > >>>> (but maybe the locking change needs to be pulled out of there). > >>> > >>> Interestingly, Ubuntu''s 4.1 fix has exactly the same problem. > >>> > >> > >> Hm, yes we should. I am pretty sure I hit that code path often enough, Wonder > >> why I never saw any dead lock there... > > > > Perhaps your dom0 kernel doesn''t register a pirq_eoi_map. > > > Would be the only explanation. And quite possible. Heck, I would need to know > what that is used for anyway. :/ The kernel is 3.0 based the interrupt I was > looking at just was a normal apic emulated through events one...Same here, I don''t have any dom0 kernels that support it. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2011-Nov-17  10:37 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
On 17/11/2011 10:28, "Stefan Bader" <stefan.bader@canonical.com> wrote:>>> Hm, yes we should. I am pretty sure I hit that code path often enough, >>> Wonder >>> why I never saw any dead lock there... >> >> Perhaps your dom0 kernel doesn''t register a pirq_eoi_map. >> > Would be the only explanation. And quite possible. Heck, I would need to know > what that is used for anyway. :/ The kernel is 3.0 based the interrupt I was > looking at just was a normal apic emulated through events one...Our automated tests still use 2.6.32. It wouldn''t surprise me if upstream Linux 3 doesn''t have the pirq_eoi_map stuff; it''s an optimisation rather than core absolutely-required functionality. It''s the dom0 PV kernel that matters in this case by the way, not the kernel that you are running in HVM mode as a domU. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2011-Nov-17  10:43 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
On Thu, 17 Nov 2011, Keir Fraser wrote:> On 17/11/2011 10:28, "Stefan Bader" <stefan.bader@canonical.com> wrote: > > >>> Hm, yes we should. I am pretty sure I hit that code path often enough, > >>> Wonder > >>> why I never saw any dead lock there... > >> > >> Perhaps your dom0 kernel doesn''t register a pirq_eoi_map. > >> > > Would be the only explanation. And quite possible. Heck, I would need to know > > what that is used for anyway. :/ The kernel is 3.0 based the interrupt I was > > looking at just was a normal apic emulated through events one... > > Our automated tests still use 2.6.32. It wouldn''t surprise me if upstream > Linux 3 doesn''t have the pirq_eoi_map stuff; it''s an optimisation rather > than core absolutely-required functionality. > > It''s the dom0 PV kernel that matters in this case by the way, not the kernel > that you are running in HVM mode as a domU.In any case considering that the PHYSDEVOP_eoi case wasn''t protected by a lock before this patch, we could just add the spinlock around the new code only, to protect accesses to the pirq_emuirq array. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Sander Eikelenboom
2011-Nov-17  11:32 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
Hello Keir, Thursday, November 17, 2011, 11:37:01 AM, you wrote:> On 17/11/2011 10:28, "Stefan Bader" <stefan.bader@canonical.com> wrote:>>>> Hm, yes we should. I am pretty sure I hit that code path often enough, >>>> Wonder >>>> why I never saw any dead lock there... >>> >>> Perhaps your dom0 kernel doesn''t register a pirq_eoi_map. >>> >> Would be the only explanation. And quite possible. Heck, I would need to know >> what that is used for anyway. :/ The kernel is 3.0 based the interrupt I was >> looking at just was a normal apic emulated through events one...> Our automated tests still use 2.6.32. It wouldn''t surprise me if upstream > Linux 3 doesn''t have the pirq_eoi_map stuff; it''s an optimisation rather > than core absolutely-required functionality.Don''t know how much room (time wise) the machines that do the automated testing have, but additional tests based on latest 3.x and/or konrad''s linux-next branch should perhaps be considered ? Especially since they upstream path is getting more and more feature complete and usable. -- Sander> It''s the dom0 PV kernel that matters in this case by the way, not the kernel > that you are running in HVM mode as a domU.> -- Keir-- Best regards, Sander mailto:linux@eikelenboom.it _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Jackson
2011-Nov-17  12:42 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
Sander Eikelenboom writes ("Re: [Xen-devel] [xen-4.1-testing test] 9805:
regressions - FAIL"):> Don''t know how much room (time wise) the machines that do the
automated testing have, but additional tests based on latest 3.x and/or
konrad''s linux-next branch should perhaps be considered ?
Yes.  This is right at the top of my todo list, except for all the
other things that are even more urgent/important...
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Pasi Kärkkäinen
2011-Nov-17  20:03 UTC
Re: [Xen-devel] [xen-4.1-testing test] 9805: regressions - FAIL
On Thu, Nov 17, 2011 at 10:37:01AM +0000, Keir Fraser wrote:> On 17/11/2011 10:28, "Stefan Bader" <stefan.bader@canonical.com> wrote: > > >>> Hm, yes we should. I am pretty sure I hit that code path often enough, > >>> Wonder > >>> why I never saw any dead lock there... > >> > >> Perhaps your dom0 kernel doesn''t register a pirq_eoi_map. > >> > > Would be the only explanation. And quite possible. Heck, I would need to know > > what that is used for anyway. :/ The kernel is 3.0 based the interrupt I was > > looking at just was a normal apic emulated through events one... > > Our automated tests still use 2.6.32. It wouldn''t surprise me if upstream > Linux 3 doesn''t have the pirq_eoi_map stuff; it''s an optimisation rather > than core absolutely-required functionality. > > It''s the dom0 PV kernel that matters in this case by the way, not the kernel > that you are running in HVM mode as a domU. >Yep, I was testing with Linux 3.1 dom0 kernel, and there this patch worked OK without problems. -- Pasi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel