flight 21490 linux-3.10 real [real] http://www.chiark.greenend.org.uk/~xensrcts/logs/21490/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-i386-pv 18 leak-check/check fail REGR. vs. 21026 test-amd64-i386-qemut-rhel6hvm-intel 11 leak-check/check fail REGR. vs. 21026 test-amd64-amd64-pv 18 leak-check/check fail REGR. vs. 21026 test-amd64-i386-qemuu-rhel6hvm-intel 11 leak-check/check fail REGR. vs. 21026 test-amd64-amd64-xl 18 leak-check/check fail REGR. vs. 21026 test-amd64-i386-pair 20 leak-check/check/src_host fail REGR. vs. 21026 test-amd64-i386-pair 21 leak-check/check/dst_host fail REGR. vs. 21026 test-amd64-amd64-pair 21 leak-check/check/dst_host fail REGR. vs. 21026 test-amd64-amd64-pair 20 leak-check/check/src_host fail REGR. vs. 21026 test-amd64-i386-rhel6hvm-intel 11 leak-check/check fail REGR. vs. 21026 Regressions which are regarded as allowable (not blocking): test-amd64-amd64-xl-sedf 18 leak-check/check fail REGR. vs. 21026 test-amd64-i386-qemuu-rhel6hvm-amd 11 leak-check/check fail blocked in 21026 test-amd64-i386-qemut-rhel6hvm-amd 11 leak-check/check fail blocked in 21026 test-amd64-i386-xl-credit2 18 leak-check/check fail blocked in 21026 test-amd64-i386-xl-multivcpu 18 leak-check/check fail blocked in 21026 test-amd64-i386-rhel6hvm-amd 11 leak-check/check fail blocked in 21026 test-amd64-amd64-xl-sedf-pin 18 leak-check/check fail REGR. vs. 21026 test-amd64-i386-xl 18 leak-check/check fail blocked in 21026 Tests which did not succeed, but are not blocking: test-amd64-amd64-xl-pcipt-intel 9 guest-start fail never pass test-amd64-i386-xend-winxpsp3 16 leak-check/check fail never pass test-amd64-i386-xl-qemut-winxpsp3-vcpus1 13 guest-stop fail never pass test-amd64-i386-xend-qemut-winxpsp3 16 leak-check/check fail never pass test-amd64-i386-xl-win7-amd64 13 guest-stop fail never pass test-amd64-amd64-xl-qemuu-winxpsp3 13 guest-stop fail never pass test-amd64-i386-xl-winxpsp3-vcpus1 13 guest-stop fail never pass test-amd64-amd64-xl-winxpsp3 13 guest-stop fail never pass test-amd64-amd64-xl-win7-amd64 13 guest-stop fail never pass test-amd64-amd64-xl-qemut-win7-amd64 13 guest-stop fail never pass test-amd64-amd64-xl-qemuu-win7-amd64 13 guest-stop fail never pass test-amd64-i386-xl-qemut-win7-amd64 13 guest-stop fail never pass test-amd64-amd64-xl-qemut-winxpsp3 13 guest-stop fail never pass version targeted for testing: linux 4e77f7f1261f65cff06918bc5e66d02a418fc842 baseline version: linux 14e9c7db465387ede7f019c42f28c90f99fc2793 ------------------------------------------------------------ People who touched revisions under test: AKASHI Takahiro <takahiro.akashi@linaro.org> Alasdair G Kergon <agk@redhat.com> Andrew Morton <akpm@linux-foundation.org> Antonella Pellizzari <anto.pellizzari83@gmail.com> Christophe Gouault <christophe.gouault@6wind.com> Dan Carpenter <dan.carpenter@oracle.com> Dan Williams <dcbw@redhat.com> Daniel Borkmann <dborkman@redhat.com> David Henningsson <david.henningsson@canonical.com> David S. Miller <davem@davemloft.net> Diego Elio Pettenò <flameeyes@flameeyes.eu> Dmitry Kravkov <dmitry@broadcom.com> Dormando <dormando@rydia.net> Enrico Mioso <mrkiko.rs@gmail.com> Eric Dumazet <edumazet@google.com> Fabio Estevam <fabio.estevam@freescale.com> Fan Du <fan.du@windriver.com> Fengguang Wu <fengguang.wu@intel.com> François CACHEREUL <f.cachereul@alphalink.fr> Greg Kroah-Hartman <gregkh@linuxfoundation.org> Hannes Frederic Sowa <hannes@stressinduktion.org> Herbert Xu <herbert@gondor.apana.org.au> Hugh Dickins <hughd@google.com> Ilpo Järvinen <ilpo.jarvinen@helsinki.fi> James Ralston <james.d.ralston@intel.com> Jason Cooper <jason@lakedaemon.net> Jason Wang <jasowang@redhat.com> Jiri Benc <jbenc@redhat.com> Johannes Berg <johannes.berg@intel.com> Julian Anastasov <ja@ssi.bg> Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Linus Torvalds <torvalds@linux-foundation.org> Linus Walleij <linus.walleij@linaro.org> Marc Kleine-Budde <mkl@pengutronix.de> Mariusz Ceier <mceier+kernel@gmail.com> Mathias Krause <minipli@googlemail.com> Mathias Nyman <mathias.nyman@linux.intel.com> Michael S. Tsirkin <mst@redhat.com> Mika Westerberg <mika.westerberg@linux.intel.com> Mikulas Patocka <mpatocka@redhat.com> Mugunthan V N <mugunthanvnm@ti.com> Neal Cardwell <ncardwell@google.com> Neil Horman <nhorman@tuxdriver.com> Olof Johansson <olof@lixom.net> Paul Durrant <paul.durrant@citrix.com> Paul Moore <paul@paul-moore.com> Pavel Parkhomenko <ra85551@gmail.com> Roel Kluin <roel.kluin@gmail.com> Russell King <rmk+kernel@arm.linux.org.uk> Rusty Russell <rusty@rustcorp.com.au> Salva Peiró <speiro@ai2.upv.es> Sathya Perla <sathya.perla@emulex.com> Sebastian Hesselbarth <sebastian.hesselbarth@gmail.com> Seif Mazareeb <seif@marvell.com> Takashi Iwai <tiwai@suse.de> Tony Prisk <linux@prisktech.co.nz> Toralf Förster <toralf.foerster@gmx.de> Vlad Yasevich <vyasevic@redhat.com> Vlad Yasevich <vyasevich@gmail.com> Will Deacon <will.deacon@arm.com> Wolfram Sang <wsa@the-dreams.de> Yuchung Cheng <ycheng@google.com> ------------------------------------------------------------ jobs: build-amd64 pass build-i386 pass build-amd64-pvops pass build-i386-pvops pass test-amd64-amd64-xl fail test-amd64-i386-xl fail test-amd64-i386-rhel6hvm-amd fail test-amd64-i386-qemut-rhel6hvm-amd fail test-amd64-i386-qemuu-rhel6hvm-amd fail test-amd64-amd64-xl-qemut-win7-amd64 fail test-amd64-i386-xl-qemut-win7-amd64 fail test-amd64-amd64-xl-qemuu-win7-amd64 fail test-amd64-amd64-xl-win7-amd64 fail test-amd64-i386-xl-win7-amd64 fail test-amd64-i386-xl-credit2 fail test-amd64-amd64-xl-pcipt-intel fail test-amd64-i386-rhel6hvm-intel fail test-amd64-i386-qemut-rhel6hvm-intel fail test-amd64-i386-qemuu-rhel6hvm-intel fail test-amd64-i386-xl-multivcpu fail test-amd64-amd64-pair fail test-amd64-i386-pair fail test-amd64-amd64-xl-sedf-pin fail test-amd64-amd64-pv fail test-amd64-i386-pv fail test-amd64-amd64-xl-sedf fail test-amd64-i386-xl-qemut-winxpsp3-vcpus1 fail test-amd64-i386-xl-winxpsp3-vcpus1 fail test-amd64-i386-xend-qemut-winxpsp3 fail test-amd64-amd64-xl-qemut-winxpsp3 fail test-amd64-amd64-xl-qemuu-winxpsp3 fail test-amd64-i386-xend-winxpsp3 fail test-amd64-amd64-xl-winxpsp3 fail ------------------------------------------------------------ sg-report-flight on woking.cam.xci-test.com logs: /home/xc_osstest/logs images: /home/xc_osstest/images Logs, config files, etc. are available at http://www.chiark.greenend.org.uk/~xensrcts/logs Test harness code can be found at http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary Not pushing. (No revision log; it would be 1653 lines long.) --===============5393645538979071311=Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============5393645538979071311==--
On Wed, 2013-11-06 at 09:03 +0000, xen.org wrote:> flight 21490 linux-3.10 real [real] > http://www.chiark.greenend.org.uk/~xensrcts/logs/21490/ > > Regressions :-( > > Tests which did not succeed and are blocking, > including tests which could not be run: > test-amd64-i386-pv 18 leak-check/check fail REGR. vs. 21026 > test-amd64-i386-qemut-rhel6hvm-intel 11 leak-check/check fail REGR. vs. 21026 > test-amd64-amd64-pv 18 leak-check/check fail REGR. vs. 21026 > test-amd64-i386-qemuu-rhel6hvm-intel 11 leak-check/check fail REGR. vs. 21026 > test-amd64-amd64-xl 18 leak-check/check fail REGR. vs. 21026 > test-amd64-i386-pair 20 leak-check/check/src_host fail REGR. vs. 21026 > test-amd64-i386-pair 21 leak-check/check/dst_host fail REGR. vs. 21026 > test-amd64-amd64-pair 21 leak-check/check/dst_host fail REGR. vs. 21026 > test-amd64-amd64-pair 20 leak-check/check/src_host fail REGR. vs. 21026 > test-amd64-i386-rhel6hvm-intel 11 leak-check/check fail REGR. vs. 21026The bisector says, in <E1Ve0Tg-0008Kj-Il@woking.cam.xci-test.com>: *** Found and reproduced problem changeset *** Bug is in tree: linux git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git Bug introduced: f495ddc46f97dd0054ff2d5d3c7493d59f7511fb Bug not present: b9396c4c9e7f499b1dd8080e901c88705f2efa99 commit f495ddc46f97dd0054ff2d5d3c7493d59f7511fb Author: Paul Durrant <paul.durrant@citrix.com> Date: Tue Oct 8 14:22:56 2013 +0100 xen-netback: Don''t destroy the netdev until the vif is shut down http://www.chiark.greenend.org.uk/~xensrcts/logs/21490/test-amd64-amd64-xl/18.ts-leak-check.log shows: 2013-11-06 04:23:59 Z executing ssh ... root@10.80.250.32 xl list 2013-11-06 04:23:59 Z LEAKED [domain 1] domain: (null) 1 0 1 --pssd 4.6 2013-11-06 04:23:59 Z LEAKED [domain 2] domain: (null) 2 0 1 --pssd 0.9 2013-11-06 04:23:59 Z LEAKED [domain 3] domain: (null) 3 0 1 --pssd 0.5 2013-11-06 04:23:59 Z LEAKED [domain 4] domain: (null) 4 0 1 --pssd 0.9 2013-11-06 04:23:59 Z LEAKED [domain 5] domain: (null) 5 0 1 --pssd 1.1 2013-11-06 04:23:59 Z LEAKED [domain 6] domain: (null) 6 0 1 --pssd 0.4 2013-11-06 04:23:59 Z LEAKED [domain 7] domain: (null) 7 0 1 --pssd 0.4 2013-11-06 04:23:59 Z LEAKED [domain 8] domain: (null) 8 0 1 --pssd 0.4 2013-11-06 04:23:59 Z LEAKED [domain 9] domain: (null) 9 0 1 --pssd 0.4 2013-11-06 04:23:59 Z LEAKED [domain 10] domain: (null) 10 0 1 --pssd 0.4 2013-11-06 04:23:59 Z LEAKED [domain 11] domain: (null) 11 0 1 --pssd 0.4 2013-11-06 04:23:59 Z LEAKED [domain 12] domain: (null) 12 0 1 --pssd 0.4 2013-11-06 04:23:59 Z LEAKED [domain 13] domain: (null) 13 0 1 --pssd 0.4 2013-11-06 04:23:59 Z LEAKED [domain 14] domain: (null) 14 0 1 --pssd 0.4 2013-11-06 04:23:59 Z LEAKED [domain 16] domain: (null) 16 0 1 --p--d 4.7 I imagine this is because the vif isn''t being properly torn down on shutdown... ifconfig and brctl etc logs don''t show the devices living on in dom0, I expect it is failing to release the grant maps. http://www.chiark.greenend.org.uk/~xensrcts/logs/21490/test-amd64-amd64-xl/serial-fire-frog.log shows each domain still owns two pages. Which I would bet are the vif rings... Also: Nov 6 04:24:19.449065 (XEN) [ref] localdom mfn pin localdom gmfn flags Nov 6 04:24:19.449101 (XEN) grant-table for remote domain: 1 (v2) Nov 6 04:24:19.461055 (XEN) [ 10] 0 0x415f36 0x00000001 0 0x415f36 0x18 Nov 6 04:24:19.461091 (XEN) [ 11] 0 0x415f35 0x00000001 0 0x415f35 0x18 Ian.
On Wed, Nov 06, 2013 at 10:48:37AM +0000, Ian Campbell wrote:> On Wed, 2013-11-06 at 09:03 +0000, xen.org wrote: > > flight 21490 linux-3.10 real [real] > > http://www.chiark.greenend.org.uk/~xensrcts/logs/21490/ > > > > Regressions :-( > > > > Tests which did not succeed and are blocking, > > including tests which could not be run: > > test-amd64-i386-pv 18 leak-check/check fail REGR. vs. 21026 > > test-amd64-i386-qemut-rhel6hvm-intel 11 leak-check/check fail REGR. vs. 21026 > > test-amd64-amd64-pv 18 leak-check/check fail REGR. vs. 21026 > > test-amd64-i386-qemuu-rhel6hvm-intel 11 leak-check/check fail REGR. vs. 21026 > > test-amd64-amd64-xl 18 leak-check/check fail REGR. vs. 21026 > > test-amd64-i386-pair 20 leak-check/check/src_host fail REGR. vs. 21026 > > test-amd64-i386-pair 21 leak-check/check/dst_host fail REGR. vs. 21026 > > test-amd64-amd64-pair 21 leak-check/check/dst_host fail REGR. vs. 21026 > > test-amd64-amd64-pair 20 leak-check/check/src_host fail REGR. vs. 21026 > > test-amd64-i386-rhel6hvm-intel 11 leak-check/check fail REGR. vs. 21026 > > The bisector says, in <E1Ve0Tg-0008Kj-Il@woking.cam.xci-test.com>: >David posted a patch to fix this, but that patch is not in that tree yet. <1381150519-14557-1-git-send-email-david.vrabel@citrix.com> From: David Vrabel <david.vrabel@citrix.com> If a guest is destroyed without transitioning its frontend to CLOSED, the domain becomes a zombie as netback was not grant unmapping the shared rings. When removing a VIF, transition the backend to CLOSED so the VIF is disconnected if necessary (which will unmap the shared rings etc). This fixes a regression introduced by 279f438e36c0a70b23b86d2090aeec50155034a9 (xen-netback: Don''t destroy the netdev until the vif is shut down). Signed-off-by: David Vrabel <david.vrabel@citrix.com> Cc: Ian Campbell <ian.campbell@citrix.com> Cc: Wei Liu <wei.liu2@citrix.com> Cc: Paul Durrant <Paul.Durrant@citrix.com> Wei.> *** Found and reproduced problem changeset *** > > Bug is in tree: linux git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git > Bug introduced: f495ddc46f97dd0054ff2d5d3c7493d59f7511fb > Bug not present: b9396c4c9e7f499b1dd8080e901c88705f2efa99 > > > commit f495ddc46f97dd0054ff2d5d3c7493d59f7511fb > Author: Paul Durrant <paul.durrant@citrix.com> > Date: Tue Oct 8 14:22:56 2013 +0100 > > xen-netback: Don''t destroy the netdev until the vif is shut down > > http://www.chiark.greenend.org.uk/~xensrcts/logs/21490/test-amd64-amd64-xl/18.ts-leak-check.log shows: > > 2013-11-06 04:23:59 Z executing ssh ... root@10.80.250.32 xl > list > 2013-11-06 04:23:59 Z LEAKED [domain 1] domain: (null) 1 0 1 --pssd 4.6 > 2013-11-06 04:23:59 Z LEAKED [domain 2] domain: (null) 2 0 1 --pssd 0.9 > 2013-11-06 04:23:59 Z LEAKED [domain 3] domain: (null) 3 0 1 --pssd 0.5 > 2013-11-06 04:23:59 Z LEAKED [domain 4] domain: (null) 4 0 1 --pssd 0.9 > 2013-11-06 04:23:59 Z LEAKED [domain 5] domain: (null) 5 0 1 --pssd 1.1 > 2013-11-06 04:23:59 Z LEAKED [domain 6] domain: (null) 6 0 1 --pssd 0.4 > 2013-11-06 04:23:59 Z LEAKED [domain 7] domain: (null) 7 0 1 --pssd 0.4 > 2013-11-06 04:23:59 Z LEAKED [domain 8] domain: (null) 8 0 1 --pssd 0.4 > 2013-11-06 04:23:59 Z LEAKED [domain 9] domain: (null) 9 0 1 --pssd 0.4 > 2013-11-06 04:23:59 Z LEAKED [domain 10] domain: (null) 10 0 1 --pssd 0.4 > 2013-11-06 04:23:59 Z LEAKED [domain 11] domain: (null) 11 0 1 --pssd 0.4 > 2013-11-06 04:23:59 Z LEAKED [domain 12] domain: (null) 12 0 1 --pssd 0.4 > 2013-11-06 04:23:59 Z LEAKED [domain 13] domain: (null) 13 0 1 --pssd 0.4 > 2013-11-06 04:23:59 Z LEAKED [domain 14] domain: (null) 14 0 1 --pssd 0.4 > 2013-11-06 04:23:59 Z LEAKED [domain 16] domain: (null) 16 0 1 --p--d 4.7 > > I imagine this is because the vif isn''t being properly torn down on > shutdown... ifconfig and brctl etc logs don''t show the devices living on > in dom0, I expect it is failing to release the grant maps. > > http://www.chiark.greenend.org.uk/~xensrcts/logs/21490/test-amd64-amd64-xl/serial-fire-frog.log shows each domain still owns two pages. Which I would bet are the vif rings... Also: > > Nov 6 04:24:19.449065 (XEN) [ref] localdom mfn pin localdom gmfn flags > Nov 6 04:24:19.449101 (XEN) grant-table for remote domain: 1 (v2) > Nov 6 04:24:19.461055 (XEN) [ 10] 0 0x415f36 0x00000001 0 0x415f36 0x18 > Nov 6 04:24:19.461091 (XEN) [ 11] 0 0x415f35 0x00000001 0 0x415f35 0x18 > > Ian.
On Wed, 2013-11-06 at 11:15 +0000, Wei Liu wrote:> On Wed, Nov 06, 2013 at 10:48:37AM +0000, Ian Campbell wrote: > > On Wed, 2013-11-06 at 09:03 +0000, xen.org wrote: > > > flight 21490 linux-3.10 real [real] > > > http://www.chiark.greenend.org.uk/~xensrcts/logs/21490/ > > > > > > Regressions :-( > > > > > > Tests which did not succeed and are blocking, > > > including tests which could not be run: > > > test-amd64-i386-pv 18 leak-check/check fail REGR. vs. 21026 > > > test-amd64-i386-qemut-rhel6hvm-intel 11 leak-check/check fail REGR. vs. 21026 > > > test-amd64-amd64-pv 18 leak-check/check fail REGR. vs. 21026 > > > test-amd64-i386-qemuu-rhel6hvm-intel 11 leak-check/check fail REGR. vs. 21026 > > > test-amd64-amd64-xl 18 leak-check/check fail REGR. vs. 21026 > > > test-amd64-i386-pair 20 leak-check/check/src_host fail REGR. vs. 21026 > > > test-amd64-i386-pair 21 leak-check/check/dst_host fail REGR. vs. 21026 > > > test-amd64-amd64-pair 21 leak-check/check/dst_host fail REGR. vs. 21026 > > > test-amd64-amd64-pair 20 leak-check/check/src_host fail REGR. vs. 21026 > > > test-amd64-i386-rhel6hvm-intel 11 leak-check/check fail REGR. vs. 21026 > > > > The bisector says, in <E1Ve0Tg-0008Kj-Il@woking.cam.xci-test.com>: > > > > David posted a patch to fix this, but that patch is not in that tree > yet.Oh good, thanks! I assume it will be flagged for stable when the time is right (hopefully sooner rather than later).