flight 21375 qemu-upstream-unstable real [real]
http://www.chiark.greenend.org.uk/~xensrcts/logs/21375/
Regressions :-(
Tests which did not succeed and are blocking,
including tests which could not be run:
test-amd64-i386-qemuu-rhel6hvm-intel 7 redhat-install fail REGR. vs. 20054
Tests which are failing intermittently (not blocking):
test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 9 guest-localmigrate fail pass in
21372
test-amd64-i386-xl-qemuu-win7-amd64 10 guest-saverestore.2 fail pass in 21372
test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 12 guest-localmigrate/x10 fail in
21372 pass in 21365
Tests which did not succeed, but are not blocking:
test-amd64-amd64-xl-qemuu-win7-amd64 13 guest-stop fail never pass
test-amd64-amd64-xl-qemuu-winxpsp3 13 guest-stop fail never pass
test-amd64-i386-xend-qemuu-winxpsp3 16 leak-check/check fail never pass
test-amd64-i386-xl-qemuu-win7-amd64 13 guest-stop fail in 21372 never pass
test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 13 guest-stop fail in 21365 never pass
version targeted for testing:
qemuu b97307ecaad98360f41ea36cd9674ef810c4f8cf
baseline version:
qemuu 8a4bd762aa01b21c43aa24c5b743f4bd7c9db3e3
------------------------------------------------------------
People who touched revisions under test:
"M. Mohan Kumar" <mohan@in.ibm.com>
Adam Lackorzynski <adam@os.inf.tu-dresden.de>
Alasdair McLeay <alasdair.mcleay@me.com>
Alberto Garcia <agarcia@igalia.com>
Alex Bligh <alex@alex.org.uk>
Alex Horn <alex.horn@cs.ox.ac.uk>
Alex Rozenman <Alex_Rozenman@mentor.com>
Alex Williamson <alex.williamson@redhat.com>
Alexander Graf <agraf@suse.de>
Alexey Kardashevskiy <aik@ozlabs.ru>
Alexey Korolev <akorolex@gmail.com>
Alexey Zaytsev <alexey.zaytsev@gmail.com>
Alin Tomescu <tomescu.alin@gmail.com>
Alon Levy <alevy@redhat.com>
Amadeusz Sławiński <amade@asmblr.net>
Amit Shah <amit.shah@redhat.com>
Amos Kong <akong@redhat.com>
Andre Przywara <andre.przywara@amd.com>
Andre Przywara <andre.przywara@calxeda.com>
Andrea Arcangeli <aarcange@redhat.com>
Andreas F=E4rber <afaerber@suse.de>
Andreas Färber <afaerber@suse.de>
Andreas Färber <andreas.faeber@web.de>
Andreas Färber <andreas.faerber@web.de>
Andreas Färberr <afaerber@suse.de>
Andreas Niederl <andreas.niederl@iaik.tugraz.at>
Andreas Schwab <schwab@linux-m68k.org>
Andreas Schwab <schwab@suse.de>
Andrew Jones <drjones@redhat.com>
Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Anthony Green <green@moxielogic.com>
Anthony Liguori <aliguori@us.ibm.com>
Anthony PERARD <anthony.perard@citrix.com>
Antoine Mathys <barsamin@gmail.com>
Anton Blanchard <anton@au1.ibm.com>
Anton Blanchard <anton@samba.org>
Artyom Tarasenko <atar4qemu@gmail.com>
Asias He <asias@redhat.com>
Aurelien Jarno <aurelien@aurel32.net>
Avi Kivity <avi.kivity@gmail.com>
Avi Kivity <avi@redhat.com>
Avik Sil <aviksil@linux.vnet.ibm.com>
Bas van Sisseren <bas@quarantainenet.nl>
Ben Herrenschmidt <benh@kernel.crashing.org>
Benjamin Herrenschmidt <benh@kernel.crashing.org>
Benoit Canet <benoit@irqsave.net>
Bharat Bhushan <bharat.bhushan@freescale.com>
Bharata B Rao <bharata@linux.vnet.ibm.com>
Blue Swirl <blauwirbel@gmail.com>
Borislav Petkov <bp@suse.de>
Brad Smith <brad@comstyle.com>
Bruce Rogers <brogers@suse.com>
Charles Arnold <carnold@suse.com>
Chegu Vinod <chegu_vinod@hp.com>
Chen Wei-Ren <chenwj@iis.sinica.edu.tw>
Christian Borntraeger <borntraeger@de.ibm.com>
Christoffer Dall <c.dall@virtualopensystems.com>
Christoffer Dall <cdall@cs.columbia.edu>
Christophe Lyon <christophe.lyon@linaro.org>
Claudio Fontana <claudio.fontana@huawei.com>
Cole Robinson <crobinso@redhat.com>
Corey Bryant <coreyb@linux.vnet.ibm.com>
Cornelia Huck <cornelia.huck@de.ibm.com>
Daniel P. Berrange <berrange@redhat.com>
Daniel Sangorrin <dsl@ertl.jp>
David Gibson <david@gibson.dropbear.id.au>
David Gibson <david@gibson.dropbear.id.au>`z
David Holsgrove <david.holsgrove@xilinx.com>
David Woodhouse <David.Woodhouse@intel.com>
Dietmar Maurer <dietmar@proxmox.com>
Dillon Amburgey <dillona@dillona.com>
Dmitry Fleytman <dfleytma@redhat.com>
Dmitry Fleytman <dmitry@daynix.com>
Dominik Dingel <dingel@linux.vnet.ibm.com>
Don Koch <dkoch@verizon.com>
Dong Xu Wang <wdongxu@linux.vnet.ibm.com>
Dongxue Zhang <elta.era@gmail.com>
Doug Goldstein <cardoe@cardoe.com>
Dunrong Huang <huangdr@cloud-times.com>
Dunrong Huang <riegamaths@gmail.com>
Ed Maste <emaste@freebsd.org>
Edgar E. Iglesias <edgar.iglesias@gmail.com>
Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Eduardo Habkost <ehabkost@redhat.com>
Eduardo Otubo <otubo@linux.vnet.ibm.com>
Eiichi Tsukata <eiichi.tsukata.xh@hitachi.com>
Ekaterina Tumanova <tumanova@linux.vnet.ibm.com>
Eric Blake <eblake@redhat.com>
Eric Johnson <ericj@mips.com>
Erlon Cruz <erlon.cruz@br.flextronics.com>
Evgeny Budilovsky <evgeny.budilovsky@ravellosystems.com>
Evgeny Voevodin <e.voevodin@samsung.com>
Evgeny Voevodin <evgenyvoevodin@gmail.com>
Fabien Chouteau <chouteau@adacore.com>
Fam Zheng <famz@redhat.com>
Federico Simoncelli <fsimonce@redhat.com>
Felipe Franciosi <felipe@paradoxo.org>
Gabriel de Perthuis <g2p.code@gmail.com>
Gabriel Kerneis <gabriel@kerneis.info>
Gal Hammer <ghammer@redhat.com>
Gerd Hoffmann <kraxel@redhat.com>
Gertjan Halkes <qemu@ghalkes.nl>
Gleb Natapov <gleb@redhat.com>
Grant Likely <grant.likely@linaro.org>
Grant Likely <grant.likely@secretlab.ca>
Guan Xuetao <gxt@mprc.pku.edu.cn>
H. Peter Anvin <hpa@zytor.com>
Hans de Goede <hdegoede@redhat.com>
Heinz Graalfs <graalfs@linux.vnet.ibm.com>
Henry Harrington <henry.harrington@gmail.com>
Hervé Poussineau <hpoussin@reactos.org>
Hervé Poussineau <hpoussineau@reactos.org>
Hu Tao <hutao@cn.fujitsu.com>
Ian Jackson <ian.jackson@eu.citrix.com>
Ian Main <imain@redhat.com>
Ian Molton <ian.molton@collabora.co.uk>
Igor Mammedov <imammedo@redhat.com>
Igor Mammedov <imammedo@redhat.com> (for i386)
Igor Mitsyanko <i.mitsyanko@gmail.com>
Igor Mitsyanko <i.mitsyanko@samsung.com>
Isaku Yamahata <yamahata@private.email.ne.jp>
Isaku Yamahata <yamahata@valinux.co.jp>
Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
Jacob Kroon <jacob.kroon@gmail.com>
James Hogan <james.hogan@imgtec.com>
Jan Kiszka <jan.kiszka@siemens.com>
Jani Kokkoken <jani.kokkonen@huawei.com>
Jani Kokkonen <jani.kokkonen@huawei.com>
Jason Baron <jbaron@redhat.com>
Jason J. Herne <jjherne@us.ibm.com>
Jason Wang <jasowang>
Jason Wang <jasowang@redhat.com>
Jean-Christophe DUBOIS <jcd@tribudubois.net>
Jeff Cody <jcody@redhat.com>
Jens Freimann <jfrei@linux.vnet.ibm.com>
Jesse Larrew <jlarrew@linux.vnet.ibm.com>
Jia Liu <proljc@gmail.com>
Jia Liu <proljc@gmail.com> (for openrisc)
Jim Meyering <meyering@redhat.com>
John Rigby <john.rigby@linaro.org>
John Spencer <maillist-qemu@barfooze.de>
Jordan Justen <jordan.l.justen@intel.com>
Josh Durgin <josh.durgin@inktank.com>
Juan Quintela <quintela@redhat.com>
Julien Grall <julien.grall@citrix.com>
Julio Guerra <guerr@julio.in>
Ján Tomko <jtomko@redhat.com>
Jürg Billeter <j@bitron.ch>
Kazuya Saito <saito.kazuya@jp.fujitsu.com>
Keith Busch <keith.busch@intel.com>
Kevin Wolf <kwolf@redhat.com>
Kevin Wolf <mail@kevin-wolf.de>
Kim Phillips <kim.phillips@freescale.com>
Kirill Batuzov <batuzovk@ispras.ru>
Knut Omang <knut.omang@oracle.com>
KONRAD Frederic <fred.konrad@greensocs.com>
Kuo-Jung Su <dantesu@faraday-tech.com>
Kuo-Jung Su <dantesu@gmail.com>
Kusanagi Kouichi <slash@ac.auone-net.jp>
Kwok Cheung Yeung <kcy@codesourcery.com>
Laszlo Ersek <lersek@redhat.com>
Laurent Desnogues <laurent.desnogues@gmail.com>
Laurent Vivier <laurent@vivier.eu>
Lei Li <lilei@linux.vnet.ibm.com>
Leon Alrae <leon.alrae@imgtec.com>
liguang <lig.fnst@cn.fujitsu.com>
Liming Wang <walimisdev@gmail.com>
Liu Jinsong <jinsong.liu@intel.com>
Liu Ping Fan <pingfank@linux.vnet.ibm.com>
Liu Ping Fan <qemulist@gmail.com>
Liu Yuan <namei.unix@gmail.com>
Liu Yuan <tailai.ly@taobao.com>
Lluís Vilanova <vilanova@ac.upc.edu>
Lucas Meneghel Rodrigues <lmr@redhat.com>
Luigi Rizzo <rizzo@iet.unipi.it>
Luiz Capitulino <lcapitulino@redhat.com>
M. Mohan Kumar <mohan@in.ibm.com>
Mans Rullgard <mans@mansr.com>
Marc-André Lureau <marcandre.lureau@redhat.com>
Marc-André Lureau <mlureau@redhat.com>
Marcel Apfelbaum <marcel.a@redhat.com>
Marcelo Tosatti <mtosatti@redhat.com>
Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Markus Armbruster <armbru@redhat.com>
Martijn van den Broek <martijn.vdbrk@gmail.com>
Matthew Daley <mattjd@gmail.com>
Max Filippov <jcmvbkbc@gmail.com>
Max Filippov <jcmvbkbc@gmail.com> (for xtensa)
Meador Inge <meadori@codesourcery.com>
Michael Contreras <michael@inetric.com>
Michael Ellerman <michael@ellerman.id.au>
Michael Marineau <mike@marineau.org>
Michael R. Hines <mrhines@us.ibm.com>
Michael Roth <mdroth@linux.vnet.ibm.com>
Michael S. Tsirkin <mst@redhat.com>
Michael Tokarev <mjt@tls.msk.ru>
Michael Walle <michael@walle.cc>
Michael Walle <michael@walle.cc> (for lm32)
Michal Novotny <minovotn@redhat.com>
Michal Privoznik <mprivozn@redhat.com>
Mike Qiu <qiudayu@linux.vnet.ibm.com>
Miroslav Rezanina <mrezanin@redhat.com>
MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
MRatnikov <m.o.ratnikov@gmail.com>
Nathan Rossi <nathan.rossi@xilinx.com>
Nicholas Bellinger <nab@linux-iscsi.org>
Nick Thomas <nick@bytemark.co.uk>
Nickolai Zeldovich <nickolai@csail.mit.edu>
Oleksii Shevchuk <alxchk@gmail.com>
Olivier Hainque <hainque@adacore.com>
Orit Wasserman <owasserm@redhat.com>
Othmar Pasteka <pasteka@kabsi.at>
Ozan Çağlayan <ozancag@gmail.com>
Paolo Bonzini <pbonini@redhat.com
Paolo Bonzini <pbonzini@redhat.com>
Paul Brook <paul@codesourcery.com>
Paul Burton <paul.burton@imgtec.com>
Paul Durrant <paul.durrant@citrix.com>
Paul Moore <pmoore@redhat.com>
Pavel Dovgalyuk <pavel.dovgaluk@gmail.com>
Pavel Hrdina <phrdina@redhat.com>
Pawit Pornkitprasan <p.pawit@gmail.com>
Petar Jovanovic <petar.jovanovic@imgtec.com>
Petar Jovanovic <petarj@mips.com>
Peter Chubb <peter.chubb@nicta.com.au>
Peter Crosthwaite <peter.crosthwaite@petalogix.com>
Peter Crosthwaite <peter.crosthwaite@xilinx.com>
Peter Crosthwaite peter.crosthwaite@xilinx.com>
Peter Feiner <peter@gridcentric.ca>
Peter Lieven <pl@kamp.de>
Peter Maydell <peter.maydell@linaro.org>
Peter Wu <lekensteyn@gmail.com>
Petr Matousek <pmatouse@redhat.com>
Philipp Hahn <hahn@univention.de>
Prerna Saxena <prerna@linux.vnet.ibm.com>
Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Ramkumar Ramachandra <artagnon@gmail.com>
Richard Henderson <rth@twiddle.net>
Richard Henderson <rth@twiddle.net> (for alpha)
Richard Sandiford <rdsandiford@googlemail.com>
Richard W.M. Jones <rjones@redhat.com>
Riku Voipio <riku.voipio@iki.fi>
Riku Voipio <riku.voipio@linaro.org>
Robert Schiele <rschiele@gmail.com>
Roger Pau Monné <roger.pau@citrix.com>
Ronald Hecht <address@hidden>
Ronald Hecht <ronald.hecht@gmx.de>
Ronnie Sahlberg <ronniesahlberg@gmail.com>
Samuel Seay <LightningTH@GMail.com>
Sander Eikelenboom <linux@eikelenboom.it>
Satoru Moriya <satoru.moriya@hds.com>
Scott Feldman <sfeldma@cumulusnetworks.com>
Scott Wood <scottwood@freescale.com>
Seiji Aguchi <seiji.aguchi@hds.com>
Serge Hallyn <serge.hallyn@ubuntu.com>
Soren Brinkmann <soren.brinkmann@xilinx.com>
Stefan Berger <stefanb@linux.vnet.ibm.com>
Stefan Hajnoczi <stefanha@redhat.com>
Stefan Priebe <s.priebe@profihost.ag>
Stefan Weil <sw@weilnetz.de>
Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Stuart Yoder <stuart.yoder@freescale.com>
Thomas Huth <thuth@linux.vnet.ibm.com>
Thomas Schwinge <thomas@codesourcery.com>
Tiejun Chen <tiejun.chen@windriver.com>
Tim Hardeck <thardeck@suse.de>
Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
Umesh Deshpande <udeshpan@redhat.com>
Uri Lublin <uril@redhat.com>
Vadim Evard <v.e.evard@gmail.com>
Vasilis Liaskovitis <vasilis.liaskovitis@profitbricks.com>
Vijay Mohan Pandarathil <vijaymohan.pandarathil@hp.com>
Viktor Mihajlovski <mihajlov@linux.vnet.ibm.com>
Vishvananda Ishaya <vishvananda@gmail.com>
Vladimir Senkov <hangup@gmail.com>
Wanlong Gao <gaowanlong@cn.fujitsu.com>
Weidong Han <hanweidong@huawei.com>
Wen Congyang <wency@cn.fujitsu.com>
Wenchao Xia <xiawenc@linux.vnet.ibm.com>
Wendy Liang <jliang@xilinx.com>
Will Auld <will.auld@intel.com>
Xiao Guangrong <xiaoguangrong@linux.vnet.ibm.com>
Xudong Hao <xudong.hao@intel.com>
Yan Vugenfirer <yan@daynix.com>
Yeongkyoon Lee <yeongkyoon.lee@samsung.com>
Yin Yin <yin.yin@cs2c.com.cn>
Yongbok Kim <yongbok.kim@imgtec.com>
zhangleiqiang <zhangleiqiang@huawei.com>
Zhenguo Wang <wangzhenguo@huawei.com>
Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
Ákos Kovács <akoskovacs@gmx.com>
------------------------------------------------------------
jobs:
build-amd64 pass
build-i386 pass
build-amd64-oldkern pass
build-i386-oldkern pass
build-amd64-pvops pass
build-i386-pvops pass
test-amd64-i386-qemuu-rhel6hvm-amd pass
test-amd64-amd64-xl-qemuu-win7-amd64 fail
test-amd64-i386-xl-qemuu-win7-amd64 fail
test-amd64-i386-qemuu-rhel6hvm-intel fail
test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 fail
test-amd64-i386-xend-qemuu-winxpsp3 fail
test-amd64-amd64-xl-qemuu-winxpsp3 fail
------------------------------------------------------------
sg-report-flight on woking.cam.xci-test.com
logs: /home/xc_osstest/logs
images: /home/xc_osstest/images
Logs, config files, etc. are available at
http://www.chiark.greenend.org.uk/~xensrcts/logs
Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary
Not pushing.
(No revision log; it would be 73708 lines long.)
--===============4916819418375342940=Content-Type: text/plain;
charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
--===============4916819418375342940==--
Ian Campbell
2013-Nov-01 10:43 UTC
Re: [qemu-upstream-unstable test] 21375: regressions - FAIL
On Fri, 2013-11-01 at 10:38 +0000, xen.org wrote:> flight 21375 qemu-upstream-unstable real [real] > http://www.chiark.greenend.org.uk/~xensrcts/logs/21375/ > > Regressions :-( > > Tests which did not succeed and are blocking, > including tests which could not be run: > test-amd64-i386-qemuu-rhel6hvm-intel 7 redhat-install fail REGR. vs. 20054Anythony, have you made any progress on this? It''s been failing for ages now... Ian.
Sander Eikelenboom
2013-Nov-01 10:47 UTC
Re: [qemu-upstream-unstable test] 21375: regressions - FAIL
Friday, November 1, 2013, 11:38:20 AM, you wrote:> flight 21375 qemu-upstream-unstable real [real] > http://www.chiark.greenend.org.uk/~xensrcts/logs/21375/> Regressions :-(> Tests which did not succeed and are blocking, > including tests which could not be run: > test-amd64-i386-qemuu-rhel6hvm-intel 7 redhat-install fail REGR. vs. 20054> Tests which are failing intermittently (not blocking): > test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 9 guest-localmigrate fail pass in 21372 > test-amd64-i386-xl-qemuu-win7-amd64 10 guest-saverestore.2 fail pass in 21372 > test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 12 guest-localmigrate/x10 fail in 21372 pass in 21365> Tests which did not succeed, but are not blocking: > test-amd64-amd64-xl-qemuu-win7-amd64 13 guest-stop fail never pass > test-amd64-amd64-xl-qemuu-winxpsp3 13 guest-stop fail never pass > test-amd64-i386-xend-qemuu-winxpsp3 16 leak-check/check fail never pass > test-amd64-i386-xl-qemuu-win7-amd64 13 guest-stop fail in 21372 never pass > test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 13 guest-stop fail in 21365 never pass> version targeted for testing: > qemuu b97307ecaad98360f41ea36cd9674ef810c4f8cf > baseline version: > qemuu 8a4bd762aa01b21c43aa24c5b743f4bd7c9db3e3Ian, Why does the script try to reboot the guest within 2 seconds after creating it ? Also strange it doesn''t report back that the -F should be used since the guest is probably not booted yet so PV drivers can''t be loaded yet. This seems to block the push for quite some time now ... -- Sander 2013-11-01 06:30:56 Z execution took 72 seconds[<=2x600]: ssh -o StrictHostKeyChecking=no -o BatchMode=yes -o ConnectTimeout=100 -o ServerAliveInterval=100 -o PasswordAuthentication=no -o ChallengeResponseAuthentication=no -o UserKnownHostsFile=tmp/t.known_hosts_21375.test-amd64-i386-qemuu-rhel6hvm-intel root@10.80.249.149 genisoimage -o /root/21375.test-amd64-i386-qemuu-rhel6hvm-intel.rhel-server-6.1-i386-dvd.iso -R -J -T -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table /root/newiso/. 2013-11-01 06:30:56 Z runvar store: redhat_cfgpath=/etc/xen/redhat.guest.osstest.cfg 2013-11-01 06:30:56 Z executing scp ... /home/xc_osstest/logs/21375/test-amd64-i386-qemuu-rhel6hvm-intel/gall-mite--redhat.guest.osstest.cfg root@10.80.249.149:/etc/xen/redhat.guest.osstest.cfg 2013-11-01 06:30:57 Z executing ssh ... root@10.80.249.149 (echo xenvnc; echo xenvnc) | vncpasswd redhat.vncpw Password:Verify:2013-11-01 06:30:57 Z executing ssh ... root@10.80.249.149 xl create /etc/xen/redhat.guest.osstest.cfg WARNING: ignoring "kernel" directive for HVM guest. Use "firmware_override" instead if you really want a non-default firmware Parsing config from /etc/xen/redhat.guest.osstest.cfg 2013-11-01 06:30:59 Z await reboot request from redhat: waiting 2000s... 2013-11-01 06:30:59 Z executing ssh ... root@10.80.249.149 xl list 2013-11-01 06:30:59 Z guest redhat.guest.osstest state is - 2013-11-01 06:30:59 Z await reboot request from redhat: guest state is - (waiting) ... 2013-11-01 06:31:29 Z executing ssh ... root@10.80.249.149 xl list 2013-11-01 06:31:30 Z guest redhat.guest.osstest state is r 2013-11-01 06:31:30 Z await reboot request from redhat: guest state is r (waiting) ... 2013-11-01 06:31:59 Z executing ssh ... root@10.80.249.149 xl list 2013-11-01 06:31:59 Z guest redhat.guest.osstest state is r 2013-11-01 06:32:29 Z executing ssh ... root@10.80.249.149 xl list 2013-11-01 06:32:29 Z guest redhat.guest.osstest state is r 2013-11-01 06:32:59 Z executing ssh ... root@10.80.249.149 xl list
Ian Campbell
2013-Nov-01 10:52 UTC
Re: [qemu-upstream-unstable test] 21375: regressions - FAIL
On Fri, 2013-11-01 at 11:47 +0100, Sander Eikelenboom wrote:> Friday, November 1, 2013, 11:38:20 AM, you wrote: > > > flight 21375 qemu-upstream-unstable real [real] > > http://www.chiark.greenend.org.uk/~xensrcts/logs/21375/ > > > Regressions :-( > > > Tests which did not succeed and are blocking, > > including tests which could not be run: > > test-amd64-i386-qemuu-rhel6hvm-intel 7 redhat-install fail REGR. vs. 20054 > > > Tests which are failing intermittently (not blocking): > > test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 9 guest-localmigrate fail pass in 21372 > > test-amd64-i386-xl-qemuu-win7-amd64 10 guest-saverestore.2 fail pass in 21372 > > test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 12 guest-localmigrate/x10 fail in 21372 pass in 21365 > > > Tests which did not succeed, but are not blocking: > > test-amd64-amd64-xl-qemuu-win7-amd64 13 guest-stop fail never pass > > test-amd64-amd64-xl-qemuu-winxpsp3 13 guest-stop fail never pass > > test-amd64-i386-xend-qemuu-winxpsp3 16 leak-check/check fail never pass > > test-amd64-i386-xl-qemuu-win7-amd64 13 guest-stop fail in 21372 never pass > > test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 13 guest-stop fail in 21365 never pass > > > version targeted for testing: > > qemuu b97307ecaad98360f41ea36cd9674ef810c4f8cf > > baseline version: > > qemuu 8a4bd762aa01b21c43aa24c5b743f4bd7c9db3e3 > > > > Ian, > > Why does the script try to reboot the guest within 2 seconds after creating it ? > Also strange it doesn''t report back that the -F should be used since > the guest is probably not booted yet so PV drivers can''t be loaded > yet.Where do you see this? It isn''t in the logs you quoted, which only show the harness waiting for a reboot, not initiating one. This is sensible -- it is waiting for the reboot initiated by the installer from within the guest when it is finished installing.> This seems to block the push for quite some time now ... > -- > Sander > > > 2013-11-01 06:30:56 Z execution took 72 seconds[<=2x600]: ssh -o StrictHostKeyChecking=no -o BatchMode=yes -o ConnectTimeout=100 -o ServerAliveInterval=100 -o PasswordAuthentication=no -o ChallengeResponseAuthentication=no -o UserKnownHostsFile=tmp/t.known_hosts_21375.test-amd64-i386-qemuu-rhel6hvm-intel root@10.80.249.149 genisoimage -o /root/21375.test-amd64-i386-qemuu-rhel6hvm-intel.rhel-server-6.1-i386-dvd.iso -R -J -T -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table /root/newiso/. > > 2013-11-01 06:30:56 Z runvar store: redhat_cfgpath=/etc/xen/redhat.guest.osstest.cfg > 2013-11-01 06:30:56 Z executing scp ... /home/xc_osstest/logs/21375/test-amd64-i386-qemuu-rhel6hvm-intel/gall-mite--redhat.guest.osstest.cfg root@10.80.249.149:/etc/xen/redhat.guest.osstest.cfg > 2013-11-01 06:30:57 Z executing ssh ... root@10.80.249.149 (echo xenvnc; echo xenvnc) | vncpasswd redhat.vncpw > > Password:Verify:2013-11-01 06:30:57 Z executing ssh ... root@10.80.249.149 xl create /etc/xen/redhat.guest.osstest.cfg > WARNING: ignoring "kernel" directive for HVM guest. Use "firmware_override" instead if you really want a non-default firmware > Parsing config from /etc/xen/redhat.guest.osstest.cfg > 2013-11-01 06:30:59 Z await reboot request from redhat: waiting 2000s... > 2013-11-01 06:30:59 Z executing ssh ... root@10.80.249.149 xl list > 2013-11-01 06:30:59 Z guest redhat.guest.osstest state is - > 2013-11-01 06:30:59 Z await reboot request from redhat: guest state is - (waiting) ... > 2013-11-01 06:31:29 Z executing ssh ... root@10.80.249.149 xl list > 2013-11-01 06:31:30 Z guest redhat.guest.osstest state is r > 2013-11-01 06:31:30 Z await reboot request from redhat: guest state is r (waiting) ... > 2013-11-01 06:31:59 Z executing ssh ... root@10.80.249.149 xl list > 2013-11-01 06:31:59 Z guest redhat.guest.osstest state is r > 2013-11-01 06:32:29 Z executing ssh ... root@10.80.249.149 xl list > 2013-11-01 06:32:29 Z guest redhat.guest.osstest state is r > 2013-11-01 06:32:59 Z executing ssh ... root@10.80.249.149 xl list > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
Sander Eikelenboom
2013-Nov-01 11:57 UTC
Re: [qemu-upstream-unstable test] 21375: regressions - FAIL
Friday, November 1, 2013, 11:52:51 AM, you wrote:> On Fri, 2013-11-01 at 11:47 +0100, Sander Eikelenboom wrote: >> Friday, November 1, 2013, 11:38:20 AM, you wrote: >> >> > flight 21375 qemu-upstream-unstable real [real] >> > http://www.chiark.greenend.org.uk/~xensrcts/logs/21375/ >> >> > Regressions :-( >> >> > Tests which did not succeed and are blocking, >> > including tests which could not be run: >> > test-amd64-i386-qemuu-rhel6hvm-intel 7 redhat-install fail REGR. vs. 20054 >> >> > Tests which are failing intermittently (not blocking): >> > test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 9 guest-localmigrate fail pass in 21372 >> > test-amd64-i386-xl-qemuu-win7-amd64 10 guest-saverestore.2 fail pass in 21372 >> > test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 12 guest-localmigrate/x10 fail in 21372 pass in 21365 >> >> > Tests which did not succeed, but are not blocking: >> > test-amd64-amd64-xl-qemuu-win7-amd64 13 guest-stop fail never pass >> > test-amd64-amd64-xl-qemuu-winxpsp3 13 guest-stop fail never pass >> > test-amd64-i386-xend-qemuu-winxpsp3 16 leak-check/check fail never pass >> > test-amd64-i386-xl-qemuu-win7-amd64 13 guest-stop fail in 21372 never pass >> > test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 13 guest-stop fail in 21365 never pass >> >> > version targeted for testing: >> > qemuu b97307ecaad98360f41ea36cd9674ef810c4f8cf >> > baseline version: >> > qemuu 8a4bd762aa01b21c43aa24c5b743f4bd7c9db3e3 >> >> >> >> Ian, >> >> Why does the script try to reboot the guest within 2 seconds after creating it ? >> Also strange it doesn''t report back that the -F should be used since >> the guest is probably not booted yet so PV drivers can''t be loaded >> yet.> Where do you see this? It isn''t in the logs you quoted, which only show > the harness waiting for a reboot, not initiating one. This is sensible > -- it is waiting for the reboot initiated by the installer from within > the guest when it is finished installing.Ah ok, seems i misread sorry. So the guest should request a reboot itself, and thus seems to stall somewhere (given the assumption that it should be able to reach the point to reboot in 2000s) Isn''t the installing part one in part part above this in the log ? (http://www.chiark.greenend.org.uk/~xensrcts/logs/21375/test-amd64-i386-qemuu-rhel6hvm-intel/7.ts-redhat-install.log) Also in: http://www.chiark.greenend.org.uk/~xensrcts/logs/21375/test-amd64-i386-qemuu-rhel6hvm-intel/info.html There doesn''t seem to be any helpful logging of what the *guest* is actually doing (or not)>> This seems to block the push for quite some time now ... >> -- >> Sander >> >> >> 2013-11-01 06:30:56 Z execution took 72 seconds[<=2x600]: ssh -o StrictHostKeyChecking=no -o BatchMode=yes -o ConnectTimeout=100 -o ServerAliveInterval=100 -o PasswordAuthentication=no -o ChallengeResponseAuthentication=no -o UserKnownHostsFile=tmp/t.known_hosts_21375.test-amd64-i386-qemuu-rhel6hvm-intel root@10.80.249.149 genisoimage -o /root/21375.test-amd64-i386-qemuu-rhel6hvm-intel.rhel-server-6.1-i386-dvd.iso -R -J -T -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table /root/newiso/. >> >> 2013-11-01 06:30:56 Z runvar store: redhat_cfgpath=/etc/xen/redhat.guest.osstest.cfg >> 2013-11-01 06:30:56 Z executing scp ... /home/xc_osstest/logs/21375/test-amd64-i386-qemuu-rhel6hvm-intel/gall-mite--redhat.guest.osstest.cfg root@10.80.249.149:/etc/xen/redhat.guest.osstest.cfg >> 2013-11-01 06:30:57 Z executing ssh ... root@10.80.249.149 (echo xenvnc; echo xenvnc) | vncpasswd redhat.vncpw >> >> Password:Verify:2013-11-01 06:30:57 Z executing ssh ... root@10.80.249.149 xl create /etc/xen/redhat.guest.osstest.cfg >> WARNING: ignoring "kernel" directive for HVM guest. Use "firmware_override" instead if you really want a non-default firmware >> Parsing config from /etc/xen/redhat.guest.osstest.cfg >> 2013-11-01 06:30:59 Z await reboot request from redhat: waiting 2000s... >> 2013-11-01 06:30:59 Z executing ssh ... root@10.80.249.149 xl list >> 2013-11-01 06:30:59 Z guest redhat.guest.osstest state is - >> 2013-11-01 06:30:59 Z await reboot request from redhat: guest state is - (waiting) ... >> 2013-11-01 06:31:29 Z executing ssh ... root@10.80.249.149 xl list >> 2013-11-01 06:31:30 Z guest redhat.guest.osstest state is r >> 2013-11-01 06:31:30 Z await reboot request from redhat: guest state is r (waiting) ... >> 2013-11-01 06:31:59 Z executing ssh ... root@10.80.249.149 xl list >> 2013-11-01 06:31:59 Z guest redhat.guest.osstest state is r >> 2013-11-01 06:32:29 Z executing ssh ... root@10.80.249.149 xl list >> 2013-11-01 06:32:29 Z guest redhat.guest.osstest state is r >> 2013-11-01 06:32:59 Z executing ssh ... root@10.80.249.149 xl list >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xen.org >> http://lists.xen.org/xen-devel
Anthony PERARD
2013-Nov-01 11:58 UTC
Re: [qemu-upstream-unstable test] 21375: regressions - FAIL
On Fri, Nov 01, 2013 at 10:43:16AM +0000, Ian Campbell wrote:> On Fri, 2013-11-01 at 10:38 +0000, xen.org wrote: > > flight 21375 qemu-upstream-unstable real [real] > > http://www.chiark.greenend.org.uk/~xensrcts/logs/21375/ > > > > Regressions :-( > > > > Tests which did not succeed and are blocking, > > including tests which could not be run: > > test-amd64-i386-qemuu-rhel6hvm-intel 7 redhat-install fail REGR. vs. 20054 > > Anythony, have you made any progress on this? It''s been failing for ages > now...Yes, looks like the bug it trigger during a vesa resolution change. I have try to use the vgabios blob that we use for qemu-traditionnal and it works fine. But with the vgabios blob provided by qemu, it does not work... I''m still not sure of what the bug is, but I''m getting closer to it. Also, this happen only on an Intel machine, on an AMD machine, everything works like a charm. More detail, if anyone want to know: It''s look like syslinux is doing a int 10h call that never return to set video mode: Int 0x10, with AX=0x4F02 Regards, -- Anthony PERARD
Ian Campbell
2013-Nov-01 12:06 UTC
Re: [qemu-upstream-unstable test] 21375: regressions - FAIL
On Fri, 2013-11-01 at 11:58 +0000, Anthony PERARD wrote:> On Fri, Nov 01, 2013 at 10:43:16AM +0000, Ian Campbell wrote: > > On Fri, 2013-11-01 at 10:38 +0000, xen.org wrote: > > > flight 21375 qemu-upstream-unstable real [real] > > > http://www.chiark.greenend.org.uk/~xensrcts/logs/21375/ > > > > > > Regressions :-( > > > > > > Tests which did not succeed and are blocking, > > > including tests which could not be run: > > > test-amd64-i386-qemuu-rhel6hvm-intel 7 redhat-install fail REGR. vs. 20054 > > > > Anythony, have you made any progress on this? It''s been failing for ages > > now... > > Yes, looks like the bug it trigger during a vesa resolution change. I > have try to use the vgabios blob that we use for qemu-traditionnal and > it works fine. But with the vgabios blob provided by qemu, it does not > work... I''m still not sure of what the bug is, but I''m getting closer to > it.Yay!> Also, this happen only on an Intel machine, on an AMD machine, > everything works like a charm. > > More detail, if anyone want to know: > It''s look like syslinux is doing a int 10h call that never return to set > video mode: > Int 0x10, with AX=0x4F02This looks like it might be handled by SeaBIOS vgasrc/vbe.c:vbe_104f00 ? There seem to be a few changes in upstream seabios since the version referenced in xen.git:Config.mk. Many of them are cleanups/code motion but a few look worth investigating. Ian.
Anthony PERARD
2013-Nov-01 15:46 UTC
Re: [qemu-upstream-unstable test] 21375: regressions - FAIL
On Fri, Nov 01, 2013 at 12:06:51PM +0000, Ian Campbell wrote:> On Fri, 2013-11-01 at 11:58 +0000, Anthony PERARD wrote: > > On Fri, Nov 01, 2013 at 10:43:16AM +0000, Ian Campbell wrote: > > > On Fri, 2013-11-01 at 10:38 +0000, xen.org wrote: > > > > flight 21375 qemu-upstream-unstable real [real] > > > > http://www.chiark.greenend.org.uk/~xensrcts/logs/21375/ > > > > > > > > Regressions :-( > > > > > > > > Tests which did not succeed and are blocking, > > > > including tests which could not be run: > > > > test-amd64-i386-qemuu-rhel6hvm-intel 7 redhat-install fail REGR. vs. 20054 > > > > > > Anythony, have you made any progress on this? It''s been failing for ages > > > now... > > > > Yes, looks like the bug it trigger during a vesa resolution change. I > > have try to use the vgabios blob that we use for qemu-traditionnal and > > it works fine. But with the vgabios blob provided by qemu, it does not > > work... I''m still not sure of what the bug is, but I''m getting closer to > > it. > > Yay! > > > Also, this happen only on an Intel machine, on an AMD machine, > > everything works like a charm. > > > > More detail, if anyone want to know: > > It''s look like syslinux is doing a int 10h call that never return to set > > video mode: > > Int 0x10, with AX=0x4F02 > > This looks like it might be handled by SeaBIOS vgasrc/vbe.c:vbe_104f00 ? > There seem to be a few changes in upstream seabios since the version > referenced in xen.git:Config.mk. Many of them are cleanups/code motion > but a few look worth investigating.I''ve been able to get the things working by applying a patch to vgabios that is in xen tree: a0e7ccf6864c196906d58b54cd0996b4dbc1b022 This patch allow to clear the framebuffer much faster. But it those not really help be to understand why the guest freeze. A couple more printf might. -- Anthony PERARD
Anthony PERARD
2013-Nov-06 17:22 UTC
Re: [qemu-upstream-unstable test] 21375: regressions - FAIL
On Fri, Nov 01, 2013 at 03:46:36PM +0000, Anthony PERARD wrote:> On Fri, Nov 01, 2013 at 12:06:51PM +0000, Ian Campbell wrote: > > On Fri, 2013-11-01 at 11:58 +0000, Anthony PERARD wrote: > > > On Fri, Nov 01, 2013 at 10:43:16AM +0000, Ian Campbell wrote: > > > > On Fri, 2013-11-01 at 10:38 +0000, xen.org wrote: > > > > > flight 21375 qemu-upstream-unstable real [real] > > > > > http://www.chiark.greenend.org.uk/~xensrcts/logs/21375/ > > > > > > > > > > Regressions :-( > > > > > > > > > > Tests which did not succeed and are blocking, > > > > > including tests which could not be run: > > > > > test-amd64-i386-qemuu-rhel6hvm-intel 7 redhat-install fail REGR. vs. 20054 > > > > > > > > Anythony, have you made any progress on this? It''s been failing for ages > > > > now... > > > > > > Yes, looks like the bug it trigger during a vesa resolution change. I > > > have try to use the vgabios blob that we use for qemu-traditionnal and > > > it works fine. But with the vgabios blob provided by qemu, it does not > > > work... I''m still not sure of what the bug is, but I''m getting closer to > > > it. > > > > Yay! > > > > > Also, this happen only on an Intel machine, on an AMD machine, > > > everything works like a charm. > > > > > > More detail, if anyone want to know: > > > It''s look like syslinux is doing a int 10h call that never return to set > > > video mode: > > > Int 0x10, with AX=0x4F02 > > > > This looks like it might be handled by SeaBIOS vgasrc/vbe.c:vbe_104f00 ? > > There seem to be a few changes in upstream seabios since the version > > referenced in xen.git:Config.mk. Many of them are cleanups/code motion > > but a few look worth investigating. > > I''ve been able to get the things working by applying a patch to vgabios > that is in xen tree: a0e7ccf6864c196906d58b54cd0996b4dbc1b022 > This patch allow to clear the framebuffer much faster. > > But it those not really help be to understand why the guest freeze. A > couple more printf might.I finally managed to have a better understanding of the issue. So, the vgabios blob provided by QEMU have a routine to clear the video ram that take few seconds to run. That give enough time to QEMU to try to refresh is display, and this mean they will be a call to xc_hvm_track_dirty_vram(). If the function is called while the vgabios routine is running, then the guest is lost. The issue appear only with an Intel machine on an HVM guest using EPT. Having the guest using shadow works fine. So I''m going to investigate the track_dirty code in Xen. The vgabios routine is called by syslinux with an Int 0x10, I tryied to get some debug print after the call, either from the guest serial or by using the Xen debug ioport, nothing ever appear, and gdbsx only gave me some weird IP which does not appear to point to any usefull code (it''s all zeros). -- Anthony PERARD
Anthony PERARD
2013-Nov-18 17:18 UTC
Re: [qemu-upstream-unstable test] 21375: regressions - FAIL
On Wed, Nov 06, 2013 at 05:22:29PM +0000, Anthony PERARD wrote:> On Fri, Nov 01, 2013 at 03:46:36PM +0000, Anthony PERARD wrote: > > On Fri, Nov 01, 2013 at 12:06:51PM +0000, Ian Campbell wrote: > > > On Fri, 2013-11-01 at 11:58 +0000, Anthony PERARD wrote: > > > > On Fri, Nov 01, 2013 at 10:43:16AM +0000, Ian Campbell wrote: > > > > > On Fri, 2013-11-01 at 10:38 +0000, xen.org wrote: > > > > > > flight 21375 qemu-upstream-unstable real [real] > > > > > > http://www.chiark.greenend.org.uk/~xensrcts/logs/21375/ > > > > > > > > > > > > Regressions :-( > > > > > > > > > > > > Tests which did not succeed and are blocking, > > > > > > including tests which could not be run: > > > > > > test-amd64-i386-qemuu-rhel6hvm-intel 7 redhat-install fail REGR. vs. 20054 > > > > > > > > > > Anythony, have you made any progress on this? It''s been failing for ages > > > > > now... > > > > > > > > Yes, looks like the bug it trigger during a vesa resolution change. I > > > > have try to use the vgabios blob that we use for qemu-traditionnal and > > > > it works fine. But with the vgabios blob provided by qemu, it does not > > > > work... I''m still not sure of what the bug is, but I''m getting closer to > > > > it. > > > > > > Yay! > > > > > > > Also, this happen only on an Intel machine, on an AMD machine, > > > > everything works like a charm. > > > > > > > > More detail, if anyone want to know: > > > > It''s look like syslinux is doing a int 10h call that never return to set > > > > video mode: > > > > Int 0x10, with AX=0x4F02 > > > > > > This looks like it might be handled by SeaBIOS vgasrc/vbe.c:vbe_104f00 ? > > > There seem to be a few changes in upstream seabios since the version > > > referenced in xen.git:Config.mk. Many of them are cleanups/code motion > > > but a few look worth investigating. > > > > I''ve been able to get the things working by applying a patch to vgabios > > that is in xen tree: a0e7ccf6864c196906d58b54cd0996b4dbc1b022 > > This patch allow to clear the framebuffer much faster. > > > > But it those not really help be to understand why the guest freeze. A > > couple more printf might. > > I finally managed to have a better understanding of the issue. > > So, the vgabios blob provided by QEMU have a routine to clear the video > ram that take few seconds to run. That give enough time to QEMU to try > to refresh is display, and this mean they will be a call to > xc_hvm_track_dirty_vram(). If the function is called while the vgabios > routine is running, then the guest is lost. > > The issue appear only with an Intel machine on an HVM guest using EPT. > Having the guest using shadow works fine. So I''m going to investigate > the track_dirty code in Xen. > > The vgabios routine is called by syslinux with an Int 0x10, I tryied to > get some debug print after the call, either from the guest serial or > by using the Xen debug ioport, nothing ever appear, and gdbsx only gave > me some weird IP which does not appear to point to any usefull code > (it''s all zeros).An other update, we had the idee of trying this on earlier versin of Xen, and it turns out that Xen 4.3 works fine. One bisect later, and a commit turns out. commit 86781624f8df1d50eb4185cfc2ddce926798f7aa x86_emulate: PUSH <mem> must read source operand just once ... for the case of accessing MMIO. So after this commit, syslinux stop working correctly with the last version of QEMU. This happen if QEMU is calling track_dirty_vram. I also have use xentrace/xenalyze to try to grab more information about the issue, it did not really help, but it''s tell me that the guest is stock on a specific instruction (it result in vmexit EPT_VIOLATION over and over on xentrace). And that were the guest is stock: 0xa126: mov %eax,%cr0 0xa129: ljmp $0xf2e,$0xa12e 0xa130: mov $0x26,%dl 0xa132: or %bh,(%eax) 0xa134: movzww %sp,%sp 0xa138: mov %edx,%ds 0xa13a: mov %edx,%es 0xa13c: mov %edx,%fs 0xa13e: mov %edx,%gs 0xa140: jmp *%ebx 0xa142: pushf => 0xa143: lcall *%cs:(%si) 0xa147: mov $0x0,%ch Before trying on earlier version of Xen, I try to understand what when wrong on the Xen side, it turn out that, in the track_dirty_vram hypercall, a call to hap_enable_log_dirty() is all that needed to break the guest. Jan, any idee of what the issue is? Regards, -- Anthony PERARD
Ian Campbell
2013-Nov-19 11:07 UTC
Re: [qemu-upstream-unstable test] 21375: regressions - FAIL
On Mon, 2013-11-18 at 17:18 +0000, Anthony PERARD wrote:> On Wed, Nov 06, 2013 at 05:22:29PM +0000, Anthony PERARD wrote: > > On Fri, Nov 01, 2013 at 03:46:36PM +0000, Anthony PERARD wrote: > > > On Fri, Nov 01, 2013 at 12:06:51PM +0000, Ian Campbell wrote: > > > > On Fri, 2013-11-01 at 11:58 +0000, Anthony PERARD wrote: > > > > > On Fri, Nov 01, 2013 at 10:43:16AM +0000, Ian Campbell wrote: > > > > > > On Fri, 2013-11-01 at 10:38 +0000, xen.org wrote: > > > > > > > flight 21375 qemu-upstream-unstable real [real] > > > > > > > http://www.chiark.greenend.org.uk/~xensrcts/logs/21375/ > > > > > > > > > > > > > > Regressions :-( > > > > > > > > > > > > > > Tests which did not succeed and are blocking, > > > > > > > including tests which could not be run: > > > > > > > test-amd64-i386-qemuu-rhel6hvm-intel 7 redhat-install fail REGR. vs. 20054 > > > > > > > > > > > > Anythony, have you made any progress on this? It''s been failing for ages > > > > > > now... > > > > > > > > > > Yes, looks like the bug it trigger during a vesa resolution change. I > > > > > have try to use the vgabios blob that we use for qemu-traditionnal and > > > > > it works fine. But with the vgabios blob provided by qemu, it does not > > > > > work... I''m still not sure of what the bug is, but I''m getting closer to > > > > > it. > > > > > > > > Yay! > > > > > > > > > Also, this happen only on an Intel machine, on an AMD machine, > > > > > everything works like a charm. > > > > > > > > > > More detail, if anyone want to know: > > > > > It''s look like syslinux is doing a int 10h call that never return to set > > > > > video mode: > > > > > Int 0x10, with AX=0x4F02 > > > > > > > > This looks like it might be handled by SeaBIOS vgasrc/vbe.c:vbe_104f00 ? > > > > There seem to be a few changes in upstream seabios since the version > > > > referenced in xen.git:Config.mk. Many of them are cleanups/code motion > > > > but a few look worth investigating. > > > > > > I''ve been able to get the things working by applying a patch to vgabios > > > that is in xen tree: a0e7ccf6864c196906d58b54cd0996b4dbc1b022 > > > This patch allow to clear the framebuffer much faster. > > > > > > But it those not really help be to understand why the guest freeze. A > > > couple more printf might. > > > > I finally managed to have a better understanding of the issue. > > > > So, the vgabios blob provided by QEMU have a routine to clear the video > > ram that take few seconds to run. That give enough time to QEMU to try > > to refresh is display, and this mean they will be a call to > > xc_hvm_track_dirty_vram(). If the function is called while the vgabios > > routine is running, then the guest is lost. > > > > The issue appear only with an Intel machine on an HVM guest using EPT. > > Having the guest using shadow works fine. So I''m going to investigate > > the track_dirty code in Xen. > > > > The vgabios routine is called by syslinux with an Int 0x10, I tryied to > > get some debug print after the call, either from the guest serial or > > by using the Xen debug ioport, nothing ever appear, and gdbsx only gave > > me some weird IP which does not appear to point to any usefull code > > (it''s all zeros). > > An other update, > > we had the idee of trying this on earlier versin of Xen, and it turns > out that Xen 4.3 works fine. One bisect later, and a commit turns out. > > commit 86781624f8df1d50eb4185cfc2ddce926798f7aa > x86_emulate: PUSH <mem> must read source operand just once > ... for the case of accessing MMIO. > > So after this commit, syslinux stop working correctly with the last > version of QEMU. This happen if QEMU is calling track_dirty_vram. > > I also have use xentrace/xenalyze to try to grab more information about > the issue, it did not really help, but it''s tell me that the guest is > stock on a specific instruction (it result in vmexit EPT_VIOLATION over > and over on xentrace). And that were the guest is stock: > > 0xa126: mov %eax,%cr0 > 0xa129: ljmp $0xf2e,$0xa12e > 0xa130: mov $0x26,%dl > 0xa132: or %bh,(%eax) > 0xa134: movzww %sp,%sp > 0xa138: mov %edx,%ds > 0xa13a: mov %edx,%es > 0xa13c: mov %edx,%fs > 0xa13e: mov %edx,%gs > 0xa140: jmp *%ebx > 0xa142: pushf > => 0xa143: lcall *%cs:(%si) > 0xa147: mov $0x0,%chOOI what is the encoding of the bad instruction?> > Before trying on earlier version of Xen, I try to understand what when > wrong on the Xen side, it turn out that, in the track_dirty_vram > hypercall, a call to hap_enable_log_dirty() is all that needed to break > the guest. > > Jan, any idee of what the issue is? > > Regards, >
Anthony PERARD
2013-Nov-19 12:33 UTC
Re: [qemu-upstream-unstable test] 21375: regressions - FAIL
On Tue, Nov 19, 2013 at 11:07:22AM +0000, Ian Campbell wrote:> On Mon, 2013-11-18 at 17:18 +0000, Anthony PERARD wrote: > > On Wed, Nov 06, 2013 at 05:22:29PM +0000, Anthony PERARD wrote: > > > On Fri, Nov 01, 2013 at 03:46:36PM +0000, Anthony PERARD wrote: > > > > On Fri, Nov 01, 2013 at 12:06:51PM +0000, Ian Campbell wrote: > > > > > On Fri, 2013-11-01 at 11:58 +0000, Anthony PERARD wrote: > > > > > > On Fri, Nov 01, 2013 at 10:43:16AM +0000, Ian Campbell wrote: > > > > > > > On Fri, 2013-11-01 at 10:38 +0000, xen.org wrote: > > > > > > > > flight 21375 qemu-upstream-unstable real [real] > > > > > > > > http://www.chiark.greenend.org.uk/~xensrcts/logs/21375/ > > > > > > > > > > > > > > > > Regressions :-( > > > > > > > > > > > > > > > > Tests which did not succeed and are blocking, > > > > > > > > including tests which could not be run: > > > > > > > > test-amd64-i386-qemuu-rhel6hvm-intel 7 redhat-install fail REGR. vs. 20054 > > > > > > > > > > > > > > Anythony, have you made any progress on this? It''s been failing for ages > > > > > > > now... > > > > > > > > > > > > Yes, looks like the bug it trigger during a vesa resolution change. I > > > > > > have try to use the vgabios blob that we use for qemu-traditionnal and > > > > > > it works fine. But with the vgabios blob provided by qemu, it does not > > > > > > work... I''m still not sure of what the bug is, but I''m getting closer to > > > > > > it. > > > > > > > > > > Yay! > > > > > > > > > > > Also, this happen only on an Intel machine, on an AMD machine, > > > > > > everything works like a charm. > > > > > > > > > > > > More detail, if anyone want to know: > > > > > > It''s look like syslinux is doing a int 10h call that never return to set > > > > > > video mode: > > > > > > Int 0x10, with AX=0x4F02 > > > > > > > > > > This looks like it might be handled by SeaBIOS vgasrc/vbe.c:vbe_104f00 ? > > > > > There seem to be a few changes in upstream seabios since the version > > > > > referenced in xen.git:Config.mk. Many of them are cleanups/code motion > > > > > but a few look worth investigating. > > > > > > > > I''ve been able to get the things working by applying a patch to vgabios > > > > that is in xen tree: a0e7ccf6864c196906d58b54cd0996b4dbc1b022 > > > > This patch allow to clear the framebuffer much faster. > > > > > > > > But it those not really help be to understand why the guest freeze. A > > > > couple more printf might. > > > > > > I finally managed to have a better understanding of the issue. > > > > > > So, the vgabios blob provided by QEMU have a routine to clear the video > > > ram that take few seconds to run. That give enough time to QEMU to try > > > to refresh is display, and this mean they will be a call to > > > xc_hvm_track_dirty_vram(). If the function is called while the vgabios > > > routine is running, then the guest is lost. > > > > > > The issue appear only with an Intel machine on an HVM guest using EPT. > > > Having the guest using shadow works fine. So I''m going to investigate > > > the track_dirty code in Xen. > > > > > > The vgabios routine is called by syslinux with an Int 0x10, I tryied to > > > get some debug print after the call, either from the guest serial or > > > by using the Xen debug ioport, nothing ever appear, and gdbsx only gave > > > me some weird IP which does not appear to point to any usefull code > > > (it''s all zeros). > > > > An other update, > > > > we had the idee of trying this on earlier versin of Xen, and it turns > > out that Xen 4.3 works fine. One bisect later, and a commit turns out. > > > > commit 86781624f8df1d50eb4185cfc2ddce926798f7aa > > x86_emulate: PUSH <mem> must read source operand just once > > ... for the case of accessing MMIO. > > > > So after this commit, syslinux stop working correctly with the last > > version of QEMU. This happen if QEMU is calling track_dirty_vram. > > > > I also have use xentrace/xenalyze to try to grab more information about > > the issue, it did not really help, but it''s tell me that the guest is > > stock on a specific instruction (it result in vmexit EPT_VIOLATION over > > and over on xentrace). And that were the guest is stock: > > > > 0xa126: mov %eax,%cr0 > > 0xa129: ljmp $0xf2e,$0xa12e > > 0xa130: mov $0x26,%dl > > 0xa132: or %bh,(%eax) > > 0xa134: movzww %sp,%sp > > 0xa138: mov %edx,%ds > > 0xa13a: mov %edx,%es > > 0xa13c: mov %edx,%fs > > 0xa13e: mov %edx,%gs > > 0xa140: jmp *%ebx > > 0xa142: pushf > > => 0xa143: lcall *%cs:(%si) > > 0xa147: mov $0x0,%ch > > OOI what is the encoding of the bad instruction?That''s what gdb give me: 0x0000a143: 2e 67 ff 1c lcall *%cs:(%si)> > Before trying on earlier version of Xen, I try to understand what when > > wrong on the Xen side, it turn out that, in the track_dirty_vram > > hypercall, a call to hap_enable_log_dirty() is all that needed to break > > the guest. > > > > Jan, any idee of what the issue is? > > > > Regards, > > > >-- Anthony PERARD
Jan Beulich
2013-Nov-19 13:05 UTC
Re: [qemu-upstream-unstable test] 21375: regressions - FAIL
>>> On 18.11.13 at 18:18, Anthony PERARD <anthony.perard@citrix.com> wrote: > An other update, > > we had the idee of trying this on earlier versin of Xen, and it turns > out that Xen 4.3 works fine. One bisect later, and a commit turns out. > > commit 86781624f8df1d50eb4185cfc2ddce926798f7aa > x86_emulate: PUSH <mem> must read source operand just once > ... for the case of accessing MMIO. > > So after this commit, syslinux stop working correctly with the last > version of QEMU. This happen if QEMU is calling track_dirty_vram. > > I also have use xentrace/xenalyze to try to grab more information about > the issue, it did not really help, but it''s tell me that the guest is > stock on a specific instruction (it result in vmexit EPT_VIOLATION over > and over on xentrace). And that were the guest is stock: > > 0xa126: mov %eax,%cr0 > 0xa129: ljmp $0xf2e,$0xa12e > 0xa130: mov $0x26,%dl > 0xa132: or %bh,(%eax) > 0xa134: movzww %sp,%sp > 0xa138: mov %edx,%ds > 0xa13a: mov %edx,%es > 0xa13c: mov %edx,%fs > 0xa13e: mov %edx,%gs > 0xa140: jmp *%ebx > 0xa142: pushf > => 0xa143: lcall *%cs:(%si) > 0xa147: mov $0x0,%ch > > Before trying on earlier version of Xen, I try to understand what when > wrong on the Xen side, it turn out that, in the track_dirty_vram > hypercall, a call to hap_enable_log_dirty() is all that needed to break > the guest. > > Jan, any idee of what the issue is?Oh, indeed, I screwed this up without noticing. Please try the attached patch. Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Anthony PERARD
2013-Nov-19 14:28 UTC
Re: [qemu-upstream-unstable test] 21375: regressions - FAIL
On Tue, Nov 19, 2013 at 01:05:37PM +0000, Jan Beulich wrote:> >>> On 18.11.13 at 18:18, Anthony PERARD <anthony.perard@citrix.com> wrote: > > An other update, > > > > we had the idee of trying this on earlier versin of Xen, and it turns > > out that Xen 4.3 works fine. One bisect later, and a commit turns out. > > > > commit 86781624f8df1d50eb4185cfc2ddce926798f7aa > > x86_emulate: PUSH <mem> must read source operand just once > > ... for the case of accessing MMIO. > > > > So after this commit, syslinux stop working correctly with the last > > version of QEMU. This happen if QEMU is calling track_dirty_vram. > > > > I also have use xentrace/xenalyze to try to grab more information about > > the issue, it did not really help, but it''s tell me that the guest is > > stock on a specific instruction (it result in vmexit EPT_VIOLATION over > > and over on xentrace). And that were the guest is stock: > > > > 0xa126: mov %eax,%cr0 > > 0xa129: ljmp $0xf2e,$0xa12e > > 0xa130: mov $0x26,%dl > > 0xa132: or %bh,(%eax) > > 0xa134: movzww %sp,%sp > > 0xa138: mov %edx,%ds > > 0xa13a: mov %edx,%es > > 0xa13c: mov %edx,%fs > > 0xa13e: mov %edx,%gs > > 0xa140: jmp *%ebx > > 0xa142: pushf > > => 0xa143: lcall *%cs:(%si) > > 0xa147: mov $0x0,%ch > > > > Before trying on earlier version of Xen, I try to understand what when > > wrong on the Xen side, it turn out that, in the track_dirty_vram > > hypercall, a call to hap_enable_log_dirty() is all that needed to break > > the guest. > > > > Jan, any idee of what the issue is? > > Oh, indeed, I screwed this up without noticing. Please try the > attached patch.Looks like this patch fix the issue. The guest is now running fine. Thanks.> x86: fix emulation of indirect far calls and jumps > > Commit 86781624 ("x86_emulate: PUSH <mem> must read source operand > just once") corrected the operands of those of the operations of opcode > extension group 5 that only read memory from SrcMem to DstMem, but > failed to also switch the use of "dst" here to "src". > > Reported-by: Anthony Perard <anthony.perard@citrix.com> > Signed-off-by: Jan Beulich <jbeulich@suse.com> > > --- a/xen/arch/x86/x86_emulate/x86_emulate.c > +++ b/xen/arch/x86/x86_emulate/x86_emulate.c > @@ -3571,7 +3571,6 @@ x86_emulate( > _regs.eip = src.val; > src.val = dst.val; > goto push; > - break; > case 4: /* jmp (near) */ > _regs.eip = src.val; > dst.type = OP_NONE; > @@ -3580,9 +3579,9 @@ x86_emulate( > case 5: /* jmp (far, absolute indirect) */ { > unsigned long sel; > > - generate_exception_if(dst.type != OP_MEM, EXC_UD, -1); > + generate_exception_if(src.type != OP_MEM, EXC_UD, -1); > > - if ( (rc = read_ulong(dst.mem.seg, dst.mem.off+dst.bytes, > + if ( (rc = read_ulong(src.mem.seg, src.mem.off + op_bytes, > &sel, 2, ctxt, ops)) ) > goto done; > > @@ -3600,7 +3599,7 @@ x86_emulate( > > if ( (rc = load_seg(x86_seg_cs, sel, ctxt, ops)) != 0 ) > goto done; > - _regs.eip = dst.val; > + _regs.eip = src.val; > > dst.type = OP_NONE; > break;-- Anthony PERARD
Ian Jackson
2013-Nov-20 14:42 UTC
Re: [qemu-upstream-unstable test] 21375: regressions - FAIL
Anthony PERARD writes ("Re: [Xen-devel] [qemu-upstream-unstable test]
21375: regressions - FAIL"):> Looks like this patch fix the issue. The guest is now running fine.
Hooray! Well done to Anthony for finding where to point the finger.
Thanks everyone.
Ian.