I have just posted version 0.2.0 of xm-test here:
http://static.danplanet.com/xm-test/xm-test-0.2.0.tar.gz
New in this version:
- Automatic submission of reports
- New tests:
- enforce_dom0_cpus
- concurrent domu tests
- additional checks for *some* stale-state issues
We would really like to get people running xm-test on a regular basis,
so that we can start to get some ideas about regression trends from
the submitted data.
* For those who are not familiar with xm-test:
Several of us here at IBM have been working on a framework for testing
the xen tools, specifically xm. Our goal is to provide a way for
developers to _easily_ write tests for new and existing xm commands.
We believe that such a test suite will help reduce breakages in the
user-facing tools when developers modify xm and/or xend.
We would like some feedback from the community on the usefulness of
our framework, in hopes that it might be hosted by xensource so that
everyone can contribute tests to help harden xm and xend.
* Details:
The framework tests (as well as the support libraries) are written in
python, which are executed by the standard automake "make check"
facilities. We build a standardized ramdisk that can be used for
portable test writing, therefore reducing dependencies on the test
host machine.
The framework library provides several abstractions to make common and
complex tasks easier for the test writer. For example, we provide a
domain and console abstraction that allows a test writer to start a
domU and execute arbitrary commands, retrieving the status and output
of each. This allows a decent amount of automation for verifying that
(for example) "xm sysrq mydomain s" actually sent the sysrq.
--
Dan Smith
IBM Linux Technology Center
Open Hypervisor Team
email: danms@us.ibm.com
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
On Mon, 2005-10-03 at 15:52 -0700, Dan Smith wrote:> I have just posted version 0.2.0 of xm-test here: > > http://static.danplanet.com/xm-test/xm-test-0.2.0.tar.gz > > New in this version: > > - Automatic submission of reports > - New tests: > - enforce_dom0_cpus > - concurrent domu tests > - additional checks for *some* stale-state issuesI got up to 12_create_concurrent_stress_pos.test saw trying to free up IRQ281 screaming by in console, machine was totally unresponsive. Tyan 2462 SMP , 2 gigs, FC4 dom0 Regards, Ted _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Monday 03 October 2005 17:52, Dan Smith wrote:> > We would like some feedback from the community on the usefulness of > our framework, in hopes that it might be hosted by xensource so that > everyone can contribute tests to help harden xm and xend.Building xm-test takes a very long time, because among other things it takes it upon itself to download and build its very own toolchain. That is extremely silly; please have it use the existing toolchain instead. I''m using qemu, so I want to run the tests on a system other than the one I built xm-test on (building a toolchain under qemu does not sound fun or necessary). How can I do this? Right now I''m getting obnoxious autoconf messages like this: make: *** No rule to make target `../configure.ac'', needed by `Makefile.in''. Stop. I have exported the tests/ directory to qemu, and am trying to run "make check". -- Hollis Blanchard IBM Linux Technology Center _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
HB> Building xm-test takes a very long time, because among other HB> things it takes it upon itself to download and build its very own HB> toolchain. FYI, I''ve only built xm-test once. See below. HB> That is extremely silly; please have it use the existing toolchain HB> instead. It builds its own toolchain because it uses uClibc. This allows us to compile some utilities and have them all fit nicely inside a ramdisk-sized image. HB> I''m using qemu, so I want to run the tests on a system other than HB> the one I built xm-test on (building a toolchain under qemu does HB> not sound fun or necessary). How can I do this? On your fastest machine, allow xm-test to build itself by running ''make'' in the toplevel. The end result will be an initrd.img file in the ramdisk/ directory. Unpack a fresh copy of xm-test wherever you want to run it from, and copy the initrd.img file into the ramdisk/ directory. Then, you can run ''make check'' (or preferably ./runtest.sh) without the lengthy build process. At this point, you never have to build xm-test again. Just copy around the initrd.img file to anywhere you run it. HB> I have exported the tests/ directory to HB> qemu, and am trying to run "make check". You can''t just use the tests/ directory on its own, you need the whole thing. There are other things provided that are not below tests/. -- Dan Smith IBM Linux Technology Center Open Hypervisor Team email: danms@us.ibm.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Friday 07 October 2005 12:38, Hollis Blanchard wrote:> On Monday 03 October 2005 17:52, Dan Smith wrote: > > We would like some feedback from the community on the usefulness of > > our framework, in hopes that it might be hosted by xensource so that > > everyone can contribute tests to help harden xm and xend. > > Building xm-test takes a very long time, because among other things it > takes it upon itself to download and build its very own toolchain. That is > extremely silly; please have it use the existing toolchain instead.To summarize IRC conversation: - xm-test creates an initrd that''s used to boot the test DomUs. - That initrd is created with buildroot[1], which uses uClibc, which requires building a whole new toolchain to use. - If they know how, users can manually copy the initrd from one xm-test directory to another, avoiding the need to rebuild it. - The initrd is plain busybox, which could easily be statically linked with the user''s GNU libc. - In the future, people want to install other tools (like e2fsprogs) into the ramdisk. These could also be statically linked with GNU libc. - Other than statically linking, the user''s libc.so could be copied to the initrd, which doesn''t make for a very reliable testing environment. - Using uClibc makes the initrd smaller, which is of dubious value in this environment. - initrd images are limited in size, while initramfs images are not. [1] http://buildroot.uclibc.org/ -- Hollis Blanchard IBM Linux Technology Center _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Fri, Oct 07, 2005 at 02:40:27PM -0500, Hollis Blanchard wrote:> On Friday 07 October 2005 12:38, Hollis Blanchard wrote: > > On Monday 03 October 2005 17:52, Dan Smith wrote: > > > We would like some feedback from the community on the usefulness of > > > our framework, in hopes that it might be hosted by xensource so that > > > everyone can contribute tests to help harden xm and xend. > > > > Building xm-test takes a very long time, because among other things it > > takes it upon itself to download and build its very own toolchain. That is > > extremely silly; please have it use the existing toolchain instead. > > To summarize IRC conversation:Some responses, as I think your summary has a lot of personal commentary, and I want to make sure another perspective is stated.> - xm-test creates an initrd that''s used to boot the test DomUs. > - That initrd is created with buildroot[1], which uses uClibc, which requires > building a whole new toolchain to use.Corrent, the initrd uses buildroot, which is a generic tool for building embedded Linux system images. It is open source, easy to extend, and brought to you by the same people that brought you busybox & uClibc.> - If they know how, users can manually copy the initrd from one xm-test > directory to another, avoiding the need to rebuild it.Correct. Even if they don''t know the build process is 100% automated, it just takes time. Given then number of people on xen-devel running gentoo, not sure why an "ug, it has to compile?" sentiment kept coming up on IRC. ;)> - The initrd is plain busybox, which could easily be statically linked with > the user''s GNU libc.For now, we are only 6 weeks and 80 tests into this, and don''t even have full coverage for xm user commands at this point.> - In the future, people want to install other tools (like e2fsprogs) into the > ramdisk. These could also be statically linked with GNU libc.Niv talked about hping, and other more advanced network tools. Saying everything needs to be statically linked is sort of a bear for adding new bits. There is already a very modular way to add things to buildroot. We can use that our play a NIH game and reinvent it for static binaries. I prefer to stand on the shoulders of those that came before me.> - Other than statically linking, the user''s libc.so could be copied to the > initrd, which doesn''t make for a very reliable testing environment.This is a retched idea. We actually did this in systemimager, and then you start finding all the joys of the differences in glibc between distros and versions, and have to go through the joys of mklibs (a debian shared library dependency tool) to ensure you''ve copied over all the right bits. It really sucks, and breaks quite a bit.> - Using uClibc makes the initrd smaller, which is of dubious value in this > environment.Unless you use it for scale out testing (i.e. run 100 DomU). Then the memory size is important as your root is in memory. I think Dan was able to start just under 100 DomU with xm-test on his 2 GB workstation (kinda cool :))> - initrd images are limited in size, while initramfs images are not.Yep, initramfs is a good thing to look into. We did initrds because we''d all done initrds before, and they were a well known quantity. :) While some heathly debate on any subject is always a good thing, it is worth noting we tried a lot of these other approaches to begin with, and none of them really panned out in a useful way. We could spend all our time creating the *uber static link initrd*, but that''s the least interesting part of xm-test. :) The actual tests are where we need to focus. Optimizing the plumbing can come post 3.0. -Sean -- __________________________________________________________________ Sean Dague Mid-Hudson Valley sean at dague dot net Linux Users Group http://dague.net http://mhvlug.org There is no silver bullet. Plus, werewolves make better neighbors than zombies, and they tend to keep the vampire population down. __________________________________________________________________ _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel