Xen devel - Mar 2005 - New Xentest automated testing framework snapshot

Here''s a new snapshot of the automated build and test tool for Xen. We
are trying to create a tool that will be as simple and lightweight as
possible, while being configurable enough to work with a variety of
tests and environments.  The purpose of this tool is to make it easier
to run tests under Xen on nightly tarball snapshots so that bugs can be
caught and fixed quickly.

It''s still a little rough around the edges but should be functional for
a lot of people.  Right now, I have it running well enough on my test
machines that I can kick it off and have it automatically pull down xen,
build/boot it, start my VMs and run tests on all of them (and on domain
0).

One major limitation is that initrd doesn''t get built, though
I''m not
sure if this is a limitation of xentest, or if it would better be
handled by make install from xen.

I would appreciate any feedback on how we might make this more useful,
as we intend to continue working on it and making it available.

-- 
Thanks,
Paul Larson
plars@linuxtestproject.org
http://www.linuxtestproject.org

Folks,
This work is much appreciated -- many thanks.

It would be great if someone could set this up to start doing nightly
builds, tests and benchmarks, and make the results available. It would
also be useful to do some ''historical'' tests too, perhaps
running the
kernel on every Sunday night in the unstable tree all the way back to
the 2.0. 

Ian
> -----Original Message-----
> From: xen-devel-admin@lists.sourceforge.net 
> [mailto:xen-devel-admin@lists.sourceforge.net] On Behalf Of 
> Paul Larson
> Sent: 01 March 2005 22:03
> To: xen-devel
> Cc: David Barrera; Li Ge; Stephanie Glass
> Subject: [Xen-devel] New Xentest automated testing framework snapshot
> 
> Here''s a new snapshot of the automated build and test tool for
Xen. We
> are trying to create a tool that will be as simple and lightweight as
> possible, while being configurable enough to work with a variety of
> tests and environments.  The purpose of this tool is to make it easier
> to run tests under Xen on nightly tarball snapshots so that 
> bugs can be
> caught and fixed quickly.
> 
> It''s still a little rough around the edges but should be 
> functional for
> a lot of people.  Right now, I have it running well enough on my test
> machines that I can kick it off and have it automatically 
> pull down xen,
> build/boot it, start my VMs and run tests on all of them (and 
> on domain
> 0).
> 
> One major limitation is that initrd doesn''t get built, though
I''m not
> sure if this is a limitation of xentest, or if it would better be
> handled by make install from xen.
> 
> I would appreciate any feedback on how we might make this more useful,
> as we intend to continue working on it and making it available.
> 
> -- 
> Thanks,
> Paul Larson
> plars@linuxtestproject.org
> http://www.linuxtestproject.org
> 

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id396&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

I will have to figure out if there is even a chance of plan 9 testing :-)

ron



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

On Tue, 2005-03-01 at 22:45 +0000, Ian Pratt wrote:
> Folks,
> This work is much appreciated -- many thanks.
> 
> It would be great if someone could set this up to start doing nightly
> builds, tests and benchmarks, and make the results available. It would
> also be useful to do some ''historical'' tests too, perhaps
running the
> kernel on every Sunday night in the unstable tree all the way back to
> the 2.0. 
That''s the idea.  Most of my testing with xentest so far has been for
the purpose of making sure xentest is behaving properly, but this week
(hopefully) I''m going to try to get a few machines in my environment
doing this every night.  It will be LTP initially, but I hope to be
adding more tests and test suites to the mix, and I''m even more hopeful
about either writing, or getting some xen specific tests dropped in my
lap (I know you sent me one a while back, that one is first in the
queue).

As for results, my thoughts were to just do some postprocessing and
results comparison between consecutive runs to see if anything changed,
and examine any new failures to see if they are really bugs or not.  If
you''d like to see more frequent results postings, and it
wouldn''t be
bothersome, I can see what can be done.

Ideally though, this is most useful if a lot of different people are
running this on a variety of machines.

-- 
Thanks,
Paul Larson
plars@linuxtestproject.org
http://www.linuxtestproject.org

This is great. We will dedicate resources to make this happen.

K. Y
>>> "Ian Pratt" <m+Ian.Pratt@cl.cam.ac.uk> 3/1/2005
5:45:10 PM >>>
Folks,
This work is much appreciated -- many thanks.

It would be great if someone could set this up to start doing nightly
builds, tests and benchmarks, and make the results available. It would
also be useful to do some ''historical'' tests too, perhaps
running the
kernel on every Sunday night in the unstable tree all the way back to
the 2.0. 

Ian
> -----Original Message-----
> From: xen-devel-admin@lists.sourceforge.net 
> [mailto:xen-devel-admin@lists.sourceforge.net] On Behalf Of 
> Paul Larson
> Sent: 01 March 2005 22:03
> To: xen-devel
> Cc: David Barrera; Li Ge; Stephanie Glass
> Subject: [Xen-devel] New Xentest automated testing framework
snapshot
> 
> Here''s a new snapshot of the automated build and test tool for
Xen.
We
> are trying to create a tool that will be as simple and lightweight
as
> possible, while being configurable enough to work with a variety of
> tests and environments.  The purpose of this tool is to make it
easier
> to run tests under Xen on nightly tarball snapshots so that 
> bugs can be
> caught and fixed quickly.
> 
> It''s still a little rough around the edges but should be 
> functional for
> a lot of people.  Right now, I have it running well enough on my
test
> machines that I can kick it off and have it automatically 
> pull down xen,
> build/boot it, start my VMs and run tests on all of them (and 
> on domain
> 0).
> 
> One major limitation is that initrd doesn''t get built, though
I''m
not
> sure if this is a limitation of xentest, or if it would better be
> handled by make install from xen.
> 
> I would appreciate any feedback on how we might make this more
useful,
> as we intend to continue working on it and making it available.
> 
> -- 
> Thanks,
> Paul Larson
> plars@linuxtestproject.org 
> http://www.linuxtestproject.org 
> 

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real
users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id396&op=click 
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net 
https://lists.sourceforge.net/lists/listinfo/xen-devel


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

Ian Pratt wrote:
>Folks,
>This work is much appreciated -- many thanks.
>
>It would be great if someone could set this up to start doing nightly
>builds, tests and benchmarks, and make the results available. It would
>also be useful to do some ''historical'' tests too, perhaps
running the
>kernel on every Sunday night in the unstable tree all the way back to
>the 2.0. 
>
>Ian
>  
>
In Samba, we have Build Farm that''s essentially a cronjob that will do 
automated building/testing on a machine and then will post (I believe 
through rsync) the results to a central server where they can be processed.

This way, anyone with a machine to spare can participate.  I''m sure we 
could do something like that with Xen if someone was able to provide a 
central server for the site containing the results.

Samba''s build farm is located at:

http://build.samba.org/

Regards,
Anthony Liguori
>>-----Original Message-----
>>From: xen-devel-admin@lists.sourceforge.net 
>>[mailto:xen-devel-admin@lists.sourceforge.net] On Behalf Of 
>>Paul Larson
>>Sent: 01 March 2005 22:03
>>To: xen-devel
>>Cc: David Barrera; Li Ge; Stephanie Glass
>>Subject: [Xen-devel] New Xentest automated testing framework snapshot
>>
>>Here''s a new snapshot of the automated build and test tool for
Xen. We
>>are trying to create a tool that will be as simple and lightweight as
>>possible, while being configurable enough to work with a variety of
>>tests and environments.  The purpose of this tool is to make it easier
>>to run tests under Xen on nightly tarball snapshots so that 
>>bugs can be
>>caught and fixed quickly.
>>
>>It''s still a little rough around the edges but should be 
>>functional for
>>a lot of people.  Right now, I have it running well enough on my test
>>machines that I can kick it off and have it automatically 
>>pull down xen,
>>build/boot it, start my VMs and run tests on all of them (and 
>>on domain
>>0).
>>
>>One major limitation is that initrd doesn''t get built, though
I''m not
>>sure if this is a limitation of xentest, or if it would better be
>>handled by make install from xen.
>>
>>I would appreciate any feedback on how we might make this more useful,
>>as we intend to continue working on it and making it available.
>>
>>-- 
>>Thanks,
>>Paul Larson
>>plars@linuxtestproject.org
>>http://www.linuxtestproject.org
>>
>>    
>>
>
>
>-------------------------------------------------------
>SF email is sponsored by - The IT Product Guide
>Read honest & candid reviews on hundreds of IT Products from real users.
>Discover which products truly live up to the hype. Start reading now.
>http://ads.osdn.com/?ad_ide95&alloc_id396&op=click
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/xen-devel
>
>  
>


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel