Lars Kurth
2013-Apr-23 14:29 UTC
Building Xen Test Days into the conclusion of the Xen 4.3 release cycle
Hi everybody, I wanted to follow up on George''s e-mail (Xen 4.3 development update CODE FREEZE at http://lists.xen.org/archives/html/xen-devel/2013-04/msg02142.html). One way we can facilitate the goal of 1. Have a bug-free 4.3 release ... is to build in some Test Days into the coming weeks. This worked well for Xen 4.2 RC2 as well as XCP 1.6. What we do need though is a good set of test instructions for Xen 4.3 : examples for last time round were - http://wiki.xen.org/wiki/Xen_4.2_RC2_test_instructions - http://wiki.xen.org/wiki/XCP_1.6_test_instructions We have a document day coming up next Monday, which would be an ideal day to put together Xen 4.3 RC test intrstructions. I remember we got a lot of test coverage for 4.2 RC2 as well as XCP 1.6. Having a predictable schedule around RC''s would also help. It appears to me that maybe one test day every other week starting from the first RC (planned for May 6th would work) ... we could do the test day 1-2 days after the planned RC''s to provide for a buffer should an RC slip. That would give us dates for Xen Test Days for May 8, May 22 (not clashing with Hackathon), June 5 (with RC''s planned 2 days before). Views? Should I start announcing the dates and promote the Test Days? If so, I need some commitment on getting good Test instructions as well as a good sense that RC''s are possible for these dates. If so, I would like to announce these by the end of this week. Regards Lars
jacek burghardt
2013-Apr-23 14:53 UTC
Re: Building Xen Test Days into the conclusion of the Xen 4.3 release cycle
This sounds like great idea. What xen needs more clarification on bugs reporting, I had experienced many bugs with xen unstable lately and I don''t know if anyone cares to fix the issues and if anyone is even running testing unstable and if they experience same issues. HVM recently stop providing network (at least with e1000 emulation )unless xen drivers are installed and freenas does not boot up on xen and does not have network connection On Tue, Apr 23, 2013 at 8:29 AM, Lars Kurth <lars.kurth@xen.org> wrote:> Hi everybody, > > I wanted to follow up on George''s e-mail (Xen 4.3 development update CODE > FREEZE at http://lists.xen.org/archives/**html/xen-devel/2013-04/** > msg02142.html<http://lists.xen.org/archives/html/xen-devel/2013-04/msg02142.html>). > One way we can facilitate the goal of > > 1. Have a bug-free 4.3 release > > ... is to build in some Test Days into the coming weeks. This worked well > for Xen 4.2 RC2 as well as XCP 1.6. What we do need though is a good set of > test instructions for Xen 4.3 : examples for last time round were > - http://wiki.xen.org/wiki/Xen_**4.2_RC2_test_instructions<http://wiki.xen.org/wiki/Xen_4.2_RC2_test_instructions> > - http://wiki.xen.org/wiki/XCP_**1.6_test_instructions<http://wiki.xen.org/wiki/XCP_1.6_test_instructions> > We have a document day coming up next Monday, which would be an ideal day > to put together Xen 4.3 RC test intrstructions. I remember we got a lot of > test coverage for 4.2 RC2 as well as XCP 1.6. > > Having a predictable schedule around RC''s would also help. It appears to > me that maybe one test day every other week starting from the first RC > (planned for May 6th would work) ... we could do the test day 1-2 days > after the planned RC''s to provide for a buffer should an RC slip. > > That would give us dates for Xen Test Days for May 8, May 22 (not clashing > with Hackathon), June 5 (with RC''s planned 2 days before). > > Views? Should I start announcing the dates and promote the Test Days? If > so, I need some commitment on getting good Test instructions as well as a > good sense that RC''s are possible for these dates. If so, I would like to > announce these by the end of this week. > > Regards > Lars > > > ______________________________**_________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Roger Pau Monné
2013-Apr-23 15:04 UTC
Re: Building Xen Test Days into the conclusion of the Xen 4.3 release cycle
On 23/04/13 16:53, jacek burghardt wrote:> This sounds like great idea. What xen needs more clarification > on bugs reporting, I had experienced many bugs with xen unstable lately > and I don''t know if anyone cares to fix the issues and if anyone is even > running testing unstable and if they experience same issues. HVM > recently stop providing network (at least with e1000 emulation )unless > xen drivers are installed and freenas does not boot up on xen and does > not have network connectionThe problem with FreeNAS not booting is already reported and Jan has a patch for it (since FreeNAS is basically FreeBSD I guess the same patch that Jan provided for FreeBSD will apply to FreeNAS): http://marc.info/?l=xen-devel&m=136637574324172 Could you provide more info about the network problem? Which OS are you using inside the guest? Is the emulated network interface correctly attached to a bridge? Could you bisect the Xen tree in order to find the commit that breaks emulated networking?
Ian Campbell
2013-Apr-23 15:09 UTC
Re: Building Xen Test Days into the conclusion of the Xen 4.3 release cycle
On Tue, 2013-04-23 at 15:53 +0100, jacek burghardt wrote:> This sounds like great idea. What xen needs more clarification > on bugs reporting, I had experienced many bugs with xen > unstable lately and I don''t know if anyone cares to fix the issues and > if anyone is even running testing unstable and if they experience same > issues. HVM recently stop providing network (at least with e1000 > emulation )unless xen drivers are installed and freenas does not > boot up on xen and does not have network connectionPlease see http://wiki.xen.org/wiki/Reporting_Bugs_against_Xen for how to report a bug against Xen. If you follow the suggestions there and include all the useful details and logs instead of posting brief mails with little detail you will likely find you get better traction with your bug reports. Ian.
Wei Liu
2013-Apr-23 15:12 UTC
Re: Building Xen Test Days into the conclusion of the Xen 4.3 release cycle
On Tue, Apr 23, 2013 at 03:53:01PM +0100, jacek burghardt wrote:> This sounds like great idea. What xen needs more clarification on bugs reporting, I had experienced many bugs with xen unstable lately and I don''t know if anyone cares to fix the issues and if anyone is even running testing unstable and if they experience same issues. HVM recently stop providing network (at least with e1000 emulation )unless xen drivers are installed and freenas does not boot up on xen and does not have network connection > >Thanks for your help with testing Xen unstable. It is not that we don''t care about fixing problems, but sometimes we would like to see more concrete logs / figures so that we can identify the genuine causes. http://wiki.xen.org/wiki/Reporting_Bugs_against_Xen could help we interact more efficient. Wei.> On Tue, Apr 23, 2013 at 8:29 AM, Lars Kurth <lars.kurth@xen.org<mailto:lars.kurth@xen.org>> wrote: > Hi everybody, > > I wanted to follow up on George''s e-mail (Xen 4.3 development update CODE FREEZE at http://lists.xen.org/archives/html/xen-devel/2013-04/msg02142.html). One way we can facilitate the goal of > > 1. Have a bug-free 4.3 release > > ... is to build in some Test Days into the coming weeks. This worked well for Xen 4.2 RC2 as well as XCP 1.6. What we do need though is a good set of test instructions for Xen 4.3 : examples for last time round were > - http://wiki.xen.org/wiki/Xen_4.2_RC2_test_instructions > - http://wiki.xen.org/wiki/XCP_1.6_test_instructions > We have a document day coming up next Monday, which would be an ideal day to put together Xen 4.3 RC test intrstructions. I remember we got a lot of test coverage for 4.2 RC2 as well as XCP 1.6. > > Having a predictable schedule around RC''s would also help. It appears to me that maybe one test day every other week starting from the first RC (planned for May 6th would work) ... we could do the test day 1-2 days after the planned RC''s to provide for a buffer should an RC slip. > > That would give us dates for Xen Test Days for May 8, May 22 (not clashing with Hackathon), June 5 (with RC''s planned 2 days before). > > Views? Should I start announcing the dates and promote the Test Days? If so, I need some commitment on getting good Test instructions as well as a good sense that RC''s are possible for these dates. If so, I would like to announce these by the end of this week. > > Regards > Lars > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org<mailto:Xen-devel@lists.xen.org> > http://lists.xen.org/xen-devel >> _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
Lars Kurth
2013-Apr-23 15:18 UTC
Re: Building Xen Test Days into the conclusion of the Xen 4.3 release cycle
Thanks for the pointers on bug reports. I still would like to get input on planning the Test Days though and whether we should build these into the release cycle. Lars
Stefano Stabellini
2013-Apr-24 19:53 UTC
Re: Building Xen Test Days into the conclusion of the Xen 4.3 release cycle
On Tue, 23 Apr 2013, Lars Kurth wrote:> Hi everybody, > > I wanted to follow up on George''s e-mail (Xen 4.3 development update > CODE FREEZE at > http://lists.xen.org/archives/html/xen-devel/2013-04/msg02142.html). One > way we can facilitate the goal of > > 1. Have a bug-free 4.3 release > > ... is to build in some Test Days into the coming weeks. This worked > well for Xen 4.2 RC2 as well as XCP 1.6. What we do need though is a > good set of test instructions for Xen 4.3 : examples for last time round > were > - http://wiki.xen.org/wiki/Xen_4.2_RC2_test_instructions > - http://wiki.xen.org/wiki/XCP_1.6_test_instructions > We have a document day coming up next Monday, which would be an ideal > day to put together Xen 4.3 RC test intrstructions. I remember we got a > lot of test coverage for 4.2 RC2 as well as XCP 1.6. > > Having a predictable schedule around RC''s would also help. It appears to > me that maybe one test day every other week starting from the first RC > (planned for May 6th would work) ... we could do the test day 1-2 days > after the planned RC''s to provide for a buffer should an RC slip.That sounds like a good plan to me> That would give us dates for Xen Test Days for May 8, May 22 (not > clashing with Hackathon), June 5 (with RC''s planned 2 days before).Fine by me> Views? Should I start announcing the dates and promote the Test Days? If > so, I need some commitment on getting good Test instructions as well as > a good sense that RC''s are possible for these dates. If so, I would like > to announce these by the end of this week.Go ahead
Dario Faggioli
2013-Apr-24 22:14 UTC
Re: Building Xen Test Days into the conclusion of the Xen 4.3 release cycle
On mar, 2013-04-23 at 15:29 +0100, Lars Kurth wrote:> Hi everybody, > > I wanted to follow up on George''s e-mail (Xen 4.3 development update > CODE FREEZE at > http://lists.xen.org/archives/html/xen-devel/2013-04/msg02142.html). One > way we can facilitate the goal of > > 1. Have a bug-free 4.3 release > > ... is to build in some Test Days into the coming weeks. This worked > well for Xen 4.2 RC2 as well as XCP 1.6. >+1 ... I really really like the idea of test days!> What we do need though is a > good set of test instructions for Xen 4.3 : examples for last time round > were > - http://wiki.xen.org/wiki/Xen_4.2_RC2_test_instructions > - http://wiki.xen.org/wiki/XCP_1.6_test_instructions > We have a document day coming up next Monday, which would be an ideal > day to put together Xen 4.3 RC test intrstructions. I remember we got a > lot of test coverage for 4.2 RC2 as well as XCP 1.6. > > Having a predictable schedule around RC''s would also help. It appears to > me that maybe one test day every other week starting from the first RC > (planned for May 6th would work) ... we could do the test day 1-2 days > after the planned RC''s to provide for a buffer should an RC slip. > > That would give us dates for Xen Test Days for May 8, May 22 (not > clashing with Hackathon), June 5 (with RC''s planned 2 days before). >Sounds cool.> Views? Should I start announcing the dates and promote the Test Days? >Yes, I think we should. For what I''ve seen from the ''Fedora Test Days'', the success of the whole thing depends on having people around and actually following the instructions and running the tests, even if everyone just do that for a small subset of them!> If > so, I need some commitment on getting good Test instructions as well as > a good sense that RC''s are possible for these dates. If so, I would like > to announce these by the end of this week. >I''m going to be on leave next Monday (during DocsDay), but for everything else you need from May 2nd on, count me. :-) Regards, Dario -- <<This happens because I choose it to happen!>> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Ian Campbell
2013-Apr-25 07:49 UTC
Re: Building Xen Test Days into the conclusion of the Xen 4.3 release cycle
On Tue, 2013-04-23 at 15:29 +0100, Lars Kurth wrote:> > Views? Should I start announcing the dates and promote the Test Days?All sounds good, I think you should go ahead. Ian.
Lars Kurth
2013-Apr-25 08:25 UTC
Re: Building Xen Test Days into the conclusion of the Xen 4.3 release cycle
>> Views? Should I start announcing the dates and promote the Test Days? > All sounds good, I think you should go ahead.Any objections? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Lars Kurth
2013-Apr-25 11:18 UTC
Re: Building Xen Test Days into the conclusion of the Xen 4.3 release cycle
On 25/04/2013 09:25, Lars Kurth wrote:> >> Views? Should I start announcing the dates and promote the Test Days? > > All sounds good, I think you should go ahead. > Any objections? >Appears there are none. I created - http://wiki.xen.org/wiki/Xen_4.3_RC1_test_instructions which needs to be popolated - Remember Mondy is Docs Day - A perfect opportunity to do this - Also updated http://wiki.xen.org/wiki/Xen_Test_Days Lars _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
George Dunlap
2013-Apr-25 14:15 UTC
Re: Building Xen Test Days into the conclusion of the Xen 4.3 release cycle
On Thu, Apr 25, 2013 at 12:18 PM, Lars Kurth <lars.kurth@xen.org> wrote:> On 25/04/2013 09:25, Lars Kurth wrote: > >>> Views? Should I start announcing the dates and promote the Test Days? >> All sounds good, I think you should go ahead. > Any objections? > > Appears there are none. I created > - http://wiki.xen.org/wiki/Xen_4.3_RC1_test_instructions which needs to be > popolated > - Remember Mondy is Docs Day - A perfect opportunity to do this > - Also updated http://wiki.xen.org/wiki/Xen_Test_DaysI was thinking it would be good to think about a "test matrix". Below is a sort of summary of what I came up with -- I''d appreciate if people could add things I''m missing. In general I was thinking "variables" as a sort of "state of the system" V, and then we want to test specific functionality f given state V. -George Variables - Host Hardware - CPU (Intel, AMD) - NUMA topology - Amount of memory - IOMMU, &c - Host config - Domain 0 distro { Debian, Fedora, Ubuntu, Arch, Alpine, NetBSD } - Domain 0 kernel { distro, mainline most recent, mainline stable } - Network driver domain - Networking (bridging / OVS / &c) - With / without default XSM policy - Guest config - Guest OS - PV / PVHVM (for Linux / NetBSD guests) - qemu version ( qemu-xen-traditional / qemu-xen / straight qemu ) - Stubdoms - Emualted / PV devices (disk, network) - USB pass-through Functionality - Build / install - Basic create, list, shutdown, reboot, destroy, suspend/resume, migrate, pause, unpause, console - Extended basic cd-insert, cd-eject, button-press, vcpu-pin, vcpu-set, domid, domname, rename, trigger, sysrq, info, dmesg, top, config-update, info - Guest devices - PCI pass-through - Network / block attach - Host usb (config file only for 4.3) - qxl spice - Memory - Ballooning PoD, mem-max, mem-set - Page sharing - Hypervisor swap - Scheduling parameters - NUMA - debug-keys - vtpm - tmem - cpupools - remus - alternate block / network scripts - Flask Types of testing - Smoke-test Just do something quickly to see if it works - Normal functional test Try to use it the way you expect it to work - Stress-testing Automated heavy use over long period of time - Edge-case testing Try to use it in ways you don''t expect it to be used - Performance Test it specifically measuring the performance