Xen devel - Dec 2005 - [PATCH] [XM-TEST] Add support for VMX guests in xm-test

Here''s a patch to add initial support for vmx guests in xm-test. This
patch applies to changeset 8272:b215584242c4.

I''ve run it on both vmx and non-vmx guests. I got the following report
for running with vmx enabled:

Xm-test execution summary:
  PASS:  69
  FAIL:  23
  XPASS: 0
  XFAIL: 1
 
 
Details:
 
 FAIL: 01_block_attach_device_pos
         Device is not actually connected to the domU
 
 FAIL: 02_block_attach_file_device_pos
         Device is not actually connected to the domU
 
 FAIL: 04_block_attach_device_repeatedly_pos
         Device is not actually attached to domU
 
 FAIL: 05_block_attach_and_dettach_device_repeatedly_pos
         Failed to attach block device /proc/partitions does not show
that!
 
 FAIL: 09_block_attach_and_dettach_device_check_data_pos
         Failed to dettach block device /proc/partitions still showing
that!
 
 FAIL: 03_block-list_anotherbd_pos
         Failed to verify that block dev is attached on DomainU
 
 FAIL: 06_block-list_checkremove_pos
         hda1 still shown in block-list after detach!
 
 FAIL: 01_block-destroy_btblock_pos
         domU reported block device still connected!
 
 FAIL: 02_block-destroy_rtblock_pos
         block-detach failed to detach block device
 
 FAIL: 05_block-destroy_byname_pos
        Unknown reason
 
 FAIL: 06_block-destroy_check_list_pos
         block-list does not show that hda1 was removed
 
 FAIL: 11_create_concurrent_pos
         [4] Failed to attach console to 11_create_4
 
 FAIL: 12_create_concurrent_stress_pos
         Timed out waiting for console
 
 FAIL: 13_create_multinic_pos
         (9 nics) Console didn''t respond probably crashed!
 
 FAIL: 01_memset_basic_pos
         The DomU command ''cat /proc/xen/balloon'' failed.
 
 FAIL: 03_memset_random_pos
         Invalid domU meminfo line
 
 FAIL: 01_network_attach_pos
         Network device is not actually connected to domU
 
 FAIL: 02_network_attach_detach_pos
         Network device is not actually connected to domU
 
XFAIL: 03_network_attach_detach_multiple_pos
         Network device is not actually connected to domU
 
 FAIL: 01_shutdown_basic_pos
         Guest domain failed to shutdown
 
 FAIL: 02_sysrq_sync_pos
         Sync SysRq not delivered
 
 FAIL: 03_sysrq_withreboot_pos
         sysrq failed with 256 != 0



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, Dec 07, 2005 at 02:04:38PM -0800, Daniel Stekloff wrote:
> 
> Here''s a patch to add initial support for vmx guests in xm-test.
This
> patch applies to changeset 8272:b215584242c4.
I''ve applied this.  I would prefer it in the future if you could
separate out
unrelated changes into different patches.  For example, you have removed the
unused __randomize facility from XmTestDomain with this patch.  Removing it is
a useful thing to do, and appreciated, but it''s not related to VMX
guest
support, so it would be better if you could provide such things as separate
patches in the future.

That said, VMX support for xm-test is a good idea, and your work is gratefully
received.
> I''ve run it on both vmx and non-vmx guests. I got the following
report
> for running with vmx enabled:
> 
> Xm-test execution summary:
>   PASS:  69
>   FAIL:  23
>   XPASS: 0
>   XFAIL: 1
Do we have an idea as to the nature of these failures?  Are these things that
will never be testable or implementable due to the nature of VMX domains, or
are these failures, plain and simple?

I would like to see a separate XFAIL_TESTS_VMX or something like that, so that
we can indicate which tests currently fail on VMX, and a separate
DISABLED_TESTS_VMX to indicate those that will never pass on VMX.  What do you
think?

Ewan.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

EM> I would like to see a separate XFAIL_TESTS_VMX or something like
EM> that, so that we can indicate which tests currently fail on VMX,
EM> and a separate DISABLED_TESTS_VMX to indicate those that will
EM> never pass on VMX.  What do you think?

So far, any test that detects that it is unable to run on the current
hardware SKIPs instead of FAIL.  For example, a test that requires
multiple physical CPUs and detects a single CPU box will call SKIP().
I think this is better than statically defining it in the Makefile,
since it''s possible that a certain feature may be available later on
later revisions of VT hardware; the test would be the best place
(IMHO) to make that determination.

-- 
Dan Smith
IBM Linux Technology Center
Open Hypervisor Team
email: danms@us.ibm.com


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, 2005-12-09 at 11:17 +0000, Ewan Mellor wrote:
> On Wed, Dec 07, 2005 at 02:04:38PM -0800, Daniel Stekloff wrote:
> 
> > 
> > Here''s a patch to add initial support for vmx guests in
xm-test. This
> > patch applies to changeset 8272:b215584242c4.
> 
> I''ve applied this.  I would prefer it in the future if you could
separate out
> unrelated changes into different patches.  For example, you have removed
the
> unused __randomize facility from XmTestDomain with this patch.  Removing it
is
> a useful thing to do, and appreciated, but it''s not related to VMX
guest
> support, so it would be better if you could provide such things as separate
> patches in the future.


Yes, I should have done that. Sorry...


> That said, VMX support for xm-test is a good idea, and your work is
gratefully
> received.
> 
> > I''ve run it on both vmx and non-vmx guests. I got the
following report
> > for running with vmx enabled:
> > 
> > Xm-test execution summary:
> >   PASS:  69
> >   FAIL:  23
> >   XPASS: 0
> >   XFAIL: 1
> 
> Do we have an idea as to the nature of these failures?  Are these things
that
> will never be testable or implementable due to the nature of VMX domains,
or
> are these failures, plain and simple?

Some of the tests fail because they aren''t appropriate for full virt
domains. These will need to be edited to check to see if VMX is enabled
or not. 

Other tests aren''t supported yet because the functionality
hasn''t been
implemented. These too could be SKIP''ed. 

And, I imagine there are some that are simple failures.

I need to take a closer look at the tests themselves, now that it''s at
least running for vmx guests. 

Thanks,

Dan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, 2005-12-09 at 06:33 -0800, Dan Smith wrote:
> EM> I would like to see a separate XFAIL_TESTS_VMX or something like
> EM> that, so that we can indicate which tests currently fail on VMX,
> EM> and a separate DISABLED_TESTS_VMX to indicate those that will
> EM> never pass on VMX.  What do you think?
> 
> So far, any test that detects that it is unable to run on the current
> hardware SKIPs instead of FAIL.  For example, a test that requires
> multiple physical CPUs and detects a single CPU box will call SKIP().
> I think this is better than statically defining it in the Makefile,
> since it''s possible that a certain feature may be available later
on
> later revisions of VT hardware; the test would be the best place
> (IMHO) to make that determination.

As Dan Smith and I have discussed, our goal was to start by configuring
the system one way or another - either VMX enabled or not. We can
perform checks in the tests themselves to see if VMX is enabled and SKIP
if not appropriate. 

We are aiming for running para virt and full virt tests concurrently on
the appropriate hardware. Dan Smith is cleaning up XenDomain.py toward
this goal. 

Questions:

1) How shall we do reporting? Should we separate VMX systems from the
current reporting structure? Should there be x86_64 VMX separate from
32bit VMX? I haven''t submitted a test yet, because I don''t
want to drag
down the numbers of para-virt testing.

2) How shall we refer to full-virt and para-virt domains in the future?
DomU vs VMX guest? Since our goal is to test them concurrently on the
appropriate hardware, how shall we refer to them in the tests?

Thanks,

Dan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

DS> 1) How shall we do reporting? Should we separate VMX systems from
DS> the current reporting structure?

I think that if we get all of the tests modified so that they SKIP
when appropriate, then we could let VMX tests merge into the existing
reporting structure.  However, I think that the plots on the front
page should be modified to show "number of failures", or perhaps
"percent of attempted tests that passed".  I think this would still
give us a reasonable view of how well things are going, including VMX
tests. 

Perhaps someone has a better metric that would make more sense?

DS> 2) How shall we refer to full-virt and para-virt domains in the
DS> future?  DomU vs VMX guest? Since our goal is to test them
DS> concurrently on the appropriate hardware, how shall we refer to
DS> them in the tests?

For the platform-agnostic tests (i.e. tests that will just start guest
domains and will work on either paravirt or VMX), we could refer to
them simply as "guests" or "domains".  Tests that
specifically start
PV-only or VMX-only domains can refer to them as something else.
Perhaps PVDomain and HVDomain (HV for Hardware Virtualization)?  This
would cover both VT and Pacifica.

-- 
Dan Smith
IBM Linux Technology Center
Open Hypervisor Team
email: danms@us.ibm.com


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel