thr3ads.net - Xen devel - [Xen-devel] [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it [Mar 2009]

If this information is useful, please help other people find it:
Share via:

Jeremy Fitzhardinge

2009-Mar-08 05:31 UTC

[Xen-devel] [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Mask X86_FEATURE_XSAVE in cpuid leaf 1, ecx, as we don''t allow guests
to
use it (by setting cr4.OSXSAVE).

This prevents crashes in pvops dom0 kernels, as new versions of Linux try
to use this feature.

(This patch deals with dom0 cpuid; I''m not sure where the right place
to
set up the default mask for domU is.)

Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>

diff -r 7f573cb76db4 xen/arch/x86/traps.c
--- a/xen/arch/x86/traps.c	Tue Mar 03 13:22:28 2009 +0000
+++ b/xen/arch/x86/traps.c	Sat Mar 07 17:02:03 2009 -0800
@@ -757,6 +757,7 @@
         __clear_bit(X86_FEATURE_XTPR % 32, &c);
         __clear_bit(X86_FEATURE_PDCM % 32, &c);
         __clear_bit(X86_FEATURE_DCA % 32, &c);
+        __clear_bit(X86_FEATURE_XSAVE % 32, &c);
         if ( !cpu_has_apic )
            __clear_bit(X86_FEATURE_X2APIC % 32, &c);
         __set_bit(X86_FEATURE_HYPERVISOR % 32, &c);
diff -r 7f573cb76db4 xen/include/asm-x86/cpufeature.h
--- a/xen/include/asm-x86/cpufeature.h	Tue Mar 03 13:22:28 2009 +0000
+++ b/xen/include/asm-x86/cpufeature.h	Sat Mar 07 17:02:03 2009 -0800
@@ -95,6 +95,7 @@
 #define X86_FEATURE_SSE4_2	(4*32+20) /* Streaming SIMD Extensions 4.2 */
 #define X86_FEATURE_X2APIC	(4*32+21) /* Extended xAPIC */
 #define X86_FEATURE_POPCNT	(4*32+23) /* POPCNT instruction */
+#define X86_FEATURE_XSAVE	(4*32+26) /* XSAVE/XRSTOR/XSETBV/XGETBV */
 #define X86_FEATURE_HYPERVISOR	(4*32+31) /* Running under some hypervisor */
 
 /* VIA/Cyrix/Centaur-defined CPU features, CPUID level 0xC0000001, word 5 */



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2009-Mar-08 08:52 UTC

head link

[Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Performed:-

cd /usr/src/linux-2.6-xen

git pull

make

make modules_install install

mkinitramfs -o /boot/initrd-2.6.29-rc7-tip.img 2.6.29-rc7-tip

Now kernel boots on Q9550
The most recent serial console boot log attached

Boris.
P.S.
 

--- On Sun, 3/8/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
From: Jeremy Fitzhardinge <jeremy@goop.org>
Subject: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests
to use it
To: "Keir Fraser" <keir.fraser@eu.citrix.com>
Cc: "Xen-devel" <xen-devel@lists.xensource.com>, "Marc - A.
Dahlhaus [ Administration | Westermann GmbH ]" <mad@wol.de>,
"Boris Derzhavets" <bderzhavets@yahoo.com>
Date: Sunday, March 8, 2009, 12:31 AM

Mask X86_FEATURE_XSAVE in cpuid leaf 1, ecx, as we don''t allow guests
to
use it (by setting cr4.OSXSAVE).

This prevents crashes in pvops dom0 kernels, as new versions of Linux try
to use this feature.

(This patch deals with dom0 cpuid; I''m not sure where the right place
to
set up the default mask for domU is.)

Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>

diff -r 7f573cb76db4 xen/arch/x86/traps.c
--- a/xen/arch/x86/traps.c	Tue Mar 03 13:22:28 2009 +0000
+++ b/xen/arch/x86/traps.c	Sat Mar 07 17:02:03 2009 -0800
@@ -757,6 +757,7 @@
        __clear_bit(X86_FEATURE_XTPR % 32, &c);
        __clear_bit(X86_FEATURE_PDCM % 32, &c);
        __clear_bit(X86_FEATURE_DCA % 32, &c);
+        __clear_bit(X86_FEATURE_XSAVE % 32, &c);
        if ( !cpu_has_apic )
           __clear_bit(X86_FEATURE_X2APIC % 32, &c);
        __set_bit(X86_FEATURE_HYPERVISOR % 32, &c);
diff -r 7f573cb76db4 xen/include/asm-x86/cpufeature.h
--- a/xen/include/asm-x86/cpufeature.h	Tue Mar 03 13:22:28 2009 +0000
+++ b/xen/include/asm-x86/cpufeature.h	Sat Mar 07 17:02:03 2009 -0800
@@ -95,6 +95,7 @@
#define X86_FEATURE_SSE4_2	(4*32+20) /* Streaming SIMD Extensions 4.2 */
#define X86_FEATURE_X2APIC	(4*32+21) /* Extended xAPIC */
#define X86_FEATURE_POPCNT	(4*32+23) /* POPCNT instruction */
+#define X86_FEATURE_XSAVE	(4*32+26) /* XSAVE/XRSTOR/XSETBV/XGETBV */
#define X86_FEATURE_HYPERVISOR	(4*32+31) /* Running under some hypervisor */

/* VIA/Cyrix/Centaur-defined CPU features, CPUID level 0xC0000001, word 5 */





      


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2009-Mar-08 09:13 UTC

head link

[Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

On 08/03/2009 05:31, "Jeremy Fitzhardinge" <jeremy@goop.org>
wrote:
> Mask X86_FEATURE_XSAVE in cpuid leaf 1, ecx, as we don''t allow
guests to
> use it (by setting cr4.OSXSAVE).
> 
> This prevents crashes in pvops dom0 kernels, as new versions of Linux try
> to use this feature.
> 
> (This patch deals with dom0 cpuid; I''m not sure where the right
place to
> set up the default mask for domU is.)
Thanks, the whole problem stems from blacklisting rather than whitelisting
feature bits. I might try and do a more comprehensive job for xen-unstable
at least. This is good for 3.3.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2009-Mar-09 11:18 UTC

head link

[Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

I was able to go through creating PV DomUs images via corresponding
pygrub''s profiles for F10,CentOS 5.2,SNV_106 (64-bit all).
However, only Solaris Nevada PV DomU gets loaded via standard
pygrub profile. 

Attempt to load F10 PV:-

[2009-03-09 05:33:38 6351] DEBUG (XendDomainInfo:92)
XendDomainInfo.create([''vm'', [''name'',
''F10PV''], [''memory'', 2048],
[''on_reboot'', ''restart''],
[''on_crash'', ''restart''],
[''vcpus'', 1], [''on_xend_start'',
''ignore''], [''on_xend_stop'',
''ignore''], [''bootloader'',
''/usr/bin/pygrub''], [''bootloader_args'',
''-q''], [''image'', [''linux'',
[''videoram'', 4]]], [''device'',
[''vbd'', [''uname'',
''phy:/dev/sdb5''], [''dev'',
''xvda''], [''mode'', ''w'']]],
[''device'', [''vif'',
[''bridge'', ''eth0'']]],
[''device'', [''vkbd'']],
[''device'', [''vfb'',
[''vncunused'', ''1''],
[''type'', ''vnc''],
[''xauthority'', ''/root/.Xauthority'']]]])
[2009-03-09 05:33:38 6351] DEBUG (XendDomainInfo:2183)
XendDomainInfo.constructDomain
[2009-03-09 05:33:38 6351] DEBUG (balloon:166) Balloon: 7239368 KiB free; need
2048; done.
[2009-03-09 05:33:38 6351] DEBUG (XendDomain:452) Adding Domain: 8
[2009-03-09 05:33:38 6351] DEBUG (XendDomainInfo:2362)
XendDomainInfo.initDomain: 8 256
[2009-03-09 05:33:38 12975] DEBUG (XendBootloader:113) Launching bootloader as
[''/usr/bin/pygrub'',
''--output=/var/run/xend/boot/xenbl.29001'',
''-q'', ''/dev/sdb5''].
[2009-03-09 05:33:39 6351] ERROR (XendBootloader:213) Boot loader
didn''t return any data!
[2009-03-09 05:33:39 6351] ERROR (XendDomainInfo:472) VM start failed
Traceback (most recent call last):
  File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line 452, in
start
    XendTask.log_progress(31, 60, self._initDomain)
  File "//usr/lib64/python/xen/xend/XendTask.py", line 209, in
log_progress
    retval = func(*args, **kwds)
  File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line 2364, in
_initDomain
    self._configureBootloader()
  File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line 2765, in
_configureBootloader
    bootloader_args, kernel, ramdisk, args)
  File "//usr/lib64/python/xen/xend/XendBootloader.py", line 214, in
bootloader
    raise VmError, msg

Attempt to load CentOS 5.2 PV:-

[2009-03-09 06:45:39 6351] DEBUG (XendDomainInfo:92)
XendDomainInfo.create([''vm'', [''name'',
''RHEL52PV''], [''memory'', 2048],
[''vcpus'', 1], [''on_xend_start'',
''ignore''], [''on_xend_stop'',
''ignore''], [''image'',
[''linux'', [''videoram'', 4]]],
[''device'', [''vbd'',
[''uname'', ''phy:/dev/sdb5''],
[''dev'', ''xvda''], [''mode'',
''w'']]], [''device'', [''vif'',
[''bridge'', ''eth0'']]]])
[2009-03-09 06:45:39 6351] DEBUG (XendDomainInfo:2183)
XendDomainInfo.constructDomain
[2009-03-09 06:45:39 6351] DEBUG (balloon:166) Balloon: 7239368 KiB free; need
2048; done.
[2009-03-09 06:45:39 6351] DEBUG (XendDomain:452) Adding Domain: 12
[2009-03-09 06:45:39 6351] DEBUG (XendDomainInfo:2362)
XendDomainInfo.initDomain: 12 256
[2009-03-09 06:45:39 16739] DEBUG (XendBootloader:113) Launching bootloader as
[''/usr/bin/pygrub'',
''--output=/var/run/xend/boot/xenbl.15023'',
''/dev/sdb5''].
[2009-03-09 06:45:44 6351] ERROR (XendBootloader:213) Boot loader
didn''t return any data!
[2009-03-09 06:45:44 6351] ERROR (XendDomainInfo:472) VM start failed
Traceback (most recent call last):
  File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line 452, in
start
    XendTask.log_progress(31, 60, self._initDomain)
  File "//usr/lib64/python/xen/xend/XendTask.py", line 209, in
log_progress
    retval = func(*args, **kwds)
  File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line 2364, in
_initDomain
    self._configureBootloader()
  File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line 2765, in
_configureBootloader
    bootloader_args, kernel, ramdisk, args)
  File "//usr/lib64/python/xen/xend/XendBootloader.py", line 214, in
bootloader
    raise VmError, msg
VmError: Boot loader didn''t return any data!









--- On Sun, 3/8/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
From: Jeremy Fitzhardinge <jeremy@goop.org>
Subject: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests
to use it
To: "Keir Fraser" <keir.fraser@eu.citrix.com>
Cc: "Xen-devel" <xen-devel@lists.xensource.com>, "Marc - A.
Dahlhaus [ Administration | Westermann GmbH ]" <mad@wol.de>,
"Boris Derzhavets" <bderzhavets@yahoo.com>
Date: Sunday, March 8, 2009, 12:31 AM

Mask X86_FEATURE_XSAVE in cpuid leaf 1, ecx, as we don''t allow guests
to
use it (by setting cr4.OSXSAVE).

This prevents crashes in pvops dom0 kernels, as new versions of Linux try
to use this feature.

(This patch deals with dom0 cpuid; I''m not sure where the right place
to
set up the default mask for domU is.)

Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>

diff -r 7f573cb76db4 xen/arch/x86/traps.c
--- a/xen/arch/x86/traps.c	Tue Mar 03 13:22:28 2009 +0000
+++ b/xen/arch/x86/traps.c	Sat Mar 07 17:02:03 2009 -0800
@@ -757,6 +757,7 @@
        __clear_bit(X86_FEATURE_XTPR % 32, &c);
        __clear_bit(X86_FEATURE_PDCM % 32, &c);
        __clear_bit(X86_FEATURE_DCA % 32, &c);
+        __clear_bit(X86_FEATURE_XSAVE % 32, &c);
        if ( !cpu_has_apic )
           __clear_bit(X86_FEATURE_X2APIC % 32, &c);
        __set_bit(X86_FEATURE_HYPERVISOR % 32, &c);
diff -r 7f573cb76db4 xen/include/asm-x86/cpufeature.h
--- a/xen/include/asm-x86/cpufeature.h	Tue Mar 03 13:22:28 2009 +0000
+++ b/xen/include/asm-x86/cpufeature.h	Sat Mar 07 17:02:03 2009 -0800
@@ -95,6 +95,7 @@
#define X86_FEATURE_SSE4_2	(4*32+20) /* Streaming SIMD Extensions 4.2 */
#define X86_FEATURE_X2APIC	(4*32+21) /* Extended xAPIC */
#define X86_FEATURE_POPCNT	(4*32+23) /* POPCNT instruction */
+#define X86_FEATURE_XSAVE	(4*32+26) /* XSAVE/XRSTOR/XSETBV/XGETBV */
#define X86_FEATURE_HYPERVISOR	(4*32+31) /* Running under some hypervisor */

/* VIA/Cyrix/Centaur-defined CPU features, CPUID level 0xC0000001, word 5 */





      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jan Beulich

2009-Mar-09 11:26 UTC

head link

Re: [Xen-devel] [PATCH] xen: mask XSAVE in cpuid since we don''t allowguests to use it

>>> Jeremy Fitzhardinge <jeremy@goop.org> 08.03.09 06:31
>>>
>Mask X86_FEATURE_XSAVE in cpuid leaf 1, ecx, as we don''t allow
guests to
>use it (by setting cr4.OSXSAVE).
I actually disagree to this. The kernel should be dealing with the situation
by inspecting the OSXSAVE bit in CPUID when running under Xen, since
it''s
really Xen that''s the ''OS'' for that particular
feature.

Then, once Keir gets willing to take Intel xsave patch(es), no further change
is going to be needed here. And the same kernel will work (assuming it has
no other dependencies) on older Xen not having this bit masked.

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2009-Mar-09 11:26 UTC

head link

[Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

If you¹re using a recent xen-unstable then you might try again with the
current tip. A couple of changes went in recently (including one fix today).

 -- Keir

On 09/03/2009 11:18, "Boris Derzhavets" <bderzhavets@yahoo.com>
wrote:
> I was able to go through creating PV DomUs images via corresponding
pygrub''s
> profiles for F10,CentOS 5.2,SNV_106 (64-bit all).
> However, only Solaris Nevada PV DomU gets loaded via standard
> pygrub profile. 
> 
> Attempt to load F10 PV:-
> 
> [2009-03-09 05:33:38 6351] DEBUG (XendDomainInfo:92)
> XendDomainInfo.create([''vm'', [''name'',
''F10PV''], [''memory'', 2048],
> [''on_reboot'', ''restart''],
[''on_crash'', ''restart''],
[''vcpus'', 1],
> [''on_xend_start'', ''ignore''],
[''on_xend_stop'', ''ignore''],
[''bootloader'',
> ''/usr/bin/pygrub''], [''bootloader_args'',
''-q''], [''image'', [''linux'',
> [''videoram'', 4]]], [''device'',
[''vbd'', [''uname'',
''phy:/dev/sdb5''], [''dev'',
> ''xvda''], [''mode'',
''w'']]], [''device'', [''vif'',
[''bridge'', ''eth0'']]],
[''device'',
> [''vkbd'']], [''device'',
[''vfb'', [''vncunused'',
''1''], [''type'', ''vnc''],
> [''xauthority'', ''/root/.Xauthority'']]]])
> [2009-03-09 05:33:38 6351] DEBUG (XendDomainInfo:2183)
> XendDomainInfo.constructDomain
> [2009-03-09 05:33:38 6351] DEBUG (balloon:166) Balloon: 7239368 KiB free;
need
> 2048; done.
> [2009-03-09 05:33:38 6351] DEBUG (XendDomain:452) Adding Domain: 8
> [2009-03-09 05:33:38 6351] DEBUG (XendDomainInfo:2362)
> XendDomainInfo.initDomain: 8 256
> [2009-03-09 05:33:38 12975] DEBUG (XendBootloader:113) Launching bootloader
as
> [''/usr/bin/pygrub'',
''--output=/var/run/xend/boot/xenbl.29001'',
''-q'',
> ''/dev/sdb5''].
> [2009-03-09 05:33:39 6351] ERROR (XendBootloader:213) Boot loader
didn''t
> return any data!
> [2009-03-09 05:33:39 6351] ERROR (XendDomainInfo:472) VM start failed
> Traceback (most recent call last):
>   File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line 452,
in start
>     XendTask.log_progress(31, 60, self._initDomain)
>   File "//usr/lib64/python/xen/xend/XendTask.py", line 209, in
log_progress
>     retval = func(*args, **kwds)
>   File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line
2364, in
> _initDomain
>     self._configureBootloader()
>   File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line
2765, in
> _configureBootloader
>     bootloader_args, kernel, ramdisk, args)
>   File "//usr/lib64/python/xen/xend/XendBootloader.py", line 214,
in
> bootloader
>     raise VmError, msg
> 
> Attempt to load CentOS 5.2 PV:-
> 
> [2009-03-09 06:45:39 6351] DEBUG (XendDomainInfo:92)
> XendDomainInfo.create([''vm'', [''name'',
''RHEL52PV''], [''memory'', 2048],
[''vcpus'',
> 1], [''on_xend_start'', ''ignore''],
[''on_xend_stop'', ''ignore''],
[''image'',
> [''linux'', [''videoram'', 4]]],
[''device'', [''vbd'',
[''uname'', ''phy:/dev/sdb5''],
> [''dev'', ''xvda''],
[''mode'', ''w'']]],
[''device'', [''vif'',
[''bridge'', ''eth0'']]]])
> [2009-03-09 06:45:39 6351] DEBUG (XendDomainInfo:2183)
> XendDomainInfo.constructDomain
> [2009-03-09 06:45:39 6351] DEBUG (balloon:166) Balloon: 7239368 KiB free;
need
> 2048; done.
> [2009-03-09 06:45:39 6351] DEBUG (XendDomain:452) Adding Domain: 12
> [2009-03-09 06:45:39 6351] DEBUG (XendDomainInfo:2362)
> XendDomainInfo.initDomain: 12 256
> [2009-03-09 06:45:39 16739] DEBUG (XendBootloader:113) Launching bootloader
as
> [''/usr/bin/pygrub'',
''--output=/var/run/xend/boot/xenbl.15023'',
''/dev/sdb5''].
> [2009-03-09 06:45:44 6351] ERROR (XendBootloader:213) Boot loader
didn''t
> return any data!
> [2009-03-09 06:45:44 6351] ERROR (XendDomainInfo:472) VM start failed
> Traceback (most recent call last):
>   File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line 452,
in start
>     XendTask.log_progress(31, 60, self._initDomain)
>   File "//usr/lib64/python/xen/xend/XendTask.py", line 209, in
log_progress
>     retval = func(*args, **kwds)
>   File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line
2364, in
> _initDomain
>     self._configureBootloader()
>   File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line
2765, in
> _configureBootloader
>     bootloader_args, kernel, ramdisk, args)
>   File "//usr/lib64/python/xen/xend/XendBootloader.py", line 214,
in
> bootloader
>     raise VmError, msg
> VmError: Boot loader didn''t return any data!
> 
> 
> 
> 
> 
> 
> 
> 
> 
> --- On Sun, 3/8/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
>> From: Jeremy Fitzhardinge <jeremy@goop.org>
>> Subject: [PATCH] xen: mask XSAVE in cpuid since we don''t allow
guests to use
>> it
>> To: "Keir Fraser" <keir.fraser@eu.citrix.com>
>> Cc: "Xen-devel" <xen-devel@lists.xensource.com>,
"Marc - A. Dahlhaus [
>> Administration | Westermann GmbH ]" <mad@wol.de>,
"Boris Derzhavets"
>> <bderzhavets@yahoo.com>
>> Date: Sunday, March 8, 2009, 12:31 AM
>> 
>> Mask X86_FEATURE_XSAVE in cpuid leaf 1, ecx, as we don''t allow
guests to
>> use it (by setting cr4.OSXSAVE).
>> 
>> This prevents crashes in pvops dom0 kernels, as new versions of Linux
try
>> to use this feature.
>> 
>> (This patch deals with dom0 cpuid; I''m not sure where the
right place to
>> set up the default mask for domU is.)
>> 
>> Signed-off-by: Jeremy Fitzhardinge
<jeremy.fitzhardinge@citrix.com>
>> 
>> diff -r 7f573cb76db4 xen/arch/x86/traps.c
>> --- a/xen/arch/x86/traps.c Tue Mar 03 13:22:28 2009 +0000
>> +++ b/xen/arch/x86/traps.c Sat Mar 07 17:02:03 2009 -0800
>> @@ -757,6 +757,7 @@
>>         __clear_bit(X86_FEATURE_XTPR % 32, &c);
>>         __clear_bit(X86_FEATURE_PDCM % 32, &c);
>>         __clear_bit(X86_FEATURE_DCA % 32, &c);
>> +        __clear_bit(X86_FEATURE_XSAVE % 32, &c);
>>         if ( !cpu_has_apic )
>>           
>>  __clear_bit(X86_FEATURE_X2APIC % 32, &c);
>>         __set_bit(X86_FEATURE_HYPERVISOR % 32, &c);
>> diff -r 7f573cb76db4 xen/include/asm-x86/cpufeature.h
>> --- a/xen/include/asm-x86/cpufeature.h Tue Mar 03 13:22:28 2009 +0000
>> +++ b/xen/include/asm-x86/cpufeature.h Sat Mar 07 17:02:03 2009 -0800
>> @@ -95,6 +95,7 @@
>> #define X86_FEATURE_SSE4_2 (4*32+20) /* Streaming SIMD Extensions 4.2
*/
>> #define X86_FEATURE_X2APIC (4*32+21) /* Extended xAPIC */
>> #define X86_FEATURE_POPCNT (4*32+23) /* POPCNT instruction */
>> +#define X86_FEATURE_XSAVE (4*32+26) /* XSAVE/XRSTOR/XSETBV/XGETBV */
>> #define X86_FEATURE_HYPERVISOR (4*32+31) /* Running under some
hypervisor */
>> 
>> /* VIA/Cyrix/Centaur-defined CPU features, CPUID level 0xC0000001, word
5 */
>> 
>> 
> 
>  


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2009-Mar-09 11:29 UTC

head link

Re: [Xen-devel] [PATCH] xen: mask XSAVE in cpuid since we don''t allowguests to use it

On 09/03/2009 11:26, "Jan Beulich" <jbeulich@novell.com> wrote:
>>>> Jeremy Fitzhardinge <jeremy@goop.org> 08.03.09 06:31
>>>
>> Mask X86_FEATURE_XSAVE in cpuid leaf 1, ecx, as we don''t allow
guests to
>> use it (by setting cr4.OSXSAVE).
> 
> I actually disagree to this. The kernel should be dealing with the
situation
> by inspecting the OSXSAVE bit in CPUID when running under Xen, since
it''s
> really Xen that''s the ''OS'' for that particular
feature.
> 
> Then, once Keir gets willing to take Intel xsave patch(es), no further
change
> is going to be needed here. And the same kernel will work (assuming it has
> no other dependencies) on older Xen not having this bit masked.
It seems reasonable to mask XSAVE for now though. I don''t think it can
usefully be used without some Xen support.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jan Beulich

2009-Mar-09 11:31 UTC

head link

[Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''tallow guests to use it

>>> Keir Fraser <keir.fraser@eu.citrix.com> 08.03.09 10:13
>>>
>On 08/03/2009 05:31, "Jeremy Fitzhardinge" <jeremy@goop.org>
wrote:
>
>> Mask X86_FEATURE_XSAVE in cpuid leaf 1, ecx, as we don''t allow
guests to
>> use it (by setting cr4.OSXSAVE).
>> 
>> This prevents crashes in pvops dom0 kernels, as new versions of Linux
try
>> to use this feature.
>> 
>> (This patch deals with dom0 cpuid; I''m not sure where the
right place to
>> set up the default mask for domU is.)
>
>Thanks, the whole problem stems from blacklisting rather than whitelisting
>feature bits. I might try and do a more comprehensive job for xen-unstable
>at least. This is good for 3.3.
I''m not sure whitelisting will be much better than blacklisting - while
now
any feature requiring not-yet-implemented support in Xen must be turned
off explicitly, this would just turn over to requiring explicitly un-hiding any
new feature not requiring Xen''s intervention. A real solution to this
issue
would require some assistance from the hardware vendors I''d think.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2009-Mar-09 11:45 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''tallow guests to use it

On 09/03/2009 11:31, "Jan Beulich" <jbeulich@novell.com> wrote:
>> Thanks, the whole problem stems from blacklisting rather than
whitelisting
>> feature bits. I might try and do a more comprehensive job for
xen-unstable
>> at least. This is good for 3.3.
> 
> I''m not sure whitelisting will be much better than blacklisting -
while now
> any feature requiring not-yet-implemented support in Xen must be turned
> off explicitly, this would just turn over to requiring explicitly un-hiding
> any
> new feature not requiring Xen''s intervention. A real solution to
this issue
> would require some assistance from the hardware vendors I''d think.
Well, it makes the difference between working but without the new feature,
versus quite possibly simply not working at all (and failing in rare and
subtle ways, perhaps). Isn''t one obviously better than the other?

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2009-Mar-09 14:31 UTC

head link

[Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Just cloned,built and installed the most recent xen-unstable.
Seems like "make tools"  reports some errors , but still finishes 
with no
error message.
CentOS 5.2 PV DomU may be loaded OK only the very first time.
The next attempt after DomU shutdown brings it to paused state.
After "xm unpause DomainName":-

i8042.c: No controller found. in  1 seconds
Red Hat nash version 5.1.19.6 starting
  Reading all physical volumes.  This may take a while...
  No volume groups found
  Volume group "VolGroup00" not found
mount: could not find filesystem ''/dev/root''
setuproot: moving /dev failed: No such file or directory
setuproot: error mounting /proc: No such file or directory
setuproot: error mounting /sys: No such file or directory
switchroot: mount failed: No such file or directory
Kernel panic - not syncing: Attempted to kill init!
 root@ServerUbuntu810:/home/boris/vm# Error: Device 51712 (vbd) could not be
connected. Hotplug scripts not working.

/var/log/xen/xend.log:-

[2009-03-09 10:18:33 6335] DEBUG (XendDomainInfo:2131) No device model
[2009-03-09 10:18:33 6335] DEBUG (XendDomainInfo:2133) Releasing devices
[2009-03-09 10:19:29 6335] DEBUG (DevController:629) hotplugStatusCallback
/local/domain/0/backend/vbd/2/51712/hotplug-status.
[2009-03-09 10:19:29 6335] DEBUG (DevController:643) hotplugStatusCallback 1.
[2009-03-09 10:19:42 6335] ERROR (XendDomainInfo:1983) Failed to restart domain
2.
Traceback (most recent call last):
  File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line 1969, in
_restart
    new_dom.waitForDevices()
  File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line 1020, in
waitForDevices
    self.getDeviceController(devclass).waitForDevices()
  File "//usr/lib64/python/xen/xend/server/DevController.py", line
140, in waitForDevices
    return map(self.waitForDevice, self.deviceIDs())
  File "//usr/lib64/python/xen/xend/server/DevController.py", line
152, in waitForDevice
    self.destroyDevice(devid, False)
  File "//usr/lib64/python/xen/xend/server/blkif.py", line 200, in
destroyDevice
    DevController.destroyDevice(self, devid, force)
  File "//usr/lib64/python/xen/xend/server/DevController.py", line
228, in destroyDevice
    self.writeBackend(dev, ''online'', "0")
  File "//usr/lib64/python/xen/xend/server/DevController.py", line
474, in writeBackend
    raise VmError("Device %s not connected" % devid)
VmError: Device 51712 not connected
[2009-03-09 10:21:06 6335] DEBUG (DevController:629) hotplugStatusCallback
/local/domain/0/backend/vbd/3/51712/hotplug-status.
[2009-03-09 10:21:06 6335] DEBUG (DevController:643) hotplugStatusCallback 2.

Start F10 PV DomU:-


root@ServerUbuntu810:/home/boris/vm# xm create -c f10.pyrun
Using config file "./f10.pyrun".
PCI: Fatal: No config space access function found
IO APIC resources could be not be allocated.
Loading /lib/kbd/keymaps/i386/qwerty/us.map
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
IP: [<ffffffffa00010e5>] backend_changed+0x2d5/0x3af [xen_blkfront]
PGD 7d1f6067 PUD 7d143067 PMD 7d116067 PTE 0
Oops: 0000 [1] SMP 
CPU 1 
Modules linked in: xen_blkfront
Pid: 19, comm: xenwatch Not tainted 2.6.27.5-117.fc10.x86_64 #1
RIP: e030:[<ffffffffa00010e5>]  [<ffffffffa00010e5>]
backend_changed+0x2d5/0x3af [xen_blkfront]
RSP: e02b:ffff88007f40fdf0  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff88007d02d840 RCX: 0000000000000000
RDX: ffff88007f40fe28 RSI: 0000000000000005 RDI: ffff88007ff5f400
RBP: ffff88007f40fe80 R08: ffff88007f40fdd0 R09: ffff88007f40fcc0
R10: 0000000000000001 R11: ffff880000000018 R12: ffff88007ff5f400
R13: ffff88007d042000 R14: 000000000000000a R15: ffffffff817e2f44
FS:  00007fb68f67e6f0(0000) GS:ffff88007fc04880(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000000000000 CR3: 000000007d105000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process xenwatch (pid: 19, threadinfo ffff88007f40e000, task ffff88007ffd5c40)
Stack:  ffff88007f40fe00 ffff88007f40fe40 ffffffff816de500 ffffffff816de500
 ffffffff81455f53 ffffffff81462b99 ffff88007f40fe7c 0000000000000000
 000000000000000a ffff88007d02d840 ffff88007ff5f400 ffffffffa0002ac0
Call Trace:
 [<ffffffff811c91e6>] otherend_changed+0x81/0x8a
 [<ffffffff811c7e14>] xenwatch_thread+0x101/0x134
 [<ffffffff81055199>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff811c7d13>] ? xenwatch_thread+0x0/0x134
 [<ffffffff81054e53>] kthread+0x49/0x76
 [<ffffffff810116e9>] child_rip+0xa/0x11
 [<ffffffff81010a07>] ? restore_args+0x0/0x30
 [<ffffffff810116df>] ? child_rip+0x0/0x11


Code: 58 08 e9 97 00 00 00 49 8b 7d 00 48 c7 c2 b9 15 00 a0 be ed ff ff ff 48 8b
4f 10 31 c0 e8 da 5f 1c e1 e9 cf 00 00 00 49 8b 45 08 <8b> 38 c1 e7 14 0b
78 04 e8 26 55 0e e1 48 85 c0 48 89 c3 75 16
RIP  [<ffffffffa00010e5>] backend_changed+0x2d5/0x3af [xen_blkfront]
 RSP <ffff88007f40fdf0>
CR2: 0000000000000000
---[ end trace bf88f6e1c7a703d2 ]---
root@ServerUbuntu810:/home/boris/vm# Error: Device 51712 (vbd) could not be
connected. Hotplug scripts not working.





--- On Mon, 3/9/09, Keir Fraser <keir.fraser@eu.citrix.com> wrote:
From: Keir Fraser <keir.fraser@eu.citrix.com>
Subject: Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow
guests to use it
To: "bderzhavets@yahoo.com" <bderzhavets@yahoo.com>
Cc: "Xen-devel" <xen-devel@lists.xensource.com>
Date: Monday, March 9, 2009, 7:26 AM

If you¹re using a recent xen-unstable then you might try again with the
current tip. A couple of changes went in recently (including one fix today).

 -- Keir

On 09/03/2009 11:18, "Boris Derzhavets" <bderzhavets@yahoo.com>
wrote:
> I was able to go through creating PV DomUs images via corresponding
pygrub''s> profiles for F10,CentOS 5.2,SNV_106 (64-bit all).
> However, only Solaris Nevada PV DomU gets loaded via standard
> pygrub profile. 
> 
> Attempt to load F10 PV:-
> 
> [2009-03-09 05:33:38 6351] DEBUG (XendDomainInfo:92)
> XendDomainInfo.create([''vm'', [''name'',
''F10PV''],
[''memory'', 2048],> [''on_reboot'', ''restart''],
[''on_crash'',''restart''], [''vcpus'',
1],> [''on_xend_start'', ''ignore''],
[''on_xend_stop'',''ignore''],
[''bootloader'',> ''/usr/bin/pygrub''], [''bootloader_args'',
''-q''],
[''image'', [''linux'',> [''videoram'', 4]]], [''device'',
[''vbd'',[''uname'', ''phy:/dev/sdb5''],
[''dev'',> ''xvda''], [''mode'',
''w'']]], [''device'',[''vif'', [''bridge'',
''eth0'']]],
[''device'',> [''vkbd'']], [''device'',
[''vfb'',[''vncunused'', ''1''],
[''type'', ''vnc''],> [''xauthority'', ''/root/.Xauthority'']]]])
> [2009-03-09 05:33:38 6351] DEBUG (XendDomainInfo:2183)
> XendDomainInfo.constructDomain
> [2009-03-09 05:33:38 6351] DEBUG (balloon:166) Balloon: 7239368 KiB free;
need> 2048; done.
> [2009-03-09 05:33:38 6351] DEBUG (XendDomain:452) Adding Domain: 8
> [2009-03-09 05:33:38 6351] DEBUG (XendDomainInfo:2362)
> XendDomainInfo.initDomain: 8 256
> [2009-03-09 05:33:38 12975] DEBUG (XendBootloader:113) Launching
bootloader as> [''/usr/bin/pygrub'',''--output=/var/run/xend/boot/xenbl.29001'',
''-q'',> ''/dev/sdb5''].
> [2009-03-09 05:33:39 6351] ERROR (XendBootloader:213) Boot loader
didn''t> return any data!
> [2009-03-09 05:33:39 6351] ERROR (XendDomainInfo:472) VM start failed
> Traceback (most recent call last):
>   File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line
452, in start>     XendTask.log_progress(31, 60, self._initDomain)
>   File "//usr/lib64/python/xen/xend/XendTask.py", line 209, in
log_progress>     retval = func(*args, **kwds)
>   File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line
2364, in> _initDomain
>     self._configureBootloader()
>   File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line
2765, in> _configureBootloader
>     bootloader_args, kernel, ramdisk, args)
>   File "//usr/lib64/python/xen/xend/XendBootloader.py", line
214, in> bootloader
>     raise VmError, msg
> 
> Attempt to load CentOS 5.2 PV:-
> 
> [2009-03-09 06:45:39 6351] DEBUG (XendDomainInfo:92)
> XendDomainInfo.create([''vm'', [''name'',
''RHEL52PV''],[''memory'', 2048],
[''vcpus'',> 1], [''on_xend_start'', ''ignore''],
[''on_xend_stop'',
''ignore''], [''image'',> [''linux'', [''videoram'', 4]]],
[''device'',[''vbd'', [''uname'',
''phy:/dev/sdb5''],> [''dev'', ''xvda''],
[''mode'', ''w'']]],[''device'', [''vif'',
[''bridge'',
''eth0'']]]])> [2009-03-09 06:45:39 6351] DEBUG (XendDomainInfo:2183)
> XendDomainInfo.constructDomain
> [2009-03-09 06:45:39 6351] DEBUG (balloon:166) Balloon: 7239368 KiB free;
need> 2048; done.
> [2009-03-09 06:45:39 6351] DEBUG (XendDomain:452) Adding Domain: 12
> [2009-03-09 06:45:39 6351] DEBUG (XendDomainInfo:2362)
> XendDomainInfo.initDomain: 12 256
> [2009-03-09 06:45:39 16739] DEBUG (XendBootloader:113) Launching
bootloader as> [''/usr/bin/pygrub'',''--output=/var/run/xend/boot/xenbl.15023'',
''/dev/sdb5''].> [2009-03-09 06:45:44 6351] ERROR (XendBootloader:213) Boot loader
didn''t> return any data!
> [2009-03-09 06:45:44 6351] ERROR (XendDomainInfo:472) VM start failed
> Traceback (most recent call last):
>   File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line
452, in start>     XendTask.log_progress(31, 60, self._initDomain)
>   File "//usr/lib64/python/xen/xend/XendTask.py", line 209, in
log_progress>     retval = func(*args, **kwds)
>   File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line
2364, in> _initDomain
>     self._configureBootloader()
>   File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line
2765, in> _configureBootloader
>     bootloader_args, kernel, ramdisk, args)
>   File "//usr/lib64/python/xen/xend/XendBootloader.py", line
214, in> bootloader
>     raise VmError, msg
> VmError: Boot loader didn''t return any data!
> 
> 
> 
> 
> 
> 
> 
> 
> 
> --- On Sun, 3/8/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
>> From: Jeremy Fitzhardinge <jeremy@goop.org>
>> Subject: [PATCH] xen: mask XSAVE in cpuid since we don''t allow
guests to use>> it
>> To: "Keir Fraser" <keir.fraser@eu.citrix.com>
>> Cc: "Xen-devel" <xen-devel@lists.xensource.com>,
"Marc - A. Dahlhaus [>> Administration | Westermann GmbH ]" <mad@wol.de>,
"Boris Derzhavets">> <bderzhavets@yahoo.com>
>> Date: Sunday, March 8, 2009, 12:31 AM
>> 
>> Mask X86_FEATURE_XSAVE in cpuid leaf 1, ecx, as we don''t allow
guests to>> use it (by setting cr4.OSXSAVE).
>> 
>> This prevents crashes in pvops dom0 kernels, as new versions of Linux
try>> to use this feature.
>> 
>> (This patch deals with dom0 cpuid; I''m not sure where the
right
place to>> set up the default mask for domU is.)
>> 
>> Signed-off-by: Jeremy Fitzhardinge
<jeremy.fitzhardinge@citrix.com>>> 
>> diff -r 7f573cb76db4 xen/arch/x86/traps.c
>> --- a/xen/arch/x86/traps.c Tue Mar 03 13:22:28 2009 +0000
>> +++ b/xen/arch/x86/traps.c Sat Mar 07 17:02:03 2009 -0800
>> @@ -757,6 +757,7 @@
>>         __clear_bit(X86_FEATURE_XTPR % 32, &c);
>>         __clear_bit(X86_FEATURE_PDCM % 32, &c);
>>         __clear_bit(X86_FEATURE_DCA % 32, &c);
>> +        __clear_bit(X86_FEATURE_XSAVE % 32, &c);
>>         if ( !cpu_has_apic )
>>           
>>  __clear_bit(X86_FEATURE_X2APIC % 32, &c);
>>         __set_bit(X86_FEATURE_HYPERVISOR % 32, &c);
>> diff -r 7f573cb76db4 xen/include/asm-x86/cpufeature.h
>> --- a/xen/include/asm-x86/cpufeature.h Tue Mar 03 13:22:28 2009 +0000
>> +++ b/xen/include/asm-x86/cpufeature.h Sat Mar 07 17:02:03 2009 -0800
>> @@ -95,6 +95,7 @@
>> #define X86_FEATURE_SSE4_2 (4*32+20) /* Streaming SIMD Extensions 4.2
*/>> #define X86_FEATURE_X2APIC (4*32+21) /* Extended xAPIC */
>> #define X86_FEATURE_POPCNT (4*32+23) /* POPCNT instruction */
>> +#define X86_FEATURE_XSAVE (4*32+26) /* XSAVE/XRSTOR/XSETBV/XGETBV */
>> #define X86_FEATURE_HYPERVISOR (4*32+31) /* Running under some
hypervisor */>> 
>> /* VIA/Cyrix/Centaur-defined CPU features, CPUID level 0xC0000001,
word 5 */>> 
>> 
> 
>  




      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jeremy Fitzhardinge

2009-Mar-09 16:40 UTC

head link

Re: [Xen-devel] [PATCH] xen: mask XSAVE in cpuid since we don''t allowguests to use it

Keir Fraser wrote:> It seems reasonable to mask XSAVE for now though. I don''t think it
can
> usefully be used without some Xen support.
>   
You mean Xen support to deal with a new CPU context structure, or 
something requiring specific guest code, like what''s needed to support 
sysenter/syscall?

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2009-Mar-09 17:54 UTC

head link

Re: [Xen-devel] [PATCH] xen: mask XSAVE in cpuid since we don''t allowguests to use it

On 09/03/2009 16:40, "Jeremy Fitzhardinge" <jeremy@goop.org>
wrote:
> Keir Fraser wrote:
>> It seems reasonable to mask XSAVE for now though. I don''t
think it can
>> usefully be used without some Xen support.
>>   
> 
> You mean Xen support to deal with a new CPU context structure, or
> something requiring specific guest code, like what''s needed to
support
> sysenter/syscall?
I mean it indicates new CPU context requiring context switching. But, fully
generally, who knows? :-)

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2009-Mar-10 22:24 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

  Seems like new kernel brought up back some old issues with ubuntu 8.10
devices. It never happened before patching 2.6.29-rc7 kernel on C2D boxes with
the same ubuntu 8.10 server release and 2.6.29-rc5.
  In particular, CentOS 5.2 PV DomU''s xen-hotplug.log looks like:-

xenstore-read: couldn''t read path /local/domain/2/vm
xenstore-read: couldn''t read path /local/domain/9/vm
xenstore-read: couldn''t read path /local/domain/3/vm
xenstore-read: couldn''t read path /local/domain/4/vm
  
 Either xen-tools (or whatever else in xen unstable) requires correlation
with patched kernel.


--- On Mon, 3/9/09, Boris Derzhavets <bderzhavets@yahoo.com> wrote:
From: Boris Derzhavets <bderzhavets@yahoo.com>
Subject: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t allow guests to use it
To: "Keir Fraser" <keir.fraser@eu.citrix.com>
Cc: "Xen-devel" <xen-devel@lists.xensource.com>
Date: Monday, March 9, 2009, 10:31 AM

Just cloned,built and installed the most recent xen-unstable.
Seems like "make tools"  reports some errors , but still finishes 
with no
error message.
CentOS 5.2 PV DomU may be loaded OK only the very first time.
The next attempt after DomU shutdown brings it to paused state.
After "xm unpause DomainName":-

i8042.c: No controller found. in  1 seconds
Red Hat nash version 5.1.19.6 starting
  Reading all physical volumes.  This may take a while...
  No volume groups found
  Volume group "VolGroup00" not found
mount: could not find filesystem ''/dev/root''
setuproot: moving /dev failed: No such file or directory
setuproot: error mounting /proc: No such file or directory
setuproot: error mounting /sys: No such file or directory
switchroot: mount failed: No such file or
 directory
Kernel panic - not syncing: Attempted to kill init!
 root@ServerUbuntu810:/home/boris/vm# Error: Device 51712 (vbd) could not be
connected. Hotplug scripts not working.

/var/log/xen/xend.log:-

[2009-03-09 10:18:33 6335] DEBUG (XendDomainInfo:2131) No device model
[2009-03-09 10:18:33 6335] DEBUG (XendDomainInfo:2133) Releasing devices
[2009-03-09 10:19:29 6335] DEBUG (DevController:629) hotplugStatusCallback
/local/domain/0/backend/vbd/2/51712/hotplug-status.
[2009-03-09 10:19:29 6335] DEBUG (DevController:643) hotplugStatusCallback 1.
[2009-03-09 10:19:42 6335] ERROR (XendDomainInfo:1983) Failed to restart domain
2.
Traceback (most recent call last):
  File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line 1969, in
_restart
    new_dom.waitForDevices()
  File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line 1020, in
waitForDevices
   
 self.getDeviceController(devclass).waitForDevices()
  File "//usr/lib64/python/xen/xend/server/DevController.py", line
140, in waitForDevices
    return map(self.waitForDevice, self.deviceIDs())
  File "//usr/lib64/python/xen/xend/server/DevController.py", line
152, in waitForDevice
    self.destroyDevice(devid, False)
  File "//usr/lib64/python/xen/xend/server/blkif.py", line 200, in
destroyDevice
    DevController.destroyDevice(self, devid, force)
  File "//usr/lib64/python/xen/xend/server/DevController.py", line
228, in destroyDevice
    self.writeBackend(dev, ''online'', "0")
  File "//usr/lib64/python/xen/xend/server/DevController.py", line
474, in writeBackend
    raise VmError("Device %s not connected" % devid)
VmError: Device 51712 not connected
[2009-03-09 10:21:06 6335] DEBUG (DevController:629)
 hotplugStatusCallback /local/domain/0/backend/vbd/3/51712/hotplug-status.
[2009-03-09 10:21:06 6335] DEBUG (DevController:643) hotplugStatusCallback 2.

Start F10 PV DomU:-


root@ServerUbuntu810:/home/boris/vm# xm create -c f10.pyrun
Using config file "./f10.pyrun".
PCI: Fatal: No config space access function found
IO APIC resources could be not be allocated.
Loading /lib/kbd/keymaps/i386/qwerty/us.map
BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
IP: [<ffffffffa00010e5>] backend_changed+0x2d5/0x3af [xen_blkfront]
PGD 7d1f6067 PUD 7d143067 PMD 7d116067 PTE 0
Oops: 0000 [1] SMP 
CPU 1 
Modules linked in: xen_blkfront
Pid: 19, comm: xenwatch Not tainted 2.6.27.5-117.fc10.x86_64 #1
RIP: e030:[<ffffffffa00010e5>]  [<ffffffffa00010e5>]
backend_changed+0x2d5/0x3af [xen_blkfront]
RSP: e02b:ffff88007f40fdf0  EFLAGS: 00010246
RAX: 0000000000000000
 RBX: ffff88007d02d840 RCX: 0000000000000000
RDX: ffff88007f40fe28 RSI: 0000000000000005 RDI: ffff88007ff5f400
RBP: ffff88007f40fe80 R08: ffff88007f40fdd0 R09: ffff88007f40fcc0
R10: 0000000000000001 R11: ffff880000000018 R12: ffff88007ff5f400
R13: ffff88007d042000 R14: 000000000000000a R15: ffffffff817e2f44
FS:  00007fb68f67e6f0(0000) GS:ffff88007fc04880(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000000000000 CR3: 000000007d105000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process xenwatch (pid: 19, threadinfo ffff88007f40e000, task ffff88007ffd5c40)
Stack:  ffff88007f40fe00 ffff88007f40fe40 ffffffff816de500 ffffffff816de500
 ffffffff81455f53 ffffffff81462b99 ffff88007f40fe7c 0000000000000000
 000000000000000a ffff88007d02d840
 ffff88007ff5f400 ffffffffa0002ac0
Call Trace:
 [<ffffffff811c91e6>] otherend_changed+0x81/0x8a
 [<ffffffff811c7e14>] xenwatch_thread+0x101/0x134
 [<ffffffff81055199>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff811c7d13>] ? xenwatch_thread+0x0/0x134
 [<ffffffff81054e53>] kthread+0x49/0x76
 [<ffffffff810116e9>] child_rip+0xa/0x11
 [<ffffffff81010a07>] ? restore_args+0x0/0x30
 [<ffffffff810116df>] ? child_rip+0x0/0x11


Code: 58 08 e9 97 00 00 00 49 8b 7d 00 48 c7 c2 b9 15 00 a0 be ed ff ff ff 48 8b
4f 10 31 c0 e8 da 5f 1c e1 e9 cf 00 00 00 49 8b 45 08 <8b> 38 c1 e7 14 0b
78 04 e8 26 55 0e e1 48 85 c0 48 89 c3 75 16
RIP  [<ffffffffa00010e5>] backend_changed+0x2d5/0x3af [xen_blkfront]
 RSP <ffff88007f40fdf0>
CR2: 0000000000000000
---[ end trace bf88f6e1c7a703d2
 ]---
root@ServerUbuntu810:/home/boris/vm# Error: Device 51712 (vbd) could not be
connected. Hotplug scripts not working.





--- On Mon, 3/9/09, Keir Fraser <keir.fraser@eu.citrix.com> wrote:
From: Keir Fraser <keir.fraser@eu.citrix.com>
Subject: Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow
guests to use it
To: "bderzhavets@yahoo.com" <bderzhavets@yahoo.com>
Cc: "Xen-devel" <xen-devel@lists.xensource.com>
Date: Monday, March 9, 2009, 7:26 AM

If you¹re using a recent xen-unstable then you might try again with the
current tip. A couple of changes went in recently (including one fix today).

 -- Keir

On 09/03/2009 11:18, "Boris Derzhavets" <bderzhavets@yahoo.com>
wrote:
> I was able to go through creating PV DomUs images via corresponding
pygrub''s> profiles for F10,CentOS 5.2,SNV_106 (64-bit all).
> However, only Solaris Nevada PV DomU gets loaded via standard
> pygrub profile. 
> 
> Attempt to load F10 PV:-
> 
> [2009-03-09 05:33:38 6351] DEBUG (XendDomainInfo:92)
> XendDomainInfo.create([''vm'', [''name'',
''F10PV''],
[''memory'', 2048],> [''on_reboot'', ''restart''],
[''on_crash'',''restart''], [''vcpus'',
1],> [''on_xend_start'', ''ignore''],
[''on_xend_stop'',''ignore''],
[''bootloader'',> ''/usr/bin/pygrub''], [''bootloader_args'',
''-q''],
[''image'', [''linux'',> [''videoram'', 4]]], [''device'',
[''vbd'',[''uname'', ''phy:/dev/sdb5''],
[''dev'',> ''xvda''], [''mode'',
''w'']]], [''device'',[''vif'', [''bridge'',
''eth0'']]],
[''device'',> [''vkbd'']], [''device'',
[''vfb'',[''vncunused'', ''1''],
[''type'', ''vnc''],> [''xauthority'', ''/root/.Xauthority'']]]])
> [2009-03-09 05:33:38 6351] DEBUG
 (XendDomainInfo:2183)> XendDomainInfo.constructDomain
> [2009-03-09 05:33:38 6351] DEBUG (balloon:166) Balloon: 7239368 KiB free;
need> 2048; done.
> [2009-03-09 05:33:38 6351] DEBUG (XendDomain:452) Adding Domain: 8
> [2009-03-09 05:33:38 6351] DEBUG (XendDomainInfo:2362)
> XendDomainInfo.initDomain: 8 256
> [2009-03-09 05:33:38 12975] DEBUG (XendBootloader:113) Launching
bootloader as> [''/usr/bin/pygrub'',''--output=/var/run/xend/boot/xenbl.29001'',
''-q'',> ''/dev/sdb5''].
> [2009-03-09 05:33:39 6351] ERROR (XendBootloader:213) Boot loader
didn''t> return any data!
> [2009-03-09 05:33:39 6351] ERROR (XendDomainInfo:472) VM start failed
> Traceback (most recent call last):
>   File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line
452, in start>     XendTask.log_progress(31, 60, self._initDomain)
>   File "//usr/lib64/python/xen/xend/XendTask.py", line 209, in
log_progress>     retval = func(*args, **kwds)
>   File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line
2364, in> _initDomain
>     self._configureBootloader()
>   File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line
2765, in> _configureBootloader
>     bootloader_args, kernel, ramdisk, args)
>   File "//usr/lib64/python/xen/xend/XendBootloader.py", line
214, in> bootloader
>     raise VmError, msg
> 
> Attempt to load CentOS 5.2 PV:-
> 
> [2009-03-09 06:45:39 6351] DEBUG (XendDomainInfo:92)
> XendDomainInfo.create([''vm'', [''name'',
''RHEL52PV''],[''memory'', 2048],
[''vcpus'',> 1], [''on_xend_start'', ''ignore''],
[''on_xend_stop'',
''ignore''], [''image'',> [''linux'', [''videoram'', 4]]],
[''device'',[''vbd'', [''uname'',
''phy:/dev/sdb5''],> [''dev'', ''xvda''],
[''mode'', ''w'']]],
[''device'', [''vif'',
[''bridge'',
''eth0'']]]])> [2009-03-09 06:45:39 6351] DEBUG (XendDomainInfo:2183)
> XendDomainInfo.constructDomain
> [2009-03-09 06:45:39 6351] DEBUG (balloon:166) Balloon: 7239368 KiB free;
need> 2048; done.
> [2009-03-09 06:45:39 6351] DEBUG (XendDomain:452) Adding Domain: 12
> [2009-03-09 06:45:39 6351] DEBUG (XendDomainInfo:2362)
> XendDomainInfo.initDomain: 12 256
> [2009-03-09 06:45:39 16739] DEBUG (XendBootloader:113) Launching
bootloader as> [''/usr/bin/pygrub'',''--output=/var/run/xend/boot/xenbl.15023'',
''/dev/sdb5''].> [2009-03-09 06:45:44 6351] ERROR (XendBootloader:213) Boot loader
didn''t> return any data!
> [2009-03-09 06:45:44 6351] ERROR (XendDomainInfo:472) VM start failed
> Traceback (most recent call last):
>   File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line
452, in start>    
 XendTask.log_progress(31, 60, self._initDomain)>   File "//usr/lib64/python/xen/xend/XendTask.py", line 209, in
log_progress>     retval = func(*args, **kwds)
>   File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line
2364, in> _initDomain
>     self._configureBootloader()
>   File "//usr/lib64/python/xen/xend/XendDomainInfo.py", line
2765, in> _configureBootloader
>     bootloader_args, kernel, ramdisk, args)
>   File "//usr/lib64/python/xen/xend/XendBootloader.py", line
214, in> bootloader
>     raise VmError, msg
> VmError: Boot loader didn''t return any data!
> 
> 
> 
> 
> 
> 
> 
> 
> 
> --- On Sun, 3/8/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
>> From: Jeremy Fitzhardinge <jeremy@goop.org>
>> Subject: [PATCH] xen: mask XSAVE in cpuid since we don''t allow
guests to use>> it
>> To: "Keir Fraser" <keir.fraser@eu.citrix.com>
>> Cc: "Xen-devel" <xen-devel@lists.xensource.com>,
"Marc - A. Dahlhaus [>> Administration | Westermann GmbH ]" <mad@wol.de>,
"Boris Derzhavets">> <bderzhavets@yahoo.com>
>> Date: Sunday, March 8, 2009, 12:31 AM
>> 
>> Mask X86_FEATURE_XSAVE in cpuid leaf 1, ecx, as we don''t allow
guests to>> use it (by setting cr4.OSXSAVE).
>> 
>> This prevents crashes in pvops dom0 kernels, as new versions of Linux
try>> to use this feature.
>> 
>> (This patch deals with dom0 cpuid; I''m not sure where the
right
place to>> set up the default mask for domU is.)
>> 
>> Signed-off-by: Jeremy Fitzhardinge
<jeremy.fitzhardinge@citrix.com>>> 
>> diff -r 7f573cb76db4
 xen/arch/x86/traps.c>> --- a/xen/arch/x86/traps.c Tue Mar 03 13:22:28 2009 +0000
>> +++ b/xen/arch/x86/traps.c Sat Mar 07 17:02:03 2009 -0800
>> @@ -757,6 +757,7 @@
>>         __clear_bit(X86_FEATURE_XTPR % 32, &c);
>>         __clear_bit(X86_FEATURE_PDCM % 32, &c);
>>         __clear_bit(X86_FEATURE_DCA % 32, &c);
>> +        __clear_bit(X86_FEATURE_XSAVE % 32, &c);
>>         if ( !cpu_has_apic )
>>           
>>  __clear_bit(X86_FEATURE_X2APIC % 32, &c);
>>         __set_bit(X86_FEATURE_HYPERVISOR % 32, &c);
>> diff -r 7f573cb76db4 xen/include/asm-x86/cpufeature.h
>> --- a/xen/include/asm-x86/cpufeature.h Tue Mar 03 13:22:28 2009 +0000
>> +++ b/xen/include/asm-x86/cpufeature.h Sat Mar 07 17:02:03 2009 -0800
>> @@ -95,6 +95,7 @@
>> #define X86_FEATURE_SSE4_2 (4*32+20) /* Streaming SIMD Extensions 4.2
*/>> #define X86_FEATURE_X2APIC (4*32+21) /* Extended xAPIC */
>> #define X86_FEATURE_POPCNT (4*32+23) /* POPCNT instruction */
>> +#define X86_FEATURE_XSAVE (4*32+26) /* XSAVE/XRSTOR/XSETBV/XGETBV */
>> #define X86_FEATURE_HYPERVISOR (4*32+31) /* Running under some
hypervisor */>> 
>> /* VIA/Cyrix/Centaur-defined CPU features, CPUID level 0xC0000001,
word 5 */>> 
>> 
> 
>  




      _______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jeremy Fitzhardinge

2009-Mar-11 00:29 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Boris Derzhavets wrote:>
>   Seems like new kernel brought up back some old issues with ubuntu 8.10
> devices. It never happened before patching 2.6.29-rc7 kernel on C2D 
> boxes with the same ubuntu 8.10 server release and 2.6.29-rc5.
>   In particular, CentOS 5.2 PV DomU''s xen-hotplug.log looks like:-
>
> xenstore-read: couldn''t read path /local/domain/2/vm
> xenstore-read: couldn''t read path /local/domain/9/vm
> xenstore-read: couldn''t read path /local/domain/3/vm
> xenstore-read: couldn''t read path /local/domain/4/vm
>  
>  Either xen-tools (or whatever else in xen unstable) requires correlation
> with patched kernel.
>
Ah, I noticed that too, but I don''t see any obvious changes which might
have caused this.  Time to pull out "git bisect" I think.

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2009-Mar-11 07:26 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

disk= [ phy:/dev/sdb5] seems not to be released after PV DomU shutdown.
Hence , second attempt to load DomU gets paused waiting for particular
device. When i reboot xen host, DomU can be loaded again.

--- On Tue, 3/10/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
From: Jeremy Fitzhardinge <jeremy@goop.org>
Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t allow guests to use it
To: bderzhavets@yahoo.com
Cc: "Keir Fraser" <keir.fraser@eu.citrix.com>,
"Xen-devel" <xen-devel@lists.xensource.com>
Date: Tuesday, March 10, 2009, 8:29 PM

Boris Derzhavets wrote:> 
>   Seems like new kernel brought up back some old issues with ubuntu 8.10
> devices. It never happened before patching 2.6.29-rc7 kernel on C2D boxes
with the same ubuntu 8.10 server release and 2.6.29-rc5.>   In particular, CentOS 5.2 PV DomU''s xen-hotplug.log looks like:-
> 
> xenstore-read: couldn''t read path /local/domain/2/vm
> xenstore-read: couldn''t read path /local/domain/9/vm
> xenstore-read: couldn''t read path /local/domain/3/vm
> xenstore-read: couldn''t read path /local/domain/4/vm
>   Either xen-tools (or whatever else in xen unstable) requires correlation
> with patched kernel.
> 
Ah, I noticed that too, but I don''t see any obvious changes which might
have caused this.  Time to pull out "git bisect" I think.

   J



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2009-Mar-11 08:40 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

I''ve got the same issue with CentOS 5.2 PV DomU (same /dev/sdb5)
under Suse''s 2.6.27 xen-ified kernel & Xen Unstable ( the most
recent).
I believe Xen is broken in some way. It''s not about the kernel.

Boris.

--- On Wed, 3/11/09, Boris Derzhavets <bderzhavets@yahoo.com> wrote:
From: Boris Derzhavets <bderzhavets@yahoo.com>
Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t allow guests to use it
To: "Jeremy Fitzhardinge" <jeremy@goop.org>
Cc: "Xen-devel" <xen-devel@lists.xensource.com>, "Keir
Fraser" <keir.fraser@eu.citrix.com>
Date: Wednesday, March 11, 2009, 3:26 AM

disk= [ phy:/dev/sdb5] seems not to be released after PV DomU shutdown.
Hence , second attempt to load DomU gets paused waiting for particular
device. When i reboot xen host, DomU can be loaded again.

--- On Tue, 3/10/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
From: Jeremy Fitzhardinge <jeremy@goop.org>
Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t allow guests to use it
To: bderzhavets@yahoo.com
Cc: "Keir Fraser" <keir.fraser@eu.citrix.com>,
"Xen-devel" <xen-devel@lists.xensource.com>
Date: Tuesday, March 10, 2009, 8:29 PM

Boris Derzhavets wrote:> 
>   Seems like new kernel brought up back some old issues with ubuntu 8.10
> devices. It never happened before patching 2.6.29-rc7 kernel on C2D boxes
with the same ubuntu 8.10 server release and 2.6.29-rc5.>   In particular, CentOS 5.2 PV DomU''s xen-hotplug.log looks like:-
> 
> xenstore-read: couldn''t read path /local/domain/2/vm
> xenstore-read: couldn''t read path /local/domain/9/vm
> xenstore-read: couldn''t read path /local/domain/3/vm
> xenstore-read: couldn''t read path /local/domain/4/vm
>   Either xen-tools (or whatever else in xen unstable) requires correlation
> with patched kernel.
> 
Ah, I noticed that too, but I don''t see any obvious changes which might
have caused this.  Time to pull out "git bisect" I think.

   J



      _______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
 second boot up
under Novell''s 2.6.27 & Xen Unstable ( the most recent).
I believe it''s not about kernel, it''s about Xen Unstable.
It''s broken in some way.

Boris.


--- On Wed, 3/11/09, Boris Derzhavets <bderzhavets@yahoo.com> wrote:
From: Boris Derzhavets <bderzhavets@yahoo.com>
Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t allow guests to use it
To: "Jeremy Fitzhardinge" <jeremy@goop.org>
Cc: "Xen-devel" <xen-devel@lists.xensource.com>, "Keir
Fraser" <keir.fraser@eu.citrix.com>
Date: Wednesday, March 11, 2009, 3:26 AM

disk= [ phy:/dev/sdb5] seems not to be released after PV DomU shutdown.
Hence , second attempt to load DomU gets paused waiting for particular
device. When i reboot xen host, DomU can be loaded again.

--- On Tue, 3/10/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
From: Jeremy Fitzhardinge <jeremy@goop.org>
Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t allow guests to use it
To: bderzhavets@yahoo.com
Cc: "Keir Fraser" <keir.fraser@eu.citrix.com>,
"Xen-devel" <xen-devel@lists.xensource.com>
Date: Tuesday, March 10, 2009, 8:29 PM

Boris Derzhavets wrote:> 
>   Seems like new kernel brought up back some old issues with ubuntu 8.10
> devices. It never happened before patching 2.6.29-rc7 kernel on C2D boxes
with the same ubuntu 8.10 server release and 2.6.29-rc5.>   In particular, CentOS 5.2 PV DomU''s xen-hotplug.log looks like:-
> 
> xenstore-read: couldn''t read path /local/domain/2/vm
> xenstore-read: couldn''t read path /local/domain/9/vm
> xenstore-read: couldn''t read path /local/domain/3/vm
> xenstore-read: couldn''t read path /local/domain/4/vm
>   Either xen-tools (or whatever else in xen unstable) requires correlation
> with patched kernel.
> 
Ah, I noticed that too, but I don''t see any obvious changes which might
have caused this.  Time to pull out "git bisect" I think.

   J



      _______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2009-Mar-11 09:09 UTC

head link

[Xen-devel] Attempt to install SNV_106 under Suse''s 2.6.27 xen-ified kernel & Xen Unstable

root@ServerUbuntu810:/home/boris/vm# xm info
host                   : ServerUbuntu810
release                : 2.6.27.5
version                : #1 SMP Sun Mar 1 02:27:38 EST 2009
machine                : x86_64
nr_cpus                : 4
nr_nodes               : 1
cores_per_socket       : 4
threads_per_core       : 1
cpu_mhz                : 2833
hw_caps                :
bfebfbff:20100800:00000000:00000140:0408e3fd:00000000:00000001:00000000
virt_caps              : hvm
total_memory           : 8191
free_memory            : 2074
node_to_cpu            : node0:0-3
node_to_memory         : node0:2074
xen_major              : 3
xen_minor              : 4
xen_extra              : -unstable
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : Mon Mar 09 10:32:24 2009 +0000 19297:b249f3e979a5
cc_compiler            : gcc version 4.3.2 (Ubuntu 4.3.2-1ubuntu12) 
cc_compile_by          : root
cc_compile_domain      : 
cc_compile_date        : Tue Mar 10 16:03:22 EDT 2009
xend_config_format     : 4
root@ServerUbuntu810:/home/boris/vm# xm create -c snv106.py
Using config file "./snv106.py".
WARNING: Cannot extract revision on this hypervisor version: v-unstable,
unexpected version format
v3.4-unstable chgset ''Mon Mar 09 10:32:24 2009 +0000
19297:b249f3e979a5''
SunOS Release 5.11 Version snv_106 64-bit
Copyright 1983-2008 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
Configuring /dev


    1.     Solaris Interactive Text (Console session) (default)
    2.     Apply driver updates
    3.     Single user shell

Enter the number of your choice.
Selected: 1                                        

Solaris Interactive Text (Console session)

WARNING: g.e. still in use!
root@ServerUbuntu810:/home/boris/vm# Error: Device 6 (vbd) could not be
connected. Hotplug scripts not working.







      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2009-Mar-11 11:36 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Blktap helps out for multiple CentOS PV DomU restarts (with image on FS) under 
Suse''s 2.6.27 xen-ified kernel & Xen Unstable ( the most recent).
But it seems not implemented yet for 2.6.29-rc7

--- On Wed, 3/11/09, Boris Derzhavets <bderzhavets@yahoo.com> wrote:
From: Boris Derzhavets <bderzhavets@yahoo.com>
Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t allow guests to use it
To: "Jeremy Fitzhardinge" <jeremy@goop.org>
Cc: "Xen-devel" <xen-devel@lists.xensource.com>, "Keir
Fraser" <keir.fraser@eu.citrix.com>
Date: Wednesday, March 11, 2009, 4:40 AM

I''ve got the same issue with CentOS 5.2 PV DomU (same /dev/sdb5)
under Suse''s 2.6.27 xen-ified kernel & Xen Unstable ( the most
recent).
I believe Xen is broken in some way. It''s not about the kernel.

Boris.

--- On Wed, 3/11/09, Boris Derzhavets <bderzhavets@yahoo.com> wrote:
From: Boris Derzhavets <bderzhavets@yahoo.com>
Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t allow guests to use it
To: "Jeremy Fitzhardinge" <jeremy@goop.org>
Cc: "Xen-devel" <xen-devel@lists.xensource.com>, "Keir
Fraser" <keir.fraser@eu.citrix.com>
Date: Wednesday, March 11, 2009, 3:26 AM

disk= [ phy:/dev/sdb5] seems not to be released after PV DomU shutdown.
Hence , second attempt to load DomU gets paused waiting for particular
device. When i reboot xen host, DomU can be loaded again.

--- On Tue, 3/10/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
From: Jeremy Fitzhardinge <jeremy@goop.org>
Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t allow guests to use it
To: bderzhavets@yahoo.com
Cc: "Keir Fraser" <keir.fraser@eu.citrix.com>,
"Xen-devel" <xen-devel@lists.xensource.com>
Date: Tuesday, March 10, 2009, 8:29
 PM

Boris Derzhavets wrote:> 
>   Seems like new kernel brought up back some old issues with ubuntu 8.10
> devices. It never happened before patching 2.6.29-rc7 kernel on C2D boxes
with the same ubuntu 8.10 server release and 2.6.29-rc5.>   In particular, CentOS 5.2 PV DomU''s xen-hotplug.log looks like:-
> 
> xenstore-read: couldn''t read path /local/domain/2/vm
> xenstore-read: couldn''t read path /local/domain/9/vm
> xenstore-read: couldn''t read path /local/domain/3/vm
> xenstore-read: couldn''t read path /local/domain/4/vm
>   Either xen-tools (or whatever else in xen unstable) requires correlation
> with patched kernel.
> 
Ah, I noticed that too, but I don''t see any obvious changes which might
have caused this.  Time to pull out "git bisect" I think.

   J



      _______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
 second boot up
under Novell''s 2.6.27 & Xen Unstable ( the most recent).
I believe it''s not about kernel, it''s about Xen Unstable.
It''s broken in some way.

Boris.


--- On Wed, 3/11/09, Boris Derzhavets <bderzhavets@yahoo.com> wrote:
From: Boris Derzhavets <bderzhavets@yahoo.com>
Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t allow guests to use it
To: "Jeremy Fitzhardinge" <jeremy@goop.org>
Cc: "Xen-devel" <xen-devel@lists.xensource.com>, "Keir
Fraser" <keir.fraser@eu.citrix.com>
Date: Wednesday, March 11, 2009, 3:26 AM

disk= [ phy:/dev/sdb5] seems not to be released after PV DomU shutdown.
Hence , second attempt to load DomU gets paused waiting for particular
device. When i reboot xen host, DomU can be loaded again.

--- On Tue, 3/10/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
From: Jeremy Fitzhardinge <jeremy@goop.org>
Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t allow guests to use it
To: bderzhavets@yahoo.com
Cc: "Keir Fraser" <keir.fraser@eu.citrix.com>,
"Xen-devel"
 <xen-devel@lists.xensource.com>
Date: Tuesday, March 10, 2009, 8:29 PM

Boris Derzhavets wrote:> 
>   Seems like new kernel brought up back some old issues with ubuntu 8.10
> devices. It never happened before patching 2.6.29-rc7 kernel on C2D boxes
with the same ubuntu 8.10 server release and 2.6.29-rc5.>   In particular, CentOS 5.2 PV DomU''s xen-hotplug.log looks like:-
> 
> xenstore-read: couldn''t read path /local/domain/2/vm
> xenstore-read: couldn''t read path /local/domain/9/vm
> xenstore-read: couldn''t read path /local/domain/3/vm
> xenstore-read: couldn''t read path /local/domain/4/vm
>   Either xen-tools (or whatever else in xen unstable) requires correlation
> with patched kernel.
> 
Ah, I noticed that too, but I don''t see any obvious changes which might
have caused this.  Time to pull out "git bisect" I think.

  
 J



      _______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      _______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jeremy Fitzhardinge

2009-Mar-11 16:09 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Boris Derzhavets wrote:> Blktap helps out for multiple CentOS PV DomU restarts (with image on 
> FS) under  Suse''s 2.6.27 xen-ified kernel & Xen Unstable ( the
most
> recent).
> But it seems not implemented yet for 2.6.29-rc7
>
So your conclusion is that there''s a regression in the tools stack when
using blkback rather than blktab?

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2009-Mar-11 16:27 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

It''s not so important for me. The important thing is:-

CentOS 5.2 PV DomU may be loaded only the very first time under Suse''s
2.6.27 xen-ified kernel & Xen Unstable ( the most recent). Attempt to
shutdown and start it again gives VBD cannot be connected . Hotplug scripts not
working. That''s a core issue, same behavior as under 2.6.29-rc7 (with
XSAVE patch ). Kernel doesn''t seem to be a root cause.
I believe Xen unstable is broken in some way. 

Boris.



--- On Wed, 3/11/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
From: Jeremy Fitzhardinge <jeremy@goop.org>
Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t allow guests to use it
To: bderzhavets@yahoo.com
Cc: "Xen-devel" <xen-devel@lists.xensource.com>, "Keir
Fraser" <keir.fraser@eu.citrix.com>
Date: Wednesday, March 11, 2009, 12:09 PM

Boris Derzhavets wrote:> Blktap helps out for multiple CentOS PV DomU restarts (with image on FS)under  Suse''s 2.6.27 xen-ified kernel & Xen Unstable ( the most
recent).> But it seems not implemented yet for 2.6.29-rc7
> 
So your conclusion is that there''s a regression in the tools stack when
using blkback rather than blktab?

   J



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jeremy Fitzhardinge

2009-Mar-11 16:50 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Boris Derzhavets wrote:> It''s not so important for me. The important thing is:-
>
> CentOS 5.2 PV DomU may be loaded only the very first time under
Suse''s
> 2.6.27 xen-ified kernel & Xen Unstable ( the most recent). Attempt to 
> shutdown and start it again gives VBD cannot be connected . Hotplug 
> scripts not working. That''s a core issue, same behavior as under 
> 2.6.29-rc7 (with XSAVE patch ). Kernel doesn''t seem to be a root
cause.
> I believe Xen unstable is broken in some way.
>
Could you try bisecting xen-unstable to see which change started failing 
for you?  There were some recent changes to the hotplug scripts which 
could have introduced a problem.

However, I''m pretty sure I started seeing this problem without updating
my usermode tools, so there could also be a kernel problem.  But its not 
obvious what it might be.

Thanks,
    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2009-Mar-11 16:57 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

>Could you try bisecting xen-unstable to see which change started failing for
>you?  There were some recent changes to the hotplug scripts which could have
> introduced a problem.No. It''s above me.


--- On Wed, 3/11/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
From: Jeremy Fitzhardinge <jeremy@goop.org>
Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t allow guests to use it
To: bderzhavets@yahoo.com
Cc: "Xen-devel" <xen-devel@lists.xensource.com>, "Keir
Fraser" <keir.fraser@eu.citrix.com>
Date: Wednesday, March 11, 2009, 12:50 PM

Boris Derzhavets wrote:> It''s not so important for me. The important thing is:-
> 
> CentOS 5.2 PV DomU may be loaded only the very first time under
Suse''s2.6.27 xen-ified kernel & Xen Unstable ( the most recent). Attempt to
shutdown and start it again gives VBD cannot be connected . Hotplug scripts not
working. That''s a core issue, same behavior as under 2.6.29-rc7 (with
XSAVE
patch ). Kernel doesn''t seem to be a root
cause.> I believe Xen unstable is broken in some way.
> 
Could you try bisecting xen-unstable to see which change started failing for
you?  There were some recent changes to the hotplug scripts which could have
introduced a problem.

However, I''m pretty sure I started seeing this problem without updating
my
usermode tools, so there could also be a kernel problem.  But its not obvious
what it might be.

Thanks,
   J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jeremy Fitzhardinge

2009-Mar-11 17:04 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Boris Derzhavets wrote:> >Could you try bisecting xen-unstable to see which change started
failing for
> >you?  There were some recent changes to the hotplug scripts which could
have
> > introduced a problem.
> No. It''s above me.
>         
>
If you''re fetching xen-unstable with "hg" and building it,
then you have
everything you need to do bisection:

   1. hg bisect -r
   2. hg bisect -g <known good version>
   3. hg bisect -b <known bad version>
   4. build as usual and test
         1. if works: hg bisect -g
         2. if not: hg bisect -b
   5. repeat 4 until first bad version has been isolated

Thanks,
    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2009-Mar-11 17:21 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

I didn''t get a chance to notice the last good version , because pygrub
stopped working (return no data ) right after XSAVE patch (for myself, of
course) .
*****************
Then i followed
******************
Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use
it
Monday, March 9, 2009 7:26 AM
From:
"Keir Fraser" <keir.fraser@eu.citrix.com>
Add sender to Contacts
To:
"bderzhavets@yahoo.com" <bderzhavets@yahoo.com>
Cc:
"Xen-devel" <xen-devel@lists.xensource.com>
If you¹re using a recent xen-unstable then you might try again with the
current tip. A couple of changes went in recently (including one fix today).

-- Keir
********************************
and turned into current issues
********************************

--- On Wed, 3/11/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
From: Jeremy Fitzhardinge <jeremy@goop.org>
Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t allow guests to use it
To: bderzhavets@yahoo.com
Cc: "Xen-devel" <xen-devel@lists.xensource.com>, "Keir
Fraser" <keir.fraser@eu.citrix.com>
Date: Wednesday, March 11, 2009, 1:04 PM

Boris Derzhavets wrote:> >Could you try bisecting xen-unstable to see which change started
failing for> >you?  There were some recent changes to the hotplug scripts which
could have> > introduced a problem.
> No. It''s above me.
>         
If you''re fetching xen-unstable with "hg" and building it,
then
you have everything you need to do bisection:

  1. hg bisect -r
  2. hg bisect -g <known good version>
  3. hg bisect -b <known bad version>
  4. build as usual and test
        1. if works: hg bisect -g
        2. if not: hg bisect -b
  5. repeat 4 until first bad version has been isolated

Thanks,
   J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2009-Mar-11 17:43 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Sorry, i may try. Just one question

Is version a changeset number ?


--- On Wed, 3/11/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
From: Jeremy Fitzhardinge <jeremy@goop.org>
Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t allow guests to use it
To: bderzhavets@yahoo.com
Cc: "Xen-devel" <xen-devel@lists.xensource.com>, "Keir
Fraser" <keir.fraser@eu.citrix.com>
Date: Wednesday, March 11, 2009, 1:04 PM

Boris Derzhavets wrote:> >Could you try bisecting xen-unstable to see which change started
failing for> >you?  There were some recent changes to the hotplug scripts which
could have> > introduced a problem.
> No. It''s above me.
>         
If you''re fetching xen-unstable with "hg" and building it,
then
you have everything you need to do bisection:

  1. hg bisect -r
  2. hg bisect -g <known good version>
  3. hg bisect -b <known bad version>
  4. build as usual and test
        1. if works: hg bisect -g
        2. if not: hg bisect -b
  5. repeat 4 until first bad version has been isolated

Thanks,
   J



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jeremy Fitzhardinge

2009-Mar-11 17:47 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Boris Derzhavets wrote:> Sorry, i may try. Just one question
>
> Is version a changeset number ?
>
Yes.

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Andrew Lyon

2009-Mar-11 19:13 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

On Wed, Mar 11, 2009 at 4:27 PM, Boris Derzhavets <bderzhavets@yahoo.com>
wrote:> It''s not so important for me. The important thing is:-
>
> CentOS 5.2 PV DomU may be loaded only the very first time under
Suse''s
> 2.6.27 xen-ified kernel & Xen Unstable ( the most recent). Attempt to
> shutdown and start it again gives VBD cannot be connected . Hotplug scripts
> not working. That''s a core issue, same behavior as under
2.6.29-rc7 (with
> XSAVE patch ). Kernel doesn''t seem to be a root cause.
> I believe Xen unstable is broken in some way.
>
> Boris.
This sounds very much like the problem I described in thread subject
"domain id number on xen unstable", I maintain my own 2.6.27 dom0
kernel using opensuse Xen patches which I rebase to apply to vanilla
without the many other patches opensuse usually applies to the kernel
tree, so from a Xen point of view I am using a very similar kernel to
yours, I found that I could start a hvm but after shutting it down
attempting to restart it would fail or hang, sometimes I got hotplug
error that vbd could not be connected.

The script that has problems on my system is xen-hotplug-cleanup , the
first time it is run it puts a lock in /var/run/xen-hotplug and never
removes it, so the next time the script runs it blocks waiting for the
lock and eventually times out.

The offending line in the script is:

vm=$(xenstore-read "/local/domain/${path_array[2]}/vm")

putting a echo immediately after that line shows that nothing after it
is executed, which is why the lock is not released.

replacing xen-hotplug-cleanup and xen-hotplug-common with the ones
from 3.3.1 seems to help, but after starting and stopping a few vm''s
the entire system reboots, so I think some objects are not being
cleaned up, not surprising really, can hardly expect scripts to work
with the wrong version of Xen.

This problem has got me stuck into a nasty corner, only Xen unstable
can fit our virtualization requirements (need viridian for stable
windows smp), but the Xensource kernel is too old for our hardware.

I am going to put some serious effort into debugging this in the next few days.

Andy
>
>
>
> --- On Wed, 3/11/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
>
> From: Jeremy Fitzhardinge <jeremy@goop.org>
> Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t
> allow guests to use it
> To: bderzhavets@yahoo.com
> Cc: "Xen-devel" <xen-devel@lists.xensource.com>, "Keir
Fraser"
> <keir.fraser@eu.citrix.com>
> Date: Wednesday, March 11, 2009, 12:09 PM
>
> Boris Derzhavets wrote:
>> Blktap helps out for multiple CentOS PV DomU restarts (with image on
FS)
> under  Suse''s 2.6.27 xen-ified kernel & Xen Unstable ( the
most recent).
>> But it seems not implemented yet for 2.6.29-rc7
>>
>
> So your conclusion is that there''s a regression in the tools stack
when
> using blkback rather than blktab?
>
>    J
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2009-Mar-11 19:49 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

What i am doing wrong ?

hg bisect -r
hg bisect -g 19282  
hg bisect -b 19297
make install-xen OK   
make install-tools

make[5]: *** [block-qcow2.o] Error 1
make[5]: Leaving directory
`/usr/src/xen-unstable.hg/tools/blktap/drivers''
make[4]: *** [subdir-install-drivers] Error 2
make[4]: Leaving directory `/usr/src/xen-unstable.hg/tools/blktap''
make[3]: *** [subdirs-install] Error 2
make[3]: Leaving directory `/usr/src/xen-unstable.hg/tools/blktap''
make[2]: *** [subdir-install-blktap] Error 2
make[2]: Leaving directory `/usr/src/xen-unstable.hg/tools''
make[1]: *** [subdirs-install] Error 2
make[1]: Leaving directory `/usr/src/xen-unstable.hg/tools''
make: *** [install-tools] Error 2

hg bisect -r
hg bisect -g 19232  
hg bisect -b 19297

make install-xen 
make[4]: Leaving directory `/usr/src/xen-unstable.hg/xen/arch/x86''
make[3]: *** [/usr/src/xen-unstable.hg/xen/arch/x86/built_in.o] Error 2
make[3]: Leaving directory `/usr/src/xen-unstable.hg/xen/arch/x86''
make[2]: *** [/usr/src/xen-unstable.hg/xen/xen] Error 2
make[2]: Leaving directory `/usr/src/xen-unstable.hg/xen''
make[1]: *** [install] Error 2
make[1]: Leaving directory `/usr/src/xen-unstable.hg/xen''
make: *** [install-xen] Error 2

make install-tools OK


--- On Wed, 3/11/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
From: Jeremy Fitzhardinge <jeremy@goop.org>
Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t allow guests to use it
To: bderzhavets@yahoo.com
Cc: "Xen-devel" <xen-devel@lists.xensource.com>, "Keir
Fraser" <keir.fraser@eu.citrix.com>
Date: Wednesday, March 11, 2009, 1:47 PM

Boris Derzhavets wrote:> Sorry, i may try. Just one question
>
> Is version a changeset number ?
>
Yes.

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jeremy Fitzhardinge

2009-Mar-12 00:41 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Boris Derzhavets wrote:> What i am doing wrong ?
>
Not sure.  You should only need to install tools though; Xen and the 
other trees should be irrelevant.
>
> hg bisect -r
> hg bisect -g 19282 
> hg bisect -b 19297
> make install-xen OK  
> make install-tools
>
> make[5]: *** [block-qcow2.o] Error 1
>What happens before this line?
> make[5]: Leaving directory
`/usr/src/xen-unstable.hg/tools/blktap/drivers''
> make[4]: *** [subdir-install-drivers] Error 2
> make[4]: Leaving directory `/usr/src/xen-unstable.hg/tools/blktap''
> make[3]: *** [subdirs-install] Error 2
> make[3]: Leaving directory `/usr/src/xen-unstable.hg/tools/blktap''
> make[2]: *** [subdir-install-blktap] Error 2
> make[2]: Leaving directory `/usr/src/xen-unstable.hg/tools''
> make[1]: *** [subdirs-install] Error 2
> make[1]: Leaving directory `/usr/src/xen-unstable.hg/tools''
> make: *** [install-tools] Error 2
>
> hg bisect -r
> hg bisect -g 19232 
> hg bisect -b 19297
>
> make install-xen
> make[4]: Leaving directory `/usr/src/xen-unstable.hg/xen/arch/x86''
> make[3]: *** [/usr/src/xen-unstable.hg/xen/arch/x86/built_in.o] Error 2
> make[3]: Leaving directory `/usr/src/xen-unstable.hg/xen/arch/x86''
> make[2]: *** [/usr/src/xen-unstable.hg/xen/xen] Error 2
> make[2]: Leaving directory `/usr/src/xen-unstable.hg/xen''
> make[1]: *** [install] Error 2
> make[1]: Leaving directory `/usr/src/xen-unstable.hg/xen''
> make: *** [install-xen] Error 2
>
> make install-tools OK
>
    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jeremy Fitzhardinge

2009-Mar-12 01:21 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Andrew Lyon wrote:> On Wed, Mar 11, 2009 at 4:27 PM, Boris Derzhavets
<bderzhavets@yahoo.com> wrote:
>   
>> It''s not so important for me. The important thing is:-
>>
>> CentOS 5.2 PV DomU may be loaded only the very first time under
Suse''s
>> 2.6.27 xen-ified kernel & Xen Unstable ( the most recent). Attempt
to
>> shutdown and start it again gives VBD cannot be connected . Hotplug
scripts
>> not working. That''s a core issue, same behavior as under
2.6.29-rc7 (with
>> XSAVE patch ). Kernel doesn''t seem to be a root cause.
>> I believe Xen unstable is broken in some way.
>>
>> Boris.
>>     
>
> This sounds very much like the problem I described in thread subject
> "domain id number on xen unstable", I maintain my own 2.6.27 dom0
> kernel using opensuse Xen patches which I rebase to apply to vanilla
> without the many other patches opensuse usually applies to the kernel
> tree, so from a Xen point of view I am using a very similar kernel to
> yours, I found that I could start a hvm but after shutting it down
> attempting to restart it would fail or hang, sometimes I got hotplug
> error that vbd could not be connected.
>
> The script that has problems on my system is xen-hotplug-cleanup , the
> first time it is run it puts a lock in /var/run/xen-hotplug and never
> removes it, so the next time the script runs it blocks waiting for the
> lock and eventually times out.
>
> The offending line in the script is:
>
> vm=$(xenstore-read "/local/domain/${path_array[2]}/vm")
>
> putting a echo immediately after that line shows that nothing after it
> is executed, which is why the lock is not released.
>
> replacing xen-hotplug-cleanup and xen-hotplug-common with the ones
> from 3.3.1 seems to help, but after starting and stopping a few
vm''s
> the entire system reboots, so I think some objects are not being
> cleaned up, not surprising really, can hardly expect scripts to work
> with the wrong version of Xen.
>
> This problem has got me stuck into a nasty corner, only Xen unstable
> can fit our virtualization requirements (need viridian for stable
> windows smp), but the Xensource kernel is too old for our hardware.
>
> I am going to put some serious effort into debugging this in the next few
days.
>   
Change 19242:226031d62fc5 is the last one which works for me.  I
haven''t
managed to bisect which specific change the problem starts occuring in, 
but 19250:b3774712e654 seems like a good starting point.

    J> Andy
>
>   
>>
>> --- On Wed, 3/11/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
>>
>> From: Jeremy Fitzhardinge <jeremy@goop.org>
>> Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t
>> allow guests to use it
>> To: bderzhavets@yahoo.com
>> Cc: "Xen-devel" <xen-devel@lists.xensource.com>,
"Keir Fraser"
>> <keir.fraser@eu.citrix.com>
>> Date: Wednesday, March 11, 2009, 12:09 PM
>>
>> Boris Derzhavets wrote:
>>     
>>> Blktap helps out for multiple CentOS PV DomU restarts (with image
on FS)
>>>       
>> under  Suse''s 2.6.27 xen-ified kernel & Xen Unstable ( the
most recent).
>>     
>>> But it seems not implemented yet for 2.6.29-rc7
>>>
>>>       
>> So your conclusion is that there''s a regression in the tools
stack when
>> using blkback rather than blktab?
>>
>>    J
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
>>
>>     
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>   

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jeremy Fitzhardinge

2009-Mar-12 01:54 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Jeremy Fitzhardinge wrote:> Change 19242:226031d62fc5 is the last one which works for me.  I 
> haven''t managed to bisect which specific change the problem starts
> occuring in, but 19250:b3774712e654 seems like a good starting point. 
Hm, perhaps not; it doesn''t work for me now after a reboot.  I realize 
now that I wasn''t restarting xenstored between tests.

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2009-Mar-12 08:48 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Trying to get 19242 on fresh clone :-

root@ServerXen331:/usr/src/xen-unstable.hg# hg bisect -r
root@ServerXen331:/usr/src/xen-unstable.hg# hg bisect -g 19242 
root@ServerXen331:/usr/src/xen-unstable.hg# hg bisect -b 19297
Testing changeset 19269:3673926b2375 (55 changesets remaining, ~5 tests)
63 files updated, 0 files merged, 0 files removed, 0 files unresolved
root@ServerXen331:/usr/src/xen-unstable.hg# make tools

gcc  -O1 -fno-omit-frame-pointer -fno-optimize-sibling-calls -m64 -g
-fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes -Wno-unused-value
-Wdeclaration-after-statement  -D__XEN_TOOLS__ -MMD -MF .block-qcow2.o.d 
-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE  -Werror -Wno-unused -I../lib
-I../../../tools/libxc -I../../../tools/include -I../../../tools/xenstore
-I../../../tools/include -I ../../libaio/src -D_GNU_SOURCE -c -o block-qcow2.o
block-qcow2.c
cc1: warnings being treated as errors
block-qcow2.c: In function ‘qcow2_create’:
block-qcow2.c:2045: error: ignoring return value of ‘write’, declared with
attribute warn_unused_result
block-qcow2.c:2047: error: ignoring return value of ‘write’, declared with
attribute warn_unused_result
block-qcow2.c:2052: error: ignoring return value of ‘write’, declared with
attribute warn_unused_result
block-qcow2.c:2055: error: ignoring return value of ‘write’, declared with
attribute warn_unused_result
block-qcow2.c:2058: error: ignoring return value of ‘write’, declared with
attribute warn_unused_result
make[5]: *** [block-qcow2.o] Error 1
make[5]: Leaving directory
`/usr/src/xen-unstable.hg/tools/blktap/drivers''
make[4]: *** [subdir-install-drivers] Error 2
make[4]: Leaving directory `/usr/src/xen-unstable.hg/tools/blktap''
make[3]: *** [subdirs-install] Error 2
make[3]: Leaving directory `/usr/src/xen-unstable.hg/tools/blktap''
make[2]: *** [subdir-install-blktap] Error 2
make[2]: Leaving directory `/usr/src/xen-unstable.hg/tools''
make[1]: *** [subdirs-install] Error 2
make[1]: Leaving directory `/usr/src/xen-unstable.hg/tools''
make: *** [install-tools] Error 2

I can build tools only for 19232, but pygrub return no data for this release

--- On Wed, 3/11/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
From: Jeremy Fitzhardinge <jeremy@goop.org>
Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t allow guests to use it
To: "Andrew Lyon" <andrew.lyon@gmail.com>
Cc: bderzhavets@yahoo.com, "Xen-devel"
<xen-devel@lists.xensource.com>, "Keir Fraser"
<keir.fraser@eu.citrix.com>
Date: Wednesday, March 11, 2009, 9:54 PM

Jeremy Fitzhardinge wrote:> Change 19242:226031d62fc5 is the last one which works for me.  Ihaven''t managed to bisect which specific change the problem starts
occuring
in, but 19250:b3774712e654 seems like a good starting point. 

Hm, perhaps not; it doesn''t work for me now after a reboot.  I realize
now
that I wasn''t restarting xenstored between tests.

   J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Andrew Lyon

2009-Mar-12 08:55 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

On Wed, Mar 11, 2009 at 7:13 PM, Andrew Lyon <andrew.lyon@gmail.com>
wrote:> On Wed, Mar 11, 2009 at 4:27 PM, Boris Derzhavets
<bderzhavets@yahoo.com> wrote:
>> It''s not so important for me. The important thing is:-
>>
>> CentOS 5.2 PV DomU may be loaded only the very first time under
Suse''s
>> 2.6.27 xen-ified kernel & Xen Unstable ( the most recent). Attempt
to
>> shutdown and start it again gives VBD cannot be connected . Hotplug
scripts
>> not working. That''s a core issue, same behavior as under
2.6.29-rc7 (with
>> XSAVE patch ). Kernel doesn''t seem to be a root cause.
>> I believe Xen unstable is broken in some way.
>>
>> Boris.
>
> This sounds very much like the problem I described in thread subject
> "domain id number on xen unstable", I maintain my own 2.6.27 dom0
> kernel using opensuse Xen patches which I rebase to apply to vanilla
> without the many other patches opensuse usually applies to the kernel
> tree, so from a Xen point of view I am using a very similar kernel to
> yours, I found that I could start a hvm but after shutting it down
> attempting to restart it would fail or hang, sometimes I got hotplug
> error that vbd could not be connected.
>
> The script that has problems on my system is xen-hotplug-cleanup , the
> first time it is run it puts a lock in /var/run/xen-hotplug and never
> removes it, so the next time the script runs it blocks waiting for the
> lock and eventually times out.
>
> The offending line in the script is:
>
> vm=$(xenstore-read "/local/domain/${path_array[2]}/vm")
>
> putting a echo immediately after that line shows that nothing after it
> is executed, which is why the lock is not released.
>
> replacing xen-hotplug-cleanup and xen-hotplug-common with the ones
> from 3.3.1 seems to help, but after starting and stopping a few
vm''s
> the entire system reboots, so I think some objects are not being
> cleaned up, not surprising really, can hardly expect scripts to work
> with the wrong version of Xen.
>
> This problem has got me stuck into a nasty corner, only Xen unstable
> can fit our virtualization requirements (need viridian for stable
> windows smp), but the Xensource kernel is too old for our hardware.
>
> I am going to put some serious effort into debugging this in the next few
days.
>
> Andy
I have just managed to get the Xensource 2.6.18.8 kernel working on
one of my test systems and this problem still happens, I updated Xen
unstable to latest hg as well, so this is definitely not a kernel
version problem, after starting and stopping one hvm I cannot start
any more, and xenstore-ls shows that objects are not being cleaned up
properly:

tool = ""
 xenstored = ""
local = ""
 domain = ""
  0 = ""
   vm = "/vm/00000000-0000-0000-0000-000000000000"
   device = ""
   control = ""
    platform-feature-multiprocessor-suspend = "1"
   error = ""
   memory = ""
    target = "3453952"
   cpu = ""
    1 = ""
     availability = "online"
    0 = ""
     availability = "online"
   name = "Domain-0"
   console = ""
    limit = "1048576"
    type = "xenconsoled"
   domid = "0"
   backend = ""
    vfb = ""
     1 = ""
      0 = ""
       hotplug-error = "/etc/xen/scripts/xen-hotplug-cleanup failed;
error detected."
       hotplug-status = "error"
    vbd = ""
     1 = ""
      768 = ""
       domain = "xptest"
       frontend = "/local/domain/1/device/vbd/768"
       uuid = "7f03ff88-17ca-5442-2684-fee0e2d43e98"
       bootable = "1"
       dev = "hda"
       state = "6"
       params = "/root/xp"
       mode = "w"
       online = "0"
       frontend-id = "1"
       type = "file"
       node = "/dev/loop0"
       physical-device = "7:0"
       hotplug-status = "connected"
    vif = ""
     1 = ""
      0 = ""
       domain = "xptest"
       handle = "0"
       uuid = "258a56f7-721d-647b-98c3-b030eb9572de"
       script = "/etc/xen/scripts/vif-bridge"
       state = "6"
       frontend = "/local/domain/1/device/vif/0"
       mac = "00:16:3e:4c:c2:43"
       online = "0"
       frontend-id = "1"
       feature-sg = "1"
       feature-gso-tcpv4 = "1"
       feature-rx-copy = "1"
       feature-rx-flip = "0"
       hotplug-status = "connected"
    console = ""
     1 = ""
   device-model = ""
vm = ""
 00000000-0000-0000-0000-000000000000 = ""
  on_xend_stop = "ignore"
  shadow_memory = "0"
  uuid = "00000000-0000-0000-0000-000000000000"
  on_reboot = "restart"
  image = "(linux (kernel ))"
   ostype = "linux"
   kernel = ""
   cmdline = ""
   ramdisk = ""
  on_poweroff = "destroy"
  bootloader_args = ""
  on_xend_start = "ignore"
  on_crash = "restart"
  xend = ""
   restart_count = "0"
  vcpus = "2"
  vcpu_avail = "3"
  bootloader = ""
  name = "Domain-0"
  memory = "3373"

Andy
>
>>
>>
>>
>> --- On Wed, 3/11/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
>>
>> From: Jeremy Fitzhardinge <jeremy@goop.org>
>> Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t
>> allow guests to use it
>> To: bderzhavets@yahoo.com
>> Cc: "Xen-devel" <xen-devel@lists.xensource.com>,
"Keir Fraser"
>> <keir.fraser@eu.citrix.com>
>> Date: Wednesday, March 11, 2009, 12:09 PM
>>
>> Boris Derzhavets wrote:
>>> Blktap helps out for multiple CentOS PV DomU restarts (with image
on FS)
>> under  Suse''s 2.6.27 xen-ified kernel & Xen Unstable ( the
most recent).
>>> But it seems not implemented yet for 2.6.29-rc7
>>>
>>
>> So your conclusion is that there''s a regression in the tools
stack when
>> using blkback rather than blktab?
>>
>>    J
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
>>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2009-Mar-12 09:34 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

On 12/03/2009 08:55, "Andrew Lyon" <andrew.lyon@gmail.com>
wrote:
> I have just managed to get the Xensource 2.6.18.8 kernel working on
> one of my test systems and this problem still happens, I updated Xen
> unstable to latest hg as well, so this is definitely not a kernel
> version problem, after starting and stopping one hvm I cannot start
> any more, and xenstore-ls shows that objects are not being cleaned up
> properly:
Changeset 19314 reverts a problematic changeset. You might upgrade and try
with that.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2009-Mar-12 12:36 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Thank you. PV DomUs are restartable now.
Boris

--- On Thu, 3/12/09, Keir Fraser <keir.fraser@eu.citrix.com> wrote:
From: Keir Fraser <keir.fraser@eu.citrix.com>
Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
don''t allow guests to use it
To: "Andrew Lyon" <andrew.lyon@gmail.com>,
"bderzhavets@yahoo.com" <bderzhavets@yahoo.com>
Cc: "Jeremy Fitzhardinge" <jeremy@goop.org>,
"Xen-devel" <xen-devel@lists.xensource.com>
Date: Thursday, March 12, 2009, 5:34 AM

On 12/03/2009 08:55, "Andrew Lyon" <andrew.lyon@gmail.com>
wrote:
> I have just managed to get the Xensource 2.6.18.8 kernel working on
> one of my test systems and this problem still happens, I updated Xen
> unstable to latest hg as well, so this is definitely not a kernel
> version problem, after starting and stopping one hvm I cannot start
> any more, and xenstore-ls shows that objects are not being cleaned up
> properly:
Changeset 19314 reverts a problematic changeset. You might upgrade and try
with that.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2009-Mar-13 14:39 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

   Yesterday   2.6.29-rc7 ( built on 03/08/09) kernel worked fine under Xen
Unstable (rel 19317).
   Today a new fresh kernel 2.6.29-rc7 has been built. The most recent rc7
kernel under Xen Unstable (rel 19317) generates an error (vif) cannot be 
connected, hotplug scripts not working when i attempted to start f10 PV DomU
or SNV_106 PV DomU.
Status of Xen seems to be OK due to same DomUs functionality under
Suse''s 2.6.27-5 &  Xen Unstable (rel 19317), it is absolutely
normal.

Boris 






      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jeremy Fitzhardinge

2009-Mar-13 15:20 UTC

head link

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Boris Derzhavets wrote:>    Yesterday   2.6.29-rc7 ( built on 03/08/09) kernel worked fine 
> under Xen Unstable (rel 19317).
>    Today a new fresh kernel 2.6.29-rc7 has been built. The most recent rc7
> kernel under Xen Unstable (rel 19317) generates an error (vif) cannot 
> be  connected, hotplug scripts not working when i attempted to start 
> f10 PV DomU
> or SNV_106 PV DomU.
> Status of Xen seems to be OK due to same DomUs functionality under
> Suse''s 2.6.27-5 &  Xen Unstable (rel 19317), it is absolutely
normal.
>
Yes, I think there''s a second problem in dom0, which has similar 
symptoms as the other hotplug scripts bug.  The scripts bug confused 
efforts in tracking down the dom0 problem.  I''ll have another go at the
kernel side today.

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Dan Magenheimer

2009-Mar-13 15:46 UTC

head link

[Xen-devel] Hot plug scripts not working c/s 19355 (was: Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it)

A heads-up that I just got a "(tap)...Hotplug scripts not working"
completely unrelated to pvops.

I updated hypervisor and tools (but notably NOT dom0 kernel) to
xen-unstable c/s 19355.  Launching a domain now fails with the
above error.  Rolling tools (only) back to 19187 makes the
problem go away.

Is there maybe a new dependency between tools and dom0 kernel?
> -----Original Message-----
> From: Jeremy Fitzhardinge [mailto:jeremy@goop.org]
> Sent: Friday, March 13, 2009 9:20 AM
> To: bderzhavets@yahoo.com
> Cc: Xen-devel; Keir Fraser
> Subject: Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we
> don''t allow guests to use it
> 
> 
> Boris Derzhavets wrote:
> >    Yesterday   2.6.29-rc7 ( built on 03/08/09) kernel worked fine 
> > under Xen Unstable (rel 19317).
> >    Today a new fresh kernel 2.6.29-rc7 has been built. The 
> most recent rc7
> > kernel under Xen Unstable (rel 19317) generates an error 
> (vif) cannot 
> > be  connected, hotplug scripts not working when i attempted 
> to start 
> > f10 PV DomU
> > or SNV_106 PV DomU.
> > Status of Xen seems to be OK due to same DomUs functionality under
> > Suse''s 2.6.27-5 &  Xen Unstable (rel 19317), it is 
> absolutely normal.
> >
> 
> Yes, I think there''s a second problem in dom0, which has similar 
> symptoms as the other hotplug scripts bug.  The scripts bug confused 
> efforts in tracking down the dom0 problem.  I''ll have another 
> go at the 
> kernel side today.
> 
>     J
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jeremy Fitzhardinge

2009-Mar-13 16:05 UTC

head link

[Xen-devel] Re: Hot plug scripts not working c/s 19355

Dan Magenheimer wrote:> A heads-up that I just got a "(tap)...Hotplug scripts not
working"
> completely unrelated to pvops.
>
> I updated hypervisor and tools (but notably NOT dom0 kernel) to
> xen-unstable c/s 19355.  Launching a domain now fails with the
> above error.  Rolling tools (only) back to 19187 makes the
> problem go away.
>   
Any chance you could try to pinpoint the failing version?
> Is there maybe a new dependency between tools and dom0 kernel?
>   
There''s certainly nothing explicit.  Bisecting the kernel seemed to 
point to some cleanups I did around /dev/privcmd, but that''s not really
used for hotplug/xenbus stuff (I mean, it is at a low level, but surely 
xenbus would break entirely if it were broken).  And those changes work 
with older tools.  (And the bisection was probably lead astray by the 
other hotplug bug.)

Maybe some timing/ordering thing?  Unfortunately I know very little 
about how things are supposed to work at that level.  IanC said that he 
thought that xenbus watches were failing to fire, which makes me wonder 
if Gerd''s xenstore fixes might be worth looking at more closely.

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2009-Mar-13 16:59 UTC

head link

[Xen-devel] Re: Hot plug scripts not working c/s 19355 (was: Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it)

On 13/03/2009 15:46, "Dan Magenheimer"
<dan.magenheimer@oracle.com> wrote:
> A heads-up that I just got a "(tap)...Hotplug scripts not
working"
> completely unrelated to pvops.
> 
> I updated hypervisor and tools (but notably NOT dom0 kernel) to
> xen-unstable c/s 19355.  Launching a domain now fails with the
> above error.  Rolling tools (only) back to 19187 makes the
> problem go away.
> 
> Is there maybe a new dependency between tools and dom0 kernel?
You could try reverting changeset 19342, as most likely culprit.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jeremy Fitzhardinge

2009-Mar-13 17:09 UTC

head link

[Xen-devel] Re: Hot plug scripts not working c/s 19355 (was: Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it)

Keir Fraser wrote:> On 13/03/2009 15:46, "Dan Magenheimer"
<dan.magenheimer@oracle.com> wrote:
>
>   
>> A heads-up that I just got a "(tap)...Hotplug scripts not
working"
>> completely unrelated to pvops.
>>
>> I updated hypervisor and tools (but notably NOT dom0 kernel) to
>> xen-unstable c/s 19355.  Launching a domain now fails with the
>> above error.  Rolling tools (only) back to 19187 makes the
>> problem go away.
>>
>> Is there maybe a new dependency between tools and dom0 kernel?
>>     
>
> You could try reverting changeset 19342, as most likely culprit.
>   
Is that the fixed xen-hotplug-cleanup change?  The symptoms aren''t
quite
the same; with the broken cleanup, there were lockfiles left lying 
about, so all the "block add" scripts were blocked waiting for the
lock
to be released.  In this case, everything is just sitting there; the 
hotplug never seems to get started.

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2009-Mar-13 18:15 UTC

head link

Re: [Xen-devel] Re: Hot plug scripts not working c/s 19355 (was: Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it)

The most recent 2.6.29-rc7 under Xen (release19317) shows up
hotplug issue at the very first start up.
When i revers  back to rc7 built on 03/08/09 under same Xen,  hotplug issue
gets gone.
I gonna test Xen(19355) with Suse 2.6.27-5 kernel

Boris.

--- On Fri, 3/13/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
From: Jeremy Fitzhardinge <jeremy@goop.org>
Subject: [Xen-devel] Re: Hot plug scripts not working c/s 19355 (was: Re:
[PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it)
To: "Keir Fraser" <keir.fraser@eu.citrix.com>
Cc: "Dan Magenheimer" <dan.magenheimer@oracle.com>,
"Xen-devel" <xen-devel@lists.xensource.com>,
"bderzhavets@yahoo.com" <bderzhavets@yahoo.com>
Date: Friday, March 13, 2009, 1:09 PM

Keir Fraser wrote:> On 13/03/2009 15:46, "Dan Magenheimer"
<dan.magenheimer@oracle.com> wrote:> 
>   
>> A heads-up that I just got a "(tap)...Hotplug scripts not
working">> completely unrelated to pvops.
>> 
>> I updated hypervisor and tools (but notably NOT dom0 kernel) to
>> xen-unstable c/s 19355.  Launching a domain now fails with the
>> above error.  Rolling tools (only) back to 19187 makes the
>> problem go away.
>> 
>> Is there maybe a new dependency between tools and dom0 kernel?
>>     
> 
> You could try reverting changeset 19342, as most likely culprit.
>   
Is that the fixed xen-hotplug-cleanup change?  The symptoms aren''t
quite
the same; with the broken cleanup, there were lockfiles left lying about, so all
the "block add" scripts were blocked waiting for the lock to be
released.  In this case, everything is just sitting there; the hotplug never
seems to get started.

   J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Andrew Lyon

2009-Mar-13 18:49 UTC

head link

Re: [Xen-devel] Re: Hot plug scripts not working c/s 19355 (was: Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it)

On Fri, Mar 13, 2009 at 5:09 PM, Jeremy Fitzhardinge <jeremy@goop.org>
wrote:> Keir Fraser wrote:
>>
>> On 13/03/2009 15:46, "Dan Magenheimer"
<dan.magenheimer@oracle.com> wrote:
>>
>>
>>>
>>> A heads-up that I just got a "(tap)...Hotplug scripts not
working"
>>> completely unrelated to pvops.
>>>
>>> I updated hypervisor and tools (but notably NOT dom0 kernel) to
>>> xen-unstable c/s 19355.  Launching a domain now fails with the
>>> above error.  Rolling tools (only) back to 19187 makes the
>>> problem go away.
>>>
>>> Is there maybe a new dependency between tools and dom0 kernel?
>>>
>>
>> You could try reverting changeset 19342, as most likely culprit.
>>
>
> Is that the fixed xen-hotplug-cleanup change?  The symptoms aren''t
quite the
> same; with the broken cleanup, there were lockfiles left lying about, so
all
> the "block add" scripts were blocked waiting for the lock to be
released.
>  In this case, everything is just sitting there; the hotplug never seems to
> get started.
>
>   J
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

I think the hotplug problem where the lockfile was left lying about is
fixed, I updated to c/s 19354 today and since then no problems with
hotplug scripts, I can start and stop domains cleanly or by destroying
them and nothing is left behind, I''ve verified this by running the
xen-hotplug-cleanup script manually and it completes and removes the
lock as it should.

Like boris I am using a opensuse kernel, so if he is still having
problems I think he has another issue or there is a remaining problem
with hotplug which for some reason doesnt trigger on my test system.

There is a separate issue where Xen crashes with (XEN) **
page_alloc.c:534 -- 0/1 ffffffffffffffff which started around the same
time the hotplug scripts were fixed, it happens regardless of which
kernel is used and has been discussed in thead with subject "xen
unstable crash".

Andy

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Boris Derzhavets

2009-Mar-13 19:08 UTC

head link

Re: [Xen-devel] Re: Hot plug scripts not working c/s 19355 (was: Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it)

Suse''s  2.6.27 xen-ified kernel & Xen Unstable (19355)
don''t have any problems
with hotplug sripts. I''ve restarted F10 PV DomU several times.

Boris

--- On Fri, 3/13/09, Andrew Lyon <andrew.lyon@gmail.com> wrote:
From: Andrew Lyon <andrew.lyon@gmail.com>
Subject: Re: [Xen-devel] Re: Hot plug scripts not working c/s 19355 (was: Re: 
[PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it)
To: "Jeremy Fitzhardinge" <jeremy@goop.org>
Cc: "bderzhavets@yahoo.com" <bderzhavets@yahoo.com>, "Dan
Magenheimer" <dan.magenheimer@oracle.com>, "Xen-devel"
<xen-devel@lists.xensource.com>, "Keir Fraser"
<keir.fraser@eu.citrix.com>
Date: Friday, March 13, 2009, 2:49 PM

On Fri, Mar 13, 2009 at 5:09 PM, Jeremy Fitzhardinge <jeremy@goop.org>
wrote:> Keir Fraser wrote:
>>
>> On 13/03/2009 15:46, "Dan Magenheimer"
<dan.magenheimer@oracle.com> wrote:>>
>>
>>>
>>> A heads-up that I just got a "(tap)...Hotplug scripts not
working">>> completely unrelated to pvops.
>>>
>>> I updated hypervisor and tools (but notably NOT dom0 kernel) to
>>> xen-unstable c/s 19355.  Launching a domain now fails with the
>>> above error.  Rolling tools (only) back to 19187 makes the
>>> problem go away.
>>>
>>> Is there maybe a new dependency between tools and dom0 kernel?
>>>
>>
>> You could try reverting changeset 19342, as most likely culprit.
>>
>
> Is that the fixed xen-hotplug-cleanup change?  The symptoms aren''t
quite the> same; with the broken cleanup, there were lockfiles left lying about, so
all> the "block add" scripts were blocked waiting for the lock to be
released.>  In this case, everything is just sitting there; the hotplug never seems
to> get started.
>
>   J
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

I think the hotplug problem where the lockfile was left lying about is
fixed, I updated to c/s 19354 today and since then no problems with
hotplug scripts, I can start and stop domains cleanly or by destroying
them and nothing is left behind, I''ve verified this by running the
xen-hotplug-cleanup script manually and it completes and removes the
lock as it should.

Like boris I am using a opensuse kernel, so if he is still having
problems I think he has another issue or there is a remaining problem
with hotplug which for some reason doesnt trigger on my test system.

There is a separate issue where Xen crashes with (XEN) **
page_alloc.c:534 -- 0/1 ffffffffffffffff which started around the same
time the hotplug scripts were fixed, it happens regardless of which
kernel is used and has been discussed in thead with subject "xen
unstable crash".

Andy

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Dan Magenheimer

2009-Mar-16 17:17 UTC

head link

[Xen-devel] RE: Hot plug scripts not working c/s 19355 (was: Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it)

> Any chance you could try to pinpoint the failing version?
> > You could try reverting changeset 19342, as most likely culprit.
Well I''ve been struggling with this on and off since Friday and
am having major problems because there are now many parts that
I''m no longer sure how to synchronize.  The tools now seem to
be dependent on ioemu but I don''t know how (or if its possible)
to synchronize versions of the ioemu git tree with a specific
xen hg changeset.  Rolling xen tools back to 19187 worked fine,
but I couldn''t roll the tools back to tip due to a compile error

ioemu-dir/hw/pt-msi.c:95: error: too few arguments to xc_domain_update_msi_irq

I did a git pull in ioemu-dir and no longer got the compile error
but it didn''t help (same hotplug problem) and now I can''t
rollback
xen tools to 19187 because I get the opposite compile error.
So my test machine, which HAD been working at least with tools 19187
is now useless unless I solve the problem.

(BTW, I tried reverting 19342 and it didn''t solve the problem.)

I was able to do a fresh tip build and install on a new test machine
(that previously had no xen on it at all), so I suspect my problem was
due to a mismatch between dom0 and "something else" so am now in the
process of rebuilding dom0 from latest.

Argh!  I thought bisecting was supposed to be easy!

With so many interdependent parts (and two different change-tracking
tools needed), perhaps its time to start being more careful about
describing version interdependencies between them?

Or maybe if I were sufficiently knowledgeable about both mercurial
and git, this wouldn''t be a problem?
> -----Original Message-----
> From: Jeremy Fitzhardinge [mailto:jeremy@goop.org]
> Sent: Friday, March 13, 2009 11:09 AM
> To: Keir Fraser
> Cc: Dan Magenheimer; Xen-devel; bderzhavets@yahoo.com
> Subject: Re: Hot plug scripts not working c/s 19355 (was: Re: [PATCH]
> xen: mask XSAVE in cpuid since we don''t allow guests to use it)
> 
> 
> Keir Fraser wrote:
> > On 13/03/2009 15:46, "Dan Magenheimer" 
> <dan.magenheimer@oracle.com> wrote:
> >
> >   
> >> A heads-up that I just got a "(tap)...Hotplug scripts not
working"
> >> completely unrelated to pvops.
> >>
> >> I updated hypervisor and tools (but notably NOT dom0 kernel) to
> >> xen-unstable c/s 19355.  Launching a domain now fails with the
> >> above error.  Rolling tools (only) back to 19187 makes the
> >> problem go away.
> >>
> >> Is there maybe a new dependency between tools and dom0 kernel?
> >>     
> >
> > You could try reverting changeset 19342, as most likely culprit.
> >   
> 
> Is that the fixed xen-hotplug-cleanup change?  The symptoms 
> aren''t quite 
> the same; with the broken cleanup, there were lockfiles left lying 
> about, so all the "block add" scripts were blocked waiting 
> for the lock 
> to be released.  In this case, everything is just sitting there; the 
> hotplug never seems to get started.
> 
>     J
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jeremy Fitzhardinge

2009-Mar-16 18:00 UTC

head link

[Xen-devel] Re: Hot plug scripts not working c/s 19355

Dan Magenheimer wrote:> Well I''ve been struggling with this on and off since Friday and
> am having major problems because there are now many parts that
> I''m no longer sure how to synchronize.  The tools now seem to
> be dependent on ioemu but I don''t know how (or if its possible)
> to synchronize versions of the ioemu git tree with a specific
> xen hg changeset.  Rolling xen tools back to 19187 worked fine,
> but I couldn''t roll the tools back to tip due to a compile error
>
> ioemu-dir/hw/pt-msi.c:95: error: too few arguments to
xc_domain_update_msi_irq
>
> I did a git pull in ioemu-dir and no longer got the compile error
> but it didn''t help (same hotplug problem) and now I can''t
rollback
> xen tools to 19187 because I get the opposite compile error.
> So my test machine, which HAD been working at least with tools 19187
> is now useless unless I solve the problem.
>   
Yeah, having xen-unstable in two different source systems is extremely 
brittle; I don''t think its sustainable for much longer.
> (BTW, I tried reverting 19342 and it didn''t solve the problem.)
>
> I was able to do a fresh tip build and install on a new test machine
> (that previously had no xen on it at all), so I suspect my problem was
> due to a mismatch between dom0 and "something else" so am now in
the
> process of rebuilding dom0 from latest.
>
> Argh!  I thought bisecting was supposed to be easy!
>
> With so many interdependent parts (and two different change-tracking
> tools needed), perhaps its time to start being more careful about
> describing version interdependencies between them?
>
> Or maybe if I were sufficiently knowledgeable about both mercurial
> and git, this wouldn''t be a problem?
No, its all a big pain, especially since there are multiple bugs with 
similar symptoms.

There was the original hotplug script bug in xen-unstable, which now 
appears to be fixed.

There was also a pvops dom0 bug, which turned out to be an upstream bug 
in vsprintf()''s handling of "%.*s", which is also now fixed.

And there may well be some other bugs...

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Xen devel - Mar 2009 - [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

[Xen-devel] [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

[Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

[Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

[Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] [PATCH] xen: mask XSAVE in cpuid since we don''t allowguests to use it

[Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] [PATCH] xen: mask XSAVE in cpuid since we don''t allowguests to use it

[Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''tallow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''tallow guests to use it

[Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] [PATCH] xen: mask XSAVE in cpuid since we don''t allowguests to use it

Re: [Xen-devel] [PATCH] xen: mask XSAVE in cpuid since we don''t allowguests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

[Xen-devel] Attempt to install SNV_106 under Suse''s 2.6.27 xen-ified kernel & Xen Unstable

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

Re: [Xen-devel] Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it

[Xen-devel] Hot plug scripts not working c/s 19355 (was: Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it)

[Xen-devel] Re: Hot plug scripts not working c/s 19355

[Xen-devel] Re: Hot plug scripts not working c/s 19355 (was: Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it)

[Xen-devel] Re: Hot plug scripts not working c/s 19355 (was: Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it)

Re: [Xen-devel] Re: Hot plug scripts not working c/s 19355 (was: Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it)

Re: [Xen-devel] Re: Hot plug scripts not working c/s 19355 (was: Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it)

Re: [Xen-devel] Re: Hot plug scripts not working c/s 19355 (was: Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it)

[Xen-devel] RE: Hot plug scripts not working c/s 19355 (was: Re: [PATCH] xen: mask XSAVE in cpuid since we don''t allow guests to use it)

[Xen-devel] Re: Hot plug scripts not working c/s 19355