Jürgen Keil
2008-May-26 14:18 UTC
xvm-gate: snv_90 hvm domU crashes on 32-bit xvm-3.3-unstable hypervirsor
Trying to boot a hvm Solaris (snv_90) domU on a 32-bit matrix-unstable dom0
kernel (on an amd64 x2 cpu) crashes the hvm domain (before kmdb can grab
control). The solaris domU grub boot entry used is the standard entry
starting with "kernel$" and include $ISADIR tokens.
xm dmesg shows:
# xm dmesg
(xVM) xVM version 3.3-unstable-xvm
(xVM) Command line: com1=9600,8n1 console=vga
(xVM) Video information:
(xVM) VGA is text mode 80x25, font 8x16
(xVM) VBE/DDC methods: V2; EDID transfer time: 1 seconds
(xVM) Disc information:
(xVM) Found 2 MBR signatures
(xVM) Found 2 EDD information structures
(xVM) Xen-e820 RAM map:
(xVM) 0000000000000000 - 000000000009f000 (usable)
(xVM) 000000000009f000 - 00000000000a0000 (reserved)
(xVM) 00000000000f0000 - 0000000000100000 (reserved)
(xVM) 0000000000100000 - 000000007fee0000 (usable)
(xVM) 000000007fee0000 - 000000007fee3000 (ACPI NVS)
(xVM) 000000007fee3000 - 000000007fef0000 (ACPI data)
(xVM) 000000007fef0000 - 000000007ff00000 (reserved)
(xVM) 00000000f0000000 - 00000000f4000000 (reserved)
(xVM) 00000000fec00000 - 0000000100000000 (reserved)
(xVM) System RAM: 2046MB (2095612kB)
(xVM) Xen heap: 9MB (10012kB)
(xVM) Domain heap initialised: DMA width 32 bits
(xVM) Processor #0 15:11 APIC version 16
(xVM) Processor #1 15:11 APIC version 16
(xVM) IOAPIC[0]: apic_id 4, version 17, address 0xfec00000, GSI 0-23
(xVM) Enabling APIC mode: Flat. Using 1 I/O APICs
(xVM) Using scheduler: SMP Credit Scheduler (credit)
(xVM) Detected 2210.213 MHz processor.
(xVM) AMD SVM: ASIDs disabled.
(xVM) HVM: SVM enabled
(xVM) CPU0: AMD Athlon(tm) 64 X2 Dual Core Processor 4200+ stepping 02
(xVM) Booting processor 1/1 eip 8c000
(xVM) AMD: Disabling C1 Clock Ramping Node #0
(xVM) AMD SVM: ASIDs disabled.
(xVM) CPU1: AMD Athlon(tm) 64 X2 Dual Core Processor 4200+ stepping 02
(xVM) Total of 2 processors activated.
(xVM) ENABLING IO-APIC IRQs
(xVM) -> Using new ACK method
(xVM) Platform timer is 25.000MHz HPET
(xVM) Brought up 2 CPUs
(xVM) AMD_IOV: IOMMU not found!
(xVM) I/O virtualisation disabled
(xVM) *** LOADING DOMAIN 0 ***
(xVM) Xen kernel: 32-bit, PAE, lsb
(xVM) Dom0 kernel: 32-bit, PAE, lsb, paddr 0x40000000 -> 0x40825a3c
(xVM) PHYSICAL MEMORY ARRANGEMENT:
(xVM) Dom0 alloc.: 0000000038000000->000000003c000000 (468627 pages to be
allocated)
(xVM) VIRTUAL MEMORY ARRANGEMENT:
(xVM) Loaded kernel: 40000000->40825a3c
(xVM) Init. ramdisk: 40826000->42297800
(xVM) Phys-Mach map: 42298000->42471a4c
(xVM) Start info: 42472000->42472474
(xVM) Page tables: 42473000->4248c000
(xVM) Boot stack: 4248c000->4248d000
(xVM) TOTAL: 40000000->42800000
(xVM) ENTRY ADDRESS: 40800000
(xVM) Dom0 has maximum 2 VCPUs
(xVM) Initrd len 0x1a71800, start at 0x40826000
(xVM) Scrubbing Free RAM: .done.
(xVM) Xen trace buffers: disabled
(xVM) Std. Loglevel: Errors and warnings
(xVM) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(xVM) Xen is relinquishing VGA console.
(xVM) *** Serial input -> DOM0 (type ''CTRL-a'' three times
to switch input to Xen)
(xVM) Freed 104kB init memory.
(xVM) hvm.c:782:d12 Trying to set reserved bit in EFER: 900
(xVM) hvm.c:782:d13 Trying to set reserved bit in EFER: 900
MSR EFER bits 0x900 is: 0x100 EFER_LME, 0x800 EFER_NX
Apparently the 32-bit hypervisor refuses to enable long mode.
xen.hg/xen/arch/x86/hvm/hvm.c checks:
if ( (value & ~(EFER_FFXSE | EFER_LME | EFER_NX | EFER_SCE)) ||
((sizeof(long) != 8) && (value & EFER_LME)) ||
<<<<<<<<<<<<<<<<<<<<
(!cpu_has_nx && (value & EFER_NX)) ||
(!cpu_has_syscall && (value & EFER_SCE)) ||
(!cpu_has_ffxsr && (value & EFER_FFXSE)) )
{
gdprintk(XENLOG_WARNING, "Trying to set reserved bit in "
"EFER: %"PRIx64"\n", value);
hvm_inject_exception(TRAP_gp_fault, 0, 0);
return X86EMUL_EXCEPTION;
}
Reason is the new xen-3.3 cpuid stuff. The host''s cpuid data is
queried,
filtered, and used as data for the hvm domU cpuid instruction support.
The "long mode" X86_FEATURE_LM of the amd64 cpu is passed unmodified
to
the hvm domU. Since the 32-bit hypervisor doesn''t allow long mode in
hvm
domains, the X86_FEATURE_LM should probably not announced in such a hvm
domain.
Workaround:
==========Force booting a 32-bit kernel (remove all $ISADIR tokens from grub
boot entry).
This didn''t happen under xen-3.1.4-xvm / 32-bit snv_91 dom0
I''ve experimented with the following fix (don''t pass the
X86_FEATURE_LM cpuid feature, when running on a
hypervisor that doesn''t support 64-bit):
diff --git a/tools/libxc/xc_cpuid_x86.c b/tools/libxc/xc_cpuid_x86.c
--- a/tools/libxc/xc_cpuid_x86.c
+++ b/tools/libxc/xc_cpuid_x86.c
@@ -35,6 +35,7 @@
int xc, domid_t domid, const unsigned int *input, unsigned int *regs)
{
unsigned long pae = 0;
+ xen_capabilities_info_t xen_caps = "";
xc_get_hvm_param(xc, domid, HVM_PARAM_PAE_ENABLED, &pae);
@@ -56,6 +57,9 @@
if ( !pae )
clear_bit(X86_FEATURE_PAE & 31, regs[3]);
clear_bit(X86_FEATURE_PSE36 & 31, regs[3]);
+ if (xc_version(xc, XENVER_capabilities, &xen_caps) == 0 &&
+ strstr(xen_caps, "x86_64") == NULL)
+ clear_bit(X86_FEATURE_LM & 31, regs[3]);
/* Filter all other features according to a whitelist. */
regs[2] &= (bitmaskof(X86_FEATURE_LAHF_LM) |
@@ -80,6 +84,8 @@
static void intel_xc_cpuid_policy(
int xc, domid_t domid, const unsigned int *input, unsigned int *regs)
{
+ xen_capabilities_info_t xen_caps = "";
+
switch ( input[0] )
{
case 0x00000001:
@@ -93,6 +99,10 @@
break;
case 0x80000001:
+ if (xc_version(xc, XENVER_capabilities, &xen_caps) == 0 &&
+ strstr(xen_caps, "x86_64") == NULL)
+ clear_bit(X86_FEATURE_LM & 31, regs[3]);
+
/* Only a few features are advertised in Intel''s 0x80000001.
*/
regs[2] &= (bitmaskof(X86_FEATURE_LAHF_LM));
regs[3] &= (bitmaskof(X86_FEATURE_NX) |
This message posted from opensolaris.org
John Levon
2008-May-27 12:23 UTC
Re: xvm-gate: snv_90 hvm domU crashes on 32-bit xvm-3.3-unstable hypervirsor
On Mon, May 26, 2008 at 07:18:29AM -0700, J??rgen Keil wrote:> Reason is the new xen-3.3 cpuid stuff. The host''s cpuid data is queried, > filtered, and used as data for the hvm domU cpuid instruction support. > The "long mode" X86_FEATURE_LM of the amd64 cpu is passed unmodified to > the hvm domU. Since the 32-bit hypervisor doesn''t allow long mode in hvm > domains, the X86_FEATURE_LM should probably not announced in such a hvm > domain.I agree this seems wrong, and I think you should try sending this to xen-devel list regards john