Dan Magenheimer
2010-May-13 19:34 UTC
[Xen-devel] need known working 64-bit upstream kernel config file for PV guest
Anybody have a known working 64-bit config file for an upstream (>=2.6.33) PV kernel? I haven''t been able to find the key config difference vs a 32-bit working config file, see: http://lists.xensource.com/archives/html/xen-devel/2010-05/msg00682.html I tried another config file from a working RHEL6 PV guest (2.6.32-19) and it panics an upstream kernel built in a 64-bit PV guest also. (Preferably would like as small a config as possible, but I''ll take what I can get. I don''t need PCI passthrough or anything exotic.) Alternately, if someone with a good eye for config files can spot the culprit CONFIG_XXX that causes the crash in the attached config and tombstone, that would be even better. I tried turning off the following without any change and now am just shooting in the dark. (CONFIG_)DEVKMEM X86_SPEEDSTEP_CENTRINO X86_VERBOSE_BOOTUP FIRMWARE_IN_KERNEL DEBUG_STACK DEBUG_STACK_OVERVLOW COMPAT_BRK Thanks, Dan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Pasi Kärkkäinen
2010-May-13 20:30 UTC
Re: [Xen-devel] need known working 64-bit upstream kernel config file for PV guest
On Thu, May 13, 2010 at 12:34:27PM -0700, Dan Magenheimer wrote:> Anybody have a known working 64-bit config file for an upstream > (>=2.6.33) PV kernel? I haven''t been able to find the key > config difference vs a 32-bit working config file, see: >I just tried the Fedora 13 Linux 2.6.33 (2.6.33.3-85.fc13.x86_64) kernel and it seems to work OK.> http://lists.xensource.com/archives/html/xen-devel/2010-05/msg00682.html > > I tried another config file from a working RHEL6 PV guest (2.6.32-19) > and it panics an upstream kernel built in a 64-bit PV guest also. > > (Preferably would like as small a config as possible, but I''ll > take what I can get. I don''t need PCI passthrough or anything > exotic.) >What exact upstream kernel.org kernels you are trying to make work? I assume you''re familiar with debugging crashing guest kernels? If not, see here: http://wiki.xensource.com/xenwiki/XenCommonProblems The chapter "My Xen PV guest kernel crashes, how can I debug it?". -- Pasi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Pasi Kärkkäinen
2010-May-13 20:37 UTC
Re: [Xen-devel] need known working 64-bit upstream kernel config file for PV guest
On Thu, May 13, 2010 at 11:30:53PM +0300, Pasi Kärkkäinen wrote:> On Thu, May 13, 2010 at 12:34:27PM -0700, Dan Magenheimer wrote: > > Anybody have a known working 64-bit config file for an upstream > > (>=2.6.33) PV kernel? I haven''t been able to find the key > > config difference vs a 32-bit working config file, see: > > > > I just tried the Fedora 13 Linux 2.6.33 (2.6.33.3-85.fc13.x86_64) kernel > and it seems to work OK. > > > http://lists.xensource.com/archives/html/xen-devel/2010-05/msg00682.html > > > > I tried another config file from a working RHEL6 PV guest (2.6.32-19) > > and it panics an upstream kernel built in a 64-bit PV guest also. > > > > (Preferably would like as small a config as possible, but I''ll > > take what I can get. I don''t need PCI passthrough or anything > > exotic.) > > > > What exact upstream kernel.org kernels you are trying to make work? > > I assume you''re familiar with debugging crashing guest kernels? > > If not, see here: > http://wiki.xensource.com/xenwiki/XenCommonProblems > > The chapter "My Xen PV guest kernel crashes, how can I debug it?". >Oh, and the other way to solve configuration problems is to pass "earlyprintk=xen console=hvc0" for the domU kernel.. Please paste the boot/crash logs? -- Pasi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Dan Magenheimer
2010-May-13 20:49 UTC
RE: [Xen-devel] need known working 64-bit upstream kernel config file for PV guest
Hi Pasi -- Thanks for the reply. Hope your trip home went well!> What exact upstream kernel.org kernels you are trying to make work?Any working x86_64 config file for any upstream kernel >=2.6.32 would be fine. I have a 32-bit config file that works fine for 2.6.27 up to 2.6.32 (and I think for 2.6.33... trying again now). I''ve just found out that my config file does NOT work for x86 2.6.34-rc5 though. Not sure why...> I assume you''re familiar with debugging crashing guest kernels? > > If not, see here: > http://wiki.xensource.com/xenwiki/XenCommonProblems > > The chapter "My Xen PV guest kernel crashes, how can I debug it?".I''ve done that before but these panics are all very early and I have a tombstone for each panic. I thought enough people are using upstream kernels that I must just have some CONFIG_ incorrect. Perhaps I am wrong and I am the first one to try these upstream kernels (at least in a PV domain)? It seems that all the failures I am seeing have some kind of syscall in the stacktrace and they happen early enough in init that I suppose it is possible that this is the very first syscall that is failing. Thanks, Dan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Pasi Kärkkäinen
2010-May-13 20:57 UTC
Re: [Xen-devel] need known working 64-bit upstream kernel config file for PV guest
On Thu, May 13, 2010 at 01:49:01PM -0700, Dan Magenheimer wrote:> Hi Pasi -- > > Thanks for the reply. Hope your trip home went well! >Yeah it did.. except the jetlag :)> > What exact upstream kernel.org kernels you are trying to make work? > > Any working x86_64 config file for any upstream kernel >=2.6.32 > would be fine. I have a 32-bit config file that works fine for 2.6.27 > up to 2.6.32 (and I think for 2.6.33... trying again now). >2.6.33.0 is known to crash as Xen PV guest, I think 2.6.33.2 fixes that. So better use 2.6.33.latest update. You could grab the Fedora 13 2.6.33.3 kernel and get the .config from it? That F13 kernel works for me.> I''ve just found out that my config file does NOT work for x86 > 2.6.34-rc5 though. Not sure why... > > > I assume you''re familiar with debugging crashing guest kernels? > > > > If not, see here: > > http://wiki.xensource.com/xenwiki/XenCommonProblems > > > > The chapter "My Xen PV guest kernel crashes, how can I debug it?". > > I''ve done that before but these panics are all very early and > I have a tombstone for each panic. I thought enough people are > using upstream kernels that I must just have some CONFIG_ > incorrect. Perhaps I am wrong and I am the first one to > try these upstream kernels (at least in a PV domain)? > > It seems that all the failures I am seeing have some kind > of syscall in the stacktrace and they happen early enough > in init that I suppose it is possible that this is the very > first syscall that is failing. >Hmm.. Jeremy was thinking that some non-Xen changes in 2.6.34 might have caused problems. I haven''t tried 2.6.34 myself (yet). I''ll try some day in the near future.. -- Pasi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Dan Magenheimer
2010-May-13 23:03 UTC
RE: [Xen-devel] need known working 64-bit upstream kernel config file for PV guest
> > Any working x86_64 config file for any upstream kernel >=2.6.32 > > would be fine. I have a 32-bit config file that works fine for > 2.6.27 > > up to 2.6.32 (and I think for 2.6.33... trying again now). > > > > 2.6.33.0 is known to crash as Xen PV guest, I think 2.6.33.2 fixes > that. > So better use 2.6.33.latest update. > > You could grab the Fedora 13 2.6.33.3 kernel and get the .config from > it? > That F13 kernel works for me.Tried a 2.6.33.4 upstream kernel with the F13 config-2.6.33.3-79.rc13.x86_64 and it also fails. So there must be a kernel bug that the F13 kernel patches fix? Here''s the output... note still a system call failing... Thanks, Dan (early) Initializing cgroup subsys cpuset (early) Initializing cgroup subsys cpu (early) Linux version 2.6.33.4 (root@dmagenhe-nsvpn-dhcp-141-144-22-17.vpn.oracle.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Thu May 13 18:10:25 EDT 2010 (early) Command line: ro root=LABEL=/ earlyprintk=xen console=hvc0 (early) ACPI in unprivileged domain disabled (early) BIOS-provided physical RAM map: (early) Xen: 0000000000000000 - 00000000000a0000 (usable) (early) Xen: 00000000000a0000 - 0000000000100000 (reserved) (early) Xen: 0000000000100000 - 0000000060000000 (usable) (early) bootconsole [xenboot0] enabled (early) NX (Execute Disable) protection: active (early) DMI not present or invalid. (early) No AGP bridge found (early) last_pfn = 0x60000 max_arch_pfn = 0x400000000 (early) init_memory_mapping: 0000000000000000-0000000060000000 (early) RAMDISK: 01de0000 - 022b2000 (early) No NUMA configuration found (early) Faking a node at 0000000000000000-0000000060000000 (early) Bootmem setup node 0 0000000000000000-0000000060000000 (early) NODE_DATA [0000000000007000 - 000000000001cfff] (early) bootmap [000000000001d000 - 0000000000028fff] pages c (early) (8 early reservations) ==> bootmem [0000000000 - 0060000000] (early) #0 [0000000000 - 0000001000] BIOS data page(early) ==> [0000000000 - 0000001000] (early) #1 [00025b5000 - 00025cc000] XEN PAGETABLES(early) ==> [00025b5000 - 00025cc000] (early) #2 [0001000000 - 0001dbf358] TEXT DATA BSS(early) ==> [0001000000 - 0001dbf358] (early) #3 [0001de0000 - 00022b2000] RAMDISK(early) ==> [0001de0000 - 00022b2000] (early) #4 [00022b2000 - 00025b5000] XEN START INFO(early) ==> [00022b2000 - 00025b5000] (early) #5 [0000001000 - 0000003000] TRAMPOLINE(early) ==> [0000001000 - 0000003000] (early) #6 [0000003000 - 0000007000] ACPI WAKEUP(early) ==> [0000003000 - 0000007000] (early) #7 [0000100000 - 00003e9000] PGTABLE(early) ==> [0000100000 - 00003e9000] (early) Zone PFN ranges: (early) DMA 0x00000000 -> 0x00001000 (early) DMA32 0x00001000 -> 0x00100000 (early) Normal 0x00100000 -> 0x00100000 (early) Movable zone start PFN for each node (early) early_node_map[2] active PFN ranges (early) 0: 0x00000000 -> 0x000000a0 (early) 0: 0x00000100 -> 0x00060000 (early) SFI: Simple Firmware Interface v0.7 http://simplefirmware.org (early) SMP: Allowing 2 CPUs, 0 hotplug CPUs (early) No local APIC present (early) APIC: disable apic facility (early) APIC: switched to apic NOOP (early) PM: Registered nosave memory: 00000000000a0000 - 0000000000100000 (early) Allocating PCI resources starting at 60000000 (gap: 60000000:a0000000) (early) Booting paravirtualized kernel on Xen (early) Xen version: 4.0.0 (preserve-AD) (early) setup_percpu: NR_CPUS:256 nr_cpumask_bits:256 nr_cpu_ids:2 nr_node_ids:1 (early) PERCPU: Embedded 30 pages/cpu @ffff880002602000 s91352 r8192 d23336 u122880 (early) pcpu-alloc: s91352 r8192 d23336 u122880 alloc=30*4096(early) (early) pcpu-alloc: (early) [0] (early) 0 (early) [0] (early) 1 (early) (early) Xen: using vcpu_info placement (early) Built 1 zonelists in Node order, mobility grouping on. Total pages: 386992 (early) Policy zone: DMA32 (early) Kernel command line: ro root=LABEL=/ earlyprintk=xen console=hvc0 (early) PID hash table entries: 4096 (order: 3, 32768 bytes) (early) Checking aperture... (early) No AGP bridge found (early) Memory: 1525136k/1572864k available (4295k kernel code, 384k absent, 47344k reserved, 7564k data, 752k init) (early) SLUB: Genslabs=14, HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 (early) Hierarchical RCU implementation. (early) NR_IRQS:4352 nr_irqs:288 (early) Console: colour dummy device 80x25 (early) Initializing cgroup subsys cpuset (early) Initializing cgroup subsys cpu (early) Linux version 2.6.33.4 (root@dmagenhe-nsvpn-dhcp-141-144-22-17.vpn.oracle.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Thu May 13 18:10:25 EDT 2010 (early) Command line: ro root=LABEL=/ earlyprintk=xen console=hvc0 (early) ACPI in unprivileged domain disabled (early) BIOS-provided physical RAM map: (early) Xen: 0000000000000000 - 00000000000a0000 (usable) (early) Xen: 00000000000a0000 - 0000000000100000 (reserved) (early) Xen: 0000000000100000 - 0000000060000000 (usable) (early) bootconsole [xenboot0] enabled (early) NX (Execute Disable) protection: active (early) DMI not present or invalid. (early) No AGP bridge found (early) last_pfn = 0x60000 max_arch_pfn = 0x400000000 (early) init_memory_mapping: 0000000000000000-0000000060000000 (early) RAMDISK: 01de0000 - 022b2000 (early) No NUMA configuration found (early) Faking a node at 0000000000000000-0000000060000000 (early) Bootmem setup node 0 0000000000000000-0000000060000000 (early) NODE_DATA [0000000000007000 - 000000000001cfff] (early) bootmap [000000000001d000 - 0000000000028fff] pages c (early) (8 early reservations) ==> bootmem [0000000000 - 0060000000] (early) #0 [0000000000 - 0000001000] BIOS data page ==> [0000000000 - 0000001000] (early) #1 [00025b5000 - 00025cc000] XEN PAGETABLES ==> [00025b5000 - 00025cc000] (early) #2 [0001000000 - 0001dbf358] TEXT DATA BSS ==> [0001000000 - 0001dbf358] (early) #3 [0001de0000 - 00022b2000] RAMDISK ==> [0001de0000 - 00022b2000] (early) #4 [00022b2000 - 00025b5000] XEN START INFO ==> [00022b2000 - 00025b5000] (early) #5 [0000001000 - 0000003000] TRAMPOLINE ==> [0000001000 - 0000003000] (early) #6 [0000003000 - 0000007000] ACPI WAKEUP ==> [0000003000 - 0000007000] (early) #7 [0000100000 - 00003e9000] PGTABLE ==> [0000100000 - 00003e9000] (early) Zone PFN ranges: (early) DMA 0x00000000 -> 0x00001000 (early) DMA32 0x00001000 -> 0x00100000 (early) Normal 0x00100000 -> 0x00100000 (early) Movable zone start PFN for each node (early) early_node_map[2] active PFN ranges (early) 0: 0x00000000 -> 0x000000a0 (early) 0: 0x00000100 -> 0x00060000 (early) SFI: Simple Firmware Interface v0.7 http://simplefirmware.org (early) SMP: Allowing 2 CPUs, 0 hotplug CPUs (early) No local APIC present (early) APIC: disable apic facility (early) APIC: switched to apic NOOP (early) PM: Registered nosave memory: 00000000000a0000 - 0000000000100000 (early) Allocating PCI resources starting at 60000000 (gap: 60000000:a0000000) (early) Booting paravirtualized kernel on Xen (early) Xen version: 4.0.0 (preserve-AD) (early) setup_percpu: NR_CPUS:256 nr_cpumask_bits:256 nr_cpu_ids:2 nr_node_ids:1 (early) PERCPU: Embedded 30 pages/cpu @ffff880002602000 s91352 r8192 d23336 u122880 (early) pcpu-alloc: s91352 r8192 d23336 u122880 alloc=30*4096 (early) pcpu-alloc: [0] 0 [0] 1 (early) Xen: using vcpu_info placement (early) Built 1 zonelists in Node order, mobility grouping on. Total pages: 386992 (early) Policy zone: DMA32 (early) Kernel command line: ro root=LABEL=/ earlyprintk=xen console=hvc0 (early) PID hash table entries: 4096 (order: 3, 32768 bytes) (early) Checking aperture... (early) No AGP bridge found (early) Memory: 1525136k/1572864k available (4295k kernel code, 384k absent, 47344k reserved, 7564k data, 752k init) (early) SLUB: Genslabs=14, HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 (early) Hierarchical RCU implementation. (early) NR_IRQS:4352 nr_irqs:288 (early) Console: colour dummy device 80x25 (early) console [tty0] enabled console [hvc0] enabled, bootconsole disabled (early) console [hvc0] enabled, bootconsole disabled allocated 15728640 bytes of page_cgroup please try ''cgroup_disable=memory'' option if you don''t want memory cgroups installing Xen timer for CPU 0 Detected 2992.570 MHz processor. Calibrating delay loop (skipped), value calculated using timer frequency.. 5985.14 BogoMIPS (lpj=2992570) Security Framework initialized SELinux: Initializing. Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes) Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes) Mount-cache hash table entries: 256 Initializing cgroup subsys debug Initializing cgroup subsys ns Initializing cgroup subsys cpuacct Initializing cgroup subsys memory Initializing cgroup subsys devices Initializing cgroup subsys freezer Initializing cgroup subsys net_cls Initializing cgroup subsys blkio CPU: Physical Processor ID: 0 CPU: Processor Core ID: 0 Performance Events: unsupported p6 CPU model 15 no PMU driver, software events only. SMP alternatives: switching to UP code installing Xen timer for CPU 1 SMP alternatives: switching to SMP code Brought up 2 CPUs devtmpfs: initialized Grant table initialized regulator: core version 0.5 Time: 165:165:165 Date: 165/165/65 NET: Registered protocol family 16 PCI: Fatal: No config space access function found bio: create slab <bio-0> at 0 ACPI: Interpreter disabled. xen_balloon: Initialising balloon driver. vgaarb: loaded SCSI subsystem initialized usbcore: registered new interface driver usbfs usbcore: registered new interface driver hub usbcore: registered new device driver usb PCI: System does not support PCI PCI: System does not support PCI NetLabel: Initializing NetLabel: domain hash size = 128 NetLabel: protocols = UNLABELED CIPSOv4 NetLabel: unlabeled traffic allowed by default Switching to clocksource xen pnp: PnP ACPI: disabled NET: Registered protocol family 2 IP route cache hash table entries: 65536 (order: 7, 524288 bytes) TCP established hash table entries: 262144 (order: 10, 4194304 bytes) TCP bind hash table entries: 65536 (order: 8, 1048576 bytes) TCP: Hash tables configured (established 262144 bind 65536) TCP reno registered UDP hash table entries: 1024 (order: 3, 32768 bytes) UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes) NET: Registered protocol family 1 Trying to unpack rootfs image as initramfs... Freeing initrd memory: 4936k freed platform rtc_cmos: registered platform RTC device (no PNP device found) Intel PCLMULQDQ-NI instructions are not detected. audit: initializing netlink socket (disabled) type=2000 audit(1273791165.134:1): initialized HugeTLB registered 2 MB page size, pre-allocated 0 pages VFS: Disk quotas dquot_6.5.2 Dquot-cache hash table entries: 512 (order 0, 4096 bytes) msgmni has been set to 2988 alg: No test for stdrng (krng) Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252) io scheduler noop registered io scheduler deadline registered io scheduler cfq registered (default) pci_hotplug: PCI Hot Plug PCI Core version: 0.5 pciehp: PCI Express Hot Plug Controller Driver version: 0.4 acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 pci-stub: invalid id string "" Console: switching to colour frame buffer device 100x37 Non-volatile memory driver v1.3 Linux agpgart interface v0.103 Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled brd: module loaded loop: module loaded input: Macintosh mouse button emulation as /devices/virtual/input/input0 Fixed MDIO Bus: probed ehci_hcd: USB 2.0 ''Enhanced'' Host Controller (EHCI) Driver ohci_hcd: USB 1.1 ''Open'' Host Controller (OHCI) Driver uhci_hcd: USB Universal Host Controller Interface driver PNP: No PS/2 controller found. Probing ports directly. i8042.c: No controller found. mice: PS/2 mouse device common for all mice input: Xen Virtual Keyboard as /devices/virtual/input/input1 input: Xen Virtual Pointer as /devices/virtual/input/input2 rtc_cmos: probe of rtc_cmos failed with error -16 device-mapper: uevent: version 1.0.3 device-mapper: ioctl: 4.17.0-ioctl (2010-03-05) initialised: dm-devel@redhat.com cpuidle: using governor ladder cpuidle: using governor menu usbcore: registered new interface driver hiddev usbcore: registered new interface driver usbhid usbhid: USB HID core driver nf_conntrack version 0.5.0 (16384 buckets, 65536 max) CONFIG_NF_CT_ACCT is deprecated and will be removed soon. Please use nf_conntrack.acct=1 kernel parameter, acct=1 nf_conntrack module option or sysctl net.netfilter.nf_conntrack_acct=1 to enable it. ip_tables: (C) 2000-2006 Netfilter Core Team TCP cubic registered Initializing XFRM netlink socket NET: Registered protocol family 17 registered taskstats version 1 No TPM chip found, activating TPM-bypass! XENBUS: Device with no driver: device/vbd/51712 XENBUS: Device with no driver: device/vif/0 XENBUS: Device with no driver: device/console/0 Magic number: 1:252:3141 drivers/rtc/hctosys.c: unable to open rtc device (rtc0) Initalizing network drop monitor service Freeing unused kernel memory: 752k freed Write protecting the kernel read-only data: 10240k Freeing unused kernel memory: 1832k freed Freeing unused kernel memory: 1868k freed Red Hat nash version 5.1.19.6 starting Mounting proc filesystem Mounting sysfs filesystem Creating /dev Creating initial device nodes Setting up hotplug. Creating block device nodes. Loading xen-blkfront.ko module blkfront: xvda: barriers enabled xvda: xvda1 xvda2 xvda3 Loading xen-blkfront.ko module insmod: error inserting ''/lib/xen-blkfront.ko'': -1 File exists Loading xen-blkfront.ko module insmod: error inserting ''/lib/xen-blkfront.ko'': -1 File exists Loading xen-blkfront.ko module insmod: error inserting ''/lib/xen-blkfront.ko'': -1 File exists Scanning and configuring dmraid supported devices Trying to resume from LABEL=SWAP-VM Unable to access resume device (LABEL=SWAP-VM) Creating root device. Mounting root filesystem. mount: could not find filesystem ''/dev/root'' Setting up other filesystems. Setting up new root fs setuproot: moving /dev failed: No such file or directory no fstab.sys, mounting internal defaults setuproot: error mounting /proc: No such file or directory setuproot: error mounting /sys: No such file or directory Switching to new root and running init. unmounting old /dev unmounting old /proc unmounting old /sys switchroot: mount failed: No such file or directory Kernel panic - not syncing: Attempted to kill init! Pid: 1, comm: init Not tainted 2.6.33.4 #1 Call Trace: [<ffffffff8104b811>] panic+0xa0/0x167 [<ffffffff8109be6f>] ? call_rcu_sched+0x10/0x12 [<ffffffff81085b92>] ? __put_css_set+0x24/0x126 [<ffffffff8142b51c>] ? _raw_write_lock_irq+0xb/0x1b [<ffffffff810557af>] ? exit_ptrace+0x8e/0xfd [<ffffffff8104ec1e>] do_exit+0x77/0x677 [<ffffffff8142e00f>] ? do_page_fault+0x268/0x277 [<ffffffff8104f2b9>] sys_exit_group+0x0/0x16 [<ffffffff8104f2cb>] sys_exit_group+0x12/0x16 [<ffffffff81009b42>] system_call_fastpath+0x16/0x1b _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Pasi Kärkkäinen
2010-May-14 05:46 UTC
Re: [Xen-devel] need known working 64-bit upstream kernel config file for PV guest
On Thu, May 13, 2010 at 04:03:00PM -0700, Dan Magenheimer wrote:> > > Any working x86_64 config file for any upstream kernel >=2.6.32 > > > would be fine. I have a 32-bit config file that works fine for > > 2.6.27 > > > up to 2.6.32 (and I think for 2.6.33... trying again now). > > > > > > > 2.6.33.0 is known to crash as Xen PV guest, I think 2.6.33.2 fixes > > that. > > So better use 2.6.33.latest update. > > > > You could grab the Fedora 13 2.6.33.3 kernel and get the .config from > > it? > > That F13 kernel works for me. > > Tried a 2.6.33.4 upstream kernel with the F13 > config-2.6.33.3-79.rc13.x86_64 and it also fails. > > So there must be a kernel bug that the F13 kernel patches fix? > > Here''s the output... note still a system call failing... >It looks like the root= parameter is wrong?> (early) Linux version 2.6.33.4 (root@dmagenhe-nsvpn-dhcp-141-144-22-17.vpn.oracle.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Thu May 13 18:10:25 EDT 2010 > (early) Command line: ro root=LABEL=/ earlyprintk=xen console=hvc0 > Red Hat nash version 5.1.19.6 starting > Mounting proc filesystem > Mounting sysfs filesystem > Creating /dev > Creating initial device nodes > Setting up hotplug. > Creating block device nodes. > Loading xen-blkfront.ko module > blkfront: xvda: barriers enabled > xvda: xvda1 xvda2 xvda3 > Loading xen-blkfront.ko module > insmod: error inserting ''/lib/xen-blkfront.ko'': -1 File exists > Loading xen-blkfront.ko module > insmod: error inserting ''/lib/xen-blkfront.ko'': -1 File exists > Loading xen-blkfront.ko module > insmod: error inserting ''/lib/xen-blkfront.ko'': -1 File exists > Scanning and configuring dmraid supported devices > Trying to resume from LABEL=SWAP-VM > Unable to access resume device (LABEL=SWAP-VM) > Creating root device. > Mounting root filesystem. > mount: could not find filesystem ''/dev/root''Here..> Setting up other filesystems. > Setting up new root fs > setuproot: moving /dev failed: No such file or directory > no fstab.sys, mounting internal defaults > setuproot: error mounting /proc: No such file or directory > setuproot: error mounting /sys: No such file or directory > Switching to new root and running init. > unmounting old /dev > unmounting old /proc > unmounting old /sys > switchroot: mount failed: No such file or directory > Kernel panic - not syncing: Attempted to kill init!-- Pasi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Dan Magenheimer
2010-May-14 13:16 UTC
RE: [Xen-devel] need known working 64-bit upstream kernel config file for PV guest
> > Here''s the output... note still a system call failing... > > It looks like the root= parameter is wrong?Hi Pasi -- I''ll need to dig up a 32-bit FC13 config file to prove it, but my experience with other 32-bit and 64-bit config files predicts that the same process for 32-bit will work fine (with the same root= parameter). But I hadn''t noticed the problem in the earlier part of the boot before you pointed it out. I''m beginning to suspect that there is a bug in 64-bit upstream kernels that is exposed in nash (the lightweight shell that gets bundled in a RedHat initrd). If so, it will be very difficult to track down, though clearly it is fixed in one of the many Fedora kernel patches. So I am going to give up on 64-bit upstream kernels for now so I can get some other things done and come back to this later. Thanks, Dan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Dan Magenheimer
2010-May-14 16:58 UTC
RE: [Xen-devel] need known working 64-bit upstream kernel config file for PV guest
> But I hadn''t noticed the problem in the earlier part of > the boot before you pointed it out. > I''m beginning to suspect that there is a bug in 64-bit > upstream kernels that is exposed in nash (the lightweight > shell that gets bundled in a RedHat initrd). If so, > it will be very difficult to track down, though clearly > it is fixed in one of the many Fedora kernel patches. > So I am going to give up on 64-bit upstream kernels > for now so I can get some other things done and > come back to this later.To add one more very interesting piece of data, booting a 2.6.32 32-bit kernel.org kernel in a PV guest works, while booting a 2.6.32.13 32-bit kernel.org kernel doesn''t... same config file! And the failure looks very similar to the one I''ve been seeing on several x86_64 kernel.org kernels. I''ll confirm that I can reproduce this, and then post as a separate thread. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Pasi Kärkkäinen
2010-May-14 17:42 UTC
Re: [Xen-devel] need known working 64-bit upstream kernel config file for PV guest
On Fri, May 14, 2010 at 09:58:47AM -0700, Dan Magenheimer wrote:> > But I hadn''t noticed the problem in the earlier part of > > the boot before you pointed it out. > > I''m beginning to suspect that there is a bug in 64-bit > > upstream kernels that is exposed in nash (the lightweight > > shell that gets bundled in a RedHat initrd). If so, > > it will be very difficult to track down, though clearly > > it is fixed in one of the many Fedora kernel patches. > > So I am going to give up on 64-bit upstream kernels > > for now so I can get some other things done and > > come back to this later. > > To add one more very interesting piece of data, > booting a 2.6.32 32-bit kernel.org kernel in a PV > guest works, while booting a 2.6.32.13 32-bit > kernel.org kernel doesn''t... same config file! > And the failure looks very similar to the one > I''ve been seeing on several x86_64 kernel.org > kernels. > > I''ll confirm that I can reproduce this, and then > post as a separate thread. >I tried both 32bit and 64bit Linux 2.6.32.9 upstream kernel.org kernels, and they worked ok for me. That was actually save/restore testing, which also worked OK, for both uni-vcpu and multi-vcpu guests. Dom0 was EL5. -- Pasi _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Dan Magenheimer
2010-May-14 18:10 UTC
RE: [Xen-devel] need known working 64-bit upstream kernel config file for PV guest
> > To add one more very interesting piece of data, > > booting a 2.6.32 32-bit kernel.org kernel in a PV > > guest works, while booting a 2.6.32.13 32-bit > > kernel.org kernel doesn''t... same config file! > > And the failure looks very similar to the one > > I''ve been seeing on several x86_64 kernel.org > > kernels. > > > > I''ll confirm that I can reproduce this, and then > > post as a separate thread. > > > > I tried both 32bit and 64bit Linux 2.6.32.9 upstream > kernel.org kernels, and they worked ok for me. > > That was actually save/restore testing, which also worked OK, > for both uni-vcpu and multi-vcpu guests. > > Dom0 was EL5.Well... I couldn''t reproduce it and, as a result, I discovered another difference: 2.6.32 x86 fails also (as well as 2.6.32.9 x86), when I am booting it in EL5u5. But it succeeds in EL5u4! Perhaps there is a different nash version between the two? Sorry if this appears terribly disorganized and disjointed. I am trying to use two machines with not quite identical environments so that I can get two other things done during long kernel compiles. As a result, I''ve missed some seemingly irrelevant details that are turning out to be relevant. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Dan Magenheimer
2010-May-21 22:15 UTC
RE: [Xen-devel] need known working 64-bit upstream kernel config file for PV guest
Just thought I''d follow up on this. I''m still not positive but it appears that the problem I''m seeing is a kernel-build process problem. It appears that on some kernels, if I build with: make -j4; make -j4 modules modules_install; make install then the resultant kernel (often) fails and if I build with: make; make modules; make modules_install; make install then the resultant kernel boots. I reproduced this, but don''t really have time to track down what the failure is. I usually build in a VNC console window and don''t redirect output of make so it''s possible I''ve gotten error messages that I''ve missed/ignored. Maybe it occurs only with Xen modules because otherwise I''m sure some Linux developer would have seen it. Since the symptom is that the kernel fails looking for a root disk using exactly the same parameters and same disk that work fine for a different kernel, it may have something to do with xenblk. Weird but apparently true. And now that I have 2.6.34 working in both 32-bit and 64-bit PV, I really have to quit looking into this now. Dan> -----Original Message----- > From: Dan Magenheimer > Sent: Friday, May 14, 2010 12:11 PM > To: Pasi Kärkkäinen > Cc: xen-devel@lists.xensource.com > Subject: RE: [Xen-devel] need known working 64-bit upstream kernel > config file for PV guest > > > > To add one more very interesting piece of data, > > > booting a 2.6.32 32-bit kernel.org kernel in a PV > > > guest works, while booting a 2.6.32.13 32-bit > > > kernel.org kernel doesn''t... same config file! > > > And the failure looks very similar to the one > > > I''ve been seeing on several x86_64 kernel.org > > > kernels. > > > > > > I''ll confirm that I can reproduce this, and then > > > post as a separate thread. > > > > > > > I tried both 32bit and 64bit Linux 2.6.32.9 upstream > > kernel.org kernels, and they worked ok for me. > > > > That was actually save/restore testing, which also worked OK, > > for both uni-vcpu and multi-vcpu guests. > > > > Dom0 was EL5. > > Well... I couldn''t reproduce it and, as a result, I discovered > another difference: 2.6.32 x86 fails also (as well as 2.6.32.9 > x86), when I am booting it in EL5u5. But it succeeds in EL5u4! > Perhaps there is a different nash version between the two? > > Sorry if this appears terribly disorganized and disjointed. > I am trying to use two machines with not quite identical > environments so that I can get two other things done during > long kernel compiles. As a result, I''ve missed some seemingly > irrelevant details that are turning out to be relevant._______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Jackson
2010-May-24 10:27 UTC
RE: [Xen-devel] need known working 64-bit upstream kernel config file for PV guest
Dan Magenheimer writes ("RE: [Xen-devel] need known working 64-bit upstream
kernel config file for PV guest"):> make -j4; make -j4 modules modules_install; make install
> then the resultant kernel (often) fails and if I build with:
> make; make modules; make modules_install; make install
> then the resultant kernel boots.
If you''re right it will be due to a race in the makefiles.
> I reproduced this, but don''t really have time to track down what
> the failure is. [...]
Since it is a race it will be difficult for other people to
reproduce. The make output ought to make it clear though.
> I usually build in a VNC console window and don''t
> redirect output of make so it''s possible I''ve gotten
error messages
> that I''ve missed/ignored.
Try, starting with a clean tree, running this
(make -j4; make -j4 modules modules_install; make install) 2>&1 | tee
../log
and check that the resulting kernel does fail, and then provide us
with "log".
> Weird but apparently true. And now that I have 2.6.34 working
> in both 32-bit and 64-bit PV, I really have to quit looking into
> this now.
Then it is unlikely to get fixed any time soon.
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel