Xen devel - Dec 2009 - I cannot get any message from domU by console.

I had installed xen 3.4.2 & pvops 2.6.31.6 dom0 kernel

and, I was trying to boot domU kernel as same as dom0

distribution is ubuntu 9.10(karmic)-server, i386

I confirmed kernel configuration several time.

$> xm create ubuntu1.config -c
Using config file "./ubuntu1.config".
Started domain ubuntu1 (id=4)
<----------------------------------------->
   ( this is space)

that was all. no other message was appeared.

$> xm list
Name                                        ID   Mem VCPUs      State
Time(s)
Domain-0                                     0   508     1     r-----
36.5
ubuntu1                                      4   512     1     r-----
83.5

$>ifconfig -a
eth0
..............

vif4.0
...............

$> xm block-list 4
2049   0    0     1      -1     -1    /local/domain/0/backend/vbd/4/2049
2050   0    0     1      -1     -1    /local/domain/0/backend/vbd/4/2050

 $> xm console 4

==> I think that domU booting is good, but I cannot get the any console
message

<configuration file-ubuntu1.config>
kernel = "/boot/vmlinuz-2.6.31.6-xenU"
ramdisk = "/boot/initrd.img-2.6.31.6-xenU"
memory = 512
name = "ubuntu1"
vif = [ ''bridge=xenbr0'' ]
ip = "192.168.0.101"
netmask= "255.255.255.0"
gateway = "192.168.0.1"
disk = [
''file:/root/xen/domainU/ubuntu1/ubuntu1.img,sda1,w'',''file:/root/xen/domainU/ubuntu1/ubuntu1_swap.img,sda2,w''
]
root = "/dev/sda1 ro"
extra = "console=hvc0"

I also tried to hvc0 console by googling

</etc/init/hvc0.conf>
start on stopped rc RUNLEVEL=[2345]
stop on runlevel [!2345]

respawn
exec /sbin/getty -8 38400 hvc0

what is problem?? first, I am wondering whether domU''s booting is good
or
bad
and if good, why not console message??

<xend.log>
[2009-12-21 23:14:33 905] DEBUG (XendDomainInfo:92)
XendDomainInfo.create([''vm'', [''name'',
''ubuntu1''], [''memory'', 512],
[''vcpus'',
1], [''on_xend_start'', ''ignore''],
[''on_xend_stop'', ''ignore''],
[''image'',
[''linux'', [''kernel'',
''/boot/vmlinuz-2.6.31.6-xenU''], [''ramdisk'',
''/boot/initrd.img-2.6.31.6-xenU''], [''ip'',
''192.168.0.101:127
.0.255.255:192.168.0.1:255.255.255.0:ubuntu1:eth0:off''],
[''root'', ''/dev/sda1
ro''], [''videoram'', 4], [''args'',
''console=hvc0'']]], [''s3_integrity'', 1],
[''device'', [''vbd'',
[''uname'',
''file:/root/xen/domainU/ubuntu1/ubuntu1.img''],
[''dev'', ''sda1''], [''mode'',
''w'']]], [''device'', [''vbd'',
[''uname'',
''file:/root/xen/domainU/ubuntu1/ubuntu1_swap.img''],
[''dev'', ''sda2''],
[''mode'', ''w'']]],
[''device'', [''vif'',
[''bridge'', ''xenbr0'']]]])[2009-12-21
23:14:33 905] DEBUG (XendDomainInfo:2304) XendDomainInfo.constructDomain
[2009-12-21 23:14:33 905] DEBUG (balloon:166) Balloon: 1518304 KiB free;
need 4096; done.[2009-12-21 23:14:33 905] DEBUG (XendDomain:453) Adding
Domain: 4
[2009-12-21 23:14:33 905] DEBUG (XendDomainInfo:2505)
XendDomainInfo.initDomain: 4 256
[2009-12-21 23:14:33 905] DEBUG (XendDomainInfo:2529)
_initDomain:shadow_memory=0x0, memory_static_max=0x20000000,
memory_static_min=0x0.
[2009-12-21 23:14:33 905] DEBUG (balloon:166) Balloon: 1518304 KiB free;
need 526336; done.[2009-12-21 23:14:33 905] INFO (image:173) buildDomain
os=linux dom=4 vcpus=1[2009-12-21 23:14:33 905] DEBUG (image:663)
domid          = 4 [2009-12-21 23:14:33 905] DEBUG (image:664)
memsize        = 512 [2009-12-21 23:14:33 905] DEBUG (image:665)
image          = /boot/vmlinuz-2.6.31.6-xenU[2009-12-21 23:14:33 905] DEBUG
(image:666) store_evtchn   = 1 [2009-12-21 23:14:33 905] DEBUG (image:667)
console_evtchn = 2
[2009-12-21 23:14:33 905] DEBUG (image:668) cmdline        = root=/dev/sda1
ro ip=192.168.0.101:127.0.255.255:192.168.0.1:255.255.255.0:ubuntu1:eth0:off
console=hvc0[2009-12-21 23:14:33 905] DEBUG (image:669) ramdisk       
/boot/initrd.img-2.6.31.6-xenU
[2009-12-21 23:14:33 905] DEBUG (image:670) vcpus          = 1 [2009-12-21
23:14:33 905] DEBUG (image:671) features       =   [2009-12-21 23:14:33 905]
DEBUG (image:672) flags          = 0
[2009-12-21 23:14:36 905] INFO (XendDomainInfo:2168) createDevice: vbd :
{''uuid'':
''4132c3cc-33fb-35ef-a369-58e27865d8ce'',
''bootable'': 1, ''driver'':
''paravirtualised'', ''dev'':
''sda1'', ''uname'':
''file:/root/xen/domainU/ubuntu1/ubuntu1.img'',
''mode'': ''w''}
[2009-12-21 23:14:36 905] DEBUG (DevController:95) DevController: writing
{''virtual-device'': ''2049'',
''device-type'': ''disk'',
''protocol'': ''x86_32-abi'',
''backend-id'': ''0'',
''state'': ''1'', ''backend'':
''/local/domain/0/backend/vbd/4/2049''} to
/local/domain/4/device/vbd/2049.
[2009-12-21 23:14:36 905] DEBUG (DevController:97) DevController: writing
{''domain'': ''ubuntu1'',
''frontend'':
''/local/domain/4/device/vbd/2049'', ''uuid'':
''4132c3cc-33fb-35ef-a369-58e27865d8ce'',
''bootable'': ''1'', ''dev'':
''sda1'',
''state'': ''1'', ''params'':
''/root/xen/domainU/ubuntu1/ubuntu1.img'',
''mode'':
''w'', ''online'': ''1'',
''frontend-id'': ''4'',
''type'': ''file''} to
/local/domain/0/backend/vbd/4/2049.
[2009-12-21 23:14:36 905] INFO (XendDomainInfo:2168) createDevice: vbd :
{''uuid'':
''57151504-2045-33e1-e506-03ab4ac07d99'',
''bootable'': 0, ''driver'':
''paravirtualised'', ''dev'':
''sda2'', ''uname'':
''file:/root/xen/domainU/ubuntu1/ubuntu1_swap.img'',
''mode'': ''w''}[2009-12-21
23:14:36 905] DEBUG (DevController:95) DevController: writing
{''virtual-device'': ''2050'',
''device-type'': ''disk'',
''protocol'': ''x86_32-abi'',
''backend-id'': ''0'',
''state'': ''1'', ''backend'':
''/local/domain/0/backend/vbd/4/2050''} to
/local/domain/4/device/vbd/2050.[2009-12-21 23:14:36 905] DEBUG
(DevController:97) DevController: writing {''domain'':
''ubuntu1'', ''frontend'':
''/local/domain/4/device/vbd/2050'', ''uuid'':
''57151504-2045-33e1-e506-03ab4ac07d99'',
''bootable'': ''0'', ''dev'':
''sda2'',
''state'': ''1'', ''params'':
''/root/xen/domainU/ubuntu1/ubuntu1_swap.img'',
''mode'': ''w'', ''online'':
''1'', ''frontend-id'': ''4'',
''type'': ''file''} to
/local/domain/0/backend/vbd/4/2050.[2009-12-21 23:14:36 905] INFO
(XendDomainInfo:2168) createDevice: vif : {''bridge'':
''xenbr0'', ''mac'':
''00:16:3e:6a:c6:f7'', ''uuid'':
''3f3c2378-c7ed-bfbd-8f76-3c99c4dc87c5''}[2009-12-21 23:14:36
905] DEBUG
(DevController:95) DevController: writing {''mac'':
''00:16:3e:6a:c6:f7'',
''handle'': ''0'', ''protocol'':
''x86_32-abi'', ''backend-id'':
''0'', ''state'': ''1'',
''backend'':
''/local/domain/0/backend/vif/4/0''} to
/local/domain/4/device/vif/0.[2009-12-21 23:14:36 905] DEBUG
(DevController:97) DevController: writing {''bridge'':
''xenbr0'', ''domain'':
''ubuntu1'', ''handle'': ''0'',
''uuid'':
''3f3c2378-c7ed-bfbd-8f76-3c99c4dc87c5'',
''script'': ''/etc/xen/scripts/vif-bridge'',
''mac'': ''00:16:3e:6a:c6:f7'',
''frontend-id'': ''4'',
''state'': ''1'', ''online'':
''1'', ''frontend'':
''/local/domain/4/device/vif/0''} to
/local/domain/0/backend/vif/4/0.
[2009-12-21 23:14:36 905] DEBUG (XendDomainInfo:3060) Storing VM details:
{''on_xend_stop'': ''ignore'',
''shadow_memory'': ''0'',
''uuid'':
''9fe69105-492f-fbc2-fe46-da4800e5c1c0'',
''on_reboot'': ''restart'',
''start_time'': ''1261404876.18'',
''on_poweroff'': ''destroy'',
''bootloader_args'':
'''', ''on_xend_start'':
''ignore'', ''on_crash'':
''restart'', ''xend/restart_count'':
''0'', ''vcpus'': ''1'',
''vcpu_avail'': ''1'',
''bootloader'': '''', ''image'':
"(linux
(kernel /boot/vmlinuz-2.6.31.6-xenU) (ramdisk
/boot/initrd.img-2.6.31.6-xenU) (args ''root=/dev/sda1 ro
ip=192.168.0.101:127.0.255.255:192.168.0.1:255.255.255.0:ubuntu1:eth0:off
console=hvc0'') (videoram 4) (notes (HV_START_LOW 4118806528) (FEATURES
''!writable_page_tables|pae_pgdir_above_4gb'') (VIRT_BASE
3221225472)
(GUEST_VERSION 2.6) (PADDR_OFFSET 0) (GUEST_OS linux) (HYPERCALL_PAGE
3222282240) (LOADER generic) (SUSPEND_CANCEL 1) (PAE_MODE yes) (ENTRY
3229274112) (XEN_VERSION xen-3.0)))", ''name'':
''ubuntu1''}
[2009-12-21 23:14:36 905] DEBUG (XendDomainInfo:1622) Storing domain
details: {''console/ring-ref'': ''413280'',
''image/entry'': ''3229274112'',
''console/port'': ''2'',
''store/ring-ref'': ''413281'',
''image/loader'': ''generic'',
''vm'':
''/vm/9fe69105-492f-fbc2-fe46-da4800e5c1c0'',
''control/platform-feature-multiprocessor-suspend'':
''1'',
''image/hv-start-low'': ''4118806528'',
''image/guest-os'': ''linux'',
''image/virt-base'': ''3221225472'',
''memory/target'': ''524288'',
''image/guest-version'': ''2.6'',
''image/pae-mode'': ''yes'',
''console/limit'':
''1048576'', ''image/paddr-offset'':
''0'', ''image/hypercall-page'':
''3222282240'',
''image/suspend-cancel'': ''1'',
''cpu/0/availability'': ''online'',
''image/features/pae-pgdir-above-4gb'': ''1'',
''image/features/writable-page-tables'': ''0'',
''console/type'': ''xenconsoled'',
''name'': ''ubuntu1'',
''domid'': ''4'',
''image/xen-version'': ''xen-3.0'',
''store/port'': ''1''}
[2009-12-21 23:14:36 905] DEBUG (DevController:95) DevController: writing
{''protocol'': ''x86_32-abi'',
''state'': ''1'',
''backend-id'': ''0'',
''backend'':
''/local/domain/0/backend/console/4/0''} to
/local/domain/4/device/console/0.
[2009-12-21 23:14:36 905] DEBUG (DevController:97) DevController: writing
{''domain'': ''ubuntu1'',
''frontend'':
''/local/domain/4/device/console/0'',
''uuid'':
''10c3246b-6962-4009-bc20-a567574efbea'',
''frontend-id'': ''4'',
''state'':
''1'', ''location'': ''2'',
''online'': ''1'', ''protocol'':
''vt100''} to
/local/domain/0/backend/console/4/0.
[2009-12-21 23:14:36 905] DEBUG (XendDomainInfo:1709)
XendDomainInfo.handleShutdownWatch
[2009-12-21 23:14:36 905] DEBUG (DevController:139) Waiting for devices vif.
[2009-12-21 23:14:36 905] DEBUG (DevController:144) Waiting for 0.
[2009-12-21 23:14:36 905] DEBUG (DevController:629) hotplugStatusCallback
/local/domain/0/backend/vif/4/0/hotplug-status.
[2009-12-21 23:14:36 905] DEBUG (DevController:643) hotplugStatusCallback 1.
[2009-12-21 23:14:36 905] DEBUG (DevController:139) Waiting for devices
vkbd.
[2009-12-21 23:14:36 905] DEBUG (DevController:139) Waiting for devices
ioports.
[2009-12-21 23:14:36 905] DEBUG (DevController:139) Waiting for devices tap.
[2009-12-21 23:14:36 905] DEBUG (DevController:139) Waiting for devices
console.
[2009-12-21 23:14:36 905] DEBUG (DevController:144) Waiting for 0.
[2009-12-21 23:14:36 905] DEBUG (DevController:139) Waiting for devices
vscsi.
[2009-12-21 23:14:36 905] DEBUG (DevController:139) Waiting for devices vbd.
[2009-12-21 23:14:36 905] DEBUG (DevController:144) Waiting for 2049.
[2009-12-21 23:14:36 905] DEBUG (DevController:629) hotplugStatusCallback
/local/domain/0/backend/vbd/4/2049/hotplug-status.
[2009-12-21 23:14:36 905] DEBUG (DevController:643) hotplugStatusCallback 1.
[2009-12-21 23:14:36 905] DEBUG (DevController:144) Waiting for 2050.
[2009-12-21 23:14:36 905] DEBUG (DevController:629) hotplugStatusCallback
/local/domain/0/backend/vbd/4/2050/hotplug-status.
[2009-12-21 23:14:37 905] DEBUG (DevController:629) hotplugStatusCallback
/local/domain/0/backend/vbd/4/2050/hotplug-status.
[2009-12-21 23:14:37 905] DEBUG (DevController:643) hotplugStatusCallback 1.
[2009-12-21 23:14:37 905] DEBUG (DevController:139) Waiting for devices irq.
[2009-12-21 23:14:37 905] DEBUG (DevController:139) Waiting for devices vfb.
[2009-12-21 23:14:37 905] DEBUG (DevController:139) Waiting for devices pci.
[2009-12-21 23:14:37 905] DEBUG (DevController:139) Waiting for devices
vtpm.
[2009-12-21 23:14:37 905] INFO (XendDomain:1182) Domain ubuntu1 (4)
unpaused.

<domain-builder-ng.log>
xc_dom_allocate: cmdline="root=/dev/sda1 ro ip=192.168.0.101:127
.0.255.255:192.168.0.1:255.255.255.0:ubuntu1:eth0:off console=hvc0",
features=""
xc_dom_kernel_file: filename="/boot/vmlinuz-2.6.31.6-xenU"
xc_dom_malloc_filemap    : 3896 kB
xc_dom_ramdisk_file: filename="/boot/initrd.img-2.6.31.6-xenU"
xc_dom_malloc_filemap    : 52 MB
xc_dom_boot_xen_init: ver 3.4, caps xen-3.0-x86_32p hvm-3.0-x86_32
hvm-3.0-x86_32p
xc_dom_parse_image: called
xc_dom_find_loader: trying ELF-generic loader ... failed
xc_dom_find_loader: trying Linux bzImage loader ... xc_dom_malloc
: 8207 kB
xc_dom_do_gunzip: unzip ok, 0x3c72d0 -> 0x803e28
OK
elf_parse_binary: phdr: paddr=0x100000 memsz=0x65b000
elf_parse_binary: phdr: paddr=0x75b000 memsz=0x2b0000
elf_parse_binary: memory: 0x100000 -> 0xa0b000
elf_xen_parse_note: GUEST_OS = "linux"
elf_xen_parse_note: GUEST_VERSION = "2.6"
elf_xen_parse_note: XEN_VERSION = "xen-3.0"
elf_xen_parse_note: VIRT_BASE = 0xc0000000
elf_xen_parse_note: ENTRY = 0xc07ad000
elf_xen_parse_note: HYPERCALL_PAGE = 0xc0102000
elf_xen_parse_note: FEATURES =
"!writable_page_tables|pae_pgdir_above_4gb"
elf_xen_parse_note: PAE_MODE = "yes"
elf_xen_parse_note: LOADER = "generic"
elf_xen_parse_note: unknown xen elf note (0xd)
elf_xen_parse_note: SUSPEND_CANCEL = 0x1
elf_xen_parse_note: HV_START_LOW = 0xf5800000
elf_xen_parse_note: PADDR_OFFSET = 0x0
elf_xen_addr_calc_check: addresses:
    virt_base        = 0xc0000000
    elf_paddr_offset = 0x0
    virt_offset      = 0xc0000000
    virt_kstart      = 0xc0100000
    virt_kend        = 0xc0a0b000
    virt_entry       = 0xc07ad000
    p2m_base         = 0xffffffffffffffff
xc_dom_parse_elf_kernel: xen-3.0-x86_32p: 0xc0100000 -> 0xc0a0b000
xc_dom_mem_init: mem 512 MB, pages 0x20000 pages, 4k each
xc_dom_mem_init: 0x20000 pages
xc_dom_boot_mem_init: called
x86_compat: guest xen-3.0-x86_32p, address size 32
xc_dom_malloc            : 512 kB
xc_dom_build_image: called
xc_dom_alloc_segment:   kernel       : 0xc0100000 -> 0xc0a0b000  (pfn 0x100
+ 0x90b pages)
xc_dom_pfn_to_ptr: domU mapping: pfn 0x100+0x90b at 0xae741000
elf_load_binary: phdr 0 at 0x0xae741000 -> 0x0xaed9c000
elf_load_binary: phdr 1 at 0x0xaed9c000 -> 0x0xaee80000
xc_dom_alloc_segment:   ramdisk      : 0xc0a0b000 -> 0xc872a000  (pfn 0xa0b
+ 0x7d1f pages)
xc_dom_malloc            : 750 kB
xc_dom_pfn_to_ptr: domU mapping: pfn 0xa0b+0x7d1f at 0xa6966000
xc_dom_do_gunzip: unzip ok, 0x34c64d1 -> 0x7d1e210
xc_dom_alloc_segment:   phys2mach    : 0xc872a000 -> 0xc87aa000  (pfn 0x872a
+ 0x80 pages)
xc_dom_pfn_to_ptr: domU mapping: pfn 0x872a+0x80 at 0xa68e6000
xc_dom_alloc_page   :   start info   : 0xc87aa000 (pfn 0x87aa)
xc_dom_alloc_page   :   xenstore     : 0xc87ab000 (pfn 0x87ab)
xc_dom_alloc_page   :   console      : 0xc87ac000 (pfn 0x87ac)
nr_page_tables: 0x00000000ffffffff/32: 0x0000000000000000 ->
0xffffffffffffffff, 1 table(s)
nr_page_tables: 0x000000003fffffff/30: 0x00000000c0000000 ->
0x00000000ffffffff, 1 table(s)
nr_page_tables: 0x00000000001fffff/21: 0x00000000c0000000 ->
0x00000000c8bfffff, 70 table(s)
xc_dom_alloc_segment:   page tables  : 0xc87ad000 -> 0xc87f5000  (pfn 0x87ad
+ 0x48 pages)
xc_dom_pfn_to_ptr: domU mapping: pfn 0x87ad+0x48 at 0xa689e000
xc_dom_alloc_page   :   boot stack   : 0xc87f5000 (pfn 0x87f5)
xc_dom_build_image  : virt_alloc_end : 0xc87f6000
xc_dom_build_image  : virt_pgtab_end : 0xc8c00000
xc_dom_boot_image: called
arch_setup_bootearly: doing nothing
xc_dom_compat_check: supported guest type: xen-3.0-x86_32p <= matches
xc_dom_compat_check: supported guest type: hvm-3.0-x86_32
xc_dom_compat_check: supported guest type: hvm-3.0-x86_32p
xc_dom_update_guest_p2m: dst 32bit, pages 0x20000
clear_page: pfn 0x87ac, mfn 0x64e60
clear_page: pfn 0x87ab, mfn 0x64e61
xc_dom_pfn_to_ptr: domU mapping: pfn 0x87aa+0x1 at 0xa689d000
start_info_x86_32: called
setup_hypercall_page: vaddr=0xc0102000 pfn=0x102
domain builder memory footprint
   allocated
      malloc             : 9531 kB
      anon mmap          : 0 bytes
   mapped
      file mmap          : 56 MB
      domU mmap          : 134 MB
arch_setup_bootlate: shared_info: pfn 0x0, mfn 0x1ea
shared_info_x86_32: called
vcpu_x86_32: called
vcpu_x86_32: cr3: pfn 0x87ad mfn 0x64e5f
launch_vm: called, ctxt=0xb41642e8
xc_dom_release: called


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, Dec 21, 2009 at 11:30:51PM +0900, ?????? wrote:
>    I had installed xen 3.4.2 & pvops 2.6.31.6 dom0 kernel
> 
>    and, I was trying to boot domU kernel as same as dom0
> 
>    distribution is ubuntu 9.10(karmic)-server, i386
> 
>    I confirmed kernel configuration several time.
> 
>    $> xm create ubuntu1.config -c
>    Using config file "./ubuntu1.config".
>    Started domain ubuntu1 (id=4)
>    <----------------------------------------->
>       ( this is space)
> 
>    that was all. no other message was appeared.
> 
Did you try with "console=hvc0 earlyprintk=xen" ?

-- Pasi


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, Dec 22, 2009 at 01:02:06PM +0900, ?????? wrote:
>    thank you for response.
> 
>    I tried your advise, and my pvops kernel was panic.
> 
>    It is the log
> 
>    #> xm create ubuntu1.config -c
>    Using config file "./ubuntu1.config".
>    Started domain ubuntu1 (id=5)
>                                 (early) [    0.000000] Reserving virtual
>    address space above 0xf5800000
>    (early) [    0.000000] Initializing cgroup subsys cpuset
>    (early) [    0.000000] Initializing cgroup subsys cpu
>    (early) [    0.000000] Linux version 2.6.31.6 ([1]root@parallels) (gcc
>    version 4.4.1 (Ubuntu 4.4.1-4ubuntu8) ) #2 SMP Mon Dec 21 16:44:19 KST
>    2009
>    (early) [    0.000000] KERNEL supported cpus:
>    (early) [    0.000000]   Intel GenuineIntel
>    (early) [    0.000000]   AMD AuthenticAMD
>    (early) [    0.000000]   NSC Geode by NSC
>    (early) [    0.000000]   Cyrix CyrixInstead
>    (early) [    0.000000]   Centaur CentaurHauls
>    (early) [    0.000000]   Transmeta GenuineTMx86
>    (early) [    0.000000]   Transmeta TransmetaCPU
>    (early) [    0.000000]   UMC UMC UMC UMC
>    (early) [    0.000000] ACPI in unprivileged domain disabled
>    (early) [    0.000000] released 0 pages of unused memory
>    (early) [    0.000000] BIOS-provided physical RAM map:
>    (early) [    0.000000]  Xen: 0000000000000000 - 00000000000a0000
(usable)
>    (early) [    0.000000]  Xen: 00000000000a0000 - 0000000000100000
>    (reserved)
>    (early) [    0.000000]  Xen: 0000000000100000 - 0000000020000000
(usable)
>    (early) [    0.000000] console [xenboot0] enabled
>    (early) [    0.000000] DMI not present or invalid.
>    (early) [    0.000000] last_pfn = 0x20000 max_arch_pfn = 0x1000000
>    (early) [    0.000000] Scanning 1 areas for low memory corruption
>    (early) [    0.000000] modified physical RAM map:
>    (early) [    0.000000]  modified: 0000000000000000 - 0000000000002000
>    (early) (usable)(early)
>    (early) [    0.000000]  modified: 0000000000002000 - 0000000000006000
>    (early) (reserved)(early)
>    (early) [    0.000000]  modified: 0000000000006000 - 00000000000a0000
>    (early) (usable)(early)
>    (early) [    0.000000]  modified: 00000000000a0000 - 0000000000100000
>    (early) (reserved)(early)
>    (early) [    0.000000]  modified: 0000000000100000 - 0000000020000000
>    (early) (usable)(early)
>    (early) [    0.000000] init_memory_mapping:
>    0000000000000000-0000000020000000
>    (early) [    0.000000] NX (Execute Disable) protection: active
>    (early) [    0.000000] RAMDISK: 00a0b000 - 0872a000
>    (early) [    0.000000] 0MB HIGHMEM available.
>    (early) [    0.000000] 512MB LOWMEM available.
>    (early) [    0.000000]   mapped low ram: 0 - 20000000
>    (early) [    0.000000]   low ram: 0 - 20000000
>    (early) [    0.000000]   node 0 low ram: 00000000 - 20000000
>    (early) [    0.000000]   node 0 bootmap 00007000 - 0000b000
>    (early) [    0.000000] (9 early reservations) ==> bootmem [0000000000
-
>    0020000000]
>    (early) [    0.000000]   #0 [0000000000 - 0000001000]   BIOS data
>    page(early)  ==> [0000000000 - 0000001000]
>    (early) [    0.000000]   #1 [00087ad000 - 00087f5000]   XEN
>    PAGETABLES(early)  ==> [00087ad000 - 00087f5000]
>    (early) [    0.000000]   #2 [0000001000 - 0000002000]    EX
>    TRAMPOLINE(early)  ==> [0000001000 - 0000002000]
>    (early) [    0.000000]   #3 [0000006000 - 0000007000]
>    TRAMPOLINE(early)  ==> [0000006000 - 0000007000]
>    (early) [    0.000000]   #4 [0000100000 - 00008e5c60]    TEXT DATA
>    BSS(early)  ==> [0000100000 - 00008e5c60]
>    (early) [    0.000000]   #5 [0000a0b000 - 000872a000]
>    RAMDISK(early)  ==> [0000a0b000 - 000872a000]
>    (early) [    0.000000]   #6 [000872a000 - 00087ad000]   XEN START
>    INFO(early)  ==> [000872a000 - 00087ad000]
>    (early) [    0.000000]   #7 [00008e6000 - 000099d000]
>    PGTABLE(early)  ==> [00008e6000 - 000099d000]
>    (early) [    0.000000]   #8 [0000007000 - 000000b000]
>    BOOTMAP(early)  ==> [0000007000 - 000000b000]
>    (early) [    0.000000] Zone PFN ranges:
>    (early) [    0.000000]   DMA      0x00000000 -> 0x00001000
>    (early) [    0.000000]   Normal   0x00001000 -> 0x00020000
>    (early) [    0.000000]   HighMem  0x00020000 -> 0x00020000
>    (early) [    0.000000] Movable zone start PFN for each node
>    (early) [    0.000000] early_node_map[3] active PFN ranges
>    (early) [    0.000000]     0: 0x00000000 -> 0x00000002
>    (early) [    0.000000]     0: 0x00000006 -> 0x000000a0
>    (early) [    0.000000]     0: 0x00000100 -> 0x00020000
>    (early) [    0.000000] Using APIC driver default
>    (early) [    0.000000] SMP: Allowing 1 CPUs, 0 hotplug CPUs
>    (early) [    0.000000] Local APIC disabled by BIOS -- you can enable it
>    with "lapic"
>    (early) [    0.000000] APIC: disable apic facility
>    (early) [    0.000000] PM: Registered nosave memory: 0000000000002000 -
>    0000000000006000
>    (early) [    0.000000] PM: Registered nosave memory: 00000000000a0000 -
>    0000000000100000
>    (early) [    0.000000] Allocating PCI resources starting at 20000000
(gap:
>    20000000:e0000000)
>    (early) [    0.000000] NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:1
>    nr_node_ids:1
>    (early) [    0.000000] PERCPU: Allocated 10 4k pages, static data 39132
>    bytes
>    (early) [    0.000000] Xen: using vcpu_info placement
>    (early) [    0.000000] Built 1 zonelists in Zone order, mobility
grouping
>    on.  Total pages: 129948
>    (early) [    0.000000] Kernel command line: root=/dev/sda1 ro
>   
ip=192.168.0.101:127.0.255.255:192.168.0.1:255.255.255.0:ubuntu1:eth0:off
>    console=hvc0 earlyprintk=xen
>    (early) [    0.000000] PID hash table entries: 2048 (order: 11, 8192
>    bytes)
>    (early) [    0.000000] Dentry cache hash table entries: 65536 (order: 6,
>    262144 bytes)
>    (early) [    0.000000] Inode-cache hash table entries: 32768 (order: 5,
>    131072 bytes)
>    (early) [    0.000000] Enabling fast FPU save and restore... (early)
done.
>    (early) [    0.000000] Enabling unmasked SIMD FPU exception support...
>    (early) done.
>    (early) [    0.000000] Initializing CPU#0
>    (early) [    0.000000] allocated 2621440 bytes of page_cgroup
>    (early) [    0.000000] please try
''cgroup_disable=memory'' option if you
>    don''t want memory cgroups
>    (early) [    0.000000] PCI-DMA: Using Xen software bounce buffering for
IO
>    (Xen-SWIOTLB)
>    (early) [    0.000000] Kernel panic - not syncing:
>    <3>xen_create_contiguous_region failed
> 
Did you try cgroup_disable=memory kernel option? 

-- Pasi


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, 2009-12-22 at 08:42 +0000, Pasi Kärkkäinen
wrote:
> >    (early) [    0.000000] PCI-DMA: Using Xen software bounce
> buffering for IO
> >    (Xen-SWIOTLB)
> >    (early) [    0.000000] Kernel panic - not syncing:
> >    <3>xen_create_contiguous_region failed
> > 
> 
> Did you try cgroup_disable=memory kernel option? 
I think this is more likely to relate to the failure to initialise
swiotlb due to the failure of xen_create_contiguous_region, leading to a
panic. IIRC the guest won''t be privileged enough to make this call
unless it has PCI passthrough enabled.

In a pre-pvops kernel the swiotlb would be inactive in a domU unless
specifically requested on the command line but in Jeremy''s xen/master
branch it looks like it is unconditionally enabled for dom0 and domU as
a consequence of:
        commit 4bc6b1a9dd5d7447af8c3d27c1449f73f5f764ec
        Author: root <root@localhost.localdomain>
        Date:   Thu Nov 5 16:33:10 2009 -0500
        
            Enable Xen-SWIOTLB if running in [non-]privileged and disable the
Xen-IOMMU if an IOMMU is detected.

On a related note I don''t think the xen-swiotlb code should panic() if
it cannot allocate the contiguous region (at least in domU). Chances are
if it hasn''t got privilege to make the fixups it isn''t going
to have any
devices.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

xen-devel-bounces@lists.xensource.com wrote on 12/22/2009 04:20:15 AM:
> On Tue, 2009-12-22 at 08:42 +0000, Pasi Kärkkäinen wrote:
> > >    (early) [    0.000000] PCI-DMA: Using Xen software bounce
> > buffering for IO
> > >    (Xen-SWIOTLB)
> > >    (early) [    0.000000] Kernel panic - not syncing:
> > >    <3>xen_create_contiguous_region failed
> > > 
> > 
> > Did you try cgroup_disable=memory kernel option? 
> 
> I think this is more likely to relate to the failure to initialise
> swiotlb due to the failure of xen_create_contiguous_region, leading to a
> panic. IIRC the guest won''t be privileged enough to make this call
> unless it has PCI passthrough enabled.
> 
> In a pre-pvops kernel the swiotlb would be inactive in a domU unless
> specifically requested on the command line but in Jeremy''s
xen/master
> branch it looks like it is unconditionally enabled for dom0 and domU as
> a consequence of:
>         commit 4bc6b1a9dd5d7447af8c3d27c1449f73f5f764ec
>         Author: root <root@localhost.localdomain>
>         Date:   Thu Nov 5 16:33:10 2009 -0500
> 
>             Enable Xen-SWIOTLB if running in [non-]privileged and 
> disable the Xen-IOMMU if an IOMMU is detected.
> 
> On a related note I don''t think the xen-swiotlb code should
panic() if
> it cannot allocate the contiguous region (at least in domU). Chances are
> if it hasn''t got privilege to make the fixups it isn''t
going to have any
> devices.
I ran into the same problem yesterday, traced it back to that exact 
commit.  Reverting it makes my domUs work, but feels like the wrong thing 
to do.  How do I make sure xen_create_contiguous_region is privileged 
enough?  I''ve got the PCI frontend and backend  built in to my kernel 
(used the x86 fedora .config from the pv_ops wiki).

-Mike


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

xen-devel-bounces@lists.xensource.com wrote on 12/22/2009 08:04:45 AM:
> xen-devel-bounces@lists.xensource.com wrote on 12/22/2009 04:20:15 AM:
> 
> > On Tue, 2009-12-22 at 08:42 +0000, Pasi Kärkkäinen wrote:
> > > >    (early) [    0.000000] PCI-DMA: Using Xen software bounce
> > > buffering for IO
> > > >    (Xen-SWIOTLB)
> > > >    (early) [    0.000000] Kernel panic - not syncing:
> > > >    <3>xen_create_contiguous_region failed
> > > > 
> > > 
> > > Did you try cgroup_disable=memory kernel option? 
> > 
> > I think this is more likely to relate to the failure to initialise
> > swiotlb due to the failure of xen_create_contiguous_region, leading to
a
> > panic. IIRC the guest won''t be privileged enough to make this
call
> > unless it has PCI passthrough enabled.
> > 
> > In a pre-pvops kernel the swiotlb would be inactive in a domU unless
> > specifically requested on the command line but in Jeremy''s
xen/master
> > branch it looks like it is unconditionally enabled for dom0 and domU 
as
> > a consequence of:
> >         commit 4bc6b1a9dd5d7447af8c3d27c1449f73f5f764ec
> >         Author: root <root@localhost.localdomain>
> >         Date:   Thu Nov 5 16:33:10 2009 -0500
> > 
> >             Enable Xen-SWIOTLB if running in [non-]privileged and 
> > disable the Xen-IOMMU if an IOMMU is detected.
> > 
> > On a related note I don''t think the xen-swiotlb code should
panic() if
> > it cannot allocate the contiguous region (at least in domU). Chances 
are
> > if it hasn''t got privilege to make the fixups it
isn''t going to have
any
> > devices.
> 
> I ran into the same problem yesterday, traced it back to that exact 
> commit.  Reverting it makes my domUs work, but feels like the wrong 
thing 
> to do.  How do I make sure xen_create_contiguous_region is privileged 
> enough?  I''ve got the PCI frontend and backend  built in to my
kernel
> (used the x86 fedora .config from the pv_ops wiki).
When you said to enable PCI passthrough, did you mean by setting 
CONFIG_XEN_PCIDEV_BACKEND_PASS as apposed to the newer 
CONFIG_XEN_PCIDEV_BACKEND_VPCI?  Is Passthrough going to replace VPCI as 
the preferred PCI Backend Mode for Xen 4.0?

-Mike


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Thanks, everyone.

I built pvops domU kernel with privileged guest and normal guest, So,
XENMEM_exchange call in xen_create_contiguous_region does not work well, and
went through kernel panic.
I built vanila kernel as domU, just with paravirtulized normal guest, and It
works well.

Umm..I am xen and linux kernel newbie. but I think that privileged guest
with non-privileged guest should work well, which can allow user to use same
kernel domU & dom0.


2009/12/22 Ian Campbell <Ian.Campbell@citrix.com>
> On Tue, 2009-12-22 at 08:42 +0000, Pasi Kärkkäinen wrote:
> > >    (early) [    0.000000] PCI-DMA: Using Xen software bounce
> > buffering for IO
> > >    (Xen-SWIOTLB)
> > >    (early) [    0.000000] Kernel panic - not syncing:
> > >    <3>xen_create_contiguous_region failed
> > >
> >
> > Did you try cgroup_disable=memory kernel option?
>
> I think this is more likely to relate to the failure to initialise
> swiotlb due to the failure of xen_create_contiguous_region, leading to a
> panic. IIRC the guest won''t be privileged enough to make this call
> unless it has PCI passthrough enabled.
>
> In a pre-pvops kernel the swiotlb would be inactive in a domU unless
> specifically requested on the command line but in Jeremy''s
xen/master
> branch it looks like it is unconditionally enabled for dom0 and domU as
> a consequence of:
>        commit 4bc6b1a9dd5d7447af8c3d27c1449f73f5f764ec
>        Author: root <root@localhost.localdomain>
>        Date:   Thu Nov 5 16:33:10 2009 -0500
>
>            Enable Xen-SWIOTLB if running in [non-]privileged and disable
> the Xen-IOMMU if an IOMMU is detected.
>
> On a related note I don''t think the xen-swiotlb code should
panic() if
> it cannot allocate the contiguous region (at least in domU). Chances are
> if it hasn''t got privilege to make the fixups it isn''t
going to have any
> devices.
>
> Ian.
>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, 2009-12-22 at 13:04 +0000, Michael D Labriola
wrote:
> How do I make sure xen_create_contiguous_region is privileged 
> enough? 
By adding a PCI device to your domU config -- that''s obviously just a
workaround though and in general you do not want domUs to be able to tie
down arbitrary amounts of "special" memory.

Later on:
> When you said to enable PCI passthrough, did you mean by setting 
> CONFIG_XEN_PCIDEV_BACKEND_PASS as apposed to the newer 
> CONFIG_XEN_PCIDEV_BACKEND_VPCI?
This is independent of CONFIG_XEN_PCIDEV_BACKEND_*.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> >    (early) [    0.000000] Kernel panic - not syncing:
> >    <3>xen_create_contiguous_region failed
This implies you did not have enough DMA32 memory in the hypervisor for the
guest.
Meaning at least 64MB. When you launch your guest, run ''xm
info'' and see what you have. The latest
from xen-unstable tries to have _at least_ that amount of memory available
for the guest. Here is what I see:

free_memory            : 3938
node_to_cpu            : node0:0-1
node_to_memory         : node0:3938
node_to_dma32_mem      : node0:3514

The node_to_dma32_mem needs to have at least 64.

Or you can add ''iommu=off'' to your boot arguments that should
disable the SWIOTLB.
You only need SWIOTLB if you do PCI pass through.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, Dec 22, 2009 at 09:20:15AM +0000, Ian Campbell
wrote:
> On Tue, 2009-12-22 at 08:42 +0000, Pasi Kärkkäinen wrote:
> > >    (early) [    0.000000] PCI-DMA: Using Xen software bounce
> > buffering for IO
> > >    (Xen-SWIOTLB)
> > >    (early) [    0.000000] Kernel panic - not syncing:
> > >    <3>xen_create_contiguous_region failed
> > > 
> > 
> > Did you try cgroup_disable=memory kernel option? 
> 
> I think this is more likely to relate to the failure to initialise
> swiotlb due to the failure of xen_create_contiguous_region, leading to a
> panic. IIRC the guest won''t be privileged enough to make this call
> unless it has PCI passthrough enabled.
Not so. You can still make the call even if you don''t have PCI
passthrough
enabled. The up call is just to replace MFNs.
> 
> In a pre-pvops kernel the swiotlb would be inactive in a domU unless
> specifically requested on the command line but in Jeremy''s
xen/master
> branch it looks like it is unconditionally enabled for dom0 and domU as
> a consequence of:
>         commit 4bc6b1a9dd5d7447af8c3d27c1449f73f5f764ec
>         Author: root <root@localhost.localdomain>
>         Date:   Thu Nov 5 16:33:10 2009 -0500
>         
>             Enable Xen-SWIOTLB if running in [non-]privileged and disable
the Xen-IOMMU if an IOMMU is detected.
> 
> On a related note I don''t think the xen-swiotlb code should
panic() if
> it cannot allocate the contiguous region (at least in domU). Chances are
I concur. It should also give the error code. I am working on re-writting that
code since the upstream now has some form of registration/de-registration that
we could take advantage off.
> if it hasn''t got privilege to make the fixups it isn''t
going to have any
> devices.
> 
> Ian.
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, 2009-12-22 at 14:35 +0000, Konrad Rzeszutek Wilk
wrote:
> On Tue, Dec 22, 2009 at 09:20:15AM +0000, Ian Campbell wrote:
> > On Tue, 2009-12-22 at 08:42 +0000, Pasi Kärkkäinen wrote:
> > > >    (early) [    0.000000] PCI-DMA: Using Xen software bounce
> > > buffering for IO
> > > >    (Xen-SWIOTLB)
> > > >    (early) [    0.000000] Kernel panic - not syncing:
> > > >    <3>xen_create_contiguous_region failed
> > > > 
> > > 
> > > Did you try cgroup_disable=memory kernel option? 
> > 
> > I think this is more likely to relate to the failure to initialise
> > swiotlb due to the failure of xen_create_contiguous_region, leading to
a
> > panic. IIRC the guest won''t be privileged enough to make this
call
> > unless it has PCI passthrough enabled.
> 
> Not so. You can still make the call even if you don''t have PCI
passthrough
> enabled. The up call is just to replace MFNs.
I thought it used to be that you could only (successfully) make order>0
increase_reservation or mem_exchange hypercalls if you had I/O
privileges? Has this changed?

Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, 2009-12-22 at 14:32 +0000, Konrad Rzeszutek Wilk
wrote:
> > >    (early) [    0.000000] Kernel panic - not syncing:
> > >    <3>xen_create_contiguous_region failed
> 
> This implies you did not have enough DMA32 memory in the hypervisor for the
guest.
> Meaning at least 64MB.
Does this mean that every domU is allocating 64M for swiotlb by default?
That seems like an awful lot of wastage when the vast majority of guests
have no I/O device and therefore no need for swiotlb, especially given
that this is memory from a limited pool (OK, so 64G isn''t that low a
limit).

If we are unable to automatically determine early enough whether a guest
needs swiotlb or not (which I suspect is the case without additional
tools support) then I think the old-style behaviour of only enabling
swiotlb in domU on explicit request matches the common case better.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

xen-devel-bounces@lists.xensource.com wrote on 12/22/2009 09:32:03 AM:
> > >    (early) [    0.000000] Kernel panic - not syncing:
> > >    <3>xen_create_contiguous_region failed
> 
> This implies you did not have enough DMA32 memory in the hypervisor 
> for the guest.
> Meaning at least 64MB. When you launch your guest, run ''xm
info'' and
> see what you have. The latest
> from xen-unstable tries to have _at least_ that amount of memory 
available
> for the guest. Here is what I see:
> 
> free_memory            : 3938
> node_to_cpu            : node0:0-1
> node_to_memory         : node0:3938
> node_to_dma32_mem      : node0:3514
> 
> The node_to_dma32_mem needs to have at least 64.
I''m using Xen 3.4.2.  xm info doesn''t have a
''node_to_dma32_mem'' entry at
all.  I wasn''t setting dom0 memory constraints via grub before, so I
tried
adding dom0_mem=512M.  That makes free_memory=3533 on my system.  Still 
get the same panic.
> Or you can add ''iommu=off'' to your boot arguments that
should
> disable the SWIOTLB.
Is iommu=off for dom0 or domU boot args.  If it''s dom0, is it for the 
hypervisor or the kernel?  I''ve tried adding it to all 3 places, but I 
still get the same panic message.

-Mike

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, Dec 22, 2009 at 02:58:47PM +0000, Ian Campbell
wrote:
> On Tue, 2009-12-22 at 14:32 +0000, Konrad Rzeszutek Wilk wrote:
> > > >    (early) [    0.000000] Kernel panic - not syncing:
> > > >    <3>xen_create_contiguous_region failed
> > 
> > This implies you did not have enough DMA32 memory in the hypervisor
for the guest.
> > Meaning at least 64MB.
> 
> Does this mean that every domU is allocating 64M for swiotlb by default?
Unfortunately yes. We can remove the:

ommit 4bc6b1a9dd5d7447af8c3d27c1449f73f5f764ec
Author: root <root@localhost.localdomain>
Date:   Thu Nov 5 16:33:10 2009 -0500

    Enable Xen-SWIOTLB if running in [non-]privileged and disable the Xen-IOMMU
if an IOMMU is detected.
    
    For PCI passthrough to work correctly, we need the Xen-SWIOTLB.
    Otherwise PCI devices in the non-privileged domains might not be
    able to do DMA.
    
And revert to the old-style behaviour.
> That seems like an awful lot of wastage when the vast majority of guests
> have no I/O device and therefore no need for swiotlb, especially given
> that this is memory from a limited pool (OK, so 64G isn''t that low
a
> limit).
> 
> If we are unable to automatically determine early enough whether a guest
> needs swiotlb or not (which I suspect is the case without additional
That is possible. Which is why I am working on the new SWIOTLB that
can be turned on _after_ xen-pcifront has been activated.
> tools support) then I think the old-style behaviour of only enabling
> swiotlb in domU on explicit request matches the common case better.
> 
> Ian.
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, Dec 22, 2009 at 10:14:34AM -0500, Michael D Labriola
wrote:
> xen-devel-bounces@lists.xensource.com wrote on 12/22/2009 09:32:03 AM:
> 
> > > >    (early) [    0.000000] Kernel panic - not syncing:
> > > >    <3>xen_create_contiguous_region failed
> > 
> > This implies you did not have enough DMA32 memory in the hypervisor 
> > for the guest.
> > Meaning at least 64MB. When you launch your guest, run ''xm
info'' and
> > see what you have. The latest
> > from xen-unstable tries to have _at least_ that amount of memory 
> available
> > for the guest. Here is what I see:
> > 
> > free_memory            : 3938
> > node_to_cpu            : node0:0-1
> > node_to_memory         : node0:3938
> > node_to_dma32_mem      : node0:3514
> > 
> > The node_to_dma32_mem needs to have at least 64.
> 
> I''m using Xen 3.4.2.  xm info doesn''t have a
''node_to_dma32_mem'' entry at
> all.  I wasn''t setting dom0 memory constraints via grub before, so
I tried
> adding dom0_mem=512M.  That makes free_memory=3533 on my system.  Still 
> get the same panic.
All of your memory is below 4GB, so you should not be hitting the problem
I thought you had (not enought memory under 4GB).

Maybe Xen 3.4.2 logic is much different than 4.0 when it comes to exchanging
MFNs
and Ian''s is right. Under 4.0 it I''ve no trouble exchanging
the MFNs even without
any PCI devices being passed in.

Can you try this patch to help narrow down the problem:

diff --git a/arch/x86/xen/pci-swiotlb.c b/arch/x86/xen/pci-swiotlb.c
index ecdbfe2..37fd0e3 100644
--- a/arch/x86/xen/pci-swiotlb.c
+++ b/arch/x86/xen/pci-swiotlb.c
@@ -960,7 +960,7 @@ xen_swiotlb_fixup(void *buf, size_t size, unsigned long
nslabs)
 				dma_bits);
 		} while (rc && dma_bits++ < max_dma_bits);
 		if (rc)
-			panic(KERN_ERR "xen_create_contiguous_region failed\n");
+			panic(KERN_ERR "xen_create_contiguous_region failed: rc: %d\n",
rc);
 
 		i += slabs;
 	} while(i < nslabs);

And also ensure that your command line for Xen has: console=com1,vga
guest_loglvl=all

And tell me what environemtn you are using? Is the hypervisor a 64-bit
while dom0 is 32bit or 64bit? Is DomU 32bit or 64bit?
> 
> > Or you can add ''iommu=off'' to your boot arguments
that should
> > disable the SWIOTLB.
> 
> Is iommu=off for dom0 or domU boot args.  If it''s dom0, is it for
the
> hypervisor or the kernel?  I''ve tried adding it to all 3 places,
but I
> still get the same panic message.
Well that is a surprise. My recollection of the code vs what is there
confirms that the code logic is a bit brute. It turns on SWIOTLB
irregardless of any flags you pass in. So nevermind the
''iommu=off''
attempt.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> I thought it used to be that you could only (successfully) make order>0
> increase_reservation or mem_exchange hypercalls if you had I/O
> privileges? Has this changed?
I am looking at the 3.4 code I am not seeing any I/O privileges check.

I did not even know that those existed actually - could you give me an idea
when was the last time you saw it?


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, 2009-12-22 at 15:47 +0000, Konrad Rzeszutek Wilk
wrote:
> > I thought it used to be that you could only (successfully) make
order>0
> > increase_reservation or mem_exchange hypercalls if you had I/O
> > privileges? Has this changed?
> 
> I am looking at the 3.4 code I am not seeing any I/O privileges check.
> 
> I did not even know that those existed actually - could you give me an idea
> when was the last time you saw it?
In xen-unstable multipage_allocation_permitted is called from
memory_exchange() and increase_reservation() and is defined as 
#define multipage_allocation_permitted(d, order)        \
    (((order) <= 9) || /* allow 2MB superpages */       \
     !rangeset_is_empty((d)->iomem_caps) ||             \
     !rangeset_is_empty((d)->arch.ioport_caps))

The ((order) <= 9) || is new and isn''t present in the 3.4 tree,
previously you would have had to add a passthrough device to cause one
of the other rangesets to become non-empty. 

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, Dec 22, 2009 at 04:09:36PM +0000, Ian Campbell
wrote:
> On Tue, 2009-12-22 at 15:47 +0000, Konrad Rzeszutek Wilk wrote:
> > > I thought it used to be that you could only (successfully) make
order>0
> > > increase_reservation or mem_exchange hypercalls if you had I/O
> > > privileges? Has this changed?
> > 
> > I am looking at the 3.4 code I am not seeing any I/O privileges check.
> > 
> > I did not even know that those existed actually - could you give me an
idea
> > when was the last time you saw it?
> 
> In xen-unstable multipage_allocation_permitted is called from
> memory_exchange() and increase_reservation() and is defined as 
> #define multipage_allocation_permitted(d, order)        \
>     (((order) <= 9) || /* allow 2MB superpages */       \
>      !rangeset_is_empty((d)->iomem_caps) ||             \
>      !rangeset_is_empty((d)->arch.ioport_caps))
> 
> The ((order) <= 9) || is new and isn''t present in the 3.4 tree,
> previously you would have had to add a passthrough device to cause one
> of the other rangesets to become non-empty. 
AAh, and since the exchange of memory is done in small chunks we pass
underneath the radar even if did not have the passthrough device set.

Ian, thanks for finding the culprit. Let me roll out a patch that
will do what was done in the past (ie, turn SWIOTLB for DomU only
when iommu=soft was passed in) as a fix.

Later (next year) I can propose the more dynamic SWIOTLB mechanism
that should make this more automatic.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> Maybe Xen 3.4.2 logic is much different than 4.0 when it comes to 
> exchanging MFNs
> and Ian''s is right. Under 4.0 it I''ve no trouble
exchanging the MFNs
> even without
> any PCI devices being passed in.
> 
> Can you try this patch to help narrow down the problem:
> 
> diff --git a/arch/x86/xen/pci-swiotlb.c b/arch/x86/xen/pci-swiotlb.c
> index ecdbfe2..37fd0e3 100644
> --- a/arch/x86/xen/pci-swiotlb.c
> +++ b/arch/x86/xen/pci-swiotlb.c
> @@ -960,7 +960,7 @@ xen_swiotlb_fixup(void *buf, size_t size, 
> unsigned long nslabs)
>              dma_bits);
>        } while (rc && dma_bits++ < max_dma_bits);
>        if (rc)
> -         panic(KERN_ERR "xen_create_contiguous_region
failed\n");
> +         panic(KERN_ERR "xen_create_contiguous_region failed: rc: 
%d\n", rc);
> 
>        i += slabs;
>     } while(i < nslabs);
> 
> And also ensure that your command line for Xen has: console=com1,vga
> guest_loglvl=all
Made that change and tried again.  rc was -12.  Do you need the whole 
dump?  I''m not really sure how to capture it...

> And tell me what environemtn you are using? Is the hypervisor a 64-bit
> while dom0 is 32bit or 64bit? Is DomU 32bit or 64bit?
This particular test is on a Dell Precision M4400 w/ 4GB of RAM. 
Everything is 32-bit.  Not using HVM.

-Mike

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

xen-devel-bounces@lists.xensource.com wrote on 12/22/2009 11:44:23 AM:
> > Maybe Xen 3.4.2 logic is much different than 4.0 when it comes to 
> > exchanging MFNs
> > and Ian''s is right. Under 4.0 it I''ve no trouble
exchanging the MFNs
> > even without
> > any PCI devices being passed in.
> > 
> > Can you try this patch to help narrow down the problem:
> > 
> > diff --git a/arch/x86/xen/pci-swiotlb.c b/arch/x86/xen/pci-swiotlb.c
> > index ecdbfe2..37fd0e3 100644
> > --- a/arch/x86/xen/pci-swiotlb.c
> > +++ b/arch/x86/xen/pci-swiotlb.c
> > @@ -960,7 +960,7 @@ xen_swiotlb_fixup(void *buf, size_t size, 
> > unsigned long nslabs)
> >              dma_bits);
> >        } while (rc && dma_bits++ < max_dma_bits);
> >        if (rc)
> > -         panic(KERN_ERR "xen_create_contiguous_region
failed\n");
> > +         panic(KERN_ERR "xen_create_contiguous_region failed:
rc:
> %d\n", rc);
> > 
> >        i += slabs;
> >     } while(i < nslabs);
> > 
> > And also ensure that your command line for Xen has: console=com1,vga
> > guest_loglvl=all
> 
> Made that change and tried again.  rc was -12.  Do you need the whole 
> dump?  I''m not really sure how to capture it...
Trying to attach the domU console output such that Lotus Notes doesn''t 
garble it all... bare with me.  ;-)





_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, Dec 22, 2009 at 11:19:19AM -0500, Konrad Rzeszutek Wilk
wrote:
> On Tue, Dec 22, 2009 at 04:09:36PM +0000, Ian Campbell wrote:
> > On Tue, 2009-12-22 at 15:47 +0000, Konrad Rzeszutek Wilk wrote:
> > > > I thought it used to be that you could only (successfully)
make order>0
> > > > increase_reservation or mem_exchange hypercalls if you had
I/O
> > > > privileges? Has this changed?
> > > 
> > > I am looking at the 3.4 code I am not seeing any I/O privileges
check.
> > > 
> > > I did not even know that those existed actually - could you give
me an idea
> > > when was the last time you saw it?
> > 
> > In xen-unstable multipage_allocation_permitted is called from
> > memory_exchange() and increase_reservation() and is defined as 
> > #define multipage_allocation_permitted(d, order)        \
> >     (((order) <= 9) || /* allow 2MB superpages */       \
> >      !rangeset_is_empty((d)->iomem_caps) ||             \
> >      !rangeset_is_empty((d)->arch.ioport_caps))
> > 
> > The ((order) <= 9) || is new and isn''t present in the 3.4
tree,
> > previously you would have had to add a passthrough device to cause one
> > of the other rangesets to become non-empty. 
> 
> AAh, and since the exchange of memory is done in small chunks we pass
> underneath the radar even if did not have the passthrough device set.
> 
> Ian, thanks for finding the culprit. Let me roll out a patch that
> will do what was done in the past (ie, turn SWIOTLB for DomU only
> when iommu=soft was passed in) as a fix.
Jeremy,

Please consider this patch to the PV-OPS tree. 

[xen/swiotlb] Enable Xen-SWIOTLB only if running in privileged domain or if in
non-privileged with iommu=soft.

Previous to this patch we would unconditionally enable Xen-SWIOTLB
if running in PV context. That does not work with Xen 3.4.2 as it has a
security check to disable exchanging of MFNs if no PCI devices have been
passed through. In 4.0 there is an additional check to allow 2MB super-pages
- we accidentally circumvented that by exchanging pages under the 2MB chunk size
even without any PCI devices passed through.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

diff --git a/arch/x86/xen/pci-swiotlb.c b/arch/x86/xen/pci-swiotlb.c
index ecdbfe2..7fdfccc 100644
--- a/arch/x86/xen/pci-swiotlb.c
+++ b/arch/x86/xen/pci-swiotlb.c
@@ -960,7 +960,7 @@ xen_swiotlb_fixup(void *buf, size_t size, unsigned long
nslabs)
 				dma_bits);
 		} while (rc && dma_bits++ < max_dma_bits);
 		if (rc)
-			panic(KERN_ERR "xen_create_contiguous_region failed\n");
+			panic(KERN_ERR "xen_create_contiguous_region failed: rc: %d\n",
rc);
 
 		i += slabs;
 	} while(i < nslabs);
@@ -984,7 +984,16 @@ static struct dma_map_ops xen_swiotlb_dma_ops = {
 
 void __init xen_swiotlb_init(void)
 {
-	if (xen_domain()) {
+        int use_swiotlb = 0;
+
+        if (xen_initial_domain())
+                use_swiotlb = 1;
+
+        /* For PV guest, only if iommu=soft is passed in. */
+        if (xen_pv_domain() && !xen_initial_domain() &&
swiotlb)
+               use_swiotlb = 1;
+ 
+	if (use_swiotlb) {
 		printk(KERN_INFO "PCI-DMA: Using Xen software bounce buffering for IO
(Xen-SWIOTLB)\n");
 		xen_swiotlb_init_with_default_size(64 * (1<<20));	/* default to 64MB */
 		dma_ops = &xen_swiotlb_dma_ops;

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> 
> Trying to attach the domU console output such that Lotus Notes
doesn''t
Oooh Lotus Notes, eh? Good luck :-)
> garble it all... bare with me.  ;-)
Got another patch on the way that should solve your problem. Please try it out.

Hmm, you are using Lotus Notes, I think it eats up spaces in patches, so
here it is as an attachment.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote on 12/22/2009 
12:00:26 PM:
> > 
> > Trying to attach the domU console output such that Lotus Notes
doesn''t
> 
> Oooh Lotus Notes, eh? Good luck :-)
Oh, you have no idea.  Not only is my productivity crippled by Lotus 
Notes, our PCs are all locked down with GuardianEdge''s
"transparent"
encryption... which only has Windows client software...  which means if I 
copy that patch onto my thumb drive (or even burn it to a CD!) it ends up 
encrypted.  And then I have to take it to a windows box on our internal 
network so I can decrypt it and ftp it over to my Linux box...  sigh.

> > garble it all... bare with me.  ;-)
> 
> Got another patch on the way that should solve your problem. Please 
> try it out.
> 
> Hmm, you are using Lotus Notes, I think it eats up spaces in patches, so
> here it is as an attachment.
> 
> [attachment "a.patch" deleted by Michael D Labriola/EB/GDYN] 
Will try it.  So domUs not requiring PCI passthrough will boot with 
SWIOTLB disabled and in order to get PCI passthrough working we''ll have
to
add ''iommu=soft'' to the domU kernel args in addition to the
normal
pci=[XX:YY:ZZ] line?  Just want to make sure I''m following all this. 
Thanks!

-Mike

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> > Got another patch on the way that should solve your problem. Please 
> > try it out.
> > 
> > Hmm, you are using Lotus Notes, I think it eats up spaces in patches, 
so
> > here it is as an attachment.
> > 
> > [attachment "a.patch" deleted by Michael D Labriola/EB/GDYN]
> 
> Will try it.  So domUs not requiring PCI passthrough will boot with 
> SWIOTLB disabled and in order to get PCI passthrough working we''ll
have
to 
> add ''iommu=soft'' to the domU kernel args in addition to
the normal
> pci=[XX:YY:ZZ] line?  Just want to make sure I''m following all
this.
> Thanks!
Well, this seems to work for the first case (domU not needing PCI 
passthrough).  Haven''t tried passing anything through yet.  On a more 
general pv_ops domU note, though... I no longer get a login prompt after 
watching the system boot with ''xm create -c''.  Is there
something in my
domU config that needs to change?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, 2009-12-22 at 16:59 +0000, Konrad Rzeszutek Wilk
wrote:
> On Tue, Dec 22, 2009 at 11:19:19AM -0500, Konrad Rzeszutek Wilk wrote:
> > On Tue, Dec 22, 2009 at 04:09:36PM +0000, Ian Campbell wrote:
> > > On Tue, 2009-12-22 at 15:47 +0000, Konrad Rzeszutek Wilk wrote:
> > > > > I thought it used to be that you could only
(successfully) make order>0
> > > > > increase_reservation or mem_exchange hypercalls if you
had I/O
> > > > > privileges? Has this changed?
> > > > 
> > > > I am looking at the 3.4 code I am not seeing any I/O
privileges check.
> > > > 
> > > > I did not even know that those existed actually - could you
give me an idea
> > > > when was the last time you saw it?
> > > 
> > > In xen-unstable multipage_allocation_permitted is called from
> > > memory_exchange() and increase_reservation() and is defined as 
> > > #define multipage_allocation_permitted(d, order)        \
> > >     (((order) <= 9) || /* allow 2MB superpages */       \
> > >      !rangeset_is_empty((d)->iomem_caps) ||             \
> > >      !rangeset_is_empty((d)->arch.ioport_caps))
> > > 
> > > The ((order) <= 9) || is new and isn''t present in the
3.4 tree,
> > > previously you would have had to add a passthrough device to
cause one
> > > of the other rangesets to become non-empty. 
> > 
> > AAh, and since the exchange of memory is done in small chunks we pass
> > underneath the radar even if did not have the passthrough device set.
> > 
> > Ian, thanks for finding the culprit. Let me roll out a patch that
> > will do what was done in the past (ie, turn SWIOTLB for DomU only
> > when iommu=soft was passed in) as a fix.
> 
> Jeremy,
> 
> Please consider this patch to the PV-OPS tree. 
> 
> [xen/swiotlb] Enable Xen-SWIOTLB only if running in privileged domain or if
in non-privileged with iommu=soft.
> 
> Previous to this patch we would unconditionally enable Xen-SWIOTLB
> if running in PV context. That does not work with Xen 3.4.2 as it has a
> security check to disable exchanging of MFNs if no PCI devices have been
> passed through. In 4.0 there is an additional check to allow 2MB
super-pages
> - we accidentally circumvented that by exchanging pages under the 2MB chunk
size
> even without any PCI devices passed through.
> 
> Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> 
> diff --git a/arch/x86/xen/pci-swiotlb.c b/arch/x86/xen/pci-swiotlb.c
> index ecdbfe2..7fdfccc 100644
> --- a/arch/x86/xen/pci-swiotlb.c
> +++ b/arch/x86/xen/pci-swiotlb.c
> @@ -960,7 +960,7 @@ xen_swiotlb_fixup(void *buf, size_t size, unsigned long
nslabs)
>  				dma_bits);
>  		} while (rc && dma_bits++ < max_dma_bits);
>  		if (rc)
> -			panic(KERN_ERR "xen_create_contiguous_region failed\n");
> +			panic(KERN_ERR "xen_create_contiguous_region failed: rc:
%d\n", rc);
>  
>  		i += slabs;
>  	} while(i < nslabs);
> @@ -984,7 +984,16 @@ static struct dma_map_ops xen_swiotlb_dma_ops = {
>  
>  void __init xen_swiotlb_init(void)
>  {
> -	if (xen_domain()) {
> +        int use_swiotlb = 0;
> +
> +        if (xen_initial_domain())
> +                use_swiotlb = 1;
> +
> +        /* For PV guest, only if iommu=soft is passed in. */
> +        if (xen_pv_domain() && !xen_initial_domain() &&
swiotlb)
> +               use_swiotlb = 1;
> + 
> +	if (use_swiotlb) {
How about just 
	  if (xen_pv_domain() && (xen_initial_domain() || swiotlb))
Or depending on how/where swiotlb gets set (i.e. if it is set IFF
xen_pv_domain) simply:
	  if (xen_initial_domain() || swiotlb)

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> >  void __init xen_swiotlb_init(void)
> >  {
> > -	if (xen_domain()) {
> > +        int use_swiotlb = 0;
> > +
> > +        if (xen_initial_domain())
> > +                use_swiotlb = 1;
> > +
> > +        /* For PV guest, only if iommu=soft is passed in. */
> > +        if (xen_pv_domain() && !xen_initial_domain()
&& swiotlb)
> > +               use_swiotlb = 1;
> > + 
> > +	if (use_swiotlb) {
> 
> How about just 
> 	  if (xen_pv_domain() && (xen_initial_domain() || swiotlb))
That would work too. Let me remake the patch as it also has
spaces instead of tabs and fails the checkpatch.pl.
> Or depending on how/where swiotlb gets set (i.e. if it is set IFF
> xen_pv_domain) simply:
> 	  if (xen_initial_domain() || swiotlb)
Can''t do that, as it could on baremetal allocate the Xen-SWIOTLB.
> 
> Ian.
> 
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> Well, this seems to work for the first case (domU not needing PCI 
> passthrough).  Haven''t tried passing anything through yet.  On a
more
> general pv_ops domU note, though... I no longer get a login prompt after 
> watching the system boot with ''xm create -c''.  Is there
something in my
> domU config that needs to change?
console=hvc0

Otherwise the output is going to ''tty0'' which is your VNC
framebuffer.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> > Well, this seems to work for the first case (domU not needing PCI 
> > passthrough).  Haven''t tried passing anything through yet. 
On a more
> > general pv_ops domU note, though... I no longer get a login prompt 
after 
> > watching the system boot with ''xm create -c''.  Is
there something in
my 
> > domU config that needs to change?
> 
> console=hvc0
> 
> Otherwise the output is going to ''tty0'' which is your VNC
framebuffer.
I already had that in the ''extra'' section of my domUs config
file...  is
that not the right place for it?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, Dec 22, 2009 at 01:47:54PM -0500, Michael D Labriola
wrote:
> > > Well, this seems to work for the first case (domU not needing PCI
> > > passthrough).  Haven''t tried passing anything through
yet.  On a more
> > > general pv_ops domU note, though... I no longer get a login
prompt
> after 
> > > watching the system boot with ''xm create -c''. 
Is there something in
> my 
> > > domU config that needs to change?
> > 
> > console=hvc0
> > 
> > Otherwise the output is going to ''tty0'' which is
your VNC framebuffer.
> 
> I already had that in the ''extra'' section of my domUs
config file...  is
> that not the right place for it?
That is the right place. You are sure you don''t have another
''extra'' or ''root''
later on in the config file that would overwrite the first one?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> On Tue, Dec 22, 2009 at 01:47:54PM -0500, Michael D Labriola wrote:
> > > > Well, this seems to work for the first case (domU not
needing PCI
> > > > passthrough).  Haven''t tried passing anything
through yet.  On a
more 
> > > > general pv_ops domU note, though... I no longer get a login
prompt
> > after 
> > > > watching the system boot with ''xm create
-c''.  Is there something
in 
> > my 
> > > > domU config that needs to change?
> > > 
> > > console=hvc0
> > > 
> > > Otherwise the output is going to ''tty0'' which
is your VNC
framebuffer.
> > 
> > I already had that in the ''extra'' section of my
domUs config file...
is 
> > that not the right place for it?
> 
> That is the right place. You are sure you don''t have another
''extra''or
''root''
> later on in the config file that would overwrite the first one?
Yeah.  Last line of my config is:

extra="earlyprintk=xen iommu=soft console=hvc0"

No login for me.  Just one of those days, I guess.

A also can''t seem to pass devices through (unless I''m doing it
wrong).
When trying to pass my soundcard do the domU, it appears to be 
appropriately hidden from dom0 via xen-pciback.hide kernel param 
(/var/log/messages says ''pciback 0000:00:1b.0: seizing
device'') but when I
try to start my domU with iommu=soft and pci=["00:1b.0"] I get a dozen
or
so "BUG: scheduling while atomic" messages w/ call traces.  My domU 
eventually starts up ok, but lspci returns with no output.

I''ve attached /var/log/messages from my dom0 after attempting to create
my
domU.

-Mike



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, 2009-12-22 at 19:50 +0000, Michael D Labriola
wrote:
> 
> Yeah.  Last line of my config is:
> 
> extra="earlyprintk=xen iommu=soft console=hvc0"
> 
> No login for me.  Just one of those days, I guess.
You need a getty on hvc0 defined in /etc/inittab of the guest. The
console= stuff just controls the kernel messages.

Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> A also can''t seem to pass devices through (unless I''m
doing it wrong).
> When trying to pass my soundcard do the domU, it appears to be 
> appropriately hidden from dom0 via xen-pciback.hide kernel param 
> (/var/log/messages says ''pciback 0000:00:1b.0: seizing
device'') but when I
> try to start my domU with iommu=soft and pci=["00:1b.0"] I get a
dozen or
You are doing it right.
> so "BUG: scheduling while atomic" messages w/ call traces.  My
domU
> eventually starts up ok, but lspci returns with no output.
Hmm, I wonder if this patch is in your tree:

commit 1aa61698354ca0582b07eb865e0432a13b459f11
Author: Ian Campbell <ian.campbell@citrix.com>
Date:   Thu Dec 17 14:08:25 2009 +0000

    xen: fix hang on suspend.

and causing this (or maybe the pciback driver is the culprit and
the above patch exposes a bug?).

Try reverting that patch and seeing what happens. I the meantime let
me compile a new Dom0 with the above mentioned commit and see if I get
the failure too.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, Dec 22, 2009 at 03:07:21PM -0500, Konrad Rzeszutek Wilk
wrote:
> > A also can''t seem to pass devices through (unless
I''m doing it wrong).
> > When trying to pass my soundcard do the domU, it appears to be 
> > appropriately hidden from dom0 via xen-pciback.hide kernel param 
> > (/var/log/messages says ''pciback 0000:00:1b.0: seizing
device'') but when I
> > try to start my domU with iommu=soft and pci=["00:1b.0"] I
get a dozen or
> 
> You are doing it right.
> 
> > so "BUG: scheduling while atomic" messages w/ call traces. 
My domU
> > eventually starts up ok, but lspci returns with no output.
> 
> Hmm, I wonder if this patch is in your tree:
> 
> commit 1aa61698354ca0582b07eb865e0432a13b459f11
> Author: Ian Campbell <ian.campbell@citrix.com>
> Date:   Thu Dec 17 14:08:25 2009 +0000
> 
>     xen: fix hang on suspend.
> 
> and causing this (or maybe the pciback driver is the culprit and
> the above patch exposes a bug?).
Bummer. Can''t blame Ian for it :-(

I updated dom0 with that patch and I am not seeing the failure you described.
> 
> Try reverting that patch and seeing what happens. I the meantime let
> me compile a new Dom0 with the above mentioned commit and see if I get
> the failure too.
Instead of that recommendation try enabling debug options. You can do this
the manual way:

diff --git a/drivers/pci/xen-pcifront.c b/drivers/pci/xen-pcifront.c
index cc3b51b..ae1648a 100644
--- a/drivers/pci/xen-pcifront.c
+++ b/drivers/pci/xen-pcifront.c
@@ -30,7 +30,7 @@
 #define INVALID_GRANT_REF (0)
 #define INVALID_EVTCHN    (-1)
 
-
+#define DEBUG 1
 struct pci_bus_entry {
 	struct list_head list;
 	struct pci_bus *bus;
diff --git a/drivers/xen/blkback/blkback.c b/drivers/xen/blkback/blkback.c
index 815c0c6..a871a2c 100644
--- a/drivers/xen/blkback/blkback.c
+++ b/drivers/xen/blkback/blkback.c
@@ -64,7 +64,7 @@ MODULE_PARM_DESC(reqs, "Number of blkback requests to
allocate");
 
 /* Run-time switchable: /sys/module/blkback/parameters/ */
 static unsigned int log_stats = 0;
-static unsigned int debug_lvl = 0;
+static unsigned int debug_lvl = 1;
 module_param(log_stats, int, 0644);
 module_param(debug_lvl, int, 0644);


Or fiddle with the debug_lvl in /sys/ namespace. 

That might shed some light on what is happening. Also do provide the
output of Dom0, DomU and the lspci -vvv in Dom0 please.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ian Campbell <Ian.Campbell@citrix.com> wrote on 12/22/2009 03:03:11 PM:
> On Tue, 2009-12-22 at 19:50 +0000, Michael D Labriola wrote:
> > 
> > Yeah.  Last line of my config is:
> > 
> > extra="earlyprintk=xen iommu=soft console=hvc0"
> > 
> > No login for me.  Just one of those days, I guess.
> 
> You need a getty on hvc0 defined in /etc/inittab of the guest. The
> console= stuff just controls the kernel messages.
Ah, jeez.  I should have known that...  ;-)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote on 12/22/2009 
03:37:09 PM:
> On Tue, Dec 22, 2009 at 03:07:21PM -0500, Konrad Rzeszutek Wilk wrote:
> > > A also can''t seem to pass devices through (unless
I''m doing it
wrong). 
> > > When trying to pass my soundcard do the domU, it appears to be 
> > > appropriately hidden from dom0 via xen-pciback.hide kernel param 
> > > (/var/log/messages says ''pciback 0000:00:1b.0: seizing
device'')
> but when I 
> > > try to start my domU with iommu=soft and
pci=["00:1b.0"] I get
adozen or 
> > 
> > You are doing it right.
> > 
> > > so "BUG: scheduling while atomic" messages w/ call
traces.  My domU
> > > eventually starts up ok, but lspci returns with no output.
> > 
> > Hmm, I wonder if this patch is in your tree:
> > 
> > commit 1aa61698354ca0582b07eb865e0432a13b459f11
> > Author: Ian Campbell <ian.campbell@citrix.com>
> > Date:   Thu Dec 17 14:08:25 2009 +0000
> > 
> >     xen: fix hang on suspend.
> > 
> > and causing this (or maybe the pciback driver is the culprit and
> > the above patch exposes a bug?).
> 
> Bummer. Can''t blame Ian for it :-(
> 
> I updated dom0 with that patch and I am not seeing the failure you 
described.

Yes, I have that commit too.

> 
> > 
> > Try reverting that patch and seeing what happens. I the meantime let
> > me compile a new Dom0 with the above mentioned commit and see if I get
> > the failure too.
> 
> Instead of that recommendation try enabling debug options. You can do 
this
> the manual way:
> 
> diff --git a/drivers/pci/xen-pcifront.c b/drivers/pci/xen-pcifront.c
> index cc3b51b..ae1648a 100644
> --- a/drivers/pci/xen-pcifront.c
> +++ b/drivers/pci/xen-pcifront.c
> @@ -30,7 +30,7 @@
>  #define INVALID_GRANT_REF (0)
>  #define INVALID_EVTCHN    (-1)
> 
> -
> +#define DEBUG 1
>  struct pci_bus_entry {
>     struct list_head list;
>     struct pci_bus *bus;
> diff --git a/drivers/xen/blkback/blkback.c 
b/drivers/xen/blkback/blkback.c
> index 815c0c6..a871a2c 100644
> --- a/drivers/xen/blkback/blkback.c
> +++ b/drivers/xen/blkback/blkback.c
> @@ -64,7 +64,7 @@ MODULE_PARM_DESC(reqs, "Number of blkback requests
> to allocate");
> 
>  /* Run-time switchable: /sys/module/blkback/parameters/ */
>  static unsigned int log_stats = 0;
> -static unsigned int debug_lvl = 0;
> +static unsigned int debug_lvl = 1;
>  module_param(log_stats, int, 0644);
>  module_param(debug_lvl, int, 0644);
> 
> 
> Or fiddle with the debug_lvl in /sys/ namespace. 
> 
> That might shed some light on what is happening. Also do provide the
> output of Dom0, DomU and the lspci -vvv in Dom0 please.
> 
Recompiled with debug turned on as shown.  Attached /var/log/messages from 
dom0 and domU, lscpi output, and my kernel .config.

Any chance it''s PREEMPT related?

-Mike






_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

xen-devel-bounces@lists.xensource.com wrote on 12/22/2009 04:49:25 PM:
> Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote on 12/22/2009 
> 03:37:09 PM:
> 
> > On Tue, Dec 22, 2009 at 03:07:21PM -0500, Konrad Rzeszutek Wilk wrote:
> > > > A also can''t seem to pass devices through (unless
I''m doing it
> wrong). 
> > > > When trying to pass my soundcard do the domU, it appears to
be
> > > > appropriately hidden from dom0 via xen-pciback.hide kernel
param
> > > > (/var/log/messages says ''pciback 0000:00:1b.0:
seizing device'')
> > but when I 
> > > > try to start my domU with iommu=soft and
pci=["00:1b.0"] I get
> adozen or 
> > > 
> > > You are doing it right.
> > > 
> > > > so "BUG: scheduling while atomic" messages w/ call
traces.  My
domU 
> > > > eventually starts up ok, but lspci returns with no output.
> > > 
> > > Hmm, I wonder if this patch is in your tree:
> > > 
> > > commit 1aa61698354ca0582b07eb865e0432a13b459f11
> > > Author: Ian Campbell <ian.campbell@citrix.com>
> > > Date:   Thu Dec 17 14:08:25 2009 +0000
> > > 
> > >     xen: fix hang on suspend.
> > > 
> > > and causing this (or maybe the pciback driver is the culprit and
> > > the above patch exposes a bug?).
> > 
> > Bummer. Can''t blame Ian for it :-(
> > 
> > I updated dom0 with that patch and I am not seeing the failure you 
> described.
> 
> Yes, I have that commit too.
> 
> 
> > 
> > > 
> > > Try reverting that patch and seeing what happens. I the meantime
let
> > > me compile a new Dom0 with the above mentioned commit and see if
I
get
> > > the failure too.
> > 
> > Instead of that recommendation try enabling debug options. You can do 
> this
> > the manual way:
> > 
> > diff --git a/drivers/pci/xen-pcifront.c b/drivers/pci/xen-pcifront.c
> > index cc3b51b..ae1648a 100644
> > --- a/drivers/pci/xen-pcifront.c
> > +++ b/drivers/pci/xen-pcifront.c
> > @@ -30,7 +30,7 @@
> >  #define INVALID_GRANT_REF (0)
> >  #define INVALID_EVTCHN    (-1)
> > 
> > -
> > +#define DEBUG 1
> >  struct pci_bus_entry {
> >     struct list_head list;
> >     struct pci_bus *bus;
> > diff --git a/drivers/xen/blkback/blkback.c 
> b/drivers/xen/blkback/blkback.c
> > index 815c0c6..a871a2c 100644
> > --- a/drivers/xen/blkback/blkback.c
> > +++ b/drivers/xen/blkback/blkback.c
> > @@ -64,7 +64,7 @@ MODULE_PARM_DESC(reqs, "Number of blkback
requests
> > to allocate");
> > 
> >  /* Run-time switchable: /sys/module/blkback/parameters/ */
> >  static unsigned int log_stats = 0;
> > -static unsigned int debug_lvl = 0;
> > +static unsigned int debug_lvl = 1;
> >  module_param(log_stats, int, 0644);
> >  module_param(debug_lvl, int, 0644);
> > 
> > 
> > Or fiddle with the debug_lvl in /sys/ namespace. 
> > 
> > That might shed some light on what is happening. Also do provide the
> > output of Dom0, DomU and the lspci -vvv in Dom0 please.
> > 
> 
> Recompiled with debug turned on as shown.  Attached /var/log/messages 
from 
> dom0 and domU, lscpi output, and my kernel .config.
> 
> Any chance it''s PREEMPT related?
Turning PREEMPT off entirely made all the "scheduling while atomic" 
messages on dom0 go away.

With our witout PREEMPT, I can successfully pass PCI devices to an old 
2.6.18-xen domU as long as the devices are page-aligned.  I only can''t 
pass PCI devices into pv_ops domUs.  Is the PCI frontend code even in 
xen/master?

-Mike

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> Turning PREEMPT off entirely made all the "scheduling while
atomic"
> messages on dom0 go away.
The "scheduling while atomic" messages also go away with
PREEMPT_VOLUNTARY
instead of PREEMPT.
> 
> With our witout PREEMPT, I can successfully pass PCI devices to an old 
> 2.6.18-xen domU as long as the devices are page-aligned.  I only
can''t
> pass PCI devices into pv_ops domUs.  Is the PCI frontend code even in 
> xen/master?
Yikes.  Jeremy merged pcifront after I pulled last...  :-(  My bad.

I can now pass page-aligned PCI devices into a pv_ops domU from my dom0. 
If domU has PREEMPT, it gets tons of "scheduling while atomic"
messages
too.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel