Hi,
I am trying to set up a cluster environment with two xen hosts sharing a
common storage space where the domain-images are stored.
There was already a thread about GFS, Debian and Xen in July, but
unfortunately that couldn''t help me out.
I use a changeset of xen-unstable yesterday:
Mon Nov 14 18:55:40 2005 +0100 7796:ed2ef64fdc33
Dom0 is Debian 3.1 on both machines
The cluster environment consists of:
- RedHat Cluster Suite (cluster-1.01.00)
- device-mapper (1.02.00-cvs (2005-10-16)
- LVM2 (LVM2.2.01.15)
The GFS block device can be accessed and mounted from both hosts at the same
time. Seems to me this works fine.
But when I try to start a domU with its image-file on the GFS my domain
won''t
come up. Exactly the same domain (image) comes up perfectly when I start it
from a local fs, e.g. ext3.
This is my config file:
----------------------------------------------------------------------------------------------------------------
kernel = "/boot/vmlinuz-2.6-xenU"
memory = 64
name = "Gollum"
vif = [ ''mac=00:50:56:00:04:42, bridge=xenbr0'' ]
disk =
[
''file:/cluster/debguest/vm1disk.img,hda1,w'',''file:/cluster/debguest/vm1swap.img,hda2,w''
]
root = "/dev/hda1 ro"
----------------------------------------------------------------------------------------------------------------
This is the console output I get from the domain when I''m trying to
start it:
----------------------------------------------------------------------------------------------------------------
testpc-022:/cluster/debguest# xm create -c debguest.cfg
Using config file "debguest.cfg".
Started domain Gollum
Linux version 2.6.12.6-xenU (root@testpc-022) (gcc version 3.3.5 (Debian
1:3.3.5-13)) #1 SMP Tue Nov 15 09:57:25 CET 2005
BIOS-provided physical RAM map:
Xen: 0000000000000000 - 0000000004000000 (usable)
0MB HIGHMEM available.
72MB LOWMEM available.
IRQ lockup detection disabled
Built 1 zonelists
Kernel command line: root=/dev/hda1 ro
Initializing CPU#0
PID hash table entries: 512 (order: 9, 8192 bytes)
Xen reported: 1525.838 MHz processor.
Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
vmalloc area: c5000000-fb7fe000, maxmem 34000000
Memory: 61012k/73728k available (2256k kernel code, 4360k reserved, 647k data,
136k init, 0k highmem)
Checking if this processor honours the WP bit even in supervisor mode... Ok.
Mount-cache hash table entries: 512
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 256K (64 bytes/line)
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Checking ''hlt'' instruction... disabled
Brought up 1 CPUs
NET: Registered protocol family 16
Brought up 1 CPUs
xen_mem: Initialising balloon driver.
Grant table initialized
SGI XFS with ACLs, security attributes, no debug enabled
SGI XFS Quota Management subsystem
Initializing Cryptographic API
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
loop: loaded (max 8 devices)
Xen virtual console successfully installed as tty1
Event-channel device installed.
Registering block device major 3
netfront: Initialising virtual ethernet driver.
NET: Registered protocol family 2
IP: routing cache hash table of 256 buckets, 4Kbytes
TCP established hash table entries: 4096 (order: 4, 65536 bytes)
TCP bind hash table entries: 4096 (order: 3, 49152 bytes)
TCP: Hash tables configured (established 4096 bind 4096)
NET: Registered protocol family 1
NET: Registered protocol family 17
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.
Freeing unused kernel memory: 136k freed
kjournald starting. Commit interval 5 seconds
INIT: version 2.86 booting
Activating swap.
Adding 524280k swap on /dev/hda2. Priority:-1 extents:1
Checking root file system...
fsck 1.37 (21-Mar-2005)
/dev/hda1: clean, 31054/262144 files, 132802/524288 blocks
end_request: I/O error, dev hda1, sector 4168
Buffer I/O error on device hda1, logical block 521
lost page write due to I/O error on hda1
end_request: I/O error, dev hda1, sector 0
Buffer I/O error on device hda1, logical block 0
lost page write due to I/O error on hda1
EXT3 FS on hda1, internal journal
System time was Tue Nov 15 16:13:08 UTC 2005.
Setting the System Clock using the Hardware Clock as reference...
System Clock set. System local time is now Tue Nov 15 16:13:11 UTC 2005.
Cleaning up ifupdown...done.
Calculating module dependencies... done.
Loading modules...
All modules loaded.
Checking all file systems...
fsck 1.37 (21-Mar-2005)
Setting kernel variables ...
... done.
Mounting local filesystems...
Cleaning /tmp /var/run /var/lock.
Setting up networking...done.
Setting up IP spoofing protection: rp_filter.
Configuring network interfaces...done.
Setting the System Clock using the Hardware Clock as reference...
end_request: I/O error, dev hda1, sector 4168
Buffer I/O error on device hda1, logical block 521
lost page write due to I/O error on hda1
end_request: I/O error, dev hda1, sector 507904
Buffer I/O error on device hda1, logical block 63488
lost page write due to I/O error on hda1
Aborting journal on device hda1.
__journal_remove_journal_head: freeing b_committed_data
__journal_remove_journal_head: freeing b_committed_data
__journal_remove_journal_head: freeing b_committed_data
ext3_abort called.
EXT3-fs error (device hda1): ext3_journal_start_sb: Detected aborted journal
Remounting filesystem read-only
System Clock set. Local time: Tue Nov 15 17:13:13 CET 2005
/etc/init.d/rcS: line 18: /etc/nologin: Read-only file system
/etc/init.d/rcS: line 24: /var/run/utmp: Read-only file system
Initializing random number generator...rm: cannot remove
`/var/lib/urandom/random-seed'': Read-only file system
urandom start: failed.
done.
Recovering nvi editor sessions... done.
Setting up X server socket directory /tmp/.X11-unix...mkdir: cannot create
directory `/tmp/.X11-unix'': Read-only file system
INIT: Entering runlevel: 2
Starting system log daemon: syslogdchmod: changing permissions of
`/dev/xconsole'': Read-only file system
end_request: I/O error, dev hda1, sector 0
Buffer I/O error on device hda1, logical block 0
lost page write due to I/O error on hda1
Buffer I/O error on device hda1, logical block 1
lost page write due to I/O error on hda1
end_request: I/O error, dev hda1, sector 262160
Buffer I/O error on device hda1, logical block 32770
lost page write due to I/O error on hda1
end_request: I/O error, dev hda1, sector 262168
Buffer I/O error on device hda1, logical block 32771
lost page write due to I/O error on hda1
end_request: I/O error, dev hda1, sector 262176
Buffer I/O error on device hda1, logical block 32772
lost page write due to I/O error on hda1
end_request: I/O error, dev hda1, sector 262208
Buffer I/O error on device hda1, logical block 32776
lost page write due to I/O error on hda1
Buffer I/O error on device hda1, logical block 32777
lost page write due to I/O error on hda1
end_request: I/O error, dev hda1, sector 262232
Buffer I/O error on device hda1, logical block 32779
lost page write due to I/O error on hda1
end_request: I/O error, dev hda1, sector 262256
Buffer I/O error on device hda1, logical block 32782
lost page write due to I/O error on hda1
end_request: I/O error, dev hda1, sector 262264
Buffer I/O error on device hda1, logical block 32783
lost page write due to I/O error on hda1
end_request: I/O error, dev hda1, sector 262272
end_request: I/O error, dev hda1, sector 262320
end_request: I/O error, dev hda1, sector 262464
end_request: I/O error, dev hda1, sector 458752
end_request: I/O error, dev hda1, sector 524304
end_request: I/O error, dev hda1, sector 786464
end_request: I/O error, dev hda1, sector 1048576
end_request: I/O error, dev hda1, sector 1048584
end_request: I/O error, dev hda1, sector 1048592
end_request: I/O error, dev hda1, sector 1245192
end_request: I/O error, dev hda1, sector 1310752
end_request: I/O error, dev hda1, sector 1310760
end_request: I/O error, dev hda1, sector 1310768
end_request: I/O error, dev hda1, sector 1310784
end_request: I/O error, dev hda1, sector 1310792
end_request: I/O error, dev hda1, sector 1310832
end_request: I/O error, dev hda1, sector 1310840
end_request: I/O error, dev hda1, sector 2359736
end_request: I/O error, dev hda1, sector 2883616
end_request: I/O error, dev hda1, sector 2883632
end_request: I/O error, dev hda1, sector 3407888
end_request: I/O error, dev hda1, sector 3670016
end_request: I/O error, dev hda1, sector 3670024
end_request: I/O error, dev hda1, sector 3670392
end_request: I/O error, dev hda1, sector 3670448
end_request: I/O error, dev hda1, sector 3670480
end_request: I/O error, dev hda1, sector 4014864
end_request: I/O error, dev hda1, sector 4016104
end_request: I/O error, dev hda1, sector 4016144
----------------------------------------------------------------------------------------------------------
syslog says:
----------------------------------------------------------------------------------------------------------
Nov 15 17:37:27 testpc-022 logger: /etc/xen/scripts/block: add
XENBUS_PATH=backend/vbd/13/770
Nov 15 17:37:27 testpc-022 logger: /etc/xen/scripts/block: add
XENBUS_PATH=backend/vbd/13/769
Nov 15 17:37:27 testpc-022 logger: /etc/xen/scripts/vif-bridge: online
XENBUS_PATH=backend/vif/13/0
Nov 15 17:37:27 testpc-022 kernel: device vif13.0 entered promiscuous mode
Nov 15 17:37:27 testpc-022 kernel: xenbr0: port 3(vif13.0) entering learning
state
Nov 15 17:37:27 testpc-022 kernel: xenbr0: topology change detected,
propagating
Nov 15 17:37:27 testpc-022 kernel: xenbr0: port 3(vif13.0) entering forwarding
state
Nov 15 17:37:27 testpc-022 logger: /etc/xen/scripts/block: Writing
backend/vbd/13/770/physical-device 0x0700 backend/vbd/13/770/node /dev/loop0
to xenstore.
Nov 15 17:37:27 testpc-022 logger: /etc/xen/scripts/block: Writing
backend/vbd/13/769/physical-device 0x0701 backend/vbd/13/769/node /dev/loop1
to xenstore.
Nov 15 17:37:27 testpc-022 logger: /etc/xen/scripts/vif-bridge: Successful
vif-bridge operation for vif13.0, bridge xenbr0.
Nov 15 17:37:27 testpc-022 logger: /etc/xen/scripts/vif-bridge: Writing
backend/vif/13/0/hotplug-status connected to xenstore.
Nov 15 17:37:27 testpc-022 logger: /etc/xen/scripts/block: Writing
backend/vbd/13/770/hotplug-status connected to xenstore.
Nov 15 17:37:27 testpc-022 logger: /etc/xen/scripts/block: Writing
backend/vbd/13/769/hotplug-status connected to xenstore.
Nov 15 17:37:28 testpc-022 kernel: GFS: fsid=xencl:xen3-root.0: warning:
assertion "gfs_glock_is_locked_by_me(ip->i_gl)" failed
Nov 15 17:37:28 testpc-022 kernel: GFS: fsid=xencl:xen3-root.0: function =
gfs_prepare_write
Nov 15 17:37:28 testpc-022 kernel: GFS: fsid=xencl:xen3-root.0: file
= /usr/local/src/cluster-1.01.00/gfs-kernel/src/gfs/ops_address.c, line = 329
Nov 15 17:37:28 testpc-022 kernel: GFS: fsid=xencl:xen3-root.0: time =
1132072648
-----------------------------------------------------------------------------------------------------
xend.log:
------------------------------------------------------------------------------------------------------
[2005-11-15 17:37:26 xend.XendDomainInfo] DEBUG (XendDomainInfo:151)
XendDomainInfo.create([''vm'', [''name'',
''Gollum''], [''memory'',
''64''], [''image'',
[''linux'', [''kernel'',
''/boot/vmlinuz-2.6-xenU''], [''root'',
''/dev/hda1 ro''],
[''vcpus'', ''1''], [''vcpus'',
''1''], [''boot'', ''c'']]],
[''device'', [''vbd'',
[''uname'',
''file:/cluster/debguest/vm1disk.img''],
[''dev'', ''hda1''], [''mode'',
''w'']]],
[''device'', [''vbd'',
[''uname'',
''file:/cluster/debguest/vm1swap.img''],
[''dev'',
''hda2''], [''mode'', ''w'']]],
[''device'', [''vif'', [''mac'',
''00:50:56:00:04:42''],
[''bridge'', ''xenbr0'']]]])
[2005-11-15 17:37:26 xend.XendDomainInfo] DEBUG (XendDomainInfo:254)
parseConfig: config is [''vm'', [''name'',
''Gollum''], [''memory'',
''64''], [''image'',
[''linux'', [''kernel'',
''/boot/vmlinuz-2.6-xenU''], [''root'',
''/dev/hda1 ro''],
[''vcpus'', ''1''], [''vcpus'',
''1''], [''boot'', ''c'']]],
[''device'', [''vbd'',
[''uname'',
''file:/cluster/debguest/vm1disk.img''],
[''dev'', ''hda1''], [''mode'',
''w'']]],
[''device'', [''vbd'',
[''uname'',
''file:/cluster/debguest/vm1swap.img''],
[''dev'',
''hda2''], [''mode'', ''w'']]],
[''device'', [''vif'', [''mac'',
''00:50:56:00:04:42''],
[''bridge'', ''xenbr0'']]]]
[2005-11-15 17:37:26 xend.XendDomainInfo] DEBUG (XendDomainInfo:308)
parseConfig: result is {''ssidref'': None,
''name'': ''Gollum'',
''bootloader'':
None, ''on_reboot'': None, ''image'':
[''linux'', [''kernel'',
''/boot/vmlinuz-2.6-xenU''], [''root'',
''/dev/hda1 ro''], [''vcpus'',
''1''],
[''vcpus'', ''1''], [''boot'',
''c'']], ''on_poweroff'': None,
''on_crash'': None,
''backend'': [], ''vcpus'': 1,
''cpu_weight'': None, ''vcpu_avail'': None,
''memory'':
64, ''device'': [(''vbd'',
[''vbd'', [''uname'',
''file:/cluster/debguest/vm1disk.img''],
[''dev'', ''hda1''], [''mode'',
''w'']]),
(''vbd'', [''vbd'', [''uname'',
''file:/cluster/debguest/vm1swap.img''],
[''dev'',
''hda2''], [''mode'', ''w'']]),
(''vif'', [''vif'', [''mac'',
''00:50:56:00:04:42''],
[''bridge'', ''xenbr0'']])],
''maxmem'': None, ''cpu'': None,
''uuid'': None}
[2005-11-15 17:37:26 xend.XendDomainInfo] DEBUG (XendDomainInfo:1011)
XendDomainInfo.construct: None 0
[2005-11-15 17:37:26 xend.XendDomainInfo] DEBUG (XendDomainInfo:1043)
XendDomainInfo.initDomain: 13 1.0
[2005-11-15 17:37:26 xend] INFO (image:132) buildDomain os=linux dom=13
vcpus=1
[2005-11-15 17:37:26 xend] DEBUG (image:170) dom = 13
[2005-11-15 17:37:26 xend] DEBUG (image:171) image
= /boot/vmlinuz-2.6-xenU
[2005-11-15 17:37:26 xend] DEBUG (image:172) store_evtchn = 1
[2005-11-15 17:37:26 xend] DEBUG (image:173) console_evtchn = 2
[2005-11-15 17:37:26 xend] DEBUG (image:174) cmdline = root=/dev/hda1
ro
[2005-11-15 17:37:26 xend] DEBUG (image:175) ramdisk [2005-11-15 17:37:26
xend] DEBUG (image:176) vcpus = 1
[2005-11-15 17:37:27 xend] DEBUG (DevController:286) DevController: writing
{''virtual-device'': ''769'',
''backend-id'': ''0'',
''state'': ''1'', ''backend'':
''/local/domain/0/backend/vbd/13/769''} to
/local/domain/13/device/vbd/769.
[2005-11-15 17:37:27 xend] DEBUG (DevController:288) DevController: writing
{''domain'': ''Gollum'',
''params'': ''/cluster/debguest/vm1disk.img'',
''frontend-id'': ''13'',
''type'': ''file'', ''state'':
''1'', ''dev'': ''hda1'',
''frontend'':
''/local/domain/13/device/vbd/769''} to
/local/domain/0/backend/vbd/13/769.
[2005-11-15 17:37:27 xend] DEBUG (DevController:286) DevController: writing
{''virtual-device'': ''770'',
''backend-id'': ''0'',
''state'': ''1'', ''backend'':
''/local/domain/0/backend/vbd/13/770''} to
/local/domain/13/device/vbd/770.
[2005-11-15 17:37:27 xend] DEBUG (DevController:288) DevController: writing
{''domain'': ''Gollum'',
''params'': ''/cluster/debguest/vm1swap.img'',
''frontend-id'': ''13'',
''type'': ''file'', ''state'':
''1'', ''dev'': ''hda2'',
''frontend'':
''/local/domain/13/device/vbd/770''} to
/local/domain/0/backend/vbd/13/770.
[2005-11-15 17:37:27 xend] DEBUG (DevController:286) DevController: writing
{''backend-id'': ''0'', ''mac'':
''00:50:56:00:04:42'', ''handle'':
''0'', ''state'': ''1'',
''backend'':
''/local/domain/0/backend/vif/13/0''}
to /local/domain/13/device/vif/0.
[2005-11-15 17:37:27 xend] DEBUG (DevController:288) DevController: writing
{''bridge'': ''xenbr0'',
''domain'': ''Gollum'',
''handle'': ''0'', ''script'':
''/etc/xen/scripts/vif-bridge'', ''state'':
''1'', ''frontend'':
''/local/domain/13/device/vif/0'', ''mac'':
''00:50:56:00:04:42'', ''frontend-id'':
''13''} to /local/domain/0/backend/vif/13/0.
[2005-11-15 17:37:27 xend.XendDomainInfo] DEBUG (XendDomainInfo:549) Storing
VM details: {''ssidref'': ''0'',
''uuid'':
''77dac010-86a3-3e1c-f9b4-2e0d33d49063'',
''on_reboot'': ''restart'',
''image'': "(linux (kernel /boot/vmlinuz-2.6-xenU)
(root ''/dev/hda1 ro'') (vcpus 1) (vcpus 1) (boot c))",
''on_poweroff'':
''destroy'', ''name'':
''Gollum'', ''vcpus'': ''1'',
''vcpu_avail'': ''1'',
''memory'': ''64'',
''on_crash'': ''restart'',
''maxmem'': ''64''}
[2005-11-15 17:37:27 xend.XendDomainInfo] DEBUG (XendDomainInfo:574) Storing
domain details: {''console/ring-ref'':
''385727'', ''console/port'':
''2'', ''name'':
''Gollum'', ''console/limit'':
''1048576'', ''vm'':
''/vm/77dac010-86a3-3e1c-f9b4-2e0d33d49063'',
''domid'': ''13'',
''cpu/0/availability'': ''online'',
''memory/target'': ''65536'',
''store/ring-ref'':
''10784'', ''store/port'':
''1''}
[2005-11-15 17:37:27 xend] DEBUG (DevController:82) Waiting for devices vif.
[2005-11-15 17:37:27 xend] DEBUG (DevController:88) Waiting for 0.
[2005-11-15 17:37:27 xend] DEBUG (DevController:309) hotplugStatus 0
[2005-11-15 17:37:27 xend] DEBUG (DevController:309) hotplugStatus 0
[2005-11-15 17:37:27 xend] DEBUG (DevController:82) Waiting for devices usb.
[2005-11-15 17:37:27 xend] DEBUG (DevController:82) Waiting for devices vbd.
[2005-11-15 17:37:27 xend] DEBUG (DevController:88) Waiting for 769.
[2005-11-15 17:37:27 xend] DEBUG (DevController:309) hotplugStatus 769
[2005-11-15 17:37:27 xend] DEBUG (DevController:88) Waiting for 770.
[2005-11-15 17:37:27 xend] DEBUG (DevController:309) hotplugStatus 770
[2005-11-15 17:37:27 xend] DEBUG (DevController:82) Waiting for devices pci.
[2005-11-15 17:37:27 xend] DEBUG (DevController:82) Waiting for devices
ioports.
[2005-11-15 17:37:27 xend] DEBUG (DevController:82) Waiting for devices vtpm.
[2005-11-15 17:37:27 xend] INFO (XendDomain:349) Domain Gollum (13) unpaused.
----------------------------------------------------------------------------------------------------------
It''d be great if someone could help me out.
Michael
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel