Thomas Karsten
2006-Jul-19 10:18 UTC
[Xen-users] Error: Device 769 (vbd) could not be connected... Limit of 25 domains.
Hello,
I am running Xen 3.0.2 in an Ubuntu 6.06 Linux system (kernel 2.6.16-xen).
Xend is running and works well.
I would like to use Xen to run about 80 domains (for testing purpose). All
domains share the same image file (which therefor is mounted
''readonly'' by
Xen) and each domain is configured to use only 10 MB of RAM. I am using a
script to generate name, hostname and IP address for each guest (it simply
loops over i and calls:''xm create name=guestdom$i hostname=$i
ip=192.168.1.
$i'').
Everything works fine, but after 25 concurrent running domains Xen refuses to
create any further domains, giving me the error:
Error: Device 769 (vbd) could not be connected. Hotplug scripts not working.
On this mailing list I found several threads dealing with this error message.
However, I did not find that anybody is facing this kind of limitation that I
have. For your information, the number of loopback devices in my system is
132. Also, I am not running out of memory, this would be stated by Xen (I
have 1500 MB of RAM, where 500 MB are occupied by Xen).
I can remove a guest domain and create a new one, this is no problem. However,
on my system I am not able to create more than 25 domains at the same time.
Can anybody help please?
Thank you very much, all ideas are appreciated,
Thomas
----------------------------------------
Output of Xen below:
# xm info
host : pc27
release : 2.6.16-xen
version : #1 SMP Thu Jul 13 10:04:57 CST 2006
machine : x86_64
nr_cpus : 2
nr_nodes : 1
sockets_per_node : 1
cores_per_socket : 1
threads_per_core : 2
cpu_mhz : 3000
hw_caps : bfebfbff:20100800:00000000:00000180:0000641d
total_memory : 1510
free_memory : 722
xen_major : 3
xen_minor : 0
xen_extra : .2-2
xen_caps : xen-3.0-x86_64
platform_params : virt_start=0xffff800000000000
xen_changeset : Thu Apr 13 15:18:37 2006 +0100 9617:5802713c159b
cc_compiler : gcc version 4.0.3 (Ubuntu 4.0.3-1ubuntu5)
cc_compile_by : root
cc_compile_domain :
cc_compile_date : Thu Jul 13 11:12:54 CST 2006
----------
# xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 508 2 r----- 1092.8
guestdom120 71 10 1 ------ 2.3
guestdom121 72 10 1 ------ 2.4
guestdom122 73 10 1 ------ 1.8
guestdom123 74 10 1 ------ 1.8
guestdom124 75 10 1 ------ 1.8
guestdom125 76 10 1 ------ 1.8
guestdom126 77 10 1 ------ 2.2
guestdom127 78 10 1 ------ 1.8
guestdom128 79 10 1 ------ 1.8
guestdom129 80 10 1 ------ 1.7
guestdom130 81 10 1 ------ 1.7
guestdom131 82 10 1 ------ 1.7
guestdom132 83 10 1 ------ 1.7
guestdom133 84 10 1 ------ 1.7
guestdom134 85 10 1 ------ 1.7
guestdom135 86 10 1 ------ 1.7
guestdom136 87 10 1 ------ 1.7
guestdom137 88 10 1 ------ 1.7
guestdom138 89 10 1 ------ 1.7
guestdom139 90 10 1 ------ 1.7
guestdom140 91 10 1 ------ 1.7
guestdom141 92 10 1 ------ 1.6
guestdom142 93 10 1 ------ 1.6
guestdom143 94 10 1 ------ 1.6
guestdom144 95 10 1 ------ 1.6
----------
My config file for each domU looks like this:
#
# base settings
#
kernel = "/boot/vmlinuz-2.6.16-xenU"
memory = 10
name = "guestdom1"
disk = [''file:/xen-images/guestdom1.img,hda1,ro'']
#
# network settings:
# change the ip address for each guest
#
vif = [''bridge=xenbr0'']
ip = "192.168.1.120"
netmask = "255.255.255.0"
gateway = "192.168.1.1"
hostname = name
root = "/dev/hda1 rw"
----------
The content of xend.log (for one failed creation):
[2006-07-19 18:02:46 xend.XendDomainInfo] DEBUG (XendDomainInfo:178)
XendDomainInfo.create([''vm'', [''name'',
''guestdom145''], [''memory'', 10],
[''vcpus'', 1], [''image'',
[''linux'', [''kernel'',
''/boot/vmlinuz-2.6.16-xenU''],
[''ip'',
''192.168.1.141:1.2.3.4:192.168.1.1:255.255.255.0:guestdom1:eth0:off''],
[''root'', ''/dev/hda1 rw'']]],
[''device'', [''vbd'',
[''uname'',
''file:/xen-images/guestdom1.img''], [''dev'',
''hda1''], [''mode'', ''ro'']]],
[''device'', [''vif'',
[''bridge'', ''xenbr0'']]]])
[2006-07-19 18:02:46 xend.XendDomainInfo] DEBUG (XendDomainInfo:284)
parseConfig: config is [''vm'', [''name'',
''guestdom145''], [''memory'', 10],
[''vcpus'', 1], [''image'',
[''linux'', [''kernel'',
''/boot/vmlinuz-2.6.16-xenU''],
[''ip'',
''192.168.1.141:1.2.3.4:192.168.1.1:255.255.255.0:guestdom1:eth0:off''],
[''root'', ''/dev/hda1 rw'']]],
[''device'', [''vbd'',
[''uname'',
''file:/xen-images/guestdom1.img''], [''dev'',
''hda1''], [''mode'', ''ro'']]],
[''device'', [''vif'',
[''bridge'', ''xenbr0'']]]]
[2006-07-19 18:02:46 xend.XendDomainInfo] DEBUG (XendDomainInfo:380)
parseConfig: result is {''ssidref'': None,
''uuid'': None, ''on_crash'': None,
''on_reboot'': None, ''image'':
[''linux'', [''kernel'',
''/boot/vmlinuz-2.6.16-xenU''], [''ip'',
''192.168.1.141:1.2.3.4:192.168.1.1:255.255.255.0:guestdom1:eth0:off''],
[''root'', ''/dev/hda1 rw'']],
''on_poweroff'': None, ''cpus'': None,
''name'':
''guestdom145'', ''backend'': [],
''vcpus'': 1, ''cpu_weight'': None,
''vcpu_avail'':
None, ''memory'': 10, ''device'':
[(''vbd'', [''vbd'', [''uname'',
''file:/xen-images/guestdom1.img''], [''dev'',
''hda1''], [''mode'', ''ro'']]),
(''vif'',
[''vif'', [''bridge'',
''xenbr0'']])], ''bootloader'': None,
''cpu'': None, ''maxmem'':
None}
[2006-07-19 18:02:46 xend.XendDomainInfo] DEBUG (XendDomainInfo:1164)
XendDomainInfo.construct: None 0
[2006-07-19 18:02:46 xend.XendDomainInfo] DEBUG (XendDomainInfo:1196)
XendDomainInfo.initDomain: 96 1.0
[2006-07-19 18:02:46 xend] DEBUG (balloon:126) Balloon: free 718; need 11;
done.
[2006-07-19 18:02:46 xend] INFO (image:135) buildDomain os=linux dom=96
vcpus=1
[2006-07-19 18:02:46 xend] DEBUG (image:173) dom = 96
[2006-07-19 18:02:46 xend] DEBUG (image:174) image
= /boot/vmlinuz-2.6.16-xenU
[2006-07-19 18:02:46 xend] DEBUG (image:175) store_evtchn = 1
[2006-07-19 18:02:46 xend] DEBUG (image:176) console_evtchn = 2
[2006-07-19 18:02:46 xend] DEBUG (image:177) cmdline =
ip=192.168.1.141:1.2.3.4:192.168.1.1:255.255.255.0:guestdom1:eth0:off
root=/dev/hda1 rw
[2006-07-19 18:02:46 xend] DEBUG (image:178) ramdisk =
[2006-07-19 18:02:46 xend] DEBUG (image:179) vcpus = 1
[2006-07-19 18:02:46 xend] DEBUG (image:180) features =
[2006-07-19 18:02:46 xend] DEBUG (blkif:24) exception looking up device number
for hda1: [Errno 2] No such file or directory: ''/dev/hda1''
[2006-07-19 18:02:46 xend] DEBUG (DevController:103) DevController: writing
{''virtual-device'': ''769'',
''backend-id'': ''0'',
''state'': ''1'', ''backend'':
''/local/domain/0/backend/vbd/96/769''} to
/local/domain/96/device/vbd/769.
[2006-07-19 18:02:46 xend] DEBUG (DevController:105) DevController: writing
{''domain'': ''guestdom145'',
''frontend'':
''/local/domain/96/device/vbd/769'',
''dev'': ''hda1'', ''state'':
''1'', ''params'':
''/xen-images/guestdom1.img'', ''mode'':
''ro'', ''frontend-id'': ''96'',
''type'': ''file''}
to /local/domain/0/backend/vbd/96/769.
[2006-07-19 18:02:46 xend] DEBUG (DevController:103) DevController: writing
{''backend-id'': ''0'', ''mac'':
''00:16:3e:29:bf:6a'', ''handle'':
''0'', ''state'': ''1'',
''backend'':
''/local/domain/0/backend/vif/96/0''}
to /local/domain/96/device/vif/0.
[2006-07-19 18:02:46 xend] DEBUG (DevController:105) DevController: writing
{''bridge'': ''xenbr0'',
''domain'': ''guestdom145'',
''handle'': ''0'', ''script'':
''/etc/xen/scripts/vif-bridge'', ''state'':
''1'', ''frontend'':
''/local/domain/96/device/vif/0'', ''mac'':
''00:16:3e:29:bf:6a'', ''frontend-id'':
''96''} to /local/domain/0/backend/vif/96/0.
[2006-07-19 18:02:46 xend.XendDomainInfo] DEBUG (XendDomainInfo:671) Storing
VM details: {''ssidref'': ''0'',
''uuid'':
''7fcd771f-3975-8798-9656-225555b52bd2'',
''on_reboot'': ''restart'',
''start_time'': ''1153303366.78'',
''on_poweroff'':
''destroy'', ''name'':
''guestdom145'', ''vcpus'':
''1'', ''vcpu_avail'': ''1'',
''memory'':
''10'', ''on_crash'':
''restart'', ''image'': "(linux
(kernel /boot/vmlinuz-2.6.16-xenU) (ip
192.168.1.141:1.2.3.4:192.168.1.1:255.255.255.0:guestdom1:eth0:off) (root
''/dev/hda1 rw''))", ''maxmem'':
''10''}
[2006-07-19 18:02:46 xend.XendDomainInfo] DEBUG (XendDomainInfo:696) Storing
domain details: {''console/ring-ref'':
''325704'', ''console/port'':
''2'', ''name'':
''guestdom145'', ''console/limit'':
''1048576'', ''vm'':
''/vm/7fcd771f-3975-8798-9656-225555b52bd2'',
''domid'': ''96'',
''cpu/0/availability'': ''online'',
''memory/target'': ''10240'',
''store/ring-ref'':
''325705'', ''store/port'':
''1''}
[2006-07-19 18:02:46 xend.XendDomainInfo] DEBUG (XendDomainInfo:877)
XendDomainInfo.handleShutdownWatch
[2006-07-19 18:02:47 xend] DEBUG (DevController:132) Waiting for devices vif.
[2006-07-19 18:02:47 xend] DEBUG (DevController:138) Waiting for 0.
[2006-07-19 18:02:47 xend] DEBUG (DevController:403)
hotplugStatusCallback /local/domain/0/backend/vif/96/0/hotplug-status.
[2006-07-19 18:02:47 xend] DEBUG (DevController:403)
hotplugStatusCallback /local/domain/0/backend/vif/96/0/hotplug-status.
[2006-07-19 18:02:47 xend] DEBUG (DevController:417) hotplugStatusCallback 1.
[2006-07-19 18:02:47 xend] DEBUG (DevController:132) Waiting for devices usb.
[2006-07-19 18:02:47 xend] DEBUG (DevController:132) Waiting for devices vbd.
[2006-07-19 18:02:47 xend] DEBUG (DevController:138) Waiting for 769.
[2006-07-19 18:02:47 xend] DEBUG (DevController:403)
hotplugStatusCallback /local/domain/0/backend/vbd/96/769/hotplug-status.
[2006-07-19 18:02:57 xend.XendDomainInfo] DEBUG (XendDomainInfo:1327)
XendDomainInfo.destroy: domid=96
[2006-07-19 18:02:57 xend.XendDomainInfo] DEBUG (XendDomainInfo:1335)
XendDomainInfo.destroyDomain(96)
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Thomas Karsten
2006-Jul-20 09:25 UTC
Re: [Xen-users] Error: Device 769 (vbd) could not be connected... Limit of 25 domains.
Hi, I used the Debian GNU/Linux Etch system on the Xen 3.0 Live CD (Version 1.0) to have a look where I am doing some mistake in my system''s configuration. I used the standard config file ''/root/deb-conf'' and launched the following command: # for ((i=100; i < 140; i++)); do xm create /root/deb-conf name=guestdom$i; done This time I was able to create up to 15 guest domains, all the others resulted in the following error message: Device 2049 (vbd) could not be connected. Backend device not found. (As far as I experienced, the device 2049 is /dev/sda1.) However, I would like to know, if some of you also run into this problem, when launching this command on the same system (Xen 3.0 Live CD). I tested this on another machine, where I could only create up to 7 concurrent guest domains. Is the problem that I face related to the hardware? Or is it only related to the software (i.e. configuration of the whole system)? In latter case, why does the Live CD behave different on different computers? Or am I missing an important point in the configuration process? I am very curious about this problem and how to solve it, since it has been a topic on this mailing list many times, but I did not see any solution posted. I hope somebody can try to launch this command using that Xen Live CD and tell me about the results. Thank you very much, Thomas On Wednesday 19 July 2006 18:18, Thomas Karsten wrote:> Hello, > > I am running Xen 3.0.2 in an Ubuntu 6.06 Linux system (kernel > 2.6.16-xen). Xend is running and works well. > > I would like to use Xen to run about 80 domains (for testing purpose). > All domains share the same image file (which therefor is mounted > ''readonly'' by Xen) and each domain is configured to use only 10 MB of > RAM. I am using a script to generate name, hostname and IP address for > each guest (it simply loops over i and calls:''xm create name=guestdom$i > hostname=$i ip=192.168.1. $i''). > > Everything works fine, but after 25 concurrent running domains Xen > refuses to create any further domains, giving me the error: > > Error: Device 769 (vbd) could not be connected. Hotplug scripts not > working. > > On this mailing list I found several threads dealing with this error > message. However, I did not find that anybody is facing this kind of > limitation that I have. For your information, the number of loopback > devices in my system is 132. Also, I am not running out of memory, this > would be stated by Xen (I have 1500 MB of RAM, where 500 MB are > occupied by Xen). > > I can remove a guest domain and create a new one, this is no problem. > However, on my system I am not able to create more than 25 domains at > the same time. Can anybody help please? > > Thank you very much, all ideas are appreciated, > Thomas_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users