john maclean
2006-Jul-30 20:30 UTC
[Xen-users] Overcoming loopback device limitation - revisited
Hi there chaps, Just to get me up and running I ran `make world && make install` from source. All went well until I started running out of loopback devices to create guests. I know of the 8 devices limitation and I''m sure that the loop "module" is built into the kernel; modinfo loop filename: /lib/modules/2.6.16-xen/kernel/drivers/block/loop.ko license: GPL alias: block-major-7-* vermagic: 2.6.16-xen SMP 686 gcc-3.3 depends: parm: max_loop:Maximum number of loop devices (1-256) (int) OK, that number is more than enough for me. This is a laptop and I''d want to manage about 12. So I use this boot time parameter even though I know/think that it''s a built in:- cat /proc/cmdline root=/dev/hda3 ro console=tty0 max_loop=32 To create some more loop back devices I ran the following:- for X in $(seq 8 31); do mknod ; MAKEDEV /dev/loop$X b 7 $i ; chmod 660 /dev/loop$X; chown 0.disk /dev/loop$X; done Now I can only seem to have just one domU running! #acid~# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 512 1 r----- 108.4 vm01 1 64 1 ------ 0.3 acid:~# xm create vm00 -c /etc/xen/vm00 Using config file "/etc/xen/vm00". Error: Device 769 (vbd) could not be connected. Backend device not found. I have a few simple-ish questions that I''d a appreciate some answers for; - What is the file system that creates loopback devices a required, on the fly? - How do I know if I have this? - Did I give the newly created devices the wrong permissons? - Why does it appear that all of the /dev/loop devices bar one are "blocked"? - Do I have to do some /etc/udev mojo? version:- xen-3.0.2-2 arch:- debian sarge me:- very stuck ;) -- John Maclean - 07739 171 531 MSc (DIC) _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Mathias Diehl
2006-Jul-30 21:29 UTC
AW: [Xen-users] Overcoming loopback device limitation - revisited
Hi John, actually I''m not sure about your problem - but here''s what I did: C=8; echo; echo "Creating loop device nodes."; \ while [ $C -lt 256 ]; do mknod /dev/loop$C b 7 $C; \ echo -n .; C=`expr $C + 1`; done; echo; As yo see permissions do not appear. You create your file''s to store the VM''s, give them a filesystem and fire them up. cheers, Mathias ___ www.evoconcept.de -----Ursprüngliche Nachricht----- Von: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com]Im Auftrag von john maclean Gesendet: Sonntag, 30. Juli 2006 22:31 An: xen-users@lists.xensource.com Betreff: [Xen-users] Overcoming loopback device limitation - revisited Hi there chaps, Just to get me up and running I ran `make world && make install` from source. All went well until I started running out of loopback devices to create guests. I know of the 8 devices limitation and I''m sure that the loop "module" is built into the kernel; modinfo loop filename: /lib/modules/2.6.16-xen/kernel/drivers/block/loop.ko license: GPL alias: block-major-7-* vermagic: 2.6.16-xen SMP 686 gcc-3.3 depends: parm: max_loop:Maximum number of loop devices (1-256) (int) OK, that number is more than enough for me. This is a laptop and I''d want to manage about 12. So I use this boot time parameter even though I know/think that it''s a built in:- cat /proc/cmdline root=/dev/hda3 ro console=tty0 max_loop=32 To create some more loop back devices I ran the following:- for X in $(seq 8 31); do mknod ; MAKEDEV /dev/loop$X b 7 $i ; chmod 660 /dev/loop$X; chown 0.disk /dev/loop$X; done Now I can only seem to have just one domU running! #acid~# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 512 1 r----- 108.4 vm01 1 64 1 ------ 0.3 acid:~# xm create vm00 -c /etc/xen/vm00 Using config file "/etc/xen/vm00". Error: Device 769 (vbd) could not be connected. Backend device not found. I have a few simple-ish questions that I''d a appreciate some answers for; - What is the file system that creates loopback devices a required, on the fly? - How do I know if I have this? - Did I give the newly created devices the wrong permissons? - Why does it appear that all of the /dev/loop devices bar one are "blocked"? - Do I have to do some /etc/udev mojo? version:- xen-3.0.2-2 arch:- debian sarge me:- very stuck ;) -- John Maclean - 07739 171 531 MSc (DIC) _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Paolo Supino
2006-Jul-31 05:09 UTC
Re: [Xen-users] Overcoming loopback device limitation - revisited
Hi John I got stuck with a similar problem in the past :-( Before I reply to your question here is something learned about loop devices and Xen: Have the kernel be able to manage the same number of loop devices you have /dev directory. No less, no more (otherwise some unexpected things happen). Now to your questions: 1. A loop filesystem is nothing special. From the point of view of Dom0 it is nothing more than a file and since it nothing more than a file you create it using the command: `dd if=/dev/zero of=[filename] bs=Yk count=X (Y is a number and k stands for Kilobytes, X is also a number). Do note that block size is for DD only and has no implication for the block size used inside by the filesystem. After the file is created you simply run `mkfs -t [filesystem type] [filename]` to create the filesystem inside(mkfs will nag you about the file not being a block special device, answer yes to the proceed question). Last step is to mount the newly created filesystem somewhere in Dom0 with the command `mount -o loop [filename] [mount point] and populate it. 2. The commands `dd`, mkfs and mount are always installed in Linux you should be OK. If you don''t have one of them something is very wrong with your installation. One important thing to note: `mkfs` is only a wrapper to the filesystem specific `mkfs`. Make sure that you have the binaries for the chosen filesystem. 3. Your loop devices and permissions seem all in order (I have the same exact configuration and it works). 4. About udev mojo: I have no clue, I don''t use udev (I hate it and refuse to use it ;-)) Paolo john maclean wrote:> Hi there chaps, > > Just to get me up and running I ran `make world && make install` from > source. All went well until I started running out of loopback devices > to create guests. I know of the 8 devices limitation and I''m sure that > the loop "module" is built into the kernel; > > modinfo loop > filename: /lib/modules/2.6.16-xen/kernel/drivers/block/loop.ko > license: GPL > alias: block-major-7-* > vermagic: 2.6.16-xen SMP 686 gcc-3.3 > depends: > parm: max_loop:Maximum number of loop devices (1-256) (int) > > OK, that number is more than enough for me. This is a laptop and I''d > want to manage about 12. So I use this boot time parameter even though > I know/think that it''s a built in:- > > cat /proc/cmdline > root=/dev/hda3 ro console=tty0 max_loop=32 > > To create some more loop back devices I ran the following:- > > for X in $(seq 8 31); do mknod ; MAKEDEV /dev/loop$X b 7 $i ; chmod > 660 /dev/loop$X; chown 0.disk /dev/loop$X; done > > Now I can only seem to have just one domU running! > > #acid~# xm list > Name ID Mem(MiB) VCPUs State Time(s) > Domain-0 0 512 1 r----- 108.4 > vm01 1 64 1 ------ 0.3 > > acid:~# xm create vm00 -c /etc/xen/vm00 > Using config file "/etc/xen/vm00". > Error: Device 769 (vbd) could not be connected. Backend device not found. > > I have a few simple-ish questions that I''d a appreciate some answers for; > - What is the file system that creates loopback devices a required, > on the fly? > - How do I know if I have this? > - Did I give the newly created devices the wrong permissons? > - Why does it appear that all of the /dev/loop devices bar one are > "blocked"? > - Do I have to do some /etc/udev mojo? > > version:- xen-3.0.2-2 > arch:- debian sarge > me:- very stuck ;)_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
john maclean
2006-Jul-31 09:31 UTC
Re: [Xen-users] Overcoming loopback device limitation - revisited
Thanks Paolo,
You are correct with the info for creating loopback devices. I made a
few ealier with the following:-
mkdir -p /xen/images
for X in $(seq 0 9) do;
dd if=/zev/zero of=/xen/images/vm0$.img bs=1024k count=1024
dd if=/zev/zero of=/xen/images/vm0$-swap.img bs=1024k count=128
mkfs.ext3 /xen/images/vm0$.img
mkswap /xen/images/vm0$-swap.img
done
I then mount -o loop /xen/images/vm-foo.img /mnt/blah
Populate it with an OS (e.g. Debian, Gentoo....)
Configure domU, umount and fire up. I''ll destroy the *one* domU
that''s
running, create spme more loopback devices and see what happens.
I''ll not reboot, restart or reinstall anything! I actually want to
learn how this works!
On 31/07/06, Paolo Supino <paolo@actcom.net.il>
wrote:> Hi John
>
> I got stuck with a similar problem in the past :-( Before I reply to
> your question here is something learned about loop devices and Xen:
> Have the kernel be able to manage the same number of loop devices you
> have /dev directory. No less, no more (otherwise some unexpected things
> happen). Now to your questions:
> 1. A loop filesystem is nothing special. From the point of view of Dom0
> it is nothing more than a file and since it nothing more than a file you
> create it using the command: `dd if=/dev/zero of=[filename] bs=Yk
> count=X (Y is a number and k stands for Kilobytes, X is also a number).
> Do note that block size is for DD only and has no implication for the
> block size used inside by the filesystem. After the file is created you
> simply run `mkfs -t [filesystem type] [filename]` to create the
> filesystem inside(mkfs will nag you about the file not being a block
> special device, answer yes to the proceed question). Last step is to
> mount the newly created filesystem somewhere in Dom0 with the command
> `mount -o loop [filename] [mount point] and populate it.
> 2. The commands `dd`, mkfs and mount are always installed in Linux you
> should be OK. If you don''t have one of them something is very
wrong with
> your installation. One important thing to note: `mkfs` is only a wrapper
> to the filesystem specific `mkfs`. Make sure that you have the binaries
> for the chosen filesystem.
> 3. Your loop devices and permissions seem all in order (I have the same
> exact configuration and it works).
> 4. About udev mojo: I have no clue, I don''t use udev (I hate it
and
> refuse to use it ;-))
>
>
>
>
>
> Paolo
>
>
> john maclean wrote:
>
> > Hi there chaps,
> >
> > Just to get me up and running I ran `make world && make
install` from
> > source. All went well until I started running out of loopback devices
> > to create guests. I know of the 8 devices limitation and I''m
sure that
> > the loop "module" is built into the kernel;
> >
> > modinfo loop
> > filename: /lib/modules/2.6.16-xen/kernel/drivers/block/loop.ko
> > license: GPL
> > alias: block-major-7-*
> > vermagic: 2.6.16-xen SMP 686 gcc-3.3
> > depends:
> > parm: max_loop:Maximum number of loop devices (1-256) (int)
> >
> > OK, that number is more than enough for me. This is a laptop and
I''d
> > want to manage about 12. So I use this boot time parameter even though
> > I know/think that it''s a built in:-
> >
> > cat /proc/cmdline
> > root=/dev/hda3 ro console=tty0 max_loop=32
> >
> > To create some more loop back devices I ran the following:-
> >
> > for X in $(seq 8 31); do mknod ; MAKEDEV /dev/loop$X b 7 $i ; chmod
> > 660 /dev/loop$X; chown 0.disk /dev/loop$X; done
> >
> > Now I can only seem to have just one domU running!
> >
> > #acid~# xm list
> > Name ID Mem(MiB) VCPUs State Time(s)
> > Domain-0 0 512 1 r----- 108.4
> > vm01 1 64 1 ------ 0.3
> >
> > acid:~# xm create vm00 -c /etc/xen/vm00
> > Using config file "/etc/xen/vm00".
> > Error: Device 769 (vbd) could not be connected. Backend device not
found.
> >
> > I have a few simple-ish questions that I''d a appreciate some
answers for;
> > - What is the file system that creates loopback devices a required,
> > on the fly?
> > - How do I know if I have this?
> > - Did I give the newly created devices the wrong permissons?
> > - Why does it appear that all of the /dev/loop devices bar one are
> > "blocked"?
> > - Do I have to do some /etc/udev mojo?
> >
> > version:- xen-3.0.2-2
> > arch:- debian sarge
> > me:- very stuck ;)
>
>
>
--
John Maclean - 07739 171 531
MSc (DIC)
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
john maclean
2006-Aug-01 00:06 UTC
Fwd: [Xen-users] Overcoming loopback device limitation - revisited
still no joy. Managed to bring some domUs back to life though. Only had one, now I got four. acid:~# losetup -f losetup: could not find any free loop device acid:~# xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 512 1 r----- 64.6 centos0 1 64 1 -b---- 0.1 centos1 2 64 1 -b---- 0.1 centos2 3 64 1 -b---- 0.1 centos3 4 64 1 -b---- 0.1 acid:~# cat /proc/cmdline root=/dev/hda3 ro console=tty0 max_loop=64 ---------- Forwarded message ---------- From: Paolo Supino <paolo@actcom.net.il> Date: 31-Jul-2006 16:27 Subject: Re: [Xen-users] Overcoming loopback device limitation - revisited To: john maclean <jayeola@gmail.com> Hi John After 12 years of using Linux (of which 10 administrating) I learned that there is no reason to reboot a system. And people suggesting a reboot simply don''t know what the problem is or how to solve it. TIA Paolo PS - Actually there is one more reason for rebooting: Kernel upgrades ... ;-) john maclean wrote:> Thanks Paolo, > > You are correct with the info for creating loopback devices. I made a > few ealier with the following:- > > mkdir -p /xen/images > for X in $(seq 0 9) do; > dd if=/zev/zero of=/xen/images/vm0$.img bs=1024k count=1024 > dd if=/zev/zero of=/xen/images/vm0$-swap.img bs=1024k count=128 > mkfs.ext3 /xen/images/vm0$.img > mkswap /xen/images/vm0$-swap.img > done > > I then mount -o loop /xen/images/vm-foo.img /mnt/blah > Populate it with an OS (e.g. Debian, Gentoo....) > Configure domU, umount and fire up. I''ll destroy the *one* domU that''s > running, create spme more loopback devices and see what happens. > > I''ll not reboot, restart or reinstall anything! I actually want to > learn how this works! > > > > On 31/07/06, Paolo Supino <paolo@actcom.net.il> wrote: > >> Hi John >> >> I got stuck with a similar problem in the past :-( Before I reply to >> your question here is something learned about loop devices and Xen: >> Have the kernel be able to manage the same number of loop devices you >> have /dev directory. No less, no more (otherwise some unexpected things >> happen). Now to your questions: >> 1. A loop filesystem is nothing special. From the point of view of Dom0 >> it is nothing more than a file and since it nothing more than a file you >> create it using the command: `dd if=/dev/zero of=[filename] bs=Yk >> count=X (Y is a number and k stands for Kilobytes, X is also a number). >> Do note that block size is for DD only and has no implication for the >> block size used inside by the filesystem. After the file is created you >> simply run `mkfs -t [filesystem type] [filename]` to create the >> filesystem inside(mkfs will nag you about the file not being a block >> special device, answer yes to the proceed question). Last step is to >> mount the newly created filesystem somewhere in Dom0 with the command >> `mount -o loop [filename] [mount point] and populate it. >> 2. The commands `dd`, mkfs and mount are always installed in Linux you >> should be OK. If you don''t have one of them something is very wrong with >> your installation. One important thing to note: `mkfs` is only a wrapper >> to the filesystem specific `mkfs`. Make sure that you have the binaries >> for the chosen filesystem. >> 3. Your loop devices and permissions seem all in order (I have the same >> exact configuration and it works). >> 4. About udev mojo: I have no clue, I don''t use udev (I hate it and >> refuse to use it ;-)) >> >> >> >> >> >> Paolo >> >> >> john maclean wrote: >> >> > Hi there chaps, >> > >> > Just to get me up and running I ran `make world && make install` from >> > source. All went well until I started running out of loopback devices >> > to create guests. I know of the 8 devices limitation and I''m sure that >> > the loop "module" is built into the kernel; >> > >> > modinfo loop >> > filename: /lib/modules/2.6.16-xen/kernel/drivers/block/loop.ko >> > license: GPL >> > alias: block-major-7-* >> > vermagic: 2.6.16-xen SMP 686 gcc-3.3 >> > depends: >> > parm: max_loop:Maximum number of loop devices (1-256) (int) >> > >> > OK, that number is more than enough for me. This is a laptop and I''d >> > want to manage about 12. So I use this boot time parameter even though >> > I know/think that it''s a built in:- >> > >> > cat /proc/cmdline >> > root=/dev/hda3 ro console=tty0 max_loop=32 >> > >> > To create some more loop back devices I ran the following:- >> > >> > for X in $(seq 8 31); do mknod ; MAKEDEV /dev/loop$X b 7 $i ; chmod >> > 660 /dev/loop$X; chown 0.disk /dev/loop$X; done >> > >> > Now I can only seem to have just one domU running! >> > >> > #acid~# xm list >> > Name ID Mem(MiB) VCPUs State Time(s) >> > Domain-0 0 512 1 r----- 108.4 >> > vm01 1 64 1 ------ 0.3 >> > >> > acid:~# xm create vm00 -c /etc/xen/vm00 >> > Using config file "/etc/xen/vm00". >> > Error: Device 769 (vbd) could not be connected. Backend device not >> found. >> > >> > I have a few simple-ish questions that I''d a appreciate some >> answers for; >> > - What is the file system that creates loopback devices a required, >> > on the fly? >> > - How do I know if I have this? >> > - Did I give the newly created devices the wrong permissons? >> > - Why does it appear that all of the /dev/loop devices bar one are >> > "blocked"? >> > - Do I have to do some /etc/udev mojo? >> > >> > version:- xen-3.0.2-2 >> > arch:- debian sarge >> > me:- very stuck ;) >> >> >> > >-- John Maclean - 07739 171 531 MSc (DIC) _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Steve Kemp
2006-Aug-01 00:11 UTC
Re: Fwd: [Xen-users] Overcoming loopback device limitation - revisited
On Tue, Aug 01, 2006 at 01:06:04AM +0100, john maclean wrote:> still no joy. Managed to bring some domUs back to life though. Only > had one, now I got four.> acid:~# losetup -f > losetup: could not find any free loop deviceRemove the max_loop setting from the boot line. Instead run: echo ''options loop max_loop=255'' > /etc/modprobe.d/loop.local Now if you have no domains running you should be able to do: rmmod loop modprobe loop dmesg | grep loop The last line should show you that you have >8 loaded. Steve -- _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
john maclean
2006-Aug-01 13:35 UTC
Re: Fwd: [Xen-users] Overcoming loopback device limitation - revisited
xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 512 1 r----- 6956.8 centos0 7 32 1 ------ 15.2 centos1 9 32 1 ------ 15.2 centos2 12 32 1 ------ 15.5 centos3 13 32 1 ------ 21.4 centos4 14 32 1 ------ 15.2 centos5 15 32 1 ------ 15.0 centos6 16 32 1 ------ 28452.9 centos7 17 32 1 ------ 14.4 I finally got there at about 05:10 this morning. rmmod loop modprobe loop max_loop=64 That did it. Had a look in the source code and /usr/src/xen-3.0.2-2/pristine-linux-2.6.16/drivers/block/loop.c gave me the clue. "... * Maximum number of loop devices when compiled-in now selectable by passing * max_loop=<1-255> to the kernel on boot. ...static int max_loop = 8;" That final line threw me as I thought I''d have to change that value from 8 to something like 32. It finally occurred to me what everyone was talking about. Thanks chaps for your patience and time. On 01/08/06, Steve Kemp <steve@steve.org.uk> wrote:> On Tue, Aug 01, 2006 at 01:06:04AM +0100, john maclean wrote: > > still no joy. Managed to bring some domUs back to life though. Only > > had one, now I got four. > > > acid:~# losetup -f > > losetup: could not find any free loop device > > Remove the max_loop setting from the boot line. > > Instead run: > > echo ''options loop max_loop=255'' > /etc/modprobe.d/loop.local > > Now if you have no domains running you should be able to do: > > rmmod loop > modprobe loop > dmesg | grep loop > > The last line should show you that you have >8 loaded. > > Steve > -- >-- John Maclean - 07739 171 531 MSc (DIC) _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users