Marco Strullato
2008-Apr-23 14:10 UTC
[Xen-users] drbd and xen problem: disk not accessible
Hi all, I have a problem with drbd and xen.
After setting up drbd and after I installed a new linux vm, I can not create
the vm because of a problem accessing the disk.
I read that a solution could be setting the kernel and ramdisk and root
parameter into the vm config file but it doesn''t work for me.
What do you suggest me?
I get:
Using config file "/etc/xen/VMNAME".
No handlers could be found for logger "xend"
Error: Disk isn''t accessible
Usage: xm create <ConfigFile> [options] [vars]
...
I''m using this packages:
xen-3.0.3-41.el5
kernel-xen-2.6.18-8.el5
kmod-drbd82-xen-8.2.5-1.2.6.18_53.1.14.el5
xen-libs-3.0.3-41.el5
xen-libs-3.0.3-41.el5
kernel-xen-2.6.18-53.1.14.el5
drbd82-8.2.5-1.el5.centos
kmod-drbd82-xen-8.2.5-1.2.6.18_53.1.14.el5
my drbd.conf is:
global {
usage-count yes;
}
common {
protocol C;
startup {
become-primary-on both;
}
net {
allow-two-primaries;
}
disk {
on-io-error detach;
}
syncer {
verify-alg md5;
rate 50M;
}
}
resource r0 {
on hyp11.infolan {
device /dev/drbd0;
disk /dev/HYP11VM/INT;
address 10.100.0.2:7788;
meta-disk internal;
}
on hyp10.infolan {
device /dev/drbd0;
disk /dev/HYP10VM/INT;
address 10.100.0.1:7788;
meta-disk internal;
}
}
and the vm configuration file is:
name = "VMNAME"
uuid = "8f1e2a13-f0c1-fbf5-a1f2-2a71f8c7c144"
maxmem = 512
memory = 512
vcpus = 1
bootloader = "/usr/bin/pygrub"
on_poweroff = "destroy"
on_reboot = "restart"
on_crash = "restart"
vfb = [ ]
disk = [ "drbd:r0,xvda,w" ]
vif = [ "mac=00:16:3e:60:7d:41,bridge=xenbr0" ]
kernel=''/boot/vmlinuz-2.6.9-67.0.7.ELxenU''
ramdisk=''/boot/initrd-2.6.9-67.0.7.ELxenU.img''
root=''/dev/VolGroup00/LogVol00 ro''
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Tiago Cruz
2008-Apr-23 14:12 UTC
Re: [Xen-users] drbd and xen problem: disk not accessible
On Wed, 2008-04-23 at 16:10 +0200, Marco Strullato wrote:> device /dev/drbd0;> disk = [ "drbd:r0,xvda,w" ]If you try to use "/dev/drbr0" on xen file configuration? -- Tiago Cruz http://everlinux.com Linux User #282636 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Marco Strullato
2008-Apr-23 14:23 UTC
Re: [Xen-users] drbd and xen problem: disk not accessible
If I use this configuration file the vm boots but I don''t know it this is the right way... into the drbd guide http://www.drbd.org/users-guide/s-xen-configure-domu.html it is not suggested to use this configuration name = "VMNAME" uuid = "8f1e2a13-f0c1-fbf5-a1f2-2a71f8c7c144" maxmem = 512 memory = 512 vcpus = 1 bootloader = "/usr/bin/pygrub" on_poweroff = "destroy" on_reboot = "restart" on_crash = "restart" vfb = [ ] disk = [ "phy:/dev/drbd0,xvda,w" ] #disk = [ "drbd:r0,xvda,w" ] vif = [ "mac=00:16:3e:60:7d:41,bridge=xenbr0" ] #kernel=''/boot/vmlinuz-2.6.9-67.0.7.ELxenU'' #ramdisk=''/boot/initrd-2.6.9-67.0.7.ELxenU.img'' #root=''/dev/VolGroup00/LogVol00 ro'' 2008/4/23, Tiago Cruz <tiagocruz@forumgdh.net>:> > On Wed, 2008-04-23 at 16:10 +0200, Marco Strullato wrote: > > > device /dev/drbd0; > > > > disk = [ "drbd:r0,xvda,w" ] > > > If you try to use "/dev/drbr0" on xen file configuration? > > > -- > Tiago Cruz > http://everlinux.com > Linux User #282636 > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Ross S. W. Walker
2008-Apr-23 14:26 UTC
RE: [Xen-users] drbd and xen problem: disk not accessible
Marco Strullato wrote:> > Hi all, I have a problem with drbd and xen. > After setting up drbd and after I installed a new linux vm, I > can not create the vm because of a problem accessing the disk. > I read that a solution could be setting the kernel and > ramdisk and root parameter into the vm config file but it > doesn''t work for me. > > What do you suggest me? > > I get: > Using config file "/etc/xen/VMNAME". > No handlers could be found for logger "xend" > Error: Disk isn''t accessible > Usage: xm create <ConfigFile> [options] [vars] > ... > > I''m using this packages: > xen-3.0.3-41.el5 > kernel-xen-2.6.18-8.el5 > kmod-drbd82-xen-8.2.5-1.2.6.18_53.1.14.el5 > xen-libs-3.0.3-41.el5 > xen-libs-3.0.3-41.el5 > kernel-xen-2.6.18-53.1.14.el5 > drbd82-8.2.5-1.el5.centos > kmod-drbd82-xen-8.2.5-1.2.6.18_53.1.14.el5 > > my drbd.conf is: > global { > usage-count yes; > } > common { > protocol C; > startup { > become-primary-on both; > } > net { > allow-two-primaries; > } > disk { > on-io-error detach; > } > syncer { > verify-alg md5; > rate 50M; > } > > } > resource r0 { > on hyp11.infolan { > device /dev/drbd0; > disk /dev/HYP11VM/INT; > address 10.100.0.2:7788; > meta-disk internal; > } > on hyp10.infolan { > device /dev/drbd0; > disk /dev/HYP10VM/INT; > address 10.100.0.1:7788; > meta-disk internal; > } > } > > and the vm configuration file is: > name = "VMNAME" > uuid = "8f1e2a13-f0c1-fbf5-a1f2-2a71f8c7c144" > maxmem = 512 > memory = 512 > vcpus = 1 > bootloader = "/usr/bin/pygrub" > on_poweroff = "destroy" > on_reboot = "restart" > on_crash = "restart" > vfb = [ ] > disk = [ "drbd:r0,xvda,w" ]^^^^^^^ Where did you get this config info from, as far as I know it should be: disk = [ "phy:drbd0,xvda,w" ]> vif = [ "mac=00:16:3e:60:7d:41,bridge=xenbr0" ] > kernel=''/boot/vmlinuz-2.6.9-67.0.7.ELxenU'' > ramdisk=''/boot/initrd-2.6.9-67.0.7.ELxenU.img'' > root=''/dev/VolGroup00/LogVol00 ro''Setting drbd resources per-lv will become very messy over time as the number of guests increase. I advise replicating PVs between hosts and adding the drbdX resources to shared VGs. If you maintain all your VMs in xenstore then the VM config should migrate between machines so you don''t need to have the VM config present on both machines beforehand and therefore you will not be able to start the same VM on 2 machines at once and therefore shouldn''t need to run CLVM which would get in the way of VM migration. -Ross ______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Ross S. W. Walker
2008-Apr-23 14:28 UTC
RE: [Xen-users] drbd and xen problem: disk not accessible
Marco Strullato wrote:> > If I use this configuration file the vm boots but I don''t > know it this is the right way... into the drbd guide > http://www.drbd.org/users-guide/s-xen-configure-domu.html it > is not suggested to use this configurationAre you running Xen 3.2 though? I believe only 3.2 has added the drbd type. <snip> ______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Marco Strullato
2008-Apr-23 14:35 UTC
Re: [Xen-users] drbd and xen problem: disk not accessible
> > disk = [ "drbd:r0,xvda,w" ] > > ^^^^^^^ > Where did you get this config info from, as far as I > know it should be: > > disk = [ "phy:drbd0,xvda,w" ]I get this from this guide: http://www.drbd.org/users-guide/s-xen-configure-domu.html> Setting drbd resources per-lv will become very messy > over time as the number of guests increase. I advise > replicating PVs between hosts and adding the drbdX > resources to shared VGs. If you maintain all your > VMs in xenstore then the VM config should migrate > between machines so you don''t need to have the VM > config present on both machines beforehand and > therefore you will not be able to start the same > VM on 2 machines at once and therefore shouldn''t > need to run CLVM which would get in the way of VM > migration. >This is interesting: could you be more precise please? How do you suggest me to set up the envirorment? Thanks Marco _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Marco Strullato
2008-Apr-23 14:36 UTC
Re: [Xen-users] drbd and xen problem: disk not accessible
> Are you running Xen 3.2 though? I believe only 3.2 has added the > drbd type.I''m using this packages: xen-3.0.3-41.el5 kernel-xen-2.6.18-8.el5 kmod-drbd82-xen-8.2.5-1.2.6.18_53.1.14.el5 xen-libs-3.0.3-41.el5 xen-libs-3.0.3-41.el5 kernel-xen-2.6.18-53.1.14.el5 drbd82-8.2.5-1.el5.centos kmod-drbd82-xen-8.2.5-1.2.6.18_53.1.14.el5 Marco _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Ross S. W. Walker
2008-Apr-23 14:46 UTC
RE: [Xen-users] drbd and xen problem: disk not accessible
Marco Strullato wrote:> > > Are you running Xen 3.2 though? I believe only 3.2 has added the > > drbd type. > > I''m using this packages: > xen-3.0.3-41.el5 > kernel-xen-2.6.18-8.el5 > kmod-drbd82-xen-8.2.5-1.2.6.18_53.1.14.el5 > xen-libs-3.0.3-41.el5 > xen-libs-3.0.3-41.el5 > kernel-xen-2.6.18-53.1.14.el5 > drbd82-8.2.5-1.el5.centos > kmod-drbd82-xen-8.2.5-1.2.6.18_53.1.14.el5The answer is no, if you want to stick with RH/CentOS supplied packages change it to ''phy:drbd0,xvda,w'', if you want to use Xen 3.2 (really not worth it if all you want is drbd: type) then you will need to download it from xen.org. If you are running 64-bit I advise Xen 3.2 as it will allow 32-bit domUs on 64-bit dom0 to work properly, but unfortunately you will need to compile it from xen.org''s SRPM as they only provide 32-bit binaries. -Ross ______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Ross S. W. Walker
2008-Apr-23 15:13 UTC
RE: [Xen-users] drbd and xen problem: disk not accessible
Marco Strullato wrote:> > > disk = [ "drbd:r0,xvda,w" ] > > > > ^^^^^^^ > > Where did you get this config info from, as far as I > > know it should be: > > > > disk = [ "phy:drbd0,xvda,w" ] > > > I get this from this guide: > http://www.drbd.org/users-guide/s-xen-configure-domu.html > > > > Setting drbd resources per-lv will become very messy > > over time as the number of guests increase. I advise > > replicating PVs between hosts and adding the drbdX > > resources to shared VGs. If you maintain all your > > VMs in xenstore then the VM config should migrate > > between machines so you don''t need to have the VM > > config present on both machines beforehand and > > therefore you will not be able to start the same > > VM on 2 machines at once and therefore shouldn''t > > need to run CLVM which would get in the way of VM > > migration. > > > > This is interesting: could you be more precise please? > How do you suggest me to set up the envirorment?You will need to do this with Xen 3.2 as I do not believe that RH/CentOS libvirt will migrate the VM config! Drbd setups are really only good for 2 servers, if you want more Xen servers then 2 then look into an iSCSI implementation. So, say we have 2 servers, they have 2 SATA disks in a mirror for OS, and 4 SATA disks in a RAID5/6/50/60/10 for guests. Both servers identical. Lets say /dev/sda is OS mirror and /dev/sdb is guest RAID (could be /dev/mdX for OS and /dev/mdX+1 for RAID if doing software RAID). Create a drbd resource out of /dev/sdb on both hosts set as primary-primary and sync it up fully. Test it make sure drbd is properly setup and performing as it should. NOTE: You will need to keep the meta-data on a separate disk in drbd! I advise creating a LV on the OS mirror and putting the drbd meta-data there. Next make /dev/drbd0 a pv with pvcreate, create a VG out of the PV, then on the other host do a pvscan, and import the VG there. Then you can create an LV on host1, lvscan on host2 and it should appear there. Test it out back and forth until your comfortable it works as advertised. Create a domU config file and test it out with your first LV, make sure it boots and installs correctly, then when you are comfortable with it import it into the xenstore with ''xm new <config file>'', after that the VM should appear in ''xm list'' all the time until it is migrated to another host, whether it is up or down it will show up in the list. Then bring it up on host 1 with ''xm start <dom name>'' when the host is up and running you can do a ''xm migrate <dom name> <host name>'' and it should just move over to the second host. Once you are comfortable this works you might want to look into some kind of VM load-balancing/fail-over application. Maybe someone has a nice heartbeat with load balancing configs and scripts put together. The idea is to have the virtual machine servers self managing. -Ross ______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Ross S. W. Walker
2008-Apr-23 16:21 UTC
RE: [Xen-users] drbd and xen problem: disk not accessible
Ross S. W. Walker wrote:> Marco Strullato wrote: > > > > disk = [ "drbd:r0,xvda,w" ] > > > > > > ^^^^^^^ > > > Where did you get this config info from, as far as I > > > know it should be: > > > > > > disk = [ "phy:drbd0,xvda,w" ] > > > > > > I get this from this guide: > > http://www.drbd.org/users-guide/s-xen-configure-domu.html > > > > > > > Setting drbd resources per-lv will become very messy > > > over time as the number of guests increase. I advise > > > replicating PVs between hosts and adding the drbdX > > > resources to shared VGs. If you maintain all your > > > VMs in xenstore then the VM config should migrate > > > between machines so you don''t need to have the VM > > > config present on both machines beforehand and > > > therefore you will not be able to start the same > > > VM on 2 machines at once and therefore shouldn''t > > > need to run CLVM which would get in the way of VM > > > migration. > > > > > > > This is interesting: could you be more precise please? > > How do you suggest me to set up the envirorment? > > You will need to do this with Xen 3.2 as I do not > believe that RH/CentOS libvirt will migrate the VM > config! > > Drbd setups are really only good for 2 servers, if you > want more Xen servers then 2 then look into an iSCSI > implementation.You can by the way convert this existing setup to iSCSI by turning these 2 servers into iSCSI targets and exporting the /dev/drbd0 PV on each to say Xen blade servers setup with multi-path fail-over (don''t do it round-robin as the drbd latency and io ordering will get in the way!).> So, say we have 2 servers, they have 2 SATA disks in > a mirror for OS, and 4 SATA disks in a RAID5/6/50/60/10 > for guests. Both servers identical. > > Lets say /dev/sda is OS mirror and /dev/sdb is guest > RAID (could be /dev/mdX for OS and /dev/mdX+1 for RAID > if doing software RAID). > > Create a drbd resource out of /dev/sdb on both hosts > set as primary-primary and sync it up fully. Test it > make sure drbd is properly setup and performing as it > should. > > NOTE: You will need to keep the meta-data on a separate > disk in drbd! I advise creating a LV on the OS mirror > and putting the drbd meta-data there.After some thought, this isn''t a requirement, but adviseable anyway to allow proper VG resizing and to make sure the meta-data io doesn''t collide with the disk io.> Next make /dev/drbd0 a pv with pvcreate, create a VG > out of the PV, then on the other host do a pvscan, and > import the VG there. > > Then you can create an LV on host1, lvscan on host2 > and it should appear there. Test it out back and > forth until your comfortable it works as advertised. > > Create a domU config file and test it out with your > first LV, make sure it boots and installs correctly, > then when you are comfortable with it import it into > the xenstore with ''xm new <config file>'', after that > the VM should appear in ''xm list'' all the time until > it is migrated to another host, whether it is up or > down it will show up in the list. > > Then bring it up on host 1 with ''xm start <dom name>'' > when the host is up and running you can do a > ''xm migrate <dom name> <host name>'' and it should > just move over to the second host. > > Once you are comfortable this works you might want to > look into some kind of VM load-balancing/fail-over > application. Maybe someone has a nice heartbeat with > load balancing configs and scripts put together. The > idea is to have the virtual machine servers self > managing. > > -Ross______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users