Hi list, I installed two new servers to be our virtualization HA. This servers have CentOS 5.2, Xen 3.0.3, Drbd 8.2.6. The setup is ok, lvm drbd resources sincronizing, Guest VM running without any problem. My only doubt is that I can only use drbd resources inside guest vms when I specify the disk resource in vms config like this: disk = ['phy:/dev/VolGroup00/vappprtem01,hda,w' ] if I specify de disk like: disk = [drbd:vappprtem01,hda,w' ] The Guest VM found the disk but this appear with zero size. As far what I read the second way is the correct to live migration work without any race condiction... This link say that centos has a problem with block-drbd script provided by drbd, is anybody aware of that ? http://lists.xensource.com/archives/html/xen-users/2008-08/msg00176.html Any hints ? Best Regards,
On 8/25/08, Alexandre Biancalana <biancalana@gmail.com> wrote: > Hi list, > > I installed two new servers to be our virtualization HA. This servers > have CentOS 5.2, Xen 3.0.3, Drbd 8.2.6. The setup is ok, lvm drbd > resources sincronizing, Guest VM running without any problem. > > My only doubt is that I can only use drbd resources inside guest vms > when I specify the disk resource in vms config like this: > > disk = ['phy:/dev/VolGroup00/vappprtem01,hda,w' ] > > if I specify de disk like: > > disk = [drbd:vappprtem01,hda,w' ] > > The Guest VM found the disk but this appear with zero size. > > As far what I read the second way is the correct to live migration > work without any race condiction... > > This link say that centos has a problem with block-drbd script > provided by drbd, is anybody aware of that ? > > http://lists.xensource.com/archives/html/xen-users/2008-08/msg00176.html This is a know problem ??? Any ideas ??? Regards,
Hi, after i've read this post http://article.gmane.org/gmane.comp.emulators.xen.user/33157/match=re+xen+drbd I've given up on xen and drbd disks for now, and am using phy:, but you'll have to give it drbd device not the volume disk = [ "phy:/dev/drbd14,xvda,w" ] Unfortunately, that means that you'll have to handle setting drbd devices primary/secondary yourself. I've written a wrapper script around drbdadm+xm and it has been working more or less ok. That also means no live migration since drbd device needs to change state sometime mid-migration, but save/move/restore has been working ok for me with really minimal downtimes. There is also a primary/primary setup with newer drbd but it was awfully unstable when i was testing it a few moths ago, most of the time it was crashing and taking down the complete cluster with it. regards, Alexandre Biancalana wrote: > Hi list, > > I installed two new servers to be our virtualization HA. This servers > have CentOS 5.2, Xen 3.0.3, Drbd 8.2.6. The setup is ok, lvm drbd > resources sincronizing, Guest VM running without any problem. > > My only doubt is that I can only use drbd resources inside guest vms > when I specify the disk resource in vms config like this: > > disk = ['phy:/dev/VolGroup00/vappprtem01,hda,w' ] > > if I specify de disk like: > > disk = [drbd:vappprtem01,hda,w' ] > > The Guest VM found the disk but this appear with zero size. > > As far what I read the second way is the correct to live migration > work without any race condiction... > > This link say that centos has a problem with block-drbd script > provided by drbd, is anybody aware of that ? > > http://lists.xensource.com/archives/html/xen-users/2008-08/msg00176.html > > Any hints ? > > Best Regards, > _______________________________________________ > CentOS-virt mailing list > CentOS-virt@centos.org > http://lists.centos.org/mailman/listinfo/centos-virt > -- Aleksandar Ivanisevic Support Manager 2e Systems Tel: +49 - 6196 - 950 58 14 Fax: +49 - 6196 - 950 58 94 E-mail: Aleksandar.Ivanisevic@2e-systems.com Address: 2e Systems GmbH, Im Taubengrund 12, D-65451 Kelsterbach Company registration: Amtsgericht Darmstadt (Germany), HRB 84110 Directors: Philip Douglas, Alistair Forbes http://www.2e-systems.com - making your business fly!
Aleksandar Ivanisevic ha scritto: > > Hi, > > after i've read this post > > http://article.gmane.org/gmane.comp.emulators.xen.user/33157/match=re+xen+drbd > > > I've given up on xen and drbd disks for now, and am using phy:, but > you'll have to give it drbd device not the volume > > disk = [ "phy:/dev/drbd14,xvda,w" ] > > Unfortunately, that means that you'll have to handle setting drbd > devices primary/secondary yourself. I've written a wrapper script around > drbdadm+xm and it has been working more or less ok. > > That also means no live migration since drbd device needs to change > state sometime mid-migration, but save/move/restore has been working ok > for me with really minimal downtimes. > > There is also a primary/primary setup with newer drbd but it was awfully > unstable when i was testing it a few moths ago, most of the time it was > crashing and taking down the complete cluster with it. > > regards, > Hi Aleksandar, I was just starting doing some test following this article: http://fghaas.wordpress.com/2007/09/03/drbd-806-brings-full-live-migration-fo... So if you say that doesn't work I'll give up on this and I'll go directly to Openvz (actually proxmox) Thanks Lorenzo
Lorenzo Quatrini wrote: > I was just starting doing some test following this article: > > http://fghaas.wordpress.com/2007/09/03/drbd-806-brings-full-live-migration-fo... > > So if you say that doesn't work I'll give up on this and I'll go directly to > Openvz (actually proxmox) Eh, never trust someone you met over the Internet ;) I've actually used that same page as a starting point for my tests and it's definitely not usable for me. Don't know much about proxmox but it seems to be the right approach, use openvz for stuff that is distro/kernel/OS agnostic and KVM/XEN/whatever for full virtualization. I'm now trying to achieve the same thing using openvz-xen kernel, but only not to waste all the work i've invested in getting the xen cluster going. Maybe I'll move to kvm after all, but I don't know yet if it would read xen disk images.
Alexandre Biancalana wrote: > > Hi list, > > I installed two new servers to be our virtualization HA. This servers > have CentOS 5.2, Xen 3.0.3, Drbd 8.2.6. The setup is ok, lvm drbd > resources sincronizing, Guest VM running without any problem. > > My only doubt is that I can only use drbd resources inside guest vms > when I specify the disk resource in vms config like this: > > disk = ['phy:/dev/VolGroup00/vappprtem01,hda,w' ] > > if I specify de disk like: > > disk = [drbd:vappprtem01,hda,w' ] > > The Guest VM found the disk but this appear with zero size. > > As far what I read the second way is the correct to live migration > work without any race condiction... > > This link say that centos has a problem with block-drbd script > provided by drbd, is anybody aware of that ? > > http://lists.xensource.com/archives/html/xen-users/2008-08/msg00176.html > > Any hints ? Just specify the drbd devices directly as: disk = ['phy:/dev/drbd0,hda,w' ] And you should be ok. There is no need to use the scripts. Be sure you don't specify the underlying device, but the drbd device! Using the underlying device will cause the replicas to become out-of-sync as what is written to the underlying device will not be replicated and a consistency check will bring your drbd resources offline! You are also going to want to use the multi-master as master-slave is too difficult to get to work properly between xend and heartbeat! -Ross ______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof.
Ross S. W. Walker wrote: > Just specify the drbd devices directly as: > > disk = ['phy:/dev/drbd0,hda,w' ] > > And you should be ok. There is no need to use the scripts. You actually got it working stably, live migration and all? which kernel, xen, drbd if I may ask?
On 8/26/08, Aleksandar Ivanisevic <aleksandar.ivanisevic@2e-systems.com> wrote: > Ross S. W. Walker wrote: > > > Just specify the drbd devices directly as: > > > > disk = ['phy:/dev/drbd0,hda,w' ] > > > > And you should be ok. There is no need to use the scripts. > > > > You actually got it working stably, live migration and all? which kernel, > xen, drbd if I may ask? What about that ? http://fghaas.wordpress.com/2007/09/03/drbd-806-brings-full-live-migration-fo...
Alexandre Biancalana wrote: > On 8/26/08, Aleksandar Ivanisevic > <aleksandar.ivanisevic@2e-systems.com> wrote: > > Ross S. W. Walker wrote: > > > > > Just specify the drbd devices directly as: > > > > > > disk = ['phy:/dev/drbd0,hda,w' ] > > > > > > And you should be ok. There is no need to use the scripts. > > > > > > > You actually got it working stably, live migration and > all? which kernel, > > xen, drbd if I may ask? > > What about that ? > > http://fghaas.wordpress.com/2007/09/03/drbd-806-brings-full-li > ve-migration-for-xen-on-drbd/ Are you running Xen 3.2 from xen.org? If not then specify your devices as the top portion of the blog talks about, then config drbd with the allow-two-primaries as the bottom talks about, and ignore the middle. If you upgrade to xen.org release of Xen then you can use the block-drbd script to specify by drbd resource name instead of device, but that's just fluff in stuff, and doesn't have anything to do with the ability to live migrate which is all handled by the allow-two-primaries option. -Ross ______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof.
On 8/26/08, Ross S. W. Walker <RWalker@medallion.com> wrote: > Alexandre Biancalana wrote: > > On 8/26/08, Aleksandar Ivanisevic > > <aleksandar.ivanisevic@2e-systems.com> wrote: > > > Ross S. W. Walker wrote: > > > > > > > Just specify the drbd devices directly as: > > > > > > > > disk = ['phy:/dev/drbd0,hda,w' ] > > > > > > > > And you should be ok. There is no need to use the scripts. > > > > > > > > > > You actually got it working stably, live migration and > > all? which kernel, > > > xen, drbd if I may ask? > > > > What about that ? > > > > http://fghaas.wordpress.com/2007/09/03/drbd-806-brings-full-li > > ve-migration-for-xen-on-drbd/ > > > Are you running Xen 3.2 from xen.org? > > If not then specify your devices as the top portion of the blog > talks about, then config drbd with the allow-two-primaries as > the bottom talks about, and ignore the middle. > > If you upgrade to xen.org release of Xen then you can use the > block-drbd script to specify by drbd resource name instead of > device, but that's just fluff in stuff, and doesn't have > anything to do with the ability to live migrate which is all > handled by the allow-two-primaries option. Are you saying that we can only use block-drbd script with Xen 3.2 ?
Alexandre Biancalana wrote: > On 8/26/08, Ross S. W. Walker <RWalker@medallion.com> wrote: > > Alexandre Biancalana wrote: > > > On 8/26/08, Aleksandar Ivanisevic > > > <aleksandar.ivanisevic@2e-systems.com> wrote: > > > > Ross S. W. Walker wrote: > > > > > > > > > Just specify the drbd devices directly as: > > > > > > > > > > disk = ['phy:/dev/drbd0,hda,w' ] > > > > > > > > > > And you should be ok. There is no need to use the scripts. > > > > > > > > > > > > > You actually got it working stably, live migration and > > > > all? which kernel, > > > > xen, drbd if I may ask? > > > > > > What about that ? > > > > > > http://fghaas.wordpress.com/2007/09/03/drbd-806-brings-full-li > > > ve-migration-for-xen-on-drbd/ > > > > > > Are you running Xen 3.2 from xen.org? > > > > If not then specify your devices as the top portion of the blog > > talks about, then config drbd with the allow-two-primaries as > > the bottom talks about, and ignore the middle. > > > > If you upgrade to xen.org release of Xen then you can use the > > block-drbd script to specify by drbd resource name instead of > > device, but that's just fluff in stuff, and doesn't have > > anything to do with the ability to live migrate which is all > > handled by the allow-two-primaries option. > > Are you saying that we can only use block-drbd script with Xen 3.2 ? It might work with the Xen 3.1 libraries too, but that's also irrelevant as CentOS/RHEL has decided to use the Xen 3.0.3 libraries (though they are using the Xen 3.1 hypervisor, go figure!). Linbit wrote the block-drbd script for Xen 3.1 (maybe 3.2) and up, but definitely not pre-3.1. For the definitive answer though I recommend posting on drbd-user. Someone there might even have a work-around to get it to work on earlier Xen libraries... But like I said it's just fluff and stuff and specifying the drbd device directly will work just the same and there is no fear of it breaking in the future if the Xen libs change or some feature in bash changes! -Ross ______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof.
On Tue, Aug 26, 2008 at 11:25 AM, Ross S. W. Walker <RWalker@medallion.com> wrote: > Alexandre Biancalana wrote: >> On 8/26/08, Ross S. W. Walker <RWalker@medallion.com> wrote: >> > If you upgrade to xen.org release of Xen then you can use the >> > block-drbd script to specify by drbd resource name instead of >> > device, but that's just fluff in stuff, and doesn't have >> > anything to do with the ability to live migrate which is all >> > handled by the allow-two-primaries option. True. In my case : CentOS 5.3, incl. kernel-xen and heartbeat, Xen.org 3.x, DRBD 8.2. It works, but has no cookie-cutter howto. >> Are you saying that we can only use block-drbd script with Xen 3.2 ? It's really not so much the Xen version, but the distro. > It might work with the Xen 3.1 libraries too, but that's also > irrelevant as CentOS/RHEL has decided to use the Xen 3.0.3 > libraries (though they are using the Xen 3.1 hypervisor, go > figure!). How did you determine they are different versions? > Linbit wrote the block-drbd script for Xen 3.1 (maybe 3.2) and > up, but definitely not pre-3.1. > > For the definitive answer though I recommend posting on > drbd-user. Someone there might even have a work-around to > get it to work on earlier Xen libraries... http://lists.linbit.com/pipermail/drbd-user/2008-August/010077.html jerry -- There's plenty of youth in America - it's time we find the "fountain of smart".
yes. i replied to myself. On Tue, Aug 26, 2008 at 10:56 PM, Jerry Amundson <jamundso@gmail.com> wrote: > On Tue, Aug 26, 2008 at 11:25 AM, Ross S. W. Walker > <RWalker@medallion.com> wrote: >> Alexandre Biancalana wrote: >>> On 8/26/08, Ross S. W. Walker <RWalker@medallion.com> wrote: >>> > If you upgrade to xen.org release of Xen then you can use the >>> > block-drbd script to specify by drbd resource name instead of >>> > device, but that's just fluff in stuff, and doesn't have >>> > anything to do with the ability to live migrate which is all >>> > handled by the allow-two-primaries option. > > True. In my case : CentOS 5.3, incl. kernel-xen and heartbeat, Xen.org > 3.x, DRBD 8.2. > It works, but has no cookie-cutter howto. > >>> Are you saying that we can only use block-drbd script with Xen 3.2 ? > > It's really not so much the Xen version, but the distro. more so, within the distro, the bootstrap method used for the domU >> It might work with the Xen 3.1 libraries too, but that's also >> irrelevant as CentOS/RHEL has decided to use the Xen 3.0.3 >> libraries (though they are using the Xen 3.1 hypervisor, go >> figure!). > > How did you determine they are different versions? oh, maybe the thread I just noticed in xen-users? http://lists.xensource.com/archives/html/xen-users/2008-08/msg00791.html where Pasi says, -> To be more clear about it, RHEL 5.2 version of Xen _hypervisor_ is 3.1.2 + -> patches. -> RHEL 5.2 kernel-xen is based on older Xen (3.0.3 iirc) kernel code. >> Linbit wrote the block-drbd script for Xen 3.1 (maybe 3.2) and >> up, but definitely not pre-3.1. >> >> For the definitive answer though I recommend posting on >> drbd-user. Someone there might even have a work-around to >> get it to work on earlier Xen libraries... > > http://lists.linbit.com/pipermail/drbd-user/2008-August/010077.html > > jerry > > -- > There's plenty of youth in America - it's time we find the "fountain of smart". > -- There's plenty of youth in America - it's time we find the "fountain of smart".
Jerry Amundson wrote: > > yes. i replied to myself. Talking to yourself is the first sign of insanity! The second being, disagreeing with the first. > On Tue, Aug 26, 2008 at 10:56 PM, Jerry Amundson <jamundso@gmail.com> wrote: > > On Tue, Aug 26, 2008 at 11:25 AM, Ross S. W. Walker <RWalker@medallion.com> wrote: > >> Alexandre Biancalana wrote: > >>> On 8/26/08, Ross S. W. Walker <RWalker@medallion.com> wrote: > >>> > If you upgrade to xen.org release of Xen then you can use the > >>> > block-drbd script to specify by drbd resource name instead of > >>> > device, but that's just fluff in stuff, and doesn't have > >>> > anything to do with the ability to live migrate which is all > >>> > handled by the allow-two-primaries option. > > > > True. In my case : CentOS 5.3, incl. kernel-xen and heartbeat, Xen.org > > 3.x, DRBD 8.2. > > It works, but has no cookie-cutter howto. > > > >>> Are you saying that we can only use block-drbd script with Xen 3.2 ? > > > > It's really not so much the Xen version, but the distro. > > more so, within the distro, the bootstrap method used for the domU More specifically 'pygrub' itself as mentioned below. > >> It might work with the Xen 3.1 libraries too, but that's also > >> irrelevant as CentOS/RHEL has decided to use the Xen 3.0.3 > >> libraries (though they are using the Xen 3.1 hypervisor, go > >> figure!). > > > > How did you determine they are different versions? > > oh, maybe the thread I just noticed in xen-users? > http://lists.xensource.com/archives/html/xen-users/2008-08/msg00791.html > where Pasi says, > -> To be more clear about it, RHEL 5.2 version of Xen _hypervisor_ is 3.1.2 + > -> patches. > -> RHEL 5.2 kernel-xen is based on older Xen (3.0.3 iirc) kernel code. Actually I found the version information through a 'xm info' which shows it's running 3.1, but the libraries and supporting apps are all 3.0.3. xen_major : 3 xen_minor : 1 xen_extra : .2-92.1.10.el5 xen_caps : xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p > >> Linbit wrote the block-drbd script for Xen 3.1 (maybe 3.2) and > >> up, but definitely not pre-3.1. > >> > >> For the definitive answer though I recommend posting on > >> drbd-user. Someone there might even have a work-around to > >> get it to work on earlier Xen libraries... > > > > http://lists.linbit.com/pipermail/drbd-user/2008-August/010077.html > > Ah so the culprit is pygrub, add that tidbit to my knowledge base. So if you define the kernel and ramdisk from external sources then the block script should work, but using pygrub, it only recognizes standard block devices. I suppose this will also affect the block-iscsi script out there too... It would be nice if someone could patch pygrub to look for handler scripts for any device NOT file, tap, phy and to use them to bootload as it's better IMHO to manage the domU's kernel and ramdisk from within the domU. -Ross ______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof.