Nyers, Gabor
2007-Nov-29 10:41 UTC
[Xen-users] FW: Advice on HighAvailable/Failover Xen machines
For some reason my below post haven''t appeared on the list even after a day, so I''m re-sending it directly to the involved in the thread. Regards, Gábor Nyers -----Original Message----- From: Nyers, Gabor Sent: woensdag 28 november 2007 13:18 To: ''xen-users@lists.xensource.com'' Subject: RE: Advice on HighAvailable/Failover Xen machines Hi, Based on my own experience with EVMS and Heartbeat together, I share Florian''s assessment: EVMS can be a pain in the butt. But once you learn its quirks it does the job. I found multipathing one of the near-showstoppers with EVMS. It took a while to figure out the solution for different seemingly random problems: just let EVMS ignore all /dev/sd* devices and use only multipathed devs in /dev/mapper/* (same goes btw for LVM2 as well) Depending on what you want to achieve, having a cluster aware volume manager is not a requisite for running highly available Xen domUs. As Florian pointed out, you can just put disk images on OCFS2 and run the domUs from there. This has also worked for me quite well. Even the setup what you''re trying, where each VM has its own logical volume, does not necessarily require cluster aware volume manager. That is, if your LV layout is fairly static, even with LVM2 your domUs will run just fine. Sure, if you add or change a volume, the other node won''t see it immediately. With careful planning it can be done. I tried this and every time I let LVM2 rescan the volumes, it found out about the new layout. This might, however, not be the safest way to do it. On the other hand EVMS destroyed the layout on more than one occasion. The option I finally decided on to be perhaps the most flexible, is where each domU also has its own dedicated LUN. On these LUNs I used old-school PC partition table. No LVs, so no need for cluster aware volume manager. It''s more simple, more compartmentized and the trade-offs are something I could live with. One example for this is that you''ll need more LUNs. But since it''s a SAN, who cares if it''s 3 LUNs or 30, right? (unless of course your specific storage system has limitations on the nr. of LUNs) Resizing filesystems is also more involved this way than with LVs. Btw: if you take my tip: don''t put anything but OS related stuff on the domU''s disk. Especially databases should always have their dedicated disks. The Heartbeat resource agent for Xen does provide you with the automatic restart or fail-over of a domU. Another nice thing Heartbeat provides is that it''s supports up to 16 nodes. (I''ve stuck with 4 btw.) You don''t mention the distro that you''re using, but if it''s SLES10, Novell will provide you with enterprise grade support for any of the above configurations. Hope this helps. Regards, Gábor Nyers -----Original Message----- Date: Wed, 28 Nov 2007 12:24:19 +0100 From: "Florian Heigl" <florian.heigl@gmail.com> Subject: Re: [Xen-users] Advice on HighAvailable/Failover Xen machines To: xen-users@lists.xensource.com Message-ID: <77abe410711280324i14c7e35et8045605bdc346159@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Hi, 2007/11/28, Maximilian Wilhelm <max@rfc2324.org>:> Hi! > > I want to accomplish the following setup: > Two (maybe leter more) Xen systems running on two machines with HBAs > inside connected to a SAN. > > I would like to place the DomU block-devices on logical volumes on > top of a LUN of the SAN which is available at both (all) Dom0s. > > I read about the possibiliy to use EVMS as one solutions which could > fullfil my needs. > > Is this a good idea? > How could I do some kind of failover in case one Dom0 has gone? > Can Xen detect this and move the affected DomUs automagically? > > Any hints how to set something like this up would be highly > appreaciated.I''ve played around a lot on this path. With EMVS you can either use the cluster segment manager or the OCFS2 Plugin with SAN Storage, either has it''s advantages, both will nicely integrate with Linux-HA / heartbeat. EVMS also is greatly helpful for fully automated/scripted storage setup etc which comes to mind for HA setups. But, now for the downside: EVMS is unfortunately quite dead, the OCFS2 plugin never made it into mainline, the heartbeat integration needs patches, the BBR patches need dm patches for the dom0 kernel, there is almost no irc support left, the devs are.. well, i don''t know where AND as it was designed by IBM gurus it is really well thought through and perfectly structured, but INCREDIBLY annoying to use! Once you go into larger Xen setups you might have to handle 100s of volumes and on occassion it might be manually - personally i think the EVMS management is unbearable. I still run my system on EVMS and i don''t really see a good alternative for a logical-volume based attempt (yes clvm exists but, as always, can''t mirror LVs in a feasible way) for a new HA system i''d try to tie together opensharedroot(.org), ocfs2 and the san, booting from ocfs2 root, with another volume(s) for the domU images. But that implies you don''t use a JBOD on the san but higher end (active-active) storage. if your site has SDRF-ed EMC^2 Symmetrix that''s no issue, otherwise it might well turn into one. Regards, Florian -- ''Sie brauchen sich um Ihre Zukunft keine Gedanken zu machen _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users