CentOS virt - Feb 2010 - Can I use direct attached storage as a shared filesystem in Xen

I have a quad core server in which I want to run 4 virtual servers. On this
server I have a 1/2 terabyte raid 1 I have split between the 4 members that
have the OS on it. I have  raid 5 10 terabyte internal storage running on a
3ware 9690a card.  I want to share this storage between the servers without
partitioning it.  Is this possible?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.centos.org/pipermail/centos-virt/attachments/20100208/5f13c6ae/attachment.html

On Mon, 2010-02-08 at 16:08 -0500, Rich wrote:
> I have a quad core server in which I want to run 4 virtual servers. On
> this server I have a 1/2 terabyte raid 1 I have split between the 4
> members that have the OS on it. I have  raid 5 10 terabyte internal
> storage running on a 3ware 9690a card.  I want to share this storage
> between the servers without partitioning it.  Is this possible? 
One option, if you want VMs to share the files on the large disk, would
be to set up CIFS/NFS.


Andri

either nfs or ocfs2. nfs is the easiest route. ocfs2 will give you a
clustered filesystem.



-Adam

On Mon, Feb 8, 2010 at 4:24 PM, Andri M?ll <andri at dot.ee>
wrote:
> On Mon, 2010-02-08 at 16:08 -0500, Rich wrote:
>> I have a quad core server in which I want to run 4 virtual servers. On
>> this server I have a 1/2 terabyte raid 1 I have split between the 4
>> members that have the OS on it. I have ?raid 5 10 terabyte internal
>> storage running on a 3ware 9690a card. ?I want to share this storage
>> between the servers without partitioning it. ?Is this possible?
>
> One option, if you want VMs to share the files on the large disk, would
> be to set up CIFS/NFS.
>
>
> Andri
>
> _______________________________________________
> CentOS-virt mailing list
> CentOS-virt at centos.org
> http://lists.centos.org/mailman/listinfo/centos-virt
>

>I have a quad core server in which I want to run 4 virtual
servers. On this server I have a 1/2 terabyte raid 1 I have
split between the 4 members that have the OS on it. I have
raid 5 10 terabyte internal storage running on a 3ware 9690a
card.  I want to share this storage between the servers
without partitioning it.  Is this possible? 
 
 
You can carve up chunks of it to use as block device storage
for individual virtual machines. Doing it that way doesn't
require any specific partition on the volume. You just
include the volume in an LVM group. This would allow you to
serve the chunks from the virtual machines in various ways.
(ntfs, ext3, iscsi, etc)
 
I'm pretty sure any other way would require a large
partition on some kind. Like ext3, or whichever one you
like.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.centos.org/pipermail/centos-virt/attachments/20100208/8466c166/attachment-0001.html

Err, I meant to say:

*This would allow you to serve the chunks from the virtual
machines in various ways. (cifs, nfs, iscsi, etc)

----- "Adam Adamou" <adam0x54 at gmail.com> wrote:
> either nfs or ocfs2. nfs is the easiest route. ocfs2 will give you a
> clustered filesystem.
Except NFS doesn't follow normal filesystem semantics and you can end up
with corrupt data without knowing it, and it, along with CIFS, will give you a
free shitload of network overhead to go along with your possibly corrupt data.
OCFS2 or GFS are the only practical choices if you want it to behave like a
typical filesystem and not have to worry about catering to it or rewriting
software and/or reeducating developers, and OCFS2 is extremely easy to set up.

The original question didn't specify much about the requirements, though. A
single shared filesystem? Read-write or read-only? No filesystem at all? Without
that information, I would at first recommend not sharing. It can be a lot of
trouble, it's usually not required, and it severely complicates life when
things fail.

Well, there is always XenFS... :/

-- 
Christopher G. Stach II
http://ldsys.net/~cgs/

----- "Rich" <rhdyes at gmail.com> wrote:
> I think the better question is this. How can I make my 2 physical
> servers that are identical into the best clustered solution sharing
> the storage between the 2.
Do you want the storage to be redundant between the two like RAID 1 or just all
of it accessible to any guest on either machine like a JBOD? For either, you can
use a suitable mix of drbd, iSCSI, GNBD, LVM, heartbeat, keepalived, and a
clustered filesystem. If you do the latter, remember that this squares your
risk. If you want a simple A/B failover setup where the second physical host is
idle until the first physical host fails, you can just use drbd and heartbeat or
RHCS.
> They all use xfs.
> I want to keep the guests the same on each server.
> I will start with the backup server. I will go on each vm and delete
> this FS and block device.
> I then want make that whole 10 terabyte raid into one gfs and then
> re-attach it back to each guest.
> Is that possible or do I have to start all over again? If it can be
> done I then will sync the date from the mainserver to this backup
> server and then switch them and do the same to the mainserver.
That sounds about right. You need to make your backups (!), destroy the data on
of your servers by creating the new clustered filesystem, copy the data from the
other server, and then share it with your reconfigured guests. So, basically
what you said.
> I have one more question which is another whole thread.
> I am using rsync to sync the 2 physical boxes now. Is there a better
> way to do this?
drbd, but rsync does a pretty good job in a lot of use cases. I wouldn't say
that it fits this one, though.

-- 
Christopher G. Stach II
http://ldsys.net/~cgs/

----- "Rich" <rhdyes at gmail.com> wrote:
> I am doing rsync now. What is the difference between ocf2 and gfs?
There are comparisons on the web, but in short, cluster size, GFS can be a major
pain in the ass for some people to set up (fencing support is usually said
pain), and GFS has distribution support. For such a small installation, you are
probably better off with OCFS2. If you need something that is almost exactly the
same as a local filesystem and depend upon certain filesystem behaviors for
whatever reason (locking comes to mind), you will be better off with GFS.

-- 
Christopher G. Stach II
http://ldsys.net/~cgs/