CentOS virt - Feb 2010 - Best practices for LVM and virtualization

Hello,

I'm currently setting up a virtualization infrastructure based on
CentOS 5.4 x86_64 and KVM.

I want to use LVM because of its flexibility, especially the ability
to add more disk space to a guest if needed.
We are considering using an hardware RAID1 with BBU as the lowest layer.

My question is: where is it recommended to put the LVM?

1. On the host only
In that case, the guests would see logical volumes as hard drives and
would be installed with "plain" partitions (ext3 / swap)
When more space is needed the logical volume (from host perspective) /
drive (from guest perspective) would simply become bigger.
This feels simple, but I wonder whether there may be side effects,
since real harddrive cannot grow, and maybe some parts of the system
expect them to stay of a given size.

2. On the guest only
On the host, there would only be "plain" raw partitions, which the
guest would see as hard drives.
The guest would be installed with an LVM and when more space is
needed, an additional partition (from host perspective) / drive (from
guest perspective) would be added to the guest volume group, which
would then become bigger, allowing logical volumes to grow.
I'm not too comfortable with that, since my understanding of the role
of the host is to manage resources for the guests: memory, CPUs so I'd
like to have some powerful abstraction there for disk space as well,
which is one purpose of LVM. The guest would then need to be restarted
when space is added.

3. On both host and guest
LVM would be installed on the host, which would then provide the
logical volumes as hard drives for the guest.
The guest would use them as additional drives to be added to its own
volume group when it needs to grow.
This feels the most flexible, but I wonder whether it is not wrong to
add LVM layers upon LVM layers, from a
performance/stability/simplicity perspective. The guest would probably
have to be restarted when more space is needed (I've not tested this
extensively yet)

I would be very interested to have your opinion / experience about
this before I go further in my testing!

Cheers,

Mathieu

----- "Mathieu Baudier" <mbaudier at argeo.org> wrote:
> 1. On the host only
This always turns into a total mess when you have anything more than a tiny
installation.
> 2. On the guest only
This one just makes no sense. :)
> 3. On both host and guest
This is what I always use and recommend. It doesn't have any side effects
with modern software versions, except with layered MD RAID on the host. (I
don't know if that has been fixed yet, but I don't think it has.) The
only issue is that there is an extra step or two when you resize guest
partitions and you're out of PV space in the guest. This also doesn't
lock you into this way of doing things. You can always add more targets a la #1.

-- 
Christopher G. Stach II
http://ldsys.net/~cgs/