libvirt users - Mar 2017 - migrating from XenServer / XCP to libvirt/KVM

I've been using XenServer / XCP (with the "xe" toolset) for a
number of
years for hosting servers used for development and free software
projects and I'm now looking at migrating all those environments to
libvirt / KVM.

I had a look at the wiki[1] already and didn't see XenServer mentioned
there.

Could anybody help comment on or explain a few things:

- comparison between XenServer and libvirt: I notice many similar
concepts, for example, XenServer has storage repositories and libvirt
has storage pools.  However, in many examples such as this[2] it appears
that libvirt requires more manual effort.  For example, when you create
a storage repository in XenServer, the tool takes a physical block
device as input and creates the necessary volume group, logical volume
and a filesystem.  In that libvirt example, it appears each of those
steps must be done manually.  The networking examples are similar: the
XenServer tools do all the Open vSwitch stuff behind the scenes but in
the libvirt examples it appears necessary to create the bridge manually
before telling libvirt about it.  Is this all correct or are the
examples I've seen out of date?  Do more recent (or future) releases of
libvirt aim to automate/hide more of these things?

- what is best practice for virtual disk images?  Does libvirt always
use files (like XenServer) or can/should block devices be used
directly?  If using files, is any special care needed to avoid block
alignment problems?

- are there any guides or tools recommended for migrating small
XenServer environments (less than 50 domains on a single physical node)
to libvirt + KVM?

- is there a list of small things that need to change in a VM before
running it under libvirt / KVM?  I'm guessing this might include
bootloader config, updating the kernel, changing block device names in
/etc/fstab and changing network device names in scripts - are there any
others?

- can anybody comment on more tricky issues that may arise in such a
migration, for example, will a Windows VM be likely to run without
modification when migrated from XenServer to libvirt/KVM or will it need
extra drivers added or anything else before the migration?

Regards,

Daniel


1. http://wiki.libvirt.org/page/Main_Page
2. https://keepingitclassless.net/2014/01/libvirt-intro-basic-configuration/

On Wed, Mar 15, 2017 at 10:10:36AM +0000, Daniel Pocock
wrote:
> 
> I've been using XenServer / XCP (with the "xe" toolset) for a
number of
> years for hosting servers used for development and free software
> projects and I'm now looking at migrating all those environments to
> libvirt / KVM.
> 
> I had a look at the wiki[1] already and didn't see XenServer mentioned
> there.
Yeah, there was a tiny bit of work done to support the XenServer API,
but the person working on it changed jobs and no one else stepped
forward to continue it.
> 
> Could anybody help comment on or explain a few things:
> 
> - comparison between XenServer and libvirt: I notice many similar
> concepts, for example, XenServer has storage repositories and libvirt
> has storage pools.  However, in many examples such as this[2] it appears
> that libvirt requires more manual effort.  For example, when you create
> a storage repository in XenServer, the tool takes a physical block
> device as input and creates the necessary volume group, logical volume
> and a filesystem.  In that libvirt example, it appears each of those
> steps must be done manually.  The networking examples are similar: the
> XenServer tools do all the Open vSwitch stuff behind the scenes but in
> the libvirt examples it appears necessary to create the bridge manually
> before telling libvirt about it.  Is this all correct or are the
> examples I've seen out of date?  Do more recent (or future) releases of
> libvirt aim to automate/hide more of these things?
NB, comparing XenServer with libvirt is a bit of an apples & oranges
comparison, as they have different goals. So yeah, libvirt requires
more steps in managing the storage, but this also allows you todo
things you can't do with XenServer. Also, note that most administrators
would not use libvirt directly, but rather use it indirectly via some
management app that introduces higher level abstractions (eg, oVirt,
OpenStack, or in simpler cases virt-manager etc). 
> - what is best practice for virtual disk images?  Does libvirt always
> use files (like XenServer) or can/should block devices be used
> directly?  If using files, is any special care needed to avoid block
> alignment problems?
This is a matter for the individual hypervisor beneath libvirt. Each
has its own set of supported virtual disk image backends. QEMU/KVM
can run on prettty much anything, file, block device, network storage,
etc
> - are there any guides or tools recommended for migrating small
> XenServer environments (less than 50 domains on a single physical node)
> to libvirt + KVM?
> 
> - is there a list of small things that need to change in a VM before
> running it under libvirt / KVM?  I'm guessing this might include
> bootloader config, updating the kernel, changing block device names in
> /etc/fstab and changing network device names in scripts - are there any
> others?
> 
> - can anybody comment on more tricky issues that may arise in such a
> migration, for example, will a Windows VM be likely to run without
> modification when migrated from XenServer to libvirt/KVM or will it need
> extra drivers added or anything else before the migration?
Yes, there are lots of things which need changing when moving an existing
guest to a new hypervisor. The best thing is to try to use a pre-existing
tool for conversion. eg virt-v2v

   http://libguestfs.org/virt-v2v.1.html

though I'm not sure on the level of support for XenServer there. You
probably just have to give it a try on a guest and see how it works.
It clones the disk image, so your original guest won't be broken.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://entangle-photo.org       -o-    http://search.cpan.org/~danberr/ :|

On 15/03/17 10:22, Daniel P. Berrange wrote:
> On Wed, Mar 15, 2017 at 10:10:36AM +0000, Daniel Pocock wrote:
>> I've been using XenServer / XCP (with the "xe" toolset)
for a number of
>> years for hosting servers used for development and free software
>> projects and I'm now looking at migrating all those environments to
>> libvirt / KVM.
>>
>> I had a look at the wiki[1] already and didn't see XenServer
mentioned
>> there.
> Yeah, there was a tiny bit of work done to support the XenServer API,
> but the person working on it changed jobs and no one else stepped
> forward to continue it.
libvirt doesn't need to interact directly with XenServer in this case,
using the XenServer "xe" commands to extract any information from the
old system and then copying the settings over to XML files for libvirt /
virsh is fine
>> Could anybody help comment on or explain a few things:
>>
>> - comparison between XenServer and libvirt: I notice many similar
>> concepts, for example, XenServer has storage repositories and libvirt
>> has storage pools.  However, in many examples such as this[2] it
appears
>> that libvirt requires more manual effort.  For example, when you create
>> a storage repository in XenServer, the tool takes a physical block
>> device as input and creates the necessary volume group, logical volume
>> and a filesystem.  In that libvirt example, it appears each of those
>> steps must be done manually.  The networking examples are similar: the
>> XenServer tools do all the Open vSwitch stuff behind the scenes but in
>> the libvirt examples it appears necessary to create the bridge manually
>> before telling libvirt about it.  Is this all correct or are the
>> examples I've seen out of date?  Do more recent (or future)
releases of
>> libvirt aim to automate/hide more of these things?
> NB, comparing XenServer with libvirt is a bit of an apples & oranges
> comparison, as they have different goals. So yeah, libvirt requires
I wasn't suggesting a comparison to suggest one might be better, rather,
I was just thinking about a cheat sheet to help somebody who knows one
platform to quickly discover the basics of the other.

E.g. a list of the "xe" commands to create a storage repository,
create
a network segment and create a VM and some equivalent libvirt
terminology and commands.
> more steps in managing the storage, but this also allows you todo
> things you can't do with XenServer. Also, note that most administrators
> would not use libvirt directly, but rather use it indirectly via some
> management app that introduces higher level abstractions (eg, oVirt,
> OpenStack, or in simpler cases virt-manager etc). 
Some of the goals with this setup:

- using utilities that are already packaged in Debian so security
updates and backports can be used, e.g. virt-manager and virsh are
available in official packages, the old XenServer setup is also based on
the XCP packages on Debian wheezy (now discontinued unfortunately). 
oVirt doesn't appear to be available in Debian yet and OpenStack may not
be right for this use case.

- keeping a fairly basic setup that any other sysadmin who knows basic
virsh commands will be comfortable with

For somebody wanting to run a "vanilla" installation of libvirt / KVM,
are there any patterns that are suggested, for example, keeping all the
images as disk images like in XenServer?  Is there a somewhat standard
place to put all the commands for setting up any network bridges, can
that be done automatically by libvirt or does each site put them in
/etc/rc.local?

>> - what is best practice for virtual disk images?  Does libvirt always
>> use files (like XenServer) or can/should block devices be used
>> directly?  If using files, is any special care needed to avoid block
>> alignment problems?
> This is a matter for the individual hypervisor beneath libvirt. Each
> has its own set of supported virtual disk image backends. QEMU/KVM
> can run on prettty much anything, file, block device, network storage,
> etc
>
>> - are there any guides or tools recommended for migrating small
>> XenServer environments (less than 50 domains on a single physical node)
>> to libvirt + KVM?
>>
>> - is there a list of small things that need to change in a VM before
>> running it under libvirt / KVM?  I'm guessing this might include
>> bootloader config, updating the kernel, changing block device names in
>> /etc/fstab and changing network device names in scripts - are there any
>> others?
>>
>> - can anybody comment on more tricky issues that may arise in such a
>> migration, for example, will a Windows VM be likely to run without
>> modification when migrated from XenServer to libvirt/KVM or will it
need
>> extra drivers added or anything else before the migration?
> Yes, there are lots of things which need changing when moving an existing
> guest to a new hypervisor. The best thing is to try to use a pre-existing
> tool for conversion. eg virt-v2v
>
>    http://libguestfs.org/virt-v2v.1.html
>
> though I'm not sure on the level of support for XenServer there. You
> probably just have to give it a try on a guest and see how it works.
> It clones the disk image, so your original guest won't be broken.
OK, thanks for suggesting that.  I have a spare server for the migration
so I will probably make a trial run using copies of all the virtual
machines and hopefully discover all the little things I have to change.

Regards,

Daniel