CentOS virt - Sep 2007 - (long) kvm and virt-manager not ready for daily usage

Farkas Levente wrote:
> hi,
> in the last 2 weeks we play a lot with our new server which we but to a
> our virtual server for the development and collect some very subjective
> experience. we use kvm and virt-manager, but sometimes i'm not really
> sure about the whether it's kvm or virt-manager problem so i collect
> them together. our system:
>   
thansk!
> MB: Intel S3000AHV
> CPU: Intel Core 2 Quad CPU Q6600 @ 2.40GHz
> RAM: 8GB
> CentOS 5 x86-64 host system with:
> kernel-2.6.18-8.1.8.el5
> kmod-kvm-35-1.2.6.18_8.1.8.el5
> kvm-35-1
> libvirt-0.3.2-2
> libvirt-python-0.3.2-2
> python-virtinst-0.300.0-1
> virt-manager-0.5.0-1
> virt-top-0.3.2.5-1
> virt-viewer-0.0.2-1
>
> kvm is not ready for production use for many reason:
> it can't reboot which imho a very basic feature, what's more
can't even
> shutdown/poweroff. the centos i386 guest are not able to shutdown on the
> x86_64 host (strange the x86_64 centos and the i586 mandrake-9 are able
> to shutdown) or what's seems to be more likely it's a random think.
>   
this issue is known and will be fix soon.
> can't give 2GB ram to any guests not even 2, but 1.9999 is ok.
what's
> more i've to discover this in the hard way virt-manager gives some
> strange error message (anyway none of the virt-manager's error message
> are useful, just strange stack trace without any kind of useful info).
>   
kvm from version 36 support above 2giga of ram (it was tested with 
32giga ram guest)
> the host see as i've 4 cpu. i've got a change to gives more cpu to
the
> guest, what's more they starts, but after a few minutes running the
> system crash. not just the guest os but the host os crash without any
> kind of info, log or any useful info what was the cause of it, but hard
> reset helps:-((( first of all it's serious problem, even if it's a
known
> bug and documented somewhere (but there is not any kind of docs about
> neither kvm nor virt-manager/libvirt and what i can find that very
> limited), since all of the new hvm cpu (which is required for kvm) has
> more core, so the guest can't use the real power of the cpu this means
> i've to put a lots of guests with one virtual cpu to the host or
> currently can't exploit these cpus.
>
> libvirt not start the guest when the host started, not saved the state
> of the guests when stopped, when restarted all guest get into shutdown
> state. may be it's just libvirt but, but if kvm not able to save
it's
> guest's state that it's a real pain. this behavior not acceptable
in a
> real environment.
>
> in virt-manager can't modify virtual cpu number of the guests
> (virt-manager crash with another unusable stacks trace). it i modify the
> xml config file by hand i've to restart libvirt in order to read it.
> even if i just modify one guest's config, but as if i restart libvirtd
> the all other guest will be killed it's another nice feature.
>
> from vit-manager i can't assign swap partitions (and any other device
> to the guests) during guest creation. i precreate lvm partition on the
> host for root and swap to all of our guests (about 10) but it's a real
> pain to reload libvirtd (which kills all guests) to add a new disk to
> the guest which can be used as a swap or other partition. what's more
> it'd be useful to add swap partition during install. the same thing
> apply to cdrom too.
>
> if i made a mistake in the libvirtd xml config file, then i don't get
> any kind of error message just libvirt don't start the given guest. no
> error message service seems to start without error and don't know what
> was the reason that the guest not start (sometimes because syntactic
> error in the config file, sometimes kvm/qemu not support it eg more then
> 2gb mem). there are unwritten unix/linux tradition eg if there are error
> during a service startup report it some log file. if a service can't
> start without error then it not start at all eg. if i've apache with 5
> virtual host but one of it's config has error then the whole httpd not
> started. it'd have to be the same with libvirt too.
>
> libvirt config file has all the features as qemu command line (if not it
> can't be used by advanced users)? but i don't know since there is
not a
> reference docs for the config file.
>
> there is not a detailed reference docs about libvirtd config (and at the
> same time the gui not contains all the settings). for the the xml format
> is a bit redundant eg. "source dev", "target dev" why
not just source
> and target, inside interface type='bridge' 'source bridge'
why not just
> source (i've to modify two place when i'd like to switch from
network to
> bride. ok these are just small cosmetic thing just mention.
>
> when create a new guest in virt-manager and reach the last page and
> press finish then most of the time (2 times from 3) i've got an error
> something like "can't connect to qemu" after i press ok and
press again
> finish then it's able to create the new host (so always have to retry
it).
>
> in virt-manager when a guest is shutdown and double-click on the guest
> and try to start it it's not possible. you've to select the guest
in the
> main windows, right-click and start it.
>
> the new vnc based console is very unstable or just virt-manager not able
> to connect it. many times i double click on a guest in virt-manager i
> got the window can't connect to console while the guest are running.
> that's why i like to see again the serial console.
>
> we've got other virtual servers with xen and openvz, with has other
> problems but works. my conclusion is that currently kvm, libvirtd,
> virt-manager (et-mgmt-tools) seems very promising. but currently just in
> a very alpha stage using even in non mission critical but production
> environment still a very 'brave' decision. xen has a few years
vantage,
> but it's at least usable in real environment.
> i'm on the these list and see what kind of problem you're working
which
> is great but at the same it'd be useful to fix the most basic problems
> with these tools.
>
> just my 2c to these community.
>
>   
thanks for all this information, this exactly what we need!

hi,
in the last 2 weeks we play a lot with our new server which we but to a
our virtual server for the development and collect some very subjective
experience. we use kvm and virt-manager, but sometimes i'm not really
sure about the whether it's kvm or virt-manager problem so i collect
them together. our system:
MB: Intel S3000AHV
CPU: Intel Core 2 Quad CPU Q6600 @ 2.40GHz
RAM: 8GB
CentOS 5 x86-64 host system with:
kernel-2.6.18-8.1.8.el5
kmod-kvm-35-1.2.6.18_8.1.8.el5
kvm-35-1
libvirt-0.3.2-2
libvirt-python-0.3.2-2
python-virtinst-0.300.0-1
virt-manager-0.5.0-1
virt-top-0.3.2.5-1
virt-viewer-0.0.2-1

kvm is not ready for production use for many reason:
it can't reboot which imho a very basic feature, what's more can't
even
shutdown/poweroff. the centos i386 guest are not able to shutdown on the
x86_64 host (strange the x86_64 centos and the i586 mandrake-9 are able
to shutdown) or what's seems to be more likely it's a random think.

can't give 2GB ram to any guests not even 2, but 1.9999 is ok. what's
more i've to discover this in the hard way virt-manager gives some
strange error message (anyway none of the virt-manager's error message
are useful, just strange stack trace without any kind of useful info).

the host see as i've 4 cpu. i've got a change to gives more cpu to the
guest, what's more they starts, but after a few minutes running the
system crash. not just the guest os but the host os crash without any
kind of info, log or any useful info what was the cause of it, but hard
reset helps:-((( first of all it's serious problem, even if it's a known
bug and documented somewhere (but there is not any kind of docs about
neither kvm nor virt-manager/libvirt and what i can find that very
limited), since all of the new hvm cpu (which is required for kvm) has
more core, so the guest can't use the real power of the cpu this means
i've to put a lots of guests with one virtual cpu to the host or
currently can't exploit these cpus.

libvirt not start the guest when the host started, not saved the state
of the guests when stopped, when restarted all guest get into shutdown
state. may be it's just libvirt but, but if kvm not able to save it's
guest's state that it's a real pain. this behavior not acceptable in a
real environment.

in virt-manager can't modify virtual cpu number of the guests
(virt-manager crash with another unusable stacks trace). it i modify the
xml config file by hand i've to restart libvirt in order to read it.
even if i just modify one guest's config, but as if i restart libvirtd
the all other guest will be killed it's another nice feature.

from vit-manager i can't assign swap partitions (and any other device
to the guests) during guest creation. i precreate lvm partition on the
host for root and swap to all of our guests (about 10) but it's a real
pain to reload libvirtd (which kills all guests) to add a new disk to
the guest which can be used as a swap or other partition. what's more
it'd be useful to add swap partition during install. the same thing
apply to cdrom too.

if i made a mistake in the libvirtd xml config file, then i don't get
any kind of error message just libvirt don't start the given guest. no
error message service seems to start without error and don't know what
was the reason that the guest not start (sometimes because syntactic
error in the config file, sometimes kvm/qemu not support it eg more then
2gb mem). there are unwritten unix/linux tradition eg if there are error
during a service startup report it some log file. if a service can't
start without error then it not start at all eg. if i've apache with 5
virtual host but one of it's config has error then the whole httpd not
started. it'd have to be the same with libvirt too.

libvirt config file has all the features as qemu command line (if not it
can't be used by advanced users)? but i don't know since there is not a
reference docs for the config file.

there is not a detailed reference docs about libvirtd config (and at the
same time the gui not contains all the settings). for the the xml format
is a bit redundant eg. "source dev", "target dev" why not
just source
and target, inside interface type='bridge' 'source bridge' why
not just
source (i've to modify two place when i'd like to switch from network to
bride. ok these are just small cosmetic thing just mention.

when create a new guest in virt-manager and reach the last page and
press finish then most of the time (2 times from 3) i've got an error
something like "can't connect to qemu" after i press ok and press
again
finish then it's able to create the new host (so always have to retry it).

in virt-manager when a guest is shutdown and double-click on the guest
and try to start it it's not possible. you've to select the guest in the
main windows, right-click and start it.

the new vnc based console is very unstable or just virt-manager not able
to connect it. many times i double click on a guest in virt-manager i
got the window can't connect to console while the guest are running.
that's why i like to see again the serial console.

we've got other virtual servers with xen and openvz, with has other
problems but works. my conclusion is that currently kvm, libvirtd,
virt-manager (et-mgmt-tools) seems very promising. but currently just in
a very alpha stage using even in non mission critical but production
environment still a very 'brave' decision. xen has a few years vantage,
but it's at least usable in real environment.
i'm on the these list and see what kind of problem you're working which
is great but at the same it'd be useful to fix the most basic problems
with these tools.

just my 2c to these community.

-- 
  Levente                               "Si vis pacem para bellum!"

From: "Farkas Levente" <lfarkas at
bppiac.hu>
> RAM: 8GB
> CentOS 5 x86-64 host system with:
> can't give 2GB ram to any guests not even 2, but 1.9999 is ok.
what's
So far as i know u need to use a PAE kernel to use >2Gb.

Danny.

On 9/8/07, Farkas Levente <lfarkas at bppiac.hu>
wrote:
> kvm is not ready for production use for many reason:
> it can't reboot which imho a very basic feature, what's more
can't even
> shutdown/poweroff. the centos i386 guest are not able to shutdown on the
> x86_64 host (strange the x86_64 centos and the i586 mandrake-9 are able
> to shutdown) or what's seems to be more likely it's a random think.
Reboot failure is a regression; for shutdown you should enable APCI
with "acpi=force"; bochs BIOS doesn't have SMBIOS/DMI tables and
Linux
may refuse to use ACPI without them (it depends on the configuration).

Luca

On Sat, 2007-09-08 at 13:15 +0200, Farkas Levente wrote:
> hi,
> kvm is not ready for production use for many reason:
> it can't reboot which imho a very basic feature, what's more
can't even
> shutdown/poweroff. the centos i386 guest are not able to shutdown on the
> x86_64 host (strange the x86_64 centos and the i586 mandrake-9 are able
> to shutdown) or what's seems to be more likely it's a random think.
> 
> can't give 2GB ram to any guests not even 2, but 1.9999 is ok.
what's
> more i've to discover this in the hard way virt-manager gives some
> strange error message (anyway none of the virt-manager's error message
> are useful, just strange stack trace without any kind of useful info).
kvm-36 now supports more than 2GB of RAM per guest.  You're one version
off for that.
> the host see as i've 4 cpu. i've got a change to gives more cpu to
the
> guest, what's more they starts, but after a few minutes running the
> system crash. not just the guest os but the host os crash without any
> kind of info, log or any useful info what was the cause of it, but hard
> reset helps:-((( first of all it's serious problem, even if it's a
known
> bug and documented somewhere (but there is not any kind of docs about
> neither kvm nor virt-manager/libvirt and what i can find that very
> limited), since all of the new hvm cpu (which is required for kvm) has
> more core, so the guest can't use the real power of the cpu this means
> i've to put a lots of guests with one virtual cpu to the host or
> currently can't exploit these cpus.
I'm having difficulty understanding what your problem is.  Are you
saying that guest SMP isn't working for you?  The host OS definitely
shouldn't crash.  Can you be more specific about what configs you are
using?  There was a host oops fixed in kvm-36 so upgrading may help you.

Regards,

Anthony Liguori
> libvirt not start the guest when the host started, not saved the state
> of the guests when stopped, when restarted all guest get into shutdown
> state. may be it's just libvirt but, but if kvm not able to save
it's
> guest's state that it's a real pain. this behavior not acceptable
in a
> real environment.
> 
> in virt-manager can't modify virtual cpu number of the guests
> (virt-manager crash with another unusable stacks trace). it i modify the
> xml config file by hand i've to restart libvirt in order to read it.
> even if i just modify one guest's config, but as if i restart libvirtd
> the all other guest will be killed it's another nice feature.
> 
> from vit-manager i can't assign swap partitions (and any other device
> to the guests) during guest creation. i precreate lvm partition on the
> host for root and swap to all of our guests (about 10) but it's a real
> pain to reload libvirtd (which kills all guests) to add a new disk to
> the guest which can be used as a swap or other partition. what's more
> it'd be useful to add swap partition during install. the same thing
> apply to cdrom too.
> 
> if i made a mistake in the libvirtd xml config file, then i don't get
> any kind of error message just libvirt don't start the given guest. no
> error message service seems to start without error and don't know what
> was the reason that the guest not start (sometimes because syntactic
> error in the config file, sometimes kvm/qemu not support it eg more then
> 2gb mem). there are unwritten unix/linux tradition eg if there are error
> during a service startup report it some log file. if a service can't
> start without error then it not start at all eg. if i've apache with 5
> virtual host but one of it's config has error then the whole httpd not
> started. it'd have to be the same with libvirt too.
> 
> libvirt config file has all the features as qemu command line (if not it
> can't be used by advanced users)? but i don't know since there is
not a
> reference docs for the config file.
> 
> there is not a detailed reference docs about libvirtd config (and at the
> same time the gui not contains all the settings). for the the xml format
> is a bit redundant eg. "source dev", "target dev" why
not just source
> and target, inside interface type='bridge' 'source bridge'
why not just
> source (i've to modify two place when i'd like to switch from
network to
> bride. ok these are just small cosmetic thing just mention.
> 
> when create a new guest in virt-manager and reach the last page and
> press finish then most of the time (2 times from 3) i've got an error
> something like "can't connect to qemu" after i press ok and
press again
> finish then it's able to create the new host (so always have to retry
it).
> 
> in virt-manager when a guest is shutdown and double-click on the guest
> and try to start it it's not possible. you've to select the guest
in the
> main windows, right-click and start it.
> 
> the new vnc based console is very unstable or just virt-manager not able
> to connect it. many times i double click on a guest in virt-manager i
> got the window can't connect to console while the guest are running.
> that's why i like to see again the serial console.
> 
> we've got other virtual servers with xen and openvz, with has other
> problems but works. my conclusion is that currently kvm, libvirtd,
> virt-manager (et-mgmt-tools) seems very promising. but currently just in
> a very alpha stage using even in non mission critical but production
> environment still a very 'brave' decision. xen has a few years
vantage,
> but it's at least usable in real environment.
> i'm on the these list and see what kind of problem you're working
which
> is great but at the same it'd be useful to fix the most basic problems
> with these tools.
> 
> just my 2c to these community.
>

Anthony Liguori wrote:
>>> saying that guest SMP isn't working for you?  The host OS
definitely
>>> shouldn't crash.  Can you be more specific about what configs
you are
>>> using?  There was a host oops fixed in kvm-36 so upgrading may help
you.
>> exactly. i've got 4 phisical core (Intel Core 2 Quad) and i try to
give
>> 4 cpus for 2 guests and 2 cpus for a third guest and restart libvirtd.
>> the result was that even the host system crash without any stack trace
>> or kernel panic and only the hard reset helps.
> 
> Could you try to reproduce the problem with kvm-36?  Host crash bugs are
> very important to fix especially one so easily reproducible.
yes, when c5-testing will include kvm-36. i already recompile many rpms
to be able to use the latest tools, but since kvm is in centos-extras
i'd rather stay with that and don't mix packages.

-- 
  Levente                               "Si vis pacem para bellum!"