thr3ads.net - Xen users - [Xen-users] migrating between different hardware (xen-3.1.3) [Feb 2008]

If this information is useful, please help other people find it:
Share via:

b52@entrap.de

2008-Feb-23 10:13 UTC

[Xen-users] migrating between different hardware (xen-3.1.3)

Hi,

I setup a testbed todo some migration tests for work and got a problem..

I used two P4 2.4GHz an AMD64 4200+ (SVM) and an Intel C2D 6320 1.86GHz. I
had xen 3.1.3 with 2.6.18 and exactly the same versions on all systems
(mirrored AoE) and one 32Bit domU

Now if I created this domU on one of the P4s, I was able to migrate it
between every host.
If I created this domU on the AMD or Intel C2D, it was only possible to
migrate between these both. The domU always crashed if I tried to migrate
it to the P4s.

You can find the xm info of all 4 PCs and one xm log of source(AMD) and
one xm log of destination(P4) in close.

Any help is appreciated and please CC me
Cheers,
Holger







_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

[STD]Ein

2008-Feb-24 04:59 UTC

head link

RE: [Xen-users] migrating between different hardware (xen-3.1.3)

This has been brought up a few times on the list.  The problem is the cpu
features that the guest is using.  When you transfer a domain from the p4 to
the a64, it supports all the flags (being a generation or two newer) that
the p4 does, however in reverse it is not the case.

There was some ongoing work to create a filter for domain creation that
would allow the ''lowest common denominator''.  However, it
seems it''s never
made into the codetree.  It has been listed on the xen roadmap for a long
time now.

In short, this doesn''t work now, and unless somebody completes the
feature
filter, it may not work for some time.

You should be careful to ensure that all the hosts that you transfer to do
in fact support all the cpu flags that the creating host does, otherwise you
will run into seemingly random crashes, corruption, or hangs.

For the moment, it is only recommendable to do domain transfer between
identical (or same generation same lineup) cpus.

-----Original Message-----
From: xen-users-bounces@lists.xensource.com
[mailto:xen-users-bounces@lists.xensource.com] On Behalf Of b52@entrap.de
Sent: February 23, 2008 4:13 AM
To: Xen-users@lists.xensource.com
Subject: [Xen-users] migrating between different hardware (xen-3.1.3)

Hi,

I setup a testbed todo some migration tests for work and got a problem..

I used two P4 2.4GHz an AMD64 4200+ (SVM) and an Intel C2D 6320 1.86GHz. I
had xen 3.1.3 with 2.6.18 and exactly the same versions on all systems
(mirrored AoE) and one 32Bit domU

Now if I created this domU on one of the P4s, I was able to migrate it
between every host.
If I created this domU on the AMD or Intel C2D, it was only possible to
migrate between these both. The domU always crashed if I tried to migrate
it to the P4s.

You can find the xm info of all 4 PCs and one xm log of source(AMD) and
one xm log of destination(P4) in close.

Any help is appreciated and please CC me
Cheers,
Holger


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

b52@entrap.de

2008-Feb-24 13:35 UTC

head link

RE: [Xen-users] migrating between different hardware (xen-3.1.3)

I already thought that, but hoped there is a way to tell the domU which
cpu flags to use. Well this will be the reason my company will not use xen
for domain management. You will never know what kind of CPUs you will buy
in future and sure you don''t want to buy a complete new cluster because
you need one box more..

Btw. should the relocation daemon not at least check the CPU flags of the
remote server before migration?

And there is really no way to pin the domU to a set of CPU flags?

-- Holger
> This has been brought up a few times on the list.  The problem is the cpu
> features that the guest is using.  When you transfer a domain from the p4
> to
> the a64, it supports all the flags (being a generation or two newer) that
> the p4 does, however in reverse it is not the case.
>
> There was some ongoing work to create a filter for domain creation that
> would allow the ''lowest common denominator''.  However, it
seems it''s never
> made into the codetree.  It has been listed on the xen roadmap for a long
> time now.
>
> In short, this doesn''t work now, and unless somebody completes the
feature
> filter, it may not work for some time.
>
> You should be careful to ensure that all the hosts that you transfer to do
> in fact support all the cpu flags that the creating host does, otherwise
> you
> will run into seemingly random crashes, corruption, or hangs.
>
> For the moment, it is only recommendable to do domain transfer between
> identical (or same generation same lineup) cpus.
>
> -----Original Message-----
> From: xen-users-bounces@lists.xensource.com
> [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of b52@entrap.de
> Sent: February 23, 2008 4:13 AM
> To: Xen-users@lists.xensource.com
> Subject: [Xen-users] migrating between different hardware (xen-3.1.3)
>
> Hi,
>
> I setup a testbed todo some migration tests for work and got a problem..
>
> I used two P4 2.4GHz an AMD64 4200+ (SVM) and an Intel C2D 6320 1.86GHz. I
> had xen 3.1.3 with 2.6.18 and exactly the same versions on all systems
> (mirrored AoE) and one 32Bit domU
>
> Now if I created this domU on one of the P4s, I was able to migrate it
> between every host.
> If I created this domU on the AMD or Intel C2D, it was only possible to
> migrate between these both. The domU always crashed if I tried to migrate
> it to the P4s.
>
> You can find the xm info of all 4 PCs and one xm log of source(AMD) and
> one xm log of destination(P4) in close.
>
> Any help is appreciated and please CC me
> Cheers,
> Holger
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Stephan Seitz

2008-Feb-24 15:18 UTC

head link

Re: [Xen-users] migrating between different hardware (xen-3.1.3)

I don''t want to argue against your decision, but you''re able
to build
your domU kernel with "generic x86" support.

Anyway, we made a few assumptions:
	- heterogene clusters are harder to manage by default
	  (even without xen)
	- new x86 hardware supports virtually all features of
	  current one.
	- hardware upgrades (10Gbit NICs, Infiniband, ...) are
	  likely to be necessary earlier than x86 changes
	- regular updates and kernel patches are always necessary,
	  optimization for newer hardware can be done if support
	  for older hardware has become obsolete.
	- electricity tends to get expensive

Taking this assumptions, and having in mind that a cluster does not
necessarily need to cover every and all machine, our decision is clear ;)

But to your question. I don''t know what kind of automation
you''re using
for migration, but it''s not too hard to get /proc/cpuinfo of a remote
host to check the capabilities before calling xm migrate.
For complex scenarios, some work has always been done outside xen.

Cheers,

Stephan


b52@entrap.de schrieb:> I already thought that, but hoped there is a way to tell the domU which
> cpu flags to use. Well this will be the reason my company will not use xen
> for domain management. You will never know what kind of CPUs you will buy
> in future and sure you don''t want to buy a complete new cluster
because
> you need one box more..
> 
> Btw. should the relocation daemon not at least check the CPU flags of the
> remote server before migration?
> 
> And there is really no way to pin the domU to a set of CPU flags?
> 
> -- Holger
> 
>> This has been brought up a few times on the list.  The problem is the
cpu
>> features that the guest is using.  When you transfer a domain from the
p4
>> to
>> the a64, it supports all the flags (being a generation or two newer)
that
>> the p4 does, however in reverse it is not the case.
>>
>> There was some ongoing work to create a filter for domain creation that
>> would allow the ''lowest common denominator''. 
However, it seems it''s never
>> made into the codetree.  It has been listed on the xen roadmap for a
long
>> time now.
>>
>> In short, this doesn''t work now, and unless somebody completes
the feature
>> filter, it may not work for some time.
>>
>> You should be careful to ensure that all the hosts that you transfer to
do
>> in fact support all the cpu flags that the creating host does,
otherwise
>> you
>> will run into seemingly random crashes, corruption, or hangs.
>>
>> For the moment, it is only recommendable to do domain transfer between
>> identical (or same generation same lineup) cpus.
>>
>> -----Original Message-----
>> From: xen-users-bounces@lists.xensource.com
>> [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of
b52@entrap.de
>> Sent: February 23, 2008 4:13 AM
>> To: Xen-users@lists.xensource.com
>> Subject: [Xen-users] migrating between different hardware (xen-3.1.3)
>>
>> Hi,
>>
>> I setup a testbed todo some migration tests for work and got a
problem..
>>
>> I used two P4 2.4GHz an AMD64 4200+ (SVM) and an Intel C2D 6320
1.86GHz. I
>> had xen 3.1.3 with 2.6.18 and exactly the same versions on all systems
>> (mirrored AoE) and one 32Bit domU
>>
>> Now if I created this domU on one of the P4s, I was able to migrate it
>> between every host.
>> If I created this domU on the AMD or Intel C2D, it was only possible to
>> migrate between these both. The domU always crashed if I tried to
migrate
>> it to the P4s.
>>
>> You can find the xm info of all 4 PCs and one xm log of source(AMD) and
>> one xm log of destination(P4) in close.
>>
>> Any help is appreciated and please CC me
>> Cheers,
>> Holger
>>
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>>
> 
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users

-- 
Stephan Seitz
Senior System Administrator

*netz-haut* e.K.
multimediale kommunikation

zweierweg 22
97074 würzburg

fon: +49 931 2876247
fax: +49 931 2876248

web: www.netz-haut.de <http://www.netz-haut.de/>

registriergericht: amtsgericht würzburg, hra 5054



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

[STD]Ein

2008-Feb-25 02:49 UTC

head link

RE: [Xen-users] migrating between different hardware (xen-3.1.3)

You know, I completely forgot that you could force xens ''hand''
by building a
kernel without support for higher features.  I''ve given that advice
myself
(on this list nonetheless).. I blame it on my current (sick/feverish) state.

You do lose a considerable amount of granularity in supporting cpu features
however.

Afaik enabling support generic x86 is not a viable solution.  You have to
ensure that the processor family is set to the lowest commonly supported
feature set.  Generic x86 support only switches the feature set on boot, and
won''t work live, resulting in the same crash scenarios when migrating
from
new->old.

I do agree that the xen hypervisor should check feature flags while
migrating, it seems like it would an easy change and prevent many crashes,
but don''t ask me to do it, I''m just an end user after all.

Sorry that I gave such a incomplete / inaccurate response, the last thing I
want to do is scare a user away from this wonderful tool.

An end user really looking forward to cpu feature filters

	Richard [STD]Ein Barr.

-----Original Message-----
From: xen-users-bounces@lists.xensource.com
[mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Stephan Seitz
Sent: February 24, 2008 9:18 AM
To: b52@entrap.de
Cc: xen-users@lists.xensource.com
Subject: Re: [Xen-users] migrating between different hardware (xen-3.1.3)

I don''t want to argue against your decision, but you''re able
to build
your domU kernel with "generic x86" support.

Anyway, we made a few assumptions:
	- heterogene clusters are harder to manage by default
	  (even without xen)
	- new x86 hardware supports virtually all features of
	  current one.
	- hardware upgrades (10Gbit NICs, Infiniband, ...) are
	  likely to be necessary earlier than x86 changes
	- regular updates and kernel patches are always necessary,
	  optimization for newer hardware can be done if support
	  for older hardware has become obsolete.
	- electricity tends to get expensive

Taking this assumptions, and having in mind that a cluster does not
necessarily need to cover every and all machine, our decision is clear ;)

But to your question. I don''t know what kind of automation
you''re using
for migration, but it''s not too hard to get /proc/cpuinfo of a remote
host to check the capabilities before calling xm migrate.
For complex scenarios, some work has always been done outside xen.

Cheers,

Stephan


b52@entrap.de schrieb:> I already thought that, but hoped there is a way to tell the domU which
> cpu flags to use. Well this will be the reason my company will not use xen
> for domain management. You will never know what kind of CPUs you will buy
> in future and sure you don''t want to buy a complete new cluster
because
> you need one box more..
> 
> Btw. should the relocation daemon not at least check the CPU flags of the
> remote server before migration?
> 
> And there is really no way to pin the domU to a set of CPU flags?
> 
> -- Holger
> 
>> This has been brought up a few times on the list.  The problem is the
cpu
>> features that the guest is using.  When you transfer a domain from the
p4
>> to
>> the a64, it supports all the flags (being a generation or two newer)
that
>> the p4 does, however in reverse it is not the case.
>>
>> There was some ongoing work to create a filter for domain creation that
>> would allow the ''lowest common denominator''. 
However, it seems it''s
never>> made into the codetree.  It has been listed on the xen roadmap for a
long
>> time now.
>>
>> In short, this doesn''t work now, and unless somebody completes
the
feature>> filter, it may not work for some time.
>>
>> You should be careful to ensure that all the hosts that you transfer to
do>> in fact support all the cpu flags that the creating host does,
otherwise
>> you
>> will run into seemingly random crashes, corruption, or hangs.
>>
>> For the moment, it is only recommendable to do domain transfer between
>> identical (or same generation same lineup) cpus.
>>
>> -----Original Message-----
>> From: xen-users-bounces@lists.xensource.com
>> [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of
b52@entrap.de
>> Sent: February 23, 2008 4:13 AM
>> To: Xen-users@lists.xensource.com
>> Subject: [Xen-users] migrating between different hardware (xen-3.1.3)
>>
>> Hi,
>>
>> I setup a testbed todo some migration tests for work and got a
problem..
>>
>> I used two P4 2.4GHz an AMD64 4200+ (SVM) and an Intel C2D 6320
1.86GHz.
I>> had xen 3.1.3 with 2.6.18 and exactly the same versions on all systems
>> (mirrored AoE) and one 32Bit domU
>>
>> Now if I created this domU on one of the P4s, I was able to migrate it
>> between every host.
>> If I created this domU on the AMD or Intel C2D, it was only possible to
>> migrate between these both. The domU always crashed if I tried to
migrate
>> it to the P4s.
>>
>> You can find the xm info of all 4 PCs and one xm log of source(AMD) and
>> one xm log of destination(P4) in close.
>>
>> Any help is appreciated and please CC me
>> Cheers,
>> Holger
>>
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>>
> 
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users

-- 
Stephan Seitz
Senior System Administrator

*netz-haut* e.K.
multimediale kommunikation

zweierweg 22
97074 würzburg

fon: +49 931 2876247
fax: +49 931 2876248

web: www.netz-haut.de <http://www.netz-haut.de/>

registriergericht: amtsgericht würzburg, hra 5054


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

b52@entrap.de

2008-Feb-25 12:48 UTC

head link

RE: [Xen-users] migrating between different hardware (xen-3.1.3)

You will definitely not scare ME away from xen :)
..but may be the company I am working for.

Thanks for your second reply, actually I was going to verify your
statement right at the moment, cause I built all kernels with generix x86
support. I am going to try this again with kernels compiled only for
Pentium-Pro. Actually the strange thing was that the domU crash depends on
the creation Host. So if xen has no CPU Flag runtime detection features in
any way, it should work with Pentium-Pro compiled kernels.

I am glad that you agree with me that xend should compare the needs of a
running DomU and the possibilities of the target host.

@Richard Get well soon.

-- Holger
> You know, I completely forgot that you could force xens
''hand'' by building
> a
> kernel without support for higher features.  I''ve given that
advice myself
> (on this list nonetheless).. I blame it on my current (sick/feverish)
> state.
>
> You do lose a considerable amount of granularity in supporting cpu
> features
> however.
>
> Afaik enabling support generic x86 is not a viable solution.  You have to
> ensure that the processor family is set to the lowest commonly supported
> feature set.  Generic x86 support only switches the feature set on boot,
> and
> won''t work live, resulting in the same crash scenarios when
migrating from
> new->old.
>
> I do agree that the xen hypervisor should check feature flags while
> migrating, it seems like it would an easy change and prevent many crashes,
> but don''t ask me to do it, I''m just an end user after
all.
>
> Sorry that I gave such a incomplete / inaccurate response, the last thing
> I
> want to do is scare a user away from this wonderful tool.
>
> An end user really looking forward to cpu feature filters
>
> 	Richard [STD]Ein Barr.
>
> -----Original Message-----
> From: xen-users-bounces@lists.xensource.com
> [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Stephan Seitz
> Sent: February 24, 2008 9:18 AM
> To: b52@entrap.de
> Cc: xen-users@lists.xensource.com
> Subject: Re: [Xen-users] migrating between different hardware (xen-3.1.3)
>
> I don''t want to argue against your decision, but you''re
able to build
> your domU kernel with "generic x86" support.
>
> Anyway, we made a few assumptions:
> 	- heterogene clusters are harder to manage by default
> 	  (even without xen)
> 	- new x86 hardware supports virtually all features of
> 	  current one.
> 	- hardware upgrades (10Gbit NICs, Infiniband, ...) are
> 	  likely to be necessary earlier than x86 changes
> 	- regular updates and kernel patches are always necessary,
> 	  optimization for newer hardware can be done if support
> 	  for older hardware has become obsolete.
> 	- electricity tends to get expensive
>
> Taking this assumptions, and having in mind that a cluster does not
> necessarily need to cover every and all machine, our decision is clear ;)
>
> But to your question. I don''t know what kind of automation
you''re using
> for migration, but it''s not too hard to get /proc/cpuinfo of a
remote
> host to check the capabilities before calling xm migrate.
> For complex scenarios, some work has always been done outside xen.
>
> Cheers,
>
> Stephan
>
>
> b52@entrap.de schrieb:
>> I already thought that, but hoped there is a way to tell the domU which
>> cpu flags to use. Well this will be the reason my company will not use
>> xen
>> for domain management. You will never know what kind of CPUs you will
>> buy
>> in future and sure you don''t want to buy a complete new
cluster because
>> you need one box more..
>>
>> Btw. should the relocation daemon not at least check the CPU flags of
>> the
>> remote server before migration?
>>
>> And there is really no way to pin the domU to a set of CPU flags?
>>
>> -- Holger
>>
>>> This has been brought up a few times on the list.  The problem is
the
>>> cpu
>>> features that the guest is using.  When you transfer a domain from
the
>>> p4
>>> to
>>> the a64, it supports all the flags (being a generation or two
newer)
>>> that
>>> the p4 does, however in reverse it is not the case.
>>>
>>> There was some ongoing work to create a filter for domain creation
that
>>> would allow the ''lowest common denominator''. 
However, it seems it''s
> never
>>> made into the codetree.  It has been listed on the xen roadmap for
a
>>> long
>>> time now.
>>>
>>> In short, this doesn''t work now, and unless somebody
completes the
> feature
>>> filter, it may not work for some time.
>>>
>>> You should be careful to ensure that all the hosts that you
transfer to
> do
>>> in fact support all the cpu flags that the creating host does,
>>> otherwise
>>> you
>>> will run into seemingly random crashes, corruption, or hangs.
>>>
>>> For the moment, it is only recommendable to do domain transfer
between
>>> identical (or same generation same lineup) cpus.
>>>
>>> -----Original Message-----
>>> From: xen-users-bounces@lists.xensource.com
>>> [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of
>>> b52@entrap.de
>>> Sent: February 23, 2008 4:13 AM
>>> To: Xen-users@lists.xensource.com
>>> Subject: [Xen-users] migrating between different hardware
(xen-3.1.3)
>>>
>>> Hi,
>>>
>>> I setup a testbed todo some migration tests for work and got a
>>> problem..
>>>
>>> I used two P4 2.4GHz an AMD64 4200+ (SVM) and an Intel C2D 6320
>>> 1.86GHz.
> I
>>> had xen 3.1.3 with 2.6.18 and exactly the same versions on all
systems
>>> (mirrored AoE) and one 32Bit domU
>>>
>>> Now if I created this domU on one of the P4s, I was able to migrate
it
>>> between every host.
>>> If I created this domU on the AMD or Intel C2D, it was only
possible to
>>> migrate between these both. The domU always crashed if I tried to
>>> migrate
>>> it to the P4s.
>>>
>>> You can find the xm info of all 4 PCs and one xm log of source(AMD)
and
>>> one xm log of destination(P4) in close.
>>>
>>> Any help is appreciated and please CC me
>>> Cheers,
>>> Holger
>>>
>>>
>>> _______________________________________________
>>> Xen-users mailing list
>>> Xen-users@lists.xensource.com
>>> http://lists.xensource.com/xen-users
>>>
>>
>>
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>
>
> --
> Stephan Seitz
> Senior System Administrator
>
> *netz-haut* e.K.
> multimediale kommunikation
>
> zweierweg 22
> 97074 würzburg
>
> fon: +49 931 2876247
> fax: +49 931 2876248
>
> web: www.netz-haut.de <http://www.netz-haut.de/>
>
> registriergericht: amtsgericht würzburg, hra 5054
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Xen users - Feb 2008 - migrating between different hardware (xen-3.1.3)

[Xen-users] migrating between different hardware (xen-3.1.3)

RE: [Xen-users] migrating between different hardware (xen-3.1.3)

RE: [Xen-users] migrating between different hardware (xen-3.1.3)

Re: [Xen-users] migrating between different hardware (xen-3.1.3)

RE: [Xen-users] migrating between different hardware (xen-3.1.3)

RE: [Xen-users] migrating between different hardware (xen-3.1.3)