I am trying to get a proof-of-concept type setup going...I have a storage box and 2 xen servers...I am using file based disk that live on the aoe device...i can run the vm from either host without issue...when I run the live migration, the domain leaves the xm list on host 1 and shows up on host 2 (however there is a pause for pings of about 2 minutes?)...after it is on host 2, I can xm console and connect to it, I checked the uptime and it was inline with what I expected, but if I ssh into it or run "last" it hangs...can still be pinged, but the console just hangs indefinitely... another observation - when I go to the directory on host 1 and create a file, I do not see it show up on host 2 unless I remount it...which is why I have been thinking there is a syncing, caching issue...thinking that maybe the host moves over and the disk it accesses is not "fresh"... I have not worked with aoe or xen before so, I am hoping there is just a config stage I missed... any thoughts/suggestions would be great! thanks lee -- Everything should be made as simple as possible, but not simpler. -- Albert Einstein _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On 1/15/08, lee hartley <leeslist@gmail.com> wrote:> I have not worked with aoe or xen before so, I am hoping there is just > a config stage I missed...you''re using file-backed images, so you''re using a filesystem on your AoE-shared device, right? what filesystem do you use? if it''s not a cluster filesystem (GFS, OCFS), then you''re asking for trouble! -- Javier _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Stefan de Konink
2008-Jan-16 00:43 UTC
Re: [Xen-users] live migration breaking...aoe issue?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Javier Guerra schreef:> On 1/15/08, lee hartley <leeslist@gmail.com> wrote: >> I have not worked with aoe or xen before so, I am hoping there is just >> a config stage I missed... > > you''re using file-backed images, so you''re using a filesystem on your > AoE-shared device, right? what filesystem do you use? if it''s not a > cluster filesystem (GFS, OCFS), then you''re asking for trouble!He is not. Technically there will be no writing actions on the side where you move from there will be no ''actions'' on this Filesystem anymore. In why would iSCSI work in another way than AoE? Stefan -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHjVMWYH1+F2Rqwn0RCvK1AKCNYwVKXdKIiI+qq8K7SeqtFBLuPwCfQ4c4 3i+pF13oU8jjKD4rjxKROlY=Xwty -----END PGP SIGNATURE----- _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
"aoe-stat" command will show the state of your disks.
AoE have a 300 seconds for default timeout. Try set aoe_deadsecs to
30 seconds (or less if is your intention).
You can set aoe_deadsecs from command line (echo in /sys), with
sysctl or with a parameter to module (you can put in /etc/
modprobe.conf "alias aoe aoe_deadsecs=30".
Cheers
Marcelo Messa
Em 15/01/2008, às 18:57, lee hartley escreveu:
> I am trying to get a proof-of-concept type setup going...I have a
> storage box and 2 xen servers...I am using file based disk that live
> on the aoe device...i can run the vm from either host without
> issue...when I run the live migration, the domain leaves the xm list
> on host 1 and shows up on host 2 (however there is a pause for pings
> of about 2 minutes?)...after it is on host 2, I can xm console and
> connect to it, I checked the uptime and it was inline with what I
> expected, but if I ssh into it or run "last" it hangs...can still
be
> pinged, but the console just hangs indefinitely...
>
> another observation - when I go to the directory on host 1 and create
> a file, I do not see it show up on host 2 unless I remount it...which
> is why I have been thinking there is a syncing, caching
> issue...thinking that maybe the host moves over and the disk it
> accesses is not "fresh"...
>
> I have not worked with aoe or xen before so, I am hoping there is just
> a config stage I missed...
>
> any thoughts/suggestions would be great!
>
> thanks
> lee
>
> --
> Everything should be made as simple as possible, but not simpler. --
> Albert Einstein
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Errata: the default timeout is 180 seconds (3 * 60) Em 15/01/2008, às 22:49, Marcelo Messa escreveu:> > "aoe-stat" command will show the state of your disks. > > AoE have a 300 seconds for default timeout. Try set aoe_deadsecs > to 30 seconds (or less if is your intention). > > You can set aoe_deadsecs from command line (echo in /sys), with > sysctl or with a parameter to module (you can put in /etc/ > modprobe.conf "alias aoe aoe_deadsecs=30". > > Cheers > Marcelo Messa > > Em 15/01/2008, às 18:57, lee hartley escreveu: > >> I am trying to get a proof-of-concept type setup going...I have a >> storage box and 2 xen servers...I am using file based disk that live >> on the aoe device...i can run the vm from either host without >> issue...when I run the live migration, the domain leaves the xm list >> on host 1 and shows up on host 2 (however there is a pause for pings >> of about 2 minutes?)...after it is on host 2, I can xm console and >> connect to it, I checked the uptime and it was inline with what I >> expected, but if I ssh into it or run "last" it hangs...can still be >> pinged, but the console just hangs indefinitely... >> >> another observation - when I go to the directory on host 1 and create >> a file, I do not see it show up on host 2 unless I remount it...which >> is why I have been thinking there is a syncing, caching >> issue...thinking that maybe the host moves over and the disk it >> accesses is not "fresh"... >> >> I have not worked with aoe or xen before so, I am hoping there is >> just >> a config stage I missed... >> >> any thoughts/suggestions would be great! >> >> thanks >> lee >> >> -- >> Everything should be made as simple as possible, but not simpler. -- >> Albert Einstein >> >> _______________________________________________ >> Xen-users mailing list >> Xen-users@lists.xensource.com >> http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
coming back to this...I tried updating aoe to the latest version
# aoe-version
aoetools: 23
installed aoe driver: 57
running aoe driver: 57
set the aoe_deadsecs=30
modprobe aoe aoe_deadsecs=30
formatted the lv with gfs (just in case)
things seem basically the same...even got a little worse, getting IO
errors...so, did gfs2_fsck on the device...no more io errors...started
vm tried the migration...hung in middle the source host shows the
domain status as "b" and when you try to connect to the console these
warnings are displayed
WARNING: g.e. still in use!
WARNING: leaking g.e. and page still in use!
WARNING: g.e. still in use!
WARNING: leaking g.e. and page still in use!
netif_release_rx_bufs: 0 xfer, 46 noxfer, 210 unused
WARNING: g.e. still in use!
WARNING: leaking g.e. and page still in use!
on the target host the status is "bp" and when you try xm console you
get
xenconsole: Could not read tty from store: No such file or directory
so, I seem to be going backwards...I am thinking to go back to ext3 w/
the aoe changes...to limit my variables...any pointers would be
great...or other approaches
basically, I just want to be able to show that xen live migration
works in a easy/cheap setup, I will refactor after that
thanks a lot for your comments!
lee
On Jan 15, 2008 9:11 PM, Marcelo Messa <messa@webmetrix.com.br>
wrote:>
> Errata: the default timeout is 180 seconds (3 * 60)
>
>
> Em 15/01/2008, às 22:49, Marcelo Messa escreveu:
>
>
> >
> > "aoe-stat" command will show the state of your disks.
> >
> > AoE have a 300 seconds for default timeout. Try set aoe_deadsecs
> > to 30 seconds (or less if is your intention).
> >
> > You can set aoe_deadsecs from command line (echo in /sys), with
> > sysctl or with a parameter to module (you can put in /etc/
> > modprobe.conf "alias aoe aoe_deadsecs=30".
> >
> > Cheers
> > Marcelo Messa
> >
> > Em 15/01/2008, às 18:57, lee hartley escreveu:
> >
> >> I am trying to get a proof-of-concept type setup going...I have a
> >> storage box and 2 xen servers...I am using file based disk that
live
> >> on the aoe device...i can run the vm from either host without
> >> issue...when I run the live migration, the domain leaves the xm
list
> >> on host 1 and shows up on host 2 (however there is a pause for
pings
> >> of about 2 minutes?)...after it is on host 2, I can xm console and
> >> connect to it, I checked the uptime and it was inline with what I
> >> expected, but if I ssh into it or run "last" it
hangs...can still be
> >> pinged, but the console just hangs indefinitely...
> >>
> >> another observation - when I go to the directory on host 1 and
create
> >> a file, I do not see it show up on host 2 unless I remount
it...which
> >> is why I have been thinking there is a syncing, caching
> >> issue...thinking that maybe the host moves over and the disk it
> >> accesses is not "fresh"...
> >>
> >> I have not worked with aoe or xen before so, I am hoping there is
> >> just
> >> a config stage I missed...
> >>
> >> any thoughts/suggestions would be great!
> >>
> >> thanks
> >> lee
> >>
> >> --
> >> Everything should be made as simple as possible, but not simpler.
--
> >> Albert Einstein
> >>
> >> _______________________________________________
> >> Xen-users mailing list
> >> Xen-users@lists.xensource.com
> >> http://lists.xensource.com/xen-users
> >
>
>
--
Everything should be made as simple as possible, but not simpler. --
Albert Einstein
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
On 1/17/08, lee hartley <leeslist@gmail.com> wrote:> basically, I just want to be able to show that xen live migration > works in a easy/cheap setup, I will refactor after thatIMHO, it''s far simpler to use blockdevices (CLVM, EVMS) for live migration, instead of image files. it''s supposed to be faster too (since you don''t have the overhead of a filesystem... and a shared one to make it worse). -- Javier _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users