Xen users - Nov 2005 - Tutorial : Debian, Xen and CLUSTER / GFS Support

Hello there!

	I made Debian, Xen 2.0 and CLUSTER/GFS work together :). I wrote
this little tutorial to help you setup yours. Any feedback is welcome,
except comments about W3C validation of my code :p
	Note that I am talking about compiling CLUSTER with your XEN kernel,
I don''t explain how to setup a working cluster. You can find how to
setup
those in official RedHat docs.
	You will see, it''s not so long, just a matter of having good
versions of everything and know 1 or 2 tricks :)

http://dash.ionblast.net/xen/cluster.html

Regards,

Christophe PAINCHAUD



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Tuesday 22 November 2005 17:31, PAINCHAUD Christophe
wrote:
> Hello there!
>
> 	I made Debian, Xen 2.0 and CLUSTER/GFS work together :). I wrote
> this little tutorial to help you setup yours. Any feedback is welcome,
> except comments about W3C validation of my code :p
> 	Note that I am talking about compiling CLUSTER with your XEN kernel,
> I don''t explain how to setup a working cluster. You can find how
to setup
> those in official RedHat docs.
> 	You will see, it''s not so long, just a matter of having good
> versions of everything and know 1 or 2 tricks :)
>
> http://dash.ionblast.net/xen/cluster.html
Hey, nice tutorial :)
Have you got domUs running with image-files on a gfs?

Cheers,
Michael

-- 
----------------------------------------------------------------------------------------
Michael Mey                                  
Thinking Objects Software GmbH    |   mailto: michael.mey@to.com 
Lilienthalstrasse 2/1                         |   phone: +49 711 88770-147
70825 Stuttgart-Korntal, Germany  |   fax: +49 711 88770-449
----------------------------------------------------------------------------------------


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

>Hey, nice tutorial :)
Thank you ;)
>Have you got domUs running with image-files on a gfs?
	I tried to put some domUs images on a GFS shared (through GNBD)
filesystem. Everything was running fine until I launched the domU : I had
tons of I/O errors. On CLUSTER ML, many users are reporting the same problem
when they are trying to lock and read/write on a very large file (over 1Gb).


	So I am using another way (which is in fact the ENBD way, shared
block devices over network). I created some LVM volumes VM01, VM02, VM03
(that I formatted with EXT3) then I exported them in GNBD. All my Xen nodes
are importing volumes from my GNBD1 server and I can run my domU very
smoothly. So far I was able to produce 55MBytes/s writes on the LVM volumes!
That''s really a performant technology. I think I won''t be able
to do better
unless I use some fiber instead of my Gigabit interface (and I would run
into the SCSI speed limit of my disks :p). CPU on both machines was not much
used.

	The LVM solution is also interesting for backups : you can do a
snapshot of a volume and then make a backup of it, then destroy the
snapshot. It doesn''t require to stop the VM using the volume.

	I will retry the domU image on GFS later, may be I did something
wrong. I would be glad to know if any of you got read of the read/write/lock
error when running a domU from it.


Christophe PAINCHAUD



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Christophe,
> 	I will retry the domU image on GFS later, may be I did something
> wrong.
AFAIK, loopback over a GFS stored file is not supported. It works a bit for
read-only and neither at all for writes. This is not Xen related.


Regards,

--
Sylvain COUTANT

ADVISEO
http://www.adviseo.fr/
http://www.open-sp.fr/



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

PAINCHAUD Christophe wrote:
>         The LVM solution is also interesting for backups : you can do a
> snapshot of a volume and then make a backup of it, then destroy the
> snapshot. It doesn''t require to stop the VM using the volume.
In theory.  In practice, the snapshot target is horribly unstable and
will nuke your data sooner rather than later :-).

Been there just last week.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Tuesday 22 November 2005 18:30, PAINCHAUD Christophe
wrote:
> >Hey, nice tutorial :)
>
> Thank you ;)
>
> >Have you got domUs running with image-files on a gfs?
>
> 	I tried to put some domUs images on a GFS shared (through GNBD)
> filesystem. Everything was running fine until I launched the domU : I had
> tons of I/O errors. On CLUSTER ML, many users are reporting the same
> problem when they are trying to lock and read/write on a very large file
> (over 1Gb).
I set up a RedHat Cluster on two Debian dom0s running xen-devel (changeset 
from last week) and I made the same experience. I also got a lot of I/O 
errors within a domU once starting it with vbd as image file stored on gfs.

Thanks to Sylvain Coutant, now we know what is the problem with that scenario 
(because I also thought I did something the wrong way).
Btw, just for completion, my syslog said the following when I started a domU 
on gfs:

Nov 15 17:37:27 testpc-022 logger: /etc/xen/scripts/block: Writing 
backend/vbd/13/770/hotplug-status connected to xenstore.
Nov 15 17:37:27 testpc-022 logger: /etc/xen/scripts/block: Writing 
backend/vbd/13/769/hotplug-status connected to xenstore.
Nov 15 17:37:28 testpc-022 kernel: GFS: fsid=xencl:xen3-root.0: warning: 
assertion "gfs_glock_is_locked_by_me(ip->i_gl)" failed
Nov 15 17:37:28 testpc-022 kernel: GFS: fsid=xencl:xen3-root.0:   function = 
gfs_prepare_write
Nov 15 17:37:28 testpc-022 kernel: GFS: fsid=xencl:xen3-root.0:   file 
= /usr/local/src/cluster-1.01.00/gfs-kernel/src/gfs/ops_address.c, line = 329
Nov 15 17:37:28 testpc-022 kernel: GFS: fsid=xencl:xen3-root.0:   time = 
1132072648

Something''s seems to go wrong with the locking mechanism of gfs ( I
used
lock_dlm for distributed locking).

Since I''m evaluating different ways of storing domUs'' vbds for
migration, I''ll
go on and test some more stuff with FC and iSCSI.

Cheers,

Michael

-- 
----------------------------------------------------------------------------------------
Michael Mey                                  
Thinking Objects Software GmbH    |   mailto: michael.mey@to.com 
Lilienthalstrasse 2/1                         |   phone: +49 711 88770-147
70825 Stuttgart-Korntal, Germany  |   fax: +49 711 88770-449
----------------------------------------------------------------------------------------


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> I started a
> domU on gfs
What do you mean by "domU on gfs" ?


Regards,

--
Sylvain COUTANT

ADVISEO
http://www.adviseo.fr/
http://www.open-sp.fr/



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Wednesday 23 November 2005 12:04, Sylvain Coutant
wrote:
> > I started a
> > domU on gfs
>
> What do you mean by "domU on gfs" ?
I meant a file-backed VBD as domU''s root which is stored on a shared
device
(e.g. exported by GNBD). The filesystem of that shared device is GFS.

-- 
----------------------------------------------------------------------------------------
Michael Mey                                  
Thinking Objects Software GmbH    |   mailto: michael.mey@to.com 
Lilienthalstrasse 2/1                         |   phone: +49 711 88770-147
70825 Stuttgart-Korntal, Germany  |   fax: +49 711 88770-449
----------------------------------------------------------------------------------------


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users