-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 hi there! i''m currently building a clustered environment of dom0 for domU''s possible to migrate between them. by now, the domU''s rootfs is mounted via nfs from one single nfsd w/ reiser4 in the net. this works nice, but a COW-fs would do it better :) . i found very rare infos about this parallax fs integration in xen, but this seems to be under heavy development and not going to get productive right now? another solution with COW support, that works LOCALLY was a lvm setup (well only one vg over one pv) with the rootfs as real lv. the VM''s rootfs has been done via "lvmcreate -s" as snapshot. Well, as said, this did it LOCALLY and -except of some unremovable snapshots- worked also very well. I''m searching for a solution with networktransparent COW support. I''ve only found the gfs via gndb / ccs from the RedHat Clustersite. This looks interresting, but i think this thingy will lead me to some headaches :) Does someone here do know a netwide mountable COW fs? Or, maybe some better idea of solving/performing this issue? Thanks in advance Stephan Seitz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFDefOl0MKSfkbzHDYRAgR6AKCxkRCcZIMZL7tFjXfQ1v8xMXH7fwCcCxlk 3lDcx4ruSQRa4fY0kYr40oo=mXwG -----END PGP SIGNATURE----- _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Tuesday 15 November 2005 15:41, Stephan Seitz wrote:> Does someone here do know a netwide mountable COW fs? Or, maybe some > better idea of > solving/performing this issue?maybe have a look at unionfs. with some initrd magic, you should be able to mount a union of a read-only base nfs share and a read-write per-domain share as root filesystem. if you don''t want to have the union in domU, you could also try to export a seperate union over nfs for each domU from your fileserver, but use caution there, trying to use kernel-nfsd to export a unionfs has caused me several Oopses... /Ernst _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Why not just export your LVM-ified COW volume over NFS? Alternatively: * Russ Ross worked on a COW NFS server: http://www.russross.com/CoWNFS.html * You could export the CoW volume directly from the server using enbd. You could import this at the destination dom0 and export it to the guest as a VBD - the guest wouldn''t know about the network, or the COW. The block-enbd scripts in the unstable tree automate the network device binding and exporting to the guest: they should "just work" without manual setup (including during migrate). Cheers, Mark On Nov 15 2005, Stephan Seitz wrote:>-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > >hi there! > >i''m currently building a clustered environment of dom0 for domU''s >possible to migrate >between them. >by now, the domU''s rootfs is mounted via nfs from one single nfsd w/ >reiser4 in the net. >this works nice, but a COW-fs would do it better :) . i found very >rare infos about this >parallax fs integration in xen, but this seems to be under heavy >development and not >going to get productive right now? >another solution with COW support, that works LOCALLY was a lvm setup >(well only one >vg over one pv) with the rootfs as real lv. the VM''s rootfs has been >done via "lvmcreate -s" >as snapshot. Well, as said, this did it LOCALLY and -except of some >unremovable snapshots- >worked also very well. >I''m searching for a solution with networktransparent COW support. I''ve >only found the gfs >via gndb / ccs from the RedHat Clustersite. This looks interresting, >but i think this thingy >will lead me to some headaches :) >Does someone here do know a netwide mountable COW fs? Or, maybe some >better idea of >solving/performing this issue? > >Thanks in advance > >Stephan Seitz > > >-----BEGIN PGP SIGNATURE----- >Version: GnuPG v1.4.1 (GNU/Linux) >Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org > >iD8DBQFDefOl0MKSfkbzHDYRAgR6AKCxkRCcZIMZL7tFjXfQ1v8xMXH7fwCcCxlk >3lDcx4ruSQRa4fY0kYr40oo>=mXwG >-----END PGP SIGNATURE----- > > >_______________________________________________ >Xen-users mailing list >Xen-users@lists.xensource.com >http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> Does someone here do know a netwide mountable COW fs? Or, maybe some> better idea of > solving/performing this issue? If i understand corectly the writeable part of the COW fs should be accesible from the network too (to make it easily available after migration). If so there is prety litle point in using unionfs or similar solutions as other sugested, it would be simpler (imo) to export the LVM snapshot from wherever it''s hosted via NBD or even plain NFS. Also GFS is not that much of a headache, i have used it myself (not inside xen tho) some time ago and after the initial i''m-trying-to-use-something-new-without-reading-docs mess it worked prety simply and without too much problems :) -- Best regards, Black Dew. =============================~ bdew@bdew.yi.org ~ ICQ:2666606 ~ 2:400/567 ~ _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Thank you all for your answers,
as a first selection, i''m trying to get cownfsd working. The
serverpart (cownfsd itself) seems to work pretty simple, but i''ve got
problems to mount it...
I didn''t find any documentation about cownfs, neither at the xen-*
lists nor at google at all.
As written on russross.com, cownfsd has been developed with xen in
mind, i''m wondering that
there''s no discussion about it''s use?
Did some of you work with it succesfully? If so, how does it interact
with the existing rpc.mountd/
rpc.nfsd ?
Russ wrote, that the native rpc.mountd has to be killed, but if i do
so, an example
xen-02:~# mount -t nfs xen-01:/xenvm/mosiXen-01 /mnt
will lead (on every host, even on localhost) to:
mount: RPC: Program/version mismatch; low version = 3, high version = 3
i recompiled the nfs-utils and the util-linux (2.12r) on every dom0,
but this didn''t show any effect, the dmesg shows everytime the same
message:
nfs warning: mount version older than kernel
on the server, cownfsd shows following error(?) :
xen-01:~# cownfsd -debug /xenvm/mosiXen-01
fsinfo (fhToFake failed in nfs_api_debug.ml) -> nfs3err_stale error
getattr (fhToFake failed in nfs_api_debug.ml) -> nfs3err_stale error
fsinfo (fhToFake failed in nfs_api_debug.ml) -> nfs3err_stale error
getattr (fhToFake failed in nfs_api_debug.ml) -> nfs3err_stale error
fsinfo (fhToFake failed in nfs_api_debug.ml) -> nfs3err_stale error
timings:
getattr: count= 2 total= 0.00072622 avg= 0.00036311
fsinfo: count= 3 total= 0.06539297 avg= 0.02179766
getattr (fhToFake failed in nfs_api_debug.ml) -> nfs3err_stale error
fsinfo (fhToFake failed in nfs_api_debug.ml) -> nfs3err_stale error
getattr (fhToFake failed in nfs_api_debug.ml) -> nfs3err_stale error
i left the whole log intact, since i don''t know how (or even when) the
"timings" msg
appears. every single trial to mount results in two lines (fsinfo...
and getaddr ...)
Just as additional info, a complete rootfs, which can be chroot''ed or
with kernel-nfs used in a domU, is natively at /xenvm/mosiXen.
/xenvm/mosiXen-01 is an empty directory with one single file named
.mount with one line:
write "/xenvm/mosiXen"
All things are currently done as root. rpc is running as shown below:
on dom0 at xen-01 (currently the nfs server):
xen-01:~# ps xa | grep rpc
4321 ? S< 0:00 [rpciod/0]
4453 ? Ss 0:00 /sbin/rpc.statd
5141 pts/3 S+ 0:00 grep rpc
xen-01:~# rpcinfo -p
Program Vers Proto Port
100000 2 tcp 111 portmapper
100000 2 udp 111 portmapper
100003 2 udp 2049 nfs
100003 2 tcp 2049 nfs
100021 1 udp 1024 nlockmgr
100021 3 udp 1024 nlockmgr
100021 4 udp 1024 nlockmgr
100021 1 tcp 1025 nlockmgr
100021 3 tcp 1025 nlockmgr
100021 4 tcp 1025 nlockmgr
100024 1 udp 1026 status
100024 1 tcp 1026 status
100005 3 udp 944 mountd
100003 3 udp 944 nfs
100005 3 tcp 947 mountd
100003 3 tcp 947 nfs
on dom0 at xen-02 (an example "client"):
xen-02:~# ps xa | grep rpc
4671 ? Ss 0:00 /sbin/rpc.statd
4857 ? S< 0:00 [rpciod/0]
14591 pts/0 S+ 0:00 grep rpc
xen-02:~# rpcinfo -p
program vers proto port
100000 2 tcp 111 portmapper
100000 2 udp 111 portmapper
100024 1 udp 1024 status
100024 1 tcp 1025 status
100003 2 udp 2049 nfs
100003 3 udp 2049 nfs
100003 2 tcp 2049 nfs
100003 3 tcp 2049 nfs
100021 1 udp 1026 nlockmgr
100021 3 udp 1026 nlockmgr
100021 4 udp 1026 nlockmgr
100021 1 tcp 1027 nlockmgr
100021 3 tcp 1027 nlockmgr
100021 4 tcp 1027 nlockmgr
Well, i know that this is not exactly a xen problem, but i assume it''s
very xen-related.
Maybe someone can point me to a solution?
Thanks!
Stephan Seitz
M.A. Williamson schrieb:
> Why not just export your LVM-ified COW volume over NFS?
>
> Alternatively: * Russ Ross worked on a COW NFS server:
> http://www.russross.com/CoWNFS.html * You could export the CoW
> volume directly from the server using enbd. You could import this
> at the destination dom0 and export it to the guest as a VBD - the
> guest wouldn''t know about the network, or the COW. The block-enbd
> scripts in the unstable tree automate the network device binding
> and exporting to the guest: they should "just work" without
manual
> setup (including during migrate).
>
> Cheers, Mark
>
> On Nov 15 2005, Stephan Seitz wrote:
>
> hi there!
>
> i''m currently building a clustered environment of dom0 for
domU''s
> possible to migrate between them. by now, the domU''s rootfs is
> mounted via nfs from one single nfsd w/ reiser4 in the net. this
> works nice, but a COW-fs would do it better :) . i found very rare
> infos about this parallax fs integration in xen, but this seems to
> be under heavy development and not going to get productive right
> now? another solution with COW support, that works LOCALLY was a
> lvm setup (well only one vg over one pv) with the rootfs as real
> lv. the VM''s rootfs has been done via "lvmcreate -s" as
snapshot.
> Well, as said, this did it LOCALLY and -except of some unremovable
> snapshots- worked also very well. I''m searching for a solution
with
> networktransparent COW support. I''ve only found the gfs via gndb /
> ccs from the RedHat Clustersite. This looks interresting, but i
> think this thingy will lead me to some headaches :) Does someone
> here do know a netwide mountable COW fs? Or, maybe some better idea
> of solving/performing this issue?
>
> Thanks in advance
>
> Stephan Seitz
>
>
>>
>>
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users>>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFDfFSZ0MKSfkbzHDYRAugwAKCxLSQLwcd0bGV7B2UNAYXN9jGtmACeNd51
CpYI2Bu0/kWdot3ilgn1D3k=abtz
-----END PGP SIGNATURE-----
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 UPDATE to my last posting about cownfsd ... sorry, i missed a very basic thing in my configuration here: if i start cownfsd /xenvm/mosiXen-01 , i need to mount server:/ instead of server:/xenvm/mosiXen-01 ... :) sorry for wasting your time Stephan Seitz schrieb:> Thank you all for your answers, > > as a first selection, i''m trying to get cownfsd working. The-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFDfFx30MKSfkbzHDYRAnaiAJ4uT/g0dtTTPzGVCUw73QzJt4wMUgCcD8Dj o2+Tqgqo2kmgUbtNwarTVLk=uxWC -----END PGP SIGNATURE----- _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users