I''m currently using windows DFS to replicate changes to a fairly small (4gb) web server directory. I would like something that performs a little better then DFS especially in terms of initial replication. Currently it can take several days to replicate 4gb of data from one server to another (over local lan). Is Lustre the right filesystem to use (I am willing to set these servers up on linux if need be). I basically want 2 servers that have an identical copy of a directory (or volume). I want to be able to make changes to either of those servers and have the changes automatically propagate to the other server in a timely fashion. DFS handles this well once it''s done the initial sync for small changes. If you make a change that affects a large number of files, DFS can take forever to replicate it. Russ
On Oct 06, 2006 18:04 -0400, Ruslan Sivak wrote:> Is Lustre the right filesystem to use (I am willing to set these servers > up on linux if need be). I basically want 2 servers that have an > identical copy of a directory (or volume). I want to be able to make > changes to either of those servers and have the changes automatically > propagate to the other server in a timely fashion. DFS handles this > well once it''s done the initial sync for small changes. If you make a > change that affects a large number of files, DFS can take forever to > replicate it.Lustre does not currently do data replication at the lustre level. It will work with hardware or software RAID1, and we are working at doing OST-level data mirroring/RAID for a future release. That will still not be exactly what you are describing. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc.
If you use drbd to handle the replication at the block device level, however, Lustre may still be a good choice for the filesystem on top, especially if you have more than 2 servers that can be used to take advantage of Lustre''s parallelism (although you can do this even with just 2 active/active servers)... On Wed, 11 Oct 2006, Andreas Dilger wrote:> On Oct 06, 2006 18:04 -0400, Ruslan Sivak wrote: >> Is Lustre the right filesystem to use (I am willing to set these servers >> up on linux if need be). I basically want 2 servers that have an >> identical copy of a directory (or volume). I want to be able to make >> changes to either of those servers and have the changes automatically >> propagate to the other server in a timely fashion. DFS handles this >> well once it''s done the initial sync for small changes. If you make a >> change that affects a large number of files, DFS can take forever to >> replicate it. > > Lustre does not currently do data replication at the lustre level. It will > work with hardware or software RAID1, and we are working at doing OST-level > data mirroring/RAID for a future release. That will still not be exactly > what you are describing. > > Cheers, Andreas > -- > Andreas Dilger > Principal Software Engineer > Cluster File Systems, Inc. > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss@clusterfs.com > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss > >
Hi Brent: On 10/12/06, Brent A Nelson <brent@phys.ufl.edu> wrote:> If you use drbd to handle the replication at the block device level, > however, Lustre may still be a good choice for the filesystem on top, > especially if you have more than 2 servers that can be used to take > advantage of Lustre''s parallelism (although you can do this even with > just 2 active/active servers)...Are you doing active/active with drbd 0.8.x? I am simply curious if this is in a production environment and how well it''s working for you. Regards, -- Mustafa A. Hashmi mahashmi@gmail.com mh@stderr.net
You can''t do a drbd0.8-style active/active, i.e. you can''t have a single storage partition be served by multiple servers simultaneously. However, you can serve different partitions/disks/logical volumes/whatever from different servers and be active/active that way (Lustre will merge all these different partitions into one filesystem, or for however many you have it configured). If one node fails, you just failover its partitions to the drbd "sister" server. We''re not in production, yet, (we still have to figure out the stale nfs file handle issue when exporting via NFS that was mentioned a few messages ago), but it seems to work fine. Failover can be slow due to the client reconnect timer, but that is tweakable. Thanks, Brent Nelson Director of Computing Dept. of Physics University of Florida On Fri, 27 Oct 2006, Mustafa A. Hashmi wrote:> Hi Brent: > > On 10/12/06, Brent A Nelson <brent@phys.ufl.edu> wrote: >> If you use drbd to handle the replication at the block device level, >> however, Lustre may still be a good choice for the filesystem on top, >> especially if you have more than 2 servers that can be used to take >> advantage of Lustre''s parallelism (although you can do this even with >> just 2 active/active servers)... > > Are you doing active/active with drbd 0.8.x? I am simply curious if > this is in a production environment and how well it''s working for you. > > Regards, > -- > Mustafa A. Hashmi > mahashmi@gmail.com > mh@stderr.net >
On 10/27/06, Brent A Nelson <brent@phys.ufl.edu> wrote:> You can''t do a drbd0.8-style active/active, i.e. you can''t have a single > storage partition be served by multiple servers simultaneously.Actually, drbd 0.8 allows for both nodes to be masters now -- I was wondering if this what you meant by active/active on nodes.> However, you can serve different partitions/disks/logical volumes/whatever > from different servers and be active/active that way (Lustre will merge > all these different partitions into one filesystem, or for however many > you have it configured). If one node fails, you just failover its > partitions to the drbd "sister" server.That is indeed our current setup -- however, that makes an active/passive setup where the replicated drbd partitions are passively standing by till heartbeat fails them over (and makes them primary).> We''re not in production, yet, (we still have to figure out the stale nfs > file handle issue when exporting via NFS that was mentioned a few messages > ago), but it seems to work fine. Failover can be slow due to the > client reconnect timer, but that is tweakable.Appreciate your quick response, thank you and good luck! Regards, -- Mustafa A. Hashmi mahashmi@gmail.com mh@stderr.net
On Fri, 27 Oct 2006, Mustafa A. Hashmi wrote:> On 10/27/06, Brent A Nelson <brent@phys.ufl.edu> wrote: >> You can''t do a drbd0.8-style active/active, i.e. you can''t have a single >> storage partition be served by multiple servers simultaneously. > > Actually, drbd 0.8 allows for both nodes to be masters now -- I was > wondering if this what you meant by active/active on nodes. >Right; you can''t do that with a Lustre partition. GFS can do that, although I would expect its performance to be much lower than Lustre.>> However, you can serve different partitions/disks/logical volumes/whatever >> from different servers and be active/active that way (Lustre will merge >> all these different partitions into one filesystem, or for however many >> you have it configured). If one node fails, you just failover its >> partitions to the drbd "sister" server. > > That is indeed our current setup -- however, that makes an > active/passive setup where the replicated drbd partitions are > passively standing by till heartbeat fails them over (and makes them > primary). >active/passive from a storage partition standpoint, right, but active/active from a node-level point-of-view. One server serves out half the storage while the other server serves out the other half (even if it''s the same Lustre filesystem, so you do get the performance boost of both servers running simultaneously). Thanks, Brent