I want to build 2 servers (both running samba) to provide file storage to 2 offices (approx 100 miles apart, linked via DSL) but all data writen to 1 server must also be saved to the other server. Both servers would also allow users to access the data via a VPN thus allowing 1 office with a failed server to access the other server via the vpn and still see the data from both offices. I currently have 1 server working but we want to add the second office to the system. (Currently 1 office has 10 users and the second office has 1 user connected via VPN ) but the second office will have 20 within 12 months and the first will have 35 soon )) Has anyone done anything like this ? I am currently reading & unpicking http://www.linux-ha.org/ to see what that gives me. Any clues/comments very welcome, even if you think I am mad ! Thanks Denis
Denis Croombs <denis at croombs.org> wrote:> I want to build 2 servers (both running samba) to provide > file storage to 2 offices (approx 100 miles apart, linked > via DSL) but all data writen to 1 server must also be saved > to the other server.What do you want to synchronize? Directory services? User authentication? Or files? Directory and user authentication are very doable. File services are typically not, or limited.> Both servers would also allow users to access the data via > a VPN thus allowing 1 office with a failed server to access > the other server via the vpn and still see the data fromboth> offices.You're going to _saturate_ your link between the two offices if you start synchronizing files in real-time. Unless you have something like a 4+ T-1s, or a 6+Mbps SDSL (i.e., synchronous DSL -- 6Mbps upload as well as download), you're going to find you're not going be able to do this in "real-time." At the best, you can use rsync to synchronize files at several points in the day -- maybe lunch-time and middle of the night. To get real-time, you're going to find the load on your network will be self-defeating.> I currently have 1 server working but we want to add the > second office to the system. (Currently 1 office has 10users> and the second office has 1 user connected via VPN ) butthe> second office will have 20 within 12 months and the first > will have 35 soon )) > Has anyone done anything like this ?Yes and no. Yes, I've done it. I've done it at high speed in the same closet (or nearby closet with SAN) with the same storage. NFS/Samba with failover c/o Mission Critical Linux (now part of RHEL). You need multi-targetable hardware (not cheap). But no, I haven't done it (nor would I do it) over a VPN link. You'll saturate it quickly. File server clustering is designed for high speed connections between servers and their storage.> I am currently reading & unpicking http://www.linux-ha.org/ > to see what that gives me."High-availability" (HA) doesn't necessarily mean "failover." Most HA implementations are for read-only web services, destination NAT (DNAT), etc... When you start talking failover of network file servers, then you start talking mega-$$$, lots of bandwidth/synchronization requirements, hardware, etc... to do it real-time. GFS reduced the $$$ and same closet requirement, but it also expoentially increases bandwidth and other costs. At best, you'll have to do it non-real-time, using something like rsync. It won't be fail-over at all.> Any clues/comments very welcome, even if you think I am mad > !I don't think you're mad. But I don't think you're aware of what is all involved with real-time failover of network file services. And it's really going to be near impossible over low-throughput Internet connections. I'd look to non-real-time rsync instead, running off-hours. That's the best you can do unless you have a lot of bandwidth and a lot of money. The majority of the HA stuff will _not_ apply. ;-> -- Bryan J. Smith | Sent from Yahoo Mail mailto:b.j.smith at ieee.org | (please excuse any http://thebs413.blogspot.com/ | missing headers)
Denis Croombs wrote:> I want to build 2 servers (both running samba) to provide file storage to 2 > offices (approx 100 miles apart, linked via DSL) but all data writen to 1 > server must also be saved to the other server. > Both servers would also allow users to access the data via a VPN thus > allowing 1 office with a failed server to access the other server via the > vpn and still see the data from both offices. > I currently have 1 server working but we want to add the second office to > the system. (Currently 1 office has 10 users and the second office has 1 > user connected via VPN ) but the second office will have 20 within 12 months > and the first will have 35 soon )) > > Has anyone done anything like this ? > > I am currently reading & unpicking http://www.linux-ha.org/ to see what that > gives me. > > Any clues/comments very welcome, even if you think I am mad ! > > Thanks > > DenisI'd take a look at drbd (www.drbd.org) while you're reading on heartbeat. I use it for web servers, but I'm not sure if it would be suitable for file servers. -- Ugo -> Please don't send a copy of your reply by e-mail. I read the list. -> Please avoid top-posting, long signatures and HTML, and cut the irrelevant parts in your replies.
>I want to build 2 servers (both running samba) to provide file storage to 2 >offices (approx 100 miles apart, linked via DSL) but all data writen to 1 >server must also be saved to the other server. >Both servers would also allow users to access the data via a VPN thus >allowing 1 office with a failed server to access the other server via the >vpn and still see the data from both offices. >I currently have 1 server working but we want to add the second office to >the system. (Currently 1 office has 10 users and the second office has 1 >user connected via VPN ) but the second office will have 20 within 12months>and the first will have 35 soon ))>Has anyone done anything like this ?I am currently synchronizing multiple office locations using a program called unison. Unison (http://www.cis.upenn.edu/~bcpierce/unison/) is a very well written program that can perform 2 way file synchronization. There are many configurable options with unison and I recommend that you check it out. In each office I have a PII350 128RAM Fedora or CentOS server running unison and the files are accessed via samba. I also configure samba to hide (veto) all of the temporary files used during synchronization. For redundancy I place a slave server with each master server that backs up all the user data / file system using rsync. This way if one of my $5 PII servers catches fire I can automatically switch over with no downtime for the users. The only downfall I have encountered is with Autocad files not properly reading the synchronized .dwl lock file and more than one user working on the same file. As a work around for this I have configured Unison to keep a backup of the last 20 versions of a file. This way I can always hit my backups to retreive lost data. As a side note, if anyone knows a work around for the stubborn autocad dwl lock file let me know :))! In any case my implementation has allowed me to synchronize file systems between 4 offices (3 in Canada, 1 in USA), using recycled hardware that was otherwise going to be donated/trashed. Let me know if you have any further questions. AJ Darton, B.COMM CIS System Consultant
On 12/7/05, Denis Croombs <denis at croombs.org> wrote:> On 12/6/05, Jonathan Darton <jdarton at ibigroup.com> wrote: > >> >I want to build 2 servers (both running samba) to provide file > >> >storage to 2 offices (approx 100 miles apart, linked via DSL) but all > >> >data writen to 1 server must also be saved to the other server. > >> >Both servers would also allow users to access the data via a VPN thus > >> >allowing 1 office with a failed server to access the other server via > >> >the vpn and still see the data from both offices. > >> >I currently have 1 server working but we want to add the second > >> >office to the system. (Currently 1 office has 10 users and the second > >> >office has 1 user connected via VPN ) but the second office will have > >> >20 within 12 > >> months > >> >and the first will have 35 soon )) > >> > >> >Has anyone done anything like this ? > >> > >> I am currently synchronizing multiple office locations using a program > >> called unison. Unison (http://www.cis.upenn.edu/~bcpierce/unison/) is > >> a very well written program that can perform 2 way file > >> synchronization. There are many configurable options with unison and I > recommend that you check it out. > >> In each office I have a PII350 128RAM Fedora or CentOS server running > >> unison and the files are accessed via samba. I also configure samba to > >> hide (veto) all of the temporary files used during synchronization. > >> For redundancy I place a slave server with each master server that > >> backs up all the user data / file system using rsync. This way if one > >> of my $5 PII servers catches fire I can automatically switch over withno> downtime for the users. > >> > >> The only downfall I have encountered is with Autocad files not > >> properly reading the synchronized .dwl lock file and more than one > >> user working on the same file. As a work around for this I have > >> configured Unison to keep a backup of the last 20 versions of a file. > >> This way I can always hit my backups to retreive lost data. As a side > >> note, if anyone knows a work around for the stubborn autocad dwl lock > file let me know :))! > >> > >> In any case my implementation has allowed me to synchronize file > >> systems between 4 offices (3 in Canada, 1 in USA), using recycled > >> hardware that was otherwise going to be donated/trashed. > >> > >> Let me know if you have any further questions. > > > >I'm about to do a Unison setup on two CentOS servers, so I'm thrilled to > see this response. I also work with Architects >>sometimes, so I'm > interested to hear about the dwl lock file issue. > > > >My one compound question: how are you invoking Unison? In batch mode,with> cron? How often? Wat other options did you > >consider before settling on the scheme you use? > > I see the project is no longer supported, do you have rpms for it ? > > Thanks>It's not that it's no longer supported, really, just that the project >that initiated it is not an officially-funded academic project. It has >actually been updated since the termination of the project, it looks >like.>I haven't installed it on CentOS yet but I got FC4 rpms from fedora >extras (I just typed 'yum install unison' and it worked). I think it's >in pretty common usage in RHEL too, so it can't be too hard to find >for CentOS, right? (But let us know what you turn up :-)).I created a script that is run by cron every minute. The script creates a lock file for the session, rsyncs the changes to the backup server, synchronizes offices using unison, deletes the lock file. Unison is executed in "batch" and "silent" mode so that no output is generated and mailed by cron. One other thing you need to consider is the size of the file system you are trying to synchronize. I have a very large directory structure in two offices which required running unison a second time splitting up the synchronization into two sessions. This can easily be done by ignoring a certain path/paths and then synchronizing them in the next session. If you are thinking of synchronizing between more than 2 locations, you need to plan the fastest route (ie. A<->B<->C or B<->A<->C). I have had no problems synchronizing between more than two locations. For all of my implementations using Fedora or CentOS operating systems I have downloaded the unison source, installed the ocaml compiler (easy to find for centos/fedora use google) and compiled from source. I have never had a problem. Another benefit to the implementation that I have been using is the fact that employees in each office can work on files on the server as long as the internal network is functioning, regardless if the synchronization link / externel connection is working. The synchronization is basically transparent to the user, and if it happens to go down it will pick up where it left off as soon as it can regain a connection. Let me know if you require any further information. A. Jonathan Darton, B.COMM CIS System Consultant