Ankush Grover
2008-Feb-06 12:41 UTC
[Samba] Is Samba Shadowcopying can be used in Production Environement with more than 20 TB of data
Hi Friends, I am currently using Samba on Centos 4.4 as a domain member of AD 2003 with each user having a quota of 2GB(no of users is around 2,000). Now the management wants to increase the quota to 10GB with this there will be more than 20TB of data to be backup weekly which will take lots of hours. Currently Veritas backup software is used to backup data on tapes. There is a concept of snapshots of Samba with LVM where snapshots of samba are taken at the given interval but so far haven't found any good article or how-to on that and also what is the experience of users using this technology and also what other technologies are being to handle TBs of data. The plan is like this Samba Server with ShadowCopy Enabled + DAS (Direct Attached Storage) http://www.wlug.org.nz/SambaShadowCopyHowto Kindly let me know if you need any further inputs Thanks & Regards
Michael Heydon
2008-Feb-06 23:18 UTC
[Samba] Is Samba Shadowcopying can be used in Production Environement with more than 20 TB of data
Ankush Grover wrote:> Hi Friends, > > > I am currently using Samba on Centos 4.4 as a domain member of AD 2003 with > each user having a quota of 2GB(no of users is around 2,000). Now > the management wants to increase the quota to 10GB with this there will be > more than 20TB of data to be backup weekly which will take lots of hours. > Currently Veritas backup software is used to backup data on tapes. There is > a concept of snapshots of Samba with LVM where snapshots of samba are taken > at the given interval but so far haven't found any good article or how-to on > that and also what is the experience of users using this technology and also > what other technologies are being to handle TBs of data. > > > The plan is like this > > Samba Server with ShadowCopy Enabled + DAS (Direct Attached Storage) > > > http://www.wlug.org.nz/SambaShadowCopyHowto > > Kindly let me know if you need any further inputs > > > Thanks & Regards >My understanding of the samba ShadowCopy stuff is that it doesn't actually take snapshots itself, you need something else to take the snapshots and once they exist the Samba/ShadowCopy stuff will let the users connect to the server with the standard windows ShadowCopy client to browse the snapshots. While this might be neat, I don't see how it would help you get your data onto tape any quicker or easier. We have something setup here (on a smaller scale) that might be useful. Our main file server rsync's with our backup server every hour (using hardlinks to keep snapshots). Since relatively little data changes between each sync, it is fairly fast (approx 5 minutes with no noticable slowdown for the clients) the backup server can then take as long as it likes to write to tape/etc without affecting the main server. *Michael Heydon - IT Administrator * michaelh@jaswin.com.au <mailto:michaelh@jaswin.com.au>
Chuck Kollars
2008-Feb-07 20:32 UTC
[Samba] Is Samba Shadowcopying can be used in Production Environement with more than 20 TB of data
> ... there will be more than 20TB of data to be > backup weekly which will take lots of hours. ...Check out the `rsync` spinoff of Samba. `Rsync`s basic idea is "copy what's changed" rather than just copying everything. It does so very well and very quickly. Copying only changed files can easily be a couple of orders of magnitude quicker than copying the whole thing. The possible flaw with this strategy that used to keep people from implementing it was that the determination of what's changed had to be _perfect_. A backup's no good if it only contains 99% of the current data. The `rsync` tool provides the needed reliabilty, making this strategy possible in real life rather than just pie in the sky. (Of course your backup medium needs to be a disk farm rather than tapes...) My situation is much smaller than yours: a little over 1000 users with a total of a little over 100GB of data. When I started using `rsync`, my backups went from many hours once a month (clearly not frequent enough, but we couldn't afford to do better) to ~10 minutes every day. (I don't use any features of Samba itself, and I don't use any aspect of LVM.) And that ~10 minutes is even with the backup on a separate machine accessed over a network, so bandwidth's limited to 100MB. A SAN would probably do quite a bit better. (The completely separate machine is our way of avoiding a single point of failure.) (And because the backup is to another disk, the backup disk can be made available read-only to lots of folks. As a result, in my situation anybody can restore any individual file at any time virtually instantaneously.) (The first time will of course take a long long time, but after that daily updates will be real quick.) -Chuck Kollars ____________________________________________________________________________________ Looking for last minute shopping deals? Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping