samba - Feb 2008 - Is Samba Shadowcopying can be used in Production Environement with more than 20 TB of data

Hi Friends,


I am currently using Samba on Centos 4.4 as a domain member of AD 2003 with
each user having a quota of 2GB(no of users is around 2,000). Now
the management wants to increase the quota to 10GB with this there will be
more than 20TB of data to be backup weekly which will take lots of hours.
Currently Veritas backup software is used to backup data on tapes. There is
a concept of snapshots of Samba with LVM where snapshots of samba are taken
at the given interval but so far haven't found any good article or how-to on
that and also what is the experience of users using this technology and also
what other technologies are being to handle TBs of data.


The plan is like this

Samba Server with ShadowCopy Enabled + DAS (Direct Attached Storage)


http://www.wlug.org.nz/SambaShadowCopyHowto

Kindly let me know if you need any further inputs


Thanks & Regards

Ankush Grover wrote:
> Hi Friends,
>
>
> I am currently using Samba on Centos 4.4 as a domain member of AD 2003 with
> each user having a quota of 2GB(no of users is around 2,000). Now
> the management wants to increase the quota to 10GB with this there will be
> more than 20TB of data to be backup weekly which will take lots of hours.
> Currently Veritas backup software is used to backup data on tapes. There is
> a concept of snapshots of Samba with LVM where snapshots of samba are taken
> at the given interval but so far haven't found any good article or
how-to on
> that and also what is the experience of users using this technology and
also
> what other technologies are being to handle TBs of data.
>
>
> The plan is like this
>
> Samba Server with ShadowCopy Enabled + DAS (Direct Attached Storage)
>
>
> http://www.wlug.org.nz/SambaShadowCopyHowto
>
> Kindly let me know if you need any further inputs
>
>
> Thanks & Regards
>   
My understanding of the samba ShadowCopy stuff is that it doesn't 
actually take snapshots itself, you need something else to take the 
snapshots and once they exist the Samba/ShadowCopy stuff will let the 
users connect to the server with the standard windows ShadowCopy client 
to browse the snapshots. While this might be neat, I don't see how it 
would help you get your data onto tape any quicker or easier.

We have something setup here (on a smaller scale) that might be useful. 
Our main file server rsync's with our backup server every hour (using 
hardlinks to keep snapshots). Since relatively little data changes 
between each sync, it is fairly fast (approx 5 minutes with no noticable 
slowdown for the clients) the backup server can then take as long as it 
likes to write to tape/etc without affecting the main server.

*Michael Heydon - IT Administrator *
michaelh@jaswin.com.au <mailto:michaelh@jaswin.com.au>

> ... there will be more than 20TB of data to be 
> backup weekly which will take lots of hours. ...
Check out the `rsync` spinoff of Samba. 

`Rsync`s basic idea is "copy what's changed" rather
than just copying everything. It does so very well and
very quickly. Copying only changed files can easily be
a couple of orders of magnitude quicker than copying
the whole thing.

The possible flaw with this strategy that used to keep
people from implementing it was that the determination
of what's changed had to be _perfect_. A backup's no
good if it only contains 99% of the current data. The
`rsync` tool provides the needed reliabilty, making
this strategy possible in real life rather than just
pie in the sky. 

(Of course your backup medium needs to be a disk farm
rather than tapes...)

My situation is much smaller than yours: a little over
1000 users with a total of a little over 100GB of
data. When I started using `rsync`, my backups went
from many hours once a month (clearly not frequent
enough, but we couldn't afford to do better) to ~10
minutes every day. (I don't use any features of Samba
itself, and I don't use any aspect of LVM.)

And that ~10 minutes is even with the backup on a
separate machine accessed over a network, so
bandwidth's limited to 100MB. A SAN would probably do
quite a bit better. (The completely separate machine
is our way of avoiding a single point of failure.)

(And because the backup is to another disk, the backup
disk can be made available read-only to lots of folks.
As a result, in my situation anybody can restore any
individual file at any time virtually
instantaneously.)

(The first time will of course take a long long time,
but after that daily updates will be real quick.)

-Chuck Kollars


     
____________________________________________________________________________________
Looking for last minute shopping deals?  
Find them fast with Yahoo! Search. 
http://tools.search.yahoo.com/newsearch/category.php?category=shopping