We have about 5 servers spread out throughout the country: 2 Centos 4.4, 1 Red Hat and 2 windows machines- looking for some good ideas for a back-up solution. Our servers hold a lot of data, but only a small portion of it is crucial on any given day. That crucial portion is always changing (for example, this week the crucial files all relate to November- current and previous projects, and the next project in early December). What we need to have is a backup solution that would allow us to roll back to any given day (about four weeks of daily history). Bare-metal backups are a must. Our ideal solution would be to find someone who could design and implement a system for us. All ideas and suggestions are welcome.
On Wed, Nov 22, 2006 at 11:26:22AM -0800, System Admin wrote:> We have about 5 servers spread out throughout the country: 2 Centos 4.4, > 1 Red Hat and 2 windows machines- looking for some good ideas for a > back-up solution. Our servers hold a lot of data, but only a small > portion of it is crucial on any given day. That crucial portion is > always changing (for example, this week the crucial files all relate to > November- current and previous projects, and the next project in early > December). > What we need to have is a backup solution that would allow us to roll > back to any given day (about four weeks of daily history). Bare-metal > backups are a must. > > Our ideal solution would be to find someone who could design and > implement a system for us. All ideas and suggestions are welcome.What is the size of the dataset? What is the size of the daily change? What is the size and throughput restrictions of respective internet connectivity? Is there (and how big is the) budget? Are the machines under the control of local administrators, or are they hands-off remote hosted? Are there any restrictions (say, inability to hire or use Canadian based services) that might be relevant? -- /\oo/\ / /()\ \ David Mackintosh | Public Key: dave at xdroop.com | http://www.xdroop.com/dave/gpg.html $ gpg --recv-keys --keyserver subkeys.pgp.net 4C032504 Mystery attachment? http://wiki.xdroop.com/space/GPG -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: <http://lists.centos.org/pipermail/centos/attachments/20061122/bdeb9d27/attachment-0002.sig>
System Admin wrote:> We have about 5 servers spread out throughout the country: 2 Centos 4.4, > 1 Red Hat and 2 windows machines- looking for some good ideas for a > back-up solution. Our servers hold a lot of data, but only a small > portion of it is crucial on any given day. That crucial portion is > always changing (for example, this week the crucial files all relate to > November- current and previous projects, and the next project in early > December). > What we need to have is a backup solution that would allow us to roll > back to any given day (about four weeks of daily history). Bare-metal > backups are a must. > > Our ideal solution would be to find someone who could design and > implement a system for us. All ideas and suggestions are welcome. > > _______________________________________________ > CentOS mailing list >Hi! Have a look at: http://www.mikerubel.org/computers/rsync_snapshots/ I use this kind of setup for my clients, works like a charm. I made my own scripts and we can hold easily last 30 days in rolling scheme. It works over SSH and support compression of the stream. RSync is very efficient. For Winblows, check: http://www.itefix.no/phpws/index.php?module=pagemaster&PAGE_user_op=view_page&PAGE_id=6&MMN_position=23:23 Hope this helps! Guy Boisvert IngTegration inc.
Guy Boisvert wrote:> System Admin wrote: >> We have about 5 servers spread out throughout the country: 2 Centos >> 4.4, 1 Red Hat and 2 windows machines- looking for some good ideas >> for a back-up solution. Our servers hold a lot of data, but only a >> small portion of it is crucial on any given day. That crucial >> portion is always changing (for example, this week the crucial files >> all relate to November- current and previous projects, and the next >> project in early December). What we need to have is a backup >> solution that would allow us to roll back to any given day (about >> four weeks of daily history). Bare-metal backups are a must. >> >> Our ideal solution would be to find someone who could design and >> implement a system for us. All ideas and suggestions are welcome. >> >> _______________________________________________ >> CentOS mailing list >> > > Hi! > > Have a look at: http://www.mikerubel.org/computers/rsync_snapshots/ > > I use this kind of setup for my clients, works like a charm. I made > my own scripts and we can hold easily last 30 days in rolling scheme. > It works over SSH and support compression of the stream. RSync is > very efficient. For Winblows, check: > >http://www.itefix.no/phpws/index.php?module=pagemaster&PAGE_user_op=view_pag e&PAGE_id=6&MMN_position=23:23> > Hope this helps! > > Guy Boisvert > IngTegration inc.I've also written up a howto on this very subject. You can read about it here: http://marks-tech-pages.blogspot.com . I got started with mikerubel, and took it further. HTH Mark
On Wed, 2006-11-22 at 11:26 -0800, System Admin wrote:> We have about 5 servers spread out throughout the country: 2 Centos 4.4, > 1 Red Hat and 2 windows machines- looking for some good ideas for a > back-up solution. Our servers hold a lot of data, but only a small > portion of it is crucial on any given day. That crucial portion is > always changing (for example, this week the crucial files all relate to > November- current and previous projects, and the next project in early > December). > What we need to have is a backup solution that would allow us to roll > back to any given day (about four weeks of daily history). Bare-metal > backups are a must. > > Our ideal solution would be to find someone who could design and > implement a system for us. All ideas and suggestions are welcome. > > _______________________________________________ > CentOS mailing list > CentOS at centos.org > http://lists.centos.org/mailman/listinfo/centosif you seem to be that involved w/centos through-out your system, why wouldn't you consider Centos paid support? those guys are good!
Cian Cullinan wrote:> The only fully bare-metal ready backup solution I know of for CentOS > is mondorescue. > IME, it's the bare-metal recovery that is going to be the hardest > requirement for you to meet (at least for free-as-in-beer software), > the rest can be done by many different pieces of available software: > backuppc, bacula, rdiff-snapshot, dervish etc. > > CianMondo is great, but there's another option. Take a look at http://sourceforge.net/projects/g4l . I've used this on windows system successfully, and it'll work great on CentOS. Thanks! Mark Schoonover IS Manager American Geotechnical V: 858-450-4040 - F: 714-685-3909 - C: 858-472-3816 "Stop the Earth!! I want off!!"
Cian Cullinan wrote:> On 11/22/06, Mark Schoonover <schoon at amgt.com> wrote: >> Mondo is great, but there's another option. Take a look at >> http://sourceforge.net/projects/g4l . I've used this on windows >> system successfully, and it'll work great on CentOS. >> > > This doesn't seem to do differential backups though, only full images?Well, the way I configure my servers is that no data gets stored on an OS partition. Once the server is built, I create an image of it, and store it off on another machine. For any incremental changes, I use rsync to keep track of changes for the entire system. So, if a drive should die, I can pop in a new drive and restore with g4l. Then use the latest rsync backup to restore all the changes. I've done this on several occasions, and it works very well, even on systems with terabytes of data. The key is to keep data and OS software on seperate partitions, or better yet, drives. Thanks! Mark Schoonover IS Manager American Geotechnical V: 858-450-4040 - F: 714-685-3909 - C: 858-472-3816 "Stop the Earth!! I want off!!"
chrism at imntv.com wrote:> Cian Cullinan wrote: >> On 11/22/06, Mark Schoonover <schoon at amgt.com> wrote: >>> Mondo is great, but there's another option. Take a look at >>> http://sourceforge.net/projects/g4l . I've used this on windows >>> system successfully, and it'll work great on CentOS. >>> >> >> This doesn't seem to do differential backups though, only full >> images? > > Also, I'm not sure if it will work on a live system. Is G4L able to > image the drive even with open files running in multi-user mode? > > If so, that would be really sweet.I don't know, I don't do it this way. For file level backups, I use rsync to another system. Mark
On 23/11/06, System Admin <administrator at insurancejournal.com> wrote:> We have about 5 servers spread out throughout the country: 2 Centos 4.4, > 1 Red Hat and 2 windows machines- looking for some good ideas for a > back-up solution. Our servers hold a lot of data, but only a small > portion of it is crucial on any given day. That crucial portion is > always changing (for example, this week the crucial files all relate to > November- current and previous projects, and the next project in early > December). > What we need to have is a backup solution that would allow us to roll > back to any given day (about four weeks of daily history). Bare-metal > backups are a must. > > Our ideal solution would be to find someone who could design and > implement a system for us. All ideas and suggestions are welcome.Your best bet would be rsync on day one. Copy over the day one data to another set called day two. Then rsync day two. Copy over to the set day three and then rsync day three and so on... Since rsync does differential transfer it would be fast and easy to implement and to run from either the client server or backup server. Plus since rsync works on top of ssh you can do key-only authentication so that other logins are thwarted. -- Regards, Sudev Barar