CentOS - Jan 2009 - Hierarchial storage management or automated archival to tape

Hi list!

While regular backup solutions like amanda or bacula are very good at
their job, ie keeping point2point copies of the files currently on
disk, I find them less suited for archiving - having unused files move
to tape in duplo and stay there until requested. I've even read of
multi-tier solutions - move to slower disks after a week and further
on to tape after a month.

Does anyone have some experience or suggestions for this? The project
will deal with ~100 TB of growth per year, most files somewhere
between 2 and 50 GB.

Yes, it can be done with everything in one filesystem, but I'm
concerned about running a full backup every month of 500 TB ;-) Not to
mention the time required for recovery in case the filesys crashes....

BR Bent

>>> On Thu, Jan 8, 2009 at 10:37 AM, in message
<77c4f5c60901080737ka0ee2fbr7f6badc9fc48e4f3 at mail.gmail.com>, Bent Terp
<bent at terp.se> wrote: 
> Hi list!
> 
> While regular backup solutions like amanda or bacula are very good at
> their job, ie keeping point2point copies of the files currently on
> disk, I find them less suited for archiving - having unused files move
> to tape in duplo and stay there until requested. I've even read of
> multi-tier solutions - move to slower disks after a week and further
> on to tape after a month.
> 
> Does anyone have some experience or suggestions for this? The project
> will deal with ~100 TB of growth per year, most files somewhere
> between 2 and 50 GB.
> 
> Yes, it can be done with everything in one filesystem, but I'm
> concerned about running a full backup every month of 500 TB ;-) Not to
> mention the time required for recovery in case the filesys crashes....
> 
> BR Bent

We use SAM-FS to do just that here in the libraries. 
http://opensolaris.org/os/project/samqfs/What_are_QFS_and_SAM/

It was open sourced last year. Not a simple tool to implement but it does work
as advertised.


Tony Placilla <aplacilla at jhu.edu>
Sr. UNIX Systems Administrator
The Sheridan Libraries
Johns Hopkins University

On Thu, Jan 8, 2009 at 5:08 PM, Tony Placilla
<aplacil1 at jhuadig.admin.jhu.edu> wrote:
> We use SAM-FS to do just that here in the libraries.
> http://opensolaris.org/os/project/samqfs/What_are_QFS_and_SAM/
Thanks for the pointer! Do you use SAM with QFS or ZFS? Or some sort of combo?

/Bent