Götz Reinicke
2011-Oct-28 08:34 UTC
[CentOS] You suggestion for 'big' filesystem management Best Practice?
Hi, we plan to set up a big file storage for media files like uncompressed movies from student film projects, dvd images etc. It should be some sort of archive and will not bee accessed by more than may be 5 people at the same time. The iSCSI RAID we have is about 26TB netto and I'm again faced with the question: How many partitions, which filesystem, which mount options etc. For the User it would be the most simpel thing, to have one big filesystem she/he could fill with all the data and dont has to search e.g. on multiple volumes. On the other hand, if one big filesystem crashes or has do be checked it will destroy a lot of data or the check will take hours ... Any suggestions pro or cons are welcome! :-) My favourite for now is 3 to 4 filesystems with the default ext4 settings. (Redhat EL 5.7, may be soon 6.1) Thanks and best regards. G?tz -- G?tz Reinicke IT-Koordinator Tel. +49 7141 969 420 Fax +49 7141 969 55 420 E-Mail goetz.reinicke at filmakademie.de Filmakademie Baden-W?rttemberg GmbH Akademiehof 10 71638 Ludwigsburg www.filmakademie.de Eintragung Amtsgericht Stuttgart HRB 205016 Vorsitzender des Aufsichtsrats: J?rgen Walter MdL Staatssekret?r im Ministerium f?r Wissenschaft, Forschung und Kunst Baden-W?rttemberg Gesch?ftsf?hrer: Prof. Thomas Schadt
Rajagopal Swaminathan
2011-Oct-28 09:11 UTC
[CentOS] You suggestion for 'big' filesystem management Best Practice?
Greetings, 2011/10/28 G?tz Reinicke <goetz.reinicke at filmakademie.de>:> Hi, > > > The iSCSI RAID we have is about 26TB netto and I'm again faced with the > question: How many partitions, which filesystem, which mount options etc. > > My favourite for now is 3 to 4 filesystems with the default ext4 > settings. (Redhat EL 5.7, may be soon 6.1) >You should seriously consider XFS (supported natively in RHEL and hence Centos 6) for any single filesystem >16TB. (EXT4 supports only upto 16TB) If you are going to have an RHCS for HA, then GFS2. But I cant comment on its performance for streaming applications etc. -- Regards, Rajagopal
John Doe
2011-Oct-28 10:24 UTC
[CentOS] You suggestion for 'big' filesystem management Best Practice?
From: G?tz Reinicke <goetz.reinicke at filmakademie.de>> The iSCSI RAID we have is about 26TB netto and I'm again faced with the > question: How many partitions, which filesystem, which mount options etc. > For the User it would be the most simpel thing, to have one big > filesystem she/he could fill with all the data and dont has to search > e.g. on multiple volumes. > On the other hand, if one big filesystem crashes or has do be checked it > will destroy a lot of data or the check will take hours ...Splitting the space, if you have the option, has advantages... You already mentioned fsck. You could assign different partitions to different groups of people. You could, depending on your RAID level, create several arrays on different drives to limit disk contention (but you will lose either some space and/or some speed)... JD
Bent Terp
2011-Oct-28 10:33 UTC
[CentOS] You suggestion for 'big' filesystem management Best Practice?
2011/10/28 G?tz Reinicke <goetz.reinicke at filmakademie.de>> The iSCSI RAID we have is about 26TB netto and I'm again faced with the > question: How many partitions, which filesystem, which mount options etc. >My vote goes to XFS, if only one server needs acces to the LUN's; and GPFS (not GFS) if you need a cluster filesystem. BR Bent (130 TB in one XFS installation, and 600 TB in one gpfs cluster)> For the User it would be the most simpel thing, to have one big > filesystem she/he could fill with all the data and dont has to search > e.g. on multiple volumes. > > On the other hand, if one big filesystem crashes or has do be checked it > will destroy a lot of data or the check will take hours ... > > > Any suggestions pro or cons are welcome! :-) > > My favourite for now is 3 to 4 filesystems with the default ext4 > settings. (Redhat EL 5.7, may be soon 6.1) > > Thanks and best regards. G?tz > -- > G?tz Reinicke > IT-Koordinator > > Tel. +49 7141 969 420 > Fax +49 7141 969 55 420 > E-Mail goetz.reinicke at filmakademie.de > > Filmakademie Baden-W?rttemberg GmbH > Akademiehof 10 > 71638 Ludwigsburg > www.filmakademie.de > > Eintragung Amtsgericht Stuttgart HRB 205016 > > Vorsitzender des Aufsichtsrats: > J?rgen Walter MdL > Staatssekret?r im Ministerium f?r Wissenschaft, > Forschung und Kunst Baden-W?rttemberg > > Gesch?ftsf?hrer: > Prof. Thomas Schadt > > > > > _______________________________________________ > CentOS mailing list > CentOS at centos.org > http://lists.centos.org/mailman/listinfo/centos > >
Adam Tauno Williams
2011-Oct-28 14:04 UTC
[CentOS] You suggestion for 'big' filesystem management Best Practice?
On Fri, 2011-10-28 at 10:34 +0200, G?tz Reinicke wrote:> we plan to set up a big file storage for media files like uncompressed > movies from student film projects, dvd images etc. > It should be some sort of archive and will not bee accessed by more than > may be 5 people at the same time. > The iSCSI RAID we have is about 26TB netto and I'm again faced with the > question: How many partitions, which filesystem, which mount options etc.> For the User it would be the most simpel thing, to have one big > filesystem she/he could fill with all the data and dont has to search > e.g. on multiple volumes.Use LVM. Divide it into numerous partitions. Initialize the partitions as Physical Volumes. Create a Volume Group containing the Physical Volumes. Then create the volumes you allocate storage as using Logical Volumes. Your LV can span as many, or as few, PVs as you like. To the SAN each PV is a partition, you can move it, delete it, etc.. In the OS you can move data between PVs, migrate LVs between PVs, etc.. Even connect to another SAN, attach PVs there to you volume group, and migrate the LVs to those PVs - this migrating data from one SAN to another. Then drop the PVs on the old SAN from the VG.