Hi I''m new to the world of HPC and cluster :-( and I was given the task of setting up a new (small) HPC cluster based on 36 Sun X2200 M2 systems. Each system as 1 136GB SAS HD. I''ve installed CentOS 5.2 as the base OS on all the systems and through partitinoing managed to have a unused partition size 65GB (and with 36 nodes I get +2TB of distributed storage). Tthe rest of the HD is as follows: 4xRAM = 64GB swap and ~10GB for the OS itself. I want ot put a Clustered FS on the unused partitions and from looking around seems that Lustre is king of the hill ... The question I have is: Will I gain anything from using Lustre in such a tight environemnt or am I going to degrade the cluster''s performnce that it simply not worth the effort? -- TIA Paolo -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080905/1031eded/attachment.html
Paolo, The biggest problem with using Lustre in this environment is that the Lustre _servers_ (those providing the disk space) cannot _use_ the filesystem (cannot have a client and a server on same machine). Kevin Paolo Supino wrote:> Hi > > I''m new to the world of HPC and cluster :-( and I was given the task > of setting up a new (small) HPC cluster based on 36 Sun X2200 M2 > systems. Each system as 1 136GB SAS HD. I''ve installed CentOS 5.2 as > the base OS on all the systems and through partitinoing managed to > have a unused partition size 65GB (and with 36 nodes I get +2TB of > distributed storage). Tthe rest of the HD is as follows: 4xRAM = 64GB > swap and ~10GB for the OS itself. I want ot put a Clustered FS on the > unused partitions and from looking around seems that Lustre is king of > the hill ... > The question I have is: Will I gain anything from using Lustre in > such a tight environemnt or am I going to degrade the cluster''s > performnce that it simply not worth the effort? > > > > -- > TIA > Paolo > > ------------------------------------------------------------------------ > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss >
On Sep 05, 2008 07:59 -0600, Kevin Van Maren wrote:> The biggest problem with using Lustre in this environment is that the > Lustre _servers_ (those providing the disk space) cannot _use_ the > filesystem (cannot have a client and a server on same machine).Well, it depends on how much memory pressure there is. It definitely works with client-on-OST (I run my home system that way, and lots of Lustre developers do functional tests in that config) but it has the chance of deadlock if there are memory-hungry apps running on the same OSS. This could be avoided by forcing all writes to be to remote OSS nodes, possibly at the cost of some performance, though network IO on a 1GigE is faster than to a single disk if you have a good switch. The other (more major, IMHO) issue is that if the client/OSS node crashes, it takes its disk with it, and now other clients cannot access the data there. Similarly, without RAID of the disk, any disk loss would mean loss of 1/36 of the filesystem. This could be avoided by having RAID1 of the disks using DRBD to a remote node that is configured as the OST failover. There is a wiki page on the lustre wiki about using DRBD. Please keep the list informed (and wiki updated) if you decide to use this configuration.> Paolo Supino wrote: > > I''m new to the world of HPC and cluster :-( and I was given the task > > of setting up a new (small) HPC cluster based on 36 Sun X2200 M2 > > systems. Each system as 1 136GB SAS HD. I''ve installed CentOS 5.2 as > > the base OS on all the systems and through partitinoing managed to > > have a unused partition size 65GB (and with 36 nodes I get +2TB of > > distributed storage). Tthe rest of the HD is as follows: 4xRAM = 64GB > > swap and ~10GB for the OS itself. I want ot put a Clustered FS on the > > unused partitions and from looking around seems that Lustre is king of > > the hill ... > > The question I have is: Will I gain anything from using Lustre in > > such a tight environemnt or am I going to degrade the cluster''s > > performnce that it simply not worth the effort?Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.