Lustre discuss - Oct 2009 - typical hardware...

Hello,

I have been watching the progression of Lustre for some years now and I am
in a position to construct a storage system as proof of concept in use with
a particular application we have in house.

Although I am no stranger to the basics of high perf interconnects up until
now, all of my storage requirements have been met with FC based SAN''s
so my
practical experience with them is quite low.

My question is basically this. What are people gravitating towards these
days? For a while, it seemed like Infiniband was going to the interconnect
to use due to cost/port as compare t o 10G Ethernet, but 10G seems to be
coming down in price. What are people using for OSS/OST''s ?
I''ve been
reading about the DDN stuff which is impressive but I don''t have access
to
that kind of budget yet.

My initial plan is to tie some HP DL160''s to MSA70''s filled
with 300GB
drives. I realize this won''t give me any sort of failover capability
but
this is just a proof for now. If there is a low cost, high speed shared
device, I''d rather use a failover config in order to get a real feel
for
that this is going to require.

Also, the roadmap for Lustre used to include "raid" personalities for
OSS''s.
has that functionality been deferred or dropped altogether, or is it still
in the works?

I realize this is a bit vague but as I mentioned, right now this is proof of
concept for us and stage one is of course fact finding.

Any suggestions for a rank beginner would be greatly appreciated.


Regards


Corey
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20091030/ac4cd038/attachment.html

On 2009-10-30, at 06:47, Corey Kovacs wrote:
> My question is basically this. What are people gravitating towards  
> these days? For a while, it seemed like Infiniband was going to the  
> interconnect to use due to cost/port as compare t o 10G Ethernet,  
> but 10G seems to be coming down in price. What are people using for  
> OSS/OST''s ? I''ve been reading about the DDN stuff which
is
> impressive but I don''t have access to that kind of budget yet.
Even if the per-port cost of 10GigE was the same as SDR infiniband
(also running at 10Gb/s, DDR or QDR or EDR will be noticably faster)
the performance of the two is NOT the same, not even close in some
cases.  TCP adds a LOT of overhead to the IO processing because of
extra (unavoidable) data copies in the networking layer, due to the
lack of RDMA (iWARP hasn''t appeared anywhere that I''ve heard
about).

Also, the RPC rate of TCP is much lower than that of native IB.
> My initial plan is to tie some HP DL160''s to MSA70''s
filled with
> 300GB drives. I realize this won''t give me any sort of failover  
> capability but this is just a proof for now. If there is a low cost,  
> high speed shared device, I''d rather use a failover config in
order
> to get a real feel for that this is going to require.
You should look at the Sun Lustre Storage system, which is configured
with HA-OSS pairs, each exporting 48x 1TB SATA drives.  You can get
over 1GB/s from each OSS, and 64TB of usable space (after RAID-6,  
journaling,
and hot spares are taken into account).
> Also, the roadmap for Lustre used to include "raid" personalities
> for OSS''s. has that functionality been deferred or dropped  
> altogether, or is it still in the works?
It is still on the horizon for now, unfortunately.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.