Gluster users - May 2010 - How about quality of GluterFS?

I have a question to prepare apply Gluster for my project:
If GluterFS is enough STRONG to do a FileHosting service, like 
RapidShare, MegaUpload ...?
Thank you!

Sir -
     Gluster is used by several of the largest CDNs in the world as a step in
the distribution process. Could you be more specific as to what you are looking
for?

Craig Carl
Gluster

Sent from a mobile device, please excuse my tpyos.

On May 13, 2010, at 20:52, lesonus <lesonus at gmail.com> wrote:
> I have a question to prepare apply Gluster for my project:
> If GluterFS is enough STRONG to do a FileHosting service, like RapidShare,
MegaUpload ...?
> Thank you!
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users

I'm using glusterFs 3.03
My server is running in Heavy load, so result of your command (iozone) 
may by not exactly

But I think there are problem with IO of my system, like that:
My server store many big file, for example 10000 file, each file 100MB, 
so all is 10TB.
I user 12HDD SATAII , 2TB, RAID6, so enough to store files.

And problem:
- when 200 user download concurrent (normal work), Webserver must open 
atleast 200 file (100mbyte/file).
Then there are 200 process open 200 file concurrently. (With Gigabit 
nic, each user can reach 1Gbit /200 = 125MByte/200 ~600Kbyte/s, it's a 
normal rate!)
Total IO to RAID will be ~ 1GBit <=> 125MByte/s.
With single file access, my RAID work well with this rate. But, with 200 
concurrent read to RAID, I see, it can not reach 125MB/s (1 Gbit)

--> Result: With Not GluterFS, my server can not service 1Gbit/s access, 
then with GlusterFS, the rate is slower?

So, What solution to resolve my hardware problem? and this solution is 
still good for GlusterFS?

I see this:
http://ftp.zresearch.com/pub/gluster/glusterfs/talks/Z/GlusterFS.pdf
I see 13GBit/s can be reached, but 64 client load with single file, so 
it's simple to reach that rate?

Thank very much for your helps!

Craig Carl wrote:
> Can you tell me what version ot Gluster you are using? Also if you 
> could run "iozone -a -b lesonus_in_gluster.csv" in the Gluster
mount
> point and then again "iozone -a -b lesonus-no-gluster.csv" on the
> storage server and send the results back I would appreciate it.
>
> We should be able to get things faster for you.
>
> Craig
>
> --
> Craig Carl
> Gluster, Inc.
> Cell - (408) 829-9953 (California, USA)
> Gtalk - craig.carl at gmail.com
>
>
> ----- Original Message -----
> From: "lesonus" <lesonus at gmail.com>
> To: "Craig Carl" <craig at gluster.com>
> Sent: Friday, May 14, 2010 8:08:46 AM GMT -08:00 US/Canada Pacific
> Subject: Re: [Gluster-users] How about quality of GluterFS?
>
> I need a FileHosting system can service 10, 20, ... 50Gbit ... to
> service users of an ISP
> With 100TB, 200TB, ... 1000 TB or more
>
> I test with GlusterFS, with basic config, one client, one server with
> Centos5.4.
> I have a very basic problem: ls command is very  very slow when my
> system running! But I not use GlusterFS, It run normal
> and change to other config of vol file, but basic problem still
>
> So I wonder how about the Quality of GlusterFS when it has basic problem
> like that!
> In the future, If it can service complex and heavy load?
>
> I need choice a suitable solution in next some days!
> If GLFS can do this?
>
> Thank you!
>
> Craig Carl wrote:
> > Sir -
> >      Gluster is used by several of the largest CDNs in the world as 
> a step in the distribution process. Could you be more specific as to 
> what you are looking for?
> >
> > Craig Carl
> > Gluster
> >
> > Sent from a mobile device, please excuse my tpyos.
> >
> > On May 13, 2010, at 20:52, lesonus <lesonus at gmail.com> wrote:
> >
> >  
> >> I have a question to prepare apply Gluster for my project:
> >> If GluterFS is enough STRONG to do a FileHosting service, like 
> RapidShare, MegaUpload ...?
> >> Thank you!
> >> _______________________________________________
> >> Gluster-users mailing list
> >> Gluster-users at gluster.org
> >> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
> >>    
> >
> >  
>

While I can't talk about specific customers or their configurations, an
CDN-type organization wouldn't typically use Gluster as a method to copy
files to all their distribution points, but they would use Gluster to make each
distribution point faster and more available.



Thanks, 

Craig 

-- 
Craig Carl 













Join us for a Webinar on May 26; 
Case Studies: Deploying Open Source Storage Clouds 
Sales Engineer; Gluster, Inc. 
Cell - ( 408) 829-9953 (California, USA) 
Office - ( 408) 770-1884 
Gtalk - craig.carl at gmail.com 
Twitter - @gluster 
Installing Gluster Storage Platform, the movie! 

----- Original Message ----- 
From: "Count Zero" <countz at gmail.com> 
To: "Craig Carl" <craig at gluster.com> 
Cc: gluster-users at gluster.org 
Sent: Monday, May 24, 2010 5:49:41 PM GMT -08:00 US/Canada Pacific 
Subject: Re: [Gluster-users] How about quality of GluterFS? 


Hey Craig, 


Can you tell me what CDN uses Gluster at the moment? 


Because here's the thing, the way I see it: 


1) The point of a CDN, is to distribute content to "far away and hard to
reach" places that are closer to the users who need the content,
2) As a consequence of (1), the latency between the CDN nodes, or data centers,
will usually be high,
3) As a consequence of (2), Gluster can not currently be practically used and
becomes super slow...


I would love to be proven wrong, or even better, shown some sample
configurations that demonstrate how a CDN might replicate their data across
their data centers, despite the relatively high latency (around 100ms).






On May 14, 2010, at 5:12 PM, Craig Carl wrote: 



Sir - 
Gluster is used by several of the largest CDNs in the world as a step in the
distribution process. Could you be more specific as to what you are looking for?

Craig Carl 
Gluster 

Sent from a mobile device, please excuse my tpyos. 

On May 13, 2010, at 20:52, lesonus < lesonus at gmail.com > wrote: 



I have a question to prepare apply Gluster for my project: 


If GluterFS is enough STRONG to do a FileHosting service, like RapidShare,
MegaUpload ...?


Thank you! 


_______________________________________________ 


Gluster-users mailing list 


Gluster-users at gluster.org 


http://gluster.org/cgi-bin/mailman/listinfo/gluster-users 
_______________________________________________ 
Gluster-users mailing list 
Gluster-users at gluster.org 
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users