search for: 10pb

...-------- Original message --------From: Serkan ?oban <cobanserkan at gmail.com> Date: 6/29/17 4:39 AM (GMT-05:00) To: Jason Kiebzak <jkiebzak at gmail.com> Cc: Gluster Users <gluster-users at gluster.org> Subject: Re: [Gluster-users] Multi petabyte gluster I am currently using 10PB single volume without problems. 40PB is on the way. EC is working fine. You need to plan ahead with large installations like this. Do complete workload tests and make sure your use case is suitable for EC. On Wed, Jun 28, 2017 at 11:18 PM, Jason Kiebzak <jkiebzak at gmail.com> wrote: > H...

...;cobanserkan at gmail.com> > > Date: 6/29/17 4:39 AM (GMT-05:00) > > To: Jason Kiebzak <jkiebzak at gmail.com> > > Cc: Gluster Users <gluster-users at gluster.org> > > Subject: Re: [Gluster-users] Multi petabyte gluster > > > > I am currently using 10PB single volume without problems. 40PB is on > > the way. EC is working fine. > > You need to plan ahead with large installations like this. Do complete > > workload tests and make sure your use case is suitable for EC. > > > > > > On Wed, Jun 28, 2017 at 11:18 PM,...

...-- > From: Serkan ?oban <cobanserkan at gmail.com> > Date: 6/29/17 4:39 AM (GMT-05:00) > To: Jason Kiebzak <jkiebzak at gmail.com> > Cc: Gluster Users <gluster-users at gluster.org> > Subject: Re: [Gluster-users] Multi petabyte gluster > > I am currently using 10PB single volume without problems. 40PB is on > the way. EC is working fine. > You need to plan ahead with large installations like this. Do complete > workload tests and make sure your use case is suitable for EC. > > > On Wed, Jun 28, 2017 at 11:18 PM, Jason Kiebzak <jkiebzak at...

...m> >> > Date: 6/29/17 4:39 AM (GMT-05:00) >> > To: Jason Kiebzak <jkiebzak at gmail.com> >> > Cc: Gluster Users <gluster-users at gluster.org> >> > Subject: Re: [Gluster-users] Multi petabyte gluster >> > >> > I am currently using 10PB single volume without problems. 40PB is on >> > the way. EC is working fine. >> > You need to plan ahead with large installations like this. Do complete >> > workload tests and make sure your use case is suitable for EC. >> > >> > >> > On Wed, J...

Has anyone scaled to a multi petabyte gluster setup? How well does erasure code do with such a large setup? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170628/b030376d/attachment.html>

...e IBM >> GPFS, HP Polyserve, etc... ? > I'm fairly biased, but I think you should look at Lustre - it is currently > running on 15 of the 20 fastest systems in the world, and 40% of the top 200. > No problems with scalability at all - the ORNL Jaguar system has 37000+ > nodes, 10PB of storage, and has exceeded 150GB/s peak read/write speed. > > Lustre is GPL and freely available: http://www.lustre.org/ I am fairly biased too. However, I don't believe any shared disk cluster file system can scale upto 1000 nodes. This includes ocfs2 and gfs2. I don't know much...

Back in the spring, we started using rsync for a disk to disk backup system maintaining close to 10PB of data. I am not here to debate the issue of what is the right tool but only to discuss what we found to be a problem with rsync when doing so. We traced the various processes hoping to find what the culprit was slowing things down so much and determined pretty easily that it was the checksu...