search for: petabytes

Thanks for the reply. We will mainly use this for archival - near-cold storage. Anything, from your experience, to keep in mind while planning large installations? Sent from my Verizon, Samsung Galaxy smartphone -------- Original message --------From: Serkan ?oban <cobanserkan at gmail.com> Date: 6/29/17 4:39 AM (GMT-05:00) To: Jason Kiebzak <jkiebzak at gmail.com> Cc: Gluster

>Thanks for the reply. We will mainly use this for archival - near-cold storage. Archival usage is good for EC >Anything, from your experience, to keep in mind while planning large installations? I am using 3.7.11 and only problem is slow rebuild time when a disk fails. It takes 8 days to heal a 8TB disk.(This might be related with my EC configuration 16+4) 3.9+ versions has some

Has anyone scaled to a multi petabyte gluster setup? How well does erasure code do with such a large setup? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170628/b030376d/attachment.html>

We are using 3.10 and have a 7 PB cluster. We decided against 16+3 as the rebuild time are bottlenecked by matrix operations which scale as the square of the number of data stripes. There are some savings because of larger data chunks but we ended up using 8+3 and heal times are about half compared to 16+3. -Alastair On 30 June 2017 at 02:22, Serkan ?oban <cobanserkan at gmail.com>

Did you test healing by increasing disperse.shd-max-threads? What is your heal times per brick now? On Fri, Jun 30, 2017 at 8:01 PM, Alastair Neil <ajneil.tech at gmail.com> wrote: > We are using 3.10 and have a 7 PB cluster. We decided against 16+3 as the > rebuild time are bottlenecked by matrix operations which scale as the square > of the number of data stripes. There are

...;t started testing the metal had only arrived the day before and they where waiting for the drives to arrive. They had also changed the design to give it more network. I will try to find out more as the customer progresses..... &gt;Interesting blog: &gt;http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/ &gt; -- Trevor Pretty | Technical Account Manager | T: +64 9 639 0652 | M: +64 21 666 161 Eagle Technology Group Ltd. Gate D, Alexandra Park, Greenlane West, Epsom Private Bag 93211, Parnell, Auckland www.eagle.co.nz  This email is confide...

When I run backup and send the output to a file on the server, I get a file limitation of 2g. When I do the same backup to a snap server, there is no problem. Is there a size limitation for a single file? Thanks, Steven C. Henry stevench@xnet.com

Does Samba have any max filesytem limitations. In particular can both 2.2.8 and 3.0 support 2TB+ filesystems. For now, I am thinking of 6TB max, so I don't need to know about Petabytes or Exabytes. The other side of the question, is can Win9x, Win2K, etc. work with filesystems over 2 TB. If the above is in a FAQ somewhere, a url would be great. Greg -- Greg Freemyer

You could always use 40-gigabit between the two storage systems which would speed things dramatically, or back to back 56-gigabit IB. ---------------------------------------- From: zfs-discuss-request@opensolaris.org Sent: Monday, March 18, 2013 11:01 PM To: zfs-discuss@opensolaris.org Subject: zfs-discuss Digest, Vol 89, Issue 12 Send zfs-discuss mailing list submissions to

https://bugzilla.samba.org/show_bug.cgi?id=11067 Bug ID: 11067 Summary: add --min-depth and --max-depth options Product: rsync Version: 3.1.1 Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 Component: core Assignee: wayned at samba.org Reporter:

Hello, we are currently managing access permissions through classical user-group-others permissions on a multi-petabyte directory tree with partially very deep and broad directories. Projects are represented by directory trees and mapped through GIDs. Lately we had lots of "singular" permission request (one single user needs access to a single dataset but should not be able to

On Wed, 8 May 2019 at 13:48, Valeri Galtsev <galtsev at kicp.uchicago.edu> wrote: > > > On 2019-05-08 12:28, Stephen John Smoogen wrote: > > On Wed, 8 May 2019 at 13:24, mark <m.roth at 5-cent.us> wrote: > > >> > > Yep. Minimum for that is going to be about the same as your RESCUE. The > > other would just be to confirm that the sda has space and

...to checkpoint, it fills the partition and ends up corrupted and unrecoverable. >Neither Gluster nor Ceph are "backup solutions", so if the data is not easily replaceable it's better to have it elsewhere. Better if offline. > It's a nice idea but when you're dealing in petabytes of data, streaming in as fast as your storage will allow, it's just not physically possible.

...t; problem, it works as designed, also with maildir rsync did a good job, >> what never worked as it should was dsync ,cause of bugs ,that may >> changed now >> > > I would like to very much hear about the problems you had with dsync as we have successfully migrated tens of petabytes of mails with it successfully while preserving the UID numbers. > Imapsync would have been totally out of question as it is absolutely crucial in those migrations to keep the UID -> MAIL pairs matching legacy server. And usually also POP3 UIDLs. If not keeping the data the servers would have...

...t; problem, it works as designed, also with maildir rsync did a good job, >> what never worked as it should was dsync ,cause of bugs ,that may >> changed now >> > > I would like to very much hear about the problems you had with dsync as we have successfully migrated tens of petabytes of mails with it successfully while preserving the UID numbers. > Imapsync would have been totally out of question as it is absolutely crucial in those migrations to keep the UID -> MAIL pairs matching legacy server. And usually also POP3 UIDLs. If not keeping the data the servers would have...

.../media/disk/Films] $ ls -l total 692996 -rw-r----- 1 678756852 34537972 148381783526817280 avr 28 01:01 Cinema drwxr-xr-x 3 kikinovak kikinovak 4096 mai 9 10:07 Anime drwxrwxrwx 4 kikinovak kikinovak 4096 mai 10 12:25 Series Notice that the file size is something like petabytes :oD Is there any way to repair this obviously corrupt data? Cheers, Niki Kovacs

...ncluded the above, because I note the 32bit mmio, but the 64bit flag; also the clock speed for the controller. Now, I've been working on one with Penguin. I noticed one thing, that it was set to native IDE. After googling, I saw that the most recent spec, which included EIDE, should be good to petabytes... but I tried resetting it to AHCI anyway. The user ran one job, ok... then another last night, and it's spitting the same errors. In /var/log/messages, I see JBD: detected IO errors while flushing file data: Mar 7 00:53:28 <server> kernel: ata2.00: exception Emask 0x0 SAct 0x3 SErr 0...

Hi, after 2 years running GlusterFS without bigger problems we're facing some strange errors lately. After updating to 3.12.7 some user reported at least 4 broken directories with some invisible files. The files are at the bricks and don't start with a dot, but aren't visible in "ls". Clients still can interact with them by using the explicit path. More information:

...ementations >> to decide for themselves). > > Yes, painfully aware. Not sure what to do about it however, since the > alternative (ie. switching back to yajl) as libvirt did is not going > to be pleasant. > > Currently we're OK as long as disk sizes don't exceed 8 petabytes, if > my quick calculation is correct. The problem comes anywhere that qemu outputs an unsigned 64-bit number as unsigned AND where that value is larger than INT64_MAX (jansson uses strtoll, rather than strtoull). But since disk sizes cannot exceed off_t, which is a signed 64-bit number, it...

...with maildir rsync did a good job, >>>> what never worked as it should was dsync ,cause of bugs ,that may >>>> changed now >>>> >>> >>> I would like to very much hear about the problems you had with dsync as we have successfully migrated tens of petabytes of mails with it successfully while preserving the UID numbers. >>> Imapsync would have been totally out of question as it is absolutely crucial in those migrations to keep the UID -> MAIL pairs matching legacy server. And usually also POP3 UIDLs. If not keeping the data the servers wou...