Folks: Thought you all might be interested to know we''ve just built a "Backblaze Pod" (see http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/) running Fedora 12 and your latest (as of yesterday) GIT kernel snapshot. We mounted *all 45 disks* into one huge 62Tb btrfs RAID10 filesystem, and so far it works just great! I''d be interested in your comments as to pitfall, problems you all see in creating such a huge btrfs file system... Jim Hughes Cerro Tololo Interamerican Observatory La Serena, Chile ---------------- ferrari.27# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 30237648 20710404 7991244 73% / tmpfs 1978536 0 1978536 0% /dev/shm /dev/sda3 42555184 185304 40208188 1% /home /dev/sdb 65931236280 6917008 65924319272 1% /raid45 ferrari.28# df -h Filesystem Size Used Avail Use% Mounted on /dev/sda1 29G 20G 7.7G 73% / tmpfs 1.9G 0 1.9G 0% /dev/shm /dev/sda3 41G 181M 39G 1% /home /dev/sdb 62T 6.6G 62T 1% /raid45 ferrari.29# mount /dev/sda1 on / type ext3 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) /dev/sda3 on /home type ext4 (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) /dev/sdb on /raid45 type btrfs (rw) ferrari.30# cat /proc/version Linux version 2.6.32 (root@localhost.localdomain) (gcc version 4.4.3 20100127 (Red Hat 4.4.3-4) (GCC) ) #1 SMP Tue Mar 2 12:28:10 CLST 2010 ferrari.31# lspci 00:00.0 Host bridge: Intel Corporation 4 Series Chipset DRAM Controller (rev 03) 00:02.0 VGA compatible controller: Intel Corporation 4 Series Chipset Integrated Graphics Controller (rev 03) 00:02.1 Display controller: Intel Corporation 4 Series Chipset Integrated Graphics Controller (rev 03) 00:03.0 Communication controller: Intel Corporation 4 Series Chipset HECI Controller (rev 03) 00:19.0 Ethernet controller: Intel Corporation 82567V-2 Gigabit Network Connection 00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4 00:1a.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5 00:1a.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6 00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2 00:1b.0 Audio device: Intel Corporation 82801JI (ICH10 Family) HD Audio Controller 00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Port 1 00:1c.1 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Port 2 00:1c.2 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Port 3 00:1c.3 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Port 4 00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1 00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2 00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3 00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90) 00:1f.0 ISA bridge: Intel Corporation 82801JIB (ICH10) LPC Interface Controller 00:1f.2 IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE Controller 00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller 00:1f.5 IDE interface: Intel Corporation 82801JI (ICH10 Family) 2 port SATA IDE Controller 02:00.0 RAID bus controller: Silicon Image, Inc. SiI 3132 Serial ATA Raid II Controller (rev 01) 03:00.0 PCI bridge: Pericom Semiconductor Device e111 (rev 02) 04:04.0 RAID bus controller: Silicon Image, Inc. SiI 3124 PCI-X Serial ATA Controller (rev 01) 05:00.0 IDE interface: JMicron Technologies, Inc. JMB368 IDE controller 06:00.0 RAID bus controller: Silicon Image, Inc. SiI 3124 PCI-X Serial ATA Controller (rev 01) 06:06.0 FireWire (IEEE 1394): Agere Systems FW322/323 (rev 70) -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Mar 03, 2010 at 11:55:56AM -0300, Jim Hughes wrote:> Folks: > > Thought you all might be interested to know we''ve just built a > "Backblaze Pod" (see > http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/) > running Fedora 12 and your latest (as of yesterday) GIT kernel snapshot. > > We mounted *all 45 disks* into one huge 62Tb btrfs RAID10 filesystem, > and so far it works just great! > > I''d be interested in your comments as to pitfall, problems you all see > in creating such a huge btrfs file system... >Well thats cool. The only thing that may start to cause problems is as you fill it up, the time it takes to cache block groups is going to take _forever_. Thankfully we only cache block groups on demand, and we try to do it in a way that we save the crappiest ones for last, so even that probably won''t be terrible. As of right now balance''s will _suck_, since they try to move all block groups, but IIRC Yan is working on making balances take less time. Other than that everything should work just as well as if it were a 10gb volume. Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html