Hi, I have 4 machines, all converted to BTRFS about 6 months ago, now all running Ubuntu Quantal with kernel 3.5.0-17 The matter is that all these machines are now getting slower and slower everyday, every disk access causing the disk to be 100% busy for long periods, to the point that I''m now seriously considering migrating everything back to ext4... From the start BTRFS was "not very fast", still satisfactory, but now it becomes truly unusable. On one machine, I know have a typical complete boot time to a usable GUI that is over 4 minutes, with the HD still very busy for a couple more minutes afterwards, where it used to be around 35-40 seconds in ext4 ! Is there anything I could do to speed things back (without losing all my snapshots or doubling the size of data on disk)...? I already had made the move back from BTRFS to ext4 about 18 months ago, I found it had improved so was back to BTRFS, and I wouldn''t have to revert back again :-/ Any advice or help greatly appreciated. TIA. -- Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E Ne cherchez pas : Je ne suis pas sur Facebook. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Am Sonntag, 7. Oktober 2012 schrieb Swâmi Petaramesh:> Hi, > > I have 4 machines, all converted to BTRFS about 6 months ago, now all > running Ubuntu Quantal with kernel 3.5.0-17 > > The matter is that all these machines are now getting slower and slower > everyday, every disk access causing the disk to be 100% busy for long > periods, to the point that I''m now seriously considering migrating > everything back to ext4... > > From the start BTRFS was "not very fast", still satisfactory, but now > it becomes truly unusable. > > On one machine, I know have a typical complete boot time to a usable > GUI that is over 4 minutes, with the HD still very busy for a couple > more minutes afterwards, where it used to be around 35-40 seconds in > ext4 ! > > Is there anything I could do to speed things back (without losing all > my snapshots or doubling the size of data on disk)...? > > I already had made the move back from BTRFS to ext4 about 18 months > ago, I found it had improved so was back to BTRFS, and I wouldn''t have > to revert back again :-/ > > Any advice or help greatly appreciated.I experienced this on my oldest BTRFS filesystems on a ThinkPad T23 as well. Especially with fsync() based workloads like apt / aptitude dist- upgrade. I use eatmydata for these operations for now on this laptop. A balance didn´t work. I considering to redo those as node and leaf size 16 KiB. But then anything above kernel 3.3 doesn´t resume reliably on the machine and I did not yet get into to finding out why. A kernel bisect would take ages on this machine unless I compile the kernels somewhere else. Anyway, I think to better see what your issue is, it would be good to have some more information about free space and state of your BTRFS filesystem. Such as btrfs fi df (preferably with btrfs tools from Goffredo) btrfs fi show for starters. [1] See Goffredo Baroncelli, [PATCH][BTRFS-PROGS][V3] btrfs filesystem df last thursday: You can pull this change from http://cassiopea.homelinux.net/git/btrfs-progs-unstable.git branch disk_free Thanks, -- Martin ''Helios'' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Am Sonntag, 7. Oktober 2012 schrieb Swâmi Petaramesh:> Hi, > > I have 4 machines, all converted to BTRFS about 6 months ago, now all > running Ubuntu Quantal with kernel 3.5.0-17 > > The matter is that all these machines are now getting slower and slower > everyday, every disk access causing the disk to be 100% busy for long > periods, to the point that I''m now seriously considering migrating > everything back to ext4... > > From the start BTRFS was "not very fast", still satisfactory, but now > it becomes truly unusable. > > On one machine, I know have a typical complete boot time to a usable > GUI that is over 4 minutes, with the HD still very busy for a couple > more minutes afterwards, where it used to be around 35-40 seconds in > ext4 ! > > Is there anything I could do to speed things back (without losing all > my snapshots or doubling the size of data on disk)...?As told in my previous post its better to first understand the problem. What might help here, additionally to what I suggested already, is some output of vmstat 1 and iostat -dx 1 /dev/sda (or maybe even better 5 or 10 seconds interval) where /dev/sda is device or better partition where the workload happens. Also the amount of snapshots in use would be helpful I bet. Ciao, -- Martin ''Helios'' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Swâmi Petaramesh <swami <at> petaramesh.org> writes:> > Hi, > > I have 4 machines, all converted to BTRFS about 6 months ago, now all > running Ubuntu Quantal with kernel 3.5.0-171. Convert to a 16k or 32k leafsize. 2. defragment (each non-trivial file) every now and again [eg. find / -size +16k -type f -exec btrfs filesystem defragment -vf ''{}'' '';'' ] 3. Note that btrfs in kernel 3.6 (vs 3.5) does appear to be quite a bit quicker. I''m running Siduction kernels whilst Debian is on freeze at 3.2. I found 3.5 really quite awful - not sure why. Make sure you discount other factors. I''m not sure defrag goes so far as to consolidate files by moving bits out of the way yet, so you may find that the number of fragments in a file never reaches one. Bear with it pls. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Am Sonntag, 7. Oktober 2012 schrieb Swâmi Petaramesh:> Hi, > > I have 4 machines, all converted to BTRFS about 6 months ago, now all > running Ubuntu Quantal with kernel 3.5.0-17 > > The matter is that all these machines are now getting slower and slower > everyday, every disk access causing the disk to be 100% busy for long > periods, to the point that I''m now seriously considering migrating > everything back to ext4... > > From the start BTRFS was "not very fast", still satisfactory, but now > it becomes truly unusable. > > On one machine, I know have a typical complete boot time to a usable > GUI that is over 4 minutes, with the HD still very busy for a couple > more minutes afterwards, where it used to be around 35-40 seconds in > ext4 ! > > Is there anything I could do to speed things back (without losing all > my snapshots or doubling the size of data on disk)...?I forgot to mention mount options? Which one do you use? Please always include all details that may be important. space_cache, inode_cache? Thanks, -- Martin ''Helios'' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Le 07/10/2012 15:00, Martin Steigerwald a écrit :> I forgot to mention mount options? Which one do you use?root@tethys:/etc# grep btrfs fstab /dev/VG1/BTR_POOL / btrfs subvol=UBUNTU/@,space_cache,autodefrag,compress=lzo,relatime 0 0 /dev/sda2 /boot btrfs subvol=UBUNTU/@boot,space_cache,autodefrag,relatime 0 0 /dev/VG1/BTR_POOL /tmp btrfs subvol=UBUNTU/@tmp,space_cache,autodefrag,compress=lzo,relatime 0 0 /dev/VG1/BTR_POOL /home btrfs subvol=UBUNTU/@home,space_cache,autodefrag,compress=lzo,relatime 0 0 /dev/VG1/BTR_POOL /storage btrfs subvol=STORAGE,space_cache,autodefrag,compress=lzo,relatime 0 0 /dev/VG1/BTR_POOL /data/BIG btrfs subvol=DATA/BIG,space_cache,compress=lzo,noatime 0 0 /dev/VG1/BTR_POOL /data/VBOX_HIDDEN btrfs subvol=DATA/VBOX_HIDDEN,space_cache,compress=lzo,noatime 0 0 /dev/VG1/BTR_POOL /data/VBOX_VMS btrfs subvol=DATA/VBOX_VMS,space_cache,compress=lzo,noatime 0 0 -- Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E Ne cherchez pas : Je ne suis pas sur Facebook. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi again ;-) Le 07/10/2012 14:33, Alex a écrit :> > 1. Convert to a 16k or 32k leafsize.How should I do this ? Can I do this on a live FS, and isn''t this going to double my on-disk used space (I have active snapshots...)> 2. defragment (each non-trivial file) every now and againI believed that the "autodefrag" mount option was supposed to avoid the need for manual defrags...? Plus, with respect to snapshots, isn''t this going to increase a lot my used disk space ?> 3. Note that btrfs in kernel 3.6 (vs 3.5) does appear to be quite a bit quicker.I''m not sure that "a bit" quicker will be enough, when I see my BTRFS is currently about 8x slower than what ext4 user to... -- Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E Ne cherchez pas : Je ne suis pas sur Facebook. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Le 07/10/2012 12:59, Martin Steigerwald a écrit :> btrfs fi df (preferably with btrfs tools from Goffredo) > btrfs fi showI don''t think I miss any free space ;-) (From one of my machines, but the others have rather the same architecture...) # btrfs fi sh failed to read /dev/sr0 Label: ''BTR_POOL'' uuid: 7266e630-ba72-404c-bfbd-759d3a6ce2e3 Total devices 1 FS bytes used 278.55GB devid 1 size 912.26GB used 464.04GB path /dev/dm-1 Label: ''BTR_BOOT'' uuid: b280100e-6a9a-4488-a207-d4e3f001b4c9 Total devices 1 FS bytes used 504.39MB devid 1 size 1.00GB used 729.50MB path /dev/sda2 # btrfs fi df /boot System: total=4.00MB, used=4.00KB Data+Metadata: total=725.50MB, used=504.38MB # btrfs fi df / Data: total=426.01GB, used=273.00GB System, DUP: total=8.00MB, used=56.00KB System: total=4.00MB, used=0.00 Metadata, DUP: total=19.00GB, used=5.55GB Metadata: total=8.00MB, used=0.00 # df -h /boot / Filesystem Size Used Avail Use% Mounted on /dev/sda2 1,1G 505M 517M 50% /boot - 913G 285G 602G 33% / ...and I have quite a lot of snapshots, as I use OpenSuSE excellent "snapper" utility : # snapper list Type | # | Pre # | Date | Cleanup | Description | Userdata -------+-----+-------+----------------------------------+----------------+-----------------------+--------- single | 0 | | | | current | single | 33 | | mer. 01 août 2012 09:17:01 CEST | timeline | Precise | pre | 203 | | lun. 27 août 2012 16:30:22 CEST | empty-pre-post | AvUp Precise->QuantA3 | post | 213 | 203 | mar. 28 août 2012 10:15:02 CEST | empty-pre-post | ApUp Precise->QuantA3 | single | 265 | | sam. 01 sept. 2012 08:17:02 CEST | timeline | QuantA3 | single | 443 | | ven. 21 sept. 2012 00:17:02 CEST | timeline | QuantB1 K 3.5.0-15 | single | 456 | | sam. 22 sept. 2012 11:17:01 CEST | timeline | timeline | single | 463 | | dim. 23 sept. 2012 10:17:03 CEST | timeline | QuantB1 K 3.5.0-15 | single | 473 | | lun. 24 sept. 2012 00:17:01 CEST | timeline | QuantB1 K 3.5.0-15 | single | 486 | | mar. 25 sept. 2012 08:17:01 CEST | timeline | QuantB1 K 3.5.0-15 | single | 495 | | mer. 26 sept. 2012 09:17:11 CEST | timeline | QuantB1 K 3.5.0-15 | single | 505 | | jeu. 27 sept. 2012 08:17:03 CEST | timeline | timeline | single | 520 | | ven. 28 sept. 2012 09:17:02 CEST | timeline | QuantB2 K 3.5.0-15 | single | 534 | | sam. 29 sept. 2012 10:17:01 CEST | timeline | QuantB2 K 3.5.0-16 | single | 544 | | dim. 30 sept. 2012 10:17:02 CEST | timeline | QuantB2 K 3.5.0-16 | single | 556 | | lun. 01 oct. 2012 09:17:03 CEST | timeline | QuantB2 K 3.5.0-16 | single | 568 | | mar. 02 oct. 2012 08:17:01 CEST | timeline | timeline | single | 582 | | mer. 03 oct. 2012 10:17:03 CEST | timeline | timeline | single | 591 | | jeu. 04 oct. 2012 08:17:02 CEST | timeline | timeline | single | 602 | | jeu. 04 oct. 2012 20:17:01 CEST | timeline | timeline | single | 603 | | jeu. 04 oct. 2012 21:17:01 CEST | timeline | timeline | single | 604 | | jeu. 04 oct. 2012 22:17:01 CEST | timeline | timeline | single | 605 | | jeu. 04 oct. 2012 23:17:01 CEST | timeline | timeline | single | 608 | | ven. 05 oct. 2012 10:17:04 CEST | timeline | QuantB2 K 3.5.0-17 | single | 609 | | ven. 05 oct. 2012 11:17:01 CEST | timeline | timeline | single | 610 | | ven. 05 oct. 2012 12:17:02 CEST | timeline | timeline | single | 611 | | ven. 05 oct. 2012 13:17:02 CEST | timeline | timeline | single | 612 | | ven. 05 oct. 2012 14:17:01 CEST | timeline | timeline | single | 613 | | ven. 05 oct. 2012 15:17:01 CEST | timeline | timeline | single | 614 | | ven. 05 oct. 2012 19:17:02 CEST | timeline | timeline | single | 615 | | ven. 05 oct. 2012 20:17:01 CEST | timeline | timeline | single | 616 | | ven. 05 oct. 2012 22:18:48 CEST | timeline | timeline | single | 617 | | ven. 05 oct. 2012 23:17:01 CEST | timeline | timeline | single | 618 | | sam. 06 oct. 2012 10:17:03 CEST | timeline | timeline | single | 619 | | sam. 06 oct. 2012 11:17:01 CEST | timeline | timeline | single | 620 | | dim. 07 oct. 2012 00:17:03 CEST | timeline | timeline | single | 621 | | dim. 07 oct. 2012 10:17:02 CEST | timeline | timeline | single | 622 | | dim. 07 oct. 2012 11:17:02 CEST | timeline | timeline | single | 623 | | dim. 07 oct. 2012 12:17:01 CEST | timeline | timeline | single | 624 | | dim. 07 oct. 2012 13:17:01 CEST | timeline | timeline | single | 625 | | dim. 07 oct. 2012 14:17:01 CEST | timeline | timeline | single | 626 | | dim. 07 oct. 2012 15:17:01 CEST | timeline | timeline | ...or if you prefer, something like : # btrfs su li / ID 256 top level 5 path UBUNTU ID 259 top level 5 path UBUNTU/@ ID 261 top level 5 path UBUNTU/@tmp ID 262 top level 5 path UBUNTU/@home ID 264 top level 5 path LMDE ID 265 top level 5 path LMDE/@ ID 267 top level 5 path LMDE/@tmp ID 268 top level 5 path LMDE/@home ID 270 top level 5 path MINT ID 271 top level 5 path MINT/@ ID 273 top level 5 path MINT/@tmp ID 274 top level 5 path MINT/@home ID 276 top level 5 path BODHI ID 277 top level 5 path BODHI/@ ID 278 top level 5 path BODHI/@var ID 279 top level 5 path BODHI/@tmp ID 280 top level 5 path BODHI/@home ID 282 top level 5 path STORAGE ID 471 top level 5 path DATA/VBOX_HIDDEN ID 472 top level 5 path DATA/VBOX_VMS ID 505 top level 5 path LMDE/@/.snapshots ID 506 top level 5 path LMDE/@home/.snapshots ID 509 top level 5 path MINT/@/.snapshots ID 510 top level 5 path MINT/@home/.snapshots ID 515 top level 5 path UBUNTU/@/.snapshots ID 516 top level 5 path UBUNTU/@home/.snapshots ID 517 top level 5 path STORAGE/.snapshots ID 628 top level 5 path DATA ID 629 top level 5 path DATA/BIG ID 630 top level 5 path DATA/BIG/michel ID 1339 top level 5 path UBUNTU/@/.snapshots/33/snapshot ID 1340 top level 5 path UBUNTU/@home/.snapshots/33/snapshot ID 1341 top level 5 path STORAGE/.snapshots/35/snapshot ID 1384 top level 5 path LMDE/@/.snapshots/2/snapshot ID 1385 top level 5 path LMDE/@home/.snapshots/2/snapshot ID 1387 top level 5 path MINT/@/.snapshots/2/snapshot ID 1388 top level 5 path MINT/@home/.snapshots/2/snapshot ID 1495 top level 5 path DATA/VBOX_HIDDEN/.snapshots ID 1621 top level 5 path MINT/@/.snapshots/3/snapshot ID 1622 top level 5 path MINT/@home/.snapshots/3/snapshot ID 1651 top level 5 path MINT/@/.snapshots/4/snapshot ID 1652 top level 5 path MINT/@home/.snapshots/4/snapshot ID 1654 top level 5 path MINT/@/.snapshots/5/snapshot ID 1655 top level 5 path MINT/@home/.snapshots/5/snapshot ID 1657 top level 5 path MINT/@/.snapshots/6/snapshot ID 1658 top level 5 path MINT/@home/.snapshots/6/snapshot ID 1706 top level 5 path DATA/VBOX_HIDDEN/.snapshots/7/snapshot ID 1874 top level 5 path UBUNTU/@home/.snapshots/203/snapshot ID 1875 top level 5 path UBUNTU/@/.snapshots/203/snapshot ID 1903 top level 5 path UBUNTU/@home/.snapshots/213/snapshot ID 1904 top level 5 path UBUNTU/@/.snapshots/213/snapshot ID 1980 top level 5 path LMDE/@/.snapshots/3/snapshot ID 1981 top level 5 path LMDE/@home/.snapshots/3/snapshot ID 1983 top level 5 path MINT/@/.snapshots/7/snapshot ID 1984 top level 5 path MINT/@home/.snapshots/7/snapshot ID 2058 top level 5 path UBUNTU/@/.snapshots/265/snapshot ID 2059 top level 5 path UBUNTU/@home/.snapshots/263/snapshot ID 2060 top level 5 path STORAGE/.snapshots/268/snapshot ID 2184 top level 5 path LMDE/@/.snapshots/4/snapshot ID 2185 top level 5 path LMDE/@home/.snapshots/4/snapshot ID 2513 top level 5 path LMDE/@home/.snapshots/5/snapshot ID 2514 top level 5 path LMDE/@/.snapshots/5/snapshot ID 2515 top level 5 path LMDE/@/.snapshots/6/snapshot ID 2516 top level 5 path LMDE/@home/.snapshots/6/snapshot ID 2524 top level 5 path LMDE/@/.snapshots/9/snapshot ID 2525 top level 5 path LMDE/@home/.snapshots/9/snapshot ID 2527 top level 5 path LMDE/@/.snapshots/10/snapshot ID 2528 top level 5 path LMDE/@home/.snapshots/10/snapshot ID 2530 top level 5 path LMDE/@/.snapshots/11/snapshot ID 2531 top level 5 path LMDE/@home/.snapshots/11/snapshot ID 2533 top level 5 path LMDE/@/.snapshots/12/snapshot ID 2534 top level 5 path LMDE/@home/.snapshots/12/snapshot ID 2536 top level 5 path LMDE/@/.snapshots/13/snapshot ID 2537 top level 5 path LMDE/@home/.snapshots/13/snapshot ID 2539 top level 5 path LMDE/@/.snapshots/14/snapshot ID 2540 top level 5 path LMDE/@home/.snapshots/14/snapshot ID 2542 top level 5 path LMDE/@/.snapshots/15/snapshot ID 2543 top level 5 path LMDE/@home/.snapshots/15/snapshot ID 2545 top level 5 path LMDE/@/.snapshots/16/snapshot ID 2546 top level 5 path LMDE/@home/.snapshots/16/snapshot ID 2548 top level 5 path LMDE/@/.snapshots/17/snapshot ID 2549 top level 5 path LMDE/@home/.snapshots/17/snapshot ID 2551 top level 5 path LMDE/@/.snapshots/18/snapshot ID 2552 top level 5 path LMDE/@home/.snapshots/18/snapshot ID 2554 top level 5 path LMDE/@/.snapshots/19/snapshot ID 2555 top level 5 path LMDE/@home/.snapshots/19/snapshot ID 2557 top level 5 path LMDE/@/.snapshots/20/snapshot ID 2558 top level 5 path LMDE/@home/.snapshots/20/snapshot ID 2560 top level 5 path LMDE/@/.snapshots/21/snapshot ID 2561 top level 5 path LMDE/@home/.snapshots/21/snapshot ID 2563 top level 5 path LMDE/@/.snapshots/22/snapshot ID 2564 top level 5 path LMDE/@home/.snapshots/22/snapshot ID 2566 top level 5 path LMDE/@/.snapshots/23/snapshot ID 2567 top level 5 path LMDE/@home/.snapshots/23/snapshot ID 2569 top level 5 path LMDE/@/.snapshots/24/snapshot ID 2570 top level 5 path LMDE/@home/.snapshots/24/snapshot ID 2572 top level 5 path LMDE/@/.snapshots/25/snapshot ID 2573 top level 5 path LMDE/@home/.snapshots/25/snapshot ID 2575 top level 5 path LMDE/@/.snapshots/26/snapshot ID 2576 top level 5 path LMDE/@home/.snapshots/26/snapshot ID 2578 top level 5 path LMDE/@/.snapshots/27/snapshot ID 2579 top level 5 path LMDE/@home/.snapshots/27/snapshot ID 2581 top level 5 path LMDE/@/.snapshots/28/snapshot ID 2582 top level 5 path LMDE/@home/.snapshots/28/snapshot ID 2584 top level 5 path LMDE/@home/.snapshots/29/snapshot ID 2585 top level 5 path LMDE/@/.snapshots/29/snapshot ID 2586 top level 5 path LMDE/@/.snapshots/30/snapshot ID 2587 top level 5 path LMDE/@home/.snapshots/30/snapshot ID 2589 top level 5 path LMDE/@/.snapshots/31/snapshot ID 2590 top level 5 path LMDE/@home/.snapshots/31/snapshot ID 2676 top level 5 path UBUNTU/@/.snapshots/443/snapshot ID 2677 top level 5 path UBUNTU/@home/.snapshots/442/snapshot ID 2678 top level 5 path STORAGE/.snapshots/472/snapshot ID 2682 top level 5 path MINT/@/.snapshots/8/snapshot ID 2683 top level 5 path MINT/@home/.snapshots/8/snapshot ID 2718 top level 5 path UBUNTU/@/.snapshots/456/snapshot ID 2719 top level 5 path UBUNTU/@home/.snapshots/455/snapshot ID 2720 top level 5 path STORAGE/.snapshots/486/snapshot ID 2739 top level 5 path UBUNTU/@/.snapshots/463/snapshot ID 2740 top level 5 path UBUNTU/@home/.snapshots/462/snapshot ID 2741 top level 5 path STORAGE/.snapshots/493/snapshot ID 2769 top level 5 path UBUNTU/@/.snapshots/473/snapshot ID 2770 top level 5 path UBUNTU/@home/.snapshots/472/snapshot ID 2771 top level 5 path STORAGE/.snapshots/503/snapshot ID 2808 top level 5 path UBUNTU/@/.snapshots/486/snapshot ID 2809 top level 5 path UBUNTU/@home/.snapshots/485/snapshot ID 2810 top level 5 path STORAGE/.snapshots/516/snapshot ID 2835 top level 5 path UBUNTU/@/.snapshots/495/snapshot ID 2836 top level 5 path UBUNTU/@home/.snapshots/494/snapshot ID 2837 top level 5 path STORAGE/.snapshots/525/snapshot ID 2865 top level 5 path UBUNTU/@/.snapshots/505/snapshot ID 2866 top level 5 path UBUNTU/@home/.snapshots/504/snapshot ID 2867 top level 5 path STORAGE/.snapshots/535/snapshot ID 2910 top level 5 path UBUNTU/@/.snapshots/520/snapshot ID 2911 top level 5 path UBUNTU/@home/.snapshots/519/snapshot ID 2912 top level 5 path STORAGE/.snapshots/550/snapshot ID 2950 top level 5 path UBUNTU/@home/.snapshots/532/snapshot ID 2951 top level 5 path STORAGE/.snapshots/563/snapshot ID 2952 top level 5 path UBUNTU/@/.snapshots/534/snapshot ID 2982 top level 5 path UBUNTU/@/.snapshots/544/snapshot ID 2983 top level 5 path UBUNTU/@home/.snapshots/543/snapshot ID 2984 top level 5 path STORAGE/.snapshots/574/snapshot ID 3018 top level 5 path UBUNTU/@/.snapshots/556/snapshot ID 3019 top level 5 path UBUNTU/@home/.snapshots/555/snapshot ID 3020 top level 5 path STORAGE/.snapshots/586/snapshot ID 3045 top level 5 path MINT/@/.snapshots/9/snapshot ID 3046 top level 5 path MINT/@home/.snapshots/9/snapshot ID 3060 top level 5 path UBUNTU/@/.snapshots/568/snapshot ID 3061 top level 5 path UBUNTU/@home/.snapshots/567/snapshot ID 3062 top level 5 path STORAGE/.snapshots/600/snapshot ID 3069 top level 5 path LMDE/@/.snapshots/32/snapshot ID 3070 top level 5 path LMDE/@home/.snapshots/32/snapshot ID 3105 top level 5 path UBUNTU/@/.snapshots/582/snapshot ID 3106 top level 5 path UBUNTU/@home/.snapshots/581/snapshot ID 3107 top level 5 path STORAGE/.snapshots/615/snapshot ID 3132 top level 5 path UBUNTU/@/.snapshots/591/snapshot ID 3133 top level 5 path UBUNTU/@home/.snapshots/590/snapshot ID 3134 top level 5 path STORAGE/.snapshots/624/snapshot ID 3138 top level 5 path LMDE/@/.snapshots/33/snapshot ID 3139 top level 5 path LMDE/@home/.snapshots/33/snapshot ID 3168 top level 5 path UBUNTU/@/.snapshots/602/snapshot ID 3169 top level 5 path UBUNTU/@home/.snapshots/601/snapshot ID 3170 top level 5 path STORAGE/.snapshots/636/snapshot ID 3171 top level 5 path UBUNTU/@/.snapshots/603/snapshot ID 3172 top level 5 path UBUNTU/@home/.snapshots/602/snapshot ID 3173 top level 5 path STORAGE/.snapshots/637/snapshot ID 3174 top level 5 path UBUNTU/@/.snapshots/604/snapshot ID 3175 top level 5 path UBUNTU/@home/.snapshots/603/snapshot ID 3176 top level 5 path STORAGE/.snapshots/638/snapshot ID 3177 top level 5 path UBUNTU/@/.snapshots/605/snapshot ID 3178 top level 5 path UBUNTU/@home/.snapshots/604/snapshot ID 3179 top level 5 path STORAGE/.snapshots/639/snapshot ID 3186 top level 5 path UBUNTU/@/.snapshots/608/snapshot ID 3187 top level 5 path UBUNTU/@home/.snapshots/607/snapshot ID 3188 top level 5 path STORAGE/.snapshots/642/snapshot ID 3189 top level 5 path UBUNTU/@/.snapshots/609/snapshot ID 3190 top level 5 path UBUNTU/@home/.snapshots/608/snapshot ID 3191 top level 5 path STORAGE/.snapshots/643/snapshot ID 3192 top level 5 path UBUNTU/@/.snapshots/610/snapshot ID 3193 top level 5 path UBUNTU/@home/.snapshots/609/snapshot ID 3194 top level 5 path STORAGE/.snapshots/644/snapshot ID 3195 top level 5 path UBUNTU/@/.snapshots/611/snapshot ID 3196 top level 5 path UBUNTU/@home/.snapshots/610/snapshot ID 3197 top level 5 path STORAGE/.snapshots/645/snapshot ID 3198 top level 5 path UBUNTU/@/.snapshots/612/snapshot ID 3199 top level 5 path UBUNTU/@home/.snapshots/611/snapshot ID 3200 top level 5 path STORAGE/.snapshots/646/snapshot ID 3201 top level 5 path UBUNTU/@/.snapshots/613/snapshot ID 3202 top level 5 path UBUNTU/@home/.snapshots/612/snapshot ID 3203 top level 5 path STORAGE/.snapshots/647/snapshot ID 3204 top level 5 path UBUNTU/@/.snapshots/614/snapshot ID 3205 top level 5 path UBUNTU/@home/.snapshots/613/snapshot ID 3206 top level 5 path STORAGE/.snapshots/648/snapshot ID 3207 top level 5 path UBUNTU/@/.snapshots/615/snapshot ID 3208 top level 5 path UBUNTU/@home/.snapshots/614/snapshot ID 3209 top level 5 path STORAGE/.snapshots/649/snapshot ID 3210 top level 5 path UBUNTU/@/.snapshots/616/snapshot ID 3211 top level 5 path UBUNTU/@home/.snapshots/615/snapshot ID 3212 top level 5 path STORAGE/.snapshots/650/snapshot ID 3213 top level 5 path UBUNTU/@/.snapshots/617/snapshot ID 3214 top level 5 path UBUNTU/@home/.snapshots/616/snapshot ID 3215 top level 5 path STORAGE/.snapshots/651/snapshot ID 3216 top level 5 path UBUNTU/@/.snapshots/618/snapshot ID 3217 top level 5 path UBUNTU/@home/.snapshots/617/snapshot ID 3218 top level 5 path STORAGE/.snapshots/652/snapshot ID 3219 top level 5 path UBUNTU/@/.snapshots/619/snapshot ID 3220 top level 5 path UBUNTU/@home/.snapshots/618/snapshot ID 3221 top level 5 path STORAGE/.snapshots/653/snapshot ID 3222 top level 5 path UBUNTU/@/.snapshots/620/snapshot ID 3223 top level 5 path UBUNTU/@home/.snapshots/619/snapshot ID 3224 top level 5 path STORAGE/.snapshots/654/snapshot ID 3225 top level 5 path UBUNTU/@/.snapshots/621/snapshot ID 3226 top level 5 path UBUNTU/@home/.snapshots/620/snapshot ID 3227 top level 5 path STORAGE/.snapshots/655/snapshot ID 3228 top level 5 path UBUNTU/@/.snapshots/622/snapshot ID 3229 top level 5 path UBUNTU/@home/.snapshots/621/snapshot ID 3230 top level 5 path STORAGE/.snapshots/656/snapshot ID 3231 top level 5 path UBUNTU/@/.snapshots/623/snapshot ID 3232 top level 5 path UBUNTU/@home/.snapshots/622/snapshot ID 3233 top level 5 path STORAGE/.snapshots/657/snapshot ID 3234 top level 5 path UBUNTU/@/.snapshots/624/snapshot ID 3235 top level 5 path UBUNTU/@home/.snapshots/623/snapshot ID 3236 top level 5 path STORAGE/.snapshots/658/snapshot ID 3237 top level 5 path UBUNTU/@/.snapshots/625/snapshot ID 3238 top level 5 path UBUNTU/@home/.snapshots/624/snapshot ID 3239 top level 5 path STORAGE/.snapshots/659/snapshot ID 3240 top level 5 path UBUNTU/@/.snapshots/626/snapshot ID 3241 top level 5 path UBUNTU/@home/.snapshots/625/snapshot ID 3242 top level 5 path STORAGE/.snapshots/660/snapshot Kind regards. -- Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E Ne cherchez pas : Je ne suis pas sur Facebook. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Am Sonntag, 7. Oktober 2012 schrieb Swâmi Petaramesh:> Le 07/10/2012 15:00, Martin Steigerwald a écrit : > > I forgot to mention mount options? Which one do you use? > > root@tethys:/etc# grep btrfs fstab > /dev/VG1/BTR_POOL / btrfs > subvol=UBUNTU/@,space_cache,autodefrag,compress=lzo,relatime 0 0 > /dev/sda2 /boot btrfs > subvol=UBUNTU/@boot,space_cache,autodefrag,relatime 0 0 > /dev/VG1/BTR_POOL /tmp btrfs > subvol=UBUNTU/@tmp,space_cache,autodefrag,compress=lzo,relatime > 0 0 /dev/VG1/BTR_POOL /home btrfs > subvol=UBUNTU/@home,space_cache,autodefrag,compress=lzo,relatime 0 0 > /dev/VG1/BTR_POOL /storage btrfs > subvol=STORAGE,space_cache,autodefrag,compress=lzo,relatime 0 0 > /dev/VG1/BTR_POOL /data/BIG btrfs > subvol=DATA/BIG,space_cache,compress=lzo,noatime 0 0 > /dev/VG1/BTR_POOL /data/VBOX_HIDDEN btrfs > subvol=DATA/VBOX_HIDDEN,space_cache,compress=lzo,noatime 0 0 > /dev/VG1/BTR_POOL /data/VBOX_VMS btrfs > subvol=DATA/VBOX_VMS,space_cache,compress=lzo,noatime 0 0Hmmm, is the slowness on all of these? For the VBOX volumes I would have a possible explaination for slowness. Unless the subvolume is mounted with nodatacow, which it isn´t according to above output, every write to the VM image goes to a new location. Thus this increases fragmentation. While autodefrag might mitigate some of these, it may not be enough. AFAIK its only efficient for certain use cases. So or so it does not appear tha tthe VBOX volumes use autodefrag. Could you try filefrag -v on some of the bigger VM image files? I also wonder about the absence of inode_cache, but this could either be default by now or BTRFS remembers whether what has initially been set. But you could try this one. Especially on the filesystems with lots of inodes. Thanks, -- Martin ''Helios'' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Am Sonntag, 7. Oktober 2012 schrieb Swâmi Petaramesh:> Hi again ;-) > > Le 07/10/2012 14:33, Alex a écrit : > > 1. Convert to a 16k or 32k leafsize. > > How should I do this ? Can I do this on a live FS, and isn''t this going > to double my on-disk used space (I have active snapshots...)I think you need to backup, reformat and restore from backup for now.> > 2. defragment (each non-trivial file) every now and again > > I believed that the "autodefrag" mount option was supposed to avoid the > need for manual defrags...?In theory yes. But AFAIR it is currently limited to certain usecases.> Plus, with respect to snapshots, isn''t this going to increase a lot my > used disk space ?I don´t think so, but I will leave this to a developer to answer.> > 3. Note that btrfs in kernel 3.6 (vs 3.5) does appear to be quite a > > bit quicker. > > I''m not sure that "a bit" quicker will be enough, when I see my BTRFS > is currently about 8x slower than what ext4 user to...How did you measure this? Is this just the boot times you mentioned in the original article? -- Martin ''Helios'' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Am Sonntag, 7. Oktober 2012 schrieb Swâmi Petaramesh:> Le 07/10/2012 12:59, Martin Steigerwald a écrit : > > btrfs fi df (preferably with btrfs tools from Goffredo) > > btrfs fi show > > I don''t think I miss any free space ;-)Well I could I know this beforehand?> (From one of my machines, but the others have rather the same > architecture...) > > # btrfs fi sh > failed to read /dev/sr0 > Label: ''BTR_POOL'' uuid: 7266e630-ba72-404c-bfbd-759d3a6ce2e3 > Total devices 1 FS bytes used 278.55GB > devid 1 size 912.26GB used 464.04GB path /dev/dm-1Hmmm. Looks fine for me. Where is this volume pool located on? On which drive(s)?> Label: ''BTR_BOOT'' uuid: b280100e-6a9a-4488-a207-d4e3f001b4c9 > Total devices 1 FS bytes used 504.39MB > devid 1 size 1.00GB used 729.50MB path /dev/sda2 > > > # btrfs fi df /boot > System: total=4.00MB, used=4.00KB > Data+Metadata: total=725.50MB, used=504.38MB > > # btrfs fi df / > Data: total=426.01GB, used=273.00GB > System, DUP: total=8.00MB, used=56.00KB > System: total=4.00MB, used=0.00 > Metadata, DUP: total=19.00GB, used=5.55GB > Metadata: total=8.00MB, used=0.00 > > # df -h /boot / > Filesystem Size Used Avail Use% Mounted on > /dev/sda2 1,1G 505M 517M 50% /boot > - 913G 285G 602G 33% / > > > ...and I have quite a lot of snapshots, as I use OpenSuSE excellent > "snapper" utility :[…]> ...or if you prefer, something like : > > # btrfs su li / > ID 256 top level 5 path UBUNTU > ID 259 top level 5 path UBUNTU/@ > ID 261 top level 5 path UBUNTU/@tmp > ID 262 top level 5 path UBUNTU/@homeThis could be 100 or more subvolumes / snapshots. Maybe slowness could be related to this one. It would be good to have some capture of this slowness with vmstat 10 iostat -xd 10 /your/device or something like that. Especially for the later, cause it shows the count of reads and writes. Also how much RAM has the machine? What does free -m / cat /proc/meminfo tell. Other than that I leave this too BTRFS developers to answer to. I think that this has become easier by the data you provided. Thanks, -- Martin ''Helios'' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 10/07/2012 04:44 PM, Martin Steigerwald wrote:>> Plus, with respect to snapshots, isn''t this going to increase a lot my >> > used disk space ? > I don´t think so, but I will leave this to a developer to answer. >IIRC the defrag is not [fully] snapshot aware, so de-frag-ging a snapshot-ted file should break the COW link. Liu Bo is working to patch on that: see " Btrfs: use flag EXTENT_DEFRAG for snapshot-aware defrag" dated 2012/09/17. Please could you clarify if you are using the "autodefrag" options when you have the performance problem ? Are you in position to to reduce the number of snapshot from hundreds to few ? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Le 08/10/2012 00:47, Goffredo Baroncelli a écrit :> Please could you clarify if you are using the "autodefrag" options > when you have the performance problem ?I use autodefrag on all volumes systematically, except on volumes that I use for really big files that would always be defragmenting (i.e. virtual machines, but I seldom use virtual machines, and my performance problem is general, I don''t take VMs into consideration in this respect...)> Are you in position to to reduce the number of snapshot from hundreds > to few ?I might do this as an experiment, but it would defeat one of the prominent purposes for which I use BTRFS, and the number of snapshots would grow immediately again, as I use the excellent OpenSuSE "snapper" tool, that makes a snapshot every single hour... Kind regards. -- Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E Ne cherchez pas : Je ne suis pas sur Facebook. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Martin, Le 07/10/2012 16:48, Martin Steigerwald a écrit :> Where is this volume pool located on? On which drive(s)?All the concerned machines are laptops with a single physical HD...> This could be 100 or more subvolumes / snapshots. > > Maybe slowness could be related to this one.That''s a good estimation... I''m not sure if there is any theoretical reason why the number of snapshots of a given volume might slow down access to this volume ?> It would be good to have some capture of this slowness with > > vmstat 10 > > iostat -xd 10 /your/deviceI will try to give you some figures. But when the machine starts really crawling, it becomes very hard to start any new program unless it''s "finished being slow", so that''s not so easy...> Also how much RAM has the machine? What does free -m / cat /proc/meminfo > tell.The 4 concerned machines have either 4GB or 2GB RAM. None is heavily loaded as far as RAM is concerned (just booting into a standard Ubuntu GUI and then accessing emailn, browser, office applications etc.) Kind regards. -- Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E Ne cherchez pas : Je ne suis pas sur Facebook. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Le 07/10/2012 16:44, Martin Steigerwald a écrit :> I think you need to backup, reformat and restore from backup for now.No way. 4 machines on each of which 2 to 4 different OSes are sharing the same BTRFS volume ! If I ever need to reformat/reinstall all this, the new format won''t be BTRFS ! I would be pissed off to a point where I would stick to either ext4 or ZFS, all of which work nicely (I have a complex system that has been running very fine on ZFS for 1.5 year)...> How did you measure this? Is this just the boot times you mentioned in > the original article?No highly scientific measurement. I don''t live with benchmarks, but with the impression that my machine is nicely swift or painfully slow. When it comes to 4 minutes for boot, 40 seconds for opening a brower, 1+ minute for Thunderbird... 15 minutes for installing an innocent set of updates... I''m under the impression that it''s painfully slow. Kind regards. -- Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E Ne cherchez pas : Je ne suis pas sur Facebook. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Oct 8, 2012 at 8:08 AM, Swâmi Petaramesh <swami@petaramesh.org> wrote:> Le 08/10/2012 00:47, Goffredo Baroncelli a écrit : >> Please could you clarify if you are using the "autodefrag" options >> when you have the performance problem ? > > I use autodefrag on all volumes systematically, except on volumes that I > use for really big files that would always be defragmenting (i.e. > virtual machines, but I seldom use virtual machines, and my performance > problem is general, I don''t take VMs into consideration in this respect...)The autodefrag option is per filesystem not per subvolume. The settings of the first subvolueme is used also for the other ones. So, because you mount the same filesystem both on> /dev/VG1/BTR_POOL / btrfssubvol=UBUNTU/@,space_cache,autodefrag,compress=lzo,relatime 0 0 and> /dev/VG1/BTR_POOL /data/VBOX_HIDDEN btrfssubvol=DATA/VBOX_HIDDEN,space_cache,compress=lzo,noatime 0 0> /dev/VG1/BTR_POOL /data/VBOX_VMS btrfssubvol=DATA/VBOX_VMS,space_cache,compress=lzo,noatime 0 0 the autodefrag option is enabled also for DATA/VBOX_HIDDEN and DATA/VBOM_VMS> >> Are you in position to to reduce the number of snapshot from hundreds >> to few ? > > I might do this as an experiment, but it would defeat one of the > prominent purposes for which I use BTRFS, and the number of snapshots > would grow immediately again, as I use the excellent OpenSuSE "snapper" > tool, that makes a snapshot every single hour...I am not suggesting that as solution, but this would help to investigate the problem. If you don''t have any needs of the snapshot, you can delete it, wait that the cleaner kernel thread does its job (it could require a bit of time), then reboot the machine. If I am right the performance should goes high. I fear that both the combination of autodefrag and the high number of snapshot could be the root-cause of the the bad performance.> > Kind regards.Ciao Goffredo> -- > Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E > Ne cherchez pas : Je ne suis pas sur Facebook. >-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Le 08/10/2012 13:38, Goffredo Baroncelli a écrit :> The autodefrag option is per filesystem not per subvolume. The settings > of the first subvolueme is used also for the other ones.Uh ! So there is no interest in creating several subvols, some for which files should be autodefragged, and some not ? That''s to bad :-(> I fear that both the combination of autodefrag and the high number of > snapshot could be the root-cause of the the bad performance.As a test, I will try to remove *most* of my snapshots and see if it helps... Thanks for the suggestion ! Kind regards. -- Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E Ne cherchez pas : Je ne suis pas sur Facebook. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi again Goffredo, Le 08/10/2012 13:38, Goffredo Baroncelli a écrit :> I fear that both the combination of autodefrag and the high number of > snapshot could be the root-cause of the the bad performance.I''ve removed, on one of my machines, all snapshots but three per subvol (keeping the oldests and newest), going from about 30 per subvol to 3, for the complete filesystem from 120+ to about a dozen. Then I let btrfs-cleaner do its job After that the machine boots to GUI in a bit less than 2 minutes, where it was more than 4 minutes previously. The machine now seems much more reactive and swift. So it seems that the number or active snapshots (or is it the number of subvols whatsoever ??) dramatically impacts performance... Thanks for the suggestion. -- Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E Ne cherchez pas : Je ne suis pas sur Facebook. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Sun, Oct 07, 2012 at 03:26:32AM -0600, Swâmi Petaramesh wrote:> Hi, > > I have 4 machines, all converted to BTRFS about 6 months ago, now all > running Ubuntu Quantal with kernel 3.5.0-17 > > The matter is that all these machines are now getting slower and slower > everyday, every disk access causing the disk to be 100% busy for long > periods, to the point that I''m now seriously considering migrating > everything back to ext4... > > From the start BTRFS was "not very fast", still satisfactory, but now it > becomes truly unusable. > > On one machine, I know have a typical complete boot time to a usable GUI > that is over 4 minutes, with the HD still very busy for a couple more > minutes afterwards, where it used to be around 35-40 seconds in ext4 ! > > Is there anything I could do to speed things back (without losing all my > snapshots or doubling the size of data on disk)...? > > I already had made the move back from BTRFS to ext4 about 18 months ago, > I found it had improved so was back to BTRFS, and I wouldn''t have to > revert back again :-/ > > Any advice or help greatly appreciated. >Can you get sysrq+w when you are seeing slowness? Usually bootup slow times means you don''t have space_cache enabled or your cache is being evicted for some reason, can you check dmesg after bootup for messages related to space cache? Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Le 08/10/2012 18:09, Josef Bacik a écrit :> Can you get sysrq+w when you are seeing slowness? Usually bootup slow times > means you don''t have space_cache enabled or your cache is being evicted for some > reason, can you check dmesg after bootup for messages related to space cache? > Thanks,I have a few : Oct 8 15:27:26 tethys kernel: [16174.736603] btrfs: free space inode generation (0) did not match free space cache generation (106988) Oct 8 15:27:27 tethys kernel: [16175.976784] btrfs: free space inode generation (0) did not match free space cache generation (30727) Oct 8 15:27:28 tethys kernel: [16176.420719] btrfs: free space inode generation (0) did not match free space cache generation (48040) Oct 8 15:27:28 tethys kernel: [16176.710972] btrfs: free space inode generation (0) did not match free space cache generation (30745) ...in syslog, but that''s about all... and not during boot... I used to have much much more of these in syslog, but solved it by booting once with the "clear_cache" option, that caused boot to be extremely slow, but seemed to fix it... (Remember I have such issues on several machines, it is highly improbable that all of them would get their cache ignored...?) Kind regards. -- Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E Ne cherchez pas : Je ne suis pas sur Facebook. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Oct 08, 2012 at 10:15:51AM -0600, Swâmi Petaramesh wrote:> Le 08/10/2012 18:09, Josef Bacik a écrit : > > Can you get sysrq+w when you are seeing slowness? Usually bootup slow times > > means you don''t have space_cache enabled or your cache is being evicted for some > > reason, can you check dmesg after bootup for messages related to space cache? > > Thanks, > I have a few : > > Oct 8 15:27:26 tethys kernel: [16174.736603] btrfs: free space inode > generation (0) did not match free space cache generation (106988) > Oct 8 15:27:27 tethys kernel: [16175.976784] btrfs: free space inode > generation (0) did not match free space cache generation (30727) > Oct 8 15:27:28 tethys kernel: [16176.420719] btrfs: free space inode > generation (0) did not match free space cache generation (48040) > Oct 8 15:27:28 tethys kernel: [16176.710972] btrfs: free space inode > generation (0) did not match free space cache generation (30745) > > ...in syslog, but that''s about all... and not during boot... > > I used to have much much more of these in syslog, but solved it by > booting once with the "clear_cache" option, that caused boot to be > extremely slow, but seemed to fix it... > > (Remember I have such issues on several machines, it is highly > improbable that all of them would get their cache ignored...?)Well what happens is on a actually used fs it ends up being more fragmented than the amount we''re allowed to preallocate for our space cache, and so we don''t write anything out, so it''s very likely that all of your machines could be hitting that. I put a patch into 3.6 to increase the cache size so that wouldn''t happen as much, perhaps move to 3.6 and see if you see some improvements? Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 10/08/2012 05:50 PM, Swâmi Petaramesh wrote:> Hi again Goffredo, > > Le 08/10/2012 13:38, Goffredo Baroncelli a écrit : >> I fear that both the combination of autodefrag and the high number of >> snapshot could be the root-cause of the the bad performance. > I''ve removed, on one of my machines, all snapshots but three per subvol > (keeping the oldests and newest), going from about 30 per subvol to 3, > for the complete filesystem from 120+ to about a dozen. > > Then I let btrfs-cleaner do its job > > After that the machine boots to GUI in a bit less than 2 minutes, where > it was more than 4 minutes previously. > > The machine now seems much more reactive and swift. > > So it seems that the number or active snapshots (or is it the number of > subvols whatsoever ??) dramatically impacts performance...Does the autodefrag options still alive ? I believe that the snapshot is quite cheap, except if you update the shared files one at time. Which should be the case of the autodefrag. But it is only a my supposition.... Could you please try to avoid the autodefrag option in a machine with an high number of snapshot ? I am curios...> > Thanks for the suggestion. >-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 2012-10-07 15:19, Swâmi Petaramesh wrote:> Hi again ;-) > Le 07/10/2012 14:33, Alex a écrit : >> 1. Convert to a 16k or 32k leafsize. > How should I do this ? Can I do this on a live FS, and isn''t this going > to double my on-disk used space (I have active snapshots...) >> 2. defragment (each non-trivial file) every now and again > I believed that the "autodefrag" mount option was supposed to avoid the > need for manual defrags...? > > Plus, with respect to snapshots, isn''t this going to increase a lot my > used disk space ? > >> 3. Note that btrfs in kernel 3.6 (vs 3.5) does appear to be quite a bit quicker. > I''m not sure that "a bit" quicker will be enough, when I see my BTRFS is > currently about 8x slower than what ext4 user to... >Here is a graph to reinforce what Alex said about performance after 3.5.x. This graph was posted in #btrfs on freenode. At the end of the graph, the poster said that he upgraded from a 3.5.x to a 3.6.x https://pipebreaker.pl/dump/loadl.png I noticed the same thing going from 3.5.0 to 3.7.0rc2; system usage is much lower. Before this I would often find huge lags and then check and see btrfs-cleaner, btrfs-transactionhelpersomething, etc. taking up 100% io. So if your system usage is high, this will help. Also possibly not relevant for you: on my desktop, (btrfs or ext4, with consumer disks) I find that deadline scheduler and nr_requests of 8 is vastly superior for reducing GUI lags. (default is scheduler cfq and 128 nr_requests). But for servers, I don''t know what to recommend. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Sunday 2012-10-07 16:48, Martin Steigerwald wrote:>> >> # btrfs su li / >> ID 256 top level 5 path UBUNTU >> ID 259 top level 5 path UBUNTU/@ >> ID 261 top level 5 path UBUNTU/@tmp >> ID 262 top level 5 path UBUNTU/@home >>[...] > >This could be 100 or more subvolumes / snapshots. >Maybe slowness could be related to this one.Absolutely. COW snapshots cause severe fragmentation (it''s in their nature). "Easily" reproducible by snapshotting an fs every day. Been there, had that. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Am Sonntag, 9. Dezember 2012 schrieb Jan Engelhardt:> On Sunday 2012-10-07 16:48, Martin Steigerwald wrote: > >> > >> # btrfs su li / > >> ID 256 top level 5 path UBUNTU > >> ID 259 top level 5 path UBUNTU/@ > >> ID 261 top level 5 path UBUNTU/@tmp > >> ID 262 top level 5 path UBUNTU/@home > >>[...] > > > >This could be 100 or more subvolumes / snapshots. > >Maybe slowness could be related to this one. > > Absolutely. COW snapshots cause severe fragmentation (it''s > in their nature). "Easily" reproducible by snapshotting an fs > every day. Been there, had that.So such a workload is only possible with pleasing speed with SSD? I have about 5-10 snapshots on my backup 2TB eSATA disk which I currently rsync to. This seems to work quite well still. But I have no idea how that would turn out with 100 snapshots or more. And about 1 TB is free, so at least I´d expect no serious freespace fragmentation like on the BTRFS in my SSD (extra post for that one). Ciao, -- Martin ''Helios'' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Sun, 9 Dec 2012 06:17:39 +0100 (CET) Jan Engelhardt <jengelh@inai.de> wrote:> > On Sunday 2012-10-07 16:48, Martin Steigerwald wrote: > >> > >> # btrfs su li / > >> ID 256 top level 5 path UBUNTU > >> ID 259 top level 5 path UBUNTU/@ > >> ID 261 top level 5 path UBUNTU/@tmp > >> ID 262 top level 5 path UBUNTU/@home > >>[...] > > > >This could be 100 or more subvolumes / snapshots. > >Maybe slowness could be related to this one. > > Absolutely. COW snapshots cause severe fragmentation (it''s > in their nature).CoW filesystem incurs fragmentation by its nature, not specifically snapshots. Even without snapshots, rewriting portions of existing files will write the new blocks not over the original ones, but elsewhere, thus increasing fragmentation. -- With respect, Roman ~~~~~~~~~~~~~~~~~~~~~~~~~~~ "Stallman had a printer, with code he could not see. So he began to tinker, and set the software free."
Le 09/12/2012 11:41, Roman Mamedov a écrit :> CoW filesystem incurs fragmentation by its nature, not specifically snapshots. > Even without snapshots, rewriting portions of existing files will write the > new blocks not over the original ones, but elsewhere, thus increasing > fragmentation.Is it to expect that somewhere in the future, BTRFS will be able to defragment itself without duplicating snapshot data ? -- Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E Ne cherchez pas : Je ne suis pas sur Facebook. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Sun, Dec 09, 2012 at 12:20:46PM +0100, Swâmi Petaramesh wrote:> Le 09/12/2012 11:41, Roman Mamedov a écrit : > > CoW filesystem incurs fragmentation by its nature, not specifically snapshots. > > Even without snapshots, rewriting portions of existing files will write the > > new blocks not over the original ones, but elsewhere, thus increasing > > fragmentation. > > Is it to expect that somewhere in the future, BTRFS will be able to > defragment itself without duplicating snapshot data ?In the presence of snapshots that are modified, no, it''s impossible to fully defrag all the files. Consider a file, AAABBBCCC, which is snapshotted, and then the middle third is rewritten, so the snapshot shows AAAZZZCCC. You now have two files which share their first and last sections, but the centre section is different. One or other of those files *must* be fragmented -- you can''t eliminate fragmentation here. If you mean, will there be a defrag that does as well as it can despite the limitation above, and which doesn''t split snapshots into separate copies, then yes -- there''s already patches out for that, but they''ve had a couple of problems. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- emacs: Emacs Makes A Computer Slow. ---
On Sunday 2012-12-09 11:41, Roman Mamedov wrote:>> >> Absolutely. COW snapshots cause severe fragmentation (it''s >> in their nature). > >CoW filesystem incurs fragmentation by its nature, not specifically snapshots. >Even without snapshots, rewriting portions of existing files will write the >new blocks not over the original ones, but elsewhere, thus increasing >fragmentation.Right, of course. In the "normal" case, the originals are deleted at some point, so the length of the CoW "chain" should be bound. However, with snapshots, there is, I wager to say, no limit to the chain length, so defragmenting a file also becomes more work. One can of course use rsync without --inplace, which does a full file copy-up if the file has changed, which essentially disables CoW, and thus reduces fragmentation. Unfortunately, not all copy operations on one''s machine do use rsync, so perhaps it would be nice to have this as a subvolume mount option as well. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 12/09/12 12:38, Hugo Mills wrote:> On Sun, Dec 09, 2012 at 12:20:46PM +0100, Swâmi Petaramesh wrote: >> Le 09/12/2012 11:41, Roman Mamedov a écrit : >>> CoW filesystem incurs fragmentation by its nature, not specifically snapshots. >>> Even without snapshots, rewriting portions of existing files will write the >>> new blocks not over the original ones, but elsewhere, thus increasing >>> fragmentation. >> Is it to expect that somewhere in the future, BTRFS will be able to >> defragment itself without duplicating snapshot data ? > In the presence of snapshots that are modified, no, it''s impossible > to fully defrag all the files.Of course, but I would agree with the poster that it would be important to "partially" defrag all the files, avoiding at least unneeded fragmentation... At least the fragmentation generated by normal writes -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html