Hi list, I don''t trust theoretical benchmarks that much and prefer "real-life benchs" on the occasion, so here''s mine: Given 4 laptops, the most powerful of which was running BTRFS and the others ext3 or ext4, all machines running Ubuntu 11.04 Natty 32-bit with a stock Ubuntu 2.6.38-11 kernel, all machines were given the following FS-intensive task : - Upgrade O.S. from Ubuntu 11.04 Natty to 11.10 Oneiric (beta), using a local packages apt-cacher. Machines : 1/ Acer Aspire 3104 WLMi, AMD Sempron Mobile 3500+ @1.8 GHz, 1.5 GB RAM, 80 GB SATA HD, ext4 over standard partitions 2/ Asus EeePC 1005PE, Intel Atom N450 @1.66 GHz, 2 GB RAM, 250 GB SATA HD, ext3 over AES-128 encrypted LVM 3/ Compaq Mini CQ10-740SF, Intel Atom N455 @1.66 GHz, 1 GB RAM, 250 GB SATA HD, ext4 over standard partitions 4/ Dell XPS M1330, Intel Core 2 Duo T9300 @2.5 GHz, 2 GB RAM, 160 B SATA HD, BTRFS over AES-128 encrypted LVM Results : All 3 ext3 / ext4 machines took between 60 and 90 minutes to complete their upgrade. BTRFS machine took 20 HOURS so far, still counting (ETA 15 minutes left). Wow. Impressive. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Sep 05, 2011 at 03:51:17PM +0200, Swâmi Petaramesh wrote:> Hi list, > > I don''t trust theoretical benchmarks that much and prefer "real-life > benchs" on the occasion, so here''s mine: > > Given 4 laptops, the most powerful of which was running BTRFS and > the others ext3 or ext4, all machines running Ubuntu 11.04 Natty > 32-bit with a stock Ubuntu 2.6.38-11 kernel, all machines were given > the following FS-intensive task : > > - Upgrade O.S. from Ubuntu 11.04 Natty to 11.10 Oneiric (beta), > using a local packages apt-cacher. > > Machines : > > 1/ Acer Aspire 3104 WLMi, AMD Sempron Mobile 3500+ @1.8 GHz, 1.5 GB > RAM, 80 GB SATA HD, ext4 over standard partitions > > 2/ Asus EeePC 1005PE, Intel Atom N450 @1.66 GHz, 2 GB RAM, 250 GB > SATA HD, ext3 over AES-128 encrypted LVM > > 3/ Compaq Mini CQ10-740SF, Intel Atom N455 @1.66 GHz, 1 GB RAM, 250 > GB SATA HD, ext4 over standard partitions > > 4/ Dell XPS M1330, Intel Core 2 Duo T9300 @2.5 GHz, 2 GB RAM, 160 B > SATA HD, BTRFS over AES-128 encrypted LVM > > Results : > > All 3 ext3 / ext4 machines took between 60 and 90 minutes to > complete their upgrade. > > BTRFS machine took 20 HOURS so far, still counting (ETA 15 minutes left). > > Wow. Impressive.That''s because dpkg is known for using (f)sync very heavily. btrfs honours the sync request in all cases, so it''s much much slower than ext3, which doesn''t. There''s an LD_PRELOAD library called "eatmydata" that you can use to paper over the issue by running it as a wrapper around dpkg. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- Always be sincere, whether you mean it or not. ---
On 05/09/11 14:51, � wrote:> Given 4 laptops, the most powerful of which was running BTRFS and the others > ext3 or ext4, all machines running Ubuntu 11.04 Natty 32-bit with a stock > Ubuntu 2.6.38-11 kernel, all machines were given the following FS-intensive > task :(dpkg-intensive workload)> All 3 ext3 / ext4 machines took between 60 and 90 minutes to complete their > upgrade. > > BTRFS machine took 20 HOURS so far, still counting (ETA 15 minutes left). > > Wow. Impressive.I see similar behaviour on a single-disk workstation. This is a consequence of dpkg''s heavy use of fsync() to ensure that changes to the filesystem are done safely, to try to ensure that a sudden crash or power interruption doesn''t leave the machine in an inconsistent or unrecoverable state. (You can see that btrfs performs speedily if fsync is disabled via, for example, `eatmydata`. Which is not a production-worthy workaround for the majority of cases, but usefully highlights what''s going wrong.) See also: https://bugs.launchpad.net/ubuntu/+source/dpkg/+bug/607632 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=635993 Now, I don''t know why btrfs is particularly slow with a (relatively?) fsync-heavy workload, though I do note that a commit went into 3.0 that improved some fsync workloads; see commit: 8e531cdfeb75269c6c5aae33651cca39707848da However, in my own testing, it doesn''t seem to have improved matters significantly. There were also a number of fsync-related improvements made in 2.6.32; perhaps there have been regressions since then? It seems clear that the dpkg developers consider this to be reasonable behaviour on their part; is there any scope for improvements to be made to btrfs to cope better with this kind of workload? Or is this an unavoidable property of the datastructures it''s using? Cheers, David -- David McBride <dwm@doc.ic.ac.uk> Department of Computing, Imperial College, London -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, 5 Sep 2011 15:00:23 +0100 Hugo Mills <hugo@carfax.org.uk> wrote:> > > > BTRFS machine took 20 HOURS so far, still counting (ETA 15 minutes left). > > > > Wow. Impressive. > > That''s because dpkg is known for using (f)sync very heavily. btrfs > honours the sync request in all casesI don''t think it''s *just* that. I had the same problem, recently upgrading Debian from Stable to Testing, it was taking more than 24 hours. I stopped the upgrade, then resumed it this time via eatmydata, and it proceeded perhaps two orders of magnitude faster than before, finishing the remaining packages in 5 minutes or so. Problem is, even when I stopped the upgrade and waited a considerable time, each ''sync'' was still taking 5-7 seconds. No other disk activity in the system, no snapshot creation/deletion/cleanup going on either, just multiple consecutive syncs: rm@rm:~$ time sync real 0m4.772s user 0m0.000s sys 0m0.480s rm@rm:~$ time sync real 0m6.831s user 0m0.000s sys 0m0.472s rm@rm:~$ time sync real 0m8.069s user 0m0.000s sys 0m0.468s rm@rm:~$ time sync real 0m6.675s user 0m0.000s sys 0m0.464s rm@rm:~$ time sync real 0m4.293s user 0m0.000s sys 0m0.464s rm@rm:~$ time sync real 0m4.230s user 0m0.000s sys 0m0.472s rm@rm:~$ time sync real 0m6.924s user 0m0.000s sys 0m0.464s To be fair, this was on the 2.6.39.2 kernel, and the performance seems to be somewhat better on 3.0 (though I didn''t do tests like this one or any significant dpkg operations on it yet). -- With respect, Roman
> That''s because dpkg is known for using (f)sync very heavily. btrfs > honours the sync request in all cases, so it''s much much slower than > ext3, which doesn''t.Hmm, is it really the case with ext3/ext4 (ignoring fsync in some cases)? Sounds like a bug in ext3/ext4 then. Is it documented anywhere where ext3/ext4 would just silently ignore fsync? -- Tomasz Chmielewski http://wpkg.org -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Sep 05, 2011 at 06:23:23PM +0200, Tomasz Chmielewski wrote:> > That''s because dpkg is known for using (f)sync very heavily. btrfs > >honours the sync request in all cases, so it''s much much slower than > >ext3, which doesn''t. > > Hmm, is it really the case with ext3/ext4 (ignoring fsync in some cases)? > > Sounds like a bug in ext3/ext4 then. > > Is it documented anywhere where ext3/ext4 would just silently ignore fsync?They don''t. Unteil recently ext3 and reiserfs would not flush the disk caches unless enabled by a mount option, but even that has recently been fixed. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Sep 05, 2011 at 12:25:21PM -0400, Christoph Hellwig wrote:> On Mon, Sep 05, 2011 at 06:23:23PM +0200, Tomasz Chmielewski wrote: > > > That''s because dpkg is known for using (f)sync very heavily. btrfs > > >honours the sync request in all cases, so it''s much much slower than > > >ext3, which doesn''t. > > > > Hmm, is it really the case with ext3/ext4 (ignoring fsync in some cases)? > > > > Sounds like a bug in ext3/ext4 then. > > > > Is it documented anywhere where ext3/ext4 would just silently ignore fsync? > > They don''t. Unteil recently ext3 and reiserfs would not flush the > disk caches unless enabled by a mount option, but even that has recently > been fixed.Apologies for disseminating my misunderstanding, then. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- You can get more with a kind word and a two-by-four than you --- can with just a kind word.
On Monday 05 September 2011 16:20:00 Roman Mamedov wrote: [...]> real 0m6.924s > user 0m0.000s > sys 0m0.464s > > To be fair, this was on the 2.6.39.2 kernel, and the performance seems to > be somewhat better on 3.0 (though I didn''t do tests like this one or any > significant dpkg operations on it yet).Same here with 3.1-rc4 in my Debian SID amd64 laptop with Vertex II SSD disk. `--> time sync ; time sync; time sync; time sync;time sync ; time sync; time sync; time sync;time sync ; time sync; time sync; time sync sync 0.00s user 0.09s system 3% cpu 2.509 total sync 0.00s user 0.09s system 3% cpu 2.690 total sync 0.00s user 0.09s system 4% cpu 2.219 total sync 0.00s user 0.09s system 3% cpu 2.270 total sync 0.00s user 0.08s system 3% cpu 2.120 total sync 0.00s user 0.09s system 4% cpu 2.170 total sync 0.00s user 0.09s system 4% cpu 2.120 total sync 0.00s user 0.09s system 3% cpu 2.189 total sync 0.00s user 0.08s system 3% cpu 2.341 total sync 0.00s user 0.09s system 3% cpu 2.262 total sync 0.00s user 0.10s system 2% cpu 3.967 total sync 0.00s user 0.09s system 4% cpu 2.271 total -- Elric Milon -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, 05 Sep 2011 15:51:17 +0200 Swâmi Petaramesh <swami@petaramesh.org> wrote:> Hi list, > > I don''t trust theoretical benchmarks that much and prefer "real-life > benchs" on the occasion, so here''s mine: > > Given 4 laptops, the most powerful of which was running BTRFS and the > others ext3 or ext4, all machines running Ubuntu 11.04 Natty 32-bit with > a stock Ubuntu 2.6.38-11 kernel, all machines were given the following > FS-intensive task : > > - Upgrade O.S. from Ubuntu 11.04 Natty to 11.10 Oneiric (beta), using a > local packages apt-cacher. > > Machines : > > 1/ Acer Aspire 3104 WLMi, AMD Sempron Mobile 3500+ @1.8 GHz, 1.5 GB RAM, > 80 GB SATA HD, ext4 over standard partitions > > 2/ Asus EeePC 1005PE, Intel Atom N450 @1.66 GHz, 2 GB RAM, 250 GB SATA > HD, ext3 over AES-128 encrypted LVM > > 3/ Compaq Mini CQ10-740SF, Intel Atom N455 @1.66 GHz, 1 GB RAM, 250 GB > SATA HD, ext4 over standard partitions > > 4/ Dell XPS M1330, Intel Core 2 Duo T9300 @2.5 GHz, 2 GB RAM, 160 B SATA > HD, BTRFS over AES-128 encrypted LVM > > Results : > > All 3 ext3 / ext4 machines took between 60 and 90 minutes to complete > their upgrade. > > BTRFS machine took 20 HOURS so far, still counting (ETA 15 minutes left). > > Wow. Impressive.I''ve seen similar problem on Ubuntu-11 + Aspire One (8GB of slow "SSD"). More specifically half of ubuntu install went very fast and when disk was ~50% free things _suddenly_ gone slow. According to strace fsync() started to take ~10 seconds(!) and upper. It''s not common filesystem behaviour, but a plain (and hard to catch?) bug. I don''t have the box anymore. The root cause it to be found by some brave soul yet. I wonder if latencytop would get the profile of the issue. Maybe, btrfs degrades only after certain (insanely large?) amount of fsync() calls? -- Sergei
On Monday 5 September 2011 22:25:23 Sergei Trofimovich wrote:> I''ve seen similar problem on Ubuntu-11 + Aspire One (8GB of slow "SSD"). > More specifically half of ubuntu install went very fast and when > disk was ~50% free things suddenly gone slow.I''m just about to give up and definitely quit using BTRFS. My system has become so slow after upgrade Ubuntu Natty => Oneiric Beta, that''s it''s purely and simply unusable (although I''m the kind of old, white-haired, experienced, used to prehistoric systems, thus very patient, IT guy...) It''a most probable that all of my usage patterns, i.e. read my mail, browse the web, etc, definitely do not correspond to what BTRFS was designed for. (Sorry for the rant, but this really pisses me off...) So I''m only wonderign whether I reformat my system to ext4 or ZFS, and whether I do it right now or on thursday... -- Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Sep 6, 2011 at 10:30 PM, Swâmi Petaramesh <swami@petaramesh.org> wrote:> On Monday 5 September 2011 22:25:23 Sergei Trofimovich wrote: >> I''ve seen similar problem on Ubuntu-11 + Aspire One (8GB of slow "SSD"). >> More specifically half of ubuntu install went very fast and when >> disk was ~50% free things suddenly gone slow. > > I''m just about to give up and definitely quit using BTRFS. My system has become > so slow after upgrade Ubuntu Natty => Oneiric Beta, that''s it''s purely and > simply unusable (although I''m the kind of old, white-haired, experienced, used > to prehistoric systems, thus very patient, IT guy...)Compared to ext3/4, btrfs or zfs (on linux) in its current state would seem like snail. The only way I can get decent speed with btrfs on my laptop is after using sandforce-based SSD (which helps offset some of the slowness). But since I like lzo compression and snapshot feature, I''ll keep on using btrfs on this one :)> > It''a most probable that all of my usage patterns, i.e. read my mail, browse > the web, etc, definitely do not correspond to what BTRFS was designed for. > (Sorry for the rant, but this really pisses me off...) > > So I''m only wonderign whether I reformat my system to ext4 or ZFS, and whether > I do it right now or on thursday...Reading your post, at this point I''d actually recommend you stick with ext4. Both btrfs and zfs are great, but IMHO btrfs is not ready for daily use by "ordinary" user yet, while zfs is a memory hog (especially for laptops, which is part of the reason why I''m using btrfs instead of zfs on this one). FWIW I have another laptop with 4GB ram and Ubuntu + zfs root on it, and it would seem "stall" on some operations with no apparent cpu or disk activity. zfs compression and snapshot feature works great though (and much more stable compared to btrfs, which stiil doesn''t have a functioning fsck), so if you have the resource to spare you might want to give zfs another try later. -- Fajar -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Le Mercredi 7 Septembre 2011 00:11:25 vous avez écrit :> Reading your post, at this point I''d actually recommend you stick with > ext4.I actually shifted back from BTRFS to ext4 and fell like having offered myself a brand new computer, about 20 times faster, me happy ;-)> Both btrfs and zfs are great, but IMHO btrfs is not ready for > daily use by "ordinary" user yet, while zfs is a memory hog > (especially for laptops, which is part of the reason why I''m using > btrfs instead of zfs on this one).True, ZFS is excellent but a memory hog (and strongly advises using a 64-bit OS) but I was surprised to discover that BTRFS was such a memory eater itself, with kernel 3.0. My system was swapping like mad ! I use (kernel) ZFS on my 64-bit main machine and I''m plain happy with it, and tried ZFS on my 32-bit laptop in the hope to get more performance for less memory ; alas I just got a memory-hungry system running damn slow... For the time being I will stick to ZFS for 64-bit machines with >= 4GB RAM, and to ext4 for 32-bit systems with less RAM... I don''t feel that BTRFS gives any advantage in its current state of development. Alas. -- Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 09/05/2011 06:29 PM, Hugo Mills wrote:> On Mon, Sep 05, 2011 at 12:25:21PM -0400, Christoph Hellwig wrote: >> On Mon, Sep 05, 2011 at 06:23:23PM +0200, Tomasz Chmielewski wrote: >>>> That''s because dpkg is known for using (f)sync very heavily. btrfs >>>> honours the sync request in all cases, so it''s much much slower than >>>> ext3, which doesn''t. >>> >>> Hmm, is it really the case with ext3/ext4 (ignoring fsync in some cases)? >>> >>> Sounds like a bug in ext3/ext4 then. >>> >>> Is it documented anywhere where ext3/ext4 would just silently ignore fsync? >> >> They don''t. Unteil recently ext3 and reiserfs would not flush the >> disk caches unless enabled by a mount option, but even that has recently >> been fixed. > > Apologies for disseminating my misunderstanding, then. > > Hugo. >I''m wondering if this may be the very same unacceptable behavior as discussed earlier also in the following message threads: btrfs always writes something after sync news://news.gmane.org:119/CABVOS09hntXYUsCJBg+byEGC5y53j4DeT6b49Rx1AaRpqFV54A@mail.gmail.com Re: Writes in idle/How to debug filesystem activity news://news.gmane.org:119/4E5C8A20.7030604@wpkg.org Re: Applications using fsync cause hangs for several seconds every few minutes news://news.gmane.org:119/4E41A6A4.9070702@uvm.edu Btrfs slowdown news://news.gmane.org:119/CAO47_-9BLKWUGDEuzaLqHSq9tZkAUaO8FMQEy1pPk9A2Hb+5AQ@mail.gmail.com PLEASE TEST: Everybody who is seeing weird and long hangs news://news.gmane.org:119/4E36C47E.70309@redhat.com and this: curious writes on mounted, not used btrfs filesystem http://thread.gmane.org/gmane.comp.file-systems.btrfs/10840 and probably this: 2.6.37: Multi-second I/O latency while untarring http://www.spinics.net/lists/linux-btrfs/msg08516.html Hasn''t the slowdown been fixed yet? For those experiencing the slowdown issue btrfs is entirely useless... (I had to stop using btrfs because of these frequent slowdowns, while btrfs was doing most likely needless disk writes for multiple seconds, I couldn''t work at all) Can someone look into this if this is not fixed? There are numerous analysis reports in the above threads, without any constructive result. Thanks -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
I''m using btrfs since one year now and it''s quite fast. I don''t feel any differences to other filesystems. Never tried a benchmark but for my daily work it''s nice. I also never had any issues with the memory. Imho nowadays memory isn''t a problem at all in desktop computers. I bought 8gb of memory 2 years before because it was so damn cheap. Never used that mutch, but it was almost for free :) The advantage to ext4 for me is the build in raid1 and the snapshots. I''m using the snapshot feature for my local backups. I like it because it''s really easy and uses very few storage. A simple "Snapshot -> Rsync to a different disk -> Snapshot" script is the perfect local backup method. I really appreciate the work of the developers! Btrfs is great and I''m 110% sure it will become better and better over the next month. Best regards, Felix On 9/7/11 4:15 PM, Swâmi Petaramesh wrote:> Le Mercredi 7 Septembre 2011 00:11:25 vous avez écrit : >> Reading your post, at this point I''d actually recommend you stick with >> ext4. > > I actually shifted back from BTRFS to ext4 and fell like having offered myself > a brand new computer, about 20 times faster, me happy ;-) > >> Both btrfs and zfs are great, but IMHO btrfs is not ready for >> daily use by "ordinary" user yet, while zfs is a memory hog >> (especially for laptops, which is part of the reason why I''m using >> btrfs instead of zfs on this one). > > True, ZFS is excellent but a memory hog (and strongly advises using a 64-bit > OS) but I was surprised to discover that BTRFS was such a memory eater itself, > with kernel 3.0. My system was swapping like mad ! > > I use (kernel) ZFS on my 64-bit main machine and I''m plain happy with it, and > tried ZFS on my 32-bit laptop in the hope to get more performance for less > memory ; alas I just got a memory-hungry system running damn slow... For the > time being I will stick to ZFS for 64-bit machines with>= 4GB RAM, and to > ext4 for 32-bit systems with less RAM... > > I don''t feel that BTRFS gives any advantage in its current state of > development. Alas. >-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, Sep 16, 2011 at 2:37 AM, Felix Blanke <felixblanke@gmail.com> wrote:> I''m using btrfs since one year now and it''s quite fast. I don''t feel any > differences to other filesystems. Never tried a benchmark but for my daily > work it''s nice.Your workload must be light :)> I also never had any issues with the memory. Imho nowadays > memory isn''t a problem at all in desktop computers. I bought 8gb of memory 2 > years before because it was so damn cheap. Never used that mutch, but it was > almost for free :)It might not be possible in some situations. For example, if: - adding memory by yourself means voiding warranty (cause there''s a warranty sticker covering memory slot cover screw), while going to official service shop means 1 full day wasted plus inflated price - you''re using $300 netbook with 1 or 2 GB RAM with only one memory slot> > The advantage to ext4 for me is the build in raid1 and the snapshots. I''m > using the snapshot feature for my local backups. I like it because it''s > really easy and uses very few storage. A simple "Snapshot -> Rsync to a > different disk -> Snapshot" script is the perfect local backup method. >you''ve never used zfs have you :) For that purpose, think "same feature as btrfs snapshot + rsync" but without needing rsync. This can be very useful as the process of rsync determining what data to transfer can be quite CPU/disk intensive.> I really appreciate the work of the developers! Btrfs is great and I''m 110% > sure it will become better and better over the next month.btrfs is great, but not quite production-ready yet, with lack of working fsck being the greatest hurdle. I''m not complaining much about the speed at this point though, being still in dev and all, and since new features will have some performance cost. -- Fajar -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, 2011-09-16 at 05:16 +0700, Fajar A. Nugraha wrote:> On Fri, Sep 16, 2011 at 2:37 AM, Felix Blanke <felixblanke@gmail.com> wrote: > > I''m using btrfs since one year now and it''s quite fast. I don''t feel any > > differences to other filesystems. Never tried a benchmark but for my daily > > work it''s nice. > > Your workload must be light :) >I recently repeatedly rsync whole partitions (>30GB) without ill effects. (ok - first sync took whole 1s).> > > > The advantage to ext4 for me is the build in raid1 and the snapshots. I''m > > using the snapshot feature for my local backups. I like it because it''s > > really easy and uses very few storage. A simple "Snapshot -> Rsync to a > > different disk -> Snapshot" script is the perfect local backup method. > > > > you''ve never used zfs have you :) > > For that purpose, think "same feature as btrfs snapshot + rsync" but > without needing rsync. This can be very useful as the process of rsync > determining what data to transfer can be quite CPU/disk intensive.Now I''m curious - how do zsf get data off the partition without rsync? Regards
On Fri, Sep 16, 2011 at 1:21 PM, Maciej Marcin Piechotka <uzytkownik2@gmail.com> wrote:> On Fri, 2011-09-16 at 05:16 +0700, Fajar A. Nugraha wrote: >> On Fri, Sep 16, 2011 at 2:37 AM, Felix Blanke <felixblanke@gmail.com> wrote: >> > I''m using btrfs since one year now and it''s quite fast. I don''t feel any >> > differences to other filesystems. Never tried a benchmark but for my daily >> > work it''s nice. >> >> Your workload must be light :) >> > > I recently repeatedly rsync whole partitions (>30GB) without ill > effects. (ok - first sync took whole 1s).Wait, you mean you sync 30GB data to another partition in one second? It should not be possible for single HDD no matter what the filesystem is. Unless you''re using SSD or many HDD in raid. That''s how I''m using btrfs btw, on SSD, so I pretty much don''t see the "slowness" since the SSD is blazingly fast. Add to that the fact btrfs has the features I need (compression, snapshot), and more memory-efficient (compared to zfs), it''s suitable for my needs.> >> > >> > The advantage to ext4 for me is the build in raid1 and the snapshots. I''m >> > using the snapshot feature for my local backups. I like it because it''s >> > really easy and uses very few storage. A simple "Snapshot -> Rsync to a >> > different disk -> Snapshot" script is the perfect local backup method. >> > >> >> you''ve never used zfs have you :) >> >> For that purpose, think "same feature as btrfs snapshot + rsync" but >> without needing rsync. This can be very useful as the process of rsync >> determining what data to transfer can be quite CPU/disk intensive. > > Now I''m curious - how do zsf get data off the partition without rsync?First hit on Google: http://www.markround.com/archives/38-ZFS-Replication.html As additinal info, zfs send/receive stream is at DMU layer, which (in over-simplified term) is similar to raw disk blocks in normal partition+ext4 setup. zfs keeps track of which blocks are used, so when given two different snapshot, it can easily find out which blocks are different. When using incremental send zfs only has to send those blocks. It doesn''t have to explicitly re-examine which parts of the file is unmodified (thus not wasting disk, CPU, and network the way rsync does). IIRC there was a proposal in this list sometime ago on implementing similar functionality (send/receive) in btrfs. No actual working code (yet) that I know of though. -- Fajar -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, 2011-09-16 at 13:42 +0700, Fajar A. Nugraha wrote:> On Fri, Sep 16, 2011 at 1:21 PM, Maciej Marcin Piechotka > <uzytkownik2@gmail.com> wrote: > > On Fri, 2011-09-16 at 05:16 +0700, Fajar A. Nugraha wrote: > >> On Fri, Sep 16, 2011 at 2:37 AM, Felix Blanke <felixblanke@gmail.com> wrote: > >> > I''m using btrfs since one year now and it''s quite fast. I don''t feel any > >> > differences to other filesystems. Never tried a benchmark but for my daily > >> > work it''s nice. > >> > >> Your workload must be light :) > >> > > > > I recently repeatedly rsync whole partitions (>30GB) without ill > > effects. (ok - first sync took whole 1s). > > Wait, you mean you sync 30GB data to another partition in one second? > It should not be possible for single HDD no matter what the filesystem > is. Unless you''re using SSD or many HDD in raid. >You misparse the sentence. First sync took 1s after rsync and next were much faster. Regards