Hi all, I''m used to snapshots with LVM and I would like to compare them to btrfs. The case I want to compare is the following: At the moment a snapshot is created, no extra space is needed (maybe some metadata overhead) and all data is shared between the original and the snapshot. In LVM, snapshots work at the block-level, so any changes done to the original volume trigger a COW to the snapshot. If LVM is configured to use 4Mb blocks (default), this means that overwriting a 100k file, will lead to 4Mb "snapshot data" to be backed up. A 800Mb file will take around 800Mb. So, for small files (that are not on the same extent/block) this can waste quite some space, while for bigger files, or lots of files "close" to each other, it doesn''t matter much. How is this for btrfs snapshots? Do they work at the file-level? or also at blocks/extents? I mean, does changing a 100k file lead to 100k being snapshotted? What would happen if I have a 20G file (for example a disk image for kvm)? Would minor changes in that file lead to the entire 20G to be COWed/"backed up"? Is there a distinction between data and metadata? Or does touching (ctime/mtime) or visiting (atime) a file cause it to be COWed? Thanks for any info on this. Mathijs -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Jul 14, 2010 at 11:27:39PM +0200, Mathijs Kwik wrote:> Hi all, > > I''m used to snapshots with LVM and I would like to compare them to btrfs. > > The case I want to compare is the following: > At the moment a snapshot is created, no extra space is needed (maybe > some metadata overhead) and all data is shared between the original > and the snapshot. > In LVM, snapshots work at the block-level, so any changes done to the > original volume trigger a COW to the snapshot. > If LVM is configured to use 4Mb blocks (default), this means that > overwriting a 100k file, will lead to 4Mb "snapshot data" to be backed > up. > A 800Mb file will take around 800Mb. > So, for small files (that are not on the same extent/block) this can > waste quite some space, while for bigger files, or lots of files > "close" to each other, it doesn''t matter much. > > How is this for btrfs snapshots? > Do they work at the file-level? or also at blocks/extents? > > I mean, does changing a 100k file lead to 100k being snapshotted?Btrfs CoWs file extents, and files can use only the parts of an extent they need, so a 1-byte change would only require one additional 4K data block. Of course, metadata also needs to be updated, and will require a number of additional blocks.> What would happen if I have a 20G file (for example a disk image for kvm)? > Would minor changes in that file lead to the entire 20G to be COWed/"backed up"?No, only the relevant portion.> Is there a distinction between data and metadata? > Or does touching (ctime/mtime) or visiting (atime) a file cause it to be COWed?Metadata is CoWed separately, so there will still only be one copy of the data.> Thanks for any info on this. > Mathijs-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
thanks, sounds very promising On Thu, Jul 15, 2010 at 12:13 AM, Sean Bartell <wingedtachikoma@gmail.com> wrote:> On Wed, Jul 14, 2010 at 11:27:39PM +0200, Mathijs Kwik wrote: >> Hi all, >> >> I''m used to snapshots with LVM and I would like to compare them to btrfs. >> >> The case I want to compare is the following: >> At the moment a snapshot is created, no extra space is needed (maybe >> some metadata overhead) and all data is shared between the original >> and the snapshot. >> In LVM, snapshots work at the block-level, so any changes done to the >> original volume trigger a COW to the snapshot. >> If LVM is configured to use 4Mb blocks (default), this means that >> overwriting a 100k file, will lead to 4Mb "snapshot data" to be backed >> up. >> A 800Mb file will take around 800Mb. >> So, for small files (that are not on the same extent/block) this can >> waste quite some space, while for bigger files, or lots of files >> "close" to each other, it doesn''t matter much. >> >> How is this for btrfs snapshots? >> Do they work at the file-level? or also at blocks/extents? >> >> I mean, does changing a 100k file lead to 100k being snapshotted? > > Btrfs CoWs file extents, and files can use only the parts of an extent > they need, so a 1-byte change would only require one additional 4K data > block. Of course, metadata also needs to be updated, and will require > a number of additional blocks. > >> What would happen if I have a 20G file (for example a disk image for kvm)? >> Would minor changes in that file lead to the entire 20G to be COWed/"backed up"? > > No, only the relevant portion. > >> Is there a distinction between data and metadata? >> Or does touching (ctime/mtime) or visiting (atime) a file cause it to be COWed? > > Metadata is CoWed separately, so there will still only be one copy of > the data. > >> Thanks for any info on this. >> Mathijs >-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
<remi.urbillac@orange-ftgroup.com>
2010-Jul-15 08:50 UTC
RE: snapshotting - what data gets shared?
Hi, Just to be sure : LVM snapshots are not COW. They are Copy On First Write. The first time you update one block, the original data needs to be stored in the snapshot area. The COW, aka Copy On Write, is not a very "good" name, because it can leads to confusion. In fact, the copy means that if you partialy update -----Message d''origine----- De : linux-btrfs-owner@vger.kernel.org [mailto:linux-btrfs-owner@vger.kernel.org] De la part de Mathijs Kwik Envoyé : jeudi 15 juillet 2010 10:01 À : Mathijs Kwik; linux-btrfs@vger.kernel.org Objet : Re: snapshotting - what data gets shared? thanks, sounds very promising On Thu, Jul 15, 2010 at 12:13 AM, Sean Bartell <wingedtachikoma@gmail.com> wrote:> On Wed, Jul 14, 2010 at 11:27:39PM +0200, Mathijs Kwik wrote: >> Hi all, >> >> I''m used to snapshots with LVM and I would like to compare them to btrfs. >> >> The case I want to compare is the following: >> At the moment a snapshot is created, no extra space is needed (maybe >> some metadata overhead) and all data is shared between the original >> and the snapshot. >> In LVM, snapshots work at the block-level, so any changes done to the >> original volume trigger a COW to the snapshot. >> If LVM is configured to use 4Mb blocks (default), this means that >> overwriting a 100k file, will lead to 4Mb "snapshot data" to be >> backed up. >> A 800Mb file will take around 800Mb. >> So, for small files (that are not on the same extent/block) this can >> waste quite some space, while for bigger files, or lots of files >> "close" to each other, it doesn''t matter much. >> >> How is this for btrfs snapshots? >> Do they work at the file-level? or also at blocks/extents? >> >> I mean, does changing a 100k file lead to 100k being snapshotted? > > Btrfs CoWs file extents, and files can use only the parts of an extent > they need, so a 1-byte change would only require one additional 4K > data block. Of course, metadata also needs to be updated, and will > require a number of additional blocks. > >> What would happen if I have a 20G file (for example a disk image for kvm)? >> Would minor changes in that file lead to the entire 20G to be COWed/"backed up"? > > No, only the relevant portion. > >> Is there a distinction between data and metadata? >> Or does touching (ctime/mtime) or visiting (atime) a file cause it to be COWed? > > Metadata is CoWed separately, so there will still only be one copy of > the data. > >> Thanks for any info on this. >> Mathijs >-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ********************************* This message and any attachments (the "message") are confidential and intended solely for the addressees. Any unauthorised use or dissemination is prohibited. Messages are susceptible to alteration. France Telecom Group shall not be liable for the message if altered, changed or falsified. If you are not the intended addressee of this message, please cancel it immediately and inform the sender. ******************************** -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
<remi.urbillac@orange-ftgroup.com>
2010-Jul-15 08:57 UTC
RE: snapshotting - what data gets shared?
Hi, Just to be sure : LVM snapshots are not COW. They are Copy On First Write (COFW). The first time you update one block, the original data needs to be stored in the dedicated snapshot area. The COW, aka Copy On Write, is not a very "good" name, because it can leads to confusion. In fact, the copy means that if you partialy update one block in the filesystem, you will not update the existing one. You will copy it to a new location, and apply the update before writing data. When cow, when cofw ? - if you can store the snapshot on an other storage space, it is cofw, like snap devices on LVM. - if you have a lower write performance with a snapshot, then again, it is COFW. - on COW, snapshots and live filesystem are always on the same disks/raid/pool... Remi PS : Sorry for the partial send, buggy CTRL key on outlook ^^ .... -----Message d''origine----- De : linux-btrfs-owner@vger.kernel.org [mailto:linux-btrfs-owner@vger.kernel.org] De la part de Mathijs Kwik Envoyé : jeudi 15 juillet 2010 10:01 À : Mathijs Kwik; linux-btrfs@vger.kernel.org Objet : Re: snapshotting - what data gets shared? thanks, sounds very promising On Thu, Jul 15, 2010 at 12:13 AM, Sean Bartell <wingedtachikoma@gmail.com> wrote:> On Wed, Jul 14, 2010 at 11:27:39PM +0200, Mathijs Kwik wrote: >> Hi all, >> >> I''m used to snapshots with LVM and I would like to compare them to btrfs. >> >> The case I want to compare is the following: >> At the moment a snapshot is created, no extra space is needed (maybe >> some metadata overhead) and all data is shared between the original >> and the snapshot. >> In LVM, snapshots work at the block-level, so any changes done to the >> original volume trigger a COW to the snapshot. >> If LVM is configured to use 4Mb blocks (default), this means that >> overwriting a 100k file, will lead to 4Mb "snapshot data" to be >> backed up. >> A 800Mb file will take around 800Mb. >> So, for small files (that are not on the same extent/block) this can >> waste quite some space, while for bigger files, or lots of files >> "close" to each other, it doesn''t matter much. >> >> How is this for btrfs snapshots? >> Do they work at the file-level? or also at blocks/extents? >> >> I mean, does changing a 100k file lead to 100k being snapshotted? > > Btrfs CoWs file extents, and files can use only the parts of an extent > they need, so a 1-byte change would only require one additional 4K > data block. Of course, metadata also needs to be updated, and will > require a number of additional blocks. > >> What would happen if I have a 20G file (for example a disk image for kvm)? >> Would minor changes in that file lead to the entire 20G to be COWed/"backed up"? > > No, only the relevant portion. > >> Is there a distinction between data and metadata? >> Or does touching (ctime/mtime) or visiting (atime) a file cause it to be COWed? > > Metadata is CoWed separately, so there will still only be one copy of > the data. > >> Thanks for any info on this. >> Mathijs >-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ********************************* This message and any attachments (the "message") are confidential and intended solely for the addressees. Any unauthorised use or dissemination is prohibited. Messages are susceptible to alteration. France Telecom Group shall not be liable for the message if altered, changed or falsified. If you are not the intended addressee of this message, please cancel it immediately and inform the sender. ******************************** -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html