Hi everyone, I''m curious what would happen in btrfs if the following commands were issued: # cp file1 file2 # chown newuser:newgroup file2 Where file1 was owned by olduser:oldgroup. If I understand copy-on-write correctly the "cp" would merely create a new pointer (or whatever it is called :( ) containing the files'' metadata but the file contents would not actually be duplicated. Then the "chown" would change the metadata so that file2''s metadata indicates newuser:newgroup owned the file contents. At the same time the metadata of file1 would say that olduser:oldgroup owned the file contents. (If a user modified the file contents, the contents would be copied at that time.) Thanks, curious newbie, C. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
CSights wrote:> Hi everyone, > I''m curious what would happen in btrfs if the following commands were issued: > > # cp file1 file2 > # chown newuser:newgroup file2 > > Where file1 was owned by olduser:oldgroup. > > > If I understand copy-on-write correctly the "cp" would merely create a new > pointer (or whatever it is called :( ) containing the files'' metadata but the > file contents would not actually be duplicated.NO. The concept you describe requires "deduplicate". What you are forgetting is that "cp" is a user-space program that reads data and writes it to a new file. The kernel does not see this as a Copy. jim -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Mar 17, 2009 at 1:35 AM, CSights <csights@fastmail.fm> wrote:> Hi everyone, > I''m curious what would happen in btrfs if the following commands were issued: > > # cp file1 file2 > # chown newuser:newgroup file2 > > Where file1 was owned by olduser:oldgroup. > > > If I understand copy-on-write correctly the "cp" would merely create a new > pointer (or whatever it is called :( ) containing the files'' metadata but the > file contents would not actually be duplicated.Certainly not. cp is a userspace program with no knowledge of filesystem details like COW or, as you describe, deduplicate. The semantics you are looking for come with links, except that they don''t. Hard links are implemented at the filesystem level, but they are not copy-on-write at the user space level - if you write to the linked file it will appear via all other links too. Soft links are nothing more than a transparent "shortcut" that happen to give most of the semantics people want from hard links in a much more flexible way. But neither allows a file to have contradictory ownership - you need ACLs to hack up access rights to mimic that. Hard links and soft links make multiple paths refer to the same file, not merely the same contents with different metadata. Otherwise you could soft link /bin/sh into your home directly, setuid the link, and own the machine. Clearly this is not the case, and won''t be with btrfs either. And if you''re not talking about soft or hard links, I''ve no idea how you thought that would work. It is possible for a file system to detect identical blocks between files, but without more guidance, it would be very expensive to do so, and with questionable benefits. -- Dmitri Nikulin Centre for Synchrotron Science Monash University Victoria 3800, Australia -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Mar 17, 2009 at 8:14 AM, Dmitri Nikulin <dnikulin@gmail.com> wrote:> Otherwise you could soft link > /bin/sh into your home directly, setuid the link, and own the machine.Sorry, that was a terrible example, only root can setuid anyway. A better example is linking to /bin/sh and making your link writable, then using that to inject malicious code, which is just as good and would be possible with the semantics you described. -- Dmitri Nikulin Centre for Synchrotron Science Monash University Victoria 3800, Australia -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, 2009-03-16 at 11:28 -0400, jim owens wrote:> CSights wrote: > > Hi everyone, > > I''m curious what would happen in btrfs if the following commands were issued: > > > > # cp file1 file2 > > # chown newuser:newgroup file2 > > > > Where file1 was owned by olduser:oldgroup. > > > > > > If I understand copy-on-write correctly the "cp" would merely create a new > > pointer (or whatever it is called :( ) containing the files'' metadata but the > > file contents would not actually be duplicated. > > NO. The concept you describe requires "deduplicate". > > What you are forgetting is that "cp" is a user-space program that > reads data and writes it to a new file. The kernel does not see > this as a Copy. >This is true, cp makes a full copy of the file. The btrfs utilities include a program called bcp that makes a COW copy of a single file (or directory tree) with a btrfs specific ioctl. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> This is true, cp makes a full copy of the file. > > The btrfs utilities include a program called bcp that makes a COW copy > of a single file (or directory tree) with a btrfs specific ioctl.Okay, what happens if in the original example "cp" is changed to "bcp"? # bcp file1 file2 # chown newuser:newgroup file2 (Where file1 was owned by olduser:oldgroup.) Is using "bcp" the same as copying with hardlinks ("cp -l file1 file2")? Here is an expanded example which is how I imagined COW would handle changes to the file''s data ("file contents"). One can pretend it is an attempt to inject malicious code into /bin/sh (e.g. file1 is /bin/sh). [METADATA] --> DATA [file1 perms olduser:oldgroup] --> file contents # cp file1 file2 [file1 perms olduser:oldgroup "COW"] \ --> file contents [file1 perms olduser:oldgroup "COW"] / (A "COW" flag is set in btrfs''s (hidden) metadata.) # chown newuser:newgroup file2 [file1 perms olduser:oldgroup "COW"] \ --> file contents [file1 perms newuser:newgroup "COW"] / (chown, chmod, others? are not "writes" to file contents, so file contents don''t need to be copied-on-write yet.) # cat newcontent >> file2 [file1 perms olduser:oldgroup] --> file contents [file2 perms newuser:newgroup] --> file contents + newcontent (File contents are modified. This is a "write" that triggers COW. The file contents are copied and then modified. Metadata for file2 are hooked up to copied then modified file contents. "COW" flag is cleared.) Would this work? At least in this example it looks like the filesystem can track whether the file contents should be copied or not without hints from userspace. Thanks, newb. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, 2009-03-17 at 11:34 -0400, CSights wrote:> > This is true, cp makes a full copy of the file. > > > > The btrfs utilities include a program called bcp that makes a COW copy > > of a single file (or directory tree) with a btrfs specific ioctl. > > Okay, what happens if in the original example "cp" is changed to "bcp"? > > # bcp file1 file2 > # chown newuser:newgroup file2 > (Where file1 was owned by olduser:oldgroup.) > > Is using "bcp" the same as copying with hardlinks ("cp -l file1 file2")? >bcp makes a copy of all the metadata, creating a new inode. cp -l creates a second link to the same inode. For bcp, the only thing that is shared between the two files is the actual data extents. The sharing is COW, so any changes to either file will not be reflected in the other file.> > Here is an expanded example which is how I imagined COW would handle changes > to the file''s data ("file contents"). One can pretend it is an attempt to > inject malicious code into /bin/sh (e.g. file1 is /bin/sh). > > [METADATA] --> DATA > [file1 perms olduser:oldgroup] --> file contents > > > # cp file1 file2 > [file1 perms olduser:oldgroup "COW"] \ > --> file contents > [file1 perms olduser:oldgroup "COW"] / > (A "COW" flag is set in btrfs''s (hidden) metadata.) > > > # chown newuser:newgroup file2 > [file1 perms olduser:oldgroup "COW"] \ > --> file contents > [file1 perms newuser:newgroup "COW"] / > (chown, chmod, others? are not "writes" to file contents, so file contents > don''t need to be copied-on-write yet.) > > > # cat newcontent >> file2 > [file1 perms olduser:oldgroup] --> file contents > [file2 perms newuser:newgroup] --> file contents + newcontent > (File contents are modified. This is a "write" that triggers COW. The file > contents are copied and then modified. Metadata for file2 are hooked up to > copied then modified file contents. "COW" flag is cleared.) > > Would this work? At least in this example it looks like the filesystem can > track whether the file contents should be copied or not without hints from > userspace.It would work, but it is slightly different from how btrfs works. There are two ways to read COW (copy on write): 1) Before changing something, make a copy of the old data and put it somewhere else. Then overwrite the original location. 2) Don''t ever overwrite the original location, write somewhere new instead. The old copy stays in the original location. Btrfs does #2. The bcp command creates a second inode that points to the same data extents as the first inode. So, modifications to the inodes themselves (such as chown, chmod, touch etc) don''t touch the data extents. Modifications to the data extents go through the COW mechanism to make sure we don''t overwrite the originals. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi everyone,> > Here is an expanded example which is how I imagined COW would handle > > changes to the file''s data ("file contents"). One can pretend it is an > > attempt to inject malicious code into /bin/sh (e.g. file1 is /bin/sh). > > > > [METADATA] --> DATA > > [file1 perms olduser:oldgroup] --> file contents > > > > > > # cp file1 file2 > > [file1 perms olduser:oldgroup "COW"] \ > > --> file contents > > [file1 perms olduser:oldgroup "COW"] / > > (A "COW" flag is set in btrfs''s (hidden) metadata.) > > > > > > # chown newuser:newgroup file2 > > [file1 perms olduser:oldgroup "COW"] \ > > --> file contents > > [file1 perms newuser:newgroup "COW"] / > > (chown, chmod, others? are not "writes" to file contents, so file > > contents don''t need to be copied-on-write yet.) > > > > > > # cat newcontent >> file2 > > [file1 perms olduser:oldgroup] --> file contents > > [file2 perms newuser:newgroup] --> file contents + newcontent > > (File contents are modified. This is a "write" that triggers COW. The > > file contents are copied and then modified. Metadata for file2 are hooked > > up to copied then modified file contents. "COW" flag is cleared.) > > It would work, but it is slightly different from how btrfs works. There > are two ways to read COW (copy on write): > > 1) Before changing something, make a copy of the old data and put it > somewhere else. Then overwrite the original location. > > 2) Don''t ever overwrite the original location, write somewhere new > instead. The old copy stays in the original location. > > Btrfs does #2.Does the choice #1 or #2 change whether the extended example works or not? It seems as though either way makes sense for the example given...?> The bcp command creates a second inode that points to the same data > extents as the first inode. So, modifications to the inodes themselves > (such as chown, chmod, touch etc) don''t touch the data extents. > > Modifications to the data extents go through the COW mechanism to make > sure we don''t overwrite the originals.To me it sounds like if cp were replaced with bcp, then btrfs would behave as I imagined in my example... Why is a "bcp" separate from cp needed? Is it because with cp btrfs doesn''t "know" a simple copy is being made, but just gets a stream of data to write to disk? Is it possible to update cp to do the btrfs ioctl automatically, or must the commands always remain separate because there are situations where it would be a problem for the file contents to be COW? (It seems to me the fact that the data contents are COW would be transparent to userland apps, so the bcp ioctl could be built in to cp.) Looking forward to (a stable) btrfs! Eager User. :) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, 2009-03-17 at 13:08 -0400, CSights wrote:> Hi everyone, > > > > Here is an expanded example which is how I imagined COW would handle > > > changes to the file''s data ("file contents"). One can pretend it is an > > > attempt to inject malicious code into /bin/sh (e.g. file1 is /bin/sh). > > > > > > [METADATA] --> DATA > > > [file1 perms olduser:oldgroup] --> file contents > > > > > > > > > # cp file1 file2 > > > [file1 perms olduser:oldgroup "COW"] \ > > > --> file contents > > > [file1 perms olduser:oldgroup "COW"] / > > > (A "COW" flag is set in btrfs''s (hidden) metadata.) > > > > > > > > > # chown newuser:newgroup file2 > > > [file1 perms olduser:oldgroup "COW"] \ > > > --> file contents > > > [file1 perms newuser:newgroup "COW"] / > > > (chown, chmod, others? are not "writes" to file contents, so file > > > contents don''t need to be copied-on-write yet.) > > > > > > > > > # cat newcontent >> file2 > > > [file1 perms olduser:oldgroup] --> file contents > > > [file2 perms newuser:newgroup] --> file contents + newcontent > > > (File contents are modified. This is a "write" that triggers COW. The > > > file contents are copied and then modified. Metadata for file2 are hooked > > > up to copied then modified file contents. "COW" flag is cleared.) > > > > It would work, but it is slightly different from how btrfs works. There > > are two ways to read COW (copy on write): > > > > 1) Before changing something, make a copy of the old data and put it > > somewhere else. Then overwrite the original location. > > > > 2) Don''t ever overwrite the original location, write somewhere new > > instead. The old copy stays in the original location. > > > > Btrfs does #2. > > Does the choice #1 or #2 change whether the extended example works or not? > It seems as though either way makes sense for the example given...? >Yes, either way works. #1 is what lvm snapshotting uses, which avoids fragmentation of the original, but it doesn''t scale well to lots of snapshots.> > The bcp command creates a second inode that points to the same data > > extents as the first inode. So, modifications to the inodes themselves > > (such as chown, chmod, touch etc) don''t touch the data extents. > > > > Modifications to the data extents go through the COW mechanism to make > > sure we don''t overwrite the originals. > > To me it sounds like if cp were replaced with bcp, then btrfs would behave as > I imagined in my example...The long term goal is to get cp to use a new system call to cow link files.> Why is a "bcp" separate from cp needed? Is it because with cp btrfs > doesn''t "know" a simple copy is being made, but just gets a stream of data to > write to disk? > Is it possible to update cp to do the btrfs ioctl automatically, or must the > commands always remain separate because there are situations where it would > be a problem for the file contents to be COW? (It seems to me the fact that > the data contents are COW would be transparent to userland apps, so the bcp > ioctl could be built in to cp.) > > Looking forward to (a stable) btrfs! > Eager User. :);) -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html