Btrfs devel - Jun 2013 - Two identical copies of an image mounted result in changes to both images if only one is modified

Hi,

I''ve observed a rather strange behaviour while trying to mount two
identical copies of the same image to different mount points.
Each modification to one image is also performed in the second one.

Example:
dd if=/dev/sda? of=image1 bs=1M
cp image1 image2
mount -o loop image1 m1
mount -o loop image2 m2

touch m2/hello
ls -la m1  //will now also include a file calles "hello"

Is this behaviour intentional and known or should I create a bug-report?
I''ve deleted quite a bunch of files on my production system because of
this...

Thanks, Clemens
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Thu, Jun 20, 2013 at 3:47 PM, Clemens Eisserer <linuxhippy@gmail.com>
wrote:
> Hi,
>
> I''ve observed a rather strange behaviour while trying to mount two
> identical copies of the same image to different mount points.
> Each modification to one image is also performed in the second one.
>
> Example:
> dd if=/dev/sda? of=image1 bs=1M
> cp image1 image2
> mount -o loop image1 m1
> mount -o loop image2 m2
>
> touch m2/hello
> ls -la m1  //will now also include a file calles "hello"
What do you get if you unmount BOTH m1 and m2, and THEN mount m1
again? Is the file still there?
>
> Is this behaviour intentional and known or should I create a bug-report?
> I''ve deleted quite a bunch of files on my production system
because of this...
I''m pretty sure this is a known behavior in btrfs.
http://markmail.org/message/i522sdkrhlxhw757#query:+page:1+mid:ksdi5d4v26eqgxpi+state:results

-- 
Fajar
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Thu, Jun 20, 2013 at 10:47:53AM +0200, Clemens Eisserer
wrote:
> Hi,
> 
> I''ve observed a rather strange behaviour while trying to mount two
> identical copies of the same image to different mount points.
> Each modification to one image is also performed in the second one.
> 
> Example:
> dd if=/dev/sda? of=image1 bs=1M
> cp image1 image2
> mount -o loop image1 m1
> mount -o loop image2 m2
> 
> touch m2/hello
> ls -la m1  //will now also include a file calles "hello"
> 
> Is this behaviour intentional and known or should I create a bug-report?
   It''s known, and not desired behaviour. The problem is that
you''ve
ended up with two filesystems with the same UUID, and the FS code gets
rather confused about that. The same problem exists with LVM snapshots
(or other block-device-layer copies).

   The solution is a combination of a tool to scan an image and change
the UUID (offline), and of some code in the kernel that detects when
it''s being told about a duplicate image (rather than an additional
device in the same FS). Neither of these has been written yet, I''m
afraid.
> I''ve deleted quite a bunch of files on my production system
because of this...
   Oops. I''m sorry to hear that. :(

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ==  PGP
key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
              --- Welcome to Rivendell,  Mr Anderson... ---

On Thu, 20 Jun 2013 10:16:22 +0100, Hugo Mills wrote:
> On Thu, Jun 20, 2013 at 10:47:53AM +0200, Clemens Eisserer wrote:
>> Hi,
>> 
>> I''ve observed a rather strange behaviour while trying to mount
two
>> identical copies of the same image to different mount points.
>> Each modification to one image is also performed in the second one.
>> touch m2/hello
>> ls -la m1  //will now also include a file calles "hello"
>> 
>> Is this behaviour intentional and known or should I create a
bug-report?
> 
>    It''s known, and not desired behaviour. The problem is that
you''ve
> ended up with two filesystems with the same UUID, and the FS code gets
> rather confused about that. The same problem exists with LVM snapshots
> (or other block-device-layer copies).
> 
>    The solution is a combination of a tool to scan an image and change
> the UUID (offline), and of some code in the kernel that detects when
> it''s being told about a duplicate image (rather than an additional
> device in the same FS). Neither of these has been written yet, I''m
> afraid.
To clarify, the loop devices are properly distinct, but the first
device ends up mounted twice.

I''ve had a look at the vfs code, and it doesn''t seem to be
uuid-aware,
which makes sense because the uuid is a property of the superblock and
the fs structure doesn''t expose it.  It''s a Btrfs problem.

Instead of redirecting to a different block device, Btrfs could and
should refuse to mount an already-mounted superblock when the block
device doesn''t match, somewhere in or below btrfs_mount.  Registering
extra, distinct superblocks for an already mounted raid is a different
matter, but that isn''t done through the mount syscall anyway.
>> I''ve deleted quite a bunch of files on my production system
because of this...
> 
>    Oops. I''m sorry to hear that. :(
> 
>    Hugo.

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Thu, Jun 20, 2013 at 10:22:07AM +0000, Gabriel de Perthuis
wrote:
> On Thu, 20 Jun 2013 10:16:22 +0100, Hugo Mills wrote:
> > On Thu, Jun 20, 2013 at 10:47:53AM +0200, Clemens Eisserer wrote:
> >> Hi,
> >> 
> >> I''ve observed a rather strange behaviour while trying to
mount two
> >> identical copies of the same image to different mount points.
> >> Each modification to one image is also performed in the second
one.
> 
> >> touch m2/hello
> >> ls -la m1  //will now also include a file calles "hello"
> >> 
> >> Is this behaviour intentional and known or should I create a
bug-report?
> > 
> >    It''s known, and not desired behaviour. The problem is that
you''ve
> > ended up with two filesystems with the same UUID, and the FS code gets
> > rather confused about that. The same problem exists with LVM snapshots
> > (or other block-device-layer copies).
> > 
> >    The solution is a combination of a tool to scan an image and change
> > the UUID (offline), and of some code in the kernel that detects when
> > it''s being told about a duplicate image (rather than an
additional
> > device in the same FS). Neither of these has been written yet,
I''m
> > afraid.
> 
> To clarify, the loop devices are properly distinct, but the first
> device ends up mounted twice.
> 
> I''ve had a look at the vfs code, and it doesn''t seem to
be uuid-aware,
> which makes sense because the uuid is a property of the superblock and
> the fs structure doesn''t expose it.  It''s a Btrfs
problem.
   Yes, it is. (I didn''t intend, however obliquely, to imply that it
wasn''t).
> Instead of redirecting to a different block device, Btrfs could and
> should refuse to mount an already-mounted superblock when the block
> device doesn''t match, somewhere in or below btrfs_mount. 
Registering
> extra, distinct superblocks for an already mounted raid is a different
> matter, but that isn''t done through the mount syscall anyway.
   The problem here is that you could quite legitimately mount
/dev/sda (with UUID=AA1234) on, say, /mnt/fs-a, and /dev/sdb (with
UUID=AA1234) on /mnt/fs-b -- _provided_ that /dev/sda and /dev/sdb are
both part of the same filesystem. So you can''t simply prevent mounting
based on the device that the mount''s being done with.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ==  PGP
key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
                   --- I know of three kinds: hot, ---                   
                cool,  and what-time-does-the-tune-start?

>> Instead of redirecting to a different block device, Btrfs could and
>> should refuse to mount an already-mounted superblock when the block
>> device doesn''t match, somewhere in or below btrfs_mount. 
Registering
>> extra, distinct superblocks for an already mounted raid is a different
>> matter, but that isn''t done through the mount syscall anyway.
> 
>    The problem here is that you could quite legitimately mount
> /dev/sda (with UUID=AA1234) on, say, /mnt/fs-a, and /dev/sdb (with
> UUID=AA1234) on /mnt/fs-b -- _provided_ that /dev/sda and /dev/sdb are
> both part of the same filesystem. So you can''t simply prevent
mounting
> based on the device that the mount''s being done with.
Okay.  The check should rely on a list of known block devices
for a given filesystem uuid.


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Thu, Jun 20, 2013 at 10:41:53AM +0000, Gabriel de Perthuis
wrote:
> >> Instead of redirecting to a different block device, Btrfs could
and
> >> should refuse to mount an already-mounted superblock when the
block
> >> device doesn''t match, somewhere in or below btrfs_mount. 
Registering
> >> extra, distinct superblocks for an already mounted raid is a
different
> >> matter, but that isn''t done through the mount syscall
anyway.
> > 
> >    The problem here is that you could quite legitimately mount
> > /dev/sda (with UUID=AA1234) on, say, /mnt/fs-a, and /dev/sdb (with
> > UUID=AA1234) on /mnt/fs-b -- _provided_ that /dev/sda and /dev/sdb are
> > both part of the same filesystem. So you can''t simply prevent
mounting
> > based on the device that the mount''s being done with.
> 
> Okay.  The check should rely on a list of known block devices
> for a given filesystem uuid.
   And this is where we fail currently -- that list is held by the
btrfs module in the kernel, and is constructed on the basis of what
"btrfs dev scan" finds by looking at superblocks on block devices.
Currently, there''s no method implemented for determining whether a
block device with a legitimate btrfs superblock on it is a duplicate
of another device, or whether it''s a newly-discovered device which is
part of an as-yet incompletely specified multi-device FS.

   I think it should be possible to look up the device ID as well, and
complain (loudly, to the user, and in the kernel) at btrfs dev scan
time if we see duplicates. That would deal with the problem at the
earliest point of confusion.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ==  PGP
key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
                   --- I know of three kinds: hot, ---                   
                cool,  and what-time-does-the-tune-start?

Thank you for your reply. I appreciate it. Unfortunately this issue is a deal
killer for us. The ability to take very fast snapshots and replicate them to
another site is key for us. We just can''t us Btrfs with this setup.
That''s too bad. Good luck and thank you.

Sent from my iPhone

On Jun 20, 2013, at 5:56 AM, Hugo Mills <hugo@carfax.org.uk> wrote:
> On Thu, Jun 20, 2013 at 10:41:53AM +0000, Gabriel de Perthuis wrote:
>>>> Instead of redirecting to a different block device, Btrfs could
and
>>>> should refuse to mount an already-mounted superblock when the
block
>>>> device doesn''t match, somewhere in or below
btrfs_mount.  Registering
>>>> extra, distinct superblocks for an already mounted raid is a
different
>>>> matter, but that isn''t done through the mount syscall
anyway.
>>> 
>>>   The problem here is that you could quite legitimately mount
>>> /dev/sda (with UUID=AA1234) on, say, /mnt/fs-a, and /dev/sdb (with
>>> UUID=AA1234) on /mnt/fs-b -- _provided_ that /dev/sda and /dev/sdb
are
>>> both part of the same filesystem. So you can''t simply
prevent mounting
>>> based on the device that the mount''s being done with.
>> 
>> Okay.  The check should rely on a list of known block devices
>> for a given filesystem uuid.
> 
>   And this is where we fail currently -- that list is held by the
> btrfs module in the kernel, and is constructed on the basis of what
> "btrfs dev scan" finds by looking at superblocks on block
devices.
> Currently, there''s no method implemented for determining whether a
> block device with a legitimate btrfs superblock on it is a duplicate
> of another device, or whether it''s a newly-discovered device which
is
> part of an as-yet incompletely specified multi-device FS.
> 
>   I think it should be possible to look up the device ID as well, and
> complain (loudly, to the user, and in the kernel) at btrfs dev scan
> time if we see duplicates. That would deal with the problem at the
> earliest point of confusion.
> 
>   Hugo.
> 
> -- 
> === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk
==>  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
>                   --- I know of three kinds: hot, ---                   
>                cool,  and what-time-does-the-tune-start?                
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Thu, Jun 20, 2013 at 08:22:12AM -0500, Kevin O''Kelley
wrote:
> Thank you for your reply. I appreciate it. Unfortunately this issue
> is a deal killer for us. The ability to take very fast snapshots and
> replicate them to another site is key for us. We just can''t us
Btrfs
> with this setup. That''s too bad. Good luck and thank you.
   If you want to make fast atomic incremental copies of btrfs to a
remote system, then btrfs send/receive may be what you''re looking for.

   Hugo.
> Sent from my iPhone
> 
> On Jun 20, 2013, at 5:56 AM, Hugo Mills <hugo@carfax.org.uk> wrote:
> 
> > On Thu, Jun 20, 2013 at 10:41:53AM +0000, Gabriel de Perthuis wrote:
> >>>> Instead of redirecting to a different block device, Btrfs
could and
> >>>> should refuse to mount an already-mounted superblock when
the block
> >>>> device doesn''t match, somewhere in or below
btrfs_mount.  Registering
> >>>> extra, distinct superblocks for an already mounted raid is
a different
> >>>> matter, but that isn''t done through the mount
syscall anyway.
> >>> 
> >>>   The problem here is that you could quite legitimately mount
> >>> /dev/sda (with UUID=AA1234) on, say, /mnt/fs-a, and /dev/sdb
(with
> >>> UUID=AA1234) on /mnt/fs-b -- _provided_ that /dev/sda and
/dev/sdb are
> >>> both part of the same filesystem. So you can''t simply
prevent mounting
> >>> based on the device that the mount''s being done with.
> >> 
> >> Okay.  The check should rely on a list of known block devices
> >> for a given filesystem uuid.
> > 
> >   And this is where we fail currently -- that list is held by the
> > btrfs module in the kernel, and is constructed on the basis of what
> > "btrfs dev scan" finds by looking at superblocks on block
devices.
> > Currently, there''s no method implemented for determining
whether a
> > block device with a legitimate btrfs superblock on it is a duplicate
> > of another device, or whether it''s a newly-discovered device
which is
> > part of an as-yet incompletely specified multi-device FS.
> > 
> >   I think it should be possible to look up the device ID as well, and
> > complain (loudly, to the user, and in the kernel) at btrfs dev scan
> > time if we see duplicates. That would deal with the problem at the
> > earliest point of confusion.
> > 
> >   Hugo.
> > 
-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ==  PGP
key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
     --- Computer Science is not about computers,  any more than ---     
                     astronomy is about telescopes.

> Thank you for your reply. I appreciate it. Unfortunately this issue is a
deal killer for us. The ability to take very fast snapshots and replicate them
to another site is key for us. We just can''t us Btrfs with this setup.
That''s too bad. Good luck and thank you.
The issue we were discussing is: how to fail early when there are 
duplicate UUIDs.
Duplicate UUIDs will never be supported.
If *your* problem has to do with fast snapshots and fast replication, 
that''s supported, see btrfs send/receive.

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Thu, Jun 20, 2013 at 11:56:07AM +0100, Hugo Mills
wrote:
> On Thu, Jun 20, 2013 at 10:41:53AM +0000, Gabriel de Perthuis wrote:
> > >> Instead of redirecting to a different block device, Btrfs
could and
> > >> should refuse to mount an already-mounted superblock when the
block
> > >> device doesn''t match, somewhere in or below
btrfs_mount.  Registering
> > >> extra, distinct superblocks for an already mounted raid is a
different
> > >> matter, but that isn''t done through the mount
syscall anyway.
> > > 
> > >    The problem here is that you could quite legitimately mount
> > > /dev/sda (with UUID=AA1234) on, say, /mnt/fs-a, and /dev/sdb
(with
> > > UUID=AA1234) on /mnt/fs-b -- _provided_ that /dev/sda and
/dev/sdb are
> > > both part of the same filesystem. So you can''t simply
prevent mounting
> > > based on the device that the mount''s being done with.
> > 
> > Okay.  The check should rely on a list of known block devices
> > for a given filesystem uuid.
> 
>    And this is where we fail currently -- that list is held by the
> btrfs module in the kernel, and is constructed on the basis of what
> "btrfs dev scan" finds by looking at superblocks on block
devices.
> Currently, there''s no method implemented for determining whether a
> block device with a legitimate btrfs superblock on it is a duplicate
> of another device, or whether it''s a newly-discovered device which
is
> part of an as-yet incompletely specified multi-device FS.
> 
>    I think it should be possible to look up the device ID as well, and
> complain (loudly, to the user, and in the kernel) at btrfs dev scan
> time if we see duplicates. That would deal with the problem at the
> earliest point of confusion.
   Incidentally, there''s a conversation on IRC right now about how to
deal with this issue. A solution to prevent the data loss cases may be
arriving some time Real Soon Now™...

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ==  PGP
key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
     --- Computer Science is not about computers,  any more than ---     
                     astronomy is about telescopes.