# for i in `seq 1 20`; do btrfsck /dev/sde|grep "checksum verify failed";echo; done checksum verify failed on 31945617408 wanted 6607632D found 9CC3ED0 checksum verify failed on 31945617408 wanted 6607632D found AFBBF41 checksum verify failed on 31945617408 wanted 6607632D found A4F0ED0 checksum verify failed on 31945617408 wanted 6607632D found B7E8F41 checksum verify failed on 31945617408 wanted 6607632D found A83DED0 checksum verify failed on 31945617408 wanted 6607632D found BB35F41 checksum verify failed on 31945617408 wanted 6607632D found A0D5ED0 checksum verify failed on 31945617408 wanted 6607632D found B3CDF41 checksum verify failed on 31945617408 wanted 6607632D found A37BED0 checksum verify failed on 31945617408 wanted 6607632D found B673F41 checksum verify failed on 31945617408 wanted 6607632D found AB7FED0 checksum verify failed on 31945617408 wanted 6607632D found BE77F41 checksum verify failed on 31945617408 wanted 6607632D found AB73ED0 checksum verify failed on 31945617408 wanted 6607632D found BE6BF41 checksum verify failed on 31945617408 wanted 6607632D found A76DED0 checksum verify failed on 31945617408 wanted 6607632D found BA65F41 checksum verify failed on 31945617408 wanted 6607632D found 98E6ED0 checksum verify failed on 31945617408 wanted 6607632D found ABDEF41 checksum verify failed on 31945617408 wanted 6607632D found 9038ED0 checksum verify failed on 31945617408 wanted 6607632D found A330F41 checksum verify failed on 31945617408 wanted 6607632D found A150ED0 checksum verify failed on 31945617408 wanted 6607632D found B448F41 checksum verify failed on 31945617408 wanted 6607632D found 916CED0 checksum verify failed on 31945617408 wanted 6607632D found A464F41 checksum verify failed on 31945617408 wanted 6607632D found 9277ED0 checksum verify failed on 31945617408 wanted 6607632D found A56FF41 checksum verify failed on 31945617408 wanted 6607632D found A9C4ED0 checksum verify failed on 31945617408 wanted 6607632D found BCBCF41 checksum verify failed on 31945617408 wanted 6607632D found A81CED0 checksum verify failed on 31945617408 wanted 6607632D found BB14F41 checksum verify failed on 31945617408 wanted 6607632D found A886ED0 checksum verify failed on 31945617408 wanted 6607632D found BB7EF41 checksum verify failed on 31945617408 wanted 6607632D found 944AED0 checksum verify failed on 31945617408 wanted 6607632D found A742F41 It looks like 10XX XXXXXXXX XXXX1110 11010000 and 10XX XXXXXXXX XXXX1111 01000001 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Nov 12, 2009 at 05:08:53AM +0300, Alexander Beregalov wrote:> # for i in `seq 1 20`; do btrfsck /dev/sde|grep "checksum verify > failed";echo; done > checksum verify failed on 31945617408 wanted 6607632D found 9CC3ED0 > checksum verify failed on 31945617408 wanted 6607632D found AFBBF41Was your filesystem mounted at the time? It''s strange that the checksum keeps changing, that points to the data in the block changing. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
2009/11/12 Chris Mason <chris.mason@oracle.com>:> On Thu, Nov 12, 2009 at 05:08:53AM +0300, Alexander Beregalov wrote: >> # for i in `seq 1 20`; do btrfsck /dev/sde|grep "checksum verify >> failed";echo; done >> checksum verify failed on 31945617408 wanted 6607632D found 9CC3ED0 >> checksum verify failed on 31945617408 wanted 6607632D found AFBBF41 > > Was your filesystem mounted at the time? It''s strange that the checksum > keeps changing, that points to the data in the block changing.It was not. Any tips how I can try to find a root cause? It is raid0 of two disks. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Nov 12, 2009 at 05:16:34PM +0300, Alexander Beregalov wrote:> 2009/11/12 Chris Mason <chris.mason@oracle.com>: > > On Thu, Nov 12, 2009 at 05:08:53AM +0300, Alexander Beregalov wrote: > >> # for i in `seq 1 20`; do btrfsck /dev/sde|grep "checksum verify > >> failed";echo; done > >> checksum verify failed on 31945617408 wanted 6607632D found 9CC3ED0 > >> checksum verify failed on 31945617408 wanted 6607632D found AFBBF41 > > > > Was your filesystem mounted at the time? It''s strange that the checksum > > keeps changing, that points to the data in the block changing. > > It was not. > Any tips how I can try to find a root cause? > It is raid0 of two disks.Well, I''d start by looking for anything else that could be touching the FS or the devices. You shouldn''t see more than two different csums in this configuration (one from each disk). Somehow the disk is giving us different answers each time, which points to a problem in the storage stack. Otherwise it is possible that block 31945617408 is multiply linked and is actually in use somehow else. btrfsck is a readonly operation though, it doesn''t change the FS while it is running. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
2009/11/12 Chris Mason <chris.mason@oracle.com>:> On Thu, Nov 12, 2009 at 05:16:34PM +0300, Alexander Beregalov wrote: >> 2009/11/12 Chris Mason <chris.mason@oracle.com>: >> > On Thu, Nov 12, 2009 at 05:08:53AM +0300, Alexander Beregalov wrote: >> >> # for i in `seq 1 20`; do btrfsck /dev/sde|grep "checksum verify >> >> failed";echo; done >> >> checksum verify failed on 31945617408 wanted 6607632D found 9CC3ED0 >> >> checksum verify failed on 31945617408 wanted 6607632D found AFBBF41 >> > >> > Was your filesystem mounted at the time? It''s strange that the checksum >> > keeps changing, that points to the data in the block changing. >> >> It was not. >> Any tips how I can try to find a root cause? >> It is raid0 of two disks. > > Well, I''d start by looking for anything else that could be touching the > FS or the devices. You shouldn''t see more than two different csums in this > configuration (one from each disk).I always see only these two blocks in output.> > Somehow the disk is giving us different answers each time, which points > to a problem in the storage stack.Hm, I will try to check discs by "badblocks" with few cycles of write-read.> > Otherwise it is possible that block 31945617408 is multiply linked and > is actually in use somehow else. btrfsck is a readonly operation > though, it doesn''t change the FS while it is running.Can it be a bug in the filesystem itself - in checksum calculation code, wrong pointer to data or something else? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Nov 12, 2009 at 06:43:00PM +0300, Alexander Beregalov wrote:> 2009/11/12 Chris Mason <chris.mason@oracle.com>: > > On Thu, Nov 12, 2009 at 05:16:34PM +0300, Alexander Beregalov wrote: > >> 2009/11/12 Chris Mason <chris.mason@oracle.com>: > >> > On Thu, Nov 12, 2009 at 05:08:53AM +0300, Alexander Beregalov wrote: > >> >> # for i in `seq 1 20`; do btrfsck /dev/sde|grep "checksum verify > >> >> failed";echo; done > >> >> checksum verify failed on 31945617408 wanted 6607632D found 9CC3ED0 > >> >> checksum verify failed on 31945617408 wanted 6607632D found AFBBF41 > >> > > >> > Was your filesystem mounted at the time? It''s strange that the checksum > >> > keeps changing, that points to the data in the block changing. > >> > >> It was not. > >> Any tips how I can try to find a root cause? > >> It is raid0 of two disks. > > > > Well, I''d start by looking for anything else that could be touching the > > FS or the devices. You shouldn''t see more than two different csums in this > > configuration (one from each disk). > I always see only these two blocks in output.The wanted is always the same but the found is always different?> > > > Somehow the disk is giving us different answers each time, which points > > to a problem in the storage stack. > Hm, I will try to check discs by "badblocks" with few cycles of write-read. > > > > Otherwise it is possible that block 31945617408 is multiply linked and > > is actually in use somehow else. btrfsck is a readonly operation > > though, it doesn''t change the FS while it is running. > > Can it be a bug in the filesystem itself - in checksum calculation > code, wrong pointer to data or something else?We can check, I''ll make a program to map a logical block number to the physical sector. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
2009/11/12 Chris Mason <chris.mason@oracle.com>:> On Thu, Nov 12, 2009 at 06:43:00PM +0300, Alexander Beregalov wrote: >> 2009/11/12 Chris Mason <chris.mason@oracle.com>: >> > On Thu, Nov 12, 2009 at 05:16:34PM +0300, Alexander Beregalov wrote: >> >> 2009/11/12 Chris Mason <chris.mason@oracle.com>: >> >> > On Thu, Nov 12, 2009 at 05:08:53AM +0300, Alexander Beregalov wrote: >> >> >> # for i in `seq 1 20`; do btrfsck /dev/sde|grep "checksum verify >> >> >> failed";echo; done >> >> >> checksum verify failed on 31945617408 wanted 6607632D found 9CC3ED0 >> >> >> checksum verify failed on 31945617408 wanted 6607632D found AFBBF41 >> >> > >> >> > Was your filesystem mounted at the time? It''s strange that the checksum >> >> > keeps changing, that points to the data in the block changing. >> >> >> >> It was not. >> >> Any tips how I can try to find a root cause? >> >> It is raid0 of two disks. >> > >> > Well, I''d start by looking for anything else that could be touching the >> > FS or the devices. You shouldn''t see more than two different csums in this >> > configuration (one from each disk). >> I always see only these two blocks in output. > > The wanted is always the same but the found is always different?Yes, exactly. The bits marked as X look as random.> >> > >> > Somehow the disk is giving us different answers each time, which points >> > to a problem in the storage stack. >> Hm, I will try to check discs by "badblocks" with few cycles of write-read. >> > >> > Otherwise it is possible that block 31945617408 is multiply linked and >> > is actually in use somehow else. btrfsck is a readonly operation >> > though, it doesn''t change the FS while it is running. >> >> Can it be a bug in the filesystem itself - in checksum calculation >> code, wrong pointer to data or something else? > > We can check, I''ll make a program to map a logical block number to the > physical sector.Ok, thanks, I will not reboot it. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Nov 12, 2009 at 08:41:44PM +0300, Alexander Beregalov wrote:> 2009/11/12 Chris Mason <chris.mason@oracle.com>: > > On Thu, Nov 12, 2009 at 06:43:00PM +0300, Alexander Beregalov wrote: > >> 2009/11/12 Chris Mason <chris.mason@oracle.com>: > >> > On Thu, Nov 12, 2009 at 05:16:34PM +0300, Alexander Beregalov wrote: > >> >> 2009/11/12 Chris Mason <chris.mason@oracle.com>: > >> >> > On Thu, Nov 12, 2009 at 05:08:53AM +0300, Alexander Beregalov wrote: > >> >> >> # for i in `seq 1 20`; do btrfsck /dev/sde|grep "checksum verify > >> >> >> failed";echo; done > >> >> >> checksum verify failed on 31945617408 wanted 6607632D found 9CC3ED0 > >> >> >> checksum verify failed on 31945617408 wanted 6607632D found AFBBF41 > >> >> > > >> >> > Was your filesystem mounted at the time? It''s strange that the checksum > >> >> > keeps changing, that points to the data in the block changing. > >> >> > >> >> It was not. > >> >> Any tips how I can try to find a root cause? > >> >> It is raid0 of two disks. > >> > > >> > Well, I''d start by looking for anything else that could be touching the > >> > FS or the devices. You shouldn''t see more than two different csums in this > >> > configuration (one from each disk). > >> I always see only these two blocks in output. > > > > The wanted is always the same but the found is always different? > > Yes, exactly. The bits marked as X look as random.Ok, I''ve added a new tool to the btrfs-unstable-progs repo. It is called btrfs-map-logical and it will print the logical->physical mapping for any metadata or data block in the FS. The FS should not be mounted when you run this program...it''ll mostly work mounted but the results won''t be 100% accurate. For your case: btrfs-map-logical -l 31945617408 -c 1 -o mirror1 /dev/sde btrfs-map-logical -l 31945617408 -c 2 -o mirror2 /dev/sde This will print to the screen the mappings for that block and also save the contents of the first mirror of the block to a file called mirror1. The second command will save the contents of the second mirror. If the FS isn''t mounted and isn''t being changed by someone else, you should get exactly the same mappings every time and exactly the same contents in the files mirror1 and mirror2 every time. Just let me know if you have any problems. The most recent commit in the btrfs-progs repo is: commit ab8fb4c99516c186641bda1dbc0e788f68b4dc77 Author: Chris Mason <chris.mason@oracle.com> Date: Thu Nov 12 14:34:09 2009 -0500 Add btrfs-map-logical program to map and read logical block numbers If you check kernel.org and this commit isn''t there, just try again ;) The mirror takes a few minutes to update. I''m pretty sure it goes faster if you hit reload over and over again, but that''s just a guess. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
2009/11/12 Chris Mason <chris.mason@oracle.com>:> On Thu, Nov 12, 2009 at 08:41:44PM +0300, Alexander Beregalov wrote: >> 2009/11/12 Chris Mason <chris.mason@oracle.com>: >> > On Thu, Nov 12, 2009 at 06:43:00PM +0300, Alexander Beregalov wrote: >> >> 2009/11/12 Chris Mason <chris.mason@oracle.com>: >> >> > On Thu, Nov 12, 2009 at 05:16:34PM +0300, Alexander Beregalov wrote: >> >> >> 2009/11/12 Chris Mason <chris.mason@oracle.com>: >> >> >> > On Thu, Nov 12, 2009 at 05:08:53AM +0300, Alexander Beregalov wrote: >> >> >> >> # for i in `seq 1 20`; do btrfsck /dev/sde|grep "checksum verify >> >> >> >> failed";echo; done >> >> >> >> checksum verify failed on 31945617408 wanted 6607632D found 9CC3ED0 >> >> >> >> checksum verify failed on 31945617408 wanted 6607632D found AFBBF41 >> >> >> > >> >> >> > Was your filesystem mounted at the time? It''s strange that the checksum >> >> >> > keeps changing, that points to the data in the block changing. >> >> >> >> >> >> It was not. >> >> >> Any tips how I can try to find a root cause? >> >> >> It is raid0 of two disks. >> >> > >> >> > Well, I''d start by looking for anything else that could be touching the >> >> > FS or the devices. You shouldn''t see more than two different csums in this >> >> > configuration (one from each disk). >> >> I always see only these two blocks in output. >> > >> > The wanted is always the same but the found is always different? >> >> Yes, exactly. The bits marked as X look as random. > > Ok, I''ve added a new tool to the btrfs-unstable-progs repo. It is > called btrfs-map-logical and it will print the logical->physical mapping > for any metadata or data block in the FS. The FS should not be mounted > when you run this program...it''ll mostly work mounted but the results > won''t be 100% accurate. > > For your case: > > btrfs-map-logical -l 31945617408 -c 1 -o mirror1 /dev/sde > btrfs-map-logical -l 31945617408 -c 2 -o mirror2 /dev/sde > > This will print to the screen the mappings for that block and also save > the contents of the first mirror of the block to a file called mirror1. > The second command will save the contents of the second mirror. > > If the FS isn''t mounted and isn''t being changed by someone else, you > should get exactly the same mappings every time and exactly the same > contents in the files mirror1 and mirror2 every time. > > Just let me know if you have any problems. The most recent commit in > the btrfs-progs repo is: > > commit ab8fb4c99516c186641bda1dbc0e788f68b4dc77 > Author: Chris Mason <chris.mason@oracle.com> > Date: Thu Nov 12 14:34:09 2009 -0500 > > Add btrfs-map-logical program to map and read logical block numbers > > If you check kernel.org and this commit isn''t there, just try again ;) > The mirror takes a few minutes to update. I''m pretty sure it goes > faster if you hit reload over and over again, but that''s just a guess. > > -chris >sh-4.0$ sudo ./btrfs-map-logical -l 31945617408 -c 1 -o /tmp/mirror1 /dev/sde; md5sum /tmp/mirror1 mirror 1 logical 31945617408 physical 16913231872 device /dev/sde mirror 2 logical 31945617408 physical 16893308928 device /dev/sdf bcce5ab8cc65c948650376e936952547 /tmp/mirror1 sh-4.0$ sudo ./btrfs-map-logical -l 31945617408 -c 2 -o /tmp/mirror2 /dev/sde; md5sum /tmp/mirror2 mirror 1 logical 31945617408 physical 16913231872 device /dev/sde mirror 2 logical 31945617408 physical 16893308928 device /dev/sdf 347034331d8879096e262a736fd02ba0 /tmp/mirror2 sh-4.0$ sudo ./btrfs-map-logical -l 31945617408 -c 1 -o /tmp/mirror1 /dev/sde; md5sum /tmp/mirror1 mirror 1 logical 31945617408 physical 16913231872 device /dev/sde mirror 2 logical 31945617408 physical 16893308928 device /dev/sdf bcce5ab8cc65c948650376e936952547 /tmp/mirror1 sh-4.0$ sudo ./btrfs-map-logical -l 31945617408 -c 2 -o /tmp/mirror2 /dev/sde; md5sum /tmp/mirror2 mirror 1 logical 31945617408 physical 16913231872 device /dev/sde mirror 2 logical 31945617408 physical 16893308928 device /dev/sdf 347034331d8879096e262a736fd02ba0 /tmp/mirror2 Does that mean that the problem is not in FS and I need to check discs, IO controller, memory. The same result with mount FS, but when I unmounted it there was a bug: BUG: unable to handle kernel paging request at 6b6b6c9f IP: [<f9a3c696>] caching_kthread+0x286/0x410 [btrfs] *pde = 00000000 Oops: 0002 [#1] last sysfs file: /sys/devices/system/cpu/cpu0/cpufreq/scaling_setspeed Modules linked in: hwmon_vid btrfs zlib_deflate zlib_inflate sata_sil i2c_nforce2 Pid: 15258, comm: btrfs-cache-311 Not tainted (2.6.32-rc6-00346-gaa021ba #4) EIP: 0060:[<f9a3c696>] EFLAGS: 00010286 CPU: 0 EIP is at caching_kthread+0x286/0x410 [btrfs] EAX: 6b6b6b6b EBX: f6f9e098 ECX: 0000000e EDX: effc5f00 ESI: f4d95ab0 EDI: f26f4000 EBP: e84e7f98 ESP: e84e7f20 DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 Process btrfs-cache-311 (pid: 15258, ti=e84e6000 task=f6450000 task.ti=e84e6000) Stack: 00000000 ffffffff 81c00000 00000007 4324c000 effc5508 effc54b8 f26f5bd0 <0> f62c20c0 000b6000 00000000 effc54b0 ffffffff ffffffff 00000032 e7b14678 <0> f26f4000 f4d95ab0 c0007f98 00074324 00a80000 00000010 00000000 074324d0 Call Trace: [<f9a3c410>] ? caching_kthread+0x0/0x410 [btrfs] [<c103ce4c>] ? kthread+0x6c/0x80 [<c103cde0>] ? kthread+0x0/0x80 [<c1003513>] ? kernel_thread_helper+0x7/0x14 Code: c7 8b 45 9c b9 01 00 00 00 ba 03 00 00 00 c7 04 24 00 00 00 00 e8 fb 4d 5e c7 8b 45 b4 e8 53 d5 ff ff 8b 75 cc 8b 86 a0 00 00 00 <ff> 88 34 01 00 00 31 c0 83 c4 6c 5b 5e 5f c9 c3 66 90 77 09 3b EIP: [<f9a3c696>] caching_kthread+0x286/0x410 [btrfs] SS:ESP 0068:e84e7f20 CR2: 000000006b6b6c9f ---[ end trace 72d6516d4c697ae6 ]--- device: ''btrfs-4'': device_unregister device: ''btrfs-4'': device_create_release -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, Nov 13, 2009 at 12:44:47AM +0300, Alexander Beregalov wrote:> 2009/11/12 Chris Mason <chris.mason@oracle.com>: > > On Thu, Nov 12, 2009 at 08:41:44PM +0300, Alexander Beregalov wrote: > >> 2009/11/12 Chris Mason <chris.mason@oracle.com>: > >> > On Thu, Nov 12, 2009 at 06:43:00PM +0300, Alexander Beregalov wrote: > >> >> 2009/11/12 Chris Mason <chris.mason@oracle.com>: > >> >> > On Thu, Nov 12, 2009 at 05:16:34PM +0300, Alexander Beregalov wrote: > >> >> >> 2009/11/12 Chris Mason <chris.mason@oracle.com>: > >> >> >> > On Thu, Nov 12, 2009 at 05:08:53AM +0300, Alexander Beregalov wrote: > >> >> >> >> # for i in `seq 1 20`; do btrfsck /dev/sde|grep "checksum verify > >> >> >> >> failed";echo; done > >> >> >> >> checksum verify failed on 31945617408 wanted 6607632D found 9CC3ED0 > >> >> >> >> checksum verify failed on 31945617408 wanted 6607632D found AFBBF41 > >> >> >> > > >> >> >> > Was your filesystem mounted at the time? It''s strange that the checksum > >> >> >> > keeps changing, that points to the data in the block changing. > >> >> >> > >> >> >> It was not. > >> >> >> Any tips how I can try to find a root cause? > >> >> >> It is raid0 of two disks. > >> >> > > >> >> > Well, I''d start by looking for anything else that could be touching the > >> >> > FS or the devices. You shouldn''t see more than two different csums in this > >> >> > configuration (one from each disk). > >> >> I always see only these two blocks in output. > >> > > >> > The wanted is always the same but the found is always different? > >> > >> Yes, exactly. The bits marked as X look as random. > > > > Ok, I''ve added a new tool to the btrfs-unstable-progs repo. It is > > called btrfs-map-logical and it will print the logical->physical mapping > > for any metadata or data block in the FS. The FS should not be mounted > > when you run this program...it''ll mostly work mounted but the results > > won''t be 100% accurate. > > > > For your case: > > > > btrfs-map-logical -l 31945617408 -c 1 -o mirror1 /dev/sde > > btrfs-map-logical -l 31945617408 -c 2 -o mirror2 /dev/sde > > > > This will print to the screen the mappings for that block and also save > > the contents of the first mirror of the block to a file called mirror1. > > The second command will save the contents of the second mirror. > > > > If the FS isn''t mounted and isn''t being changed by someone else, you > > should get exactly the same mappings every time and exactly the same > > contents in the files mirror1 and mirror2 every time. > > > > Just let me know if you have any problems. The most recent commit in > > the btrfs-progs repo is: > > > > commit ab8fb4c99516c186641bda1dbc0e788f68b4dc77 > > Author: Chris Mason <chris.mason@oracle.com> > > Date: Thu Nov 12 14:34:09 2009 -0500 > > > > Add btrfs-map-logical program to map and read logical block numbers > > > > If you check kernel.org and this commit isn''t there, just try again ;) > > The mirror takes a few minutes to update. I''m pretty sure it goes > > faster if you hit reload over and over again, but that''s just a guess. > > > > -chris > > > > sh-4.0$ sudo ./btrfs-map-logical -l 31945617408 -c 1 -o /tmp/mirror1 > /dev/sde; md5sum /tmp/mirror1 > mirror 1 logical 31945617408 physical 16913231872 device /dev/sde > mirror 2 logical 31945617408 physical 16893308928 device /dev/sdf > bcce5ab8cc65c948650376e936952547 /tmp/mirror1 > > sh-4.0$ sudo ./btrfs-map-logical -l 31945617408 -c 2 -o /tmp/mirror2 > /dev/sde; md5sum /tmp/mirror2 > mirror 1 logical 31945617408 physical 16913231872 device /dev/sde > mirror 2 logical 31945617408 physical 16893308928 device /dev/sdf > 347034331d8879096e262a736fd02ba0 /tmp/mirror2 > > sh-4.0$ sudo ./btrfs-map-logical -l 31945617408 -c 1 -o /tmp/mirror1 > /dev/sde; md5sum /tmp/mirror1 > mirror 1 logical 31945617408 physical 16913231872 device /dev/sde > mirror 2 logical 31945617408 physical 16893308928 device /dev/sdf > bcce5ab8cc65c948650376e936952547 /tmp/mirror1 > > sh-4.0$ sudo ./btrfs-map-logical -l 31945617408 -c 2 -o /tmp/mirror2 > /dev/sde; md5sum /tmp/mirror2 > mirror 1 logical 31945617408 physical 16913231872 device /dev/sde > mirror 2 logical 31945617408 physical 16893308928 device /dev/sdf > 347034331d8879096e262a736fd02ba0 /tmp/mirror2You''re getting different answers for the two blocks, which is ok (one could be corrupt). What doesn''t make sense is that btrfsck isn''t dealing with this correctly. When you read both copies again, you get the same answers as the first time, so this looks like a btrfsck bug to me. I''ll try to reproduce. If you''re in a hurry, you could do something like this (with the FS unmounted). Keep a good copy of both mirror1 and mirror2 dd if=/tmp/mirror1 of=/dev/sde bs=1 seek=16913231872 count=4096 dd if=/tmp/mirror1 of=/dev/sdf bs=1 seek=16893308928 count=4096 sync btrfsck. If that doesn''t work: dd if=/tmp/mirror2 of=/dev/sde bs=1 seek=16913231872 count=4096 dd if=/tmp/mirror2 of=/dev/sdf bs=1 seek=16893308928 count=4096 btrfsck again. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html