Hi, I have been having a kernel crash since kernel 3.3.2 and hoped it had been fixed by the time 3.8.7 (FC18) but no joy. I take backups of large image files (around 100GB) and btrfs snapshot them. If I then use rsync --inplace to update the images, I get a btrfs stack trace and btrfs hangs: This happens every night. Here is the stack trace: Any idea what is causing this? Thanks, Mark Apr 22 13:29:29 bkserver kernel: [66320.561671] ------------[ cut here ]---- -------- Apr 22 13:29:29 bkserver kernel: [66320.561818] kernel BUG at fs/btrfs/ ctree.c:2952! Apr 22 13:29:29 bkserver kernel: [66320.561906] invalid opcode: 0000 [#1] SMP Apr 22 13:29:29 bkserver kernel: [66320.562212] Modules linked in: ppdev parport_pc vmxnet3 coretemp vmw_balloon parport shpchp i2c_piix4 microcode btrfs zlib_deflate libcrc32c vmwgfx ttm drm vmw_pvscsi i2c_core Apr 22 13:29:29 bkserver kernel: [66320.562538] CPU 6Apr 22 13:29:29 bkserver kernel: [66320.562605] Pid: 30965, comm: btrfs-endio-wri Not tainted 3.8.7-201.fc18.x86_64 #1 VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform Apr 22 13:29:29 bkserver kernel: [66320.562820] RIP: 0010:[<ffffffffa00bd5c1>] [<ffffffffa00bd5c1>] btrfs_set_item_key_safe+0x141/0x150 [btrfs]Apr 22 13:29:29 bkserver kernel: [66320.563046] RSP: 0018:ffff88014763dae8 EFLAGS: 00010286 Apr 22 13:29:29 bkserver kernel: [66320.563147] RAX: 00000000ffffffff RBX: 000000000000001a RCX: 0000000ce75ee000 Apr 22 13:29:29 bkserver kernel: [66320.563258] RDX: 00000000ffffffff RSI: ffff88014763dc16 RDI: ffff88014763dac7 Apr 22 13:29:29 bkserver kernel: [66320.563372] RBP: ffff88014763db48 R08: 0000000000004b58 R09: 00000ce75e80006c Apr 22 13:29:29 bkserver kernel: [66320.563475] R10: 6c0000000000004b R11: 0000000ce75e8000 R12: ffff88014763db07 Apr 22 13:29:29 bkserver kernel: [66320.563586] R13: ffff880101d151b0 R14: ffff88014763dc16 R15: ffff8800afb69d90 Apr 22 13:29:29 bkserver kernel: [66320.563729] FS: 0000000000000000(0000) GS:ffff8803bfd80000(0000) knlGS:0000000000000000 Apr 22 13:29:29 bkserver kernel: [66320.563842] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b Apr 22 13:29:29 bkserver kernel: [66320.563931] CR2: 0000000000440be0 CR3: 00000003ab6ab000 CR4: 00000000000007e0 Apr 22 13:29:29 bkserver kernel: [66320.564051] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Apr 22 13:29:29 bkserver kernel: [66320.564185] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Apr 22 13:29:29 bkserver kernel: [66320.564298] Process btrfs-endio-wri (pid: 30965, threadinfo ffff88014763c000, task ffff8803ab79aec0) Apr 22 13:29:29 bkserver kernel: [66320.564455] Stack: Apr 22 13:29:29 bkserver kernel: [66320.564558] ffff88014763db48 ffff8800066a9800 ffff880131387000 5800000000000000 Apr 22 13:29:29 bkserver kernel: [66320.564662] 6c0000000000004b 0000000ce75e8000 ffff880100000000 ffff8800afb69d90 Apr 22 13:29:29 bkserver kernel: [66320.564755] ffff880101d151b0 0000000000000a69 0000000000000000 0000000ce75d8000 Apr 22 13:29:29 bkserver kernel: [66320.564865] Call Trace: Apr 22 13:29:29 bkserver kernel: [66320.564948] [<ffffffffa00f1b50>] __btrfs_drop_extents+0x460/0xbc0 [btrfs] Apr 22 13:29:29 bkserver kernel: [66320.565048] [<ffffffffa00cbe46>] ? run_clustered_refs+0x256/0xb50 [btrfs] Apr 22 13:29:29 bkserver kernel: [66320.565143] [<ffffffffa00f740e>] ? alloc_extent_state+0x2e/0xc0 [btrfs] Apr 22 13:29:29 bkserver kernel: [66320.565300] [<ffffffffa00f2d2b>] btrfs_drop_extents+0x6b/0xa0 [btrfs] Apr 22 13:29:29 bkserver kernel: [66320.565406] [<ffffffffa00e31ab>] insert_reserved_file_extent.constprop.58+0x7b/0x2c0 [btrfs] Apr 22 13:29:29 bkserver kernel: [66320.566955] [<ffffffffa00e06b5>] ? start_transaction+0x95/0x460 [btrfs] Apr 22 13:29:29 bkserver kernel: [66320.567792] [<ffffffffa00e9004>] btrfs_finish_ordered_io+0x364/0x3f0 [btrfs] Apr 22 13:29:29 bkserver kernel: [66320.568534] [<ffffffffa00e90a5>] finish_ordered_fn+0x15/0x20 [btrfs] Apr 22 13:29:29 bkserver kernel: [66320.569362] [<ffffffffa0109e56>] worker_loop+0x136/0x580 [btrfs] Apr 22 13:29:29 bkserver kernel: [66320.570101] [<ffffffffa0109d20>] ? btrfs_queue_worker+0x300/0x300 [btrfs] Apr 22 13:29:29 bkserver kernel: [66320.570858] [<ffffffff81081f40>] kthread+0xc0/0xd0 Apr 22 13:29:29 bkserver kernel: [66320.571561] [<ffffffff81088f30>] ? set_security_override_from_ctx+0x50/0x50 Apr 22 13:29:29 bkserver kernel: [66320.572263] [<ffffffff81010000>] ? perf_trace_xen_cpu_write_gdt_entry+0xa0/0x100 Apr 22 13:29:29 bkserver kernel: [66320.572924] [<ffffffff81081e80>] ? kthread_create_on_node+0x120/0x120 Apr 22 13:29:29 bkserver kernel: [66320.573588] [<ffffffff8165baac>] ret_from_fork+0x7c/0xb0 Apr 22 13:29:29 bkserver kernel: [66320.574225] [<ffffffff81081e80>] ? kthread_create_on_node+0x120/0x120 Apr 22 13:29:29 bkserver kernel: [66320.574883] Code: 00 00 4c 89 e6 4c 89 ff 48 98 48 8d 04 80 48 8d 54 80 65 e8 02 0c 04 00 4c 89 f6 4c 89 e7 e8 07 f2 ff ff 85 c0 0f 8f 5e ff ff ff <0f> 0b 0f 0b 66 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 Apr 22 13:29:29 bkserver kernel: [66320.576813] RIP [<ffffffffa00bd5c1>] btrfs_set_item_key_safe+0x141/0x150 [btrfs] Apr 22 13:29:29 bkserver kernel: [66320.577476] RSP <ffff88014763dae8> Apr 22 13:29:29 bkserver kernel: [66320.579362] --- [ end trace 0ae203d38f3d2f34 ]--- -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Apr 22, 2013 at 01:19:41PM +0000, Mark Ridley wrote:> If I then use rsync --inplace to update the images, I get a btrfs stack trace > and btrfs hangs: > > This happens every night.> [<ffffffffa00bd5c1>] btrfs_set_item_key_safe+0x141/0x150 [btrfs]The check fails because it finds keys in reverted order. Given the conditions under which it happens I think it''s an on-disk corruption and fsck should be able to at least detect it. david -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Thanks, David. What causes this corruption and how can I fix it? I''m very worried about running btrfs.fsck as last time it made a slight corruption like this worse and the whole volume had to be trashed. After fsck the "available space" on df ended up being negative so nothing could be written to the volume. Mark On 22/04/2013 14:42, "David Sterba" <dsterba@suse.cz> wrote:>On Mon, Apr 22, 2013 at 01:19:41PM +0000, Mark Ridley wrote: >> If I then use rsync --inplace to update the images, I get a btrfs stack >>trace >> and btrfs hangs: >> >> This happens every night. > >> [<ffffffffa00bd5c1>] btrfs_set_item_key_safe+0x141/0x150 [btrfs] > >The check fails because it finds keys in reverted order. Given the >conditions under which it happens I think it''s an on-disk corruption and >fsck should be able to at least detect it. > >david-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Apr 22, 2013 at 4:01 PM, Mark Ridley <mark@backupsystems.co.uk> wrote:> Thanks, David. > > What causes this corruption and how can I fix it? > > I''m very worried about running btrfs.fsck as last time it made a slight > corruption like this worse and the whole volume had to be trashed. > > After fsck the "available space" on df ended up being negative so nothing > could be written to the volume. > > Mark > > On 22/04/2013 14:42, "David Sterba" <dsterba@suse.cz> wrote: > >>On Mon, Apr 22, 2013 at 01:19:41PM +0000, Mark Ridley wrote: >>> If I then use rsync --inplace to update the images, I get a btrfs stack >>>trace >>> and btrfs hangs: >>> >>> This happens every night. >> >>> [<ffffffffa00bd5c1>] btrfs_set_item_key_safe+0x141/0x150 [btrfs] >> >>The check fails because it finds keys in reverted order. Given the >>conditions under which it happens I think it''s an on-disk corruption and >>fsck should be able to at least detect it. >> >>david > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.htmlThis happened without --repair? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
No. Without --repair pointed out some problems, but whats the point of knowing about the problems if they can''t be fixed so I ran with --repair and it broke the volume. On 22/04/2013 15:02, "Harald Glatt" <mail@hachre.de> wrote:>On Mon, Apr 22, 2013 at 4:01 PM, Mark Ridley <mark@backupsystems.co.uk> >wrote: >> Thanks, David. >> >> What causes this corruption and how can I fix it? >> >> I''m very worried about running btrfs.fsck as last time it made a slight >> corruption like this worse and the whole volume had to be trashed. >> >> After fsck the "available space" on df ended up being negative so >>nothing >> could be written to the volume. >> >> Mark >> >> On 22/04/2013 14:42, "David Sterba" <dsterba@suse.cz> wrote: >> >>>On Mon, Apr 22, 2013 at 01:19:41PM +0000, Mark Ridley wrote: >>>> If I then use rsync --inplace to update the images, I get a btrfs >>>>stack >>>>trace >>>> and btrfs hangs: >>>> >>>> This happens every night. >>> >>>> [<ffffffffa00bd5c1>] btrfs_set_item_key_safe+0x141/0x150 [btrfs] >>> >>>The check fails because it finds keys in reverted order. Given the >>>conditions under which it happens I think it''s an on-disk corruption and >>>fsck should be able to at least detect it. >>> >>>david >> >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" >>in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > >This happened without --repair?-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Yeah, --repair is not recommended as of now. Maybe posting the btrfsck output just from checking would be helpful in this case. On Mon, Apr 22, 2013 at 4:04 PM, Mark Ridley <mark@backupsystems.co.uk> wrote:> No. > > Without --repair pointed out some problems, but whats the point of knowing > about the problems if they can''t be fixed so I ran with --repair and it > broke the volume. > > On 22/04/2013 15:02, "Harald Glatt" <mail@hachre.de> wrote: > >>On Mon, Apr 22, 2013 at 4:01 PM, Mark Ridley <mark@backupsystems.co.uk> >>wrote: >>> Thanks, David. >>> >>> What causes this corruption and how can I fix it? >>> >>> I''m very worried about running btrfs.fsck as last time it made a slight >>> corruption like this worse and the whole volume had to be trashed. >>> >>> After fsck the "available space" on df ended up being negative so >>>nothing >>> could be written to the volume. >>> >>> Mark >>> >>> On 22/04/2013 14:42, "David Sterba" <dsterba@suse.cz> wrote: >>> >>>>On Mon, Apr 22, 2013 at 01:19:41PM +0000, Mark Ridley wrote: >>>>> If I then use rsync --inplace to update the images, I get a btrfs >>>>>stack >>>>>trace >>>>> and btrfs hangs: >>>>> >>>>> This happens every night. >>>> >>>>> [<ffffffffa00bd5c1>] btrfs_set_item_key_safe+0x141/0x150 [btrfs] >>>> >>>>The check fails because it finds keys in reverted order. Given the >>>>conditions under which it happens I think it''s an on-disk corruption and >>>>fsck should be able to at least detect it. >>>> >>>>david >>> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" >>>in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >>This happened without --repair? >-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
What does btrfs scrub do? Is that meant to detect and fix problems? Thanks, Mark On 22/04/2013 15:13, "Harald Glatt" <mail@hachre.de> wrote:>Yeah, --repair is not recommended as of now. Maybe posting the btrfsck >output just from checking would be helpful in this case. > >On Mon, Apr 22, 2013 at 4:04 PM, Mark Ridley <mark@backupsystems.co.uk> >wrote: >> No. >> >> Without --repair pointed out some problems, but whats the point of >>knowing >> about the problems if they can''t be fixed so I ran with --repair and it >> broke the volume. >> >> On 22/04/2013 15:02, "Harald Glatt" <mail@hachre.de> wrote: >> >>>On Mon, Apr 22, 2013 at 4:01 PM, Mark Ridley <mark@backupsystems.co.uk> >>>wrote: >>>> Thanks, David. >>>> >>>> What causes this corruption and how can I fix it? >>>> >>>> I''m very worried about running btrfs.fsck as last time it made a >>>>slight >>>> corruption like this worse and the whole volume had to be trashed. >>>> >>>> After fsck the "available space" on df ended up being negative so >>>>nothing >>>> could be written to the volume. >>>> >>>> Mark >>>> >>>> On 22/04/2013 14:42, "David Sterba" <dsterba@suse.cz> wrote: >>>> >>>>>On Mon, Apr 22, 2013 at 01:19:41PM +0000, Mark Ridley wrote: >>>>>> If I then use rsync --inplace to update the images, I get a btrfs >>>>>>stack >>>>>>trace >>>>>> and btrfs hangs: >>>>>> >>>>>> This happens every night. >>>>> >>>>>> [<ffffffffa00bd5c1>] btrfs_set_item_key_safe+0x141/0x150 [btrfs] >>>>> >>>>>The check fails because it finds keys in reverted order. Given the >>>>>conditions under which it happens I think it''s an on-disk corruption >>>>>and >>>>>fsck should be able to at least detect it. >>>>> >>>>>david >>>> >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" >>>>in >>>> the body of a message to majordomo@vger.kernel.org >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >>>This happened without --repair? >>-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Only data errors (from CRC checks), maybe also some structure errors - I''m not sure. A remount should fix all errors. If it doesn''t I think it''s considered a bug - ultimately the goal is that no --repair should ever be required... Scrub is generally safe though, unlike btrfsck --repair. On Mon, Apr 22, 2013 at 4:16 PM, Mark Ridley <mark@backupsystems.co.uk> wrote:> What does btrfs scrub do? > > Is that meant to detect and fix problems? > > Thanks, > > Mark > > On 22/04/2013 15:13, "Harald Glatt" <mail@hachre.de> wrote: > >>Yeah, --repair is not recommended as of now. Maybe posting the btrfsck >>output just from checking would be helpful in this case. >> >>On Mon, Apr 22, 2013 at 4:04 PM, Mark Ridley <mark@backupsystems.co.uk> >>wrote: >>> No. >>> >>> Without --repair pointed out some problems, but whats the point of >>>knowing >>> about the problems if they can''t be fixed so I ran with --repair and it >>> broke the volume. >>> >>> On 22/04/2013 15:02, "Harald Glatt" <mail@hachre.de> wrote: >>> >>>>On Mon, Apr 22, 2013 at 4:01 PM, Mark Ridley <mark@backupsystems.co.uk> >>>>wrote: >>>>> Thanks, David. >>>>> >>>>> What causes this corruption and how can I fix it? >>>>> >>>>> I''m very worried about running btrfs.fsck as last time it made a >>>>>slight >>>>> corruption like this worse and the whole volume had to be trashed. >>>>> >>>>> After fsck the "available space" on df ended up being negative so >>>>>nothing >>>>> could be written to the volume. >>>>> >>>>> Mark >>>>> >>>>> On 22/04/2013 14:42, "David Sterba" <dsterba@suse.cz> wrote: >>>>> >>>>>>On Mon, Apr 22, 2013 at 01:19:41PM +0000, Mark Ridley wrote: >>>>>>> If I then use rsync --inplace to update the images, I get a btrfs >>>>>>>stack >>>>>>>trace >>>>>>> and btrfs hangs: >>>>>>> >>>>>>> This happens every night. >>>>>> >>>>>>> [<ffffffffa00bd5c1>] btrfs_set_item_key_safe+0x141/0x150 [btrfs] >>>>>> >>>>>>The check fails because it finds keys in reverted order. Given the >>>>>>conditions under which it happens I think it''s an on-disk corruption >>>>>>and >>>>>>fsck should be able to at least detect it. >>>>>> >>>>>>david >>>>> >>>>> -- >>>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" >>>>>in >>>>> the body of a message to majordomo@vger.kernel.org >>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>> >>>>This happened without --repair? >>> >-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Thanks for getting back to me. Do you think remount fixes the keys reverted problem? On 22/04/2013 15:18, "Harald Glatt" <mail@hachre.de> wrote:>Only data errors (from CRC checks), maybe also some structure errors - >I''m not sure. A remount should fix all errors. If it doesn''t I think >it''s considered a bug - ultimately the goal is that no --repair should >ever be required... Scrub is generally safe though, unlike btrfsck >--repair. > >On Mon, Apr 22, 2013 at 4:16 PM, Mark Ridley <mark@backupsystems.co.uk> >wrote: >> What does btrfs scrub do? >> >> Is that meant to detect and fix problems? >> >> Thanks, >> >> Mark >> >> On 22/04/2013 15:13, "Harald Glatt" <mail@hachre.de> wrote: >> >>>Yeah, --repair is not recommended as of now. Maybe posting the btrfsck >>>output just from checking would be helpful in this case. >>> >>>On Mon, Apr 22, 2013 at 4:04 PM, Mark Ridley <mark@backupsystems.co.uk> >>>wrote: >>>> No. >>>> >>>> Without --repair pointed out some problems, but whats the point of >>>>knowing >>>> about the problems if they can''t be fixed so I ran with --repair and >>>>it >>>> broke the volume. >>>> >>>> On 22/04/2013 15:02, "Harald Glatt" <mail@hachre.de> wrote: >>>> >>>>>On Mon, Apr 22, 2013 at 4:01 PM, Mark Ridley >>>>><mark@backupsystems.co.uk> >>>>>wrote: >>>>>> Thanks, David. >>>>>> >>>>>> What causes this corruption and how can I fix it? >>>>>> >>>>>> I''m very worried about running btrfs.fsck as last time it made a >>>>>>slight >>>>>> corruption like this worse and the whole volume had to be trashed. >>>>>> >>>>>> After fsck the "available space" on df ended up being negative so >>>>>>nothing >>>>>> could be written to the volume. >>>>>> >>>>>> Mark >>>>>> >>>>>> On 22/04/2013 14:42, "David Sterba" <dsterba@suse.cz> wrote: >>>>>> >>>>>>>On Mon, Apr 22, 2013 at 01:19:41PM +0000, Mark Ridley wrote: >>>>>>>> If I then use rsync --inplace to update the images, I get a btrfs >>>>>>>>stack >>>>>>>>trace >>>>>>>> and btrfs hangs: >>>>>>>> >>>>>>>> This happens every night. >>>>>>> >>>>>>>> [<ffffffffa00bd5c1>] btrfs_set_item_key_safe+0x141/0x150 [btrfs] >>>>>>> >>>>>>>The check fails because it finds keys in reverted order. Given the >>>>>>>conditions under which it happens I think it''s an on-disk corruption >>>>>>>and >>>>>>>fsck should be able to at least detect it. >>>>>>> >>>>>>>david >>>>>> >>>>>> -- >>>>>> To unsubscribe from this list: send the line "unsubscribe >>>>>>linux-btrfs" >>>>>>in >>>>>> the body of a message to majordomo@vger.kernel.org >>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>> >>>>>This happened without --repair? >>>> >>-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
I think it won''t... That would just be the goal eventually. If scrub sees no errors all the data should be in tact and your best bet to get things working perfectly again is to create a new filesystem and transfer the data into it. I''m not a dev and I can''t say if a btrfs-image for debugging purposes would be helpful in this case or not, maybe this kind of corrution has been fixed a long time ago and wouldn''t happen again in up-to-date versions of btrfs anyway. On Mon, Apr 22, 2013 at 4:24 PM, Mark Ridley <mark@backupsystems.co.uk> wrote:> Thanks for getting back to me. > > Do you think remount fixes the keys reverted problem? > > On 22/04/2013 15:18, "Harald Glatt" <mail@hachre.de> wrote: > >>Only data errors (from CRC checks), maybe also some structure errors - >>I''m not sure. A remount should fix all errors. If it doesn''t I think >>it''s considered a bug - ultimately the goal is that no --repair should >>ever be required... Scrub is generally safe though, unlike btrfsck >>--repair. >> >>On Mon, Apr 22, 2013 at 4:16 PM, Mark Ridley <mark@backupsystems.co.uk> >>wrote: >>> What does btrfs scrub do? >>> >>> Is that meant to detect and fix problems? >>> >>> Thanks, >>> >>> Mark >>> >>> On 22/04/2013 15:13, "Harald Glatt" <mail@hachre.de> wrote: >>> >>>>Yeah, --repair is not recommended as of now. Maybe posting the btrfsck >>>>output just from checking would be helpful in this case. >>>> >>>>On Mon, Apr 22, 2013 at 4:04 PM, Mark Ridley <mark@backupsystems.co.uk> >>>>wrote: >>>>> No. >>>>> >>>>> Without --repair pointed out some problems, but whats the point of >>>>>knowing >>>>> about the problems if they can''t be fixed so I ran with --repair and >>>>>it >>>>> broke the volume. >>>>> >>>>> On 22/04/2013 15:02, "Harald Glatt" <mail@hachre.de> wrote: >>>>> >>>>>>On Mon, Apr 22, 2013 at 4:01 PM, Mark Ridley >>>>>><mark@backupsystems.co.uk> >>>>>>wrote: >>>>>>> Thanks, David. >>>>>>> >>>>>>> What causes this corruption and how can I fix it? >>>>>>> >>>>>>> I''m very worried about running btrfs.fsck as last time it made a >>>>>>>slight >>>>>>> corruption like this worse and the whole volume had to be trashed. >>>>>>> >>>>>>> After fsck the "available space" on df ended up being negative so >>>>>>>nothing >>>>>>> could be written to the volume. >>>>>>> >>>>>>> Mark >>>>>>> >>>>>>> On 22/04/2013 14:42, "David Sterba" <dsterba@suse.cz> wrote: >>>>>>> >>>>>>>>On Mon, Apr 22, 2013 at 01:19:41PM +0000, Mark Ridley wrote: >>>>>>>>> If I then use rsync --inplace to update the images, I get a btrfs >>>>>>>>>stack >>>>>>>>>trace >>>>>>>>> and btrfs hangs: >>>>>>>>> >>>>>>>>> This happens every night. >>>>>>>> >>>>>>>>> [<ffffffffa00bd5c1>] btrfs_set_item_key_safe+0x141/0x150 [btrfs] >>>>>>>> >>>>>>>>The check fails because it finds keys in reverted order. Given the >>>>>>>>conditions under which it happens I think it''s an on-disk corruption >>>>>>>>and >>>>>>>>fsck should be able to at least detect it. >>>>>>>> >>>>>>>>david >>>>>>> >>>>>>> -- >>>>>>> To unsubscribe from this list: send the line "unsubscribe >>>>>>>linux-btrfs" >>>>>>>in >>>>>>> the body of a message to majordomo@vger.kernel.org >>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>> >>>>>>This happened without --repair? >>>>> >>> >-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Any idea when the 3.9 kernel will be out. I see there are lots of btrfs fixes going into it. I''m currently running FC18 on 3.8.7, although 10 minutes ago it got updated to 3.8.8. On 22/04/2013 15:28, "Harald Glatt" <mail@hachre.de> wrote:>I think it won''t... That would just be the goal eventually. If scrub >sees no errors all the data should be in tact and your best bet to get >things working perfectly again is to create a new filesystem and >transfer the data into it. > >I''m not a dev and I can''t say if a btrfs-image for debugging purposes >would be helpful in this case or not, maybe this kind of corrution has >been fixed a long time ago and wouldn''t happen again in up-to-date >versions of btrfs anyway. > >On Mon, Apr 22, 2013 at 4:24 PM, Mark Ridley <mark@backupsystems.co.uk> >wrote: >> Thanks for getting back to me. >> >> Do you think remount fixes the keys reverted problem? >> >> On 22/04/2013 15:18, "Harald Glatt" <mail@hachre.de> wrote: >> >>>Only data errors (from CRC checks), maybe also some structure errors - >>>I''m not sure. A remount should fix all errors. If it doesn''t I think >>>it''s considered a bug - ultimately the goal is that no --repair should >>>ever be required... Scrub is generally safe though, unlike btrfsck >>>--repair. >>> >>>On Mon, Apr 22, 2013 at 4:16 PM, Mark Ridley <mark@backupsystems.co.uk> >>>wrote: >>>> What does btrfs scrub do? >>>> >>>> Is that meant to detect and fix problems? >>>> >>>> Thanks, >>>> >>>> Mark >>>> >>>> On 22/04/2013 15:13, "Harald Glatt" <mail@hachre.de> wrote: >>>> >>>>>Yeah, --repair is not recommended as of now. Maybe posting the btrfsck >>>>>output just from checking would be helpful in this case. >>>>> >>>>>On Mon, Apr 22, 2013 at 4:04 PM, Mark Ridley >>>>><mark@backupsystems.co.uk> >>>>>wrote: >>>>>> No. >>>>>> >>>>>> Without --repair pointed out some problems, but whats the point of >>>>>>knowing >>>>>> about the problems if they can''t be fixed so I ran with --repair and >>>>>>it >>>>>> broke the volume. >>>>>> >>>>>> On 22/04/2013 15:02, "Harald Glatt" <mail@hachre.de> wrote: >>>>>> >>>>>>>On Mon, Apr 22, 2013 at 4:01 PM, Mark Ridley >>>>>>><mark@backupsystems.co.uk> >>>>>>>wrote: >>>>>>>> Thanks, David. >>>>>>>> >>>>>>>> What causes this corruption and how can I fix it? >>>>>>>> >>>>>>>> I''m very worried about running btrfs.fsck as last time it made a >>>>>>>>slight >>>>>>>> corruption like this worse and the whole volume had to be trashed. >>>>>>>> >>>>>>>> After fsck the "available space" on df ended up being negative so >>>>>>>>nothing >>>>>>>> could be written to the volume. >>>>>>>> >>>>>>>> Mark >>>>>>>> >>>>>>>> On 22/04/2013 14:42, "David Sterba" <dsterba@suse.cz> wrote: >>>>>>>> >>>>>>>>>On Mon, Apr 22, 2013 at 01:19:41PM +0000, Mark Ridley wrote: >>>>>>>>>> If I then use rsync --inplace to update the images, I get a >>>>>>>>>>btrfs >>>>>>>>>>stack >>>>>>>>>>trace >>>>>>>>>> and btrfs hangs: >>>>>>>>>> >>>>>>>>>> This happens every night. >>>>>>>>> >>>>>>>>>> [<ffffffffa00bd5c1>] btrfs_set_item_key_safe+0x141/0x150 [btrfs] >>>>>>>>> >>>>>>>>>The check fails because it finds keys in reverted order. Given the >>>>>>>>>conditions under which it happens I think it''s an on-disk >>>>>>>>>corruption >>>>>>>>>and >>>>>>>>>fsck should be able to at least detect it. >>>>>>>>> >>>>>>>>>david >>>>>>>> >>>>>>>> -- >>>>>>>> To unsubscribe from this list: send the line "unsubscribe >>>>>>>>linux-btrfs" >>>>>>>>in >>>>>>>> the body of a message to majordomo@vger.kernel.org >>>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>>> >>>>>>>This happened without --repair? >>>>>> >>>> >>-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Apr 22, 2013 at 03:40:00PM +0100, Mark Ridley wrote:> Any idea when the 3.9 kernel will be out.Most likely next Sunday/Monday, unless something drastic shows up this week (or there are too many fixes going in).> I see there are lots of btrfs fixes going into it. > > I''m currently running FC18 on 3.8.7, although 10 minutes ago it got > updated to 3.8.8.Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk == PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- We believe in free will because we have no choice. ---
I heard it''s coming out next week. On Mon, Apr 22, 2013 at 4:40 PM, Mark Ridley <mark@backupsystems.co.uk> wrote:> Any idea when the 3.9 kernel will be out. > > I see there are lots of btrfs fixes going into it. > > I''m currently running FC18 on 3.8.7, although 10 minutes ago it got > updated to 3.8.8. > > On 22/04/2013 15:28, "Harald Glatt" <mail@hachre.de> wrote: > >>I think it won''t... That would just be the goal eventually. If scrub >>sees no errors all the data should be in tact and your best bet to get >>things working perfectly again is to create a new filesystem and >>transfer the data into it. >> >>I''m not a dev and I can''t say if a btrfs-image for debugging purposes >>would be helpful in this case or not, maybe this kind of corrution has >>been fixed a long time ago and wouldn''t happen again in up-to-date >>versions of btrfs anyway. >> >>On Mon, Apr 22, 2013 at 4:24 PM, Mark Ridley <mark@backupsystems.co.uk> >>wrote: >>> Thanks for getting back to me. >>> >>> Do you think remount fixes the keys reverted problem? >>> >>> On 22/04/2013 15:18, "Harald Glatt" <mail@hachre.de> wrote: >>> >>>>Only data errors (from CRC checks), maybe also some structure errors - >>>>I''m not sure. A remount should fix all errors. If it doesn''t I think >>>>it''s considered a bug - ultimately the goal is that no --repair should >>>>ever be required... Scrub is generally safe though, unlike btrfsck >>>>--repair. >>>> >>>>On Mon, Apr 22, 2013 at 4:16 PM, Mark Ridley <mark@backupsystems.co.uk> >>>>wrote: >>>>> What does btrfs scrub do? >>>>> >>>>> Is that meant to detect and fix problems? >>>>> >>>>> Thanks, >>>>> >>>>> Mark >>>>> >>>>> On 22/04/2013 15:13, "Harald Glatt" <mail@hachre.de> wrote: >>>>> >>>>>>Yeah, --repair is not recommended as of now. Maybe posting the btrfsck >>>>>>output just from checking would be helpful in this case. >>>>>> >>>>>>On Mon, Apr 22, 2013 at 4:04 PM, Mark Ridley >>>>>><mark@backupsystems.co.uk> >>>>>>wrote: >>>>>>> No. >>>>>>> >>>>>>> Without --repair pointed out some problems, but whats the point of >>>>>>>knowing >>>>>>> about the problems if they can''t be fixed so I ran with --repair and >>>>>>>it >>>>>>> broke the volume. >>>>>>> >>>>>>> On 22/04/2013 15:02, "Harald Glatt" <mail@hachre.de> wrote: >>>>>>> >>>>>>>>On Mon, Apr 22, 2013 at 4:01 PM, Mark Ridley >>>>>>>><mark@backupsystems.co.uk> >>>>>>>>wrote: >>>>>>>>> Thanks, David. >>>>>>>>> >>>>>>>>> What causes this corruption and how can I fix it? >>>>>>>>> >>>>>>>>> I''m very worried about running btrfs.fsck as last time it made a >>>>>>>>>slight >>>>>>>>> corruption like this worse and the whole volume had to be trashed. >>>>>>>>> >>>>>>>>> After fsck the "available space" on df ended up being negative so >>>>>>>>>nothing >>>>>>>>> could be written to the volume. >>>>>>>>> >>>>>>>>> Mark >>>>>>>>> >>>>>>>>> On 22/04/2013 14:42, "David Sterba" <dsterba@suse.cz> wrote: >>>>>>>>> >>>>>>>>>>On Mon, Apr 22, 2013 at 01:19:41PM +0000, Mark Ridley wrote: >>>>>>>>>>> If I then use rsync --inplace to update the images, I get a >>>>>>>>>>>btrfs >>>>>>>>>>>stack >>>>>>>>>>>trace >>>>>>>>>>> and btrfs hangs: >>>>>>>>>>> >>>>>>>>>>> This happens every night. >>>>>>>>>> >>>>>>>>>>> [<ffffffffa00bd5c1>] btrfs_set_item_key_safe+0x141/0x150 [btrfs] >>>>>>>>>> >>>>>>>>>>The check fails because it finds keys in reverted order. Given the >>>>>>>>>>conditions under which it happens I think it''s an on-disk >>>>>>>>>>corruption >>>>>>>>>>and >>>>>>>>>>fsck should be able to at least detect it. >>>>>>>>>> >>>>>>>>>>david >>>>>>>>> >>>>>>>>> -- >>>>>>>>> To unsubscribe from this list: send the line "unsubscribe >>>>>>>>>linux-btrfs" >>>>>>>>>in >>>>>>>>> the body of a message to majordomo@vger.kernel.org >>>>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>>>> >>>>>>>>This happened without --repair? >>>>>>> >>>>> >>> >-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
For me anyway, I hope the fedora team don''t take too long to make it available. Thanks for all your help today. On 22/04/2013 15:41, "Harald Glatt" <mail@hachre.de> wrote:>I heard it''s coming out next week. > >On Mon, Apr 22, 2013 at 4:40 PM, Mark Ridley <mark@backupsystems.co.uk> >wrote: >> Any idea when the 3.9 kernel will be out. >> >> I see there are lots of btrfs fixes going into it. >> >> I''m currently running FC18 on 3.8.7, although 10 minutes ago it got >> updated to 3.8.8. >> >> On 22/04/2013 15:28, "Harald Glatt" <mail@hachre.de> wrote: >> >>>I think it won''t... That would just be the goal eventually. If scrub >>>sees no errors all the data should be in tact and your best bet to get >>>things working perfectly again is to create a new filesystem and >>>transfer the data into it. >>> >>>I''m not a dev and I can''t say if a btrfs-image for debugging purposes >>>would be helpful in this case or not, maybe this kind of corrution has >>>been fixed a long time ago and wouldn''t happen again in up-to-date >>>versions of btrfs anyway. >>> >>>On Mon, Apr 22, 2013 at 4:24 PM, Mark Ridley <mark@backupsystems.co.uk> >>>wrote: >>>> Thanks for getting back to me. >>>> >>>> Do you think remount fixes the keys reverted problem? >>>> >>>> On 22/04/2013 15:18, "Harald Glatt" <mail@hachre.de> wrote: >>>> >>>>>Only data errors (from CRC checks), maybe also some structure errors - >>>>>I''m not sure. A remount should fix all errors. If it doesn''t I think >>>>>it''s considered a bug - ultimately the goal is that no --repair should >>>>>ever be required... Scrub is generally safe though, unlike btrfsck >>>>>--repair. >>>>> >>>>>On Mon, Apr 22, 2013 at 4:16 PM, Mark Ridley >>>>><mark@backupsystems.co.uk> >>>>>wrote: >>>>>> What does btrfs scrub do? >>>>>> >>>>>> Is that meant to detect and fix problems? >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Mark >>>>>> >>>>>> On 22/04/2013 15:13, "Harald Glatt" <mail@hachre.de> wrote: >>>>>> >>>>>>>Yeah, --repair is not recommended as of now. Maybe posting the >>>>>>>btrfsck >>>>>>>output just from checking would be helpful in this case. >>>>>>> >>>>>>>On Mon, Apr 22, 2013 at 4:04 PM, Mark Ridley >>>>>>><mark@backupsystems.co.uk> >>>>>>>wrote: >>>>>>>> No. >>>>>>>> >>>>>>>> Without --repair pointed out some problems, but whats the point of >>>>>>>>knowing >>>>>>>> about the problems if they can''t be fixed so I ran with --repair >>>>>>>>and >>>>>>>>it >>>>>>>> broke the volume. >>>>>>>> >>>>>>>> On 22/04/2013 15:02, "Harald Glatt" <mail@hachre.de> wrote: >>>>>>>> >>>>>>>>>On Mon, Apr 22, 2013 at 4:01 PM, Mark Ridley >>>>>>>>><mark@backupsystems.co.uk> >>>>>>>>>wrote: >>>>>>>>>> Thanks, David. >>>>>>>>>> >>>>>>>>>> What causes this corruption and how can I fix it? >>>>>>>>>> >>>>>>>>>> I''m very worried about running btrfs.fsck as last time it made a >>>>>>>>>>slight >>>>>>>>>> corruption like this worse and the whole volume had to be >>>>>>>>>>trashed. >>>>>>>>>> >>>>>>>>>> After fsck the "available space" on df ended up being negative >>>>>>>>>>so >>>>>>>>>>nothing >>>>>>>>>> could be written to the volume. >>>>>>>>>> >>>>>>>>>> Mark >>>>>>>>>> >>>>>>>>>> On 22/04/2013 14:42, "David Sterba" <dsterba@suse.cz> wrote: >>>>>>>>>> >>>>>>>>>>>On Mon, Apr 22, 2013 at 01:19:41PM +0000, Mark Ridley wrote: >>>>>>>>>>>> If I then use rsync --inplace to update the images, I get a >>>>>>>>>>>>btrfs >>>>>>>>>>>>stack >>>>>>>>>>>>trace >>>>>>>>>>>> and btrfs hangs: >>>>>>>>>>>> >>>>>>>>>>>> This happens every night. >>>>>>>>>>> >>>>>>>>>>>> [<ffffffffa00bd5c1>] btrfs_set_item_key_safe+0x141/0x150 >>>>>>>>>>>>[btrfs] >>>>>>>>>>> >>>>>>>>>>>The check fails because it finds keys in reverted order. Given >>>>>>>>>>>the >>>>>>>>>>>conditions under which it happens I think it''s an on-disk >>>>>>>>>>>corruption >>>>>>>>>>>and >>>>>>>>>>>fsck should be able to at least detect it. >>>>>>>>>>> >>>>>>>>>>>david >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> To unsubscribe from this list: send the line "unsubscribe >>>>>>>>>>linux-btrfs" >>>>>>>>>>in >>>>>>>>>> the body of a message to majordomo@vger.kernel.org >>>>>>>>>> More majordomo info at >>>>>>>>>>http://vger.kernel.org/majordomo-info.html >>>>>>>>> >>>>>>>>>This happened without --repair? >>>>>>>> >>>>>> >>>> >>-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html