I just wanted to resend my last update to this thread in case it got lost
during the holiday weekend, Happy New Year everyone!
thanks for your reply Changwei,>
> no I can't say that any of the nodes lost power or rebooted. It
isn't
> impossible, but when I assessed the situation none of the nodes where down.
> there is other stuck stacks as well yes.
>
> sorry for the long email but below I have pasted what I believe is logs
> from the original "stuck stack" 3-4 days before the
"ls" stuck stack pasted
> in my original email.
> This happened on node-103, the node that was at that point modifying for
> the file(s) in the directory I was later ls-ing on. qemu is the underlying
> KVM hypervior openstack is using.
>
>
> My ocfs2 filesystem and openstack environment is back up after I rebooted
> all the nodes and the storage device. Even the files in that troubled
> directory are fine. (this isn't a production environment, only a
testing
> environment, still important but not crucial, crucial.
>
> Please let me know any observations or comments. Also please let me know
> if this occurs again how to easiest resolve and stabilize the ocfs2
> (rebooting node-103 did not seem to fix anything).
>
> Also, I am new the the concept of fencing, is ocfs2 fenced sufficiently by
> default, or should I have set up some other mechanism....?
>
> thanks
>
> 2017-12-17T23:53:42.511398+00:00 node-103 kernel: [974474.883386]
> qemu-system-x86 D ffff880ef621b9c8 0 26593 1 0x00000000
> 2017-12-17T23:53:42.511399+00:00 node-103 kernel: [974474.883390]
> ffff880ef621b9c8 ffff880ef621b9b0 ffff882038edb800 ffff88102c102a00
> 2017-12-17T23:53:42.511408+00:00 node-103 kernel: [974474.883392]
> ffff880ef621c000 ffff880ef621bb70 ffff880ef621bb68 ffff88102c102a00
> 2017-12-17T23:53:42.511410+00:00 node-103 kernel: [974474.883393]
> 0000000000000004 ffff880ef621b9e0 ffffffff81840585 7fffffffffffffff
> 2017-12-17T23:53:42.511410+00:00 node-103 kernel: [974474.883395] Call
> Trace:
> 2017-12-17T23:53:42.511411+00:00 node-103 kernel: [974474.883403]
> [<ffffffff81840585>] schedule+0x35/0x80
> 2017-12-17T23:53:42.511412+00:00 node-103 kernel: [974474.883407]
> [<ffffffff818436d5>] schedule_timeout+0x1b5/0x270
> 2017-12-17T23:53:42.511412+00:00 node-103 kernel: [974474.883411]
> [<ffffffff810ac642>] ? default_wake_function+0x12/0x20
> 2017-12-17T23:53:42.511443+00:00 node-103 kernel: [974474.883416]
> [<ffffffff810c4422>] ? autoremove_wake_function+0x12/0x40
> 2017-12-17T23:53:42.511444+00:00 node-103 kernel: [974474.883418]
> [<ffffffff810c3d52>] ? __wake_up_common+0x52/0x90
> 2017-12-17T23:53:42.511445+00:00 node-103 kernel: [974474.883420]
> [<ffffffff81840fe3>] wait_for_completion+0xb3/0x140
> 2017-12-17T23:53:42.511446+00:00 node-103 kernel: [974474.883421]
> [<ffffffff810ac630>] ? wake_up_q+0x70/0x70
> 2017-12-17T23:53:42.511446+00:00 node-103 kernel: [974474.883466]
> [<ffffffffc0896145>] __ocfs2_cluster_lock.isra.34+0x415/0x750 [ocfs2]
> 2017-12-17T23:53:42.511447+00:00 node-103 kernel: [974474.883469]
> [<ffffffff810f634b>] ? ktime_get+0x3b/0xb0
> 2017-12-17T23:53:42.511453+00:00 node-103 kernel: [974474.883482]
> [<ffffffffc089720a>] ocfs2_inode_lock_full_nested+0x16a/0x920 [ocfs2]
> 2017-12-17T23:53:42.511453+00:00 node-103 kernel: [974474.883494]
> [<ffffffffc089fe20>] ? ocfs2_check_range_for_refcount+0x150/0x150
[ocfs2]
> 2017-12-17T23:53:42.511454+00:00 node-103 kernel: [974474.883505]
> [<ffffffffc08a0045>] ocfs2_file_write_iter+0x225/0xdf0 [ocfs2]
> 2017-12-17T23:53:42.511455+00:00 node-103 kernel: [974474.883508]
> [<ffffffff812252c0>] ? poll_select_copy_remaining+0x140/0x140
> 2017-12-17T23:53:42.511455+00:00 node-103 kernel: [974474.883511]
> [<ffffffff81349a6d>] ? security_file_permission+0x3d/0xc0
> 2017-12-17T23:53:42.511456+00:00 node-103 kernel: [974474.883522]
> [<ffffffffc089fe20>] ? ocfs2_check_range_for_refcount+0x150/0x150
[ocfs2]
> 2017-12-17T23:53:42.511462+00:00 node-103 kernel: [974474.883525]
> [<ffffffff812613ea>] aio_run_iocb+0x26a/0x2d0
> 2017-12-17T23:53:42.511463+00:00 node-103 kernel: [974474.883528]
> [<ffffffff8122e8e5>] ? __fget_light+0x25/0x60
> 2017-12-17T23:53:42.511464+00:00 node-103 kernel: [974474.883529]
> [<ffffffff8122e933>] ? __fdget+0x13/0x20
> 2017-12-17T23:53:42.511464+00:00 node-103 kernel: [974474.883530]
> [<ffffffff812622cf>] do_io_submit+0x25f/0x500
> 2017-12-17T23:53:42.511482+00:00 node-103 kernel: [974474.883532]
> [<ffffffff81262580>] SyS_io_submit+0x10/0x20
> 2017-12-17T23:53:42.511490+00:00 node-103 kernel: [974474.883534]
> [<ffffffff818446b2>] entry_SYSCALL_64_fastpath+0x16/0x71
> 2017-12-17T23:53:42.511495+00:00 node-103 kernel: [974474.883545]
> qemu-img D ffff880f19ec7948 0 40743 5019 0x00000000
> 2017-12-17T23:53:42.511495+00:00 node-103 kernel: [974474.883547]
> ffff880f19ec7948 ffff882033fff060 ffff882038f3f000 ffff880b39739c00
> 2017-12-17T23:53:42.511502+00:00 node-103 kernel: [974474.883549]
> ffff880f19ec8000 ffff880f19ec7af0 ffff880f19ec7ae8 ffff880b39739c00
> 2017-12-17T23:53:42.511503+00:00 node-103 kernel: [974474.883550]
> 0000000000000004 ffff880f19ec7960 ffffffff81840585 7fffffffffffffff
> 2017-12-17T23:53:42.511503+00:00 node-103 kernel: [974474.883552] Call
> Trace:
> 2017-12-17T23:53:42.511504+00:00 node-103 kernel: [974474.883554]
> [<ffffffff81840585>] schedule+0x35/0x80
> 2017-12-17T23:53:42.511504+00:00 node-103 kernel: [974474.883555]
> [<ffffffff818436d5>] schedule_timeout+0x1b5/0x270
> 2017-12-17T23:53:42.511505+00:00 node-103 kernel: [974474.883557]
> [<ffffffff8183fed6>] ? __schedule+0x3b6/0xa30
> 2017-12-17T23:53:42.511511+00:00 node-103 kernel: [974474.883559]
> [<ffffffff81840fe3>] wait_for_completion+0xb3/0x140
> 2017-12-17T23:53:42.511512+00:00 node-103 kernel: [974474.883560]
> [<ffffffff810ac630>] ? wake_up_q+0x70/0x70
> 2017-12-17T23:53:42.511513+00:00 node-103 kernel: [974474.883573]
> [<ffffffffc0896145>] __ocfs2_cluster_lock.isra.34+0x415/0x750 [ocfs2]
> 2017-12-17T23:53:42.511513+00:00 node-103 kernel: [974474.883595]
> [<ffffffffc089720a>] ocfs2_inode_lock_full_nested+0x16a/0x920 [ocfs2]
> 2017-12-17T23:53:42.511514+00:00 node-103 kernel: [974474.883605]
> [<ffffffffc0898d6e>] ? ocfs2_extent_map_trunc+0x10e/0x150 [ocfs2]
> 2017-12-17T23:53:42.511514+00:00 node-103 kernel: [974474.883620]
> [<ffffffffc08f9b32>] ocfs2_iop_get_acl+0x52/0x100 [ocfs2]
> 2017-12-17T23:53:42.511520+00:00 node-103 kernel: [974474.883623]
> [<ffffffff812730f1>] get_acl+0x41/0x60
> 2017-12-17T23:53:42.511521+00:00 node-103 kernel: [974474.883625]
> [<ffffffff8121aeab>] generic_permission+0x13b/0x190
> 2017-12-17T23:53:42.511522+00:00 node-103 kernel: [974474.883636]
> [<ffffffffc089aeea>] ocfs2_permission+0xca/0xe0 [ocfs2]
> 2017-12-17T23:53:42.511522+00:00 node-103 kernel: [974474.883638]
> [<ffffffff8121af77>] __inode_permission+0x77/0xc0
> 2017-12-17T23:53:42.511523+00:00 node-103 kernel: [974474.883640]
> [<ffffffff8121afd4>] inode_permission+0x14/0x50
> 2017-12-17T23:53:42.511524+00:00 node-103 kernel: [974474.883641]
> [<ffffffff8121b0fb>] may_open+0x5b/0xf0
> 2017-12-17T23:53:42.511534+00:00 node-103 kernel: [974474.883642]
> [<ffffffff8121efe8>] path_openat+0x188/0x1330
> 2017-12-17T23:53:42.511549+00:00 node-103 kernel: [974474.883644]
> [<ffffffff81221381>] do_filp_open+0x91/0x100
> 2017-12-17T23:53:42.511551+00:00 node-103 kernel: [974474.883645]
> [<ffffffff8122edb6>] ? __alloc_fd+0x46/0x190
> 2017-12-17T23:53:42.511556+00:00 node-103 kernel: [974474.883647]
> [<ffffffff8120f738>] do_sys_open+0x138/0x2a0
> 2017-12-17T23:53:42.511556+00:00 node-103 kernel: [974474.883649]
> [<ffffffff8106b594>] ? __do_page_fault+0x1b4/0x400
> 2017-12-17T23:53:42.511557+00:00 node-103 kernel: [974474.883651]
> [<ffffffff8120f8be>] SyS_open+0x1e/0x20
> 2017-12-17T23:53:42.511558+00:00 node-103 kernel: [974474.883653]
> [<ffffffff818446b2>] entry_SYSCALL_64_fastpath+0x16/0x71
> 2017-12-17T23:55:42.511102+00:00 node-103 kernel: [974594.892385]
> qemu-system-x86 D ffff880ef621b9c8 0 26593 1 0x00000000
> 2017-12-17T23:55:42.511103+00:00 node-103 kernel: [974594.892388]
> ffff880ef621b9c8 ffff880ef621b9b0 ffff882038edb800 ffff88102c102a00
> 2017-12-17T23:55:42.511121+00:00 node-103 kernel: [974594.892390]
> ffff880ef621c000 ffff880ef621bb70 ffff880ef621bb68 ffff88102c102a00
> 2017-12-17T23:55:42.511123+00:00 node-103 kernel: [974594.892391]
> 0000000000000004 ffff880ef621b9e0 ffffffff81840585 7fffffffffffffff
> 2017-12-17T23:55:42.511124+00:00 node-103 kernel: [974594.892393] Call
> Trace:
> 2017-12-17T23:55:42.511125+00:00 node-103 kernel: [974594.892399]
> [<ffffffff81840585>] schedule+0x35/0x80
> 2017-12-17T23:55:42.511125+00:00 node-103 kernel: [974594.892402]
> [<ffffffff818436d5>] schedule_timeout+0x1b5/0x270
> 2017-12-17T23:55:42.511126+00:00 node-103 kernel: [974594.892406]
> [<ffffffff810ac642>] ? default_wake_function+0x12/0x20
> 2017-12-17T23:55:42.511127+00:00 node-103 kernel: [974594.892409]
> [<ffffffff810c4422>] ? autoremove_wake_function+0x12/0x40
> 2017-12-17T23:55:42.511128+00:00 node-103 kernel: [974594.892411]
> [<ffffffff810c3d52>] ? __wake_up_common+0x52/0x90
> 2017-12-17T23:55:42.511129+00:00 node-103 kernel: [974594.892413]
> [<ffffffff81840fe3>] wait_for_completion+0xb3/0x140
> 2017-12-17T23:55:42.511130+00:00 node-103 kernel: [974594.892414]
> [<ffffffff810ac630>] ? wake_up_q+0x70/0x70
> 2017-12-17T23:55:42.511131+00:00 node-103 kernel: [974594.892448]
> [<ffffffffc0896145>] __ocfs2_cluster_lock.isra.34+0x415/0x750 [ocfs2]
> 2017-12-17T23:55:42.511131+00:00 node-103 kernel: [974594.892451]
> [<ffffffff810f634b>] ? ktime_get+0x3b/0xb0
> 2017-12-17T23:55:42.511133+00:00 node-103 kernel: [974594.892463]
> [<ffffffffc089720a>] ocfs2_inode_lock_full_nested+0x16a/0x920 [ocfs2]
> 2017-12-17T23:55:42.511134+00:00 node-103 kernel: [974594.892475]
> [<ffffffffc089fe20>] ? ocfs2_check_range_for_refcount+0x150/0x150
[ocfs2]
> 2017-12-17T23:55:42.511135+00:00 node-103 kernel: [974594.892486]
> [<ffffffffc08a0045>] ocfs2_file_write_iter+0x225/0xdf0 [ocfs2]
> 2017-12-17T23:55:42.511136+00:00 node-103 kernel: [974594.892490]
> [<ffffffff812252c0>] ? poll_select_copy_remaining+0x140/0x140
> 2017-12-17T23:55:42.511136+00:00 node-103 kernel: [974594.892493]
> [<ffffffff81349a6d>] ? security_file_permission+0x3d/0xc0
> 2017-12-17T23:55:42.511137+00:00 node-103 kernel: [974594.892504]
> [<ffffffffc089fe20>] ? ocfs2_check_range_for_refcount+0x150/0x150
[ocfs2]
> 2017-12-17T23:55:42.511139+00:00 node-103 kernel: [974594.892507]
> [<ffffffff812613ea>] aio_run_iocb+0x26a/0x2d0
> 2017-12-17T23:55:42.511140+00:00 node-103 kernel: [974594.892510]
> [<ffffffff8122e8e5>] ? __fget_light+0x25/0x60
> 2017-12-17T23:55:42.511141+00:00 node-103 kernel: [974594.892511]
> [<ffffffff8122e933>] ? __fdget+0x13/0x20
> 2017-12-17T23:55:42.511142+00:00 node-103 kernel: [974594.892513]
> [<ffffffff812622cf>] do_io_submit+0x25f/0x500
> 2017-12-17T23:55:42.511158+00:00 node-103 kernel: [974594.892515]
> [<ffffffff81262580>] SyS_io_submit+0x10/0x20
> 2017-12-17T23:55:42.511160+00:00 node-103 kernel: [974594.892517]
> [<ffffffff818446b2>] entry_SYSCALL_64_fastpath+0x16/0x71
> 2017-12-17T23:55:42.511163+00:00 node-103 kernel: [974594.892527]
> qemu-img D ffff880f19ec7948 0 40743 5019 0x00000000
> 2017-12-17T23:55:42.511163+00:00 node-103 kernel: [974594.892529]
> ffff880f19ec7948 ffff882033fff060 ffff882038f3f000 ffff880b39739c00
> 2017-12-17T23:55:42.511165+00:00 node-103 kernel: [974594.892530]
> ffff880f19ec8000 ffff880f19ec7af0 ffff880f19ec7ae8 ffff880b39739c00
> 2017-12-17T23:55:42.511166+00:00 node-103 kernel: [974594.892532]
> 0000000000000004 ffff880f19ec7960 ffffffff81840585 7fffffffffffffff
> 2017-12-17T23:55:42.511167+00:00 node-103 kernel: [974594.892533] Call
> Trace:
> 2017-12-17T23:55:42.511167+00:00 node-103 kernel: [974594.892535]
> [<ffffffff81840585>] schedule+0x35/0x80
> 2017-12-17T23:55:42.511168+00:00 node-103 kernel: [974594.892537]
> [<ffffffff818436d5>] schedule_timeout+0x1b5/0x270
> 2017-12-17T23:55:42.511168+00:00 node-103 kernel: [974594.892538]
> [<ffffffff8183fed6>] ? __schedule+0x3b6/0xa30
> 2017-12-17T23:55:42.511170+00:00 node-103 kernel: [974594.892540]
> [<ffffffff81840fe3>] wait_for_completion+0xb3/0x140
> 2017-12-17T23:55:42.511171+00:00 node-103 kernel: [974594.892542]
> [<ffffffff810ac630>] ? wake_up_q+0x70/0x70
> 2017-12-17T23:55:42.511172+00:00 node-103 kernel: [974594.892553]
> [<ffffffffc0896145>] __ocfs2_cluster_lock.isra.34+0x415/0x750 [ocfs2]
> 2017-12-17T23:55:42.511173+00:00 node-103 kernel: [974594.892565]
> [<ffffffffc089720a>] ocfs2_inode_lock_full_nested+0x16a/0x920 [ocfs2]
> 2017-12-17T23:55:42.511174+00:00 node-103 kernel: [974594.892576]
> [<ffffffffc0898d6e>] ? ocfs2_extent_map_trunc+0x10e/0x150 [ocfs2]
> 2017-12-17T23:55:42.511174+00:00 node-103 kernel: [974594.892592]
> [<ffffffffc08f9b32>] ocfs2_iop_get_acl+0x52/0x100 [ocfs2]
> 2017-12-17T23:55:42.511176+00:00 node-103 kernel: [974594.892594]
> [<ffffffff812730f1>] get_acl+0x41/0x60
> 2017-12-17T23:55:42.511177+00:00 node-103 kernel: [974594.892596]
> [<ffffffff8121aeab>] generic_permission+0x13b/0x190
> 2017-12-17T23:55:42.511178+00:00 node-103 kernel: [974594.892608]
> [<ffffffffc089aeea>] ocfs2_permission+0xca/0xe0 [ocfs2]
> 2017-12-17T23:55:42.511179+00:00 node-103 kernel: [974594.892610]
> [<ffffffff8121af77>] __inode_permission+0x77/0xc0
> 2017-12-17T23:55:42.511179+00:00 node-103 kernel: [974594.892612]
> [<ffffffff8121afd4>] inode_permission+0x14/0x50
> 2017-12-17T23:55:42.511180+00:00 node-103 kernel: [974594.892613]
> [<ffffffff8121b0fb>] may_open+0x5b/0xf0
> 2017-12-17T23:55:42.511181+00:00 node-103 kernel: [974594.892615]
> [<ffffffff8121efe8>] path_openat+0x188/0x1330
> 2017-12-17T23:55:42.511183+00:00 node-103 kernel: [974594.892616]
> [<ffffffff81221381>] do_filp_open+0x91/0x100
> 2017-12-17T23:55:42.511184+00:00 node-103 kernel: [974594.892618]
> [<ffffffff8122edb6>] ? __alloc_fd+0x46/0x190
> 2017-12-17T23:55:42.511187+00:00 node-103 kernel: [974594.892620]
> [<ffffffff8120f738>] do_sys_open+0x138/0x2a0
> 2017-12-17T23:55:42.511188+00:00 node-103 kernel: [974594.892622]
> [<ffffffff8106b594>] ? __do_page_fault+0x1b4/0x400
> 2017-12-17T23:55:42.511188+00:00 node-103 kernel: [974594.892624]
> [<ffffffff8120f8be>] SyS_open+0x1e/0x20
> 2017-12-17T23:55:42.511197+00:00 node-103 kernel: [974594.892626]
> [<ffffffff818446b2>] entry_SYSCALL_64_fastpath+0x16/0x71
> 2017-12-17T23:57:42.511168+00:00 node-103 kernel: [974714.901454]
> qemu-system-x86 D ffff880ef621b9c8 0 26593 1 0x00000000
> 2017-12-17T23:57:42.511169+00:00 node-103 kernel: [974714.901457]
> ffff880ef621b9c8 ffff880ef621b9b0 ffff882038edb800 ffff88102c102a00
> 2017-12-17T23:57:42.511170+00:00 node-103 kernel: [974714.901459]
> ffff880ef621c000 ffff880ef621bb70 ffff880ef621bb68 ffff88102c102a00
> 2017-12-17T23:57:42.511183+00:00 node-103 kernel: [974714.901461]
> 0000000000000004 ffff880ef621b9e0 ffffffff81840585 7fffffffffffffff
> 2017-12-17T23:57:42.511185+00:00 node-103 kernel: [974714.901463] Call
> Trace:
> 2017-12-17T23:57:42.511185+00:00 node-103 kernel: [974714.901470]
> [<ffffffff81840585>] schedule+0x35/0x80
> 2017-12-17T23:57:42.511186+00:00 node-103 kernel: [974714.901473]
> [<ffffffff818436d5>] schedule_timeout+0x1b5/0x270
> 2017-12-17T23:57:42.511186+00:00 node-103 kernel: [974714.901477]
> [<ffffffff810ac642>] ? default_wake_function+0x12/0x20
> 2017-12-17T23:57:42.511188+00:00 node-103 kernel: [974714.901481]
> [<ffffffff810c4422>] ? autoremove_wake_function+0x12/0x40
> 2017-12-17T23:57:42.511189+00:00 node-103 kernel: [974714.901482]
> [<ffffffff810c3d52>] ? __wake_up_common+0x52/0x90
> 2017-12-17T23:57:42.511190+00:00 node-103 kernel: [974714.901484]
> [<ffffffff81840fe3>] wait_for_completion+0xb3/0x140
> 2017-12-17T23:57:42.511197+00:00 node-103 kernel: [974714.901486]
> [<ffffffff810ac630>] ? wake_up_q+0x70/0x70
> 2017-12-17T23:57:42.511198+00:00 node-103 kernel: [974714.901527]
> [<ffffffffc0896145>] __ocfs2_cluster_lock.isra.34+0x415/0x750 [ocfs2]
> 2017-12-17T23:57:42.511199+00:00 node-103 kernel: [974714.901530]
> [<ffffffff810f634b>] ? ktime_get+0x3b/0xb0
> 2017-12-17T23:57:42.511201+00:00 node-103 kernel: [974714.901543]
> [<ffffffffc089720a>] ocfs2_inode_lock_full_nested+0x16a/0x920 [ocfs2]
> 2017-12-17T23:57:42.511202+00:00 node-103 kernel: [974714.901555]
> [<ffffffffc089fe20>] ? ocfs2_check_range_for_refcount+0x150/0x150
[ocfs2]
> 2017-12-17T23:57:42.511203+00:00 node-103 kernel: [974714.901566]
> [<ffffffffc08a0045>] ocfs2_file_write_iter+0x225/0xdf0 [ocfs2]
> 2017-12-17T23:57:42.511204+00:00 node-103 kernel: [974714.901569]
> [<ffffffff812252c0>] ? poll_select_copy_remaining+0x140/0x140
> 2017-12-17T23:57:42.511204+00:00 node-103 kernel: [974714.901572]
> [<ffffffff81349a6d>] ? security_file_permission+0x3d/0xc0
> 2017-12-17T23:57:42.511205+00:00 node-103 kernel: [974714.901583]
> [<ffffffffc089fe20>] ? ocfs2_check_range_for_refcount+0x150/0x150
[ocfs2]
> 2017-12-17T23:57:42.511207+00:00 node-103 kernel: [974714.901587]
> [<ffffffff812613ea>] aio_run_iocb+0x26a/0x2d0
> 2017-12-17T23:57:42.511208+00:00 node-103 kernel: [974714.901590]
> [<ffffffff8122e8e5>] ? __fget_light+0x25/0x60
> 2017-12-17T23:57:42.511209+00:00 node-103 kernel: [974714.901591]
> [<ffffffff8122e933>] ? __fdget+0x13/0x20
> 2017-12-17T23:57:42.511210+00:00 node-103 kernel: [974714.901593]
> [<ffffffff812622cf>] do_io_submit+0x25f/0x500
> 2017-12-17T23:57:42.511227+00:00 node-103 kernel: [974714.901595]
> [<ffffffff81262580>] SyS_io_submit+0x10/0x20
> 2017-12-17T23:57:42.511229+00:00 node-103 kernel: [974714.901598]
> [<ffffffff818446b2>] entry_SYSCALL_64_fastpath+0x16/0x71
> 2017-12-17T23:57:42.511233+00:00 node-103 kernel: [974714.901609]
> qemu-img D ffff880f19ec7948 0 40743 5019 0x00000000
> 2017-12-17T23:57:42.511233+00:00 node-103 kernel: [974714.901610]
> ffff880f19ec7948 ffff882033fff060 ffff882038f3f000 ffff880b39739c00
> 2017-12-17T23:57:42.511235+00:00 node-103 kernel: [974714.901612]
> ffff880f19ec8000 ffff880f19ec7af0 ffff880f19ec7ae8 ffff880b39739c00
> 2017-12-17T23:57:42.511236+00:00 node-103 kernel: [974714.901613]
> 0000000000000004 ffff880f19ec7960 ffffffff81840585 7fffffffffffffff
> 2017-12-17T23:57:42.511237+00:00 node-103 kernel: [974714.901615] Call
> Trace:
> 2017-12-17T23:57:42.511238+00:00 node-103 kernel: [974714.901617]
> [<ffffffff81840585>] schedule+0x35/0x80
> 2017-12-17T23:57:42.511238+00:00 node-103 kernel: [974714.901618]
> [<ffffffff818436d5>] schedule_timeout+0x1b5/0x270
> 2017-12-17T23:57:42.511239+00:00 node-103 kernel: [974714.901620]
> [<ffffffff8183fed6>] ? __schedule+0x3b6/0xa30
> 2017-12-17T23:57:42.511240+00:00 node-103 kernel: [974714.901622]
> [<ffffffff81840fe3>] wait_for_completion+0xb3/0x140
> 2017-12-17T23:57:42.511242+00:00 node-103 kernel: [974714.901623]
> [<ffffffff810ac630>] ? wake_up_q+0x70/0x70
> 2017-12-17T23:57:42.511243+00:00 node-103 kernel: [974714.901636]
> [<ffffffffc0896145>] __ocfs2_cluster_lock.isra.34+0x415/0x750 [ocfs2]
> 2017-12-17T23:57:42.511243+00:00 node-103 kernel: [974714.901648]
> [<ffffffffc089720a>] ocfs2_inode_lock_full_nested+0x16a/0x920 [ocfs2]
> 2017-12-17T23:57:42.511244+00:00 node-103 kernel: [974714.901659]
> [<ffffffffc0898d6e>] ? ocfs2_extent_map_trunc+0x10e/0x150 [ocfs2]
> 2017-12-17T23:57:42.511244+00:00 node-103 kernel: [974714.901685]
> [<ffffffffc08f9b32>] ocfs2_iop_get_acl+0x52/0x100 [ocfs2]
> 2017-12-17T23:57:42.511246+00:00 node-103 kernel: [974714.901687]
> [<ffffffff812730f1>] get_acl+0x41/0x60
> 2017-12-17T23:57:42.511247+00:00 node-103 kernel: [974714.901690]
> [<ffffffff8121aeab>] generic_permission+0x13b/0x190
> 2017-12-17T23:57:42.511248+00:00 node-103 kernel: [974714.901701]
> [<ffffffffc089aeea>] ocfs2_permission+0xca/0xe0 [ocfs2]
> 2017-12-17T23:57:42.511249+00:00 node-103 kernel: [974714.901703]
> [<ffffffff8121af77>] __inode_permission+0x77/0xc0
> 2017-12-17T23:57:42.511249+00:00 node-103 kernel: [974714.901704]
> [<ffffffff8121afd4>] inode_permission+0x14/0x50
> 2017-12-17T23:57:42.511250+00:00 node-103 kernel: [974714.901706]
> [<ffffffff8121b0fb>] may_open+0x5b/0xf0
> 2017-12-17T23:57:42.511252+00:00 node-103 kernel: [974714.901707]
> [<ffffffff8121efe8>] path_openat+0x188/0x1330
> 2017-12-17T23:57:42.511253+00:00 node-103 kernel: [974714.901708]
> [<ffffffff81221381>] do_filp_open+0x91/0x100
> 2017-12-17T23:57:42.511254+00:00 node-103 kernel: [974714.901710]
> [<ffffffff8122edb6>] ? __alloc_fd+0x46/0x190
> 2017-12-17T23:57:42.511257+00:00 node-103 kernel: [974714.901712]
> [<ffffffff8120f738>] do_sys_open+0x138/0x2a0
> 2017-12-17T23:57:42.511257+00:00 node-103 kernel: [974714.901714]
> [<ffffffff8106b594>] ? __do_page_fault+0x1b4/0x400
> 2017-12-17T23:57:42.511258+00:00 node-103 kernel: [974714.901715]
> [<ffffffff8120f8be>] SyS_open+0x1e/0x20
> 2017-12-17T23:57:42.511260+00:00 node-103 kernel: [974714.901717]
> [<ffffffff818446b2>] entry_SYSCALL_64_fastpath+0x16/0x71
> 2017-12-17T23:59:42.511080+00:00 node-103 kernel: [974834.910524]
> qemu-system-x86 D ffff880ef621b9c8 0 26593 1 0x00000000
> 2017-12-17T23:59:42.511080+00:00 node-103 kernel: [974834.910528]
> ffff880ef621b9c8 ffff880ef621b9b0 ffff882038edb800 ffff88102c102a00
> 2017-12-17T23:59:42.511081+00:00 node-103 kernel: [974834.910529]
> ffff880ef621c000 ffff880ef621bb70 ffff880ef621bb68 ffff88102c102a00
> 2017-12-17T23:59:42.511083+00:00 node-103 kernel: [974834.910531]
> 0000000000000004 ffff880ef621b9e0 ffffffff81840585 7fffffffffffffff
> 2017-12-17T23:59:42.511084+00:00 node-103 kernel: [974834.910533] Call
> Trace:
> 2017-12-17T23:59:42.511085+00:00 node-103 kernel: [974834.910540]
> [<ffffffff81840585>] schedule+0x35/0x80
> 2017-12-17T23:59:42.511086+00:00 node-103 kernel: [974834.910543]
> [<ffffffff818436d5>] schedule_timeout+0x1b5/0x270
> 2017-12-17T23:59:42.511086+00:00 node-103 kernel: [974834.910547]
> [<ffffffff810ac642>] ? default_wake_function+0x12/0x20
> 2017-12-17T23:59:42.511087+00:00 node-103 kernel: [974834.910551]
> [<ffffffff810c4422>] ? autoremove_wake_function+0x12/0x40
> 2017-12-17T23:59:42.511089+00:00 node-103 kernel: [974834.910553]
> [<ffffffff810c3d52>] ? __wake_up_common+0x52/0x90
> 2017-12-17T23:59:42.511090+00:00 node-103 kernel: [974834.910555]
> [<ffffffff81840fe3>] wait_for_completion+0xb3/0x140
> 2017-12-17T23:59:42.511091+00:00 node-103 kernel: [974834.910557]
> [<ffffffff810ac630>] ? wake_up_q+0x70/0x70
> 2017-12-17T23:59:42.511091+00:00 node-103 kernel: [974834.910594]
> [<ffffffffc0896145>] __ocfs2_cluster_lock.isra.34+0x415/0x750 [ocfs2]
> 2017-12-17T23:59:42.511092+00:00 node-103 kernel: [974834.910596]
> [<ffffffff810f634b>] ? ktime_get+0x3b/0xb0
> 2017-12-17T23:59:42.511093+00:00 node-103 kernel: [974834.910609]
> [<ffffffffc089720a>] ocfs2_inode_lock_full_nested+0x16a/0x920 [ocfs2]
> 2017-12-17T23:59:42.511095+00:00 node-103 kernel: [974834.910633]
> [<ffffffffc089fe20>] ? ocfs2_check_range_for_refcount+0x150/0x150
[ocfs2]
> 2017-12-17T23:59:42.511096+00:00 node-103 kernel: [974834.910644]
> [<ffffffffc08a0045>] ocfs2_file_write_iter+0x225/0xdf0 [ocfs2]
> 2017-12-17T23:59:42.511096+00:00 node-103 kernel: [974834.910647]
> [<ffffffff812252c0>] ? poll_select_copy_remaining+0x140/0x140
> 2017-12-17T23:59:42.511097+00:00 node-103 kernel: [974834.910649]
> [<ffffffff81349a6d>] ? security_file_permission+0x3d/0xc0
> 2017-12-17T23:59:42.511098+00:00 node-103 kernel: [974834.910660]
> [<ffffffffc089fe20>] ? ocfs2_check_range_for_refcount+0x150/0x150
[ocfs2]
> 2017-12-17T23:59:42.511129+00:00 node-103 kernel: [974834.910663]
> [<ffffffff812613ea>] aio_run_iocb+0x26a/0x2d0
> 2017-12-17T23:59:42.511133+00:00 node-103 kernel: [974834.910665]
> [<ffffffff8122e8e5>] ? __fget_light+0x25/0x60
> 2017-12-17T23:59:42.511135+00:00 node-103 kernel: [974834.910666]
> [<ffffffff8122e933>] ? __fdget+0x13/0x20
> 2017-12-17T23:59:42.511137+00:00 node-103 kernel: [974834.910668]
> [<ffffffff812622cf>] do_io_submit+0x25f/0x500
> 2017-12-17T23:59:42.511154+00:00 node-103 kernel: [974834.910670]
> [<ffffffff81262580>] SyS_io_submit+0x10/0x20
> 2017-12-17T23:59:42.511156+00:00 node-103 kernel: [974834.910672]
> [<ffffffff818446b2>] entry_SYSCALL_64_fastpath+0x16/0x71
> 2017-12-17T23:59:42.511161+00:00 node-103 kernel: [974834.910686]
> qemu-img D ffff880f19ec7948 0 40743 5019 0x00000000
> 2017-12-17T23:59:42.511162+00:00 node-103 kernel: [974834.910688]
> ffff880f19ec7948 ffff882033fff060 ffff882038f3f000 ffff880b39739c00
> 2017-12-17T23:59:42.511163+00:00 node-103 kernel: [974834.910689]
> ffff880f19ec8000 ffff880f19ec7af0 ffff880f19ec7ae8 ffff880b39739c00
> 2017-12-17T23:59:42.511164+00:00 node-103 kernel: [974834.910691]
> 0000000000000004 ffff880f19ec7960 ffffffff81840585 7fffffffffffffff
> 2017-12-17T23:59:42.511165+00:00 node-103 kernel: [974834.910692] Call
> Trace:
> 2017-12-17T23:59:42.511166+00:00 node-103 kernel: [974834.910694]
> [<ffffffff81840585>] schedule+0x35/0x80
> 2017-12-17T23:59:42.511167+00:00 node-103 kernel: [974834.910696]
> [<ffffffff818436d5>] schedule_timeout+0x1b5/0x270
> 2017-12-17T23:59:42.511167+00:00 node-103 kernel: [974834.910697]
> [<ffffffff8183fed6>] ? __schedule+0x3b6/0xa30
> 2017-12-17T23:59:42.511168+00:00 node-103 kernel: [974834.910699]
> [<ffffffff81840fe3>] wait_for_completion+0xb3/0x140
> 2017-12-17T23:59:42.511170+00:00 node-103 kernel: [974834.910700]
> [<ffffffff810ac630>] ? wake_up_q+0x70/0x70
> 2017-12-17T23:59:42.511171+00:00 node-103 kernel: [974834.910712]
> [<ffffffffc0896145>] __ocfs2_cluster_lock.isra.34+0x415/0x750 [ocfs2]
> 2017-12-17T23:59:42.511172+00:00 node-103 kernel: [974834.910722]
> [<ffffffffc089720a>] ocfs2_inode_lock_full_nested+0x16a/0x920 [ocfs2]
> 2017-12-17T23:59:42.511172+00:00 node-103 kernel: [974834.910733]
> [<ffffffffc0898d6e>] ? ocfs2_extent_map_trunc+0x10e/0x150 [ocfs2]
> 2017-12-17T23:59:42.511173+00:00 node-103 kernel: [974834.910748]
> [<ffffffffc08f9b32>] ocfs2_iop_get_acl+0x52/0x100 [ocfs2]
> 2017-12-17T23:59:42.511174+00:00 node-103 kernel: [974834.910751]
> [<ffffffff812730f1>] get_acl+0x41/0x60
> 2017-12-17T23:59:42.511176+00:00 node-103 kernel: [974834.910753]
> [<ffffffff8121aeab>] generic_permission+0x13b/0x190
> 2017-12-17T23:59:42.511177+00:00 node-103 kernel: [974834.910777]
> [<ffffffffc089aeea>] ocfs2_permission+0xca/0xe0 [ocfs2]
> 2017-12-17T23:59:42.511178+00:00 node-103 kernel: [974834.910778]
> [<ffffffff8121af77>] __inode_permission+0x77/0xc0
> 2017-12-17T23:59:42.511179+00:00 node-103 kernel: [974834.910780]
> [<ffffffff8121afd4>] inode_permission+0x14/0x50
> 2017-12-17T23:59:42.511179+00:00 node-103 kernel: [974834.910782]
> [<ffffffff8121b0fb>] may_open+0x5b/0xf0
> 2017-12-17T23:59:42.511180+00:00 node-103 kernel: [974834.910783]
> [<ffffffff8121efe8>] path_openat+0x188/0x1330
> 2017-12-17T23:59:42.511182+00:00 node-103 kernel: [974834.910785]
> [<ffffffff81221381>] do_filp_open+0x91/0x100
> 2017-12-17T23:59:42.511183+00:00 node-103 kernel: [974834.910786]
> [<ffffffff8122edb6>] ? __alloc_fd+0x46/0x190
> 2017-12-17T23:59:42.511185+00:00 node-103 kernel: [974834.910789]
> [<ffffffff8120f738>] do_sys_open+0x138/0x2a0
> 2017-12-17T23:59:42.511186+00:00 node-103 kernel: [974834.910791]
> [<ffffffff8106b594>] ? __do_page_fault+0x1b4/0x400
> 2017-12-17T23:59:42.511187+00:00 node-103 kernel: [974834.910793]
> [<ffffffff8120f8be>] SyS_open+0x1e/0x20
> 2017-12-17T23:59:42.511188+00:00 node-103 kernel: [974834.910795]
> [<ffffffff818446b2>] entry_SYSCALL_64_fastpath+0x16/0x71
> 2017-12-18T00:00:01.271777+00:00 node-103 kernel: [974853.675776] Process
> accounting resumed
> 2017-12-18T00:01:42.511127+00:00 node-103 kernel: [974954.919618]
> qemu-system-x86 D ffff880ef621b9c8 0 26593 1 0x00000000
> 2017-12-18T00:01:42.511128+00:00 node-103 kernel: [974954.919621]
> ffff880ef621b9c8 ffff880ef621b9b0 ffff882038edb800 ffff88102c102a00
> 2017-12-18T00:01:42.511128+00:00 node-103 kernel: [974954.919623]
> ffff880ef621c000 ffff880ef621bb70 ffff880ef621bb68 ffff88102c102a00
> 2017-12-18T00:01:42.511130+00:00 node-103 kernel: [974954.919625]
> 0000000000000004 ffff880ef621b9e0 ffffffff81840585 7fffffffffffffff
> 2017-12-18T00:01:42.511131+00:00 node-103 kernel: [974954.919627] Call
> Trace:
> 2017-12-18T00:01:42.511132+00:00 node-103 kernel: [974954.919634]
> [<ffffffff81840585>] schedule+0x35/0x80
> 2017-12-18T00:01:42.511133+00:00 node-103 kernel: [974954.919638]
> [<ffffffff818436d5>] schedule_timeout+0x1b5/0x270
> 2017-12-18T00:01:42.511134+00:00 node-103 kernel: [974954.919643]
> [<ffffffff810ac642>] ? default_wake_function+0x12/0x20
> 2017-12-18T00:01:42.511134+00:00 node-103 kernel: [974954.919647]
> [<ffffffff810c4422>] ? autoremove_wake_function+0x12/0x40
> 2017-12-18T00:01:42.511136+00:00 node-103 kernel: [974954.919649]
> [<ffffffff810c3d52>] ? __wake_up_common+0x52/0x90
> 2017-12-18T00:01:42.511138+00:00 node-103 kernel: [974954.919651]
> [<ffffffff81840fe3>] wait_for_completion+0xb3/0x140
> 2017-12-18T00:01:42.511138+00:00 node-103 kernel: [974954.919653]
> [<ffffffff810ac630>] ? wake_up_q+0x70/0x70
> 2017-12-18T00:01:42.511139+00:00 node-103 kernel: [974954.919702]
> [<ffffffffc0896145>] __ocfs2_cluster_lock.isra.34+0x415/0x750 [ocfs2]
> 2017-12-18T00:01:42.511139+00:00 node-103 kernel: [974954.919705]
> [<ffffffff810f634b>] ? ktime_get+0x3b/0xb0
> 2017-12-18T00:01:42.511141+00:00 node-103 kernel: [974954.919719]
> [<ffffffffc089720a>] ocfs2_inode_lock_full_nested+0x16a/0x920 [ocfs2]
> 2017-12-18T00:01:42.511142+00:00 node-103 kernel: [974954.919732]
> [<ffffffffc089fe20>] ? ocfs2_check_range_for_refcount+0x150/0x150
[ocfs2]
> 2017-12-18T00:01:42.511143+00:00 node-103 kernel: [974954.919744]
> [<ffffffffc08a0045>] ocfs2_file_write_iter+0x225/0xdf0 [ocfs2]
> 2017-12-18T00:01:42.511144+00:00 node-103 kernel: [974954.919746]
> [<ffffffff812252c0>] ? poll_select_copy_remaining+0x140/0x140
> 2017-12-18T00:01:42.511145+00:00 node-103 kernel: [974954.919749]
> [<ffffffff81349a6d>] ? security_file_permission+0x3d/0xc0
> 2017-12-18T00:01:42.511176+00:00 node-103 kernel: [974954.919761]
> [<ffffffffc089fe20>] ? ocfs2_check_range_for_refcount+0x150/0x150
[ocfs2]
> 2017-12-18T00:01:42.511181+00:00 node-103 kernel: [974954.919764]
> [<ffffffff812613ea>] aio_run_iocb+0x26a/0x2d0
> 2017-12-18T00:01:42.511182+00:00 node-103 kernel: [974954.919766]
> [<ffffffff8122e8e5>] ? __fget_light+0x25/0x60
> 2017-12-18T00:01:42.511184+00:00 node-103 kernel: [974954.919767]
> [<ffffffff8122e933>] ? __fdget+0x13/0x20
> 2017-12-18T00:01:42.511185+00:00 node-103 kernel: [974954.919769]
> [<ffffffff812622cf>] do_io_submit+0x25f/0x500
> 2017-12-18T00:01:42.511203+00:00 node-103 kernel: [974954.919771]
> [<ffffffff81262580>] SyS_io_submit+0x10/0x20
> 2017-12-18T00:01:42.511205+00:00 node-103 kernel: [974954.919773]
> [<ffffffff818446b2>] entry_SYSCALL_64_fastpath+0x16/0x71
> 2017-12-18T00:01:42.511209+00:00 node-103 kernel: [974954.919786]
> qemu-img D ffff880f19ec7948 0 40743 5019 0x00000000
> 2017-12-18T00:01:42.511210+00:00 node-103 kernel: [974954.919788]
> ffff880f19ec7948 ffff882033fff060 ffff882038f3f000 ffff880b39739c00
> 2017-12-18T00:01:42.511211+00:00 node-103 kernel: [974954.919789]
> ffff880f19ec8000 ffff880f19ec7af0 ffff880f19ec7ae8 ffff880b39739c00
> 2017-12-18T00:01:42.511212+00:00 node-103 kernel: [974954.919791]
> 0000000000000004 ffff880f19ec7960 ffffffff81840585 7fffffffffffffff
> 2017-12-18T00:01:42.511213+00:00 node-103 kernel: [974954.919792] Call
> Trace:
> 2017-12-18T00:01:42.511215+00:00 node-103 kernel: [974954.919794]
> [<ffffffff81840585>] schedule+0x35/0x80
> 2017-12-18T00:01:42.511215+00:00 node-103 kernel: [974954.919795]
> [<ffffffff818436d5>] schedule_timeout+0x1b5/0x270
> 2017-12-18T00:01:42.511216+00:00 node-103 kernel: [974954.919797]
> [<ffffffff8183fed6>] ? __schedule+0x3b6/0xa30
> 2017-12-18T00:01:42.511217+00:00 node-103 kernel: [974954.919799]
> [<ffffffff81840fe3>] wait_for_completion+0xb3/0x140
> 2017-12-18T00:01:42.511218+00:00 node-103 kernel: [974954.919801]
> [<ffffffff810ac630>] ? wake_up_q+0x70/0x70
> 2017-12-18T00:01:42.511220+00:00 node-103 kernel: [974954.919826]
> [<ffffffffc0896145>] __ocfs2_cluster_lock.isra.34+0x415/0x750 [ocfs2]
> 2017-12-18T00:01:42.511220+00:00 node-103 kernel: [974954.919838]
> [<ffffffffc089720a>] ocfs2_inode_lock_full_nested+0x16a/0x920 [ocfs2]
> 2017-12-18T00:01:42.511221+00:00 node-103 kernel: [974954.919850]
> [<ffffffffc0898d6e>] ? ocfs2_extent_map_trunc+0x10e/0x150 [ocfs2]
> 2017-12-18T00:01:42.511222+00:00 node-103 kernel: [974954.919866]
> [<ffffffffc08f9b32>] ocfs2_iop_get_acl+0x52/0x100 [ocfs2]
> 2017-12-18T00:01:42.511223+00:00 node-103 kernel: [974954.919869]
> [<ffffffff812730f1>] get_acl+0x41/0x60
> 2017-12-18T00:01:42.511224+00:00 node-103 kernel: [974954.919872]
> [<ffffffff8121aeab>] generic_permission+0x13b/0x190
> 2017-12-18T00:01:42.511226+00:00 node-103 kernel: [974954.919895]
> [<ffffffffc089aeea>] ocfs2_permission+0xca/0xe0 [ocfs2]
> 2017-12-18T00:01:42.511226+00:00 node-103 kernel: [974954.919897]
> [<ffffffff8121af77>] __inode_permission+0x77/0xc0
> 2017-12-18T00:01:42.511227+00:00 node-103 kernel: [974954.919898]
> [<ffffffff8121afd4>] inode_permission+0x14/0x50
> 2017-12-18T00:01:42.511228+00:00 node-103 kernel: [974954.919900]
> [<ffffffff8121b0fb>] may_open+0x5b/0xf0
> 2017-12-18T00:01:42.511229+00:00 node-103 kernel: [974954.919901]
> [<ffffffff8121efe8>] path_openat+0x188/0x1330
> 2017-12-18T00:01:42.511231+00:00 node-103 kernel: [974954.919903]
> [<ffffffff81221381>] do_filp_open+0x91/0x100
> 2017-12-18T00:01:42.511232+00:00 node-103 kernel: [974954.919904]
> [<ffffffff8122edb6>] ? __alloc_fd+0x46/0x190
> 2017-12-18T00:01:42.511235+00:00 node-103 kernel: [974954.919907]
> [<ffffffff8120f738>] do_sys_open+0x138/0x2a0
> 2017-12-18T00:01:42.511235+00:00 node-103 kernel: [974954.919909]
> [<ffffffff8106b594>] ? __do_page_fault+0x1b4/0x400
> 2017-12-18T00:01:42.511236+00:00 node-103 kernel: [974954.919910]
> [<ffffffff8120f8be>] SyS_open+0x1e/0x20
> 2017-12-18T00:01:42.511238+00:00 node-103 kernel: [974954.919912]
> [<ffffffff818446b2>] entry_SYSCALL_64_fastpath+0x16/0x71
>
>
> -- Jim
>
> On Wed, Dec 27, 2017 at 8:03 PM, Changwei Ge <ge.changwei at h3c.com>
wrote:
>
>> On 2017/12/28 3:02, Jim Okken wrote:
>> > Peter,
>> >
>> > I did not want to flood my first email with details and make it 3
pages
>> long. i gladly will provide more details. first I'd like to ask
that you be
>> less condescending. You have no idea the journey I took toward using
ocfs2
>> in this environment, and also the requirements I needed to meet.
>> > you were amazed and astonished by my question, and I was amazed
and
>> astonished by your answer.
>> >
>> > let's start over:
>> > if ocfs2 isnt the right solution for what I'm doing I can
admit that,
>> and move off of it.
>> > if OpenStack and perhaps newer kernels do not necessarily work
with
>> ocfs2 I can admit that too, and move off of it.
>> > I had high hopes it was the right solution, and at first it did
the job.
>> >
>> > I have a healthy HP MSA 2040 storage appliance connected to via
fiber
>> channel. It has a 7TB storage volume on a fiber channel LUN. From what
I
>> know I need a shared storage filesystem so each of my client systems,
also
>> on the fiber channel network, can access this storage simultaneously
with
>> corrupting data (I need file locking). This HP MSA is healthy and
stable.
>> This isn't exactly local storage I know, but each client system
sees this
>> MSA storage volume as a local drive, ie: /dev/sdb
>> >
>> > what could cause a "lost" wakeup from the OCFS2 lock
manager?
>>
>> Hi Jim,
>> Did a node crash or lose power supply before the stuck stack was found?
>> And is the stuck stack the only one you can find in your kernel log?
>>
>> Thanks,
>> Changwei
>>
>> >
>> > Ubuntu has ocfs2 packages in it's repos. So I hope it has some
level of
>> support in it's OSs and distributed kernels...
>> > I am not well versed in storage concepts but i'll surprise
you, and
>> today my employer (who signs my paycheck) asks me, and tasks me, with
>> making this storage solution work better.
>> >
>> > please let me know if I can provide more details. please let me
know
>> any further comments
>> >
>> > thanks!
>> >
>> > -- Jim
>> >
>> > On Wed, Dec 27, 2017 at 1:16 PM, Peter Grandi <pg at
ocfs.list.sabi.co.uk
>> <mailto:pg at ocfs.list.sabi.co.uk>> wrote:
>> >
>> > > I have a ocfs2 filesystem setup as a shared filesystem
between
>> > > 12 openstack compute nodes which are Ubuntu 16.04.3.
>> >
>> > I am amazed by how unconstrained are the imaginations of some
>> > other people. That is a truly astonishing setup.
>> >
>> > > I have a very big concern of stability. A month ago I
lost a
>> > > good deal of files, I don't know the real reason,
but things
>> > > seemed to point to the ofcs2 cluster.
>> >
>> > That also seems to me unconstrained by concern about mere
>> > details.
>> >
>> > > Last week I found many of my compute nodes with the nova
>> > > service down. The node which went down first has a
"stuck"
>> > > file/directory in the ocfs2 filesystem [ ... ]
>> >
>> > The stack trace seems to point at a "lost" wakeup
from the OCFS2
>> > lock manager.
>> >
>> > > I have other openstack compute nodes that are identical
except
>> > > they use local storage and do not use ocfs2 and these
have
>> > > always been stable.
>> >
>> > But OCFS2 is meant to work with local physical storage on a
>> > local phyical machine. What's your current setup?
>> >
>> > > maybe ocfs2 just isn't stable on Ubuntu 16.04.3? I
am using
>> > > version 1.6.4-3.1
>> >
>> > OCFS2 has been extremely stable for many years on very high
load
>> > share-disk clusters for many users. OpenStack and perhaps
newer
>> > kernels not necessarily so.
>> >
>> > Also OCSF2 requires a storage subsystem with specific features
>> > and a high degree of reliable operation. It is astonishing but
>> > fairly typical that this reports contains no mention of the
>> > setup or of the state of the storage subsystem.
>> >
>> > _______________________________________________
>> > Ocfs2-users mailing list
>> > Ocfs2-users at oss.oracle.com <mailto:Ocfs2-users at
oss.oracle.com>
>> > https://oss.oracle.com/mailman/listinfo/ocfs2-users <
>> https://oss.oracle.com/mailman/listinfo/ocfs2-users>
>> >
>> >
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://oss.oracle.com/pipermail/ocfs2-users/attachments/20180102/32f857d7/attachment-0001.html