Nikolay Aleksandrov
2020-Apr-29 21:23 UTC
[Bridge] BUG: soft lockup while deleting tap interface from vlan aware bridge
On 4/29/20 11:52 PM, Stefan Priebe - Profihost AG wrote:> Hello, > > while running a stable vanilla kernel 4.19.115 i'm reproducably get this > one: > > watchdog: BUG: soft lockup - CPU#38 stuck for 22s! [bridge:3570653] > > ... > > Call > Trace:nbp_vlan_delete+0x59/0xa0br_vlan_info+0x66/0xd0br_afspec+0x18c/0x1d0br_dellink+0x74/0xd0rtnl_bridge_dellink+0x110/0x220rtnetlink_rcv_msg+0x283/0x360 > ? > rtnl_calcit.isra.29+0x110/0x110netlink_rcv_skb+0xde/0x110netlink_unicast+0x1be/0x250netlink_sendmsg+0x2c1/0x3b0___sys_sendmsg+0x308/0x320 > ? > __sys_sendmsg+0x63/0xa0__sys_sendmsg+0x63/0xa0do_syscall_64+0x5b/0x190entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > full dump below. This only happens if there is enough load on the > machine. I can't reproduce this on a test machine while running only 1-5 > VMs with tap devices enslaved to a bridge. But i was able to shut down 3 > qemu nodes in a row while just executing: > bridge vlan del dev tap123 vid 1-4094 > > Is this a known bug in 4.19.X? > > Thanks for your help. > > Greets, > Stefan >Hi Stefan, It seems something (your email client?) has mangled the email below and has made it harder to understand the traces. I can guess a few ways this might've happened (e.g. way too many fdb entries per-vlan with thousands of vlans, although I've tested with over 10 million and couldn't reach it). Could you please share more about the setup ? How many fdb entries does it have ? How many ports per bridge, how many vlans per port ? Most importantly could you please get a trace on the latest kernel ? Also what does "enough load" mean ? Thank you, Nik> watchdog: BUG: soft lockup - CPU#38 stuck for 22s! [bridge:3570653] > Modules linked in: act_police cls_basic sch_ingress sch_htb netconsole > ebtable_filter ebtables ip6t_REJECT nf_reject_ipv6 ip6table_filter > ip6_tables xt_physdev xt_comment xt_tcpudp xt_mark xt_addrtype > xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_multiport > ipt_REJECT nf_reject_ipv4 xt_set ip_set_hash_net ip_set iptable_filter > bpfilter 8021q garp bonding fuse ipmi_watchdog nfnetlink_log nfnetlink > tcp_htcp sch_fq_codel mgag200 ttm drm_kms_helper dcdbas edac_mce_amd > dell_smbios kvm_amd dell_wmi_descriptor drm kvm fb_sys_fops syscopyarea > sysfillrect ipmi_si irqbypass sysimgblt ipmi_devintf crc32_pclmul > i2c_algo_bit ghash_clmulni_intel sg wmi k10temp ipmi_msghandler button > vhost_net tun vhost tap ip_tables x_tables btrfs zstd_decompress > zstd_compress xxhash raid10 raid456 async_raid6_recov > async_memcpy async_pq async_xor async_tx xor usbhid raid6_pq raid0 > multipath linear raid1 mlx5_core md_mod devlink tg3 sd_mod xhci_pci ptp > dm_mod xhci_hcd nvme i2c_piix4 libphy pps_core nvme_core usbcore ahci > i2c_core usb_common libahci [last unloaded: cpuid] > CPU: 38 PID: 3570653 Comm: bridge Not tainted 4.19.115+53-ph #1 > Hardware name: Dell Inc. PowerEdge R7515/04F3CJ, BIOS 1.2.14 01/28/2020 > RIP: 0010:br_fdb_delete_by_port+0x48/0xb0 > Code: d5 48 89 ef e8 09 51 08 00 49 8b 84 24 98 04 00 00 48 85 c0 75 0a > eb 44 48 85 db 74 3f 48 89 d8 48 83 e8 18 74 36 48 8b 70 08 <48> 8b 58 > 18 4c 39 fe 75 e5 45 85 f6 0f b6 50 28 75 0b f6 c2 02 75 > RSP: 0018:ffffbbc9f21afa58 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff13 > RAX: ffff9cabf1a74a80 RBX: ffff9cabf1a74a98 RCX: 0000000000000000 > RDX: 0000000000000000 RSI: ffff9cae289f8a00 RDI: ffff9baebe6a9844 > RBP: ffff9baebe6a9844 R08: 0000000000000000 R09: ffffffff933d3840 > R10: fffff675235b9800 R11: 0000000000000001 R12: ffff9baebe6a9840 > R13: 0000000000000e23 R14: 0000000000000000 R15: ffff9cad832f5c00 > FS: 00007f446531a540(0000) GS:ffff9baebff80000(0000) knlGS:0000000000000000 > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > CR2: 00007f44652c4443 CR3: 000000127b3f8000 CR4: 0000000000340ee0 > Call > Trace:nbp_vlan_delete+0x59/0xa0br_vlan_info+0x66/0xd0br_afspec+0x18c/0x1d0br_dellink+0x74/0xd0rtnl_bridge_dellink+0x110/0x220rtnetlink_rcv_msg+0x283/0x360 > ? > rtnl_calcit.isra.29+0x110/0x110netlink_rcv_skb+0xde/0x110netlink_unicast+0x1be/0x250netlink_sendmsg+0x2c1/0x3b0___sys_sendmsg+0x308/0x320 > ? > __sys_sendmsg+0x63/0xa0__sys_sendmsg+0x63/0xa0do_syscall_64+0x5b/0x190entry_SYSCALL_64_after_hwframe+0x44/0xa9 > RIP: 0033:0x7f4465252c74 > Code: 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b5 0f 1f 80 00 00 00 00 > 48 8d 05 89 5a 0c 00 8b 00 85 c0 75 13 b8 2e 00 00 00 0f 05 <48> 3d 00 > f0 ff ff 77 54 c3 0f 1f 00 41 54 41 89 d4 55 48 89 f5 53 > RSP: 002b:00007fff564feba8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e > RAX: ffffffffffffffda RBX: 000000005ea9d931 RCX: 00007f4465252c74 > RDX: 0000000000000000 RSI: 00007fff564febf0 RDI: 0000000000000003 > RBP: 00007fff564febf0 R08: 0000000000000004 R09: 0000000000000008 > R10: fffffffffffffb8e R11: 0000000000000246 R12: 00007fff564fec30 > R13: 000056355724f2c0 R14: 00007fff56506ca0 R15: 0000000000000000 > Kernel panic - not syncing: softlockup: hung tasks > CPU: 38 PID: 3570653 Comm: bridge Tainted: G L 4.19.115+53-ph #1 > Hardware name: Dell Inc. PowerEdge R7515/04F3CJ, BIOS 1.2.14 01/28/2020 > Call > Trace:<IRQ>dump_stack+0x66/0x8bpanic+0xe4/0x252watchdog_timer_fn+0x263/0x270 > ? > softlockup_fn+0x40/0x40__hrtimer_run_queues+0xfc/0x270hrtimer_interrupt+0x101/0x240smp_apic_timer_interrupt+0x6a/0x130apic_timer_interrupt+0xf/0x20</IRQ> > RIP: 0010:br_fdb_delete_by_port+0x48/0xb0 > Code: d5 48 89 ef e8 09 51 08 00 49 8b 84 24 98 04 00 00 48 85 c0 75 0a > eb 44 48 85 db 74 3f 48 89 d8 48 83 e8 18 74 36 48 8b 70 08 <48> 8b 58 > 18 4c 39 fe 75 e5 45 85 f6 0f b6 50 28 75 0b f6 c2 02 75 > RSP: 0018:ffffbbc9f21afa58 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff13 > RAX: ffff9cabf1a74a80 RBX: ffff9cabf1a74a98 RCX: 0000000000000000 > RDX: 0000000000000000 RSI: ffff9cae289f8a00 RDI: ffff9baebe6a9844 > RBP: ffff9baebe6a9844 R08: 0000000000000000 R09: ffffffff933d3840 > R10: fffff675235b9800 R11: 0000000000000001 R12: ffff9baebe6a9840 > R13: 0000000000000e23 R14: 0000000000000000 R15: ffff9cad832f5c00 > ? > br_fdb_delete_by_port+0x27/0xb0nbp_vlan_delete+0x59/0xa0br_vlan_info+0x66/0xd0br_afspec+0x18c/0x1d0br_dellink+0x74/0xd0rtnl_bridge_dellink+0x110/0x220rtnetlink_rcv_msg+0x283/0x360 > ? > rtnl_calcit.isra.29+0x110/0x110netlink_rcv_skb+0xde/0x110netlink_unicast+0x1be/0x250netlink_sendmsg+0x2c1/0x3b0___sys_sendmsg+0x308/0x320 > ? > __sys_sendmsg+0x63/0xa0__sys_sendmsg+0x63/0xa0do_syscall_64+0x5b/0x190entry_SYSCALL_64_after_hwframe+0x44/0xa9 > RIP: 0033:0x7f4465252c74 > Code: 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b5 0f 1f 80 00 00 00 00 > 48 8d 05 89 5a 0c 00 8b 00 85 c0 75 13 b8 2e 00 00 00 0f 05 <48> 3d 00 > f0 ff ff 77 54 c3 0f 1f 00 41 54 41 89 d4 55 48 89 f5 53 > RSP: 002b:00007fff564feba8 EFLAGS: 00000246 ORIG_RAX: 000000000000002e > RAX: ffffffffffffffda RBX: 000000005ea9d931 RCX: 00007f4465252c74 > RDX: 0000000000000000 RSI: 00007fff564febf0 RDI: 0000000000000003 > RBP: 00007fff564febf0 R08: 0000000000000004 R09: 0000000000000008 > R10: fffffffffffffb8e R11: 0000000000000246 R12: 00007fff564fec30 > R13: 000056355724f2c0 R14: 00007fff56506ca0 R15: 0000000000000000 > ------------[ cut here ]------------ > WARNING: CPU: 35 PID: 2224815 at kernel/sched/core.c:1168 > set_task_cpu+0x173/0x180 > ------------[ cut here ]------------ > sched: Unexpected reschedule of offline CPU#13! > WARNING: CPU: 46 PID: 0 at arch/x86/kernel/smp.c:128 > native_smp_send_reschedule+0x34/0x40 > ------------[ cut here ]------------ > Modules linked in: > sched: Unexpected reschedule of offline CPU#0!act_police > WARNING: CPU: 33 PID: 2699426 at arch/x86/kernel/smp.c:128 > native_smp_send_reschedule+0x34/0x40cls_basic > Modules linked > in:sch_ingressact_policesch_htbcls_basicnetconsolesch_ingressebtable_filtersch_htbebtablesnetconsoleip6t_REJECTebtable_filternf_reject_ipv6ebtablesip6table_filterip6t_REJECT > ------------[ cut here ]------------ip6_tablesnf_reject_ipv6xt_physdev > sched: Unexpected reschedule of offline CPU#37!ip6table_filterxt_comment > WARNING: CPU: 39 PID: 0 at arch/x86/kernel/smp.c:128 > native_smp_send_reschedule+0x34/0x40ip6_tablesxt_tcpudpxt_physdev > Modules linked > in:xt_markxt_commentact_policext_addrtypext_tcpudpcls_basicxt_conntrackxt_marksch_ingressnf_conntrackxt_addrtypesch_htbnf_defrag_ipv6xt_conntracknetconsolenf_defrag_ipv4nf_conntrackebtable_filterxt_multiportnf_defrag_ipv6ebtablesipt_REJECTnf_defrag_ipv4ip6t_REJECTnf_reject_ipv4xt_multiportnf_reject_ipv6xt_setipt_REJECTip6table_filterip_set_hash_netnf_reject_ipv4ip6_tablesip_setxt_setxt_physdeviptable_filterip_set_hash_netxt_commentbpfilterip_setxt_tcpudp8021qiptable_filterxt_markgarpbpfilterxt_addrtypebonding8021qxt_conntrackfusegarpnf_conntrackipmi_watchdogbondingnf_defrag_ipv6nfnetlink_logfusenf_defrag_ipv4nfnetlinkipmi_watchdogxt_multiporttcp_htcpnfnetlink_logipt_REJECTsch_fq_codelnfnetlinknf_reject_ipv4mgag200tcp_htcpxt_setttmsch_fq_codelip_set_hash_netdrm_kms_helpermgag200ip_setdcdbasttmiptable_filteredac_mce_amddrm_kms_helperbpfilterdell_smbiosdcdbas8021qkvm_amdedac_mce_amdgarpdell_wmi_descriptordell_smbiosbondingdrmkvm_amdfusekvmdell_wmi_descriptoripmi_watchdogfb_sys_fopsdrmnfnetlink_logsyscopyareakvmnfnetlinksysfillrectfb_sys_fopstcp_htcpipmi_sisyscopyareasch_fq_codelirqbypasssysfillrectmgag200sysimgbltipmi_sittmipmi_devintfirqbypassdrm_kms_helpercrc32_pclmulsysimgbltdcdbasi2c_algo_bitipmi_devintfedac_mce_amdghash_clmulni_intelcrc32_pclmuldell_smbiossgi2c_algo_bitkvm_amdwmighash_clmulni_inteldell_wmi_descriptork10tempsgdrmipmi_msghandlerwmikvmbuttonk10temp > ? __x64_sys_futex+0x88/0x180 > RDX: 00047009e9d4a290 RSI: 00000000334bd47f RDI: > 0000000000000000act_policeksys_ioctl+0x70/0x80 > RBP: ffffffff9337a340 R08: 0000000000000002 R09: > 0000000000020100cls_basic__x64_sys_ioctl+0x16/0x20 > R10: 00000000ffffffff R11: 0000000000000081 R12: > ffff9b704145a580sch_ingressdo_syscall_64+0x5b/0x190 > R13: 0000000000000001 R14: 0000000000000001 R15: > 00047009e9d4a290sch_htbentry_SYSCALL_64_after_hwframe+0x44/0xa9netconsoledo_idle+0x1de/0x240 > RIP: 0033:0x7f56000eb777ebtable_filtercpu_startup_entry+0x6f/0x80 > Code: 00 00 90 48 8b 05 19 a7 0c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff > ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 > f0 ff ff 73 01 c3 48 8b 0d e9 a6 0c 00 f7 d8 64 89 01 > 48ebtablesstart_secondary+0x1a7/0x200 > RSP: 002b:00007f55db7fb628 EFLAGS: 00000246ip6t_REJECT > ORIG_RAX: 0000000000000010secondary_startup_64+0xa4/0xb0nf_reject_ipv6 > RAX: ffffffffffffffda RBX: 000000000000ae80 RCX: > 00007f56000eb777ip6table_filter > ---[ end trace 0d5897b92aff0c87 ]--- > RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000025ip6_tables > RBP: 0000000000000000 R08: 00007f55ec0d2330 R09: 00000000ffffffffxt_physdev > R10: 00005585bae5c8e0 R11: 0000000000000246 R12: 00007f55e8d8b000xt_comment > R13: 00007f5604388000 R14: 0000000000000000 R15: 00007f55e8d8b000 > xt_tcpudp xt_mark > ---[ end trace 0d5897b92aff0c88 ]--- > xt_addrtype xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 > xt_multiport ipt_REJECT nf_reject_ipv4 xt_set ip_set_hash_net ip_set > iptable_filter bpfilter 8021q garp bonding fuse ipmi_watchdog > nfnetlink_log nfnetlink tcp_htcp sch_fq_codel mgag200 ttm drm_kms_helper > dcdbas edac_mce_amd dell_smbios kvm_amd dell_wmi_descriptor drm kvm > fb_sys_fops syscopyarea sysfillrect ipmi_si irqbypass sysimgblt > ipmi_devintf crc32_pclmul i2c_algo_bit ghash_clmulni_intel > ------------[ cut here ]------------sg > sched: Unexpected reschedule of offline CPU#1!wmi > WARNING: CPU: 33 PID: 2699426 at arch/x86/kernel/smp.c:128 > native_smp_send_reschedule+0x34/0x40k10temp > Modules linked > in:ipmi_msghandleract_policebuttoncls_basicvhost_netsch_ingresstunsch_htbvhostnetconsoletapebtable_filterip_tableshrtimer_interrupt+0x101/0x240smp_apic_timer_interrupt+0x6a/0x130apic_timer_interrupt+0xf/0x20</IRQ> > RIP: 0010:cpuidle_enter_state+0xb2/0x330 > Code: 64 65 9a 6d e8 ef b3 a5 ff 49 89 c7 0f 1f 44 00 00 31 ff e8 30 c4 > a5 ff 80 7c 24 17 00 0f 85 1c 02 00 00 fb 66 0f 1f 44 00 00 <4c> 2b 7c > 24 08 48 ba cf f7 53 e3 a5 9b c4 20 4c 89 f8 49 c1 ff 3f > RSP: 0018:ffffbbc9d8b03e70 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13 > RAX: ffff9beebfba0840 RBX: ffff9baebe0ff800 RCX: 000000000000001f > RDX: 00047009e9d427d5 RSI: 00000000334bd47f RDI: 0000000000000000 > RBP: ffffffff9337a340 R08: 0000000000000002 R09: 0000000000020100 > R10: 00000000ffffffff R11: 000000000000012f R12: ffff9bafc1ec4b00 > R13: 0000000000000001 R14: 0000000000000001 R15: > 00047009e9d427d5do_idle+0x1de/0x240cpu_startup_entry+0x6f/0x80start_secondary+0x1a7/0x200secondary_startup_64+0xa4/0xb0 > ---[ end trace 0d5897b92aff0c8b ]--- > Modules linked in: act_police cls_basic sch_ingress sch_htb netconsole > ebtable_filter ebtables ip6t_REJECT nf_reject_ipv6 ip6table_filter > ip6_tables xt_physdev xt_comment xt_tcpudp xt_mark xt_addrtype > xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_multiport > ipt_REJECT nf_reject_ipv4 xt_set ip_set_hash_net ip_set iptable_filter > bpfilter 8021q garp bonding fuse ipmi_watchdog nfnetlink_log nfnetlink > tcp_htcp sch_fq_codel mgag200 ttm drm_kms_helper dcdbas edac_mce_amd > dell_smbios kvm_amd dell_wmi_descriptor drm kvm fb_sys_fops syscopyarea > sysfillrect ipmi_si irqbypass sysimgblt ipmi_devintf crc32_pclmul > i2c_algo_bit ghash_clmulni_intel sg wmi k10temp ipmi_msghandler button > vhost_net tun vhost tap ip_tables x_tables btrfs zstd_decompress > zstd_compress xxhash raid10 raid456 async_raid6_recov > WARNING: CPU: 34 PID: 0 at arch/x86/kernel/smp.c:128 > native_smp_send_reschedule+0x34/0x40 > async_memcpy async_pq async_xor async_tx xor usbhid raid6_pq raid0 > multipath linear raid1 mlx5_core md_mod devlink tg3 sd_mod xhci_pci ptp > dm_mod xhci_hcd nvme i2c_piix4 libphy pps_core nvme_core usbcore ahci > i2c_core usb_common libahci [last unloaded: cpuid] > Modules linked in: act_police cls_basic sch_ingress sch_htb netconsole > ebtable_filter ebtables ip6t_REJECT nf_reject_ipv6 ip6table_filter > ip6_tables xt_physdev xt_comment xt_tcpudp xt_mark xt_addrtype > xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_multiport > ipt_REJECT nf_reject_ipv4 xt_set ip_set_hash_net ip_set iptable_filter > bpfilter 8021q garp bonding fuse ipmi_watchdog nfnetlink_log nfnetlink > tcp_htcp sch_fq_codel mgag200 ttm drm_kms_helper dcdbas edac_mce_amd > dell_smbios kvm_amd dell_wmi_descriptor drm kvm fb_sys_fops syscopyarea > sysfillrect ipmi_si irqbypass sysimgblt ipmi_devintf crc32_pclmul > i2c_algo_bit ghash_clmulni_intel sg wmi k10temp ipmi_msghandler button > vhost_net tun vhost tap ip_tables x_tables btrfs zstd_decompress > zstd_compress xxhash raid10 raid456 async_raid6_recov > CPU: 35 PID: 2224815 Comm: kvm Tainted: G W L 4.19.115+53-ph #1 > async_memcpy async_pq async_xor async_tx xor usbhid raid6_pq raid0 > multipath linear raid1 mlx5_core md_mod devlink tg3 sd_mod xhci_pci ptp > dm_mod xhci_hcd nvme i2c_piix4 libphy pps_core nvme_core usbcore ahci > i2c_core usb_common libahci [last unloaded: cpuid] > Hardware name: Dell Inc. PowerEdge R7515/04F3CJ, BIOS 1.2.14 01/28/2020 > RIP: 0010:set_task_cpu+0x173/0x180 > CPU: 34 PID: 0 Comm: swapper/34 Tainted: G W L 4.19.115+53-ph #1 > Code: e9 54 ff ff ff 0f 0b e9 dc fe ff ff 8b 43 60 85 c0 0f 84 e8 fe ff > ff 8b 43 60 83 f8 02 0f 84 dc fe ff ff 0f 0b e9 d5 fe ff ff <0f> 0b e9 > df fe ff ff 66 0f 1f 44 00 00 0f 1f 44 00 00 41 55 41 54 > Hardware name: Dell Inc. PowerEdge R7515/04F3CJ, BIOS 1.2.14 01/28/2020 > RIP: 0010:native_smp_send_reschedule+0x34/0x40 > RSP: 0018:ffff9baebfec3e50 EFLAGS: 00010006 > Code: 05 01 ed 3f 01 73 15 48 8b 05 f8 94 09 01 be fd 00 00 00 48 8b 40 > 30 e9 4a 29 bc 00 89 fe 48 c7 c7 c0 13 03 93 e8 ec 0d 05 00 <0f> 0b c3 > 66 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 53 48 83 ec 20 > RAX: 0000000000000200 RBX: ffff9be89db94b00 RCX: ffff9be89db94b00 > RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff9be89db94b00 > RSP: 0018:ffff9baebfe837f8 EFLAGS: 00010082 > RBP: ffff9be89db94e68 R08: 0000000000000000 R09: 0000000000000000 > R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000003 > RAX: 0000000000000000 RBX: ffff9ca1948da580 RCX: 0000000000000006 > RDX: 0000000000000007 RSI: 0000000000000092 RDI: ffff9baebfe955c0 > R13: 0000000000000003 R14: 0000000000000046 R15: 0000000000000003 > FS: 0000000000000000(0000) GS:ffff9baebfec0000(0000) knlGS:0000000000000000 > RBP: 0000000000020840 R08: 0000000000000001 R09: 0000000000000000 > R10: 00000000f4240000 R11: 000000000000002f R12: ffff9ca1948db0cc > ------------[ cut here ]------------ > sched: Unexpected reschedule of offline CPU#15! > WARNING: CPU: 35 PID: 2224815 at arch/x86/kernel/smp.c:128 > native_smp_send_reschedule+0x34/0x40 > Modules linked in: act_police cls_basic sch_ingress sch_htb netconsole > ebtable_filter ebtables ip6t_REJECT nf_reject_ipv6 ip6table_filter > ip6_tables xt_physdev xt_comment xt_tcpudp xt_mark xt_addrtype > xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_multiport > ipt_REJECT nf_reject_ipv4 xt_set ip_set_hash_net ip_set iptable_filter > bpfilter 8021q garp bonding fuse ipmi_watchdog nfnetlink_log nfnetlink > tcp_htcp sch_fq_codel mgag200 ttm drm_kms_helper dcdbas edac_mce_amd > dell_smbios kvm_amd dell_wmi_descriptor drm kvm fb_sys_fops syscopyarea > sysfillrect ipmi_si irqbypass sysimgblt ipmi_devintf crc32_pclmul > i2c_algo_bit ghash_clmulni_intel sg wmi k10temp ipmi_msghandler button > vhost_net tun vhost tap ip_tables x_tables btrfs zstd_decompress > zstd_compress xxhash raid10 raid456 async_raid6_recov > async_memcpy async_pq async_xor async_tx xor usbhid raid6_pq raid0 > multipath linear raid1 mlx5_core md_mod devlink tg3 sd_mod xhci_pci ptp > dm_mod xhci_hcd nvme i2c_piix4 libphy pps_core nvme_core usbcore ahci > i2c_core usb_common libahci [last unloaded: cpuid] > CPU: 35 PID: 2224815 Comm: kvm Tainted: G W L 4.19.115+53-ph #1 > Hardware name: Dell Inc. PowerEdge R7515/04F3CJ, BIOS 1.2.14 01/28/2020 > RIP: 0010:native_smp_send_reschedule+0x34/0x40 > Code: 05 01 ed 3f 01 73 15 48 8b 05 f8 94 09 01 be fd 00 00 00 48 8b 40 > 30 e9 4a 29 bc 00 89 fe 48 c7 c7 c0 13 03 93 e8 ec 0d 05 00 <0f> 0b c3 > 66 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 53 48 83 ec 20 > RSP: 0018:ffff9baebfec3d78 EFLAGS: 00010086 > RAX: 0000000000000000 RBX: ffff9bae9a8fcb00 RCX: 0000000000000006 > RDX: 0000000000000007 RSI: 0000000000000082 RDI: ffff9baebfed55c0 > RBP: 0000000000020840 R08: 0000000000000001 R09: 0000000000000000 > R10: ffff9baebfec3d28 R11: 0000000000000034 R12: ffff9bae9a8fd64c > R13: 0000000000000004 R14: 0000000000000046 R15: 000000000000000f > FS: 0000000000000000(0000) GS:ffff9baebfec0000(0000) knlGS:0000000000000000 > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > CR2: 00007efdd36ffab8 CR3: 0000012c3faa2000 CR4: 0000000000340ee0 > Call > Trace:<IRQ>try_to_wake_up+0x3fc/0x510autoremove_wake_function+0x11/0x50__wake_up_common+0x96/0x180__wake_up_common_lock+0x7c/0xc0irq_work_run_list+0x50/0x80 > ? > tick_sched_do_timer+0x80/0x80update_process_times+0x3b/0x50tick_sched_handle+0x22/0x60tick_sched_timer+0x37/0x70__hrtimer_run_queues+0xfc/0x270hrtimer_interrupt+0x101/0x240smp_apic_timer_interrupt+0x6a/0x130apic_timer_interrupt+0xf/0x20</IRQ> > RIP: 0010:svm_handle_external_intr+0xc/0x20 [kvm_amd] > Code: 83 0c 03 00 00 20 75 02 5b c3 48 89 df 5b e9 cb e4 ff ff 90 66 2e > 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 fb 66 0f 1f 44 00 00 <90> fa 66 > 0f 1f 44 00 00 c3 90 66 2e 0f 1f 84 00 00 00 00 00 0f 1f > RSP: 0018:ffffbbc9f1117d18 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13 > RAX: ffffffffc0930790 RBX: ffff9c7f9d2c0000 RCX: 0000000100000000 > RDX: 000b133f62ad4458 RSI: fff6d37b61b9824a RDI: ffff9c7f9d2c0000 > RBP: ffffbbc9f1117da0 R08: 0000000000000000 R09: 0000000000000000 > R10: 0000000000000000 R11: 0000000000000000 R12: 000b133f62ad4458 > R13: 0000000000000000 R14: ffff9c7f9d2c4328 R15: ffff9be510f24000 > ? svm_sync_dirty_debug_regs+0xc0/0xc0 [kvm_amd] > vcpu_enter_guest+0x3e9/0x1520 [kvm] > ? ktime_get+0x3e/0xb0 > ? __apic_accept_irq+0x2ae/0x320 [kvm] > ? kvm_arch_vcpu_ioctl_run+0x2f6/0x520 [kvm] > kvm_arch_vcpu_ioctl_run+0x2f6/0x520 [kvm] > kvm_vcpu_ioctl+0x388/0x5d0 [kvm] > ? do_futex+0x12c/0x5d0do_vfs_ioctl+0xa2/0x640 > Kernel Offset: 0x11000000 from 0xffffffff81000000 (relocation range: > 0xffffffff80000000-0xffffffffbfffffff) > Rebooting in 20 seconds.. >