bugzilla-daemon at netfilter.org
2018-Apr-23 17:50 UTC
[Bug 1247] New: Module crash due to broken count
https://bugzilla.netfilter.org/show_bug.cgi?id=1247
Bug ID: 1247
Summary: Module crash due to broken count
Product: netfilter/iptables
Version: unspecified
Hardware: other
OS: other
Status: NEW
Severity: blocker
Priority: P5
Component: unknown
Assignee: netfilter-buglog at lists.netfilter.org
Reporter: netfilternetfilter at gmail.com
The configuration I use is a NixOS keepalived setup on AWS (which uses Xen)
configured with per packet round robin routing in a direct routing setup, which
in turn uses ipvs.
Symptoms are ksoftirqd using 100% of the CPU resulting in what looks like a DoS
attack on the machine.
The module repeatedly crashes when in this mode with a message similar to the
one below.
[root at keepalive:~]# uname -a
Linux keepalive 4.15.17 #1-NixOS SMP Thu Apr 12 10:31:21 UTC 2018 x86_64
GNU/Linux
It is the same problem as described on
https://github.com/NixOS/nixpkgs/issues/39078.
[ 782.525457] ------------[ cut here ]------------
[ 782.525467] refcount_t hit zero at ip_vs_conn_put+0x31/0x40 [ip_vs] in
sh[15519], uid/euid: 497/497
[ 782.525477] WARNING: CPU: 0 PID: 15519 at ../kernel/panic.c:657
refcount_error_report+0x94/0x9e
[ 782.525477] Modules linked in: ip_vs_rr cirrus ttm sb_edac edac_core
drm_kms_helper crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcbc mousedev
drm aesni_intel aes_x86_64 crypto_simd glue_helper cryptd psmouse evdev
input_leds led_class intel_agp fb_sys_fops syscopyarea sysfillrect
intel_rapl_perf mac_hid intel_gtt serio_raw sysimgblt agpgart i2c_piix4
i2c_core ata_generic pata_acpi floppy cfg80211 rfkill button loop macvlan ip_vs
nf_conntrack libcrc32c crc32c_generic ip_tables x_tables ipv6 crc_ccitt autofs4
ext4 crc16 mbcache jbd2 fscrypto ata_piix libata atkbd libps2 scsi_mod
crc32c_intel i8042 rtc_cmos serio af_packet dm_mod dax fuse xen_netfront
xen_blkfront
[ 782.525532] CPU: 0 PID: 15519 Comm: sh Tainted: G W 4.15.17
#1-NixOS
[ 782.525533] Hardware name: Xen HVM domU, BIOS 4.2.amazon 08/24/2006
[ 782.525535] RIP: 0010:refcount_error_report+0x94/0x9e
[ 782.525536] RSP: 0000:ffffa344dde039c8 EFLAGS: 00010296
[ 782.525537] RAX: 0000000000000057 RBX: ffffffff92f20e06 RCX:
0000000000000006
[ 782.525541] RDX: 0000000000000007 RSI: 0000000000000086 RDI:
ffffa344dde165c0
[ 782.525541] RBP: ffffa344dde03b08 R08: 0000000000000218 R09:
0000000000000004
[ 782.525542] R10: ffffffff93006a80 R11: 0000000000000001 R12:
ffffa344d68cd100
[ 782.525543] R13: 00000000000001f1 R14: ffffffff92f12fb0 R15:
0000000000000004
[ 782.525544] FS: 00007fc9d2040fc0(0000) GS:ffffa344dde00000(0000)
knlGS:0000000000000000
[ 782.525545] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 782.525546] CR2: 000000000262a000 CR3: 0000000016a0c004 CR4:
00000000001606f0
[ 782.525548] Call Trace:
[ 782.525551] <IRQ>
[ 782.525557] ex_handler_refcount+0x4e/0x80
[ 782.525558] fixup_exception+0x33/0x40
[ 782.525563] do_trap+0x83/0x140
[ 782.525565] do_error_trap+0x83/0xf0
[ 782.525568] ? ip_vs_conn_drop_conntrack+0x120/0x1a5 [ip_vs]
[ 782.525574] ? ip_finish_output2+0x29c/0x390
[ 782.525575] ? ip_finish_output2+0x1a2/0x390
[ 782.525580] invalid_op+0x1b/0x40
[ 782.525582] RIP: 0010:ip_vs_conn_put+0x31/0x40 [ip_vs]
[ 782.525583] RSP: 0000:ffffa344dde03bb8 EFLAGS: 00010246
[ 782.525584] RAX: 0000000000000001 RBX: ffffa344df31cf00 RCX:
ffffa344d7450198
[ 782.525585] RDX: 0000000000000003 RSI: 00000000fffffe01 RDI:
ffffa344d7450140
[ 782.525586] RBP: 0000000000000002 R08: 0000000000000476 R09:
0000000000000000
[ 782.525587] R10: ffffa344dde03b28 R11: ffffa344df200000 R12:
ffffa344d7d09000
[ 782.525587] R13: ffffa344def3a980 R14: ffffffffc04f6e20 R15:
0000000000000008
[ 782.525591] ip_vs_in.part.29.constprop.36+0x34f/0x640 [ip_vs]
[ 782.525593] ? ip_vs_conn_out_get+0xe0/0xe0 [ip_vs]
[ 782.525596] ip_vs_remote_request4+0x47/0xa0 [ip_vs]
[ 782.525598] ? ip_vs_in.part.29.constprop.36+0x640/0x640 [ip_vs]
[ 782.525600] nf_hook_slow+0x43/0xc0
[ 782.525602] ip_local_deliver+0xac/0xc0
[ 782.525604] ? ip_rcv_finish+0x400/0x400
[ 782.525606] ip_rcv+0x26c/0x380
[ 782.525610] __netif_receive_skb_core+0x3a0/0xb10
[ 782.525629] ? inet_gro_receive+0x23c/0x2b0
[ 782.525631] ? netif_receive_skb_internal+0x24/0xb0
[ 782.525633] netif_receive_skb_internal+0x24/0xb0
[ 782.525635] napi_gro_receive+0xb8/0xe0
[ 782.525638] xennet_poll+0x676/0xb40 [xen_netfront]
[ 782.525641] net_rx_action+0x139/0x3a0
[ 782.525644] __do_softirq+0xde/0x2b4
[ 782.525646] irq_exit+0xae/0xb0
[ 782.525651] xen_evtchn_do_upcall+0x2c/0x40
[ 782.525653] xen_hvm_callback_vector+0x7d/0x90
[ 782.525654] </IRQ>
[ 782.525656] RIP: 0033:0x7fc9d11c91f9
[ 782.525656] RSP: 002b:00007ffebe8a2ea0 EFLAGS: 00000202 ORIG_RAX:
ffffffffffffff0c
[ 782.525658] RAX: 00000000ffffffff RBX: 0000000002609808 RCX:
0000000000000054
[ 782.525658] RDX: 0000000000000001 RSI: 0000000002605440 RDI:
00000000025f940e
[ 782.525659] RBP: 00000000025f940e R08: 000000000260213d R09:
1999999999999999
[ 782.525660] R10: 000000000262a808 R11: 00000000025f942d R12:
00000000025f940e
[ 782.525661] R13: 00007fc9d1301e20 R14: 00000000025f9408 R15:
00007fc9d1302720
[ 782.525662] Code: 48 8b 95 80 00 00 00 41 55 49 8d 8c 24 e0 05 00 00 45 8b
84 24 38 04 00 00 41 89 c1 48 89 de 48 c7 c7 a8 2f f2 92 e8 7c fa ff ff
<0f> 0b
58 5b 5d 41 5c 41 5d c3 0f 1f 44 00 00 55 48 89 e5 41 56
[ 782.525684] ---[ end trace 429a7a27ee858cfb ]---
--
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20180423/0209d4c7/attachment.html>
bugzilla-daemon at netfilter.org
2018-Apr-23 17:51 UTC
[Bug 1247] Module crash due to broken count
https://bugzilla.netfilter.org/show_bug.cgi?id=1247
Net Filter <netfilternetfilter at gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |netfilternetfilter at gmail.co
| |m
--
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20180423/ebf83201/attachment.html>
bugzilla-daemon at netfilter.org
2018-Apr-23 18:22 UTC
[Bug 1247] Module crash due to broken count
https://bugzilla.netfilter.org/show_bug.cgi?id=1247
Pablo Neira Ayuso <pablo at netfilter.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |pablo at netfilter.org
--- Comment #1 from Pablo Neira Ayuso <pablo at netfilter.org> ---
Please, report this bugzilla ticket to lvs-devel mailing list:
http://www.linuxvirtualserver.org/mailing.html
--
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20180423/dc26fa4e/attachment.html>
bugzilla-daemon at netfilter.org
2018-Apr-23 19:24 UTC
[Bug 1247] Module crash due to broken count
https://bugzilla.netfilter.org/show_bug.cgi?id=1247 --- Comment #2 from Net Filter <netfilternetfilter at gmail.com> --- I reported it to the lvs-devel mailing list as requested. -- You are receiving this mail because: You are watching all bug changes. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20180423/42e2cd39/attachment-0001.html>