As I'm about to commit my AES-GCM work, I've been trying to do
some testing to make sure I didn't break IPsec.
The first major issue I ran across was transport mode... ae@ has been
nice enough to get ICMP working in transport mode for IPv4 and IPv6,
but it looks like TCP is still broken. I haven't tested UDP yet...
So, IPsec even w/o crypto is fundamentally broken here... It's clear
that not many people run transport mode...
If someone could create a good test suite that ensures makes sure basic
IPsec traffic passes, that would be a huge win for us. The tests
should test a complete cross product of: { tunnel, transport } x
{ TCP, UDP, ICMP, any others? } x { IPv4, IPv6 }. Please add to this
list.
There is a lock order reversal which someone should look at:
# setkey -DP
lock order reversal:
1st 0xffffffff817fc658 sptree (fast ipsec security policy database) @
netipsec/key.c:2453
2nd 0xffffffff818bd920 rawcb (rawcb) @ netipsec/keysock.c:303
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe001de9d330
kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe001de9d3e0
witness_checkorder() at witness_checkorder+0xdad/frame 0xfffffe001de9d470
__mtx_lock_flags() at __mtx_lock_flags+0xa8/frame 0xfffffe001de9d4c0
key_sendup_mbuf() at key_sendup_mbuf+0xb0/frame 0xfffffe001de9d500
key_spddump() at key_spddump+0x188/frame 0xfffffe001de9d540
key_parse() at key_parse+0x8fa/frame 0xfffffe001de9d740
key_output() at key_output+0xeb/frame 0xfffffe001de9d7a0
sosend_generic() at sosend_generic+0x405/frame 0xfffffe001de9d850
kern_sendit() at kern_sendit+0x20b/frame 0xfffffe001de9d900
sendit() at sendit+0x129/frame 0xfffffe001de9d950
sys_sendto() at sys_sendto+0x4d/frame 0xfffffe001de9d9a0
amd64_syscall() at amd64_syscall+0x25a/frame 0xfffffe001de9dab0
Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe001de9dab0
--- syscall (133, FreeBSD ELF64, sys_sendto), rip = 0x800b5c5aa, rsp =
0x7fffffff6ae8, rbp = 0x7fffffffeb50 ---
The following is more OpenCrypto or aesni related, but as IPsec is a
consumer, they are related.
It also looks like OpenCrypto session management wasn't on anyone's
list, partly because the docs for it aren't very good, and until
AES-NI came along, if you provided an explicit IV, the software stack
would "magicly" work. Now w/ AES-NI, that is no longer the case...
Currently, the aesni driver only has one FPU context allocated per
session, but IPsec will reuse the same session for different requests
from different threads. This means that you can get panics like:
panic: System call sendto returing with kernel FPU ctx leaked
or:
panic: dummy ctx
As the same session fpu context will be assigned to different kernel
threads stomping on each other's state. Both transport and tunnel
mode suffer (panics) from this issue. I'm working w/ kib on a
possible solution to this.
There are other issues, such as all the session management code is
looked up via linked lists... This isn't good for performance... One
of the reasons why this was done was to allow migrating sessions between
capable devices, such that if one was overloaded/busy, you could use
another one that isn't as busy... There are better solutions/patterns
now that weren't used back then that should get us around this issue..
--
John-Mark Gurney Voice: +1 415 225 5579
"All that I will do, has been done, All that I have, has not."
On 21.11.2014 00:35, John-Mark Gurney wrote:> As I'm about to commit my AES-GCM work, I've been trying to do > some testing to make sure I didn't break IPsec. > > The first major issue I ran across was transport mode... ae@ has been > nice enough to get ICMP working in transport mode for IPv4 and IPv6, > but it looks like TCP is still broken. I haven't tested UDP yet... > So, IPsec even w/o crypto is fundamentally broken here... It's clear > that not many people run transport mode... > > If someone could create a good test suite that ensures makes sure basic > IPsec traffic passes, that would be a huge win for us. The tests > should test a complete cross product of: { tunnel, transport } x > { TCP, UDP, ICMP, any others? } x { IPv4, IPv6 }. Please add to this > list.I usually do tests for both transport and tunnel modes with and without gif(4)/gre(4). So, just tried between two CURRENT hosts and it works. I use racoon and isakmpd for IKE. ICMP, TCP (ssh) and UDP (ike) works for me. How do you test? Do you use software crypto or aesni? -- WBR, Andrey V. Elsukov -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 538 bytes Desc: OpenPGP digital signature URL: <http://lists.freebsd.org/pipermail/freebsd-security/attachments/20141121/77888051/attachment.sig>
Andrey V. Elsukov wrote this message on Fri, Nov 21, 2014 at 01:20 +0300:> On 21.11.2014 00:35, John-Mark Gurney wrote: > > As I'm about to commit my AES-GCM work, I've been trying to do > > some testing to make sure I didn't break IPsec. > > > > The first major issue I ran across was transport mode... ae@ has been > > nice enough to get ICMP working in transport mode for IPv4 and IPv6, > > but it looks like TCP is still broken. I haven't tested UDP yet... > > So, IPsec even w/o crypto is fundamentally broken here... It's clear > > that not many people run transport mode... > > > > If someone could create a good test suite that ensures makes sure basic > > IPsec traffic passes, that would be a huge win for us. The tests > > should test a complete cross product of: { tunnel, transport } x > > { TCP, UDP, ICMP, any others? } x { IPv4, IPv6 }. Please add to this > > list. > > I usually do tests for both transport and tunnel modes with and without > gif(4)/gre(4). So, just tried between two CURRENT hosts and it works. > I use racoon and isakmpd for IKE. ICMP, TCP (ssh) and UDP (ike) works > for me. How do you test? Do you use software crypto or aesni?Hmm... weird... Just tested again and TCP seems to be working now... Not sure what changed... It could be that I didn't retest after fixing AES-NI's mbuf issue, but I thought I had... Though I thought I had tested a clean HEAD too... I've only been testing w/ static associations to make testing easier.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."
On 21/11/2014 8:35 AM, John-Mark Gurney wrote:> As I'm about to commit my AES-GCM work, I've been trying to do > some testing to make sure I didn't break IPsec. > > The first major issue I ran across was transport mode... ae@ has been > nice enough to get ICMP working in transport mode for IPv4 and IPv6, > but it looks like TCP is still broken. I haven't tested UDP yet... > So, IPsec even w/o crypto is fundamentally broken here... It's clear > that not many people run transport mode... > > If someone could create a good test suite that ensures makes sure basic > IPsec traffic passes, that would be a huge win for us. The tests > should test a complete cross product of: { tunnel, transport } x > { TCP, UDP, ICMP, any others? } x { IPv4, IPv6 }. Please add to this > list. > > There is a lock order reversal which someone should look at: > # setkey -DP > lock order reversal: > 1st 0xffffffff817fc658 sptree (fast ipsec security policy database) @ netipsec/key.c:2453 > 2nd 0xffffffff818bd920 rawcb (rawcb) @ netipsec/keysock.c:303 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe001de9d330 > kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe001de9d3e0 > witness_checkorder() at witness_checkorder+0xdad/frame 0xfffffe001de9d470 > __mtx_lock_flags() at __mtx_lock_flags+0xa8/frame 0xfffffe001de9d4c0 > key_sendup_mbuf() at key_sendup_mbuf+0xb0/frame 0xfffffe001de9d500 > key_spddump() at key_spddump+0x188/frame 0xfffffe001de9d540 > key_parse() at key_parse+0x8fa/frame 0xfffffe001de9d740 > key_output() at key_output+0xeb/frame 0xfffffe001de9d7a0 > sosend_generic() at sosend_generic+0x405/frame 0xfffffe001de9d850 > kern_sendit() at kern_sendit+0x20b/frame 0xfffffe001de9d900 > sendit() at sendit+0x129/frame 0xfffffe001de9d950 > sys_sendto() at sys_sendto+0x4d/frame 0xfffffe001de9d9a0 > amd64_syscall() at amd64_syscall+0x25a/frame 0xfffffe001de9dab0 > Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe001de9dab0 > --- syscall (133, FreeBSD ELF64, sys_sendto), rip = 0x800b5c5aa, rsp = 0x7fffffff6ae8, rbp = 0x7fffffffeb50 --- > > > The following is more OpenCrypto or aesni related, but as IPsec is a > consumer, they are related. > > It also looks like OpenCrypto session management wasn't on anyone's > list, partly because the docs for it aren't very good, and until > AES-NI came along, if you provided an explicit IV, the software stack > would "magicly" work. Now w/ AES-NI, that is no longer the case... > > Currently, the aesni driver only has one FPU context allocated per > session, but IPsec will reuse the same session for different requests > from different threads. This means that you can get panics like: > panic: System call sendto returing with kernel FPU ctx leaked > or: > panic: dummy ctx > > As the same session fpu context will be assigned to different kernel > threads stomping on each other's state. Both transport and tunnel > mode suffer (panics) from this issue. I'm working w/ kib on a > possible solution to this. > > There are other issues, such as all the session management code is > looked up via linked lists... This isn't good for performance... One > of the reasons why this was done was to allow migrating sessions between > capable devices, such that if one was overloaded/busy, you could use > another one that isn't as busy... There are better solutions/patterns > now that weren't used back then that should get us around this issue.. >John-Mark, I'm unsure how I can help, only to say that I've been using ipsec over IPv4 between multiple customer sites (branch to Head Offices) since FreeBSD 4.9, they're now 9.2R. We're using both transport and tunnel modes, and use TCP, UDP, ICMP between devices. I intend to migrate them from 9.2 to 10.1Stable within a fortnight, though we're only using IPv4. We also use strongswan for iphone (IKEv1) and Windows7 (IKEv2) VPN connections, ipsec works ok. All boundary devices and the VPN server use VIA-padlock chips (C7). Kind regards, Dewayne.
On 20 Nov 2014, at 21:35 , John-Mark Gurney <jmg at funkthat.com> wrote:> The first major issue I ran across was transport mode... ae@ has been > nice enough to get ICMP working in transport mode for IPv4 and IPv6, > but it looks like TCP is still broken. I haven't tested UDP yet... > So, IPsec even w/o crypto is fundamentally broken here... It's clear > that not many people run transport mode?Or that some people lately broke the code.> If someone could create a good test suite that ensures makes sure basic > IPsec traffic passes, that would be a huge win for us. The tests > should test a complete cross product of: { tunnel, transport } x > { TCP, UDP, ICMP, any others? } x { IPv4, IPv6 }. Please add to this > list.+ SCTP + IPcomp + IPv4/IPv6 mixes inside/outside + properly working enc(4) If you get all that then add IPIP (gif) and GRE on the inside as well. ? Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames, 1983