HP-UX 11 Corrupted MAC errors

Deron Meranda
Fri, 26 Jul 2002 11:04:30 -0000

Using 3.4p1 under HP-UX 11.0 I am repeatedly getting disconnected with
Corrupted MAC on input.  I am connecting from a RedHat Linux client
(at 3.1p1).

The incorrect MAC is appearing on the server packet receive side.
Never get an invalid MAC on the client side.  I'm currently diving
into packet.c to try to find this, but the behavior is so strange and
predictable I thought I'd see if anybody else has ever seen this.

The strange thing is that the MAC error always occurs when starting an
X application (emacs) using X forwarding, sometime between the X
authentication check and when the window gets mapped (because it never
appears).  I don't get corrupted MAC errors anyplace else.  I've even
forwarded other TCP ports through the ssh session and they cause no
corrupt MACs either...only X sessions.  This happens with about 75%
regularity..in those cases where it successfully gets the window
mapped that channel never has problems no matter how long I use it.
But each additional X forwarding channel I open has about a 75% chance
of a corrupt MAC during or around the X authentication phase.

Enabling or disabling compression has no effect.
Choice of hmac-md5 or hmac-sha1 has no effect.
Choice of cipher aes128-cbc or 3des-cbc has no effect.

I have linked against several OpenSSL versions
  0.9.6d - hp-parisc   (optimized for PA-RISC 1.1, no assembly)
  0.9.6d - hp-parisc2  (optimized for PA-RISC 2.0 with assembly)
  0.9.7beta2 - hp-parisc2

All OpenSSL's pass their tests.  I'm also using these same OpenSSL
libraries in Apache/mod_ssl and have not seen any errors there yet.

Any obvious thoughts before I spend a lot of time tracing through the
packet and crypto code?  The obvious first question to answer; is this
OpenSSL or OpenSSH...I just don't know yet, but I suspect the later.

Deron Meranda