openssh unix dev - Feb 2016 - [Bug] Regression problem in transfer.sh for OpenSSH 7.1 P2 on HPE NSE above dd-size 32k

G'Day,

I am requesting help in resolving an issue on the HPE NonStop platform in
OpenSSH 7.1 P2 in the regression suite for all dd-size above 32k. Previous
tests are all passing, but in the for-loop inside regress/transfer.sh, when
s is 64k, the command:

                dd if=$DATA obs=${s} 2> /dev/null | \
                        ${SSH} -q -$p -F $OBJ/ssh_proxy somehost "cat >
${COPY}"

does not transfer any data. regress/copy is empty and the test fails. I have
checked the platform's API for read/write code, which has no issue
transferring data at 1Mb - so that's probably not it. Is there some ioctl or
other system call might be new that may be falling in this specific
situation? I have been looking for math problems but have not found any -
int/long/size_t are 32bit on the box. 56k is a common boundary in some
functions, but I don't get what OpenSSH would be doing in this regard. Our
previous version, 6.7 based at commit 28453d5, did not have this problem,
and regress/transfer.sh does not have any substantively differences compared
to back then. The code is built using OpenSSL 1.0.2f.

Getting detailed debug logs of the regression test would also really help
here. Anyone have any pointers on where I can look to try to resolve this?

Thanks,

Randall
--
-- Randall Becker
-- Brief whoami: NonStop&UNIX developer since approximately
UNIX(421664400)/NonStop(211288444200000000)
-- In my real life, I talk too much.

On Tue, Feb 9, 2016 at 6:36 AM, Randall S. Becker
<rsbecker at nexbridge.com> wrote:
[...]
> Getting detailed debug logs of the regression test would also really help
> here. Anyone have any pointers on where I can look to try to resolve this?
The logs should be in the regress directory.  There should be ssh.log
and ssh.log for the currently running test and failed-ssh.log and
failed-sshd.log containing the concatenation of the logs from any
failed tests.  You can run "make t-exec LTESTS=transfer" to run just
that single test while you're playing around to save some time.

I have seen something similar on old FreeBSD systems when the test was
run on an NFS mount.  I never figured out why this was, but running
the test on local disk worked in that case.

-- 
Darren Tucker (dtucker at zip.com.au)
GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4  37C9 C982 80C7 8FF4 FA69
    Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.

On February 8, 2016 3:30 PM, Darren Tucker wrote:
> To: Randall S. Becker <rsbecker at nexbridge.com>
> Cc: OpenSSH Devel List <openssh-unix-dev at mindrot.org>
> Subject: Re: [Bug] Regression problem in transfer.sh for OpenSSH 7.1 P2 on
> HPE NSE above dd-size 32k
> 
> On Tue, Feb 9, 2016 at 6:36 AM, Randall S. Becker
> <rsbecker at nexbridge.com> wrote:
> [...]
> > Getting detailed debug logs of the regression test would also really
> > help here. Anyone have any pointers on where I can look to try to
resolve
> this?
> 
> The logs should be in the regress directory.  There should be ssh.log and
> ssh.log for the currently running test and failed-ssh.log and
failed-sshd.log
> containing the concatenation of the logs from any failed tests.  You can
run
> "make t-exec LTESTS=transfer" to run just that single test while
you're
playing
> around to save some time.
> 
> I have seen something similar on old FreeBSD systems when the test was run
> on an NFS mount.  I never figured out why this was, but running the test
on
> local disk worked in that case.
I've got those logs. Unfortunately, hoping for more details. The logs do
show the test failure, but no details in failed-sshd.log indicating a
specific problem. Some of the formats are a bit wonky but I can live with
those. Since you mentioned NFS, which could have been constrained by UDP
packet sizes, and this platform sometimes cannot read beyond 56Kb off disk
(in some situations), it may be the read of regress/data that is failing.
Any pointers where that is hiding so that I can verify that data is actually
getting into sshd from the disk?

Cheers,
Randall

On February 8, 2016 3:30 PM, Darren Tucker wrote:
> To: Randall S. Becker <rsbecker at nexbridge.com>
> Cc: OpenSSH Devel List <openssh-unix-dev at mindrot.org>
> Subject: Re: [Bug] Regression problem in transfer.sh for OpenSSH 7.1 P2 on
> HPE NSE above dd-size 32k
> 
> On Tue, Feb 9, 2016 at 6:36 AM, Randall S. Becker
> <rsbecker at nexbridge.com> wrote:
> [...]
> > Getting detailed debug logs of the regression test would also really
> > help here. Anyone have any pointers on where I can look to try to
resolve
> this?
> 
> The logs should be in the regress directory.  There should be ssh.log and
> ssh.log for the currently running test and failed-ssh.log and
failed-sshd.log
> containing the concatenation of the logs from any failed tests.  You can
run
> "make t-exec LTESTS=transfer" to run just that single test while
you're
playing
> around to save some time.
> 
> I have seen something similar on old FreeBSD systems when the test was run
> on an NFS mount.  I never figured out why this was, but running the test
on
> local disk worked in that case.
So here's where the 5% comes in. The dd code explicitly checks SSIZE_MAX on
the platform and restricts the obs values that are legitimate. From limits.h
on NSE:
#define SSIZE_MAX        53248    /* max single I/O size, 52K       */
The result is that dd is failing parameter checks above 32k, so 64k, 128k,
and 256k are invalid and should have been omitted on the platform in any
event. Is this a legit bug in the OpenSSH regression suite?

Cheers,
Randall