openssh bugs - Feb 2011 - [Bug 1213] ssh-keyscan exits in mid-way

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

Andreas Kotes <count-mindrot at flatline.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |count-mindrot at flatline.de
           Severity|normal                      |major

--- Comment #6 from Andreas Kotes <count-mindrot at flatline.de>
2011-02-18 02:36:54 EST ---
I'm running into the same problem on recent versions.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

--- Comment #7 from Andreas Kotes <count-mindrot at flatline.de>
2011-02-18 05:31:44 EST ---
btw: I've elevated this to 'major', as it completely breaks the
usefulness for ssh-keyscan in large networks, as the error condition
(len == 0 in packet_read_seqnr() in packet.c; resulting in
logit("Connection closed ... etc") and cleanup_exit(255);) is much
easier to hit. On 10 runs of ssh-keyscan over ~3800 IPs I couldn't get
a single complete run without hitting this. Please fix.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

--- Comment #8 from aab at purdue.edu 2011-02-18 10:58:23 EST ---
Mr. Kotes, I have a patch against openssh-5.[678]p1 for our problem
that could be called a workaround or a fix depending on your way of
looking at it.  The probable reason that `packet_read_seqnr()' gets the
len==0 is that one of the IPs from which your attempting to get a key
has a bad `sshd' server that times out because of the
"LoginGraceTime".
 This, in turn, causes almost all of the other servers that have open
sockets at that time to "LoginGraceTime" out as well.  To back up a
bit, `packet_read_seqnr()' calls the vanilla `cleanup_exit()' that in
the current ssh-keyscan aborts immediately rather than continuing like
ssh-keyscan's `fatal()' call does.  This is part 1 of the fix.  The
second part is to teach ssh-keyscan how to deal with the problem when a
bad server times out.  My patch does both although the code seems a bit
kludgy to me.

Unfortunately, we haven't had a bad server recently so I can't
completely test the patch (I'm using it in test mode now) and, until
then, I don't want to send it to the OpenSSH folks.  FWIW - our host
farm is 3500+ with an additional 1200+ to be online soon and probably
more in the late summer.

In my opioion, this should be marked as a bug against the current
openssh variant.  How do I go about doing that?

If you'd like to have a copy of the current patch so you can test it,
please tell me where to send it.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

aab at purdue.edu changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|4.3p2                       |5.8p1
          Component|Miscellaneous               |ssh-keygen

--- Comment #9 from aab at purdue.edu 2011-02-18 11:04:12 EST ---
I've noted that this is a ssh-keyscan bug and I've attached it to the
openssh-5.8p1 release.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

aab at purdue.edu changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|ssh-keygen                  |Miscellaneous

--- Comment #10 from aab at purdue.edu 2011-02-18 11:07:01 EST ---
Oops, can't read.  ssh-keygen ain't ssh-keyscan.  Changed the component
back to Miscellaneous.  Hey, isn't ssh-keyscan a component also?

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

Daniel Richard G. <skunk at iSKUNK.ORG> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |skunk at iSKUNK.ORG

--- Comment #11 from Daniel Richard G. <skunk at iSKUNK.ORG> 2011-02-24
02:55:54 EST ---
I reported this a while ago on the Ubuntu Launchpad bug tracker:

    https://bugs.launchpad.net/openssh/+bug/483928

I've also confirmed that the bug persists in OpenSSH 5.8p1, and I gave
your patch a try to scan a corporate network of 6000+ hosts.

Most of the hosts don't appear to be running SSH, but I can't be sure
if that's really the case, or if ssh-keyscan(1) is bugging out on many
of the connections. It does run through to the end of the list, but
with some anomalies, like "Connection closed by A.B.C.D" or
"Received
disconnect from A.B.C.D: 2: Client Disconnect" messages that crop up
multiple times for the same IP address.

Is it possible that one bad connection can still take down active good
connections, even with this patch?

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

--- Comment #12 from aab at purdue.edu 2011-02-24 04:32:02 EST ---
Ummm.  If you're referring to the "original" patch that I
submitted,
It's out-of-date.  It was written before I had a complete(?) handle on
what was going wrong.  Included with this comment is an attachment with
the newer patch against the openssh-5.8p1 source.

A bit of explanation.  Some of the mods are for clarity.  When your
working, as we are, with a large number of hosts, "socket" doesn't
tell
you very much as to where the problem is occuring.  Same with "Bad
hostkey alg".

In the patch, I've attempted to allow `ssh-keyscan' to continue if the
encountered problem is external in origin.  Some of the items that you
noticed are (I think) addressed by this patch.

NOTE - NOTE - NOTE - this patch has NOT been completely verified.  The
closed by remote because of LoginGraceTime" outs needs a bad remote
server so that that can be done.  Unfortunately, all of our servers are
playing nice-nice at present.  I did have an earlier buggy variant of
the patch that "tried" to execute the patch code but I screwed up and
generated an infinite loop instead.  The basic code is running as the
`ssh-keyscan' of choice in our setup.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

aab at purdue.edu changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #1969|0                           |1
        is obsolete|                            |

--- Comment #13 from aab at purdue.edu 2011-02-24 04:40:19 EST ---
Created attachment 2000
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2000
openssh-5.8p1 - patch for ssh-keyscan

Is this comment different from the other one????

Later (better?) patch to fix `ssh-keyscan's premature aborting observed
in large network scans.  Hopefully, there are sufficient comments in
the code to describe the fix.  Please ask if you find something
annoying.  I also have patches for 5.6p1 and 5.7p1.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

--- Comment #14 from Daniel Richard G. <skunk at iSKUNK.ORG> 2011-02-25
02:50:54 EST ---
With this updated patch, I'm seeing at least twice as many host keys
returned than before (up to ~2400, from ~1000), and the "multiple
errors from the same IP" oddness is gone now.

The more-specific error messages are very helpful. I do notice that
hosts which are firewalled or otherwise fail to yield a server banner
are not cited with an error message to stderr. I think this would be
useful if it can be done, that every host listed in the input is spoken
for one way or the other in the output, because that way you can be
sure that no host is being silently dropped by the program.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

--- Comment #15 from aab at purdue.edu 2011-02-27 13:24:50 EST ---
Created attachment 2005
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2005
Upgraded(?) patch to include extra ssh-keyscan logging.

Try this to log all attempt failures.  I put it under control of a
command line option, '-L'.  One failure noted by ssh-keyscan is the
ECONNREFUSED that I think should have caused a standard error message
to be elided.  Except for the ECONNREFUSED, all of the new messages are
written by the `logit()' function.  FWIW - this patch may or may not
obsolete the patch supplied with attachment 2000 so I didn't check the
obsolete:2000 box.  I didn't test this patch out very thoroughly but
what testing I did showed what I wanted.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

--- Comment #16 from Daniel Richard G. <skunk at iSKUNK.ORG> 2011-02-27
20:09:55 EST ---
aab, thanks for putting together this updated patch. I gave it a try,
and whether due to the patch or another issue that I hadn't encountered
before, it bombed out with this error:

[...]
# A.B.C.D SSH-2.0-dropbear_0.50
# W.X.Y.Z SSH-1.99-OpenSSH_3.9p1
# A.B.C.E SSH-2.0-dropbear_0.50
Connection closed by A.B.C.E
conalloc: attempt to reuse fdno 47
make: *** [ssh_known_hosts.unx.new] Error 255

A couple of ancillary notes on the patch:

1. The old and new filenames both have the .orig extension! I had to
edit one of each pair so that the patch could apply.

2. IMO, there isn't a need to add a new -L option... are "Connection
closed" and e.g. "no 'blah' hostkey alg(s)" really
categorically
distinct to the end user?

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

--- Comment #17 from aab at purdue.edu 2011-02-28 05:16:05 EST
---
># A.B.C.D SSH-2.0-dropbear_0.50
># W.X.Y.Z SSH-1.99-OpenSSH_3.9p1
># A.B.C.E SSH-2.0-dropbear_0.50
>Connection closed by A.B.C.E
>conalloc: attempt to reuse fdno 47
>make: *** [ssh_known_hosts.unx.new] Error 255
Oh boy, I missed something.  Is this repeatable?  I think I saw this
myself somewhere along the line but I thought I had fixed the problem. 
Since my time is pretty much taken up for the next week or so, I don't
know when I'll be able to check.
>1. The old and new filenames both have the .orig extension! I had to
>edit one of each pair so that the patch could apply.
I just looked at the attachment.  There are two ".orig"s per file. 
One
is on the `diff' statement and is ignored (I hope) by `patch'.  The
second is one line down on the "old" file identifier (---) and
`patch'
does use that.  Which one was your `patch' making complaints about?
>2. IMO, there isn't a need to add a new -L option... are
"Connection
>closed" and e.g. "no 'blah' hostkey alg(s)" really
categorically
>distinct to the end user?
STDERR is extremely noisy as it is.  In my case, at this time, I think
I'd add on the order of 7000+ extra lines when I use '-L' that
I'd need
to winnow to find any important data.  Besides, you can't forget that
god called "upward compatibility" you know (;-}).

And yes, if you meant "Connection timed out", I think that they are
distinct at least from a Systems Administrator (me) point of view.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

--- Comment #18 from Daniel Richard G. <skunk at iSKUNK.ORG> 2011-03-01
14:42:38 EST ---
(In reply to comment #17)
> 
> Oh boy, I missed something.  Is this repeatable?  I think I saw this
> myself somewhere along the line but I thought I had fixed the problem. 
> Since my time is pretty much taken up for the next week or so, I don't
> know when I'll be able to check.
Well, I tried it again, and it ran to completion. Must be a rare
failure mode.
> I just looked at the attachment.  There are two ".orig"s per
file.  One
> is on the `diff' statement and is ignored (I hope) by `patch'.  The
> second is one line down on the "old" file identifier (---) and
`patch'
> does use that.  Which one was your `patch' making complaints about?
Presumably the second one. It was looking for e.g. kex.c.orig rather
than kex.c.
> STDERR is extremely noisy as it is.  In my case, at this time, I think
> I'd add on the order of 7000+ extra lines when I use '-L' that
I'd need
> to winnow to find any important data.  Besides, you can't forget that
> god called "upward compatibility" you know (;-}).
> 
> And yes, if you meant "Connection timed out", I think that they
are
> distinct at least from a Systems Administrator (me) point of view.
*shrugs* I'd pretty much expect a flood of information anyway. Given a
large network, you have to use grep(1) or the like to make any sense of
it.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

aab at purdue.edu changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #2000|0                           |1
        is obsolete|                            |
   Attachment #2005|0                           |1
        is obsolete|                            |

--- Comment #19 from aab at purdue.edu 2011-03-02 13:29:26 EST ---
Created attachment 2008
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2008
patch - fixes bug in previous patch
>> Oh boy, I missed something.  Is this repeatable?  I think I saw this
>> myself somewhere along the line but I thought I had fixed the problem. 
>> Since my time is pretty much taken up for the next week or so, I
don't
>> know when I'll be able to check.
>
>Well, I tried it again, and it ran to completion. Must be a rare
>failure mode.
Yep, I missed something.  The sockets associated with ALL connections
processed by the `keygrab_ssh2()' function are closed twice.  I missed
the close in the `packet.c:packet_close()' function that's called at
the bottom of the `keygrab_ssh2()' function.  I had assumed (bad bad
word) that the only close was in the `confree()' function.  Work/not
work is up to the gods and the relative connection timings I think.
>> I just looked at the attachment.  There are two ".orig"s per
file.  One
>> is on the `diff' statement and is ignored (I hope) by `patch'. 
The
>> second is one line down on the "old" file identifier (---)
and `patch'
>> does use that.  Which one was your `patch' making complaints about?
>
>Presumably the second one. It was looking for e.g. kex.c.orig rather
>than kex.c.
The format of this patch is the same as before.  If you are using the
current GNU `patch', you should be able to `patch [-p0] < patch' in
the
"openssh-5.8p1" parent directory.  If your in the
"openssh-5.8p1"
directory itself, you should be able to `patch -p1 <patch'.
>> STDERR is extremely noisy as it is.  In my case, at this time, I think
>> I'd add on the order of 7000+ extra lines when I use '-L'
that I'd need
>> to winnow to find any important data.  Besides, you can't forget
that
>> god called "upward compatibility" you know (;-}).
>> 
>> And yes, if you meant "Connection timed out", I think that
they are
>> distinct at least from a Systems Administrator (me) point of view.
>
>*shrugs* I'd pretty much expect a flood of information anyway. Given a
>large network, you have to use grep(1) or the like to make any sense of
>it.
I think that, if/when this patch is actually submitted to the OpenSSH
folks, I'll let the mavins there decide whether or not to have a
'-L'
option.

To satisfy my curiosity, did you observe any missing hosts when you use
the '-L' option (and it actually completes)?

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

--- Comment #20 from Daniel Richard G. <skunk at iSKUNK.ORG> 2011-03-02
19:23:42 EST ---
(In reply to comment #19)
> 
> Yep, I missed something.  The sockets associated with ALL connections
> processed by the `keygrab_ssh2()' function are closed twice.  I missed
> the close in the `packet.c:packet_close()' function that's called
at
> the bottom of the `keygrab_ssh2()' function.  I had assumed (bad bad
> word) that the only close was in the `confree()' function.  Work/not
> work is up to the gods and the relative connection timings I think.
I tried the new patch, and no errors. I'll give it a few more runs to
see if anything breaks again.
> The format of this patch is the same as before.  If you are using the
> current GNU `patch', you should be able to `patch [-p0] < patch'
in the
> "openssh-5.8p1" parent directory.  If your in the
"openssh-5.8p1"
> directory itself, you should be able to `patch -p1 <patch'.
Oh, I know about -p0 vs. -p1 and such. The problem is that the patch,
as up currently, looks for foo.c.orig instead of foo.c. In other words,

    --- dir/foo.c.orig
    +++ dir/foo.c.orig  (WRONG)

    --- dir/foo.c.orig
    +++ dir/foo.c       (CORRECT)
> I think that, if/when this patch is actually submitted to the OpenSSH
> folks, I'll let the mavins there decide whether or not to have a
'-L'
> option.
Fair enough, though I think there might be more value in just
(unconditionally) printing a tally at the end of how many valid hosts
were found, how many had no host algs, etc. (a bit like what "md5sum
-c" does when it encounters errors).
> To satisfy my curiosity, did you observe any missing hosts when you use
> the '-L' option (and it actually completes)?
Ah, I forgot to report on this; my bad!

I do see a few hosts in the input list that are not mentioned anywhere
in the stderr output. These appear to be strictly "alias" IP
addresses,
e.g. for an input line of

    10.0.0.1,10.0.0.2,10.0.0.3 host.example.com,10.0.0.1,10.0.0.2,...
             ^^^^^^^^ ^^^^^^^^
                   these

This is the correct behavior, I take it?

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

--- Comment #21 from aab at purdue.edu 2011-03-03 05:34:29 EST ---
(In reply to comment #20)
> (In reply to comment #19)
> 
>> The format of this patch is the same as before.  If you are using the
>> current GNU `patch', you should be able to `patch [-p0] <
patch' in the
>> "openssh-5.8p1" parent directory.  If your in the
"openssh-5.8p1"
>> directory itself, you should be able to `patch -p1 <patch'.
> 
>Oh, I know about -p0 vs. -p1 and such. The problem is that the patch,
>as up currently, looks for foo.c.orig instead of foo.c. In other words,
> 
>     --- dir/foo.c.orig
>     +++ dir/foo.c.orig  (WRONG)
> 
>     --- dir/foo.c.orig
>     +++ dir/foo.c       (CORRECT)
Hmmm, but the patch doesn't have two consecutive lines with
".orig" as
you describe above.  From observation, the first three lines for each
modified file are similar to

diff -u openssh-5.8p1/kex.c.orig openssh-5.8p1/kex.c
--- openssh-5.8p1/kex.c.orig    2010-09-24 08:11:14.000000000 -0400
+++ openssh-5.8p1/kex.c    2011-02-11 18:14:03.396688000 -0500

Are you using the GNU patch?  The attached patch text works for me with
no changes whatsoever.  Or to ask it somewhat differently, does your
`patch' process WRONG even though the text is actually CORRECT?  Is it
possible that your`patch' is not ignoring the "diff" line?
>> I think that, if/when this patch is actually submitted to the OpenSSH
>> folks, I'll let the mavins there decide whether or not to have a
'-L'
>> option.
> 
> Fair enough, though I think there might be more value in just
> (unconditionally) printing a tally at the end of how many valid hosts
> were found, how many had no host algs, etc. (a bit like what "md5sum
> -c" does when it encounters errors).
Actually, after I had sent the previous, I thought I should have added
that the described approach is a cop out on my part (;-}).
>> To satisfy my curiosity, did you observe any missing hosts when you use
>> the '-L' option (and it actually completes)?
> 
> Ah, I forgot to report on this; my bad!
> 
> I do see a few hosts in the input list that are not mentioned anywhere
> in the stderr output. These appear to be strictly "alias" IP
addresses,
> e.g. for an input line of
> 
>     10.0.0.1,10.0.0.2,10.0.0.3 host.example.com,10.0.0.1,10.0.0.2,...
>              ^^^^^^^^ ^^^^^^^^
>                    these
> 
> This is the correct behavior, I take it?
I submit hosts, one per line, as the data to ssh-keyscan and am not
familiar with the "alias" format.  In fact, your comments clarified it
somewhat for me.  If you meant that "10.0.0.1" was seen in stderr and
the others weren't, I believe that this is the "correct" behavior
if
ssh-keyscan had success with "10.0.0.1".  I think the code tells me
that it stops looking after the first IP/host with which it has
success.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

--- Comment #22 from Daniel Richard G. <skunk at iSKUNK.ORG> 2011-03-03
15:02:12 EST ---
(In reply to comment #21)
> 
> Hmmm, but the patch doesn't have two consecutive lines with
".orig" as
> you describe above.  From observation, the first three lines for each
> modified file are similar to
> 
> diff -u openssh-5.8p1/kex.c.orig openssh-5.8p1/kex.c
> --- openssh-5.8p1/kex.c.orig    2010-09-24 08:11:14.000000000 -0400
> +++ openssh-5.8p1/kex.c    2011-02-11 18:14:03.396688000 -0500
Um. Are we looking at the same file? Here are the first three lines of
your most recent patch (attachment 2008, in comment #19):

--- openssh-5.8p1/kex.c.orig    2010-09-24 08:11:14.000000000 -0400
+++ openssh-5.8p1/kex.c.orig    2011-02-11 18:14:03.396688000 -0500
@@ -49,6 +49,7 @@ 
> Are you using the GNU patch?  The attached patch text works for me with
> no changes whatsoever.  Or to ask it somewhat differently, does your
> `patch' process WRONG even though the text is actually CORRECT?  Is it
> possible that your`patch' is not ignoring the "diff" line?
This is on an Ubuntu Linux system:

host:/tmp/openssh-5.8p1$ patch -p1 --dry-run <aab-2008.patch 
patching file kex.c.orig
Hunk #1 FAILED at 49.
Hunk #2 FAILED at 367.
2 out of 2 hunks FAILED -- saving rejects to file kex.c.orig.rej
patching file packet.c.orig
Hunk #1 FAILED at 1025.
Hunk #2 FAILED at 1035.
Hunk #3 FAILED at 1100.
3 out of 3 hunks FAILED -- saving rejects to file packet.c.orig.rej
[...]

If I edit each "+++" line in the patch, it applies cleanly.
> I submit hosts, one per line, as the data to ssh-keyscan and am not
> familiar with the "alias" format.  In fact, your comments
clarified it
> somewhat for me.  If you meant that "10.0.0.1" was seen in stderr
and
> the others weren't, I believe that this is the "correct"
behavior if
> ssh-keyscan had success with "10.0.0.1".  I think the code tells
me
> that it stops looking after the first IP/host with which it has
> success.
Okay, that seems reasonable. (Yes, I only saw 10.0.0.1 and not the
other two.)

The sample "Input format" line in the ssh-keyscan man page has two IP
addresses in the first column, though the semantics of this are left
unexplained. My assumption is that it's meant for hosts with
round-robined DNS names, where the SSH server at each address uses the
same host keys. (Which would be consistent with what you describe.)

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

--- Comment #23 from aab at purdue.edu 2011-03-03 16:03:01 EST ---
(In reply to comment #22)
> (In reply to comment #21)
>> 
>> Hmmm, but the patch doesn't have two consecutive lines with
".orig" as
>> you describe above.  From observation, the first three lines for each
>> modified file are similar to
>> 
>> diff -u openssh-5.8p1/kex.c.orig openssh-5.8p1/kex.c
>> --- openssh-5.8p1/kex.c.orig    2010-09-24 08:11:14.000000000 -0400
>> +++ openssh-5.8p1/kex.c    2011-02-11 18:14:03.396688000 -0500
> 
> Um. Are we looking at the same file? Here are the first three lines of
> your most recent patch (attachment 2008 [details], in comment #19):
> 
> --- openssh-5.8p1/kex.c.orig    2010-09-24 08:11:14.000000000 -0400
> +++ openssh-5.8p1/kex.c.orig    2011-02-11 18:14:03.396688000 -0500
> @@ -49,6 +49,7 @@ 
Boy, I'm not sure that we are looking at the same file.  I just did a 

  wget -Ojunk https://bugzilla.mindrot.org/attachment.cgi?id=2008

and got my version.  When I click on the attachment line near the top
of the bug #1213 comments (this page - "patch - fixes bug ..."), I get
my version.  Clicking on the "details" button that you specified
above,
I get my version.

Have we encountered a bug in yet another utility?  Browser problem?

I should have thanked you earlier for "testing" the patch so I'll
do so
now - THANKS.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

--- Comment #24 from Daniel Richard G. <skunk at iSKUNK.ORG> 2011-03-03
17:51:48 EST ---
Okay, I think I see what's going on here.

When you click on the "attachment 2008" link, you're taken to a
fancy
side-by-side rendition of the diff. At the top, there are a series of
links:

    View | Details | Raw Unified | Return to bug 1213 | Differences ...

I was clicking on "Raw Unified," and got the broken patch.
"View" goes
to the URL you gave (which yields the correct patch). Confusing, isn't
it?

Anyway, I'm happy to test your patches, because that means I can get
the company-wide ssh_known_hosts file I've been needing so much :-)

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

--- Comment #25 from aab at purdue.edu 2011-03-04 12:07:11 EST ---
(In reply to comment #24)
> Okay, I think I see what's going on here.
> 
> When you click on the "attachment 2008 [details]" link,
you're taken to a fancy
> side-by-side rendition of the diff. At the top, there are a series of
> links:
> 
>     View | Details | Raw Unified | Return to bug 1213 | Differences ...
> 
> I was clicking on "Raw Unified," and got the broken patch.
"View" goes
> to the URL you gave (which yields the correct patch). Confusing, isn't
> it?
Yes, it is indeed confusing.  I've never used the exact path you used
to get to the patch so I missed seeing the "bad" representation of it.

One of the things that I've observed in generating the
"ssh_known_hosts" file is that it can end up having a quite variable
keyset as it depends on ALL of the hosts ALWAYS being up (don't we
wish).  It's probably overkill but we generate the "hosts" file
once an
hour via a set of wrapper scripts.  Included within the scripts is a
database that contains the current keys for all hosts that are
currently supposed to be active (previously acquired via these same
scripts).  This allows us two capabilities: 1) if there is no key
returned for some host, the database can supply the last one and 2) it
allows us to see if there have been any changes in the keys that might
signify a security break.

A second part is a condensation of the keys via globbing. This assumes
that a number of the hosts have the same key.  The cluster nodes on our
private networks are basically all cloned so we do get considerable
condensation.  Right now, for 4700+ hosts, the "hosts" file has 334
entries.

The core script is a highly modified variant of the GNU licensed
script, "make_ssh_known_hosts.pl", that was in "ssh-1.0.0"
(circa
1998).  Note that's "ssh" not "openssh".  My original
came from
"ssh-1.2.26".  For some reason, it disappeared when the OpenSSH folks
took over.   For Linux boxes, it's still dependent on my bind 9 hack of
`nslookup' as I haven't had time to modify it to use the current GNU
`host'.

Would you be interested in anything like this?

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

--- Comment #26 from Daniel Richard G. <skunk at iSKUNK.ORG> 2011-03-05
19:08:28 EST ---
(In reply to comment #25)
> 
> Yes, it is indeed confusing.  I've never used the exact path you used
> to get to the patch so I missed seeing the "bad" representation
of it.
Lord knows what the point of that link even is... I clicked on it only
because "Raw" suggested that it would yield the "real"
text/plain diff
instead of a fancy HTML rendition.
> Would you be interested in anything like this?
I appreciate the offer, but a database would be overkill for my use
case. I'm not in my company's IT department, and metamorphosing host
keys on those 6000+ hosts are waaaay out of my purview. (I can't get
too worked up over the security implications, either, since much worse
than that is officially tolerated.)

If anything, the most I would do is put together a Perl script to merge
an old and new known_hosts file, such that new entries override old
ones, and old ones that don't have a newer replacement are kept.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

--- Comment #27 from Paul Wouters <paul at cypherpunks.ca> 2011-03-06
06:04:20 EST ---
(In reply to comment #26)
> (In reply to comment #25)
> If anything, the most I would do is put together a Perl script to merge
> an old and new known_hosts file, such that new entries override old
> ones, and old ones that don't have a newer replacement are kept.
You really want to look at SSHFP DNS records protected by DNSSEC, and
setting VerifyHostKeyDNS ask in your /etc/ssh/ssh_config

you can use the "sshfp" tool for that, which is exactly why I was
interested in this bug. sshfp can AXFR a zone, and use ssh-keyscan to
connect to all A records in the zone and print the SSHFP record to add
in your zones.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

--- Comment #28 from Daniel Richard G. <skunk at iSKUNK.ORG> 2011-03-06
06:13:53 EST ---
(In reply to comment #27)
> 
> You really want to look at SSHFP DNS records protected by DNSSEC, and
> setting VerifyHostKeyDNS ask in your /etc/ssh/ssh_config
I would, if I were in my company's IT department :-)

(All I'm doing is generating an ssh_known_hosts file that is accessible
to a handful of clients via a local fileserver. The network
infrastructure beyond that is completely out of my hands.)
> you can use the "sshfp" tool for that, which is exactly why I was
> interested in this bug. sshfp can AXFR a zone, and use ssh-keyscan to
> connect to all A records in the zone and print the SSHFP record to add
> in your zones.
Hmm, that could be useful. While I couldn't do much with the SSHFP
records, the AXFR->keyscan functionality would be useful. (Right now,
I'm doing the AXFR via host(1), and using a Perl script to reformat
that into a hosts list for ssh-keyscan(1).)

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

aab at purdue.edu changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #1961|0                           |1
        is obsolete|                            |

--- Comment #29 from aab at purdue.edu 2011-03-08 18:03:53 EST ---
Comment on attachment 1961
  --> https://bugzilla.mindrot.org/attachment.cgi?id=1961
One attempt at getting the rsa key from a remote server that was having
a number of problems.

This has been resolved with attachment 2008.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1213

--- Comment #30 from aab at purdue.edu 2011-03-15 12:14:37 EST ---
One of our `sshd' servers finally gave me sufficient problems to test
the last of the patched code and, as far as I can tell, it worked.

Is there anybody out there that has any issues with the current patch? 
If not, I wonder if I can catch the attention of any of the OpenSSH
folks.  I note that this problem has yet to be assigned to anyone.

Or is there another route that I should take for attention?

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.