openssh bugs - Sep 2006 - [Bug 1239] OpenSSL ENGINE support broken on Solaris using Sun Studio compiler

http://bugzilla.mindrot.org/show_bug.cgi?id=1239

           Summary: OpenSSL ENGINE support broken on Solaris using Sun
                    Studio compiler
           Product: Portable OpenSSH
           Version: 4.4p1
          Platform: Other
        OS/Version: Solaris
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Build system
        AssignedTo: bitbucket at mindrot.org
        ReportedBy: andrew.benham at thus.net


Solaris 8,9, and 10, x86 and sparc

configuring with '--with-ssl-engine' gives an error:

checking for OpenSSL ENGINE support... configure: error: OpenSSL ENGINE
support not found


Cause:
configure.ac lines 1916 onward state:

                AC_TRY_COMPILE(
                        [ #include <openssl/engine.h>],
                        [
int
main(void){ENGINE_load_builtin_engines();ENGINE_register_all_complete();}
                        ],
                        [ AC_MSG_RESULT(yes)
                          AC_DEFINE(USE_OPENSSL_ENGINE, 1,
                             [Enable OpenSSL engine support])
                        ],
                        [ AC_MSG_ERROR(OpenSSL ENGINE support not
found)]
                )


This generates code in configure lines 17887 onward:

  int
  main ()
  {

  int
main(void){ENGINE_load_builtin_engines();ENGINE_register_all_complet   
    e();}

    ;
    return 0;
  }


which the Sun Studio compiler doesn't regard as valid C: it's
declaring a function 'main()' inside a function 'main()', and
terminates with an error.

gcc doesn't seem to mind compiling this code, but the Sun Studio
compiler isn't having it.


Fix:
configure.ac, line 1919

Lose the leading "int main(void){" and the trailing "}"




------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

http://bugzilla.mindrot.org/show_bug.cgi?id=1239





------- Comment #1 from andrew.benham at thus.net  2006-09-29 00:07 -------
Created an attachment (id=1191)
 --> (http://bugzilla.mindrot.org/attachment.cgi?id=1191&action=view)
Fix for configure.ac




------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

http://bugzilla.mindrot.org/show_bug.cgi?id=1239


dtucker at zip.com.au changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED
OtherBugsDependingO|                            |1222
              nThis|                            |




------- Comment #2 from dtucker at zip.com.au  2006-09-29 20:20 -------
I must admit that I didn't read what autoconf produced for this test,
but I'm surprised that gcc accepted it.

Patch applied to both -HEAD and the 4.4 branch.  It will be next
release.

Thanks.




------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

http://bugzilla.mindrot.org/show_bug.cgi?id=1239





------- Comment #3 from andrew.benham at thus.net  2006-09-29 23:12 -------
Well, apparently gcc supports 'nested functions', but the world is
divided over whether this is a good idea or not.
Whatever the individual view on this, it ain't standard C.




------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.