So, I just stumbled across this article[1] that talks about compiling OpenSSL with Mingw for Ruby and Python. I thought I''d give it a try to see if it would help with the openssl test failures. I''m having a few problems. The first problem is in the extract step. Bsdtar errors out when it tries to extract the latest OpenSSL tar.gz[2]. So, I''ve sort of skipped that step for the moment. I run rake extract, then I manually extract these files where I need them. The second problem is getting the order of things right in rake. I''m assuming I would need to configure, compile, and install the OpenSSL files (*.dll and *.a) before I try to build Ruby, and my rake skills aren''t good enough to figure out how to do this. So, I think I''m going to try to configure msys and mingw properly, and try this as a manual process to see if there''s any worth to this approach. But, before I do that, I thought I''d ask here. Does this seem like a worthwhile approach, or am I on a wild goose chase? :-) [1] http://bowmansolutions.com/mingw-openldap/ [2] http://www.openssl.org/source/openssl-0.9.8g.tar.gz Thanks, Gordon
On Thu, Apr 10, 2008 at 2:27 PM, Gordon Thiesfeld <gthiesfeld at gmail.com> wrote:> So, I just stumbled across this article[1] that talks about compiling > OpenSSL with Mingw for Ruby and Python. I thought I''d give it a try > to see if it would help with the openssl test failures. I''m having a > few problems. >Hello Gordon! I wasn''t aware there are OpenSSL failures in the MinGW build. AFAIK the only blocker I still have is test_readline, which hangs and halt the whole ''rake check'' process. If you see the recipes, I''ve been using a pre-build binary from GNUwin32 project instead of build it myself. Right now the way the Rake tasks are organized is not good enough, and can get brittle over time.> The first problem is in the extract step. Bsdtar errors out when it > tries to extract the latest OpenSSL tar.gz[2]. So, I''ve sort of > skipped that step for the moment. I run rake extract, then I manually > extract these files where I need them. >Can you provide the output or the error message you''re getting when it tries to extract it? Also, this OpenSSL need ot be extracted in it''s own directory, instead of using the OpenSSL package target location, instead of MinGW one.> The second problem is getting the order of things right in rake. I''m > assuming I would need to configure, compile, and install the OpenSSL > files (*.dll and *.a) before I try to build Ruby, and my rake skills > aren''t good enough to figure out how to do this. >Yes and no, configure will be chained with configure of ruby interpreter, and for compile will happen the same. That wouldn''t work unless we provide --with-openssl-dir or something to the ruby configure process (to avoid clutter mingw and msys installation with leftovers of these packages). I think we need a new abstraction layer on top of Rake, that can be more descriptive and precise of the dependencies, but don''t have enough time right now to design something :-P> So, I think I''m going to try to configure msys and mingw properly, and > try this as a manual process to see if there''s any worth to this > approach. But, before I do that, I thought I''d ask here. Does this > seem like a worthwhile approach, or am I on a wild goose chase? :-) >Please do, I''ll suggest you revert your changes, use latest.zip and let rake do it''s stuff, then use the sandbox/mingw and sandbox/msys as base MinGW and start playing with manual build of OpenSSL. I think is real worthy, since one of the goals of these recipes is cut the dependency of externals builds, for ruby or it''s dependencies. The bad is some GNU projects (like readline) mess a lot the packages and make almost impossible replicate the build procedure from sources.> Thanks,Thanks to you man!, please keep us posted on this! I''ll hang out at #ruby-lang the weekend, trying to get other gems working on Windows and MinGW. Regards, -- Luis Lavena Multimedia systems - Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so. Douglas Adams
On Thu, Apr 10, 2008 at 3:01 PM, Luis Lavena <luislavena at gmail.com> wrote:> Hello Gordon! > > I wasn''t aware there are OpenSSL failures in the MinGW build. AFAIK > the only blocker I still have is test_readline, which hangs and halt > the whole ''rake check'' process.I''m running bzr revision 52. I get this on ''rake check''. ./ruby_1_8/test/openssl/test_hmac.rb:14: [BUG] Segmentation fault ruby 1.8.6 (2008-03-03) [i386-mingw32] This application has requested the Runtime to terminate it in an unusual way. Please contact the application''s support team for more information. make: *** [test-all] Error 3 rake aborted! Command failed with status (2): ["C:/ruby/repo/installer3/sandbox/msys/bin/...] (See full trace by running task with --trace)> Can you provide the output or the error message you''re getting when it > tries to extract it?"C:/ruby/repo/oci/sandbox/extract_utils/bsdtar.exe" xzf "C:/ruby/repo/oci/downloads/openssl-0.9.8g.tar.gz" openssl-0.9.8g/apps/md4.c: Can''t create ''openssl-0.9.8g/apps/md4.c'': No such file or directory openssl-0.9.8g/include/openssl/dtls1.h: Can''t create ''openssl-0.9.8g/include/openssl/dtls1.h'': No such file or directory openssl-0.9.8g/include/openssl/kssl.h: Can''t create ''openssl-0.9.8g/include/openssl/kssl.h'': No such file or directory openssl-0.9.8g/include/openssl/ssl.h: Can''t create ''openssl-0.9.8g/include/openssl/ssl.h'': No such file or directory openssl-0.9.8g/include/openssl/ssl2.h: Can''t create ''openssl-0.9.8g/include/openssl/ssl2.h'': No such file or directory openssl-0.9.8g/include/openssl/ssl23.h: Can''t create ''openssl-0.9.8g/include/openssl/ssl23.h'': No such file or directory openssl-0.9.8g/include/openssl/ssl3.h: Can''t create ''openssl-0.9.8g/include/openssl/ssl3.h'': No such file or directory openssl-0.9.8g/include/openssl/tls1.h: Can''t create ''openssl-0.9.8g/include/openssl/tls1.h'': No such file or directory bsdtar.exe: Error exit delayed from previous errors. rake aborted! Command failed with status (1): ["C:/ruby/repo/oci/sandbox/extract_utils/bs...] (See full trace by running task with --trace)> Also, this OpenSSL need ot be extracted in it''s > own directory, instead of using the OpenSSL package target location, > instead of MinGW one.Right, I had set it up with a similar folder structure to Ruby''s (openssl, openssl_build, openssl_mingw).> Please do, I''ll suggest you revert your changes, use latest.zip and > let rake do it''s stuff, then use the sandbox/mingw and sandbox/msys as > base MinGW and start playing with manual build of OpenSSL.I did bzr revert, and I''m going to give this a try.> I''ll hang out at #ruby-lang the weekend, trying to get other gems > working on Windows and MinGW.I''ll try to stop by. I could use some help ;) Gordon Thiesfeld
On Fri, Apr 11, 2008 at 5:57 PM, Gordon Thiesfeld <gthiesfeld at gmail.com> wrote:> On Thu, Apr 10, 2008 at 3:01 PM, Luis Lavena <luislavena at gmail.com> wrote: > > Hello Gordon! > > > > I wasn''t aware there are OpenSSL failures in the MinGW build. AFAIK > > the only blocker I still have is test_readline, which hangs and halt > > the whole ''rake check'' process. > > I''m running bzr revision 52. I get this on ''rake check''. > > ./ruby_1_8/test/openssl/test_hmac.rb:14: [BUG] Segmentation fault > ruby 1.8.6 (2008-03-03) [i386-mingw32] >That segfault came from your OpenSSL change, or that is generated by "out of the box" recipes? Can you provide information about your Windows installation? (XP/Vista, home, ultimate, service pack level, etc). Also will be helpful know your PATH environment variable: ECHO %PATH% (or SET PATH without params)> > This application has requested the Runtime to terminate it in an unusual way. > Please contact the application''s support team for more information. > make: *** [test-all] Error 3 > rake aborted! > Command failed with status (2): ["C:/ruby/repo/installer3/sandbox/msys/bin/...] > > (See full trace by running task with --trace) > > > > > Can you provide the output or the error message you''re getting when it > > tries to extract it? > > "C:/ruby/repo/oci/sandbox/extract_utils/bsdtar.exe" xzf > "C:/ruby/repo/oci/downloads/openssl-0.9.8g.tar.gz" > openssl-0.9.8g/apps/md4.c: Can''t create ''openssl-0.9.8g/apps/md4.c'': > No such file or directoryOk, no doubt bsdtar cannot handle latest openssl packages. I''m checking right now what version can and cannot be unpacked: 0.9.7l, 0.9.7m, 0.9.8d, 0.9.8e, 0.9.8f and 0.9.8g all fail in the same files: openssl-0.9.7l/apps/md4.c: Can''t create ''openssl-0.9.7l/apps/md4.c'': No such file or directory openssl-0.9.7l/include/openssl/kssl.h: Can''t create ''openssl-0.9.7l/include/openssl/kssl.h'': No such file or directory openssl-0.9.7l/include/openssl/ssl23.h: Can''t create ''openssl-0.9.7l/include/openssl/ssl23.h'': No such file or directory openssl-0.9.7l/include/openssl/ssl2.h: Can''t create ''openssl-0.9.7l/include/openssl/ssl2.h'': No such file or directory openssl-0.9.7l/include/openssl/ssl3.h: Can''t create ''openssl-0.9.7l/include/openssl/ssl3.h'': No such file or directory openssl-0.9.7l/include/openssl/ssl.h: Can''t create ''openssl-0.9.7l/include/openssl/ssl.h'': No such file or directory openssl-0.9.7l/include/openssl/tls1.h: Can''t create ''openssl-0.9.7l/include/openssl/tls1.h'': No such file or directory openssl-0.9.7l/test/fips_aes_data: Can''t create ''openssl-0.9.7l/test/fips_aes_data'': No such file or directory If OpenSSL cannot be extracted using simple tar, then bsdtar is not enough or openssl packages are corrupt somehow.> > > Also, this OpenSSL need ot be extracted in it''s > > own directory, instead of using the OpenSSL package target location, > > instead of MinGW one. > > Right, I had set it up with a similar folder structure to Ruby''s > (openssl, openssl_build, openssl_mingw). > > > > Please do, I''ll suggest you revert your changes, use latest.zip and > > let rake do it''s stuff, then use the sandbox/mingw and sandbox/msys as > > base MinGW and start playing with manual build of OpenSSL. > > I did bzr revert, and I''m going to give this a try. > > > > I''ll hang out at #ruby-lang the weekend, trying to get other gems > > working on Windows and MinGW. > > I''ll try to stop by. I could use some help ;) >Good, me too ;-) -- Luis Lavena Multimedia systems - Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so. Douglas Adams
On Fri, Apr 11, 2008 at 4:57 PM, Luis Lavena <luislavena at gmail.com> wrote:> > That segfault came from your OpenSSL change, or that is generated by > "out of the box" recipes? >With the "out of the box" recipes. I haven''t actually gotten OpenSSL to compile yet ;)> Can you provide information about your Windows installation? > (XP/Vista, home, ultimate, service pack level, etc). >Win XP Pro SP2> Also will be helpful know your PATH environment variable: > > ECHO %PATH% (or SET PATH without params)C:\ruby\repo\oci>path PATH=c:\wix;c:\ruby\ruby186\bin;c:\windows\system32;c:\windows\system;C:\Program Files\Subversion\bin;C:\Program Files\Git\cmd;C:\Program Files\Bazaar;C:\bin;C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727;C:\Program Files\Putty;c:\bin\console2 I also tried putting ruby_mingw\bin folder to the front of the path, and that didn''t help.> Ok, no doubt bsdtar cannot handle latest openssl packages. I''m > checking right now what version can and cannot be unpacked:> If OpenSSL cannot be extracted using simple tar, then bsdtar is not > enough or openssl packages are corrupt somehow.7zip can open them ok, and Winzip too. I was thinking about writing a new extract method using 7zip (7z.exe), but that would be a dependency (required to be installed and required to be in path). I guess before I even think about that, I should worry about getting OpenSSL compiled, and working ;) Thanks, Gordon
On Fri, Apr 11, 2008 at 9:12 PM, Gordon Thiesfeld <gthiesfeld at gmail.com> wrote:> On Fri, Apr 11, 2008 at 4:57 PM, Luis Lavena <luislavena at gmail.com> wrote: > > > > That segfault came from your OpenSSL change, or that is generated by > > "out of the box" recipes? > > > > With the "out of the box" recipes. I haven''t actually gotten OpenSSL > to compile yet ;) >Hehehe, ok, then it seems something smells fishy down there... Can you use Dependency Walker on top of the openssl.so file? it will try to resolve to msvcrt-ruby18.dll, libeay32.dll and libssl32.dll. Please take note the path where these files resolves to, it seems openssl.so, even is linked correctly at build time, at runtime it''s resolving the location of one of these dependencies wrong.> > > If OpenSSL cannot be extracted using simple tar, then bsdtar is not > > enough or openssl packages are corrupt somehow. > > 7zip can open them ok, and Winzip too. I was thinking about writing a > new extract method using 7zip (7z.exe), but that would be a dependency > (required to be installed and required to be in path). I guess before > I even think about that, I should worry about getting OpenSSL > compiled, and working ;) >Maybe we can get a 7zip command line version package somewhere, as long is .zip file (enough to bootstrap it with rubyzip). Thank you for your time, I''ll try to see what I can do for this later tonight. Regards, -- Luis Lavena Multimedia systems - Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so. Douglas Adams
On Fri, Apr 11, 2008 at 7:18 PM, Luis Lavena <luislavena at gmail.com> wrote:> On Fri, Apr 11, 2008 at 9:12 PM, Gordon Thiesfeld <gthiesfeld at gmail.com> wrote: > > On Fri, Apr 11, 2008 at 4:57 PM, Luis Lavena <luislavena at gmail.com> wrote: > > > > > > That segfault came from your OpenSSL change, or that is generated by > > > "out of the box" recipes? > > > > > > > With the "out of the box" recipes. I haven''t actually gotten OpenSSL > > to compile yet ;) > > > > Hehehe, ok, then it seems something smells fishy down there... > > Can you use Dependency Walker on top of the openssl.so file? it will > try to resolve to msvcrt-ruby18.dll, libeay32.dll and libssl32.dll. > > Please take note the path where these files resolves to, it seems > openssl.so, even is linked correctly at build time, at runtime it''s > resolving the location of one of these dependencies wrong.Good call. I have libeay32.dll and libssl32.dll in C:\Windows\system32. I''ll clean that up and try again tomorrow morning.> > > > > If OpenSSL cannot be extracted using simple tar, then bsdtar is not > > > enough or openssl packages are corrupt somehow. > > > > 7zip can open them ok, and Winzip too. I was thinking about writing a > > new extract method using 7zip (7z.exe), but that would be a dependency > > (required to be installed and required to be in path). I guess before > > I even think about that, I should worry about getting OpenSSL > > compiled, and working ;) > > > > Maybe we can get a 7zip command line version package somewhere, as > long is .zip file (enough to bootstrap it with rubyzip). >Yes, here it is: http://downloads.sourceforge.net/sevenzip/7za457.zip The only thing is that 7z.exe can''t extract files in a single step, you have to extract from gz or bz2, etc., then extract from tar. I don''t think that would be too big of a problem, though. Thanks again, Luis. Gordon