On 21/03/2008, Luis Lavena <luislavena at gmail.com> wrote:> On Mar 20, 5:52 pm, Michal Suchanek <hramr... at centrum.cz> wrote: > > > > Well, I consider myself sort of developer but I haven''t used Rake so far. > > > > > I''m just used to do every repetitive task with Rake and I think other > developers will agree.There are also other alternatives like make, shell scripts, etc. And it does not help with the repetitiveness of the download that much as there are the timeouts. I do not quite understand how the stuff fits together but I suspect these are somewhere in the rake itself so perhaps rake needs a retry option for downloads.> > Anyway, I don''t feel very happy with the layout I got for this > project, since the separated task for every dependency are over- > engineered :-P > > > > > > That''s why the script would be helpful ;-) > > > > > Hmm, a script that just fire ''rake'' don''t sound too useful :-P >When that''s the only thing needed there is no reason to write a script, yes ;-)> > > I do not get a download timeout or it is quite well obfuscated at > > least. I get slow download of packages one after another and then I > > get something like: > > > > execution timed out > > rake aborted > > > > > The error message you get can vary, that depends on the sourceforge > mirror you got. sorry about that. > > The idea is that you don''t need to download the sources package again > unless you ''clobber'' or nuke your downloads directory (rake clean will > only clean the sandbox)Good, but if you unpack a fresh copy of the installer the downloads aren''t there unless you save them and put them into the fresh copy manually.> > > > > > I installed the OCI but did not do anything to put the ruby bin > > directory into my path, and the installer did not do it either. There > > must be a zlib1.dll somewhere in your system but there''s none in mine. > > It is quite common so it might be installed by some other software. > > > > > If the installer didn''t add Ruby\bin into the PATH then you''re running > without administrative privileges or something like that. > > I just added a README.txt file that roughly indicates zlib1.dll as > requirement.I still think it would be overall easier to just put a copy of zlib1.dll into the extract_tools folder.> > > > Yes, I tried some tasks until I was sure there''s nothing that does not > > produce any error but it might be that executing the tasks out of > > order leaves some fallout. > > > > > To make you happy: the Rakefile now have a defaults (recipes/ > defaults.rake) that: > > download, > extract, > prepare, > configure, > compile and > installVery nice, now running rake gives me a ruby build :-) However, it would be nice if this copy also included rubygems and/or rake so that rebuilding with this copy new was easier. To rebuild I had to - download rubygems - run the extract process of the installer so that I got bsdtar - extract rubygems - in rubygems run setup.rb configure (this for some reason performed the installation already) - run gem install rake After this I could nuke the install3 folder, extract a new one, and build ruby with the newly installed ruby. Thanks Michal
Welcome Michal! On Fri, Mar 21, 2008 at 3:49 PM, Michal Suchanek <hramrach at centrum.cz> wrote:> On 21/03/2008, Luis Lavena <luislavena at gmail.com> wrote: > > On Mar 20, 5:52 pm, Michal Suchanek <hramr... at centrum.cz> wrote: > > > > > > Well, I consider myself sort of developer but I haven''t used Rake so far. > > > > > > > > > I''m just used to do every repetitive task with Rake and I think other > > developers will agree. > > There are also other alternatives like make, shell scripts, etc. > > And it does not help with the repetitiveness of the download that much > as there are the timeouts. I do not quite understand how the stuff > fits together but I suspect these are somewhere in the rake itself so > perhaps rake needs a retry option for downloads. >Well, the ideas, as I said: use the most ruby you could possible use, that means use Ruby to build ruby :-) There is no shell script that could perform what are we doing right now without external tools for download and unpack (yes, Is a bummer have zlib1.dll around, will try to fix this weekend). If you have used the garbagagecollect builds for your "Yet another Ruby Installer for Windows", then you know there is no documentation or way to recreate the process locally, and thus there is no way to simple fix the problems the current Ruby for Windows faces. Then you''re stuck each time to the releases made by somebody else and reply lots of tickets and support request when things don''t work out of the box, like WIN32OLE in 1.8.6-p110 (which was replaced soon enough by p111)... I think you get the picture. Instead of gargabecollect release, I''m not using Visual C as compiler, since is not available anymore, and couldn''t be freely obtained like MinGW (there is also the lack of good speed we got from MinGW build). This compilation of recipes try to get MinGW+MSYS tools to fully build Ruby without you prior requiring nothing more than a working Ruby installation. So there is no manual setup of MinGW or something similar.> > > I do not get a download timeout or it is quite well obfuscated at > > > least. I get slow download of packages one after another and then I > > > get something like: > > > > > > execution timed out > > > rake aborted > > > > > > > > > The error message you get can vary, that depends on the sourceforge > > mirror you got. sorry about that. > > > > The idea is that you don''t need to download the sources package again > > unless you ''clobber'' or nuke your downloads directory (rake clean will > > only clean the sandbox) > > Good, but if you unpack a fresh copy of the installer the downloads > aren''t there unless you save them and put them into the fresh copy > manually. >Hmm, don''t understand what are you saying, let me explain how I designed this in the first place. 1) after you unpack latest.zip (I actually name the folder installer3.dev, anyway) you jump in and perform the ''rake download'' task. This is the first and only time you require to do so, since that task will download all the zip, tar.bz and tar.gz files to build a clean ruby each time, unless you update some of the required files (version updates or security fixes, etc). 2) since you start with a clean checkout of the installer, you proceed to "extract" and "prepare", which will create the required directories in sandbox and unpack all the tools. Then it arrange (prepare) some of the files and dlls to proceed with the next step. This is required every time you "clean" your sandbox (that means when you change something in the recipes and want to do a fresh start). 3) configure task actually run the GNU autoconf script that is bundled with ruby, and generate the proper config file and the makefile required by the compile task. 4) compile will try (at this time) to build ruby with the bindings that extract and prepare made available, at this time, only "zlib" and "readline" are in place. 5) Install task emulates the directory structure the garbage collect distribution have (is a ruby installation). The idea is later in the process rubygems gets installed over there, but there are no recipes for it right now. 6) check task will ensure that everything built in the compile task works out of the box. It runs the ruby test units to garantee that. Since readline is failing, I didn''t proceed to work on the MSI installer generators or the RubyGems installation.> > I still think it would be overall easier to just put a copy of > zlib1.dll into the extract_tools folder. >Is a bad practice add binaries to your repository... doing that you''re forced to keep it updated from release to release... I''ll fix the extract utils to expand the downloaded zlib package.> > > > > > > Yes, I tried some tasks until I was sure there''s nothing that does not > > > produce any error but it might be that executing the tasks out of > > > order leaves some fallout. > > > > > > > > > To make you happy: the Rakefile now have a defaults (recipes/ > > defaults.rake) that: > > > > download, > > extract, > > prepare, > > configure, > > compile and > > install > > Very nice, now running rake gives me a ruby build :-) > > However, it would be nice if this copy also included rubygems and/or > rake so that rebuilding with this copy new was easier. To rebuild I > had to > > - download rubygems > - run the extract process of the installer so that I got bsdtar > - extract rubygems > - in rubygems run setup.rb configure (this for some reason performed > the installation already) > - run gem install rake >As explain above, RubyGems is not my main focus right now, but get a stable working ruby build instead (there are lot of extensions don''t build right now, like iconv, curses and others). However, if you want it included right now, I think a rubygems recipe can be added to dependencies, and download, extract and install tasks too.> After this I could nuke the install3 folder, extract a new one, and > build ruby with the newly installed ruby. >That''s part of the idea, but how you nuke installer3? just use rake clean to erase the sandbox, don''t stress the servers doing several downloads (clean will not remove the downloads, clobber or a new checkout will).> ThanksRegards, -- Luis Lavena Multimedia systems - Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so. Douglas Adams
On 21/03/2008, Luis Lavena <luislavena at gmail.com> wrote:> Welcome Michal! > > > On Fri, Mar 21, 2008 at 3:49 PM, Michal Suchanek <hramrach at centrum.cz> wrote: > > On 21/03/2008, Luis Lavena <luislavena at gmail.com> wrote: > > > On Mar 20, 5:52 pm, Michal Suchanek <hramr... at centrum.cz> wrote: > > > > > > > > Well, I consider myself sort of developer but I haven''t used Rake so far. > > > > > > > > > > > > > I''m just used to do every repetitive task with Rake and I think other > > > developers will agree. > > > > There are also other alternatives like make, shell scripts, etc. > > > > And it does not help with the repetitiveness of the download that much > > as there are the timeouts. I do not quite understand how the stuff > > fits together but I suspect these are somewhere in the rake itself so > > perhaps rake needs a retry option for downloads. > > > > > Well, the ideas, as I said: use the most ruby you could possible use, > that means use Ruby to build ruby :-) > > There is no shell script that could perform what are we doing right > now without external tools for download and unpack (yes, Is a bummer > have zlib1.dll around, will try to fix this weekend).I am not arguing against Rake, I am just saying that there are other tools that could do the job, and thus there are people who aren''t familiar with Rake.> > If you have used the garbagagecollect builds for your "Yet another > Ruby Installer for Windows", then you know there is no documentation > or way to recreate the process locally, and thus there is no way to > simple fix the problems the current Ruby for Windows faces. > > Then you''re stuck each time to the releases made by somebody else and > reply lots of tickets and support request when things don''t work out > of the box, like WIN32OLE in 1.8.6-p110 (which was replaced soon > enough by p111)... I think you get the picture.Yes, I have seen the earlier discussions about installers. It''s actually not mine "yet another installer", I was just replying to the thread. And I am aware of the grief with binary-only releases :-/> > Instead of gargabecollect release, I''m not using Visual C as compiler, > since is not available anymore, and couldn''t be freely obtained like > MinGW (there is also the lack of good speed we got from MinGW build). > > This compilation of recipes try to get MinGW+MSYS tools to fully build > Ruby without you prior requiring nothing more than a working Ruby > installation. So there is no manual setup of MinGW or something > similar. >Yes, this is very nice. The better automation you get the more likely people are going to use the scripts and possibly find or even fix problems.> > > > > I do not get a download timeout or it is quite well obfuscated at > > > > least. I get slow download of packages one after another and then I > > > > get something like: > > > > > > > > execution timed out > > > > rake aborted > > > > > > > > > > > > > The error message you get can vary, that depends on the sourceforge > > > mirror you got. sorry about that. > > > > > > The idea is that you don''t need to download the sources package again > > > unless you ''clobber'' or nuke your downloads directory (rake clean will > > > only clean the sandbox) > > > > Good, but if you unpack a fresh copy of the installer the downloads > > aren''t there unless you save them and put them into the fresh copy > > manually. > > > > > Hmm, don''t understand what are you saying, let me explain how I > designed this in the first place. > > 1) after you unpack latest.zip (I actually name the folder > installer3.dev, anyway) you jump in and perform the ''rake download'' > task.Well, the zip archives contain an installer3 folder ...> > This is the first and only time you require to do so, since that task > will download all the zip, tar.bz and tar.gz files to build a clean > ruby each time, unless you update some of the required files (version > updates or security fixes, etc). > > 2) since you start with a clean checkout of the installer, you proceed > to "extract" and "prepare", which will create the required directories > in sandbox and unpack all the tools. Then it arrange (prepare) some of > the files and dlls to proceed with the next step. > > This is required every time you "clean" your sandbox (that means when > you change something in the recipes and want to do a fresh start). > > 3) configure task actually run the GNU autoconf script that is bundled > with ruby, and generate the proper config file and the makefile > required by the compile task. > > 4) compile will try (at this time) to build ruby with the bindings > that extract and prepare made available, at this time, only "zlib" and > "readline" are in place. > > 5) Install task emulates the directory structure the garbage collect > distribution have (is a ruby installation). > > The idea is later in the process rubygems gets installed over there, > but there are no recipes for it right now. > > 6) check task will ensure that everything built in the compile task > works out of the box. It runs the ruby test units to garantee that. > > Since readline is failing, I didn''t proceed to work on the MSI > installer generators or the RubyGems installation. >7) and then I download a new latest.zip, and since I am not sure nothing was removed between the different version the only safe way is to start with a fresh copy of the installer3 folder. BTW stuff breaks if the path to installer3 contains a space, this could be mentioned in README it it is not already.> > > > > I still think it would be overall easier to just put a copy of > > zlib1.dll into the extract_tools folder. > > > > > Is a bad practice add binaries to your repository... doing that you''re > forced to keep it updated from release to release... > > I''ll fix the extract utils to expand the downloaded zlib package.Yes, that''s what I had in mind. There''s even a separate download for zlib1.dll only.> > > > > > > > > > > > Yes, I tried some tasks until I was sure there''s nothing that does not > > > > produce any error but it might be that executing the tasks out of > > > > order leaves some fallout. > > > > > > > > > > > > > To make you happy: the Rakefile now have a defaults (recipes/ > > > defaults.rake) that: > > > > > > download, > > > extract, > > > prepare, > > > configure, > > > compile and > > > install > > > > Very nice, now running rake gives me a ruby build :-) > > > > However, it would be nice if this copy also included rubygems and/or > > rake so that rebuilding with this copy new was easier. To rebuild I > > had to > > > > - download rubygems > > - run the extract process of the installer so that I got bsdtar > > - extract rubygems > > - in rubygems run setup.rb configure (this for some reason performed > > the installation already) > > - run gem install rake > > > > > As explain above, RubyGems is not my main focus right now, but get a > stable working ruby build instead (there are lot of extensions don''t > build right now, like iconv, curses and others). > > However, if you want it included right now, I think a rubygems recipe > can be added to dependencies, and download, extract and install tasks > too.I guess it is time to add it because with rubygems and rake added you have enough to rebuild ruby with the image that is built by the recipes. Also you could release the image as .zip and somebody would perhaps try it out and test their favourite gems. I guess iconv and openssl are important but curses are broken anyway. Actually they might work on some language versions of Windows that do not (and cannot) use multibyte characters but then you have the additional problem of finding out the encoding that''s currently in use.> > > > After this I could nuke the install3 folder, extract a new one, and > > build ruby with the newly installed ruby. > > > > > That''s part of the idea, but how you nuke installer3? just use rake > clean to erase the sandbox, don''t stress the servers doing several > downloads (clean will not remove the downloads, clobber or a new > checkout will).But how do you update the scripts then? Thanks Michal
On Sat, Mar 22, 2008 at 6:33 AM, Michal Suchanek <hramrach at centrum.cz> wrote:> On 21/03/2008, Luis Lavena <luislavena at gmail.com> wrote: > > > > That''s part of the idea, but how you nuke installer3? just use rake > > clean to erase the sandbox, don''t stress the servers doing several > > downloads (clean will not remove the downloads, clobber or a new > > checkout will). > > But how do you update the scripts then? >It is part of a bazaar [0] repository. (bzr) is a distributed revision software similar to git or mercurial, but offer a more flexible workflow than these two and also has Windows as supported platform :-) If you don''t have Python installed, you can grab bazaar standalone setup [1] and add it to your PATH, bzr commands are quite similar to those from subversion, but just to get you started: 0) bzr whoami "your name <your at mail.com>" This is required to identify yourself on each commit :-D 1) bzr branch http://code.mmediasys.com/installer3/dev installer3.dev Doing that you will grab a copy of the current repository (dev branch) of installer3. 2) bzr update (inside installer3.dev) to get latest changes 3) (outside installer3.dev) bzr branch installer3.dev my-feature-or-fix-branch This will create a new branch (locally) that will let you fix, enhance and commit your enhancements without requiring write access to dev branch 4) when you have something to contribute back (until I set a better repository) just generate a bundle patch with: bzr send my-feature-or-fix-branch installer3.dev -o my-first-bundle.bundle Then you attach it to your mail and send it to the list :-) I''ll reply your other comments right now :-) Regards, -- Luis Lavena Multimedia systems - Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so. Douglas Adams
On Sat, Mar 22, 2008 at 11:05 AM, Luis Lavena <luislavena at gmail.com> wrote: Forgot to provide the links! :-P [0] http://bazaar-vcs.org/ [1] http://bazaar-vcs.org/WindowsDownloads -- Luis Lavena Multimedia systems - Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so. Douglas Adams
On Sat, Mar 22, 2008 at 6:33 AM, Michal Suchanek <hramrach at centrum.cz> wrote:> > I am not arguing against Rake, I am just saying that there are other > tools that could do the job, and thus there are people who aren''t > familiar with Rake. >Don''t take my comments personal. I self learned english and sometimes what I express in words are not actually 100% accurate on what I think it should express :-) I had used in the past a series of batch files to perform this, and if you take a closer look of the current code in installer2 in the rubyforge repository, you will see Curt and Andy already used something similar to bootstrap building One Click Installer By "people" I meant other Ruby developers, working on Windows, willing to collaborate in the project. The end-user of the project wouldn''t care about all the effort of download MinGW or build everything from scratch, they will only use the generated output (the MSI installer).> > > > > If you have used the garbagagecollect builds for your "Yet another > > Ruby Installer for Windows", then you know there is no documentation > > or way to recreate the process locally, and thus there is no way to > > simple fix the problems the current Ruby for Windows faces. > > > > Then you''re stuck each time to the releases made by somebody else and > > reply lots of tickets and support request when things don''t work out > > of the box, like WIN32OLE in 1.8.6-p110 (which was replaced soon > > enough by p111)... I think you get the picture. > > Yes, I have seen the earlier discussions about installers. It''s > actually not mine "yet another installer", I was just replying to the > thread. And I am aware of the grief with binary-only releases :-/ >Sorry about that, got lost in the translation :-P The dependency on external and non directly related developers to release fixes is a no-no. As example, take Mongrel project. I shared the most as possible of my Windows environment setup with the other Mongrel developers, so they don''t need me to be around when a bug/security fix show up. Right now, we depend 110% on garbagecollect releases, but the few times I politely asked the developers for his scripts (to recreate the build environment locally) he didn''t replied back. So I ended creating, from scratch again, the build scripts for VC6 and VC8 at that time, just to found it too complicated and decided to go for a Pure Ruby alternative :-)> > > > 1) after you unpack latest.zip (I actually name the folder > > installer3.dev, anyway) you jump in and perform the ''rake download'' > > task. > > Well, the zip archives contain an installer3 folder ... >Oh yes, the zip files contains installer3 folder, I meant the Bazaar branch :-)> > 7) and then I download a new latest.zip, and since I am not sure > nothing was removed between the different version the only safe way is > to start with a fresh copy of the installer3 folder. >That will end download *everything* again... please don''t do that... a huge hit to SF server is not polite :-(> BTW stuff breaks if the path to installer3 contains a space, this > could be mentioned in README it it is not already.Hehehe, I just get used to not use spaces in my path that forgot about it. I''ll rewrite README for better reading of requirements and limitations.> > > > I''ll fix the extract utils to expand the downloaded zlib package. > > Yes, that''s what I had in mind. There''s even a separate download for > zlib1.dll only. >We already have it, the download is zlib123-dll.zip I''ll patch the extract utils in the following minutes.> > I guess it is time to add it because with rubygems and rake added you > have enough to rebuild ruby with the image that is built by the > recipes.You''re correct, maybe I''ll do that before putting all the pending dependencies in place :-P> Also you could release the image as .zip and somebody would perhaps > try it out and test their favourite gems. >I''m still concerned by readline test being broken and that this generates IRB uses 50% of the processor being idle! (yes, that is a huge bug that need to be backported to 1.8 from trunk branch of ruby).> I guess iconv and openssl are important but curses are broken anyway. > Actually they might work on some language versions of Windows that do > not (and cannot) use multibyte characters but then you have the > additional problem of finding out the encoding that''s currently in > use. >We need to determine what built-in extensions are really used. OpenSSL is a must since signed gems (like Mongrel) will not work without it. for Iconv I think will follow the suggestions from _why and use the win32 alternative, the gnuwin32 packages for these tools are older and unmaintained :-( Thank you for your feedback Michal! I really appreciate it and it give me a push to fix these things! :-) Regards and have a nice weekend! -- Luis Lavena Multimedia systems - Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so. Douglas Adams