Hello Railers, I''ve been thinking about packaging Rails apps again this morning. I know there are a few projects out there that are at or approaching a releasable state and I''m wondering how everyone is going to go about releasing their software into the wild. I''ve also got about 10 Rails apps at work that I''d like to make easily maintainable. I''ve brought this up once or twice on IRC, but topics there tend to fly by so I thought I''d post here to get some kind of discussion (and archive thereof) going. Here are my thoughts. RubyGems - This seems like a fairly attractive way to release an application but I''m running into a couple walls while thinking about it. I''ve seen mention that there''s an Instiki gem available now, and I can wrap my head around the process of packaging up a rails app that uses its own webrick (as I believe Instiki does), that seems like it might be easy enough (given that some kind soul has provided a place to start with the Instiki gem). But what about applications that want to use Apache? Are there any thoughts on this? Could the gem just use the basic rails application structure and then you''d point Apache down into there? What about vendor/ and log/? I haven''t used the rails gem yet, but I imagine the gem for a rails based app would require the rails gem and this stuff would be taken care of magically. I am just digging into the gems chapter of PickAxe2 and the more I read the more I think that this may be the way to go. I''ll bang on a couple gemspecs later this morning. RPA - I don''t think Rails itself has been formally packaged up as an RPA, and I know nothing at all about RPA, so I don''t even know how or if this is a viable solution. Instiki Fashion - I was browsing through the Instiki tarball last night and a "no step 3" set up looks pretty nice. Though there may have to be a step 3, or 4, to configure a database, but this doesn''t seem as sexily upgradable as a gem. Tarball - The way I''m currently doing it now is I start with a fresh rails tarball and extract my application over top of it and go from there. It''s pretty simple (for an application author) and brute force, but not very maintainable and I wouldn''t expect a user of my application to want to do this. Tarballs that extract files into ''.'' are the work of Satan himself and I could not, in good conscience, distribute one to others. Well, all of that rabble is my current set of ideas. I''d love to hear any other ideas, or if you have any additions or corrections to my thinking please share them. I myself am leaning heavily toward using RubyGems for my open source and work projects both and if I have any success I''ll post it back here. Thanks for listening -Scott
On Fri, 5 Nov 2004 09:24:43 -0500, Scott Barron <scott-HDQKq3lYuGDk1uMJSBkQmQ@public.gmane.org> wrote:> Hello Railers, > > I''ve been thinking about packaging Rails apps again this morning. I > know there are a few projects out there that are at or approaching a > releasable state and I''m wondering how everyone is going to go about > releasing their software into the wild. I''ve also got about 10 Rails > apps at work that I''d like to make easily maintainable. I''ve brought > this up once or twice on IRC, but topics there tend to fly by so I > thought I''d post here to get some kind of discussion (and archive > thereof) going. Here are my thoughts. > > RubyGems - This seems like a fairly attractive way to release an > application but I''m running into a couple walls while thinking about it. > I''ve seen mention that there''s an Instiki gem available now, and I can > wrap my head around the process of packaging up a rails app that uses > its own webrick (as I believe Instiki does), that seems like it might be > easy enough (given that some kind soul has provided a place to start > with the Instiki gem). But what about applications that want to use > Apache? Are there any thoughts on this? Could the gem just use the > basic rails application structure and then you''d point Apache down into > there? What about vendor/ and log/? I haven''t used the rails gem yet, > but I imagine the gem for a rails based app would require the rails gem > and this stuff would be taken care of magically. I am just digging into > the gems chapter of PickAxe2 and the more I read the more I think that > this may be the way to go. I''ll bang on a couple gemspecs later this > morning. >Jim Weirich has made a lot of progress in the area of packaging Rails apps as gems. I think it''s a fantastic idea. You''re right about making the gem depend on Rails itself. The issues center around, as you say, post "install" setup. Jim and I were discussing the possibility of creating a generalized post-installer for Rails apps. The idea of being able to: `gem install somerailsapp` and have it ready to go with all of its dependencies is about as exciting as it gets. I''ve CC''d Jim on this message in case he has anything to add. You''ve got Apache config to worry about but you also have things like loading database schemas, etc. Jim has spent a lot more time thinking about the issues than I have. There is a working Instiki gem that has been contributed by a Ruby community member (Stoyan Zekhov). We did some successful testing last night on irc. It works now, but there are a couple of small tweaks that would be nice to add before releasing it. I''m hoping to get to that this evening and give the final version to David. As I said, gem-installed Rails apps would be _killer_. I''d love to see any efforts made, and to hear feedback about how we could change RubyGems to make it easier.> RPA - I don''t think Rails itself has been formally packaged up as an > RPA, and I know nothing at all about RPA, so I don''t even know how or if > this is a viable solution. >I think there''s an old version of Rails in RPA. I''m sure it would be possible to do this with RPA. But, the focus of RPA is not on developers distributing their applications but on a centralized team repackaging and distributing them. Given that fact, they can do whatever they need to do to get the Rails apps working. I would definitely see this as a possible means of distribution but not so much a developer-driven means.> Instiki Fashion - I was browsing through the Instiki tarball last night > and a "no step 3" set up looks pretty nice. Though there may have to be > a step 3, or 4, to configure a database, but this doesn''t seem as sexily > upgradable as a gem. > > Tarball - The way I''m currently doing it now is I start with a fresh > rails tarball and extract my application over top of it and go from > there. It''s pretty simple (for an application author) and brute force, > but not very maintainable and I wouldn''t expect a user of my application > to want to do this. Tarballs that extract files into ''.'' are the work > of Satan himself and I could not, in good conscience, distribute one to > others. >:) That Satan and his messy tarballs, damn him! (oh, too late)> Well, all of that rabble is my current set of ideas. I''d love to hear > any other ideas, or if you have any additions or corrections to my > thinking please share them. I myself am leaning heavily toward using > RubyGems for my open source and work projects both and if I have any > success I''ll post it back here. >While confessing to being horribly time-strapped these days, please share your pains and successes and don''t hesitate to ask for RubyGems help. I''m on irc (irc.freenode.net) as "chad" and (more often) on AOL IM as "cefogc". Also, Marcel Molina may not appreciate me volunteering him, but he''s done quite a bit of great work with packaging gems and would probably be a good resource. "noradio" on irc. -- Chad Fowler http://chadfowler.com http://rubycentral.org http://rubygarden.org http://rubygems.rubyforge.org (over 20,000 gems served!)
>>>>> "Scott" == Scott Barron <scott-HDQKq3lYuGDk1uMJSBkQmQ@public.gmane.org> writes:Scott> Hello Railers, I''ve been thinking about packaging Rails Scott> apps again this morning. I know there are a few projects Scott> out there that are at or approaching a releasable state and Scott> I''m wondering how everyone is going to go about releasing Scott> their software into the wild. Scott, What kind of users would you expect? For Ruby enthusiasts .tar.gz with INSTALL file or gems seems reasonable. For production use distribution would depend on target OS. For Linux .tar.gz with detailed INSTALL file is lowest common denominotar, while .deb and .rpn would allow proper pre-install dependency checking, and install-time configuration. For Windows, .zip+INSTALL or a graphical installer with install-time dependency checking would be native. To really "do things as Romans do" you''d want to comply with Linux filesystem standard (e.g. logs go to /var/log). Either way, for *users* it seems best to include full Rails distribution that is known to work. Without "UNIX shared library major/minor version number"-style versioning there is no real way to safely share same instance of Rails from multiple applications. Nor there is much point, as ABI gets backward incompatible changes with every release and Rails distribution size is trivial. For fellow developers you''d want an equivalent of "configure/build/install(possibly in-place)" process, as opposed to "install from installer medium" process for users. Rails distribution should be decoupled, allowing one to track both Rails and your app releases, and to make local change to both. I''ve been thinking of how to do this best. I think something conceptually similar to Rails gems does currently would work: * Rails distribution is changed to only contain current vendor directory and utility stuff. So, when you d/l .tar.gz from the website it expands to: ./rails-1.23/ ./rails-1.23/activerecord/... similarly Edge Railers create mkdir ./rails-cvs/ cd ./rails-cvs cvs update Conceptually $(RAILS_DIR)/ hierarchy is kept read-only. * When you want to build a new app you do mkdir ./my_app cd ./my_app $(RAILS_DIR)/bin/create_app which creates ./app/ ./app/controllers/ ./config/ etc copies/creates ./Rakefile ./config/database.yml ./public/dispatch.cgi etc while configuring load_path in dispatch.* to point to $(RAILS_DIR). This allows $(RAILS_DIR) and $(YOUR_APP_DIR) to be independent and lets one share single, and possibly locally modified, $(RAILS_DIR) between multiple app instances. * At this point you can checkin $(YOUR_APP_DIR)/... into your own source control and start adding controllers, etc. You can also cvs update Rails. The tricky part is when Rails changes something that is copied/created by $(RAILS_DIR)/bin/create_app. Reasonable behavior can be implemented for $(RAILS_DIR)/bin/create_app to do the safe thing when re-running it. Also, these kind of files could be split into $(RAILS_DIR)/skel/... hierarchy, with changes to any files here prominently mentioned in checking and release notes, so that we can manually merge new Rails changes into our apps. Gleb
On Fri, Nov 05, 2004 at 02:35:23PM -0500, Chad Fowler wrote:> On Fri, 5 Nov 2004 09:24:43 -0500, Scott Barron <scott-HDQKq3lYuGDk1uMJSBkQmQ@public.gmane.org> wrote: > > Hello Railers, > >[Snipping out all non-gems related topics for brevity]> Jim Weirich has made a lot of progress in the area of packaging Rails > apps as gems. I think it''s a fantastic idea. You''re right about > making the gem depend on Rails itself. The issues center around, as > you say, post "install" setup. Jim and I were discussing the > possibility of creating a generalized post-installer for Rails apps. > The idea of being able to: `gem install somerailsapp` and have it > ready to go with all of its dependencies is about as exciting as it > gets. I''ve CC''d Jim on this message in case he has anything to add. > You''ve got Apache config to worry about but you also have things like > loading database schemas, etc. Jim has spent a lot more time thinking > about the issues than I have. > > There is a working Instiki gem that has been contributed by a Ruby > community member (Stoyan Zekhov). We did some successful testing last > night on irc. It works now, but there are a couple of small tweaks > that would be nice to add before releasing it. I''m hoping to get to > that this evening and give the final version to David. > > As I said, gem-installed Rails apps would be _killer_. I''d love to > see any efforts made, and to hear feedback about how we could change > RubyGems to make it easier. >[...]> > While confessing to being horribly time-strapped these days, please > share your pains and successes and don''t hesitate to ask for RubyGems > help. I''m on irc (irc.freenode.net) as "chad" and (more often) on AOL > IM as "cefogc". Also, Marcel Molina may not appreciate me > volunteering him, but he''s done quite a bit of great work with > packaging gems and would probably be a good resource. "noradio" on > irc.Hey Chad, I was able to work a bit on it on Friday morning, and here are some of the issues I ran into (I''ll try to spare any "drama" for those who can''t handle truth). #1. It seems that if you try to install a local gem and it depends on a remote gem, it will then try to get the first gem remotely instead of using it locally. This would mostly be a problem for someone who is maintaining some gems. I''m thinking of a situation like this: I''ve got 10 rails apps at work I want to make easily maintainable and deployable as gems, and of course I wouldn''t want them on the main gems server. It seems that one can run his own gems server, if one did that would he have to maintain the other gems (eg rails et al) on the local gem server? I haven''t looked too much into running my own local gem server yet, but Marcel did seem to confirm the initial problem I was having. #2. I guess the rest is all "post-install" issues. It seemed that RubyGems was not maintaining the +x bit on the files (eg dispatch.cgi) that need to be executable. I was able to run a rails app (as apache CGI) from the gem with a little tweaking. Of course, if you bump the revision of the app you''d have to change your apache config, but I don''t know if that''s a terribly big deal. So the main parts of post-install would be apache config, db config, setting a thing or to back to +x. I think those were all the major issues I had. It can be made to work as is, though it would be sexy if RubyGems had some sort of post-install script facility (I think RPM has this?). It would definitely solve all the problems in #2, though I suppose an install script could be included as an executable in the rails app gem for now that might alleviate the problems. At any rate, I''d love for the world to have gem-installable rails apps. It''s a sexy thought as I would like to both release an app or two to the world and ease my internal app management with gems. I''m here to help in any way that I can (I''m htonl on irc, by the way). Cheers -Scott after a little tweaking. I had to set that
On Sun, 7 Nov 2004 10:05:34 -0500, Scott Barron <scott-HDQKq3lYuGDk1uMJSBkQmQ@public.gmane.org> wrote:> On Fri, Nov 05, 2004 at 02:35:23PM -0500, Chad Fowler wrote: > > On Fri, 5 Nov 2004 09:24:43 -0500, Scott Barron <scott-HDQKq3lYuGDk1uMJSBkQmQ@public.gmane.org> wrote: > > > Hello Railers, > > > > > > Hey Chad, > > I was able to work a bit on it on Friday morning, and here are some of > the issues I ran into (I''ll try to spare any "drama" for those who can''t > handle truth). > > #1. It seems that if you try to install a local gem and it depends on a > remote gem, it will then try to get the first gem remotely instead of > using it locally. This would mostly be a problem for someone who is > maintaining some gems.Yea, this is a known bug that will be fixed in the next release. Sorry. :)> > #2. I guess the rest is all "post-install" issues. It seemed that > RubyGems was not maintaining the +x bit on the files (eg dispatch.cgi) > that need to be executable. I was able to run a rails app (as apache > CGI) from the gem with a little tweaking. Of course, if you bump the > revision of the app you''d have to change your apache config, but I don''t > know if that''s a terribly big deal. So the main parts of post-install > would be apache config, db config, setting a thing or to back to +x. >I''m surprised about the +x bit. Need to look into that one. So, you''re saying you pointed apache directly at the location of the installed gem? Interesting. Not what I was expecting at all. I was envisioning some kind of install-then-deploy scenario. Do you have a sample app you were playing with that we could try?> I think those were all the major issues I had. It can be made to work > as is, though it would be sexy if RubyGems had some sort of post-install > script facility (I think RPM has this?). It would definitely solve all > the problems in #2, though I suppose an install script could be included > as an executable in the rails app gem for now that might alleviate the > problems. >Yea. There has been a lot of discussion and not very much work done regarding post-install scripts. I''m sure the day will come, but I have yet to find the time to implement it (nor has anyone else). You''re right, though.> At any rate, I''d love for the world to have gem-installable rails apps. > It''s a sexy thought as I would like to both release an app or two to the > world and ease my internal app management with gems. I''m here to help > in any way that I can (I''m htonl on irc, by the way). >Ahh, now I know who you are! You people and your weird irc nicks! ;) Thanks, Chad Fowler http://chadfowler.com http://rubycentral.org http://rubygarden.org http://rubygems.rubyforge.org (over 20,000 gems served!)
> > #1. It seems that if you try to install a local gem and it depends on a > > remote gem, it will then try to get the first gem remotely instead of > > using it locally. This would mostly be a problem for someone who is > > maintaining some gems. > > Yea, this is a known bug that will be fixed in the next release. Sorry. :)Most excellent!> So, you''re saying you pointed apache directly at the location of the > installed gem? Interesting. Not what I was expecting at all. I was > envisioning some kind of install-then-deploy scenario. Do you have a > sample app you were playing with that we could try?This is what I did, but mostly as a "proof of concept" thing to get started. It didn''t really "feel good" pointing apache down into the gems directory, but it did work. A deploy script would sit much more cozier with me. The app I was playing with is internal and not releasable but if I get some time I''ll do up the Elite Journal app and maybe bang on a deploy script. Have you or Jim Weirich done any work (or even outlining) of such a deploy script? I''m thinking it''d have to do at least the following: - Get from the user the place the application would be insatlled - With this it should probably check if there is an existing application installed there, and possibly check against version numbers? Thoughts on updating this? - Run the `rails'' command to start a nice fresh rails directory. - Add the application files to the new rails set up. - Get some DB config parameters from the user to set up database.yml. - Spit out a hunk of code for Apache configuration. - If this is going to be a non-vhost apache it would also need to make some changes to .htaccess. - I don''t know anything about webrick, fcgi, or mod_ruby installations, anyone with thoughts on that?> Yea. There has been a lot of discussion and not very much work done > regarding post-install scripts. I''m sure the day will come, but I > have yet to find the time to implement it (nor has anyone else). > You''re right, though.Indeed, time is hard to come by these days. I think if some kind of deploy script is made, which can be called manually now, can easily graft into a post-install script once that facility becomes available in RubyGems. I''ve got a few important things on the old TODO list for the beginning of the week, but once I clear those out I''ll have some more time to work on this (unless someone beats me to it ;).> Ahh, now I know who you are! You people and your weird irc nicks! ;)Weird, creative, it''s all the same ;) (I think someone already owns "scott" so I went with the next best thing). -Scott
Some comments based on what I did with storycards ... Scott Barron said:> [...] Have you or Jim Weirich done any work > (or even outlining) of such a deploy script? I''m thinking it''d have to > do at least the following: > - Get from the user the place the application would be insatlledI only stored site changable stuff outside the gem directory. This included configuration information and the logging directory.> - With this it should probably check if there is an existing > application installed there, and possibly check against version > numbers? Thoughts on updating this?Since I only stored config information, I would read the configs if they already existed.> - Run the `rails'' command to start a nice fresh rails directory.No, because much of the application resided in the gem directory. No need to run rails. Even if the app were stored outside of the gem area, I think I would just copy it from the gem area master. It should all be there, no need to run the rails initialize directory script.> - Add the application files to the new rails set up.No need, if config only.> - Get some DB config parameters from the user to set up database.yml.Yes, done through an interactive configuration script.> - Spit out a hunk of code for Apache configuration. > - If this is going to be a non-vhost apache it would also need to > make some changes to .htaccess.I didn''t address this since I am using webrick only for now.> - I don''t know anything about webrick, fcgi, or mod_ruby > installations, anyone with thoughts on that?Webrick was easy. The main program was installed into /usr/local/bin by gem (as all commands are) and you just ran it. The main program took care of default ports and what not, but would accept options (as does the original webrick dispatcher). Not hard at all. You also need to address the initialization of the database schema. I didn''t do anything here but make the schema available in a MySQL format for manual creation. -- -- Jim Weirich jim-Fxty1mrVU9GlFc2d6oM/ew@public.gmane.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)
On Mon, Nov 08, 2004 at 02:26:04PM -0000, Jim Weirich wrote:> Some comments based on what I did with storycards ... > > Scott Barron said: > > [...] Have you or Jim Weirich done any work > > (or even outlining) of such a deploy script? I''m thinking it''d have to > > do at least the following: > > - Get from the user the place the application would be insatlled > > I only stored site changable stuff outside the gem directory. This > included configuration information and the logging directory.Well, wouldn''t the user want to choose where, outside the gem directory, these things would go? If one is using apache, then either public/ will have to go outside of the gems directory (preferred), or apache will have to point at public/ in the gems directory.> > - Run the `rails'' command to start a nice fresh rails directory. > > No, because much of the application resided in the gem directory. No need > to run rails. > > Even if the app were stored outside of the gem area, I think I would just > copy it from the gem area master. It should all be there, no need to run > the rails initialize directory script.Indeed, you''re right here. I remember now that I did have to change the way rails was pulled in in shared.rb from "require ''active_record''" to "require_gem ''activerecord''" and such. So yeah, a new ''rails'' would not have to be run. My mistake.> > - Spit out a hunk of code for Apache configuration. > > - If this is going to be a non-vhost apache it would also need to > > make some changes to .htaccess. > > I didn''t address this since I am using webrick only for now.Good, then we now have someone addressing Apache and someone addressing webrick, so we should get all the bases covered (though I still have not set up fast cgi and don''t know what that entails yet).> > - I don''t know anything about webrick, fcgi, or mod_ruby > > installations, anyone with thoughts on that? > > Webrick was easy. The main program was installed into /usr/local/bin by > gem (as all commands are) and you just ran it. The main program took care > of default ports and what not, but would accept options (as does the > original webrick dispatcher). Not hard at all.I figured webrick would be quite easy to set up (given the example of Instiki and it''s lack of a step 3). Just as long as nothing assumes /usr/local/bin (I know the dispatch and script files of rails do, something I feel should be changed, but that''s another issue).> You also need to address the initialization of the database schema. I > didn''t do anything here but make the schema available in a MySQL format > for manual creation.I think this is fine. To auto-load it you''d have to get from them db credentials for a user with appropriate permissions (create database, create tables, etc) which they may or may not have. Probably more trouble than it''s worth (though I could be convinced otherwise). Thanks for your input Jim, very valuable and much appreciated (story cards is pretty cool, by the way). Thanks -Scott
Scott Barron said:>> I only stored site changable stuff outside the gem directory. This >> included configuration information and the logging directory. > > Well, wouldn''t the user want to choose where, outside the gem directory, > these things would go? If one is using apache, then either public/ will > have to go outside of the gems directory (preferred), or apache will > have to point at public/ in the gems directory.Hmmm ... the public area wasn''t an issue with the webrick version. Let''s see what''s in the public area... the dispatch code, javascript code, stylesheets, the .htaccess file, images and static html. If any of those things need to be customized by the end user, then they should definitately reside outside the gem directory. If it makes it easier for apache to access the data, then I have no problem with that either. Let me just add this too. The way I implemented a lot of this stuff in storycards is to replace the require ''../../../../../../../'' stuff with utility calls. For example, to access something in the log directory was: log_file = File.join(Storycards.log_dir, "file.log") The log_dir method would hunt down the proper location the first time it is called. Thereafter it would return whatever it found the first time. I also had a Storycards.data_dir. This gave me a lot of freedom in moving parts of Rails around to where I wanted them. -- -- Jim Weirich jim-Fxty1mrVU9GlFc2d6oM/ew@public.gmane.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)