Duane Johnson
2005-Dec-19 18:25 UTC
How about a forked session supervisor in dispatch.fcgi?
I was talking to a friend of mine who''s developing PHP on trax and we came up with a solution to the ruby session file problem. I''m not sure if this idea has come up before, so I wanted to run it by everyone before making any code contributations and/or talking to the Rails core about it. It seems to me that the current situation we''re in with regard to session files in the /tmp folder is a problem. Without a cron job or other service to supervise the sessions, they can quickly get out of hand. What if dispatch.fcgi (and script/server) had an additional thread running in the background that would obey session clean-up rules such as "Delete all sessions older than one week every 24 hours". Is this a solution that would work? Has it been attempted? Regards, Duane Johnson (canadaduane) http://blog.inquirylabs.com/
Matthew Beale
2005-Dec-19 18:28 UTC
Re: How about a forked session supervisor in dispatch.fcgi?
On Mon, 2005-12-19 at 11:25 -0700, Duane Johnson wrote:> I was talking to a friend of mine who''s developing PHP on trax and we > came up with a solution to the ruby session file problem. I''m not > sure if this idea has come up before, so I wanted to run it by > everyone before making any code contributations and/or talking to the > Rails core about it. > > It seems to me that the current situation we''re in with regard to > session files in the /tmp folder is a problem. Without a cron job or > other service to supervise the sessions, they can quickly get out of > hand. > > What if dispatch.fcgi (and script/server) had an additional thread > running in the background that would obey session clean-up rules such > as "Delete all sessions older than one week every 24 hours".Ruby doesn''t do native threads, so the dispatch.fcgi process would still be running as long as the cleanup was running. There are some third party libraries for ruby that attempt to do native threads, but they are not cross platform, last time I checked. -- -Matthew Beale mixonic-8rZIAEcCR/xWk0Htik3J/w@public.gmane.org :: 607 227 0871 Resume and Portfolio @ http://madhatted.com
Matt S Trout
2005-Dec-19 22:19 UTC
Re: How about a forked session supervisor in dispatch.fcgi?
Duane Johnson wrote:> I was talking to a friend of mine who''s developing PHP on trax and we > came up with a solution to the ruby session file problem. I''m not > sure if this idea has come up before, so I wanted to run it by > everyone before making any code contributations and/or talking to the > Rails core about it. > > It seems to me that the current situation we''re in with regard to > session files in the /tmp folder is a problem. Without a cron job or > other service to supervise the sessions, they can quickly get out of > hand. > > What if dispatch.fcgi (and script/server) had an additional thread > running in the background that would obey session clean-up rules such > as "Delete all sessions older than one week every 24 hours". > > Is this a solution that would work? Has it been attempted?You might want to have a quick look at http://search.cpan.org/~agrundma/Catalyst-Plugin-Scheduler-0.03/lib/Catalyst/Plugin/Scheduler.pm which is how we''re getting round this in Catalyst. Backgrounds threads etc. were discussed but the simple answer won out for portability reasons. -- Posted via http://www.ruby-forum.com/.
Duane Johnson
2005-Dec-20 05:56 UTC
Re: How about a forked session supervisor in dispatch.fcgi?
On Dec 19, 2005, at 11:28 AM, Matthew Beale wrote:>> What if dispatch.fcgi (and script/server) had an additional thread >> running in the background that would obey session clean-up rules such >> as "Delete all sessions older than one week every 24 hours". > > Ruby doesn''t do native threads, so the dispatch.fcgi process would > still > be running as long as the cleanup was running. > > There are some third party libraries for ruby that attempt to do > native > threads, but they are not cross platform, last time I checked.No, that''s true. But what about just a regular old cross-platform Ruby Thread? Duane Johnson (canadaduane) http://blog.inquirylabs.com/
Ezra Zygmuntowicz
2005-Dec-20 17:22 UTC
Re: How about a forked session supervisor in dispatch.fcgi?
On Dec 19, 2005, at 9:56 PM, Duane Johnson wrote:> > On Dec 19, 2005, at 11:28 AM, Matthew Beale wrote: >>> What if dispatch.fcgi (and script/server) had an additional thread >>> running in the background that would obey session clean-up rules >>> such >>> as "Delete all sessions older than one week every 24 hours". >> >> Ruby doesn''t do native threads, so the dispatch.fcgi process would >> still >> be running as long as the cleanup was running. >> >> There are some third party libraries for ruby that attempt to do >> native >> threads, but they are not cross platform, last time I checked. > > No, that''s true. But what about just a regular old cross-platform > Ruby Thread? > > Duane Johnson > (canadaduane) > http://blog.inquirylabs.com/Duane- I tried to use a plain old ruby thread in a site i built in order to flush the cache at timed intervals and clear the sessions like you want to do. It was a nightmare and didn''t work well at all. Long story short, I switched to cron jobs and ruby/shell scripts to do the same thing for me and it works out great this way. I think if you are trying to flush the sessions at a timed interval that cron is the way to go. You can make a quick script to check the mtime on the file and delete all the sessions older then 12 hours(or whatever you want) Cheers- -Ezra Zygmuntowicz Yakima Herald-Republic WebMaster http://yakimaherald.com 509-577-7732 ezra-gdxLOakOTQ9oetBuM9ipNAC/G2K4zDHf@public.gmane.org
Duane Johnson
2005-Dec-20 19:04 UTC
Re: How about a forked session supervisor in dispatch.fcgi?
On Dec 20, 2005, at 10:22 AM, Ezra Zygmuntowicz wrote:> > On Dec 19, 2005, at 9:56 PM, Duane Johnson wrote: > >> >> On Dec 19, 2005, at 11:28 AM, Matthew Beale wrote: >>>> What if dispatch.fcgi (and script/server) had an additional thread >>>> running in the background that would obey session clean-up rules >>>> such >>>> as "Delete all sessions older than one week every 24 hours". >>> >>> Ruby doesn''t do native threads, so the dispatch.fcgi process >>> would still >>> be running as long as the cleanup was running. >>> >>> There are some third party libraries for ruby that attempt to do >>> native >>> threads, but they are not cross platform, last time I checked. >> >> No, that''s true. But what about just a regular old cross-platform >> Ruby Thread? >> >> Duane Johnson >> (canadaduane) >> http://blog.inquirylabs.com/ > > > Duane- > > I tried to use a plain old ruby thread in a site i built in order > to flush the cache at timed intervals and clear the sessions like > you want to do. It was a nightmare and didn''t work well at all. > Long story short, I switched to cron jobs and ruby/shell scripts to > do the same thing for me and it works out great this way. I think > if you are trying to flush the sessions at a timed interval that > cron is the way to go. You can make a quick script to check the > mtime on the file and delete all the sessions older then 12 hours > (or whatever you want)Thanks for the great feedback, Ezra. It sounds like this is some hard-earned advice. I''ll stick with cron jobs and "tmpwatch" that my friend John pointed me to:> There is a Freebsd port in /usr/ports/sysutils/tmpwatch > > /usr/local/sbin/tmpwatch -m -f 1 /tmp/ruby_sessions > > -m = makes it look at last modified time > -f = force removal > 1 = deletes only files (-m modified time) 1 hour or older > /tmp/ruby_sessions = directory to delete files in(For anyone trying to implement the above, it assumes you''ve put your ruby session files in /tmp/ruby_sessions which is NOT the default for Rails. If you want to configure your app to do that also, use this line in environment.rb: ActionController::CgiRequest::DEFAULT_SESSION_OPTIONS[:tmpdir] = ''/ tmp/ruby_sessions'' and make sure /tmp/ruby_sessions exists and has the correct permissions) Ezra, is there any further detail you can give me on the cause for failure regarding ruby Threads? I''m still curious to know what stumbling blocks lie along that path :) Thanks, Duane Johnson (canadaduane) http://blog.inquirylabs.com/
Ezra Zygmuntowicz
2005-Dec-21 00:37 UTC
Re: How about a forked session supervisor in dispatch.fcgi?
On Dec 20, 2005, at 11:04 AM, Duane Johnson wrote:> > On Dec 20, 2005, at 10:22 AM, Ezra Zygmuntowicz wrote: > >> >> On Dec 19, 2005, at 9:56 PM, Duane Johnson wrote: >> >>> >>> On Dec 19, 2005, at 11:28 AM, Matthew Beale wrote: >>>>> What if dispatch.fcgi (and script/server) had an additional thread >>>>> running in the background that would obey session clean-up >>>>> rules such >>>>> as "Delete all sessions older than one week every 24 hours". >>>> >>>> Ruby doesn''t do native threads, so the dispatch.fcgi process >>>> would still >>>> be running as long as the cleanup was running. >>>> >>>> There are some third party libraries for ruby that attempt to do >>>> native >>>> threads, but they are not cross platform, last time I checked. >>> >>> No, that''s true. But what about just a regular old cross- >>> platform Ruby Thread? >>> >>> Duane Johnson >>> (canadaduane) >>> http://blog.inquirylabs.com/ >> >> >> Duane- >> >> I tried to use a plain old ruby thread in a site i built in order >> to flush the cache at timed intervals and clear the sessions like >> you want to do. It was a nightmare and didn''t work well at all. >> Long story short, I switched to cron jobs and ruby/shell scripts >> to do the same thing for me and it works out great this way. I >> think if you are trying to flush the sessions at a timed interval >> that cron is the way to go. You can make a quick script to check >> the mtime on the file and delete all the sessions older then 12 >> hours(or whatever you want) > > Thanks for the great feedback, Ezra. It sounds like this is some > hard-earned advice. I''ll stick with cron jobs and "tmpwatch" that > my friend John pointed me to: > >> There is a Freebsd port in /usr/ports/sysutils/tmpwatch >> >> /usr/local/sbin/tmpwatch -m -f 1 /tmp/ruby_sessions >> >> -m = makes it look at last modified time >> -f = force removal >> 1 = deletes only files (-m modified time) 1 hour or older >> /tmp/ruby_sessions = directory to delete files in > > (For anyone trying to implement the above, it assumes you''ve put > your ruby session files in /tmp/ruby_sessions which is NOT the > default for Rails. If you want to configure your app to do that > also, use this line in environment.rb: > ActionController::CgiRequest::DEFAULT_SESSION_OPTIONS[:tmpdir] = ''/ > tmp/ruby_sessions'' > and make sure /tmp/ruby_sessions exists and has the correct > permissions) > > Ezra, is there any further detail you can give me on the cause for > failure regarding ruby Threads? I''m still curious to know what > stumbling blocks lie along that path :) > > Thanks, > Duane Johnson > (canadaduane) > http://blog.inquirylabs.com/Duane- The main problem with ruby threads was that with a fastcgi architecture, there isn''t any easy way I could find to run an extra ruby thread on only one instance of the fcgi runners for my app. So trying to use a ruby thread to watch /tmp and to expire caches ended up running one of these threads in each fcgi listener which in turn caused them to start conflicting with each other and contending for file locks and other stuff that wasn''t any fun. I have cached pages in my newspaper website that need to be expired on a specific schedule and these pages get their data from other servers not controlled by rails(proprietary BaseView DB). So there was no hook to be observed to start my cache expiries. Also I found that running extra long running threads off of my rails processes caused other random errors that were hard to debug. And once I gave up on it and went with cron, all of these hassles went away and things "just worked". Granted my app is not your normal rails app. Very little of the data is managed by ActiveRecord so there may be other issues I had that you wouldn''t necessarily run into with a more standard rails app. If you want to make a system in ruby for watching your sessions and expiring caches on a time based system, I would recommend using a drb server to do this and having your rails app send commands to it or something like that. Which reminds me, I have a new plugin that I need to polish up that I will release in the next week or two. Its a rails''/drb system for queueing up long running jobs. So its a drb based server that can still use ActiveRecord and access your domain models but it does not depend on the request/response cycle. Its main use would be if you have expensive queries for reporting or other long running system tasks, you send them to be done by the drb server disconnected from your main rails app. And then you can come back later and query the drb server for the results. So for instance say you need to run a few large queries and run some system/shell scripts from your rails app, you could have a page that kicks off control of these tasks to the drb server while your page shows a progress bar that queries the drb server every 3 or 4 seconds for an update of how far along it is with its task. And then once it finishes and your app quries it again for its progress, it gives you the results. Anyway , I''m rambling off topic here. The short of it all is that I have run into quite a few problems trying to use threads in my rails apps. And pulling them out into either a drb server or just cron jobs has eliminated these hard to debug errors. Hope that helps- -Ezra Zygmuntowicz Yakima Herald-Republic WebMaster http://yakimaherald.com 509-577-7732 ezra-gdxLOakOTQ9oetBuM9ipNAC/G2K4zDHf@public.gmane.org
Duane Johnson
2005-Dec-21 04:38 UTC
Re: How about a forked session supervisor in dispatch.fcgi?
On Dec 20, 2005, at 5:37 PM, Ezra Zygmuntowicz wrote:> <snip> > Anyway , I''m rambling off topic here. The short of it all is that > I have run into quite a few problems trying to use threads in my > rails apps. And pulling them out into either a drb server or just > cron jobs has eliminated these hard to debug errors. > > Hope that helps-Absolutely. Thanks! Duane Johnson (canadaduane) http://blog.inquirylabs.com/ _______________________________________________ Rails mailing list Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org http://lists.rubyonrails.org/mailman/listinfo/rails
Apparently Analagous Threads
- Can''t get fcgi working with Apache2 on Mac OS X
- OpenSSL ''no ciphers'' error on OS X when using Net::HTTP
- Family Connection 1.1.2
- ror/lighttpd - HTML files via port 80 become application/octet-stream?
- apache2 fastcgi (fcgi) internal server errors, still :(