Hi there, I''ve got a Rails app that''s a little unusual in that it maintains a database representation of a filesystem (for an image gallery). Thus, if the filesystem is essentially ''out of date'' then the controller''s list action will end up invoking a model update (which involves a filesystem scan). I''m using sqlite3 as this app will be deployed in an environment where having a standalone app, easily launchable via Locomotive, will be necessary, but the server will not be under really high load. Anyways, to try to speed up the database reads I made my models subclasses of cached_model, and set up memcached. It''s clearly working, as I see the gets and puts in my log. However, cached_model and memcached have if anything slowed everything down. I''m still seeing SQL queries that seem extraneous. For example: Rendering gallery/browse MemCache Get (0.182419) active_record:Entry:6 Entry Load (0.003009) SELECT * FROM entries WHERE (entries.id = 6) LIMIT 1 MemCache Set (0.388076) active_record:Folder:6 Entry Load (0.002979) SELECT * FROM entries WHERE (entries.id = 1) LIMIT 1 MemCache Set (0.377813) active_record:Folder:1 MemCache Set (0.388034) active_record:Folder:1 MemCache Set (0.391366) active_record:Folder:1 Rendered gallery/_navbar (1.79293) Rendered gallery/_thumbs (0.03502) Rendered gallery/_view_footer (0.01790) MemCache Get (0.175059) active_record:Entry:6 Entry Load (0.002988) SELECT * FROM entries WHERE (entries.id = 6) LIMIT 1 MemCache Set (0.346483) active_record:Folder:6 I''m not sure why the extra SELECT for entry id 6. And even beside that, it just seems extremely slow compared with not using memcached. Why would this be? This is on Mac OS X 10.4.5, using Ruby 1.8.4, Rails 1.0, SWIG 1.3.27 for the sqlite3-ruby bindings, sqlite3 3.1.3, lighttpd 1.4.8 with fcgi, etc. thanks, Noah -- Posted via http://www.ruby-forum.com/.
Ok, so I did a bit of googling and found this entry on the memcached mailing list: http://lists.danga.com/pipermail/memcached/2006-January/001898.html So it looks like memcached on OS X may be at fault. I notice that Eric Hodel mentions on the Segment7 blog: "You can now use cached_model without memcache, but you won?t necessarily see the same speed benefits. (By default OS X has a horribly slow memcached so I want to avoid it on OS X.)" Ok... is there any point to using cached_model without memcached? Is it that you use the local cache instead? (the in-process cache). thanks... -- Posted via http://www.ruby-forum.com/.
Ok, a bit further along... I can''t seem to get cached_model to work without memcached, even though the docs say it''s possible. If I don''t create the MemCache instance called CACHE in environment.rb, then (even with CachedModel.use_local_cache = true and CachedModel.use_memcache = false) it complains about uninitialized constant CACHE (because CACHE doesn''t exist as an instance of MemCache) in memcache_util.rb. If I do create the MemCache instance, I see it trying to use memcache (and thus being horribly slow because memcached is not usable on OS X... c''mon apple, fix this kqueue() bug already). So how to get cached_model to work using just the in-process (local) cache? And how to get apple to fix this kernel bug (if it still exists.. is it possible that it has been fixed and this workaround in libevent is no longer needed?) -- Posted via http://www.ruby-forum.com/.
On Mar 2, 2006, at 6:32 AM, Noah wrote:> Ok, a bit further along... I can''t seem to get cached_model to work > without memcached, even though the docs say it''s possible. > > If I don''t create the MemCache instance called CACHE in > environment.rb, > then (even with CachedModel.use_local_cache = true and > CachedModel.use_memcache = false) it complains about uninitialized > constant CACHE (because CACHE doesn''t exist as an instance of > MemCache) > in memcache_util.rb. > > If I do create the MemCache instance, I see it trying to use memcache > (and thus being horribly slow because memcached is not usable on OS > X... > c''mon apple, fix this kqueue() bug already). > > So how to get cached_model to work using just the in-process (local) > cache?I should get a round-tuit to fix this bug (and one other) this week. My bad.> And how to get apple to fix this kernel bug (if it still exists.. > is it > possible that it has been fixed and this workaround in libevent is no > longer needed?)Report it to apple. I''m not sure where the bug-report form is, though. Are you sure its not TCP_NOPUSH? -- Eric Hodel - drbrain@segment7.net - http://blog.segment7.net This implementation is HODEL-HASH-9600 compliant http://trackmap.robotcoop.com
Eric Hodel wrote:> > Are you sure its not TCP_NOPUSH?It is TCP_NOPUSH _and_ kqueue. Both are broken. Turns out there''s a fix to get memcached to be fast on OS X, but it will not perform well under very high load (high in terms of concurrency). Set the EVENT_NOKQUEUE environment variable to 1 before running memcached, and undef TCP_NOPUSH in memcached.c and rebuild memcached. -- Posted via http://www.ruby-forum.com/.