I''m trying to decide which path to go down for developing a new web app. Scalability in the long run is a huge requirement, but without sacrificing the development time. The rails community is huge, there''s support everywhere, documentation is fantastic, but I keep hearing things from others about how if we use rails we''re not going to be able to scale. Some have said Rails is great for prototyping, but going to production it''s going to fail. The largest sites out there never use Rails. I believe Twitter tried and failed. I''m not sure what they ended up doing though. I really want to develop using RoR, but I''m hesitant because of the scale factor. Can anyone give me some information concerning this? Thanks! --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
On 8 Feb 2008, at 21:22, -kinetic wrote:> > I''m trying to decide which path to go down for developing a new web > app. Scalability in the long run is a huge requirement, but without > sacrificing the development time. > > The rails community is huge, there''s support everywhere, documentation > is fantastic, but I keep hearing things from others about how if we > use rails we''re not going to be able to scale. > > Some have said Rails is great for prototyping, but going to production > it''s going to fail. The largest sites out there never use Rails. I > believe Twitter tried and failed. I''m not sure what they ended up > doing though.Twitter is still using rails. Fred> > > I really want to develop using RoR, but I''m hesitant because of the > scale factor. > > Can anyone give me some information concerning this? > > Thanks! > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
also large sites like Revolution Health are using rails I think that if you''re smart about your use of the DB, if you use caching, etc you can get pretty far. ...but I also wonder about this --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Do you think the troubles Twitter were realizing was just a well advertise demise while all the others go through just as much scaling problems? In short, would scaling be less of a problem with a PHP framework like Symfony? I think the reason why not many of the larger sites are using RoR is because it''s so new. And twitter was the newest one to go big, and because of the quick development, maybe they staggered into the scaling problems early? I''ve read a ton of articles this afternoon, and they all say how much more processing power Ruby uses, and that scaling is seemingly more complicated. On Feb 8, 2:54 pm, lunaclaire <szager...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> also large sites like Revolution Health are using rails > > I think that if you''re smart about your use of the DB, if you use > caching, etc you can get pretty far. > > ...but I also wonder about this--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
-kinetic wrote:> > I''ve read a ton of articles this afternoon, and they all say how much > more processing power Ruby uses, and that scaling is seemingly more > complicated. >Usually people writing these articles never had any real application to scale themselves. Sure, some applications might have better performance on a single box with Java in a micro-benchmark because they are CPU bound and Java is much faster than Ruby (now) for CPU intensive processing but this has nothing to do with scaling as most people mean it which usually is : "go beyond the point where your application can fit on a single server and get more performance by adding pizza boxes". So saying that an application framework scales more than another is pure bulls**t. Every modern web framework uses a "share-nothing" approach to solve scalability problems so about anyone can scale by adding application servers today. When anybody comes to the point where the only shared component becomes a bottleneck (usually the relational database but might be filesystems, LDAP servers, network, whatever) scaling is a matter of architecture and solving the bottleneck is specific to the application not a problem the framework can come with a solution for. The problem is that the "share-nothing" term is misleading: you always share something and this is what comes back to bite you when your application is under stress. You''ll need one(/several) good specialist(s) to find the bottlenecks and re-architecture your application when the problem arises. For example, I recently had to spend some time optimizing some parts of one Ruby process that was getting slower. After I finished, on the same hardware the application was 10 times faster. Turns out that some database queries weren''t optimal, the database lacked some indexes and PostgreSQL 8.0 was much slower than 8.2. Ruby didn''t have anything to do with it. I''m pretty confident that even if I spent the rest of my life recoding this application from Ruby to raw assembler I wouldn''t get more than a 10% speedup and a ton of bugs. In the software industry believing that software in a shiny box will solve your problems is amateurism: people solve your high-level problems using tools, the tools are only meant to low-level ones like minimizing the time spent coding simple tasks and providing easy access to lots of libraries from which to extend your application as quickly as possible (in fact most of them probably create more problems than they solve :-) ). So if you have problems to solve, I suggest you look for bright people who know how to solve them, not bright software, as bright software is only a marketing myth... Whow, looking at what I just typed it seems I was tired of coding and needed to vent a bit :-) Lionel --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
The fact that Ruby uses more processing power is actually irrelevant to the scalability of a RoR app. Scalability is "how effectively the system architecture can grow in proportion to increasing demand." (The Rails Way by Obie Fernandez). The important thing is your algorithms, not the language they are written in. If your algorithms are well written, then you can hopefully just double the number of servers when you double your users. This isn''t always possible, but that has nothing to do with Ruby and everything to do with the complexity of the business problem at hand. So if you write it in PHP, you might need to buy a new $1000 server for every 100 new users. If you write it in Ruby, then you might need to spend $1200 (or even $2000, but not likely) for the server that supports 100 users. So if you have 1,000,000 users, in PHP you''d need 100 servers X $1000 equals $100,000. In RoR you''d need 100 servers X $1200 equals $120,000. You spend more on hardware, but far less on software development and maintenance. I think part of why people still wonder if Rails can scale is that there are many parts of ActiveRecord that put simplicity before performance. There are 1000s of projects using rails that don''t need high performance so I think this is a great decision. It is also a rails convention (or maybe best practice) to not pre-maturely optimize. That means it is ok to have unscalable algorithms when you don''t have tons of users--start by keeping your code simple. Ruby (and the good practices that the rails community encourages) makes refactoring pretty easy. Cheers, On Feb 8, 3:37 pm, -kinetic <kineti...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> Do you think the troubles Twitter were realizing was just a well > advertise demise while all the others go through just as much scaling > problems? > > In short, would scaling be less of a problem with a PHP framework like > Symfony? > > I think the reason why not many of the larger sites are using RoR is > because it''s so new. And twitter was the newest one to go big, and > because of the quick development, maybe they staggered into the > scaling problems early? > > I''ve read a ton of articles this afternoon, and they all say how much > more processing power Ruby uses, and that scaling is seemingly more > complicated. > > On Feb 8, 2:54 pm, lunaclaire <szager...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > > also large sites like Revolution Health are using rails > > > I think that if you''re smart about your use of the DB, if you use > > caching, etc you can get pretty far. > > > ...but I also wonder about this--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Interesting article on Twitter: http://assetbar.wordpress.com/2008/02/08/twitter-proxy-any-interest/ --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Agreed. This issue of scalability is much more an issue of application design than anything else. If you make good design choices, taking into account things like caching, background processing, etc up front, you''ll be able to scale. But those design decisions will exist in every framework you choose. The main advantage that you''ll get in RoR iss in the code base. Well- written Ruby code is more expressive than most other languages and consequently tends to be easier to maintain. Similarly, the power of the language itself tends to reduce the number of lines of code that are required. I recently read of an application (forget the name... maybe YellowPages.com?) that reduced its code base from 200K+ LOC to around 120K. If you''re like me, less is better... less to break, less to remember, less to archive.... Also, the goal of Matz in designing Ruby was to make programming enjoyable, even fun. Ask any hardcore RoR developer and i think you''ll find general agreement that he''s achieved his goal. Writing in Ruby is fun. One other note, the latest version of Ruby (1.9.x) had performance as its primary target. Rumors are that there has been significant gains across the board. Part of the Rails core is working towards Ruby 1.9 compatability so we should see that fairly soon (maybe at RailsConf?). Soon you may not be shouting back to Scotty for more power... On Feb 8, 7:57 pm, DyingToLearn <phy...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> The fact that Ruby uses more processing power is actually irrelevant > to the scalability of a RoR app. Scalability is "how effectively the > system architecture can grow in proportion to increasing demand." (The > Rails Way by Obie Fernandez). The important thing is your algorithms, > not the language they are written in. > > If your algorithms are well written, then you can hopefully just > double the number of servers when you double your users. This isn''t > always possible, but that has nothing to do with Ruby and everything > to do with the complexity of the business problem at hand. > > So if you write it in PHP, you might need to buy a new $1000 server > for every 100 new users. If you write it in Ruby, then you might need > to spend $1200 (or even $2000, but not likely) for the server that > supports 100 users. So if you have 1,000,000 users, in PHP you''d need > 100 servers X $1000 equals $100,000. In RoR you''d need 100 servers X > $1200 equals $120,000. You spend more on hardware, but far less on > software development and maintenance. > > I think part of why people still wonder if Rails can scale is that > there are many parts of ActiveRecord that put simplicity before > performance. There are 1000s of projects using rails that don''t need > high performance so I think this is a great decision. It is also a > rails convention (or maybe best practice) to not pre-maturely > optimize. That means it is ok to have unscalable algorithms when you > don''t have tons of users--start by keeping your code simple. Ruby (and > the good practices that the rails community encourages) makes > refactoring pretty easy. > > Cheers, > > On Feb 8, 3:37 pm, -kinetic <kineti...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > > Do you think the troubles Twitter were realizing was just a well > > advertise demise while all the others go through just as much scaling > > problems? > > > In short, would scaling be less of a problem with a PHP framework like > > Symfony? > > > I think the reason why not many of the larger sites are using RoR is > > because it''s so new. And twitter was the newest one to go big, and > > because of the quick development, maybe they staggered into the > > scaling problems early? > > > I''ve read a ton of articles this afternoon, and they all say how much > > more processing power Ruby uses, and that scaling is seemingly more > > complicated. > > > On Feb 8, 2:54 pm, lunaclaire <szager...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > > > also large sites like Revolution Health are using rails > > > > I think that if you''re smart about your use of the DB, if you use > > > caching, etc you can get pretty far. > > > > ...but I also wonder about this--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Wow, great information everyone. My app needs to scale to hundreds of thousands of users per day to a million or so. The numbers are ideal, but as all engineers say, you must prepare for the worst, and you must prepare for the best. There''s a huge need to develop the application with scaling in mind every step of the way. Ruby on Rails seems to skim over that initially, and just have you develop quickly and efficiently. Would you recommend using RoR for something like this? The trade off might not be worth it? Thanks again! On Feb 9, 11:51 am, AndyV <a...-HmMyXyqgL2CVc3sceRu5cw@public.gmane.org> wrote:> Agreed. This issue of scalability is much more an issue of > application design than anything else. If you make good design > choices, taking into account things like caching, background > processing, etc up front, you''ll be able to scale. But those design > decisions will exist in every framework you choose. > > The main advantage that you''ll get in RoR iss in the code base. Well- > written Ruby code is more expressive than most other languages and > consequently tends to be easier to maintain. Similarly, the power of > the language itself tends to reduce the number of lines of code that > are required. I recently read of an application (forget the name... > maybe YellowPages.com?) that reduced its code base from 200K+ LOC to > around 120K. If you''re like me, less is better... less to break, less > to remember, less to archive.... Also, the goal of Matz in designing > Ruby was to make programming enjoyable, even fun. Ask any hardcore > RoR developer and i think you''ll find general agreement that he''s > achieved his goal. Writing in Ruby is fun. > > One other note, the latest version of Ruby (1.9.x) had performance as > its primary target. Rumors are that there has been significant gains > across the board. Part of the Rails core is working towards Ruby 1.9 > compatability so we should see that fairly soon (maybe at > RailsConf?). Soon you may not be shouting back to Scotty for more > power... > > On Feb 8, 7:57 pm, DyingToLearn <phy...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > > The fact that Ruby uses more processing power is actually irrelevant > > to the scalability of a RoR app. Scalability is "how effectively the > > system architecture can grow in proportion to increasing demand." (The > > Rails Way by Obie Fernandez). The important thing is your algorithms, > > not the language they are written in. > > > If your algorithms are well written, then you can hopefully just > > double the number of servers when you double your users. This isn''t > > always possible, but that has nothing to do with Ruby and everything > > to do with the complexity of the business problem at hand. > > > So if you write it in PHP, you might need to buy a new $1000 server > > for every 100 new users. If you write it in Ruby, then you might need > > to spend $1200 (or even $2000, but not likely) for the server that > > supports 100 users. So if you have 1,000,000 users, in PHP you''d need > > 100 servers X $1000 equals $100,000. In RoR you''d need 100 servers X > > $1200 equals $120,000. You spend more on hardware, but far less on > > software development and maintenance. > > > I think part of why people still wonder if Rails can scale is that > > there are many parts of ActiveRecord that put simplicity before > > performance. There are 1000s of projects using rails that don''t need > > high performance so I think this is a great decision. It is also a > > rails convention (or maybe best practice) to not pre-maturely > > optimize. That means it is ok to have unscalable algorithms when you > > don''t have tons of users--start by keeping your code simple. Ruby (and > > the good practices that the rails community encourages) makes > > refactoring pretty easy. > > > Cheers, > > > On Feb 8, 3:37 pm, -kinetic <kineti...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > > > Do you think the troubles Twitter were realizing was just a well > > > advertise demise while all the others go through just as much scaling > > > problems? > > > > In short, would scaling be less of a problem with a PHP framework like > > > Symfony? > > > > I think the reason why not many of the larger sites are using RoR is > > > because it''s so new. And twitter was the newest one to go big, and > > > because of the quick development, maybe they staggered into the > > > scaling problems early? > > > > I''ve read a ton of articles this afternoon, and they all say how much > > > more processing power Ruby uses, and that scaling is seemingly more > > > complicated. > > > > On Feb 8, 2:54 pm, lunaclaire <szager...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > > > > also large sites like Revolution Health are using rails > > > > > I think that if you''re smart about your use of the DB, if you use > > > > caching, etc you can get pretty far. > > > > > ...but I also wonder about this--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
I could write a lot, to answer this question, but Chris @TheKeyboard has phrased it very simple and beutiful: http://www.littlehart.net/atthekeyboard/2007/01/02/just-build-it-damnit/ - Carsten --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
In my view all this talk of scalability is premature. People plan for a million users from start and that is a premature optimization. It''s much easier to do the site in Rails and when bottlenecks become a pain scale like this 1. Optimize using fragment/page cahing, server static files from apache, profile and optimize the code(check for N+1 bugs and stuff) 2. Optimize the SQLs. Use ''explain'' on your mysql queries and see how much you can optimize. 3. Use more mongrels, write mongrel handlers for slow requests(or requests which depend on 3rd party servers like fetching rss feeds et al) 4. Use memcache servers to cache the queries for data that queried a lot. 5. Add more application servers 6. Write cpu intensive functions in C and add a ruby wrapper for these 7. Cluster you database servers I run a few sites which serve upto 100-150k requests per day much of the above(1 app server, 2 mongrels, 1 db server and step 1). Scaling upto 1 million with all the above steps should not be a big problem) Piyush On Feb 10, 2008 6:36 PM, Carsten Gehling <carsten.gehling-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> > I could write a lot, to answer this question, but Chris @TheKeyboard > has phrased it very simple and beutiful: > > http://www.littlehart.net/atthekeyboard/2007/01/02/just-build-it-damnit/ > > - Carsten > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
> My app needs to scale to hundreds of thousands of users per day to a > million or so.It need right now? Or you think it will need? There is a big difference here. Some wisdom from the 37 signals: http://gettingreal.37signals.com/ch04_Scale_Later.php Regards, Rimantas -- http://rimantas.com/ --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
> My app needs to scale to hundreds of thousands of users per day to a > million or so. The numbers are ideal, but as all engineers say, you > must prepare for the worst, and you must prepare for the best.That''s not really a useful question. For example... my site has 10 million users (it doesn''t but let''s pretend :) but they only visit once a month. Or, perhaps I have 10,000 and they visit my site 5-6 times a minute each. You need to look at how many requests you are going to get over a day, or an hour, what your peaks are, how much *traffic* you have to be able to sustain during those times. That all said... I agree with everyone else... just build it :)> > There''s a huge need to develop the application with scaling in mind > every step of the way. Ruby on Rails seems to skim over that > initially, and just have you develop quickly and efficiently. Would > you recommend using RoR for something like this? The trade off might > not be worth it? > > Thanks again! > > On Feb 9, 11:51 am, AndyV <a...-HmMyXyqgL2CVc3sceRu5cw@public.gmane.org> wrote: >> Agreed. This issue of scalability is much more an issue of >> application design than anything else. If you make good design >> choices, taking into account things like caching, background >> processing, etc up front, you''ll be able to scale. But those design >> decisions will exist in every framework you choose. >> >> The main advantage that you''ll get in RoR iss in the code base. Well- >> written Ruby code is more expressive than most other languages and >> consequently tends to be easier to maintain. Similarly, the power of >> the language itself tends to reduce the number of lines of code that >> are required. I recently read of an application (forget the name... >> maybe YellowPages.com?) that reduced its code base from 200K+ LOC to >> around 120K. If you''re like me, less is better... less to break, less >> to remember, less to archive.... Also, the goal of Matz in designing >> Ruby was to make programming enjoyable, even fun. Ask any hardcore >> RoR developer and i think you''ll find general agreement that he''s >> achieved his goal. Writing in Ruby is fun. >> >> One other note, the latest version of Ruby (1.9.x) had performance as >> its primary target. Rumors are that there has been significant gains >> across the board. Part of the Rails core is working towards Ruby 1.9 >> compatability so we should see that fairly soon (maybe at >> RailsConf?). Soon you may not be shouting back to Scotty for more >> power... >> >> On Feb 8, 7:57 pm, DyingToLearn <phy...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: >> >>> The fact that Ruby uses more processing power is actually irrelevant >>> to the scalability of a RoR app. Scalability is "how effectively the >>> system architecture can grow in proportion to increasing demand." (The >>> Rails Way by Obie Fernandez). The important thing is your algorithms, >>> not the language they are written in. >> >>> If your algorithms are well written, then you can hopefully just >>> double the number of servers when you double your users. This isn''t >>> always possible, but that has nothing to do with Ruby and everything >>> to do with the complexity of the business problem at hand. >> >>> So if you write it in PHP, you might need to buy a new $1000 server >>> for every 100 new users. If you write it in Ruby, then you might need >>> to spend $1200 (or even $2000, but not likely) for the server that >>> supports 100 users. So if you have 1,000,000 users, in PHP you''d need >>> 100 servers X $1000 equals $100,000. In RoR you''d need 100 servers X >>> $1200 equals $120,000. You spend more on hardware, but far less on >>> software development and maintenance. >> >>> I think part of why people still wonder if Rails can scale is that >>> there are many parts of ActiveRecord that put simplicity before >>> performance. There are 1000s of projects using rails that don''t need >>> high performance so I think this is a great decision. It is also a >>> rails convention (or maybe best practice) to not pre-maturely >>> optimize. That means it is ok to have unscalable algorithms when you >>> don''t have tons of users--start by keeping your code simple. Ruby (and >>> the good practices that the rails community encourages) makes >>> refactoring pretty easy. >> >>> Cheers, >> >>> On Feb 8, 3:37 pm, -kinetic <kineti...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: >> >>>> Do you think the troubles Twitter were realizing was just a well >>>> advertise demise while all the others go through just as much scaling > > > problems? >> >>>> In short, would scaling be less of a problem with a PHP framework like >>>> Symfony? >> >>>> I think the reason why not many of the larger sites are using RoR is >>>> because it''s so new. And twitter was the newest one to go big, and >>>> because of the quick development, maybe they staggered into the >>>> scaling problems early? >> >>>> I''ve read a ton of articles this afternoon, and they all say how much >>>> more processing power Ruby uses, and that scaling is seemingly more >>>> complicated. >> >>>> On Feb 8, 2:54 pm, lunaclaire <szager...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: >> >>>>> also large sites like Revolution Health are using rails >> >>>>> I think that if you''re smart about your use of the DB, if you use >>>>> caching, etc you can get pretty far. >> >>>>> ...but I also wonder about this > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
> I''ve read a ton of articles this afternoon, and they all say how much > more processing power Ruby uses, and that scaling is seemingly more > complicated.RoR is a LAMP system. The P stands for an interPreted language, and the M stands for MySQL. Write developer-friendly code with the P, and put the M on another server. If the P gets slow, put more servers under it. This system allows you to optimize developer time, while keeping the potentially slowest part of the system - MySQL - on a dedicated server. -- Phlip --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
On Feb 10, 2008, at 5:22 PM, Phlip wrote:> >> I''ve read a ton of articles this afternoon, and they all say how much >> more processing power Ruby uses, and that scaling is seemingly more >> complicated. > > RoR is a LAMP system. The P stands for an interPreted language, and > the M > stands for MySQL. >Um, LAMP actually stands for something more along the lines of Linux, Apache, MySQL, and Perl/PHP. RoR can run on Windows, Linux, or Mac and can use any of quite a few databases. Some (maybe many) choose MySQL. I don''t. I use PostgreSQL. Peace, Phillip --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
You guys interested in scaling with RoR might be interested in http://blog.caboo.se/articles/2007/7/29/scale-rails-from-one-box-to-three-four-and-five Basically covers how to scale and the steps to take. -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
On Sun, Feb 10, 2008 at 02:06:18PM +0100, Carsten Gehling wrote:> I could write a lot, to answer this question, but Chris @TheKeyboard > has phrased it very simple and beutiful: > > http://www.littlehart.net/atthekeyboard/2007/01/02/just-build-it-damnit/That post is good advice, in general, but it starts from a premise that is never explicitly mentioned -- that the site you are building is new. While it''s refreshing when clients ask for something brand-spanking-new, the work is more likely to be "do exactly what my site already does, plus this set of additional features." In that case, the traffic is already coming to the site. If you''re lucky, the traffic is light enough that you can still ignore scaling issues while you''re developing the new Rails app to replace the existing whatever app. On the other hand, the client may be coming to you to reimplement their site because it is getting slow and they want you to create the new one so you can be responsible for scaling it. This may not be the most common Rails project, but such work is definitely out there. It means that, yes, you need to think about scaling from the start. You don''t have the luxury of a gradual user uptake to see where the bottlenecks are, because the moment your new site goes live you have a full user base (see below for a way around that). You need to address problems before they happen, because downtime and slowness are not just growing pains but regressions that the client will be justifiably screaming about. I''d be interested in some thoughts toward handling the situation starting from that premise, where "just build it, damnit (sic)" doesn''t apply. As an example, one can play tricks with a frontend server to put some segment of the user base on the new system while leaving the majority on the old to simulate a gradual user uptake. Of course, that may not be feasible depending on the particular site''s functionality. Consider Twitter, for example, where the users'' interaction with one another is central to the site.> - Carsten--Greg --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
On 2/10/08, Gregory Seidman <gsslist+ror-dNXPQ6k9rNiG6BJUYyje5axOck334EZe@public.gmane.org> wrote:> On Sun, Feb 10, 2008 at 02:06:18PM +0100, Carsten Gehling wrote: > > I could write a lot, to answer this question, but Chris @TheKeyboard > > has phrased it very simple and beutiful: > > > > http://www.littlehart.net/atthekeyboard/2007/01/02/just-build-it-damnit/ > > That post is good advice, in general, but it starts from a premise that is > never explicitly mentioned -- that the site you are building is new. > While it''s refreshing when clients ask for something brand-spanking-new, > the work is more likely to be "do exactly what my site already does, plus > this set of additional features."No difference: just build it, damnit; scale to meet traffic demands; launch. You''re going to have more trouble with integration and compatibility, anyway. Iteratively, jeremy --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
On Feb 10, 2008 5:37 AM, Piyush Ranjan <piyush.pr-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> In my view all this talk of scalability is premature. People plan for a > million users from start and that is a premature optimization. It''s much > easier to do the site in Rails and when bottlenecks become a pain scale like > thisBingo. People who worry too much about scaling off the bat never get their projects launched. Make something worthwhile first, scale it when necessary. -------------------- seth at subimage interactive ----- http://sublog.subimage.com ----- Cashboard - Estimates, invoices, and time tracking software - for free! http://www.getcashboard.com ----- Substruct - Open source RoR e-commerce software. http://code.google.com/p/substruct/ --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Gregory Seidman wrote:> [...] > In that case, the traffic is already coming to the site. If you''re lucky, > the traffic is light enough that you can still ignore scaling issues while > you''re developing the new Rails app to replace the existing whatever app. > On the other hand, the client may be coming to you to reimplement their > site because it is getting slow and they want you to create the new one so > you can be responsible for scaling it. > > This may not be the most common Rails project, but such work is definitely > out there. It means that, yes, you need to think about scaling from the > start. You don''t have the luxury of a gradual user uptake to see where the > bottlenecks are, because the moment your new site goes live you have a full > user base (see below for a way around that). You need to address problems > before they happen, because downtime and slowness are not just growing > pains but regressions that the client will be justifiably screaming about. > > I''d be interested in some thoughts toward handling the situation starting > from that premise, where "just build it, damnit (sic)" doesn''t apply. >If it is getting slow you can check out the whole architecture to find out where the bottlenecks are and rebuild it accordingly. As I already said, when you have performance problems you need people knowledgeable enough to find out why, the framework itself is only a tool that might need some tuning (or not). Lionel --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Advice from smart and well-intentioned people notwithstanding, I firmly believe you should think about scale early and often. Google thought about scale early - their very name is scale. My deep concern is that those who work on improving the Ruby on Rails framework may have this "worry about scale later" mentality. I''m concerned that many of the programming paradigms in Rails lead to unscalable practices like "transparent" joins that novice programmers won''t know to avoid (e.g. image.user.comments.categories - in views no less). It would be very helpful if RoR actually solved some of the tough scale issues by, for example: - Including support for Amazon''s Simple DB service - Intelligently unwrapping joins in AR layer, or allowing/encouraging developers to do that - Providing a mysql proxy (better than mysql-proxy) which is scriptable in Ruby, that provides for db load-balancing, failover, multi-master, master-per-model, index-modulo multi-master, etc. acts_as_readonlyable is a great start, but you can only scale so far without multi-master. - Support for lazy insertion, insert ignore, insert later, insert many, batch insertion, etc. Perhaps "acts_as_batched" ? - Providing better examples and support for logging using syslog over udp (there actually may be plenty of support for this already, I just haven''t had time to dig into it yet). Educate/encourage through examples how to write certain log-type info to logs instead of writing to db. m On Feb 11, 2008 2:30 AM, Lionel Bouton <lionel-subscription-WTamNBQcZIx7tPAFqOLdPg@public.gmane.org> wrote:> > Gregory Seidman wrote: > > [...] > > In that case, the traffic is already coming to the site. If you''re > lucky, > > the traffic is light enough that you can still ignore scaling issues > while > > you''re developing the new Rails app to replace the existing whatever > app. > > On the other hand, the client may be coming to you to reimplement their > > site because it is getting slow and they want you to create the new one > so > > you can be responsible for scaling it. > > > > This may not be the most common Rails project, but such work is > definitely > > out there. It means that, yes, you need to think about scaling from the > > start. You don''t have the luxury of a gradual user uptake to see where > the > > bottlenecks are, because the moment your new site goes live you have a > full > > user base (see below for a way around that). You need to address > problems > > before they happen, because downtime and slowness are not just growing > > pains but regressions that the client will be justifiably screaming > about. > > > > I''d be interested in some thoughts toward handling the situation > starting > > from that premise, where "just build it, damnit (sic)" doesn''t apply. > > > > If it is getting slow you can check out the whole architecture to find > out where the bottlenecks are and rebuild it accordingly. As I already > said, when you have performance problems you need people knowledgeable > enough to find out why, the framework itself is only a tool that might > need some tuning (or not). > > Lionel > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
> - Providing a mysql proxy (better than mysql-proxy) which isYou may find this useful... http://blog.rapleaf.com/dev/?p=5 --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
I guess the main difference between mysql-proxy and things like this and acts_as_readonlyable is that mysql-proxy acts at the mysql layer. You can connect to mysql-proxy from the command line just as if it were a mysql server. It allows you to say things like "if it''s an INSERT or UPDATE and the table is users, then use this db" - thus it hides those details from the RoR code, which may be kept clean, simple, and flexible if/when you need to add a slave, move a master, etc. My only real problem with mysql-proxy is lua - what''s up with that? ;^) Just my $0.02, m> > - Providing a mysql proxy (better than mysql-proxy) which is > > You may find this useful... > > http://blog.rapleaf.com/dev/?p=5 > > > > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Marc Byrd wrote:> Advice from smart and well-intentioned people notwithstanding, I > firmly believe you should think about scale early and often. Google > thought about scale early - their very name is scale. > > My deep concern is that those who work on improving the Ruby on Rails > framework may have this "worry about scale later" mentality. I''m > concerned that many of the programming paradigms in Rails lead to > unscalable practices like "transparent" joins that novice programmers > won''t know to avoid (e.g. image.user.comments.categories - in views no > less).I''m quite sure Google architects knew these pitfalls and worked out by themselves how to avoid them... As they simply *coded* their own framework (and probably several times along their path). If you really believe you can throw a "scaling" framework into the hands of "novice programmers" and expect them to code something that scales anything like the Google underlying architecture, just wait a minute, I''ll fetch a good seat and popcorn to watch the show. I''m sure I''ll find it either quite instructive or good laughing material :-) Good luck, Lionel --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
My experience taught me to think about it often, but code for it only when needed. Just build it sounds cool to some and stupid to others, but it does works. Until you have real people smashing around in your carefully crafted application, you will never know where and more importantly when to address scaling. You can assume it is in location X, Y, or Z, but who knew users of the application would be clicking on A, B, C like hamsters on a treadmill. When I started with Rails, it was only a few weeks after David and 37Signals released it to the world. At the time my application ran under Apache, FastCGI and MySQL. That was the entire technology chain using 1 web server and 1 database server. Today, the very same application runs on 5 servers, using MySQL, Nginx, Mongrel, Amazon S3, a dedicated email platform and numerous back processing scripts on dedicated servers. Some of my scaling challenges: - Concurrent users blocking others with long running actions, which are now offloaded to the background. - Storing large objects in the database, are instead sent to Amazon S3 or the filesystem and only store metadata in the database - Unreliable stability of FastCGI under heavy loads, are now running on proxy mongrels - Slow loading of images and javascripts, are now served statically outside application using a couple lines of Nginx configuration - Long render times for large pages were retooled to avoid slow methods like url_for and more intelligently paginated - Sluggish performance when using MySQL Fulltext searches, now use an external search tool like Sphinx After several years of real world use by customers, I now have enough experience to see some issues down the road. Rails can and will scale if you think of it as only one part of the solution. If you think every tool needed is in the framework itself, your application will not scale. -- Lon --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---