Hey folks- I''ve recently been running up against the memory limits on TextDrive for a Ruby on Rails app under development. It''s running on FCGI/lighttpd with 2 listeners. I can restart it and within a few clicks on random pages I can get it to fail for memory allocation problems. While I haven''t written off the idea of a memory leak, I have been unable to reproduce the problem on my development machine. I can run mongrel or webrick and don''t see memory usage go up after a LOT of browsing around and hitting it through the webservices API. I think my app just gets me close to the 48MB limit. So, my questions: 1. Is the TxD memory limit too low? My app isn''t enormous, but it is bigger than a simple blog or other basic Rails app like Strongspace or something. I''ve seen memory usage posts from other folks that put their app in the 40MB-70MB range. If this is the case, and TxD claims to be the premier RoR host, why is their limit so low? 2. What techniques can I use to reduce memory usage? 3. Would another server technology be easier on memory? 4. I really like TxD, but they claim they can''t increase memory allotment for me (even just to TEST the damn thing for a day). So, are there other hosts that would be more amenable to this app? Business hosting is a possibility (at TxD) and it will be necessary once the app is released, but before I go through the trouble of moving the app, I wanted to test some stuff out. Jake -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk -~----------~----~----~----~------~----~------~--~---
Jake Janovetz wrote:> Hey folks- > > I''ve recently been running up against the memory limits on TextDrive for > a Ruby on Rails app under development.I thought I''d mention -- this has recently been a problem because my app just got bigger. But when I migrated to Edge Rails, it got even worse. I wonder if there will be even more such problems when 1.2 is released and people''s code just gets bigger. Jake -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk -~----------~----~----~----~------~----~------~--~---
On 10/30/06, Jake Janovetz <rails-mailing-list-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> wrote:> > Hey folks- > > I''ve recently been running up against the memory limits on TextDrive for > a Ruby on Rails app under development. > > It''s running on FCGI/lighttpd with 2 listeners. I can restart it and > within a few clicks on random pages I can get it to fail for memory > allocation problems.Why do you need two listeners? Most apps would not. -- _Deirdre http://deirdre.net/ --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk -~----------~----~----~----~------~----~------~--~---
I am also running up against the memory limits of Text Drive but only since I upgraded my app to use edge rails. Are you running on edge? If so, I would recommend running on 1.1.6 and see how that goes. I never had a problem with memory, except for very occasionally on TextDrive. Since I upgraded to edge, I am seeing memory spike up to 3.5 MBs on a single request that only grabs 10 records out of 139 or them (something about the total impacts the size). I am in the process of downgrading my app back to 1.1.6 and will post my results to the list if it fixes my memory issues. In theory, this could be something that I added in this release but from my initial analysis it does not appear to be. Also, a lot of people say you don''t need more than one fcgi and while in theory I would agree, when you are on a server like TextDrive that will *kill* your fcgis if they hit system resources the second fcgi is a good idea so your site is not immediately dead. You have two chances. Tom On 10/30/06, Deirdre Saoirse Moen <dsmoen-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> > On 10/30/06, Jake Janovetz <rails-mailing-list-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> wrote: > > > > Hey folks- > > > > I''ve recently been running up against the memory limits on TextDrive for > > a Ruby on Rails app under development. > > > > It''s running on FCGI/lighttpd with 2 listeners. I can restart it and > > within a few clicks on random pages I can get it to fail for memory > > allocation problems. > > Why do you need two listeners? Most apps would not. > > -- > _Deirdre http://deirdre.net/ > > > >-- Tom Davies http://atomgiant.com http://gifthat.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Deirdre Saoirse Moen wrote:> On 10/30/06, Jake Janovetz <rails-mailing-list-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> wrote: >> >> Hey folks- >> >> I''ve recently been running up against the memory limits on TextDrive for >> a Ruby on Rails app under development. >> >> It''s running on FCGI/lighttpd with 2 listeners. I can restart it and >> within a few clicks on random pages I can get it to fail for memory >> allocation problems. > > Why do you need two listeners? Most apps would not. >That point is irrelevant actually. Each process is limited to 48 MB on TxD. I don''t know what their limit it on the sum. Jake -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Tom Davies wrote:> I am also running up against the memory limits of Text Drive but only > since I upgraded my app to use edge rails. Are you running on edge? > If so, I would recommend running on 1.1.6 and see how that goes. I > never had a problem with memory, except for very occasionally on > TextDrive. > > Since I upgraded to edge, I am seeing memory spike up to 3.5 MBs on a > single request that only grabs 10 records out of 139 or them > (something about the total impacts the size). I am in the process of > downgrading my app back to 1.1.6 and will post my results to the list > if it fixes my memory issues. In theory, this could be something that > I added in this release but from my initial analysis it does not > appear to be. > > Also, a lot of people say you don''t need more than one fcgi and while > in theory I would agree, when you are on a server like TextDrive that > will *kill* your fcgis if they hit system resources the second fcgi is > a good idea so your site is not immediately dead. You have two > chances. > > TomTom-- Yes, the memory limit came into play when I upgraded to edge. I definitely think edge has higher memory requirements. Unfortunately, it''s something I needed to do, so couldn''t be avoided. We moved to a container on TxD and that solved the memory problems. It''s also more expensive, but reasonable for this app. The increased speed is a huge benefit. I think I just recognize the limitations of shared hosting. The move to edge just accelerated the need to switch off of it, but that was inevitable. Perhaps TxD will increase their memory limits when 1.2 releases. Jake -- Posted via http://www.ruby-forum.com/. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Yeah, from my limited testing, I witnessed edge rails requiring in the neighborhood of 6-8MBs more per fcgi. That is a lot to jump from one point release to another so I suspect it was an accident. I posted another email to the list to see if anyone has any feedback on it. I would hope they can bring the memory requirements down closer to the 1.1.6 level. Tom On 11/7/06, Jake Janovetz <rails-mailing-list-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> wrote:> > Tom Davies wrote: > > I am also running up against the memory limits of Text Drive but only > > since I upgraded my app to use edge rails. Are you running on edge? > > If so, I would recommend running on 1.1.6 and see how that goes. I > > never had a problem with memory, except for very occasionally on > > TextDrive. > > > > Since I upgraded to edge, I am seeing memory spike up to 3.5 MBs on a > > single request that only grabs 10 records out of 139 or them > > (something about the total impacts the size). I am in the process of > > downgrading my app back to 1.1.6 and will post my results to the list > > if it fixes my memory issues. In theory, this could be something that > > I added in this release but from my initial analysis it does not > > appear to be. > > > > Also, a lot of people say you don''t need more than one fcgi and while > > in theory I would agree, when you are on a server like TextDrive that > > will *kill* your fcgis if they hit system resources the second fcgi is > > a good idea so your site is not immediately dead. You have two > > chances. > > > > Tom > > Tom-- > > Yes, the memory limit came into play when I upgraded to edge. I > definitely think edge has higher memory requirements. Unfortunately, > it''s something I needed to do, so couldn''t be avoided. > > We moved to a container on TxD and that solved the memory problems. > It''s also more expensive, but reasonable for this app. The increased > speed is a huge benefit. > > I think I just recognize the limitations of shared hosting. The move to > edge just accelerated the need to switch off of it, but that was > inevitable. Perhaps TxD will increase their memory limits when 1.2 > releases. > > Jake > > -- > Posted via http://www.ruby-forum.com/. > > > >-- Tom Davies http://atomgiant.com http://gifthat.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
I have a shared TD account. How do I see my memory consumption? Is it the Size column listed in the output from the top command? last pid: 87116; load averages: 2.60, 3.94, 5.15 up 83+06:27:38 22:00:11 17 processes: 1 running, 16 sleeping CPU states: 11.8% user, 4.2% nice, 17.5% system, 0.4% interrupt, 66.1%idle Mem: 2976M Active, 391M Inact, 410M Wired, 201M Cache, 112M Buf, 13M Free Swap: 12G Total, 5239M Used, 7049M Free, 42% Inuse, 320K Out PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND 28834 bikle10 4 10 42452K 8K select 2 0:06 0.00% 0.00%ruby 28832 bikle10 4 10 4144K 776K select 2 0:03 0.00% 0.00%lighttpd -Dan On 11/7/06, Tom Davies <atomgiant-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> > Yeah, from my limited testing, I witnessed edge rails requiring in the > neighborhood of 6-8MBs more per fcgi. That is a lot to jump from one > point release to another so I suspect it was an accident. > > I posted another email to the list to see if anyone has any feedback > on it. I would hope they can bring the memory requirements down > closer to the 1.1.6 level. > > Tom > > On 11/7/06, Jake Janovetz <rails-mailing-list-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> wrote: > > > > Tom Davies wrote: > > > I am also running up against the memory limits of Text Drive but only > > > since I upgraded my app to use edge rails. Are you running on edge? > > > If so, I would recommend running on 1.1.6 and see how that goes. I > > > never had a problem with memory, except for very occasionally on > > > TextDrive. > > > > > > Since I upgraded to edge, I am seeing memory spike up to 3.5 MBs on a > > > single request that only grabs 10 records out of 139 or them > > > (something about the total impacts the size). I am in the process of > > > downgrading my app back to 1.1.6 and will post my results to the list > > > if it fixes my memory issues. In theory, this could be something that > > > I added in this release but from my initial analysis it does not > > > appear to be. > > > > > > Also, a lot of people say you don''t need more than one fcgi and while > > > in theory I would agree, when you are on a server like TextDrive that > > > will *kill* your fcgis if they hit system resources the second fcgi is > > > a good idea so your site is not immediately dead. You have two > > > chances. > > > > > > Tom > > > > Tom-- > > > > Yes, the memory limit came into play when I upgraded to edge. I > > definitely think edge has higher memory requirements. Unfortunately, > > it''s something I needed to do, so couldn''t be avoided. > > > > We moved to a container on TxD and that solved the memory problems. > > It''s also more expensive, but reasonable for this app. The increased > > speed is a huge benefit. > > > > I think I just recognize the limitations of shared hosting. The move to > > edge just accelerated the need to switch off of it, but that was > > inevitable. Perhaps TxD will increase their memory limits when 1.2 > > releases. > > > > Jake > > > > -- > > Posted via http://www.ruby-forum.com/. > > > > > > > > > > -- > Tom Davies > > http://atomgiant.com > http://gifthat.com > > > >-- bikle-FMj8menRjGsAvxtiuMwx3w@public.gmane.org http://bikle.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Hi Dan, Yeah top should work. You can also do "ps auxww | grep dispatch". The size is the virtual memory size. The RES is the resident memory. Virtual memory is limited to 80MB on TextDrive. Resident memory is limited to 48MB. In my case, I was hitting the resident memory limits. Tom On 11/7/06, Dan Bikle <dan.bikle-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> I have a shared TD account. > > How do I see my memory consumption? > > Is it the Size column listed in the output from the top command? > > last pid: 87116; load averages: 2.60, 3.94, 5.15 > > up 83+06:27:38 22:00:11 > 17 processes: 1 running, 16 sleeping > CPU states: 11.8% user, 4.2% nice, 17.5% system, 0.4% interrupt, 66.1% > idle > Mem: 2976M Active, 391M Inact, 410M Wired, 201M Cache, 112M Buf, 13M Free > Swap: 12G Total, 5239M Used, 7049M Free, 42% Inuse, 320K Out > > PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU > COMMAND > 28834 bikle10 4 10 42452K 8K select 2 0:06 0.00% 0.00% > ruby > 28832 bikle10 4 10 4144K 776K select 2 0:03 0.00% 0.00% > lighttpd > > -Dan > > > On 11/7/06, Tom Davies <atomgiant-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > > > Yeah, from my limited testing, I witnessed edge rails requiring in the > > neighborhood of 6-8MBs more per fcgi. That is a lot to jump from one > > point release to another so I suspect it was an accident. > > > > I posted another email to the list to see if anyone has any feedback > > on it. I would hope they can bring the memory requirements down > > closer to the 1.1.6 level. > > > > Tom > > > > On 11/7/06, Jake Janovetz > <rails-mailing-list-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> wrote: > > > > > > Tom Davies wrote: > > > > I am also running up against the memory limits of Text Drive but only > > > > since I upgraded my app to use edge rails. Are you running on edge? > > > > If so, I would recommend running on 1.1.6 and see how that goes. I > > > > never had a problem with memory, except for very occasionally on > > > > TextDrive. > > > > > > > > Since I upgraded to edge, I am seeing memory spike up to 3.5 MBs on a > > > > single request that only grabs 10 records out of 139 or them > > > > (something about the total impacts the size). I am in the process of > > > > downgrading my app back to 1.1.6 and will post my results to the list > > > > if it fixes my memory issues. In theory, this could be something that > > > > I added in this release but from my initial analysis it does not > > > > appear to be. > > > > > > > > Also, a lot of people say you don''t need more than one fcgi and while > > > > in theory I would agree, when you are on a server like TextDrive that > > > > will *kill* your fcgis if they hit system resources the second fcgi is > > > > a good idea so your site is not immediately dead. You have two > > > > chances. > > > > > > > > Tom > > > > > > Tom-- > > > > > > Yes, the memory limit came into play when I upgraded to edge. I > > > definitely think edge has higher memory requirements. Unfortunately, > > > it''s something I needed to do, so couldn''t be avoided. > > > > > > We moved to a container on TxD and that solved the memory problems. > > > It''s also more expensive, but reasonable for this app. The increased > > > speed is a huge benefit. > > > > > > I think I just recognize the limitations of shared hosting. The move to > > > edge just accelerated the need to switch off of it, but that was > > > inevitable. Perhaps TxD will increase their memory limits when 1.2 > > > releases. > > > > > > Jake > > > > > > -- > > > Posted via http://www.ruby-forum.com/. > > > > > > > > > > > > > > > > -- > > Tom Davies > > > > http://atomgiant.com > > http://gifthat.com > > > > > > >-- Tom Davies http://atomgiant.com http://gifthat.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
Nope, TextDrive no longer allows you to use ''top''. I''ve run into the same problem with them - hitting the 48M limit on my production app. After trying in vain to get the memory consumption down and seeing my site bouncing like a yoyo, I''m getting a VPS at Slicehost. I''ll setup Mongrel and be much happier, I think. On 11/8/06, Tom Davies <atomgiant-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> > Hi Dan, > > Yeah top should work. You can also do "ps auxww | grep dispatch". > > The size is the virtual memory size. The RES is the resident memory. > Virtual memory is limited to 80MB on TextDrive. Resident memory is > limited to 48MB. In my case, I was hitting the resident memory > limits. > > Tom > > On 11/7/06, Dan Bikle <dan.bikle-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > I have a shared TD account. > > > > How do I see my memory consumption? > > > > Is it the Size column listed in the output from the top command? > > > > last pid: 87116; load averages: 2.60, 3.94, 5.15 > > > > up 83+06:27:38 22:00:11 > > 17 processes: 1 running, 16 sleeping > > CPU states: 11.8% user, 4.2% nice, 17.5% system, 0.4% interrupt, 66.1% > > idle > > Mem: 2976M Active, 391M Inact, 410M Wired, 201M Cache, 112M Buf, 13M Free > > Swap: 12G Total, 5239M Used, 7049M Free, 42% Inuse, 320K Out > > > > PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU > > COMMAND > > 28834 bikle10 4 10 42452K 8K select 2 0:06 0.00% 0.00% > > ruby > > 28832 bikle10 4 10 4144K 776K select 2 0:03 0.00% 0.00% > > lighttpd > > > > -Dan > > > > > > On 11/7/06, Tom Davies <atomgiant-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > > > > > > Yeah, from my limited testing, I witnessed edge rails requiring in the > > > neighborhood of 6-8MBs more per fcgi. That is a lot to jump from one > > > point release to another so I suspect it was an accident. > > > > > > I posted another email to the list to see if anyone has any feedback > > > on it. I would hope they can bring the memory requirements down > > > closer to the 1.1.6 level. > > > > > > Tom > > > > > > On 11/7/06, Jake Janovetz > > <rails-mailing-list-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> wrote: > > > > > > > > Tom Davies wrote: > > > > > I am also running up against the memory limits of Text Drive but only > > > > > since I upgraded my app to use edge rails. Are you running on edge? > > > > > If so, I would recommend running on 1.1.6 and see how that goes. I > > > > > never had a problem with memory, except for very occasionally on > > > > > TextDrive. > > > > > > > > > > Since I upgraded to edge, I am seeing memory spike up to 3.5 MBs on a > > > > > single request that only grabs 10 records out of 139 or them > > > > > (something about the total impacts the size). I am in the process of > > > > > downgrading my app back to 1.1.6 and will post my results to the list > > > > > if it fixes my memory issues. In theory, this could be something that > > > > > I added in this release but from my initial analysis it does not > > > > > appear to be. > > > > > > > > > > Also, a lot of people say you don''t need more than one fcgi and while > > > > > in theory I would agree, when you are on a server like TextDrive that > > > > > will *kill* your fcgis if they hit system resources the second fcgi is > > > > > a good idea so your site is not immediately dead. You have two > > > > > chances. > > > > > > > > > > Tom > > > > > > > > Tom-- > > > > > > > > Yes, the memory limit came into play when I upgraded to edge. I > > > > definitely think edge has higher memory requirements. Unfortunately, > > > > it''s something I needed to do, so couldn''t be avoided. > > > > > > > > We moved to a container on TxD and that solved the memory problems. > > > > It''s also more expensive, but reasonable for this app. The increased > > > > speed is a huge benefit. > > > > > > > > I think I just recognize the limitations of shared hosting. The move to > > > > edge just accelerated the need to switch off of it, but that was > > > > inevitable. Perhaps TxD will increase their memory limits when 1.2 > > > > releases. > > > > > > > > Jake > > > > > > > > -- > > > > Posted via http://www.ruby-forum.com/. > > > > > > > > > > > > > > > > > > > > > > -- > > > Tom Davies > > > > > > http://atomgiant.com > > > http://gifthat.com > > > > > > > > > > > > > > -- > Tom Davies > > http://atomgiant.com > http://gifthat.com > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---