All, I''m trying to figure out the best way to estimate the amount of memory usage that will be taken up by Mongrel and I''m hoping that you have some estimates that can help me. I''m currently estimating the following: Memory Required = Number of running puppetmasters * size of manifests * number of clients managed The mongrel model doesn''t seem to *ever* reclaim memory. Suggestions on reclaiming memory welcome though I''m sure most people will just urge me to move to Passenger. Thanks, Trevor --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
On Aug 13, 2009, at 4:36 PM, Trevor Vaughan wrote:> > All, > > I''m trying to figure out the best way to estimate the amount of memory > usage that will be taken up by Mongrel and I''m hoping that you have > some estimates that can help me. > > I''m currently estimating the following: > > Memory Required = Number of running puppetmasters * size of manifests > * number of clients managed > > The mongrel model doesn''t seem to *ever* reclaim memory. > > Suggestions on reclaiming memory welcome though I''m sure most people > will just urge me to move to Passenger.Passenger is slowly becoming the ''official'' way to run puppetmaster, and is recommended by Reductive Labs. As far as memory, there are many dependancies like 32-bit over 64-bit, the size, amount of files transfered through puppet, your manifests, variables, clients used, etc. Some is also dependent upon the version ruby itself and it''s optimizations. Out of all of these variables what I CAN say are most important: - files and the size of files transfered though Puppet - 32 bit over 64 bit. 0.25 will definitely help with the memory consumption with file usage. -L -- Larry Ludwig Reductive Labs --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Trevor Vaughan <peiriannydd@gmail.com> writes:> All, > > I''m trying to figure out the best way to estimate the amount of memory > usage that will be taken up by Mongrel and I''m hoping that you have > some estimates that can help me. > > I''m currently estimating the following: > > Memory Required = Number of running puppetmasters * size of manifests > * number of clients managed > > The mongrel model doesn''t seem to *ever* reclaim memory. > > Suggestions on reclaiming memory welcome though I''m sure most people > will just urge me to move to Passenger.You''re right--use Passenger :) When we were running Mongrel out puppetmasterd processes started out at 100MB and grew to 1G in a couple of weeks. With Passenger processes don''t stick around as long so if nothing else you dodge the memory leaks. And we''ve heard that some of the memory leak problems are in Ruby itself--not Puppet. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Interesting, thanks for the information. My only concern with a 32 bit architecture, is that my memory usage may legitimately grow above 2G which would toast the puppetmaster process pretty quickly. Thanks, Trevor On 08/13/2009 05:53 PM, Brice Figureau wrote:> On 13/08/09 22:36, Trevor Vaughan wrote: >> All, >> >> I''m trying to figure out the best way to estimate the amount of memory >> usage that will be taken up by Mongrel and I''m hoping that you have >> some estimates that can help me. >> >> I''m currently estimating the following: >> >> Memory Required = Number of running puppetmasters * size of manifests >> * number of clients managed > > In 0.24, what cost the most is the file transfer, hopefully 0.25 is > waaay better in this respect. > >> The mongrel model doesn''t seem to *ever* reclaim memory. > > My master don''t grow, but they''re not really under load. > One of the issue with MRI, is that if your master are constantly under > load the ruby garbage collector never really start, and thus memory is > not reclaimed. > >> Suggestions on reclaiming memory welcome though I''m sure most people >> will just urge me to move to Passenger. > > Make sure you''re using a recent ruby. It is known that some older ruby > version are leaking memory. > > I can''t comment on Passenger since I never tried it. But in the near > future (understand when I''ll have some spare cycle) you''ll have another > promising alternative: Puppet on JRuby on Glassfish. > I was able to run this stack in the lab, and limited testing show that > it was at least 30% faster than using mongrel. For the memory part, I > didn''t check, but the JVM has a superior garbage collector than the ruby > one. I really expect some memory gains, just because you''ll be running > only one process... >-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAkqEtk8ACgkQyjMdFR1108BxlQCaA10cbRfhT8TpSRaI2jOlDb6r TkIAn0Cv09xARvbAyLZBrFbs+rx2/3Tr =JftR -----END PGP SIGNATURE----- --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
On Aug 13, 2009, at 8:56 PM, Trevor Vaughan wrote:> > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Interesting, thanks for the information. > > My only concern with a 32 bit architecture, is that my memory usage > may > legitimately grow above 2G which would toast the puppetmaster process > pretty quickly. >The rub is Ruby itself is not optimized for 64 bit. an int for example is double in size. -L -- Larry Ludwig Reductive Labs --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
When I was using mongrels (yes also switched to passenger) I had large memory abuse per mongrel. I was using monit to control the size of the mongrels (e.g. restart a certain mongrel process when its too big), maybe this will be useful for you: http://github.com/ohadlevy/puppet-multipuppetmaster/blob/755dd82b45295db104a15290840eddf8686afda5/templates/monit.erb (note that if you browse the repo, a lot of stuff are already removed - e.g. all the mongrel part can only be found at the repo older tree). Ohad On Fri, Aug 14, 2009 at 8:56 AM, Trevor Vaughan <peiriannydd@gmail.com>wrote:> > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Interesting, thanks for the information. > > My only concern with a 32 bit architecture, is that my memory usage may > legitimately grow above 2G which would toast the puppetmaster process > pretty quickly. > > Thanks, > > Trevor > > On 08/13/2009 05:53 PM, Brice Figureau wrote: > > On 13/08/09 22:36, Trevor Vaughan wrote: > >> All, > >> > >> I''m trying to figure out the best way to estimate the amount of memory > >> usage that will be taken up by Mongrel and I''m hoping that you have > >> some estimates that can help me. > >> > >> I''m currently estimating the following: > >> > >> Memory Required = Number of running puppetmasters * size of manifests > >> * number of clients managed > > > > In 0.24, what cost the most is the file transfer, hopefully 0.25 is > > waaay better in this respect. > > > >> The mongrel model doesn''t seem to *ever* reclaim memory. > > > > My master don''t grow, but they''re not really under load. > > One of the issue with MRI, is that if your master are constantly under > > load the ruby garbage collector never really start, and thus memory is > > not reclaimed. > > > >> Suggestions on reclaiming memory welcome though I''m sure most people > >> will just urge me to move to Passenger. > > > > Make sure you''re using a recent ruby. It is known that some older ruby > > version are leaking memory. > > > > I can''t comment on Passenger since I never tried it. But in the near > > future (understand when I''ll have some spare cycle) you''ll have another > > promising alternative: Puppet on JRuby on Glassfish. > > I was able to run this stack in the lab, and limited testing show that > > it was at least 30% faster than using mongrel. For the memory part, I > > didn''t check, but the JVM has a superior garbage collector than the ruby > > one. I really expect some memory gains, just because you''ll be running > > only one process... > > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (GNU/Linux) > > iEYEARECAAYFAkqEtk8ACgkQyjMdFR1108BxlQCaA10cbRfhT8TpSRaI2jOlDb6r > TkIAn0Cv09xARvbAyLZBrFbs+rx2/3Tr > =JftR > -----END PGP SIGNATURE----- > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Thanks Had. I had been looking at doing something like this and this will save some time. Trevor On 08/13/2009 09:43 PM, Had Levy wrote:> When I was using mongrels (yes also switched to passenger) I had large > memory abuse per mongrel. > I was using monit to control the size of the mongrels (e.g. restart a > certain mongrel process when its too big), maybe this will be useful for > you: > http://github.com/ohadlevy/puppet-multipuppetmaster/blob/755dd82b45295db104a15290840eddf8686afda5/templates/monit.erb > (note that if you browse the repo, a lot of stuff are already removed - e.g. > all the mongrel part can only be found at the repo older tree). > > Ohad > > On Fri, Aug 14, 2009 at 8:56 AM, Trevor Vaughan <peiriannydd@gmail.com>wrote: > > Interesting, thanks for the information. > > My only concern with a 32 bit architecture, is that my memory usage may > legitimately grow above 2G which would toast the puppetmaster process > pretty quickly. > > Thanks, > > Trevor > > On 08/13/2009 05:53 PM, Brice Figureau wrote: >>>> On 13/08/09 22:36, Trevor Vaughan wrote: >>>>> All, >>>>> >>>>> I''m trying to figure out the best way to estimate the amount of memory >>>>> usage that will be taken up by Mongrel and I''m hoping that you have >>>>> some estimates that can help me. >>>>> >>>>> I''m currently estimating the following: >>>>> >>>>> Memory Required = Number of running puppetmasters * size of manifests >>>>> * number of clients managed >>>> In 0.24, what cost the most is the file transfer, hopefully 0.25 is >>>> waaay better in this respect. >>>> >>>>> The mongrel model doesn''t seem to *ever* reclaim memory. >>>> My master don''t grow, but they''re not really under load. >>>> One of the issue with MRI, is that if your master are constantly under >>>> load the ruby garbage collector never really start, and thus memory is >>>> not reclaimed. >>>> >>>>> Suggestions on reclaiming memory welcome though I''m sure most people >>>>> will just urge me to move to Passenger. >>>> Make sure you''re using a recent ruby. It is known that some older ruby >>>> version are leaking memory. >>>> >>>> I can''t comment on Passenger since I never tried it. But in the near >>>> future (understand when I''ll have some spare cycle) you''ll have another >>>> promising alternative: Puppet on JRuby on Glassfish. >>>> I was able to run this stack in the lab, and limited testing show that >>>> it was at least 30% faster than using mongrel. For the memory part, I >>>> didn''t check, but the JVM has a superior garbage collector than the ruby >>>> one. I really expect some memory gains, just because you''ll be running >>>> only one process... >>>> >>>-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAkqFMj8ACgkQyjMdFR1108CERwCgjgRiVRHrC6TgdI1NAGTlg3PZ mfEAnAr0tT5OWV47HE9cEqq69bnJZYqI =SdR9 -----END PGP SIGNATURE----- --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---