I''ve been messing around with Puppet on a VM on my personal desktop. It looks descent. I was wondering what kind of load this thing would have managing about 400 nodes. Does this thing require a beefy server? -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Hi, On 05/10/2011 03:04 PM, Panaman wrote:> I''ve been messing around with Puppet on a VM on my personal desktop. > It looks descent. I was wondering what kind of load this thing would > have managing about 400 nodes. > Does this thing require a beefy server? >we manage 320 nodes on a XEN VM based Puppet Master. 4 GB RAM, 1 Core with 2.3 GHz 80% used. We have dashboard and puppetmaster running on apache passenger and mysql running locally. hth, Martin -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
when you say 80% used.... do you meen like your cpu is 80%? On May 10, 10:16 am, Martin Alfke <tux...@gmail.com> wrote:> Hi, > > On 05/10/2011 03:04 PM, Panaman wrote: > > > I''ve been messing around with Puppet on a VM on my personal desktop. > > It looks descent. I was wondering what kind of load this thing would > > have managing about 400 nodes. > > Does this thing require a beefy server? > > we manage 320 nodes on a XEN VM based Puppet Master. > 4 GB RAM, 1 Core with 2.3 GHz 80% used. > We have dashboard and puppetmaster running on apache passenger and mysql > running locally. > > hth, > > Martin-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Charles Johnson
2011-May-10 14:39 UTC
Re: [Puppet Users] Puppet Master System Requirements
We have puppet managing 800 nodes from a single dedicated CentOS server with 16GB ram and 8 cores, which is our standard infrastructure box. We use apache, passenger and ruby 1.8.7. Altogether about 10GB is used for puppet. All 800 nodes check in every 30 minutes. Load levels via uptime are typically 0.4 across the board. ~Charles~ On Tue, May 10, 2011 at 8:04 AM, Panaman <panaman@gmail.com> wrote:> I''ve been messing around with Puppet on a VM on my personal desktop. > It looks descent. I was wondering what kind of load this thing would > have managing about 400 nodes. > Does this thing require a beefy server? > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Martin Alfke
2011-May-10 14:45 UTC
Re: [Puppet Users] Re: Puppet Master System Requirements
On 05/10/2011 04:25 PM, Panaman wrote:> when you say 80% used.... do you meen like your cpu is 80%?CPU is 20% unused and 80% used (according to /proc/stat)> > On May 10, 10:16 am, Martin Alfke <tux...@gmail.com> wrote: >> Hi, >> >> On 05/10/2011 03:04 PM, Panaman wrote: >> >>> I''ve been messing around with Puppet on a VM on my personal desktop. >>> It looks descent. I was wondering what kind of load this thing would >>> have managing about 400 nodes. >>> Does this thing require a beefy server? >> >> we manage 320 nodes on a XEN VM based Puppet Master. >> 4 GB RAM, 1 Core with 2.3 GHz 80% used. >> We have dashboard and puppetmaster running on apache passenger and mysql >> running locally. >> >> hth, >> >> Martin >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
thanks guys, this is very useful to me. On May 10, 10:45 am, Martin Alfke <tux...@gmail.com> wrote:> On 05/10/2011 04:25 PM, Panaman wrote: > > > when you say 80% used.... do you meen like your cpu is 80%? > > CPU is 20% unused and 80% used (according to /proc/stat) > > > > > > > > > > > On May 10, 10:16 am, Martin Alfke <tux...@gmail.com> wrote: > >> Hi, > > >> On 05/10/2011 03:04 PM, Panaman wrote: > > >>> I''ve been messing around with Puppet on a VM on my personal desktop. > >>> It looks descent. I was wondering what kind of load this thing would > >>> have managing about 400 nodes. > >>> Does this thing require a beefy server? > > >> we manage 320 nodes on a XEN VM based Puppet Master. > >> 4 GB RAM, 1 Core with 2.3 GHz 80% used. > >> We have dashboard and puppetmaster running on apache passenger and mysql > >> running locally. > > >> hth, > > >> Martin-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
I''m thinking quad xeon with 8gb of ram should be sufficient. I am probably going to be using FreeBSD On May 10, 11:31 am, Panaman <pana...@gmail.com> wrote:> thanks guys, this is very useful to me. > > On May 10, 10:45 am, Martin Alfke <tux...@gmail.com> wrote: > > > > > > > > > On 05/10/2011 04:25 PM, Panaman wrote: > > > > when you say 80% used.... do you meen like your cpu is 80%? > > > CPU is 20% unused and 80% used (according to /proc/stat) > > > > On May 10, 10:16 am, Martin Alfke <tux...@gmail.com> wrote: > > >> Hi, > > > >> On 05/10/2011 03:04 PM, Panaman wrote: > > > >>> I''ve been messing around with Puppet on a VM on my personal desktop. > > >>> It looks descent. I was wondering what kind of load this thing would > > >>> have managing about 400 nodes. > > >>> Does this thing require a beefy server? > > > >> we manage 320 nodes on a XEN VM based Puppet Master. > > >> 4 GB RAM, 1 Core with 2.3 GHz 80% used. > > >> We have dashboard and puppetmaster running on apache passenger and mysql > > >> running locally. > > > >> hth, > > > >> Martin-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Nigel Kersten
2011-May-10 15:35 UTC
Re: [Puppet Users] Re: Puppet Master System Requirements
On Tue, May 10, 2011 at 3:32 PM, Panaman <panaman@gmail.com> wrote:> I''m thinking quad xeon with 8gb of ram should be sufficient. I am > probably going to be using FreeBSD >Unless you''re running a particularly leaky Ruby stack, I tend to find that with that sort of CPU you''re unlikely to make use of more than 4GB RAM. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On May 10, 2011, at 6:04 AM, Panaman wrote:> I''ve been messing around with Puppet on a VM on my personal desktop. > It looks descent. I was wondering what kind of load this thing would > have managing about 400 nodes. > Does this thing require a beefy server?With that many clients, make sure you''re using Passenger if possible. Do not use Webrick. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
My setup handles about 40 clients on a default half hour schedule. Manifests take anywhere from .1 to .4 seconds to compile on the master. The master is a 3.46ghz hex-core xeon server with 24GB of ram and a raid 6 (lsi hardware raid) on 7200rpm sata drives. 15 minute load on the server is usually around .17 and that''s with 3 other vm''s(testing vm''s, usually idle) running on the server that is the puppetmaster. Obviously really really complicated manifests will probably take a tad bit longer to compile so YMMV. On May 10, 9:04 am, Panaman <pana...@gmail.com> wrote:> I''ve been messing around with Puppet on a VM on my personal desktop. > It looks descent. I was wondering what kind of load this thing would > have managing about 400 nodes. > Does this thing require a beefy server?-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Nigel Kersten
2011-May-10 19:00 UTC
Re: [Puppet Users] Re: Puppet Master System Requirements
On Tue, May 10, 2011 at 6:32 PM, Blazer40 <blazer40@gmail.com> wrote:> My setup handles about 40 clients on a default half hour schedule. > Manifests take anywhere from .1 to .4 seconds to compile on the > master. The master is a 3.46ghz hex-core xeon server with 24GB of ram > and a raid 6 (lsi hardware raid) on 7200rpm sata drives. 15 minute > load on the server is usually around .17 and that''s with 3 other > vm''s(testing vm''s, usually idle) running on the server that is the > puppetmaster. Obviously really really complicated manifests will > probably take a tad bit longer to compile so YMMV. >Great detail. How much peak RAM usage do you usually end up with? Are you running with Passenger/Apache ? Unicorn? nginx? mongrel? Details on the number of backends would be great.> > On May 10, 9:04 am, Panaman <pana...@gmail.com> wrote: > > I''ve been messing around with Puppet on a VM on my personal desktop. > > It looks descent. I was wondering what kind of load this thing would > > have managing about 400 nodes. > > Does this thing require a beefy server? > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- Nigel Kersten Product, Puppet Labs @nigelkersten -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Matthew Marlowe
2011-May-11 05:41 UTC
Re: [Puppet Users] Puppet Master System Requirements
Keep in mind that there are many ways to run puppet. We manage ~100 nodes with just a single puppet master running within a gentoo VM w/ only single cpu core and 2GB ram. Catalog compile times average under 0.6 seconds. This is also w/ web brick. The puppet master VM also serves as a master nfs server and gentoo build server. Thats a lot of stuff on a single small VM, but it works perfectly for us because: a) our default puppet run interval is 4hrs (if something goes wrong w/ one of our manifests or the server, we''ll probably notice it and stop it before too many servers get updated - for our purposes, we don''t see any benefit to using an interval less than 4hrs. 4hrs is certainly sufficient for most common security updates and we also do not want to have normal updates impacting production performance during peak business hours - so 25% of servers updating every hour is perfect for us. ). b) Many of our servers, mostly the gentoo ones, only execute puppet when puppetrun is invoked either manually by systems administrators for the specific nodes they are reconfiguring or automatically as part of a nightly update systems maintenance cron job). Basically, puppet is extremely flexible w/ hardware, and it is likely your own preferences and production requirements will dictate the hardware needed rather than puppet itself. On Tuesday, May 10, 2011 06:04:22 am Panaman wrote:> I''ve been messing around with Puppet on a VM on my personal desktop. > It looks descent. I was wondering what kind of load this thing would > have managing about 400 nodes. > Does this thing require a beefy server?Matt -- Matthew Marlowe / 858-400-7430 / DeployLinux Consulting, Inc Professional Linux Hosting and Systems Administration Services www.deploylinux.net * matt@deploylinux.net ''MattM'' @ irc.freenode.net -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Since I use passenger on ruby enterprise 2011.03, here is the output from passenger-memory-stats: ---------- Apache processes ---------- PID PPID VMSize Private Name -------------------------------------- 7900 26907 229.3 MB 0.5 MB /usr/sbin/httpd 21341 26907 229.3 MB 0.5 MB /usr/sbin/httpd 26907 1 229.0 MB 0.4 MB /usr/sbin/httpd 26924 26907 229.3 MB 0.5 MB /usr/sbin/httpd 26925 26907 229.3 MB 0.5 MB /usr/sbin/httpd 26926 26907 229.3 MB 0.6 MB /usr/sbin/httpd 26927 26907 229.3 MB 0.5 MB /usr/sbin/httpd 26928 26907 229.3 MB 0.5 MB /usr/sbin/httpd 26929 26907 229.3 MB 0.5 MB /usr/sbin/httpd 26930 26907 229.3 MB 0.5 MB /usr/sbin/httpd 26931 26907 229.3 MB 0.5 MB /usr/sbin/httpd 27245 26907 229.3 MB 0.5 MB /usr/sbin/httpd ### Processes: 12 ### Total private dirty RSS: 6.25 MB -------- Nginx processes -------- ### Processes: 0 ### Total private dirty RSS: 0.00 MB ----- Passenger processes ----- PID VMSize Private Name ------------------------------- 1707 182.1 MB 75.2 MB Rails: /var/www/apps/puppet-dashboard 3596 208.7 MB 99.0 MB Rack: /usr/share/puppet/rack/puppetmasterd 26909 16.5 MB 0.2 MB PassengerWatchdog 26912 155.8 MB 0.9 MB PassengerHelperAgent 26914 56.8 MB 7.7 MB Passenger spawn server 26919 50.2 MB 0.4 MB PassengerLoggingAgent ### Processes: 6 ### Total private dirty RSS: 183.50 MB AKA, it doesn''t eat alot (at least compared to some of the other rails apps I have running on other servers!) On May 10, 3:00 pm, Nigel Kersten <ni...@puppetlabs.com> wrote:> On Tue, May 10, 2011 at 6:32 PM, Blazer40 <blaze...@gmail.com> wrote: > > My setup handles about 40 clients on a default half hour schedule. > > Manifests take anywhere from .1 to .4 seconds to compile on the > > master. The master is a 3.46ghz hex-core xeon server with 24GB of ram > > and a raid 6 (lsi hardware raid) on 7200rpm sata drives. 15 minute > > load on the server is usually around .17 and that''s with 3 other > > vm''s(testing vm''s, usually idle) running on the server that is the > > puppetmaster. Obviously really really complicated manifests will > > probably take a tad bit longer to compile so YMMV. > > Great detail. > > How much peak RAM usage do you usually end up with? > > Are you running with Passenger/Apache ? Unicorn? nginx? mongrel? > > Details on the number of backends would be great. > > > > > > > > > > > > > On May 10, 9:04 am, Panaman <pana...@gmail.com> wrote: > > > I''ve been messing around with Puppet on a VM on my personal desktop. > > > It looks descent. I was wondering what kind of load this thing would > > > have managing about 400 nodes. > > > Does this thing require a beefy server? > > > -- > > You received this message because you are subscribed to the Google Groups > > "Puppet Users" group. > > To post to this group, send email to puppet-users@googlegroups.com. > > To unsubscribe from this group, send email to > > puppet-users+unsubscribe@googlegroups.com. > > For more options, visit this group at > >http://groups.google.com/group/puppet-users?hl=en. > > -- > Nigel Kersten > Product, Puppet Labs > @nigelkersten-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Since I use passenger on ruby enterprise 2011.03, here is the output from passenger-memory-stats: ---------- Apache processes ---------- PID PPID VMSize Private Name -------------------------------------- 7900 26907 229.3 MB 0.5 MB /usr/sbin/httpd 21341 26907 229.3 MB 0.5 MB /usr/sbin/httpd 26907 1 229.0 MB 0.4 MB /usr/sbin/httpd 26924 26907 229.3 MB 0.5 MB /usr/sbin/httpd 26925 26907 229.3 MB 0.5 MB /usr/sbin/httpd 26926 26907 229.3 MB 0.6 MB /usr/sbin/httpd 26927 26907 229.3 MB 0.5 MB /usr/sbin/httpd 26928 26907 229.3 MB 0.5 MB /usr/sbin/httpd 26929 26907 229.3 MB 0.5 MB /usr/sbin/httpd 26930 26907 229.3 MB 0.5 MB /usr/sbin/httpd 26931 26907 229.3 MB 0.5 MB /usr/sbin/httpd 27245 26907 229.3 MB 0.5 MB /usr/sbin/httpd ### Processes: 12 ### Total private dirty RSS: 6.25 MB -------- Nginx processes -------- ### Processes: 0 ### Total private dirty RSS: 0.00 MB ----- Passenger processes ----- PID VMSize Private Name ------------------------------- 1707 182.1 MB 75.2 MB Rails: /var/www/apps/puppet-dashboard 3596 208.7 MB 99.0 MB Rack: /usr/share/puppet/rack/puppetmasterd 26909 16.5 MB 0.2 MB PassengerWatchdog 26912 155.8 MB 0.9 MB PassengerHelperAgent 26914 56.8 MB 7.7 MB Passenger spawn server 26919 50.2 MB 0.4 MB PassengerLoggingAgent ### Processes: 6 ### Total private dirty RSS: 183.50 MB AKA, it doesn''t eat alot (at least compared to some of the other rails apps I have running on other servers!) On May 10, 3:00 pm, Nigel Kersten <ni...@puppetlabs.com> wrote:> On Tue, May 10, 2011 at 6:32 PM, Blazer40 <blaze...@gmail.com> wrote: > > My setup handles about 40 clients on a default half hour schedule. > > Manifests take anywhere from .1 to .4 seconds to compile on the > > master. The master is a 3.46ghz hex-core xeon server with 24GB of ram > > and a raid 6 (lsi hardware raid) on 7200rpm sata drives. 15 minute > > load on the server is usually around .17 and that''s with 3 other > > vm''s(testing vm''s, usually idle) running on the server that is the > > puppetmaster. Obviously really really complicated manifests will > > probably take a tad bit longer to compile so YMMV. > > Great detail. > > How much peak RAM usage do you usually end up with? > > Are you running with Passenger/Apache ? Unicorn? nginx? mongrel? > > Details on the number of backends would be great. > > > > > > > > > > > > > On May 10, 9:04 am, Panaman <pana...@gmail.com> wrote: > > > I''ve been messing around with Puppet on a VM on my personal desktop. > > > It looks descent. I was wondering what kind of load this thing would > > > have managing about 400 nodes. > > > Does this thing require a beefy server? > > > -- > > You received this message because you are subscribed to the Google Groups > > "Puppet Users" group. > > To post to this group, send email to puppet-users@googlegroups.com. > > To unsubscribe from this group, send email to > > puppet-users+unsubscribe@googlegroups.com. > > For more options, visit this group at > >http://groups.google.com/group/puppet-users?hl=en. > > -- > Nigel Kersten > Product, Puppet Labs > @nigelkersten-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
I''m on the extremely small scale end from what has been previously described, but this could prove useful to those who want to manage a small number of hosts and are curious how little they need to run Puppet. I''ve just begun using Puppet, and so only have 10 hosts right now, but here''s what I''m running... CentOS 5.6 x64 VMware ESXi virtual machine - 512MB RAM - 1 x 2.33GHz CPU - Paravirtual Roles are... - Puppetmaster - puppet-dashboard w/ mysql & apache + passanger - local yum repo over http - OSSEC server. - Trey According to zabbix my system averages about .20 CPU load over 15 minute average. I occassionally peak 1.0 but that''s typically when I''m running puppet manually very rapidly during testing. Memory usage is about 70% on average. On May 11, 12:41 am, Matthew Marlowe <m...@deploylinux.net> wrote:> Keep in mind that there are many ways to run puppet. > > We manage ~100 nodes with just a single puppet master running within a gentoo > VM w/ only single cpu core and 2GB ram. Catalog compile times average under > 0.6 seconds. This is also w/ web brick. The puppet master VM also serves as > a master nfs server and gentoo build server. > > Thats a lot of stuff on a single small VM, but it works perfectly for us > because: > a) our default puppet run interval is 4hrs (if something goes wrong w/ one of > our manifests or the server, we''ll probably notice it and stop it before too > many servers get updated - for our purposes, we don''t see any benefit to using > an interval less than 4hrs. 4hrs is certainly sufficient for most common > security updates and we also do not want to have normal updates impacting > production performance during peak business hours - so 25% of servers updating > every hour is perfect for us. ). > b) Many of our servers, mostly the gentoo ones, only execute puppet when > puppetrun is invoked either manually by systems administrators for the > specific nodes they are reconfiguring or automatically as part of a nightly > update systems maintenance cron job). > > Basically, puppet is extremely flexible w/ hardware, and it is likely your own > preferences and production requirements will dictate the hardware needed > rather than puppet itself. > > On Tuesday, May 10, 2011 06:04:22 am Panaman wrote: > > > I''ve been messing around with Puppet on a VM on my personal desktop. > > It looks descent. I was wondering what kind of load this thing would > > have managing about 400 nodes. > > Does this thing require a beefy server? > > Matt > -- > Matthew Marlowe / 858-400-7430 / DeployLinux Consulting, Inc > Professional Linux Hosting and Systems Administration Services > www.deploylinux.net * m...@deploylinux.net > ''MattM'' @ irc.freenode.net-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.