Hi list, I am looking at configuration management tool options. I have a large fleet (> 3,000 hosts) and highly heterogeneous (RHEL3-5, CentOS, 5RH7, Solaris 10 LDOMs/zones, Solaris 8-9, AIX 5.3 & 6.1 LPARs, HP-UX & Tru64 + Windows). We care mainly about RHEL and new versions of Solaris & AIX but ability to compile on older versions is an advantage. Probably, the Windows will be managed by SCCM. I have read that Puppet could have scalability problems to a site as large as ours. To keep this simple, I''d like feedback on whether that is likely to be true for us. A rep from CFengine has told me that ours would be the largest Puppet site in the world (I think that''s not true). Could someone confirm? General feedback also welcome. Kind regards, Alex Harvey -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
3k? I know of larger but can''t comment on who they are, but they are close to 8-10k nodes AFAIK if not more by now. On Mon, May 14, 2012 at 2:53 AM, Alex Harvey <alexharv074@gmail.com> wrote:> Hi list, > > I am looking at configuration management tool options. > > I have a large fleet (> 3,000 hosts) and highly heterogeneous > (RHEL3-5, CentOS, 5RH7, Solaris 10 LDOMs/zones, Solaris 8-9, AIX 5.3 & > 6.1 LPARs, HP-UX & Tru64 + Windows). We care mainly about RHEL and > new versions of Solaris & AIX but ability to compile on older versions > is an advantage. Probably, the Windows will be managed by SCCM. > > I have read that Puppet could have scalability problems to a site as > large as ours. To keep this simple, I''d like feedback on whether that > is likely to be true for us. > > A rep from CFengine has told me that ours would be the largest Puppet > site in the world (I think that''s not true). Could someone confirm? > > General feedback also welcome. > > Kind regards, > Alex Harvey > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Sun, May 13, 2012 at 11:53 PM, Alex Harvey <alexharv074@gmail.com> wrote:> Hi list, > > I am looking at configuration management tool options. > > I have a large fleet (> 3,000 hosts) and highly heterogeneous > (RHEL3-5, CentOS, 5RH7, Solaris 10 LDOMs/zones, Solaris 8-9, AIX 5.3 & > 6.1 LPARs, HP-UX & Tru64 + Windows). We care mainly about RHEL and > new versions of Solaris & AIX but ability to compile on older versions > is an advantage. Probably, the Windows will be managed by SCCM. > > I have read that Puppet could have scalability problems to a site as > large as ours. To keep this simple, I''d like feedback on whether that > is likely to be true for us. > > A rep from CFengine has told me that ours would be the largest Puppet > site in the world (I think that''s not true). Could someone confirm? >There are at least a couple of deployments that are more like 30k hosts than 3k.... It''s absolutely not true.> > General feedback also welcome. > > Kind regards, > Alex Harvey > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- Nigel Kersten | http://puppetlabs.com | @nigelkersten Schedule Meetings at: http://tungle.me/nigelkersten -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Nigel, Don''t we have one or two an order of magnitude larger than that? http://indico.cern.ch/getFile.py/access?contribId=29&sessionId=3&resId=3&materialId=slides&confId=160737 (warning: large PDF) I think they''re looking at 300K or so. And I believe we have a few more at that size as well. -Eric -- Eric Shamow Professional Services http://puppetlabs.com/ (c)631.871.6441 On Monday, May 14, 2012 at 12:35 PM, Nigel Kersten wrote:> > > On Sun, May 13, 2012 at 11:53 PM, Alex Harvey <alexharv074@gmail.com (mailto:alexharv074@gmail.com)> wrote: > > Hi list, > > > > I am looking at configuration management tool options. > > > > I have a large fleet (> 3,000 hosts) and highly heterogeneous > > (RHEL3-5, CentOS, 5RH7, Solaris 10 LDOMs/zones, Solaris 8-9, AIX 5.3 & > > 6.1 LPARs, HP-UX & Tru64 + Windows). We care mainly about RHEL and > > new versions of Solaris & AIX but ability to compile on older versions > > is an advantage. Probably, the Windows will be managed by SCCM. > > > > I have read that Puppet could have scalability problems to a site as > > large as ours. To keep this simple, I''d like feedback on whether that > > is likely to be true for us. > > > > A rep from CFengine has told me that ours would be the largest Puppet > > site in the world (I think that''s not true). Could someone confirm? > > There are at least a couple of deployments that are more like 30k hosts than 3k.... > > It''s absolutely not true. > > > > > > General feedback also welcome. > > > > Kind regards, > > Alex Harvey > > > > -- > > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > > To post to this group, send email to puppet-users@googlegroups.com (mailto:puppet-users@googlegroups.com). > > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com (mailto:puppet-users%2Bunsubscribe@googlegroups.com). > > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. > > > > > > -- > Nigel Kersten | http://puppetlabs.com | @nigelkersten > Schedule Meetings at: http://tungle.me/nigelkersten > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com (mailto:puppet-users@googlegroups.com). > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com (mailto:puppet-users+unsubscribe@googlegroups.com). > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Sun, May 13, 2012 at 11:53 PM, Alex Harvey <alexharv074@gmail.com> wrote:> > A rep from CFengine has told me that ours would be the largest Puppet > site in the world (I think that''s not true). Could someone confirm? > >Here''s a recording of a short talk I gave about some performance testing I did last summer. These are the numbers I gathered from a real world production deployment of tens of thousands of nodes running against multiple Puppet Masters: http://www.youtube.com/watch?v=wjXrJjKymM4 The CFengine rep is certainly not speaking the truth on this matter. I hope they''re simply ignorant and not willingly lying. Cheers, -- Jeff McCune -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Hi Alex > I have read that Puppet could have scalability problems to a site as> large as ours. To keep this simple, I''d like feedback on whether that > is likely to be true for us.On the actual scaling question... Puppet (2.6.x+) is easy to scale as it is just HTTP(S) traffic. You can front-end Puppet with a load balancer and then just cluster master nodes to scale to your needs in the same way you can grow any HTTP service. Regards James Turnbull -- James Turnbull Puppet Labs 1-503-734-8571 To schedule a meeting with me: http://tungle.me/jamtur01 -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Hi Alex, Just a quick point, Glad to see Team Puppet out in force to show you the light! That CF rep, needs a swift kick in the …. well, I don''t want to advocate violence here, I think he can do with being added to your junk mail list. 3000 hosts? No problem! You have to serve one session per client every 30 mins, and your puppet master doesn''t process much unless you''ve changed something. That''s about a hundred hits a minute, so use apache instead of webrick, and your good to go! CERN as I understand it, are the guys trying to "recreate the big bang". Not simulate it, like NASA, or do some complex maths research about it… actually do it. Now, if anyone on the planet needs to be 100% sure that their gazillion or so hosts are all playing nice, it''s them. They use Puppet. Next question!? On 14 May 2012, at 07:53, Alex Harvey wrote:> Hi list, > > I am looking at configuration management tool options. > > I have a large fleet (> 3,000 hosts) and highly heterogeneous > (RHEL3-5, CentOS, 5RH7, Solaris 10 LDOMs/zones, Solaris 8-9, AIX 5.3 & > 6.1 LPARs, HP-UX & Tru64 + Windows). We care mainly about RHEL and > new versions of Solaris & AIX but ability to compile on older versions > is an advantage. Probably, the Windows will be managed by SCCM. > > I have read that Puppet could have scalability problems to a site as > large as ours. To keep this simple, I''d like feedback on whether that > is likely to be true for us. > > A rep from CFengine has told me that ours would be the largest Puppet > site in the world (I think that''s not true). Could someone confirm? > > General feedback also welcome. > > Kind regards, > Alex Harvey > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Wanted to point out I manage over 3000 x86 tablets with puppet. This doesn''t include production servers. Sent via BlackBerry -----Original Message----- From: Alex Harvey <alexharv074@gmail.com> Sender: puppet-users@googlegroups.com Date: Sun, 13 May 2012 23:53:52 To: Puppet Users<puppet-users@googlegroups.com> Reply-To: puppet-users@googlegroups.com Subject: [Puppet Users] Puppet / scalability Hi list, I am looking at configuration management tool options. I have a large fleet (> 3,000 hosts) and highly heterogeneous (RHEL3-5, CentOS, 5RH7, Solaris 10 LDOMs/zones, Solaris 8-9, AIX 5.3 & 6.1 LPARs, HP-UX & Tru64 + Windows). We care mainly about RHEL and new versions of Solaris & AIX but ability to compile on older versions is an advantage. Probably, the Windows will be managed by SCCM. I have read that Puppet could have scalability problems to a site as large as ours. To keep this simple, I''d like feedback on whether that is likely to be true for us. A rep from CFengine has told me that ours would be the largest Puppet site in the world (I think that''s not true). Could someone confirm? General feedback also welcome. Kind regards, Alex Harvey -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Monday, 14 May 2012 18:38:29 UTC+2, Eric Shamow wrote:> > Nigel, > > Don''t we have one or two an order of magnitude larger than that? > > > http://indico.cern.ch/getFile.py/access?contribId=29&sessionId=3&resId=3&materialId=slides&confId=160737 (warning: > large PDF) > > I think they''re looking at 300K or so. And I believe we have a few more > at that size as well. > > We are currently along way from that with a single master and around 200nodes while we learn puppet and migrate away from quattor. The 300k figures are a couple of years away, we will have a respectable figure sometime this year we would hope. Steve (CERN IT)> -Eric > > -- > > Eric Shamow > Professional Services > http://puppetlabs.com/ > (c)631.871.6441 > > On Monday, May 14, 2012 at 12:35 PM, Nigel Kersten wrote: > > > > On Sun, May 13, 2012 at 11:53 PM, Alex Harvey <alexharv074@gmail.com>wrote: > > Hi list, > > I am looking at configuration management tool options. > > I have a large fleet (> 3,000 hosts) and highly heterogeneous > (RHEL3-5, CentOS, 5RH7, Solaris 10 LDOMs/zones, Solaris 8-9, AIX 5.3 & > 6.1 LPARs, HP-UX & Tru64 + Windows). We care mainly about RHEL and > new versions of Solaris & AIX but ability to compile on older versions > is an advantage. Probably, the Windows will be managed by SCCM. > > I have read that Puppet could have scalability problems to a site as > large as ours. To keep this simple, I''d like feedback on whether that > is likely to be true for us. > > A rep from CFengine has told me that ours would be the largest Puppet > site in the world (I think that''s not true). Could someone confirm? > > > There are at least a couple of deployments that are more like 30k hosts > than 3k.... > > It''s absolutely not true. > > > > > General feedback also welcome. > > Kind regards, > Alex Harvey > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > > > > > -- > Nigel Kersten | http://puppetlabs.com | @nigelkersten > Schedule Meetings at: http://tungle.me/nigelkersten > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/P6IzZ2PT1DEJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 14 May 2012 18:35, Nigel Kersten <nigel@puppetlabs.com> wrote:> > > On Sun, May 13, 2012 at 11:53 PM, Alex Harvey <alexharv074@gmail.com> wrote: >> >> Hi list, >> >> I am looking at configuration management tool options. >> >> I have a large fleet (> 3,000 hosts) and highly heterogeneous >> (RHEL3-5, CentOS, 5RH7, Solaris 10 LDOMs/zones, Solaris 8-9, AIX 5.3 & >> 6.1 LPARs, HP-UX & Tru64 + Windows). We care mainly about RHEL and >> new versions of Solaris & AIX but ability to compile on older versions >> is an advantage. Probably, the Windows will be managed by SCCM. >> >> I have read that Puppet could have scalability problems to a site as >> large as ours. To keep this simple, I''d like feedback on whether that >> is likely to be true for us. >> >> A rep from CFengine has told me that ours would be the largest Puppet >> site in the world (I think that''s not true). Could someone confirm? > > > There are at least a couple of deployments that are more like 30k hosts than > 3k.... > > It''s absolutely not true. >However, storeconfigs does have some scalability issues, so I guess that they aren''t using that feature with 30k hosts? (soon there is of course the puppetdb that should replace it and be more scalable afaik) -- Erik Dalén -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Thanks kindly to all for the thoughtful responses. I''ll be looking closely at the idea of using load balanced Puppet servers. I wonder if anyone has any thoughts on the other problem I''m told I''ll probably encounter, namely the difficulty in compiling Ruby and other packages in AIX, HPUX & earlier versions of Solaris? -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Wednesday, May 16, 2012 4:20:23 PM UTC+2, Alex Harvey wrote:> > Thanks kindly to all for the thoughtful responses. I''ll be looking > closely at the idea of using load balanced Puppet servers. > > I wonder if anyone has any thoughts on the other problem I''m told I''ll > probably encounter, namely the difficulty in compiling Ruby and other > packages in AIX, HPUX & earlier versions of Solaris? >I''m running puppet with Solaris and I haven''t had to compile anything. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/LQUmnKs43DsJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Tue, May 15, 2012 at 7:03 AM, Erik Dalén <erik.gustav.dalen@gmail.com>wrote:> On 14 May 2012 18:35, Nigel Kersten <nigel@puppetlabs.com> wrote: > > > > > > On Sun, May 13, 2012 at 11:53 PM, Alex Harvey <alexharv074@gmail.com> > wrote: > >> > >> Hi list, > >> > >> I am looking at configuration management tool options. > >> > >> I have a large fleet (> 3,000 hosts) and highly heterogeneous > >> (RHEL3-5, CentOS, 5RH7, Solaris 10 LDOMs/zones, Solaris 8-9, AIX 5.3 & > >> 6.1 LPARs, HP-UX & Tru64 + Windows). We care mainly about RHEL and > >> new versions of Solaris & AIX but ability to compile on older versions > >> is an advantage. Probably, the Windows will be managed by SCCM. > >> > >> I have read that Puppet could have scalability problems to a site as > >> large as ours. To keep this simple, I''d like feedback on whether that > >> is likely to be true for us. > >> > >> A rep from CFengine has told me that ours would be the largest Puppet > >> site in the world (I think that''s not true). Could someone confirm? > > > > > > There are at least a couple of deployments that are more like 30k hosts > than > > 3k.... > > > > It''s absolutely not true. > > > > However, storeconfigs does have some scalability issues, so I guess > that they aren''t using that feature with 30k hosts? > > (soon there is of course the puppetdb that should replace it and be > more scalable afaik) > >Let me firm that up a bit :) PuppetDB *does* replace the existing storeconfigs, and *is* much more scalable :) -- Nigel Kersten | http://puppetlabs.com | @nigelkersten Schedule Meetings at: http://tungle.me/nigelkersten -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 15 May 2012 22:10, Steve Traylen <steve.traylen@cern.ch> wrote:> > We are currently along way from that with a single master and around 200 > nodes > while we learn puppet and migrate away from quattor. > The 300k figures are a couple of years away, we will have a respectable > figure > sometime this year we would hope. > > Steve (CERN IT) >I have had a careful look at http://projects.puppetlabs.com/projects/1/wiki/Whos_Using_Puppet This list is, in fairness, consistent with what the CFEngine rep told me. If there are sites out there with >3K or even >10K nodes, but not listed at Puppet''s website, it is reasonable to suppose that the CFEngine rep doesn''t know about them. :) Is it possible that this list needs updating? Maybe large sites aren''t as keen to share their stories with the world? From the list, I also get the feeling that sites using Puppet to manage their infrastructure tend to be Linux sites. Google appears to be using Puppet to manage "all recent Mac OS X and Linux desktops, laptops, servers in the corporate infrastructure". They don''t use it "in production", whatever that means. For those taking it beyond Linux and Mac OS X, I see Solaris mentioned a bit. Are Sun/Oracle still using it for their Solaris kit? I don''t see any mention of AIX at all. I believe I have heard the same rumour of the large company at 10K nodes that uses Solaris, AIX and Linux. So, I am no longer doubting that Puppet can scale to a site as large as ours - especially given what James Turnbull wrote above - but I''d like to ensure that I get my facts right. Is it possible that the large ~ 10K site is one of the first of its kind in the world - especially to use it for AIX? What about the 30K site? A CFEngine document I downloaded claims, "Puppet has recently retracted claims of running 35000 servers at a major bank." Is this also true? Best wishes. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.