I have a specific need to use Puppet to Autoscale a few applications I have deployed on EC2. I''m using Puppet now to manage and create instances and saw a neat video on this subject. I''m missing the a piece of the puzzle. I''m assuming I should use Cloudwatch to send my puppetmaster a notification to then create a new instance with the cloudformation module. Does that sound about right? I looked into cloudformation, but that doesn''t seem right. Can someone point me in the right direction. Thanks. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/ps2yIUWdTMQJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
I use scalr for this purpose.. http://scalr.net/ I think they did away with the free version of it recently. But if you ask me what they''re asking is well worth it! Autoscales up and down based on server load. Works with both AWS and rackspace cloud. -tim On Sun, May 13, 2012 at 9:28 PM, de <dustyeike@gmail.com> wrote:> I have a specific need to use Puppet to Autoscale a few applications I have > deployed on EC2. I''m using Puppet now to manage and create instances and saw > a neat video on this subject. I''m missing the a piece of the puzzle. I''m > assuming I should use Cloudwatch to send my puppetmaster a notification to > then create a new instance with the cloudformation module. > > Does that sound about right? I looked into cloudformation, but that doesn''t > seem right. > > Can someone point me in the right direction. > > Thanks. > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/puppet-users/-/ps2yIUWdTMQJ. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en.-- GPG me!! gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
> http://scalr.net/ > > I think they did away with the free version of it recently. But if you > ask me what they''re asking is well worth it! Autoscales up and down > based on server load. Works with both AWS and rackspace cloud. > > -tim > > On Sun, May 13, 2012 at 9:28 PM, de <dustyeike@gmail.com> wrote: > > I have a specific need to use Puppet to Autoscale a few applications I > have > > deployed on EC2. I''m using Puppet now to manage and create instances and > saw > > a neat video on this subject. I''m missing the a piece of the puzzle. I''m > > assuming I should use CloudWatch to send my puppetmaster a notification > to > > then create a new instance with the cloudformation module. >To create autoscaling webfarms I use Scalr too. As Tim reported is a powerful cloud orchestration tool, available as an online service or as an opensource software (PHP/Python/JS) that you can install on your servers. I must confirm that there is no more free trial for the service and the pricing is based on the number of hosts you manage. > Does that sound about right? I looked into cloudformation, but that doesn''t> > seem right. >I haven''t tried it but I suppose that Cloudformation has some orchestration feature too: I think you can define an AutoScalingGroup and have CloudFormation starts a new host based on a predefined AMI when loadaverage of the hosts in the group reach some predefined level; you can choose a different metric if you prefer, like a webpage response time. Then I suppose you have to update your ELB configuration and probably something else ... All these tasks can be configured with a tool like Scalr through a web interface and this simplifies your life, but it''s something to pay for, while cloudformation is a free feature on AWS, for what I remember. Hope this helps, L. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Can you provide a link to the video? I''m guessing the workflow could be much simpler, in that you could have an autoscaling group, that spins up instances that are based on AMIs that are either preconfigured for puppet, or have the cloud-init info passed to them to get puppet up and running. -Brian On Sun, May 13, 2012 at 9:28 PM, de <dustyeike@gmail.com> wrote:> I have a specific need to use Puppet to Autoscale a few applications I > have deployed on EC2. I''m using Puppet now to manage and create instances > and saw a neat video on this subject. I''m missing the a piece of the > puzzle. I''m assuming I should use Cloudwatch to send my puppetmaster a > notification to then create a new instance with the cloudformation module. > > Does that sound about right? I looked into cloudformation, but that > doesn''t seem right. > > Can someone point me in the right direction. > > Thanks. > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/puppet-users/-/ps2yIUWdTMQJ. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. >-- <http://aws.amazon.com/solutions/solution-providers/brandorr/> -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Thanks everyone. Here''s the video I mentioned if you''re interested. He makes it look really simple. http://vimeo.com/27335693 To clarify, how can I tell Puppet(master) about this new instance in my autoscale group when it becomes available? I''d like to automate running puppet master to configure new nodes. Thanks. On Monday, May 14, 2012 8:04:28 AM UTC-4, Brian Gupta wrote:> > Can you provide a link to the video? I''m guessing the workflow could be > much simpler, in that you could have an autoscaling group, that spins up > instances that are based on AMIs that are either preconfigured for puppet, > or have the cloud-init info passed to them to get puppet up and running. > > -Brian > > On Sun, May 13, 2012 at 9:28 PM, de <dustyeike@gmail.com> wrote: > >> I have a specific need to use Puppet to Autoscale a few applications I >> have deployed on EC2. I''m using Puppet now to manage and create instances >> and saw a neat video on this subject. I''m missing the a piece of the >> puzzle. I''m assuming I should use Cloudwatch to send my puppetmaster a >> notification to then create a new instance with the cloudformation module. >> >> Does that sound about right? I looked into cloudformation, but that >> doesn''t seem right. >> >> Can someone point me in the right direction. >> >> Thanks. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Users" group. >> To view this discussion on the web visit >> https://groups.google.com/d/msg/puppet-users/-/ps2yIUWdTMQJ. >> To post to this group, send email to puppet-users@googlegroups.com. >> To unsubscribe from this group, send email to >> puppet-users+unsubscribe@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/puppet-users?hl=en. >> > > > > -- > <http://aws.amazon.com/solutions/solution-providers/brandorr/> > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/hrrQpsMYTCsJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Scalr looks okay. I''ll look into this. What I really meant was how does Puppet(master) detect a new instance is available in your autoscale group and deploy your modules. I guess that has more to do with notifications than autoscale, but thought I''d clarify. Thanks. On Monday, May 14, 2012 6:00:59 AM UTC-4, LoreLLo wrote:> > > >> http://scalr.net/ >> >> I think they did away with the free version of it recently. But if you >> ask me what they''re asking is well worth it! Autoscales up and down >> based on server load. Works with both AWS and rackspace cloud. >> >> -tim >> >> On Sun, May 13, 2012 at 9:28 PM, de <dustyeike@gmail.com> wrote: >> > I have a specific need to use Puppet to Autoscale a few applications I >> have >> > deployed on EC2. I''m using Puppet now to manage and create instances >> and saw >> > a neat video on this subject. I''m missing the a piece of the puzzle. I''m >> > assuming I should use CloudWatch to send my puppetmaster a notification >> to >> > then create a new instance with the cloudformation module. >> > > To create autoscaling webfarms I use Scalr too. As Tim reported is a > powerful cloud orchestration tool, available as an online service or as an > opensource software (PHP/Python/JS) that you can install on your servers. > I must confirm that there is no more free trial for the service and the > pricing is based on the number of hosts you manage. > > > Does that sound about right? I looked into cloudformation, but that > doesn''t > >> > seem right. >> > > I haven''t tried it but I suppose that Cloudformation has some > orchestration feature too: I think you can define an AutoScalingGroup > and have CloudFormation starts a new host based on a predefined AMI when > loadaverage of the hosts in the group reach some predefined level; you can > choose a different metric if you prefer, like a webpage response time. Then > I suppose you have to update your ELB configuration and probably something > else ... All these tasks can be configured with a tool like Scalr through a > web interface and this simplifies your life, but it''s something to pay for, > while cloudformation is a free feature on AWS, for what I remember. > > Hope this helps, > L. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/U41DUaMkB3sJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Generally puppet doesn''t "detect" new systems as such. The new systems tell the puppet server about themselves. The way this is handled is your base instance install would include puppet. So, as the system boots up puppet would be running. The puppet agent will contact the master. The master would have autosigning setup, so it will automatically exchange certs with the new system. Then the behavior would continue as defined in the puppet configuration. Steven Date: Mon, 14 May 2012 11:30:04 -0700 From: dustyeike@gmail.com To: puppet-users@googlegroups.com Subject: Re: [Puppet Users] Autoscaling with Puppet Scalr looks okay. I''ll look into this. What I really meant was how does Puppet(master) detect a new instance is available in your autoscale group and deploy your modules. I guess that has more to do with notifications than autoscale, but thought I''d clarify. Thanks. On Monday, May 14, 2012 6:00:59 AM UTC-4, LoreLLo wrote: http://scalr.net/ I think they did away with the free version of it recently. But if you ask me what they''re asking is well worth it! Autoscales up and down based on server load. Works with both AWS and rackspace cloud. -tim On Sun, May 13, 2012 at 9:28 PM, de <dustyeike@gmail.com> wrote:> I have a specific need to use Puppet to Autoscale a few applications I have> deployed on EC2. I''m using Puppet now to manage and create instances and saw> a neat video on this subject. I''m missing the a piece of the puzzle. I''m> assuming I should use CloudWatch to send my puppetmaster a notification to> then create a new instance with the cloudformation module.To create autoscaling webfarms I use Scalr too. As Tim reported is a powerful cloud orchestration tool, available as an online service or as an opensource software (PHP/Python/JS) that you can install on your servers. I must confirm that there is no more free trial for the service and the pricing is based on the number of hosts you manage. > Does that sound about right? I looked into cloudformation, but that doesn''t> seem right.I haven''t tried it but I suppose that Cloudformation has some orchestration feature too: I think you can define an AutoScalingGroup and have CloudFormation starts a new host based on a predefined AMI when loadaverage of the hosts in the group reach some predefined level; you can choose a different metric if you prefer, like a webpage response time. Then I suppose you have to update your ELB configuration and probably something else ... All these tasks can be configured with a tool like Scalr through a web interface and this simplifies your life, but it''s something to pay for, while cloudformation is a free feature on AWS, for what I remember. Hope this helps,L. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/U41DUaMkB3sJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
This might be what I was looking for. Havent'' yet looked into this. https://github.com/ccaum/puppet-autoami On Sunday, May 13, 2012 9:28:25 PM UTC-4, de wrote:> > I have a specific need to use Puppet to Autoscale a few applications I > have deployed on EC2. I''m using Puppet now to manage and create instances > and saw a neat video on this subject. I''m missing the a piece of the > puzzle. I''m assuming I should use Cloudwatch to send my puppetmaster a > notification to then create a new instance with the cloudformation module. > > Does that sound about right? I looked into cloudformation, but that > doesn''t seem right. > > Can someone point me in the right direction. > > Thanks. >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/NSZ1uqdjTeIJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
I''m in the same boat here. I have a puppet master server that provides config to a few different classes of machines. I''m struggling with a way to have instances from one autoscale group come up, and attach to a "webgroup-a" or "webgroup-b" puppet configuration. Is there a way for a puppet client to pass along some bit of information to the master, other than a hostname, to identify itself as having booted into a particular class, and thus to receive particular configuration? On Sunday, May 13, 2012 7:28:25 PM UTC-6, de wrote:> > I have a specific need to use Puppet to Autoscale a few applications I > have deployed on EC2. I''m using Puppet now to manage and create instances > and saw a neat video on this subject. I''m missing the a piece of the > puzzle. I''m assuming I should use Cloudwatch to send my puppetmaster a > notification to then create a new instance with the cloudformation module. > > Does that sound about right? I looked into cloudformation, but that > doesn''t seem right. > > Can someone point me in the right direction. > > Thanks. >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/xBJAK7PIOM4J. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Mon, Jun 4, 2012 at 1:01 PM, Matthew <matthew@sympoz.com> wrote:> I''m in the same boat here. I have a puppet master server that provides > config to a few different classes of machines. I''m struggling with a way to > have instances from one autoscale group come up, and attach to a > "webgroup-a" or "webgroup-b" puppet configuration. > > Is there a way for a puppet client to pass along some bit of information to > the master, other than a hostname, to identify itself as having booted into > a particular class, and thus to receive particular configuration?This sounds like a custom fact. If you''re doing this from a script a quick and dirty (but not ideal long term since it''s not clear where this fact came from) is to simply export an environment variable on the agent: export FACTER_FACT_AUTOSCALE_GROUP=webgroup-b puppet agent --test When the puppet agent connects to the master there will be a fact named fact_autoscale_group with the value of "webgroup-b". I''ve used this before to signal to the master how the agent should configure a new Puppet master worker behind a load balancer. -Jeff -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Thanks, that''s very helpful. What are other users doing with autoscale to define their nodes.pp? A wildcard match? Basing replies off a custom fact like this? I''m already looking at using a non-fqdn based certname via user-data... but that general part in autoscale is a bit obscured for me. Or, phrased differently, if Amazon is init''ing the instance creation request from one of my AMI''s, how do I get a config in place on the puppet side to match the host. I''m thinking that''s certname and a custom fact, but am open to other ideas. -Matthew On Mon, Jun 4, 2012 at 2:20 PM, Jeff McCune <jeff@puppetlabs.com> wrote:> On Mon, Jun 4, 2012 at 1:01 PM, Matthew <matthew@sympoz.com> wrote: > > I''m in the same boat here. I have a puppet master server that provides > > config to a few different classes of machines. I''m struggling with a way > to > > have instances from one autoscale group come up, and attach to a > > "webgroup-a" or "webgroup-b" puppet configuration. > > > > Is there a way for a puppet client to pass along some bit of information > to > > the master, other than a hostname, to identify itself as having booted > into > > a particular class, and thus to receive particular configuration? > > This sounds like a custom fact. > > If you''re doing this from a script a quick and dirty (but not ideal > long term since it''s not clear where this fact came from) is to simply > export an environment variable on the agent: > > export FACTER_FACT_AUTOSCALE_GROUP=webgroup-b > puppet agent --test > > When the puppet agent connects to the master there will be a fact > named fact_autoscale_group with the value of "webgroup-b". > > I''ve used this before to signal to the master how the agent should > configure a new Puppet master worker behind a load balancer. > > -Jeff > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- Matthew Boeckman Director - Technical Operationsp: 303.997.0345 x832 <#SafeHtmlFilter_> matthew@sympoz.comwww.sympoz.com | www.craftsy.comSympoz, Inc. 2150 W. 29th Ave., Suite 400 Denver‚ CO 80211 Toll Free: (888) 979.6769 <#SafeHtmlFilter_>[image: Sympoz.com]<http://www.sympoz.com>[image: Craftsy.com] <http://www.craftsy.com> -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
> What are other users doing with autoscale to define their nodes.pp? A wildcard match? Basing replies off a custom > fact like this? I''m already looking at using a non-fqdn based certname via user-data... but that general part in > autoscale is a bit obscured for me.I use the certname (=FQDN) to match the right configuration in nodes.pp. The right name is set at boot-time by the cloud orchestrator (Scalr). While Puppet knows only the configuration it gives to each role (webserver or mysql server or mysql slave...), the orchestrator knows the exact status of the farm (how many instances of each role). So if you have a farm composed by two autoscaling "roles", web & db, i change the hostname from ec2-xx-xx-xx-xx to something like: - myservice-web01 - myservice-web02 match myservice-web0[0-9] in site.pp - myservice-web03 - myservice-db-master match myservice-db-master in site.pp - myservice-db-slave01 - myservice-db-slave02 match myservice-db-slave0[0-9] in site.pp - myservice-db-slave03 A simple advantage of this approach is that I don''t need to build, pay and maintain a customized AMI for each role in each farm, I can simply start from a fresh OS at boot time. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Hello, On Tue, 2012-06-05 at 08:43 -0600, Matthew Boeckman wrote:> Thanks, that''s very helpful. > > > What are other users doing with autoscale to define their nodes.pp? A > wildcard match? Basing replies off a custom fact like this? I''m > already looking at using a non-fqdn based certname via user-data... > but that general part in autoscale is a bit obscured for me.When you create the autoscaling group, one of the things you do is create a launch config. In this launch config you can add scripts to the userdata that will then be run by the node on bootup (via cloud-init). This is very helpful for injecting custom data into the node for use when it connects to the puppetmaster.> Or, phrased differently, if Amazon is init''ing the instance creation > request from one of my AMI''s, how do I get a config in place on the > puppet side to match the host. I''m thinking that''s certname and a > custom fact, but am open to other ideas.Cheers, -- Stephen Gran Senior Systems Integrator - guardian.co.uk Please consider the environment before printing this email. ------------------------------------------------------------------ Visit guardian.co.uk - newspaper of the year www.guardian.co.uk www.observer.co.uk www.guardiannews.com On your mobile, visit m.guardian.co.uk or download the Guardian iPhone app www.guardian.co.uk/iphone To save up to 30% when you subscribe to the Guardian and the Observer visit www.guardian.co.uk/subscriber --------------------------------------------------------------------- This e-mail and all attachments are confidential and may also be privileged. If you are not the named recipient, please notify the sender and delete the e-mail and all attachments immediately. Do not disclose the contents to another person. You may not use the information for any purpose, or store, or copy, it in any way. Guardian News & Media Limited is not liable for any computer viruses or other material transmitted with or as part of this e-mail. You should employ virus checking software. Guardian News & Media Limited A member of Guardian Media Group plc Registered Office PO Box 68164 Kings Place 90 York Way London N1P 2AP Registered in England Number 908396 -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.