We''re using Puppet as part of a broader toolchain that relies on delivering software for deployment using sets of Yum-based RPM packages. We''ve setup system, role and application specific Yum repositories on an environment-by-environment basis that ensure that the required set of RPM versions flow appropriately (e.g. from development to QA to staging and hence to production). In this spirit we''re packaging our Puppet modules as sets of RPMs too so the correct versions of the system, role and application specific modules flow along with everything else. The problem arises when you consider the conflict that arises between the "natural" use of Yum-based RPM installation and the Puppet master''s module delivery mechanisms. Puppet allows "modulepath" to be set on an environment-by-environment basis, of course, thus supporting delivering different versions of modules from a master managing several environments. The restriction lies with Yum/RPM''s inability to allow multiple versions of the same (relocatable) package to be installed on the same system (even good old System V packages could do that!). I''m looking for workarounds that aren''t too egregious to either system! Here are the ideas we''ve come up with so far: Hack the RPM package names to include a version discriminator (e.g. "packageV1-1.0-noarch.rpm" rather than "package-1.0-noarch.rpm") to allow them all to be installed on Puppet master Use Yum/RPM to install the modules directly on the client systems and find a way to restrict the Puppet master to managing the manifests rather than attempting to install the modules too. Is the second method workable? It seems to be a blend between agent and apply modes. We don''t want to use apply mode since we really value using the master (even supplemented with Hiera) to act as the resource model provider to deliver configuration attributes to the agents as well as act as the node provider for Rundeck (used for distributed orchestration) using the Puppet/Rundeck plug-in (which doesn''t seem to be environment aware - but that''s another story!). We''d appreciate any comments and feedback on this. Thanks, Anthony. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Sorry for the top posting. Imnsho, rpm had always permitted to have multiple package version if they not conflict, in fact the usual case is the kernel. Anyway your question is most rpm related: so if you like i suggest you to ask to a rpm mailing list. Best regards 2012/5/27, Anthony Shortland <anthony@dtosolutions.com>:> We''re using Puppet as part of a broader toolchain that relies on delivering > software for deployment using sets of Yum-based RPM packages. We''ve setup > system, role and application specific Yum repositories on an > environment-by-environment basis that ensure that the required set of RPM > versions flow appropriately (e.g. from development to QA to staging and > hence to production). > > In this spirit we''re packaging our Puppet modules as sets of RPMs too so the > correct versions of the system, role and application specific modules flow > along with everything else. > > The problem arises when you consider the conflict that arises between the > "natural" use of Yum-based RPM installation and the Puppet master''s module > delivery mechanisms. > > Puppet allows "modulepath" to be set on an environment-by-environment basis, > of course, thus supporting delivering different versions of modules from a > master managing several environments. > > The restriction lies with Yum/RPM''s inability to allow multiple versions of > the same (relocatable) package to be installed on the same system (even good > old System V packages could do that!). > > I''m looking for workarounds that aren''t too egregious to either system! > > Here are the ideas we''ve come up with so far: > > Hack the RPM package names to include a version discriminator (e.g. > "packageV1-1.0-noarch.rpm" rather than "package-1.0-noarch.rpm") to allow > them all to be installed on Puppet master > Use Yum/RPM to install the modules directly on the client systems and find a > way to restrict the Puppet master to managing the manifests rather than > attempting to install the modules too. > > Is the second method workable? It seems to be a blend between agent and > apply modes. > > We don''t want to use apply mode since we really value using the master (even > supplemented with Hiera) to act as the resource model provider to deliver > configuration attributes to the agents as well as act as the node provider > for Rundeck (used for distributed orchestration) using the Puppet/Rundeck > plug-in (which doesn''t seem to be environment aware - but that''s another > story!). > > We''d appreciate any comments and feedback on this. > > Thanks, > > Anthony. > > > > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- Inviato dal mio dispositivo mobile -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Hi, [snip]> > - Hack the RPM package names to include a version discriminator (e.g. > "packageV1-1.0-noarch.rpm" rather than "package-1.0-noarch.rpm") to allow > them all to be installed on Puppet maste > > This is a practice that is already used for the kernel, so it seems like alegitimate way of working. Some time ago, I faced the exact same problem you are describing. One possible solution I devised was: * Use the version of the module in the packagename (as you already considered) to make multiple versions on one system possible, and install all modules in the same basedir, using the version as part of the path, like /etc/puppet/modules/<modname>/<version>/ * Create some script that mimics the functionality of the debian apache a2enmod, a2dismod scripts (they create symlinks for apache modules) that creates a symlink of the version you want in the modulepath of the environment you want (eg. /etc/puppet/modules/prd/users -> /etc/puppet/modules/users/0.1.2) You could also create a simple puppet class instead of the script, to manage these symlinks. If you use this with hiera, you get a nice yaml file that contains the deployed versions of your modules per environment. In the end I turned to a different solution (create a puppetmaster for each environment) because of a few other reasons. But I think the above should work without lots of problems. Kind regards, k -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Anthony Shortland
2012-May-28 23:32 UTC
Re: [Puppet Users] Managing Puppet modules as RPMs
Very interesting that you should say that you created a Puppet master for each environment ... it''s the obvious way to accommodate the limits of RPM packaging while retaining the ability to use Puppet in full agent mode. Is this considered a best practice? What other considerations drive the decision to have a Puppet master per environment? Anthony. On May 27, 2012, at 5:03 PM, Kristof Willaert wrote:> Hi, > > [snip] > Hack the RPM package names to include a version discriminator (e.g. "packageV1-1.0-noarch.rpm" rather than "package-1.0-noarch.rpm") to allow them all to be installed on Puppet maste > This is a practice that is already used for the kernel, so it seems like a legitimate way of working. > > Some time ago, I faced the exact same problem you are describing. One possible solution I devised was: > > * Use the version of the module in the packagename (as you already considered) to make multiple versions on one system possible, and install all modules in the > same basedir, using the version as part of the path, like /etc/puppet/modules/<modname>/<version>/ > > * Create some script that mimics the functionality of the debian apache a2enmod, a2dismod scripts (they create symlinks for apache modules) that creates a symlink > of the version you want in the modulepath of the environment you want (eg. /etc/puppet/modules/prd/users -> /etc/puppet/modules/users/0.1.2) > > You could also create a simple puppet class instead of the script, to manage these symlinks. If you use this with hiera, you get a nice yaml file that contains the > deployed versions of your modules per environment. > > In the end I turned to a different solution (create a puppetmaster for each environment) because of a few other reasons. But I think the above should work without lots of problems. > > Kind regards, > > k > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
inline On Mon, May 28, 2012 at 04:32:03PM -0700, Anthony Shortland wrote:> Very interesting that you should say that you created a Puppet master for > each environment ... it''s the obvious way to accommodate the limits of RPM > packaging while retaining the ability to use Puppet in full agent mode. > Is this considered a best practice? What other considerations drive the > decision to have a Puppet master per environment?Being able to test puppetmaster upgrades with all supported builds without affecting production. Being able to leave my lab puppetmaster and some agents for a weekend with an angrier chaos monkey chewing on them, and not worry that it''s affecting production. Etc., etc., doing things without touching production.> Anthony. > On May 27, 2012, at 5:03 PM, Kristof Willaert wrote: > > Hi, > > [snip] > > * Hack the RPM package names to include a version discriminator > (e.g. "packageV1-1.0-noarch.rpm" rather than > "package-1.0-noarch.rpm") to allow them all to be installed on > Puppet maste > > This is a practice that is already used for the kernel, so it seems like > a legitimate way of working. > > Some time ago, I faced the exact same problem you are describing. One > possible solution I devised was: > > * Use the version of the module in the packagename (as you already > considered) to make multiple versions on one system possible, and > install all modules in the > same basedir, using the version as part of the path, like > /etc/puppet/modules/<modname>/<version>/ > > * Create some script that mimics the functionality of the debian apache > a2enmod, a2dismod scripts (they create symlinks for apache modules) that > creates a symlink > of the version you want in the modulepath of the environment you want > (eg. /etc/puppet/modules/prd/users -> /etc/puppet/modules/users/0.1.2) > > You could also create a simple puppet class instead of the script, to > manage these symlinks. If you use this with hiera, you get a nice yaml > file that contains the > deployed versions of your modules per environment. > > In the end I turned to a different solution (create a puppetmaster for > each environment) because of a few other reasons. But I think the above > should work without lots of problems. > > Kind regards, > > k > -- > You received this message because you are subscribed to the Google > Groups "Puppet Users" group. > To post to this group, send email to [1]puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > [2]puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > [3]http://groups.google.com/group/puppet-users?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > > References > > Visible links > 1. mailto:puppet-users@googlegroups.com > 2. mailto:puppet-users+unsubscribe@googlegroups.com > 3. http://groups.google.com/group/puppet-users?hl=en-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Sun, May 27, 2012 at 9:10 AM, devzero2000 <pinto.elia@gmail.com> wrote:> Sorry for the top posting. > > Imnsho, rpm had always permitted to have multiple package version if > they not conflict, in fact the usual case is the kernel. Anyway your > question is most rpm related: so if you like i suggest you to ask to a > rpm mailing list. > > as far as i know, kernel is a very specific usage case, and while rpm doessupport it, yum does not (kernel multi version in yum is hard coded). Ohad> > Best regards > > 2012/5/27, Anthony Shortland <anthony@dtosolutions.com>: > > We''re using Puppet as part of a broader toolchain that relies on > delivering > > software for deployment using sets of Yum-based RPM packages. We''ve setup > > system, role and application specific Yum repositories on an > > environment-by-environment basis that ensure that the required set of RPM > > versions flow appropriately (e.g. from development to QA to staging and > > hence to production). > > > > In this spirit we''re packaging our Puppet modules as sets of RPMs too so > the > > correct versions of the system, role and application specific modules > flow > > along with everything else. > > > > The problem arises when you consider the conflict that arises between the > > "natural" use of Yum-based RPM installation and the Puppet master''s > module > > delivery mechanisms. > > > > Puppet allows "modulepath" to be set on an environment-by-environment > basis, > > of course, thus supporting delivering different versions of modules from > a > > master managing several environments. > > > > The restriction lies with Yum/RPM''s inability to allow multiple versions > of > > the same (relocatable) package to be installed on the same system (even > good > > old System V packages could do that!). > > > > I''m looking for workarounds that aren''t too egregious to either system! > > > > Here are the ideas we''ve come up with so far: > > > > Hack the RPM package names to include a version discriminator (e.g. > > "packageV1-1.0-noarch.rpm" rather than "package-1.0-noarch.rpm") to allow > > them all to be installed on Puppet master > > Use Yum/RPM to install the modules directly on the client systems and > find a > > way to restrict the Puppet master to managing the manifests rather than > > attempting to install the modules too. > > > > Is the second method workable? It seems to be a blend between agent and > > apply modes. > > > > We don''t want to use apply mode since we really value using the master > (even > > supplemented with Hiera) to act as the resource model provider to deliver > > configuration attributes to the agents as well as act as the node > provider > > for Rundeck (used for distributed orchestration) using the Puppet/Rundeck > > plug-in (which doesn''t seem to be environment aware - but that''s another > > story!). > > > > We''d appreciate any comments and feedback on this. > > > > Thanks, > > > > Anthony. > > > > > > > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Puppet Users" group. > > To post to this group, send email to puppet-users@googlegroups.com. > > To unsubscribe from this group, send email to > > puppet-users+unsubscribe@googlegroups.com. > > For more options, visit this group at > > http://groups.google.com/group/puppet-users?hl=en. > > > > > > -- > Inviato dal mio dispositivo mobile > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Hi, [snip] Very interesting that you should say that you created a Puppet master for> each environment ... it''s the obvious way to accommodate the limits of RPM > packaging while retaining the ability to use Puppet in full agent mode. > > Is this considered a best practice? What other considerations drive the > decision to have a Puppet master per environment? >We are a hosting/service provider company and have a number of customers that have completely isolated setups. So a single puppetmaster was never really an option. We put some effort in the easy and automated creation of puppetmasters and module distribution (hence the RPMs :)), to be able to use Puppet agent mode. From there on, the decision to split them per environment (only for large customers ofcourse), was trivial. And it indeed allows us to test upgrades of our masters, while protecting us from "development" code accidently ending up on production machines. Don''t know if it''s a best practice. There is the overhead of a number of puppetmasters, and some loss of centralizatiion (for reports, ...), but that is something we knew from the start. Kind regards, k -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On May 26, 8:56 pm, Anthony Shortland <anth...@dtosolutions.com> wrote:> We''re using Puppet as part of a broader toolchain that relies on delivering software for deployment using sets of Yum-based RPM packages. We''ve setup system, role and application specific Yum repositories on an environment-by-environment basis that ensure that the required set of RPM versions flow appropriately (e.g. from development to QA to staging and hence to production). > > In this spirit we''re packaging our Puppet modules as sets of RPMs too so the correct versions of the system, role and application specific modules flow along with everything else.+1> Hack the RPM package names to include a version discriminator (e.g. "packageV1-1.0-noarch.rpm" rather than "package-1.0-noarch.rpm") to allow them all to be installed on Puppet masterI don''t think that''s too bad. Indeed, yum-based distros do exactly that for so-called "compatibility" libraries.> Use Yum/RPM to install the modules directly on the client systems and find a way to restrict the Puppet master to managing the manifests rather than attempting to install the modules too.Turning off pluginsync would probably be all you need to make that work configuration-wise. I''d be a bit nervous about keeping all the plugins synchronized though. I can imagine schemes to make it work, but you need more care and admin effort that way. Or there''s at least one other option: cut out Yum for installing these particular RPMs on the master. Since they''re targeted only for the master, delivering them via Yum provides fewer advantages than many other uses of Yum. The underlying RPM system should not object as long as the packages don''t conflict. I think this is more of a hack than your first option, however, only at a higher level. I think there is some confusion here surrounding the role and purpose of these RPMs. If the primary purpose of each one is to provide a puppetmaster the ability to support a particular system, role, or application (or combination of the above), then what you''re characterizing as different versions of the same RPM in fact provide separate and distinct capabilities. As such they *should* have different names. Contrast that with the situation where the primary purpose of the RPMs is to provide Puppet modules to client systems. In that case, the different "versions" provide alternative, conflicting versions of logically the same features. In that case they should have the same name, and it follows naturally that yum objects to installing more than one. All things considered, I think your option 1 is right, good, and proper, not a hack at all. John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.