Hi,
For the past week, I have been trying to figure out the best way to do
package management with Puppet on an Ubuntu system. I have studied many
solutons I found on the web, but none of them seem to do exactly what I want.
Among other things, I have tried everything that is discussed here:
http://groups.google.com/group/puppet-users/browse_thread/thread/af7349171a5821a4/db96b809c97f6006
but none of that really works.
What I would like, is for Puppet to run `aptitude update` before it
installs or upgrades packages. That doesn''t seem out of the ordinary to
me, but I can''t make it work in Puppet.
The approach I like best, is this one:
class apt-update {
exec { "/usr/bin/aptitude update":
refreshonly => true,
}
}
define apt_package($ensure="latest") {
include apt-update
package { $name:
ensure => $ensure,
require => Class["apt-update"],
}
}
apt_package { "foo":
}
but that doesn''t work, it doesn''t run apt-update before
installing ''foo'',
because of the "refreshonly => true". If I remove it, it works, but
then
`aptitude update` is run on /every/ Puppet run, which is worse than not
running it at all.
Is it at all possible to achieve what I want with Puppet? If so, how?
I have tried to fiddle around with the new run stages feature, but that
got me in a circular dependency mess I couldn''t get out of.
Desperately hoping for hints...
Best regards,
Martijn.
Actually nothing to do with the content of this post but is anyone else getting a lot of mails from puppet-users at least twice? This particular one came through 3 times with the same timestamp and same Message-Id. On 31/08/2010 15:03, Martijn Grendelman wrote:> Hi, > > For the past week, I have been trying to figure out the best way to do > package management with Puppet on an Ubuntu system. I have studied many > solutons I found on the web, but none of them seem to do exactly what I want. > > Among other things, I have tried everything that is discussed here: > http://groups.google.com/group/puppet-users/browse_thread/thread/af7349171a5821a4/db96b809c97f6006 > > but none of that really works. > > What I would like, is for Puppet to run `aptitude update` before it > installs or upgrades packages. That doesn''t seem out of the ordinary to > me, but I can''t make it work in Puppet. > > The approach I like best, is this one: > > class apt-update { > exec { "/usr/bin/aptitude update": > refreshonly => true, > } > } > > define apt_package($ensure="latest") { > include apt-update > package { $name: > ensure => $ensure, > require => Class["apt-update"], > } > } > > apt_package { "foo": > } > > but that doesn''t work, it doesn''t run apt-update before installing ''foo'', > because of the "refreshonly => true". If I remove it, it works, but then > `aptitude update` is run on /every/ Puppet run, which is worse than not > running it at all. > > Is it at all possible to achieve what I want with Puppet? If so, how? > > I have tried to fiddle around with the new run stages feature, but that > got me in a circular dependency mess I couldn''t get out of. > > Desperately hoping for hints... > > Best regards, > Martijn. >-- Trevor Hemsley Infrastructure Engineer ................................................. * C A L Y P S O * Brighton, UK OFFICE +44 (0) 1273 666 350 FAX +44 (0) 1273 666 351 ................................................. www.calypso.com This electronic-mail might contain confidential information intended only for the use by the entity named. If the reader of this message is not the intended recipient, the reader is hereby notified that any dissemination, distribution or copying is strictly prohibited. * P * /*/ Please consider the environment before printing this e-mail /*/ -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Tue, Aug 31, 2010 at 10:03 AM, Martijn Grendelman <martijn@grendelman.net> wrote:> > What I would like, is for Puppet to run `aptitude update` before it > installs or upgrades packages. That doesn''t seem out of the ordinary to > me, but I can''t make it work in Puppet.The run stages of puppet 2.6 addresses this need. I recommend you look into adding a class to a stage which runs before the main stage. The class would contain an exec resource to run aptitude update periodically. Hope this helps, -- Jeff McCune http://www.puppetlabs.com/ -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Tue, Aug 31, 2010 at 7:03 AM, Martijn Grendelman <martijn@grendelman.net> wrote:> Hi, > > For the past week, I have been trying to figure out the best way to do > package management with Puppet on an Ubuntu system. I have studied many > solutons I found on the web, but none of them seem to do exactly what I want. > > Among other things, I have tried everything that is discussed here: > http://groups.google.com/group/puppet-users/browse_thread/thread/af7349171a5821a4/db96b809c97f6006 > > but none of that really works. > > What I would like, is for Puppet to run `aptitude update` before it > installs or upgrades packages. That doesn''t seem out of the ordinary to > me, but I can''t make it work in Puppet. > > The approach I like best, is this one: > > class apt-update { > exec { "/usr/bin/aptitude update": > refreshonly => true, > } > } > > define apt_package($ensure="latest") { > include apt-update > package { $name: > ensure => $ensure, > require => Class["apt-update"], > } > } > > apt_package { "foo": > } > > but that doesn''t work, it doesn''t run apt-update before installing ''foo'', > because of the "refreshonly => true". If I remove it, it works, but then > `aptitude update` is run on /every/ Puppet run, which is worse than not > running it at all. > > Is it at all possible to achieve what I want with Puppet? If so, how?Why do you want to run this before every package install? That will add a fair bit of overhead. I don''t quite understand the desire to run before every package install, and yet not simply have it run once before all package installs. We''ve used two approaches, both using our package::apt::update class which runs apt-get update/upgrade/dpkg --configure -a etc. 1) Resource defaults: In the global scope, set a resource default for package to require the package::apt::update class The downside is that if you wish to set an additional require for a specific package, you need to remember to include the global default as well. 2) Defined types: Define a type much like yours that has the same require on the package resource definition inside the defined type. The downside is that you are no longer just using ''package'' resources, which makes the use of other people''s modules difficult. The upside is that we have a similar defined type ''apt_repo'' that sets up repositories. With appropriate use of require in apt_package and before in apt_repo, you know your repositories are being laid down before apt-get update etc get run, and that the packages are installed afterwards. Run stages make this a bit simpler, but you can achieve the same functionality without them.> > I have tried to fiddle around with the new run stages feature, but that > got me in a circular dependency mess I couldn''t get out of. > > Desperately hoping for hints... > > Best regards, > Martijn. > >-- nigel -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Hi Nigel Kersten, Thank you for your elaborate answer. I am not sure how to proceed, though...>> What I would like, is for Puppet to run `aptitude update` before it >> installs or upgrades packages. That doesn''t seem out of the ordinary to >> me, but I can''t make it work in Puppet. >> >> The approach I like best, is this one: >> >> class apt-update { >> exec { "/usr/bin/aptitude update": >> refreshonly => true, >> } >> } >> >> define apt_package($ensure="latest") { >> include apt-update >> package { $name: >> ensure => $ensure, >> require => Class["apt-update"], >> } >> } >> >> apt_package { "foo": >> } >> >> but that doesn''t work, it doesn''t run apt-update before installing ''foo'', >> because of the "refreshonly => true". If I remove it, it works, but then >> `aptitude update` is run on /every/ Puppet run, which is worse than not >> running it at all. >> >> Is it at all possible to achieve what I want with Puppet? If so, how? > > Why do you want to run this before every package install? That will > add a fair bit of overhead. I don''t quite understand the desire to run > before every package install, and yet not simply have it run once > before all package installs.I am not sure how I said I want to do an update before /every/ package install. Once at the start of a Puppet-run, /IF/ one or more packages need to be installed, would suffice.> We''ve used two approaches, both using our package::apt::update class > which runs apt-get update/upgrade/dpkg --configure -a etc.I think I read that in the posting that I referenced.> 1) Resource defaults: > > In the global scope, set a resource default for package to require the > package::apt::update class > The downside is that if you wish to set an additional require for a > specific package, you need to remember to include the global default > as well. > > 2) Defined types: > > Define a type much like yours that has the same require on the package > resource definition inside the defined type.Isn''t that what I decribed above? The problem is, that by including the class in the define, it runs everytime, unless ''refreshonly => true'' is set, in which case it never runs. If I don''t include the class in the define, Puppet can''t find the class when it evaluates the ''require''. How does that work for you? Would it be possible for you to share (relevant parts of) your configuration as-is?> The downside is that you are no longer just using ''package'' resources, > which makes the use of other people''s modules difficult. > The upside is that we have a similar defined type ''apt_repo'' that sets > up repositories. With appropriate use of require in apt_package and > before in apt_repo, you know your repositories are being laid down > before apt-get update etc get run, and that the packages are installed > afterwards. > > Run stages make this a bit simpler, but you can achieve the same > functionality without them.I have been looking for examples using run stages, but they are scarce. The examples in the release notes are too simple to go by.>> I have tried to fiddle around with the new run stages feature, but that >> got me in a circular dependency mess I couldn''t get out of. >>Best regards, Martijn.
On Tue, Aug 31, 2010 at 1:02 PM, Martijn Grendelman <martijn@grendelman.net> wrote:> Hi Nigel Kersten, > > Thank you for your elaborate answer. I am not sure how to proceed, though... > >>> What I would like, is for Puppet to run `aptitude update` before it >>> installs or upgrades packages. That doesn''t seem out of the ordinary to >>> me, but I can''t make it work in Puppet. >>> >>> The approach I like best, is this one: >>> >>> class apt-update { >>> exec { "/usr/bin/aptitude update": >>> refreshonly => true, >>> } >>> } >>> >>> define apt_package($ensure="latest") { >>> include apt-update >>> package { $name: >>> ensure => $ensure, >>> require => Class["apt-update"], >>> } >>> } >>> >>> apt_package { "foo": >>> } >>> >>> but that doesn''t work, it doesn''t run apt-update before installing ''foo'', >>> because of the "refreshonly => true". If I remove it, it works, but then >>> `aptitude update` is run on /every/ Puppet run, which is worse than not >>> running it at all. >>> >>> Is it at all possible to achieve what I want with Puppet? If so, how? >> >> Why do you want to run this before every package install? That will >> add a fair bit of overhead. I don''t quite understand the desire to run >> before every package install, and yet not simply have it run once >> before all package installs. > > I am not sure how I said I want to do an update before /every/ package > install. Once at the start of a Puppet-run, /IF/ one or more packages > need to be installed, would suffice.Ah. I''m sorry, I must have misread your post.> >> We''ve used two approaches, both using our package::apt::update class >> which runs apt-get update/upgrade/dpkg --configure -a etc. > > I think I read that in the posting that I referenced. > >> 1) Resource defaults: >> >> In the global scope, set a resource default for package to require the >> package::apt::update class >> The downside is that if you wish to set an additional require for a >> specific package, you need to remember to include the global default >> as well. >> >> 2) Defined types: >> >> Define a type much like yours that has the same require on the package >> resource definition inside the defined type. > > Isn''t that what I decribed above? The problem is, that by including the > class in the define, it runs everytime, unless ''refreshonly => true'' is > set, in which case it never runs. If I don''t include the class in the > define, Puppet can''t find the class when it evaluates the ''require''.So I don''t have it refreshonly. I''m perfectly happy with apt-get update etc running on every puppet run. Are you using modules and environments? You will need to make sure the class is included somewhere, but inside the define isn''t the logical spot in my opinion.> > How does that work for you? Would it be possible for you to share > (relevant parts of) your configuration as-is?It really isn''t much more complicated than that :) but I''ll try and post something.> >> The downside is that you are no longer just using ''package'' resources, >> which makes the use of other people''s modules difficult. >> The upside is that we have a similar defined type ''apt_repo'' that sets >> up repositories. With appropriate use of require in apt_package and >> before in apt_repo, you know your repositories are being laid down >> before apt-get update etc get run, and that the packages are installed >> afterwards. >> >> Run stages make this a bit simpler, but you can achieve the same >> functionality without them. > > I have been looking for examples using run stages, but they are scarce. > The examples in the release notes are too simple to go by. > >>> I have tried to fiddle around with the new run stages feature, but that >>> got me in a circular dependency mess I couldn''t get out of. >>> > > Best regards, > Martijn. >-- nigel -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
* Martijn Grendelman <martijn at grendelman.net> [2010/08/31 22:02]:> What I would like, is for Puppet to run `aptitude update` before > it installs or upgrades packages. That doesn''t seem out of the > ordinary to me, but I can''t make it work in Puppet.I would do the `aptitude update` in cron, outside of puppet, and manage the cronjob via the cron type. That would eliminate your precedence and ordering problems completely. -- There are only two hard things in Computer Science: cache invalidation and naming things. -- Phil Karlton -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Tue, Aug 31, 2010 at 1:41 PM, Darren Chamberlain <darren@boston.com> wrote:> * Martijn Grendelman <martijn at grendelman.net> [2010/08/31 22:02]: >> What I would like, is for Puppet to run `aptitude update` before >> it installs or upgrades packages. That doesn''t seem out of the >> ordinary to me, but I can''t make it work in Puppet. > > I would do the `aptitude update` in cron, outside of puppet, and > manage the cronjob via the cron type. That would eliminate your > precedence and ordering problems completely.Not if you wish to use puppet to manage your repositories as well, and we do a lot of that. We do this in our wrapper script * apt-get update * apt-get upgrade our-config-tools-metapackage * run puppetd That way we can push out a new version of puppet and associated support tools out of band of the actual puppet run, and then puppet runs can continue that add additional repositories and install packages. This also gives us an out of band way of fixing issues with broken puppet clients by adding a package to our-config-tools-metapackage and performing the fixes in the package scripts.> > -- > There are only two hard things in Computer Science: cache invalidation > and naming things. > -- Phil Karlton > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. > >-- nigel -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Hi, Excerpts from Martijn Grendelman''s message of Tue Aug 31 16:02:43 -0400 2010:> Hi Nigel Kersten, > > >> What I would like, is for Puppet to run `aptitude update` before it > >> installs or upgrades packages. That doesn''t seem out of the ordinary to > >> me, but I can''t make it work in Puppet. > >> > >> The approach I like best, is this one: > >> > >> class apt-update { > >> exec { "/usr/bin/aptitude update": > >> refreshonly => true, > >> } > >> } > >> > >> define apt_package($ensure="latest") { > >> include apt-update > >> package { $name: > >> ensure => $ensure, > >> require => Class["apt-update"], > >> } > >> } > >> > >> apt_package { "foo": > >> } > >> > >> but that doesn''t work, it doesn''t run apt-update before installing ''foo'', > >> because of the "refreshonly => true". If I remove it, it works, but then > >> `aptitude update` is run on /every/ Puppet run, which is worse than not > >> running it at all. > >> > >> Is it at all possible to achieve what I want with Puppet? If so, how? > > > > Why do you want to run this before every package install? That will > > add a fair bit of overhead. I don''t quite understand the desire to run > > before every package install, and yet not simply have it run once > > before all package installs. > > I am not sure how I said I want to do an update before /every/ package > install. Once at the start of a Puppet-run, /IF/ one or more packages > need to be installed, would suffice. >You may run into a chicken-egg problem. Packages that are configured as "ensure => latest" can only be upgraded by puppet if the local apt cache files are up-to-date. IOW puppet relies on the local apt files to figure out if packages need to be updated. And to get new local apt files apt-get update needs to be run. -- Mathias Gug Ubuntu Developer http://www.ubuntu.com -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Nigel Kersten schreef:> So I don''t have it refreshonly. I''m perfectly happy with apt-get > update etc running on every puppet run.Ah, that explains ;-) But I was hoping that wouldn''t be necessary.> Are you using modules and environments? You will need to make sure the > class is included somewhere, but inside the define isn''t the logical > spot in my opinion.Well, that''s a big shortcoming in my understanding of how Puppet works, namespaces and stuff like that. But I''m learning ;-)>> How does that work for you? Would it be possible for you to share >> (relevant parts of) your configuration as-is? > > It really isn''t much more complicated than that :) but I''ll try and > post something.If this is it, then I guess I''m OK, thanks! Best regards, Martijn.
Mathias Gug schreef:>> I am not sure how I said I want to do an update before /every/ package >> install. Once at the start of a Puppet-run, /IF/ one or more packages >> need to be installed, would suffice. > > You may run into a chicken-egg problem. Packages that are configured as > "ensure => latest" can only be upgraded by puppet if the local apt cache > files are up-to-date. > > IOW puppet relies on the local apt files to figure out if packages need > to be updated. And to get new local apt files apt-get update needs to be > run. >I am aware of that, but that''s not a problem. I could easily run an update every day outside of Puppet (on Debian, APT installs a cronjob that can take care of this by default, it just needs to be configured). That way, Puppet would pick up the new versions sooner or later. But what I am trying to achieve, has a different background. Our development team regularly releases new software, packaged in a Debian package with a unique name. The idea is that we just give Puppet a new package {foo: } and have these new packages be installed ASAP. We even use triggered runs to speed things up and to make sure that several servers remain identical. But for this to work, the Puppet run needs to run aptitude update to pick up the new package name. Running the update periodically isn''t enough, but running an update on every catalog run is just overkill. Hope this explains my motives, and maybe someone has any more tips! Best regards, Martijn.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 08/31/2010 06:41 PM, Trevor Hemsley wrote:> Actually nothing to do with the content of this post but is anyone else > getting a lot of mails from puppet-users at least twice? This particular > one came through 3 times with the same timestamp and same Message-Id./me too. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkx+RL8ACgkQbwltcAfKi3+YHgCfTZZnTNBNTb/IRxkSeoKR3OW5 cSMAn2cbYt6D290CQv08KObEe0VujALP =tMxT -----END PGP SIGNATURE----- -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Tue, Aug 31, 2010 at 04:03:35PM +0200, Martijn Grendelman wrote:> What I would like, is for Puppet to run `aptitude update` before it > installs or upgrades packages.FYI you''re likely to encounter problems when using aptitude provider for package management. See Debian bugs #445034 #445035 regards, -- Marcin Owsiany <marcin@owsiany.pl> http://marcin.owsiany.pl/ GnuPG: 1024D/60F41216 FE67 DA2D 0ACA FC5E 3F75 D6F6 3A0D 8AA0 60F4 1216 "Every program in development at MIT expands until it can read mail." -- Unknown -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 05-09-10 12:02, Marcin Owsiany wrote:> On Tue, Aug 31, 2010 at 04:03:35PM +0200, Martijn Grendelman wrote: >> What I would like, is for Puppet to run `aptitude update` before it >> installs or upgrades packages. > > FYI you''re likely to encounter problems when using aptitude provider for > package management. See Debian bugs #445034 #445035Thanks. The former is not really important to me, since I never use aptitude that way. The latter is probably a little more tricky, but at least the ''unknown package'' case is handled by Puppet itself: # Yay, stupid aptitude doesn''t throw an error when the package is missing. if args.include?(:install) and output =~ /Couldn''t find any package/ raise Puppet::Error.new( "Could not find package #{self.name}" ) end I have now solved my problem by using a custom provider for packages, that is based on the aptitude provider, but simply adds this: # See if we need to run aptitude update before installing this package if not File.exists?("/tmp/puppet-iphionapt") FileUtils.touch("/tmp/puppet-iphionapt") aptitude("update") end I just haven''t found a suitable way to clean up the tempfile from within Puppet without touching any of Puppet''s own files, so I use a cronjob for that now. Maybe I''ll switch to a ''postrun_command'', or does anyone have a better idea? Best regards, Martijn.
On Tue, Aug 31, 2010 at 3:35 PM, Martijn Grendelman <martijn@grendelman.net> wrote:> But for this to work, the Puppet run needs to run aptitude update to > pick up the new package name. Running the update periodically isn''t > enough, but running an update on every catalog run is just overkill.I understand your concern here, but have you done the timing tests? How long are your average puppet runs and how much time does an "apt-get update" take? I just wonder if you''re prematurely optimizing your runs. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 01.09.2010 01:35, Martijn Grendelman wrote:> Mathias Gug schreef: >>> I am not sure how I said I want to do an update before /every/ package >>> install. Once at the start of a Puppet-run, /IF/ one or more packages >>> need to be installed, would suffice. >> You may run into a chicken-egg problem. Packages that are configured as >> "ensure => latest" can only be upgraded by puppet if the local apt cache >> files are up-to-date. >> >> IOW puppet relies on the local apt files to figure out if packages need >> to be updated. And to get new local apt files apt-get update needs to be >> run. >> > I am aware of that, but that''s not a problem. I could easily run an > update every day outside of Puppet (on Debian, APT installs a cronjob > that can take care of this by default, it just needs to be configured). > > That way, Puppet would pick up the new versions sooner or later. But > what I am trying to achieve, has a different background. > > Our development team regularly releases new software, packaged in a > Debian package with a unique name. The idea is that we just give Puppet > a new package {foo: } and have these new packages be installed ASAP. We > even use triggered runs to speed things up and to make sure that several > servers remain identical. > > But for this to work, the Puppet run needs to run aptitude update to > pick up the new package name. Running the update periodically isn''t > enough, but running an update on every catalog run is just overkill. > > Hope this explains my motives, and maybe someone has any more tips! > > Best regards, > Martijn.As it was said earlier if you have a ensure => latest you should probably have (also) a cron job for updates. For your case you may try a define like (using http://forge.puppetlabs.com/ripienaar/concat ; read the comments in the pp files) define fastupdate_package( $reason, $date ) { concat::fragment {"${name}_install": target => "/var/log/fastupdate_package.log", order => $date, content => "${date} Installed package ${name} (${reason})", } package {$name: #... stuff require => Exec["aptupdate"], } } when using you should also have included in a class something similar to concat { "/var/log/fastupdate_package.log": notify => Exec["aptupdate"], } exec { "aptupdate": command => "/usr/bin/aptitude update": refreshonly => true, } This should create something like a log file (not an actual log file, since the new data is not appended, but recreated when necessary). This would actually create some chaos if used for normal packages, so I highly recommend using this in a module that only involves your packages, and for the rest of packages use the cron solution, which is way way more cleaner than this. Hope it helps, :) Silviu -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Mon, Sep 6, 2010 at 10:56 AM, Silviu Paragina <silviu@paragina.ro> wrote:> As it was said earlier if you have a ensure => latest you should probably > have (also) a cron job for updates.I think it''s much more difficult to manage multiple schedules, and to deal with inconsistencies when people manually run Puppet at times that aren''t in sync with your cron job than it is for Puppet to simply ensure the appropriate commands are run before your packages are installed. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 06.09.2010 21:03, Nigel Kersten wrote:> On Mon, Sep 6, 2010 at 10:56 AM, Silviu Paragina<silviu@paragina.ro> wrote: > >> As it was said earlier if you have a ensure => latest you should probably >> have (also) a cron job for updates. > I think it''s much more difficult to manage multiple schedules, and to > deal with inconsistencies when people manually run Puppet at times > that aren''t in sync with your cron job than it is for Puppet to simply > ensure the appropriate commands are run before your packages are > installed. >Could you be a little more specific, I''m not sure if we are referring to the same cron job. I''m not sure if you are referring to a "apt-get update" or a "puppet" cron job. I was referring to a apt-get update cron. You are right of course. If the apt-get update cron fails (or it hasn''t got the chance to run), puppet will have no ideea that it failed, and will still run the package resources, which in some scenarios might be catastrophic. But on the other hand if it just works like that and you really really need the resources, you might consider using it. Thinking about it an apt-get update shouldn''t have more than 3-4 small files per repository if the cache is hit, and otherwise it should have at most 10 files around a few hundred bytes (in case of incremental repositories, not sure if incremental is the right technical word, but I hope you understand what I meant) so there isn''t much of a penalty there, more supporting your solution. I should of probably noted this in my answer, but most of it was already discussed in the thread already. Now my solution was proposed in the idea that under no circumstance apt-get update should be run every run (and, yes, I didn''t take into account that the reason for that might be wrong). I feel that I should point out again, that the solution is hackish and as such, if used, it should be used only in a single module, preferably in a single manifest, detailing in comments all the reasons for doing things the way it does. And most of all if somebody doesn''t follow the convention it all goes down the drain. :) So my solution should be avoided if possible, but if it can not be avoided, it works with the notes above. Silviu -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Mon, Sep 6, 2010 at 11:52 AM, Silviu Paragina <silviu@paragina.ro> wrote:> On 06.09.2010 21:03, Nigel Kersten wrote: >> >> On Mon, Sep 6, 2010 at 10:56 AM, Silviu Paragina<silviu@paragina.ro> >> wrote: >> >>> As it was said earlier if you have a ensure => latest you should >>> probably >>> have (also) a cron job for updates. >> >> I think it''s much more difficult to manage multiple schedules, and to >> deal with inconsistencies when people manually run Puppet at times >> that aren''t in sync with your cron job than it is for Puppet to simply >> ensure the appropriate commands are run before your packages are >> installed. >> > Could you be a little more specific, I''m not sure if we are referring to the > same cron job. I''m not sure if you are referring to a "apt-get update" or a > "puppet" cron job. I was referring to a apt-get update cron.Yep. That''s the one I was talking about.> > You are right of course. If the apt-get update cron fails (or it hasn''t got > the chance to run), puppet will have no ideea that it failed, and will still > run the package resources, which in some scenarios might be catastrophic. > But on the other hand if it just works like that and you really really need > the resources, you might consider using it. Thinking about it an apt-get > update shouldn''t have more than 3-4 small files per repository if the cache > is hit, and otherwise it should have at most 10 files around a few hundred > bytes (in case of incremental repositories, not sure if incremental is the > right technical word, but I hope you understand what I meant) so there isn''t > much of a penalty there, more supporting your solution. I should of probably > noted this in my answer, but most of it was already discussed in the thread > already. > > Now my solution was proposed in the idea that under no circumstance apt-get > update should be run every run (and, yes, I didn''t take into account that > the reason for that might be wrong). > > I feel that I should point out again, that the solution is hackish and as > such, if used, it should be used only in a single module, preferably in a > single manifest, detailing in comments all the reasons for doing things the > way it does. And most of all if somebody doesn''t follow the convention it > all goes down the drain. :) So my solution should be avoided if possible, > but if it can not be avoided, it works with the notes above.I''m pretty sure we''re in agreement here :)> > > > Silviu > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- nigel -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Hi,>> But for this to work, the Puppet run needs to run aptitude update to >> pick up the new package name. Running the update periodically isn''t >> enough, but running an update on every catalog run is just overkill. > > I understand your concern here, but have you done the timing tests? > > How long are your average puppet runs and how much time does an > "apt-get update" take? > > I just wonder if you''re prematurely optimizing your runs.It''s not actually about optimizing puppet runs, it''s about the load imposed on the package repository servers, when hundreds of servers do updates twice an hour. And even if that load isn''t a real problem (which it probably isn''t), it''s about politeness to the sysadmins of those servers. The behaviour that I''m trying to avoid could easily be explained as abuse. I just like to play nice. Best regards, Martijn.
Hi, Just to let you know how I implemented it in the end. Earlier I wrote:> I have now solved my problem by using a custom provider for packages, that > is based on the aptitude provider, but simply adds this: > > # See if we need to run aptitude update before installing this package > if not File.exists?("/tmp/puppet-iphionapt") > FileUtils.touch("/tmp/puppet-iphionapt") > aptitude("update") > end > > I just haven''t found a suitable way to clean up the tempfile from within > Puppet without touching any of Puppet''s own files, so I use a cronjob for > that now. Maybe I''ll switch to a ''postrun_command'', or does anyone have a > better idea?I have put the custom package provider in a module. This module also has a ''cleanup'' class, that removes the tempfile. The cleanup class is run in a ''post'' runstage. The custom provider is used by default for all packages. On the first package install, aptitude update is run, but not for subsequent installs in the same run. At the end of the run, the tempfile is removed. Best regards, Martijn.
> For the past week, I have been trying to figure out the best way to do > package management with Puppet on an Ubuntu system. I have studied many > solutons I found on the web, but none of them seem to do exactly what I want.I am not sure if somebody proposed this already, but may be you can use this approach: In the puppetmaster: * you have a puppet-file repository called files/debian-reps-sums or something like that. * You have an script scheduled each 5, 15 or 30m, that will write files for each repository with the sums. Something like this: wget http://ftp.fr.debian.org/debian/dists/lenny/Release -q -O - |grep main/binary-i386 > ftp.fr.debian.org_lenny_main_binary-i386.sums In the configuration: * Each host that depens on that repository, will has a reference on that file, storing it somewhere. You add a "notify => Exec["update-repository"]. Exec["update-repository"], of cuorse, should be notify-only. # Force apt-get update if needed file { "${debianmirror}_${distribution}_${component}_binary-${architecture}.sums": source => "puppet:///debian-reps-sums/${debianmirror}_${distribution}_${component}_binary-${architecture}.sums", path => "/var/lib/puppet/debian-reps-sums/${debianmirror}_${distribution}_${component}_binary-${architecture}.sums", owner => "root", group => 0, mode => "0664", notify => Exec["apt-get_update"] } You can create a definition to configure repositories and add this dependency. I found this method in a to control "stow" installations from http://trac.cae.tntech.edu/infrastructure/ With this method: * apt-get update will run on client only when needed. * Only one node (puppetmaster) will check if there are new updates. * You can easily force apt-get update if needed -- Atentamente Héctor Rivas -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Sep 7, 2010, at 1:22 AM, Martijn Grendelman wrote:> Hi, > >>> But for this to work, the Puppet run needs to run aptitude update to >>> pick up the new package name. Running the update periodically isn''t >>> enough, but running an update on every catalog run is just overkill. >> >> I understand your concern here, but have you done the timing tests? >> >> How long are your average puppet runs and how much time does an >> "apt-get update" take? >> >> I just wonder if you''re prematurely optimizing your runs. > > It''s not actually about optimizing puppet runs, it''s about the load > imposed on the package repository servers, when hundreds of servers do > updates twice an hour.I use the apt-cacher-ng proxy to reduce load on the package servers and for speed. It means that I can download packages over a 100Mbps connection.> And even if that load isn''t a real problem (which it probably isn''t), it''s > about politeness to the sysadmins of those servers. The behaviour that I''m > trying to avoid could easily be explained as abuse. I just like to play nice. > > Best regards, > Martijn. > > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Patrick <kc7zzv@gmail.com> writes:> On Sep 7, 2010, at 1:22 AM, Martijn Grendelman wrote: > >>>> But for this to work, the Puppet run needs to run aptitude update to pick >>>> up the new package name. Running the update periodically isn''t enough, >>>> but running an update on every catalog run is just overkill. >>> >>> I understand your concern here, but have you done the timing tests? >>> >>> How long are your average puppet runs and how much time does an >>> "apt-get update" take? >>> >>> I just wonder if you''re prematurely optimizing your runs. >> >> It''s not actually about optimizing puppet runs, it''s about the load imposed >> on the package repository servers, when hundreds of servers do updates >> twice an hour. > > I use the apt-cacher-ng proxy to reduce load on the package servers and for > speed. It means that I can download packages over a 100Mbps connection.So do we; this is nothing but polite behaviour when you have a large pool of machines hitting the same resources time and again. A word of warning from painful discoveries of the past: while apt can be a bit of a mirror and link load if you make it update, yum is a huge risk: it will do a mirror update every time you invoke it that it thinks the lists are out of date. On one deployment we had that running every hour and yum updated every hour, generating gigabytes of traffic until we put a proxy in place between it and the outside world. You can tune these things, but out of the box you run a huge risk if you don''t funnel your updates through some sort of sensible proxy server... Daniel -- ✣ Daniel Pittman ✉ daniel@rimspace.net ☎ +61 401 155 707 ♽ made with 100 percent post-consumer electrons -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.