Hi, I am using puppet (version 0.25.5-1.e15 for redhat) for password management for non-system users. This morning, users on some of my puppet clients had their encrypted password strings in /etc/shadow replaced with the following string: YAML::syck::BadAlias That has effectively broken the users'' ability to login to those servers. Puppet will not overwrite that string with the correct encrypted string, and I can''t even change the password manually using ''passwd'', because I get an ''Authentication token manipulation error''. The only way I can fix this is by manually editing /etc/shadow, replaced that YAML string with something valid (I''ve been using an ''*''), and then changing the password manually or letting puppet overwrite it with the correct password. What could have caused this? -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Gus F. wrote:> I am using puppet (version 0.25.5-1.e15 for redhat) for password > management for non-system users. This morning, users on some of my > puppet clients had their encrypted password strings in /etc/shadow > replaced with the following string: > > YAML::syck::BadAliasEeeww. That''s no damn good.> That has effectively broken the users'' ability to login to those > servers. Puppet will not overwrite that string with the correct > encrypted string, and I can''t even change the password manually > using ''passwd'', because I get an ''Authentication token manipulation > error''. The only way I can fix this is by manually editing > /etc/shadow, replaced that YAML string with something valid (I''ve > been using an ''*''), and then changing the password manually or > letting puppet overwrite it with the correct password. > > What could have caused this?If you run puppet again, does it attempt to change the entries back? You could run it with --noop to test quickly without risking a change. Though depending on the cause, it might not show up unless you run it without --noop. If no one else chimes in with better ideas, you might want to run "puppetd --test --trace --debug" (after backing up /etc/shadow). Maybe that would help determine the source of the problem. -- Todd OpenPGP -> KeyID: 0xBEAF0CE3 | URL: www.pobox.com/~tmz/pgp ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ I do not believe in the collective wisdom of individual ignorance. -- Thomas Carlyle
Yeah, it''s nasty. Running the puppet commands you listed doesn''t show any problems. The logs show the password being set correctly, but it isn''t really. Once I manually change the password, then puppet can make those password changes again. It looks like the problem may be limited to my clients which are running different (older) versions of puppet than the server is running. I am updating all of my out of date clients, manually forcing password changes and I''ll have to watch it to see what happens. Thanks for your help. I''ll let you know if I find anything out. On Jun 15, 6:29 pm, Todd Zullinger <t...@pobox.com> wrote:> Gus F. wrote: > > I am using puppet (version 0.25.5-1.e15 for redhat) for password > > management for non-system users. This morning, users on some of my > > puppet clients had their encrypted password strings in /etc/shadow > > replaced with the following string: > > > YAML::syck::BadAlias > > Eeeww. That''s no damn good. > > > That has effectively broken the users'' ability to login to those > > servers. Puppet will not overwrite that string with the correct > > encrypted string, and I can''t even change the password manually > > using ''passwd'', because I get an ''Authentication token manipulation > > error''. The only way I can fix this is by manually editing > > /etc/shadow, replaced that YAML string with something valid (I''ve > > been using an ''*''), and then changing the password manually or > > letting puppet overwrite it with the correct password. > > > What could have caused this? > > If you run puppet again, does it attempt to change the entries back? > You could run it with --noop to test quickly without risking a change. > Though depending on the cause, it might not show up unless you run it > without --noop. If no one else chimes in with better ideas, you might > want to run "puppetd --test --trace --debug" (after backing up > /etc/shadow). Maybe that would help determine the source of the > problem. > > -- > Todd OpenPGP -> KeyID: 0xBEAF0CE3 | URL:www.pobox.com/~tmz/pgp > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > I do not believe in the collective wisdom of individual ignorance. > -- Thomas Carlyle > > application_pgp-signature_part > < 1KViewDownload-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Jun 15, 2:39 pm, "Gus F." <gus.fer...@gmail.com> wrote:> I am using puppet (version 0.25.5-1.e15 for redhat) for password > management for non-system users. This morning, users on some of my > puppet clients had their encrypted password strings in /etc/shadow > replaced with the following string: > > YAML::syck::BadAliasWhoa. I just saw this on a .24.8 client for the first time today. I poked at it for a few minutes and couldnt reproduce, so I shrugged it off. This is what I found in syslog: puppetd: Starting catalog run puppetd: (//#<YAML::Syck::BadAlias:0x2a98ef0a20>/File[/usr/local/bin/ concatfragments.sh]) Failed to retrieve current state of resource: can''t convert YAML::Syck::BadAlias into Integer puppetd: (//#<YAML::Syck::BadAlias:0x2a9859e940>/Concat[/etc/ filename.txt]/Exec[concat_/etc/filename.txt]) Dependency file[/usr/ local/bin/concatfragments.sh] has 1 failures puppetd: (//#<YAML::Syck::BadAlias:0x2a9859e940>/Concat[/etc/ filename.txt]/Exec[concat_/etc/filename.txt]) Skipping because of failed dependencies puppetd: (//#<YAML::Syck::BadAlias:0x2a9859e940>/Concat[/etc/ filename.txt]/File[/etc/filename.txt]) Dependency file[/usr/local/bin/ concatfragments.sh] has 1 failures puppetd: (//#<YAML::Syck::BadAlias:0x2a9859e940>/Concat[/etc/ filename.txt]/File[/etc/filename.txt]) Skipping because of failed dependencies puppetd: Finished catalog run in 9.82 seconds> What could have caused this?+1 to this. Where to even look? -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Tue, Jun 15, 2010 at 6:32 PM, donavan <donavan@desinc.net> wrote:> On Jun 15, 2:39 pm, "Gus F." <gus.fer...@gmail.com> wrote: > > I am using puppet (version 0.25.5-1.e15 for redhat) for password > > management for non-system users. This morning, users on some of my > > puppet clients had their encrypted password strings in /etc/shadow > > replaced with the following string: > > > > YAML::syck::BadAlias > > Whoa. I just saw this on a .24.8 client for the first time today. I > poked at it for a few minutes and couldnt reproduce, so I shrugged it > off. > > This is what I found in syslog: > puppetd: Starting catalog run > puppetd: (//#<YAML::Syck::BadAlias:0x2a98ef0a20>/File[/usr/local/bin/ > concatfragments.sh]) Failed to retrieve current state of resource: > can''t convert YAML::Syck::BadAlias into Integer > puppetd: (//#<YAML::Syck::BadAlias:0x2a9859e940>/Concat[/etc/ > filename.txt]/Exec[concat_/etc/filename.txt]) Dependency file[/usr/ > local/bin/concatfragments.sh] has 1 failures > puppetd: (//#<YAML::Syck::BadAlias:0x2a9859e940>/Concat[/etc/ > filename.txt]/Exec[concat_/etc/filename.txt]) Skipping because of > failed dependencies > puppetd: (//#<YAML::Syck::BadAlias:0x2a9859e940>/Concat[/etc/ > filename.txt]/File[/etc/filename.txt]) Dependency file[/usr/local/bin/ > concatfragments.sh] has 1 failures > puppetd: (//#<YAML::Syck::BadAlias:0x2a9859e940>/Concat[/etc/ > filename.txt]/File[/etc/filename.txt]) Skipping because of failed > dependencies > puppetd: Finished catalog run in 9.82 seconds > > > What could have caused this? > > +1 to this. Where to even look? >The first thing to do is to work out whether this has occurred with any resource types other than Concat, so we know whether this is an issue in the core puppet distribution or just in the concat type/provider.> > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- nigel -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
if this happens on a 0.25.5 server (and maybe client), it might be happening because of the zaml implementation, can you reproduce this problem with 0.25.4 master? in anycase, it sounds like a bug :) Ohad On Wed, Jun 16, 2010 at 9:35 AM, Nigel Kersten <nigelk@google.com> wrote:> > > On Tue, Jun 15, 2010 at 6:32 PM, donavan <donavan@desinc.net> wrote: > >> On Jun 15, 2:39 pm, "Gus F." <gus.fer...@gmail.com> wrote: >> > I am using puppet (version 0.25.5-1.e15 for redhat) for password >> > management for non-system users. This morning, users on some of my >> > puppet clients had their encrypted password strings in /etc/shadow >> > replaced with the following string: >> > >> > YAML::syck::BadAlias >> >> Whoa. I just saw this on a .24.8 client for the first time today. I >> poked at it for a few minutes and couldnt reproduce, so I shrugged it >> off. >> >> This is what I found in syslog: >> puppetd: Starting catalog run >> puppetd: (//#<YAML::Syck::BadAlias:0x2a98ef0a20>/File[/usr/local/bin/ >> concatfragments.sh]) Failed to retrieve current state of resource: >> can''t convert YAML::Syck::BadAlias into Integer >> puppetd: (//#<YAML::Syck::BadAlias:0x2a9859e940>/Concat[/etc/ >> filename.txt]/Exec[concat_/etc/filename.txt]) Dependency file[/usr/ >> local/bin/concatfragments.sh] has 1 failures >> puppetd: (//#<YAML::Syck::BadAlias:0x2a9859e940>/Concat[/etc/ >> filename.txt]/Exec[concat_/etc/filename.txt]) Skipping because of >> failed dependencies >> puppetd: (//#<YAML::Syck::BadAlias:0x2a9859e940>/Concat[/etc/ >> filename.txt]/File[/etc/filename.txt]) Dependency file[/usr/local/bin/ >> concatfragments.sh] has 1 failures >> puppetd: (//#<YAML::Syck::BadAlias:0x2a9859e940>/Concat[/etc/ >> filename.txt]/File[/etc/filename.txt]) Skipping because of failed >> dependencies >> puppetd: Finished catalog run in 9.82 seconds >> >> > What could have caused this? >> >> +1 to this. Where to even look? >> > > The first thing to do is to work out whether this has occurred with any > resource types other than Concat, so we know whether this is an issue in the > core puppet distribution or just in the concat type/provider. > > > > >> >> -- >> >> You received this message because you are subscribed to the Google Groups >> "Puppet Users" group. >> To post to this group, send email to puppet-users@googlegroups.com. >> To unsubscribe from this group, send email to >> puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> >> . >> For more options, visit this group at >> http://groups.google.com/group/puppet-users?hl=en. >> >> > > > -- > nigel > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
It is happening on 0.25.5-1 server, but so far I have only seen the problem on 0.24.X clients. I don''t have an 0.25.4 master to test against, but at any rate I am unable to reproduce the problem. The problem seems to have occurred in a relatively small time across the affected clients, and hasn''t happened again (yet). I have seen errors similar to the following on all of the affected clients. The "''@name=" variable has a different value on different clients: ./puppet.log:Mon Jun 14 00:05:26 EDT 2010 Puppet (err): Could not apply complete catalog: Invalid ''should'' value for mode: #<YAML::Syck::BadAlias:0xb7dbdf94 @name="id050"> I agree that it sounds like a bug. I''m hoping that getting all of my clients up to date will prevent another occurrence. On Jun 15, 10:44 pm, Ohad Levy <ohadl...@gmail.com> wrote:> if this happens on a 0.25.5 server (and maybe client), it might be happening > because of the zaml implementation, can you reproduce this problem with > 0.25.4 master? > > in anycase, it sounds like a bug :) > > Ohad > > On Wed, Jun 16, 2010 at 9:35 AM, Nigel Kersten <nig...@google.com> wrote: > > > On Tue, Jun 15, 2010 at 6:32 PM, donavan <dona...@desinc.net> wrote: > > >> On Jun 15, 2:39 pm, "Gus F." <gus.fer...@gmail.com> wrote: > >> > I am using puppet (version 0.25.5-1.e15 for redhat) for password > >> > management for non-system users. This morning, users on some of my > >> > puppet clients had their encrypted password strings in /etc/shadow > >> > replaced with the following string: > > >> > YAML::syck::BadAlias > > >> Whoa. I just saw this on a .24.8 client for the first time today. I > >> poked at it for a few minutes and couldnt reproduce, so I shrugged it > >> off. > > >> This is what I found in syslog: > >> puppetd: Starting catalog run > >> puppetd: (//#<YAML::Syck::BadAlias:0x2a98ef0a20>/File[/usr/local/bin/ > >> concatfragments.sh]) Failed to retrieve current state of resource: > >> can''t convert YAML::Syck::BadAlias into Integer > >> puppetd: (//#<YAML::Syck::BadAlias:0x2a9859e940>/Concat[/etc/ > >> filename.txt]/Exec[concat_/etc/filename.txt]) Dependency file[/usr/ > >> local/bin/concatfragments.sh] has 1 failures > >> puppetd: (//#<YAML::Syck::BadAlias:0x2a9859e940>/Concat[/etc/ > >> filename.txt]/Exec[concat_/etc/filename.txt]) Skipping because of > >> failed dependencies > >> puppetd: (//#<YAML::Syck::BadAlias:0x2a9859e940>/Concat[/etc/ > >> filename.txt]/File[/etc/filename.txt]) Dependency file[/usr/local/bin/ > >> concatfragments.sh] has 1 failures > >> puppetd: (//#<YAML::Syck::BadAlias:0x2a9859e940>/Concat[/etc/ > >> filename.txt]/File[/etc/filename.txt]) Skipping because of failed > >> dependencies > >> puppetd: Finished catalog run in 9.82 seconds > > >> > What could have caused this? > > >> +1 to this. Where to even look? > > > The first thing to do is to work out whether this has occurred with any > > resource types other than Concat, so we know whether this is an issue in the > > core puppet distribution or just in the concat type/provider. > > >> -- > > >> You received this message because you are subscribed to the Google Groups > >> "Puppet Users" group. > >> To post to this group, send email to puppet-users@googlegroups.com. > >> To unsubscribe from this group, send email to > >> puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> > >> . > >> For more options, visit this group at > >>http://groups.google.com/group/puppet-users?hl=en. > > > -- > > nigel > > > -- > > You received this message because you are subscribed to the Google Groups > > "Puppet Users" group. > > To post to this group, send email to puppet-users@googlegroups.com. > > To unsubscribe from this group, send email to > > puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/puppet-users?hl=en.-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Well, I''ve worked around this for now by: - uninstalling puppet and facter with mac ports (so there''s no puppetfoo executables under /opt/local/sbin or /opt/local/bin anymore - used gem to install puppet and facter into the system ruby, so now the executables are under /usr/bin/puppetfoo This also means a version bump to 0.25.5 from 0.25.4 Current system ruby version in 10.6.3: bash-3.2# /usr/bin/ruby --version ruby 1.8.7 (2009-06-08 patchlevel 173) [universal-darwin10.0] So perhaps it''s a problem with how macports installs ruby, or something. Cheers Jesse -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Jun 15, 8:24 pm, "Gus F." <gus.fer...@gmail.com> wrote:> It is happening on 0.25.5-1 server, but so far I have only seen the > problem on 0.24.X clients. I don''t have an 0.25.4 master to test > against, but at any rate I am unable to reproduce the problem. The > problem seems to have occurred in a relatively small time across the > affected clients, and hasn''t happened again (yet).Sounds like the same setup here. We''re in the middle of transitioning, masters are .25.5 and clients .24.8. Wasn''t able to reproduce, and a very low occurrence rate. I only see one failure in the past ~6 hours. Maybe a dozen for the entire day. And that''s at ~1400 puppetruns/hr. Nigel, Doesnt look like an issue with Concat define. This is a straight File resource here: file { "/etc/mail": owner => "root", mode => 0755, group => "root", ensure => directory; } Wed Jun 16 00:00:42 -0700 2010 //#<YAML::Syck::BadAlias:0x2a9881c690>/ File[/etc/mail] (err): Failed to retrieve current state of resource: can''t convert YAML::Syck::BadAlias into Integer Wed Jun 16 00:00:42 -0700 2010 //#<YAML::Syck::BadAlias:0x2a9881c690>/ Remotefile[/etc/mail/sendmail.cf]/File[/etc/mail/sendmail.cf] (notice): Dependency file[/etc/mail] has 1 failures Wed Jun 16 00:00:42 -0700 2010 //#<YAML::Syck::BadAlias:0x2a9881c690>/ Remotefile[/etc/mail/sendmail.cf]/File[/etc/mail/sendmail.cf] (warning): Skipping because of failed dependencies Wed Jun 16 00:00:42 -0700 2010 //#<YAML::Syck::BadAlias:0x2a9881c690>/ Remotefile[/etc/mail/aliases]/File[/etc/mail/aliases] (notice): Dependency file[/etc/mail] has 1 failures Wed Jun 16 00:00:42 -0700 2010 //#<YAML::Syck::BadAlias:0x2a9881c690>/ Remotefile[/etc/mail/aliases]/File[/etc/mail/aliases] (warning): Skipping because of failed dependencies -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.