Puppet users - Jan 2007 - error "Stack level too deep" on ''exec[ "perl Makefile.PL" ] ?

Hi,

I''m testing puppet from the subversion trunk. I have a rather
hacked-together system to install a CRM into one of my nodes, which
worked (more or less) with version 0.19.3. When I tried to do a puppet
run on a completely clean node it seemed to stall on the
''compilation''
part (I do an exec of ''perl Makefile.PL'' in the appropriate
dir). When
I broke it off, the debug showed the following:

stack level too deep
err:
/base/enam/eserver/perlinstall[everything]/Exec[makefile-everything]/returns:
change from notrun to 0 failed: stack level too deep

And then a number of failed steps due to unresolved dependencies. Is
this an error message from puppet or is the "stack level too deep"
thing originating from something else?

Thijs

On Jan 2, 2007, at 8:37 PM, Thijs Oppermann wrote:
> Hi,
>
> I''m testing puppet from the subversion trunk. I have a rather
> hacked-together system to install a CRM into one of my nodes, which
> worked (more or less) with version 0.19.3. When I tried to do a puppet
> run on a completely clean node it seemed to stall on the
''compilation''
> part (I do an exec of ''perl Makefile.PL'' in the
appropriate dir). When
> I broke it off, the debug showed the following:
>
> stack level too deep
> err: /base/enam/eserver/perlinstall[everything]/Exec[makefile- 
> everything]/returns:
> change from notrun to 0 failed: stack level too deep
>
> And then a number of failed steps due to unresolved dependencies. Is
> this an error message from puppet or is the "stack level too
deep"
> thing originating from something else?
''stack level too deep'' is a ruby error.  What is the exact
code that
caused this?

  --
  One of the keys to happiness is a bad memory.   -- Rita Mae Brown
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

OK, after some more testing I can say that this error does not occur
with 0.19.3. Same procedure (clean node, first run: 0.19.3 finishes
with all OK, latest svn checkout (yesterday) does not).

The code where it happens:

* a node calls a define with the following:

    perlinstall { "everything":
        ploverride      => "everything",
        require         => wgetfile[ "everything.tar.gz" ]
    }

* the define downloads a tar.gz file, extracts it, substitutes a
custom Makefile.PL into the extract dir and tries to do a ''perl
Makefile.PL'', which seems to hang. On a ctrl-c I get the "stack
level
too deep" error. The code where it hangs looks like this (nothing
fancy):

    exec { "makefile-$name":
        command => "cd $( ls -A -d $name*/ ) && perl
Makefile.PL",
        path    => "/usr/bin:/bin",
        cwd     => $dest,
        require => [ package[ "perl" ], exec[
"untar-$name" ] ]
    }

It''s probably a bit of a hack, but I''m trying this
off-the-cuff-like... ;) And in my defense it did work in 0.19.3. ;)

Thijs

P.S. I attached the complete perlinstall.pp, I hope that covers your
request for "exact code".



On 03/01/07, Luke Kanies <luke@madstop.com> wrote:
> On Jan 2, 2007, at 8:37 PM, Thijs Oppermann wrote:
>
> > Hi,
> >
> > I''m testing puppet from the subversion trunk. I have a rather
> > hacked-together system to install a CRM into one of my nodes, which
> > worked (more or less) with version 0.19.3. When I tried to do a puppet
> > run on a completely clean node it seemed to stall on the
''compilation''
> > part (I do an exec of ''perl Makefile.PL'' in the
appropriate dir). When
> > I broke it off, the debug showed the following:
> >
> > stack level too deep
> > err: /base/enam/eserver/perlinstall[everything]/Exec[makefile-
> > everything]/returns:
> > change from notrun to 0 failed: stack level too deep
> >
> > And then a number of failed steps due to unresolved dependencies. Is
> > this an error message from puppet or is the "stack level too
deep"
> > thing originating from something else?
>
> ''stack level too deep'' is a ruby error.  What is the
exact code that
> caused this?
>
>   --
>   One of the keys to happiness is a bad memory.   -- Rita Mae Brown
>   ---------------------------------------------------------------------
>   Luke Kanies | http://reductivelabs.com | http://madstop.com
>
>
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users
>

_______________________________________________
Puppet-users mailing list
Puppet-users@madstop.com
https://mail.madstop.com/mailman/listinfo/puppet-users

On Jan 3, 2007, at 3:09 AM, Thijs Oppermann wrote:
> OK, after some more testing I can say that this error does not occur
> with 0.19.3. Same procedure (clean node, first run: 0.19.3 finishes
> with all OK, latest svn checkout (yesterday) does not).
>
> The code where it happens:
>
> * a node calls a define with the following:
>
>    perlinstall { "everything":
>        ploverride      => "everything",
>        require         => wgetfile[ "everything.tar.gz" ]
>    }
>
> * the define downloads a tar.gz file, extracts it, substitutes a
> custom Makefile.PL into the extract dir and tries to do a ''perl
> Makefile.PL'', which seems to hang. On a ctrl-c I get the
"stack level
> too deep" error. The code where it hangs looks like this (nothing
> fancy):
>
>    exec { "makefile-$name":
>        command => "cd $( ls -A -d $name*/ ) && perl
Makefile.PL",
>        path    => "/usr/bin:/bin",
>        cwd     => $dest,
>        require => [ package[ "perl" ], exec[
"untar-$name" ] ]
>    }
>
> It''s probably a bit of a hack, but I''m trying this
> off-the-cuff-like... ;) And in my defense it did work in 0.19.3. ;)
Can you reproduce the problem if you run the command by itself?  That  
is, if you do the manual setup and then just run the command using  
puppet?

Also, if you run puppetd with --trace, do you get a stack trace?  I  
expect it''d be a *huge* trace, but if you can give me the top few  
lines (the bits before the deep stack start, probably including the  
top few lines of the deep stack), it''ll be much easier to track down  
the problem.

This should certainly work, so it''s not something you''re doing
wrong,
at least not that I can tell, and if it is a configuration problem,  
then Puppet should fail more gracefully than that.
> Thijs
>
> P.S. I attached the complete perlinstall.pp, I hope that covers your
> request for "exact code".
Yep, thanks.

  --
  It is curious that physical courage should be so common in the  
world and
  moral courage so rare.            -- Mark Twain
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

On Jan 3, 2007, at 3:09 AM, Thijs Oppermann wrote:
> OK, after some more testing I can say that this error does not occur
> with 0.19.3. Same procedure (clean node, first run: 0.19.3 finishes
> with all OK, latest svn checkout (yesterday) does not).
Also, can you file this as a bug with ''minor'' as the
milestone?

Thanks.

  --
  Seize opportunity by the beard, for it is bald behind.  --  
Bulgarian Proverb
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

OK, will do, but I need to reset my test machine for this, and do it
all remotely. My internet connectivity is not very reliable at the
moment, so it may take some time. Thanks for looking at it.

Thijs

On 04/01/07, Luke Kanies <luke@madstop.com> wrote:
> On Jan 3, 2007, at 3:09 AM, Thijs Oppermann wrote:
>
> > OK, after some more testing I can say that this error does not occur
> > with 0.19.3. Same procedure (clean node, first run: 0.19.3 finishes
> > with all OK, latest svn checkout (yesterday) does not).
>
> Also, can you file this as a bug with ''minor'' as the
milestone?
>
> Thanks.
>
>   --
>   Seize opportunity by the beard, for it is bald behind.  --
> Bulgarian Proverb
>   ---------------------------------------------------------------------
>   Luke Kanies | http://reductivelabs.com | http://madstop.com
>
>
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users
>

Hmm... running with --trace gave me some weird behaviour. Now it fails
to do some basic package installs, such as postfix, mailx and make,
while others go fine. The error these give is:

Failed to retrieve current state: Failed to handle dpkg-query output

The first couple of lines of trace for this error are thus:

/usr/local/lib/site_ruby/1.8/puppet/provider/package/dpkg.rb:80:in
`query''
/usr/local/lib/site_ruby/1.8/puppet/type/package.rb:396:in `retrieve''
/usr/local/lib/site_ruby/1.8/puppet/metatype/evaluation.rb:22:in
`evaluate''
/usr/local/lib/site_ruby/1.8/puppet/transaction.rb:42:in `apply''
/usr/local/lib/site_ruby/1.8/puppet/transaction.rb:207:in
`eval_resource''
/usr/local/lib/site_ruby/1.8/puppet/util.rb:397:in `thinmark''

And then the source of the error I wanted to look at gets skipped as
it has the make package as a requirement.

Don''t know if this relates, but I did a ''svn up'' on
both the master as
the client before trying this.

I''ll try the manual setup thing to test further...

Thijs


P.S. the full trace is manageable, so here it is:

notice: /base/enam/eserver/Package[libcgi-perl]/ensure: created
/usr/local/lib/site_ruby/1.8/puppet/provider/package/dpkg.rb:80:in
`query''
/usr/local/lib/site_ruby/1.8/puppet/type/package.rb:396:in `retrieve''
/usr/local/lib/site_ruby/1.8/puppet/metatype/evaluation.rb:22:in
`evaluate''
/usr/local/lib/site_ruby/1.8/puppet/transaction.rb:42:in `apply''
/usr/local/lib/site_ruby/1.8/puppet/transaction.rb:207:in
`eval_resource''
/usr/local/lib/site_ruby/1.8/puppet/util.rb:397:in `thinmark''
/usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
/usr/lib/ruby/1.8/benchmark.rb:307:in `realtime''
/usr/local/lib/site_ruby/1.8/puppet/util.rb:396:in `thinmark''
/usr/local/lib/site_ruby/1.8/puppet/transaction.rb:206:in
`eval_resource''
/usr/local/lib/site_ruby/1.8/puppet/transaction.rb:264:in `evaluate''
/usr/local/lib/site_ruby/1.8/puppet/util.rb:397:in `thinmark''
/usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
/usr/lib/ruby/1.8/benchmark.rb:307:in `realtime''
/usr/local/lib/site_ruby/1.8/puppet/util.rb:396:in `thinmark''
/usr/local/lib/site_ruby/1.8/puppet/transaction.rb:263:in `evaluate''
/usr/local/lib/site_ruby/1.8/puppet/transaction.rb:257:in `evaluate''
/usr/local/lib/site_ruby/1.8/puppet/client/master.rb:116:in `apply''
/usr/local/lib/site_ruby/1.8/puppet/client/master.rb:332:in `run''
/usr/local/lib/site_ruby/1.8/puppet/util.rb:212:in `benchmark''
/usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
/usr/lib/ruby/1.8/benchmark.rb:307:in `realtime''
/usr/local/lib/site_ruby/1.8/puppet/util.rb:211:in `benchmark''
/usr/local/lib/site_ruby/1.8/puppet/client/master.rb:331:in `run''
/usr/lib/ruby/1.8/sync.rb:229:in `synchronize''
/usr/local/lib/site_ruby/1.8/puppet/client/master.rb:317:in `run''
/usr/bin/puppetd:427
/usr/local/lib/site_ruby/1.8/puppet/provider/package/dpkg.rb:80:in
`query''
/usr/local/lib/site_ruby/1.8/puppet/type/package.rb:396:in `retrieve''
/usr/local/lib/site_ruby/1.8/puppet/metatype/evaluation.rb:22:in
`evaluate''
/usr/local/lib/site_ruby/1.8/puppet/transaction.rb:42:in `apply''
/usr/local/lib/site_ruby/1.8/puppet/transaction.rb:207:in
`eval_resource''
/usr/local/lib/site_ruby/1.8/puppet/util.rb:397:in `thinmark''
/usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
/usr/lib/ruby/1.8/benchmark.rb:307:in `realtime''
/usr/local/lib/site_ruby/1.8/puppet/util.rb:396:in `thinmark''
/usr/local/lib/site_ruby/1.8/puppet/transaction.rb:206:in
`eval_resource''
/usr/local/lib/site_ruby/1.8/puppet/transaction.rb:264:in `evaluate''
/usr/local/lib/site_ruby/1.8/puppet/util.rb:397:in `thinmark''
/usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
/usr/lib/ruby/1.8/benchmark.rb:307:in `realtime''
/usr/local/lib/site_ruby/1.8/puppet/util.rb:396:in `thinmark''
/usr/local/lib/site_ruby/1.8/puppet/transaction.rb:263:in `evaluate''
/usr/local/lib/site_ruby/1.8/puppet/transaction.rb:257:in `evaluate''
/usr/local/lib/site_ruby/1.8/puppet/client/master.rb:116:in `apply''
/usr/local/lib/site_ruby/1.8/puppet/client/master.rb:332:in `run''
/usr/local/lib/site_ruby/1.8/puppet/util.rb:212:in `benchmark''
/usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
/usr/lib/ruby/1.8/benchmark.rb:307:in `realtime''
/usr/local/lib/site_ruby/1.8/puppet/util.rb:211:in `benchmark''
/usr/local/lib/site_ruby/1.8/puppet/client/master.rb:331:in `run''
/usr/lib/ruby/1.8/sync.rb:229:in `synchronize''
/usr/local/lib/site_ruby/1.8/puppet/client/master.rb:317:in `run''
/usr/bin/puppetd:427
err:
//base/enam/eserver/apache2server[apache2eserver]/Package[libapache2-mod-perl2]:
Failed to retrieve current state: Failed to handle dpkg-query output




On 04/01/07, Thijs Oppermann <thijso+puppet@gmail.com>
wrote:
> OK, will do, but I need to reset my test machine for this, and do it
> all remotely. My internet connectivity is not very reliable at the
> moment, so it may take some time. Thanks for looking at it.
>
> Thijs
>
> On 04/01/07, Luke Kanies <luke@madstop.com> wrote:
> > On Jan 3, 2007, at 3:09 AM, Thijs Oppermann wrote:
> >
> > > OK, after some more testing I can say that this error does not
occur
> > > with 0.19.3. Same procedure (clean node, first run: 0.19.3
finishes
> > > with all OK, latest svn checkout (yesterday) does not).
> >
> > Also, can you file this as a bug with ''minor'' as the
milestone?
> >
> > Thanks.
> >
> >   --
> >   Seize opportunity by the beard, for it is bald behind.  --
> > Bulgarian Proverb
> >  
---------------------------------------------------------------------
> >   Luke Kanies | http://reductivelabs.com | http://madstop.com
> >
> >
> > _______________________________________________
> > Puppet-users mailing list
> > Puppet-users@madstop.com
> > https://mail.madstop.com/mailman/listinfo/puppet-users
> >
>

Ok, on the second run I still get stuff as in the previous reply, but
it does hang on the "old" error... And looking at the source of the
supposedly custom Makefile.PL I see that even though puppet tells me
it substituted my custom version, it''s still the original one. And
that explains the hang, because it wants user input, which it won''t
get from puppet of course...

So now the question becomes why it doesn''t replace the file, while it
says it does... (and the errors from the previous reply of course...)

Thijs

On 04/01/07, Thijs Oppermann <thijso+puppet@gmail.com>
wrote:
> Hmm... running with --trace gave me some weird behaviour. Now it fails
> to do some basic package installs, such as postfix, mailx and make,
> while others go fine. The error these give is:
>
> Failed to retrieve current state: Failed to handle dpkg-query output
>
> The first couple of lines of trace for this error are thus:
>
> /usr/local/lib/site_ruby/1.8/puppet/provider/package/dpkg.rb:80:in
`query''
> /usr/local/lib/site_ruby/1.8/puppet/type/package.rb:396:in
`retrieve''
> /usr/local/lib/site_ruby/1.8/puppet/metatype/evaluation.rb:22:in
`evaluate''
> /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:42:in `apply''
> /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:207:in
`eval_resource''
> /usr/local/lib/site_ruby/1.8/puppet/util.rb:397:in `thinmark''
>
> And then the source of the error I wanted to look at gets skipped as
> it has the make package as a requirement.
>
> Don''t know if this relates, but I did a ''svn up''
on both the master as
> the client before trying this.
>
> I''ll try the manual setup thing to test further...
>
> Thijs
>
>
> P.S. the full trace is manageable, so here it is:
>
> notice: /base/enam/eserver/Package[libcgi-perl]/ensure: created
> /usr/local/lib/site_ruby/1.8/puppet/provider/package/dpkg.rb:80:in
`query''
> /usr/local/lib/site_ruby/1.8/puppet/type/package.rb:396:in
`retrieve''
> /usr/local/lib/site_ruby/1.8/puppet/metatype/evaluation.rb:22:in
`evaluate''
> /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:42:in `apply''
> /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:207:in
`eval_resource''
> /usr/local/lib/site_ruby/1.8/puppet/util.rb:397:in `thinmark''
> /usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
> /usr/lib/ruby/1.8/benchmark.rb:307:in `realtime''
> /usr/local/lib/site_ruby/1.8/puppet/util.rb:396:in `thinmark''
> /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:206:in
`eval_resource''
> /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:264:in
`evaluate''
> /usr/local/lib/site_ruby/1.8/puppet/util.rb:397:in `thinmark''
> /usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
> /usr/lib/ruby/1.8/benchmark.rb:307:in `realtime''
> /usr/local/lib/site_ruby/1.8/puppet/util.rb:396:in `thinmark''
> /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:263:in
`evaluate''
> /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:257:in
`evaluate''
> /usr/local/lib/site_ruby/1.8/puppet/client/master.rb:116:in
`apply''
> /usr/local/lib/site_ruby/1.8/puppet/client/master.rb:332:in `run''
> /usr/local/lib/site_ruby/1.8/puppet/util.rb:212:in `benchmark''
> /usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
> /usr/lib/ruby/1.8/benchmark.rb:307:in `realtime''
> /usr/local/lib/site_ruby/1.8/puppet/util.rb:211:in `benchmark''
> /usr/local/lib/site_ruby/1.8/puppet/client/master.rb:331:in `run''
> /usr/lib/ruby/1.8/sync.rb:229:in `synchronize''
> /usr/local/lib/site_ruby/1.8/puppet/client/master.rb:317:in `run''
> /usr/bin/puppetd:427
> /usr/local/lib/site_ruby/1.8/puppet/provider/package/dpkg.rb:80:in
`query''
> /usr/local/lib/site_ruby/1.8/puppet/type/package.rb:396:in
`retrieve''
> /usr/local/lib/site_ruby/1.8/puppet/metatype/evaluation.rb:22:in
`evaluate''
> /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:42:in `apply''
> /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:207:in
`eval_resource''
> /usr/local/lib/site_ruby/1.8/puppet/util.rb:397:in `thinmark''
> /usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
> /usr/lib/ruby/1.8/benchmark.rb:307:in `realtime''
> /usr/local/lib/site_ruby/1.8/puppet/util.rb:396:in `thinmark''
> /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:206:in
`eval_resource''
> /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:264:in
`evaluate''
> /usr/local/lib/site_ruby/1.8/puppet/util.rb:397:in `thinmark''
> /usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
> /usr/lib/ruby/1.8/benchmark.rb:307:in `realtime''
> /usr/local/lib/site_ruby/1.8/puppet/util.rb:396:in `thinmark''
> /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:263:in
`evaluate''
> /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:257:in
`evaluate''
> /usr/local/lib/site_ruby/1.8/puppet/client/master.rb:116:in
`apply''
> /usr/local/lib/site_ruby/1.8/puppet/client/master.rb:332:in `run''
> /usr/local/lib/site_ruby/1.8/puppet/util.rb:212:in `benchmark''
> /usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
> /usr/lib/ruby/1.8/benchmark.rb:307:in `realtime''
> /usr/local/lib/site_ruby/1.8/puppet/util.rb:211:in `benchmark''
> /usr/local/lib/site_ruby/1.8/puppet/client/master.rb:331:in `run''
> /usr/lib/ruby/1.8/sync.rb:229:in `synchronize''
> /usr/local/lib/site_ruby/1.8/puppet/client/master.rb:317:in `run''
> /usr/bin/puppetd:427
> err:
//base/enam/eserver/apache2server[apache2eserver]/Package[libapache2-mod-perl2]:
> Failed to retrieve current state: Failed to handle dpkg-query output
>
>
>
>
> On 04/01/07, Thijs Oppermann <thijso+puppet@gmail.com> wrote:
> > OK, will do, but I need to reset my test machine for this, and do it
> > all remotely. My internet connectivity is not very reliable at the
> > moment, so it may take some time. Thanks for looking at it.
> >
> > Thijs
> >
> > On 04/01/07, Luke Kanies <luke@madstop.com> wrote:
> > > On Jan 3, 2007, at 3:09 AM, Thijs Oppermann wrote:
> > >
> > > > OK, after some more testing I can say that this error does
not occur
> > > > with 0.19.3. Same procedure (clean node, first run: 0.19.3
finishes
> > > > with all OK, latest svn checkout (yesterday) does not).
> > >
> > > Also, can you file this as a bug with ''minor''
as the milestone?
> > >
> > > Thanks.
> > >
> > >   --
> > >   Seize opportunity by the beard, for it is bald behind.  --
> > > Bulgarian Proverb
> > >  
---------------------------------------------------------------------
> > >   Luke Kanies | http://reductivelabs.com | http://madstop.com
> > >
> > >
> > > _______________________________________________
> > > Puppet-users mailing list
> > > Puppet-users@madstop.com
> > > https://mail.madstop.com/mailman/listinfo/puppet-users
> > >
> >
>

Sorry for spamming the list.

Another test, after removing all the files relating to the perl error
(such as the tar.gz and the resulting directory), seems to point to a
sequencing error, as the subsequent puppet run completed the ''perl
Makefile.PL'' step (and then stops because make is missing due to the
other error).

So it seems as if under some conditions the move of the makefile is
done *before* the untar of the source package, which of course results
in bad stuff happening (and which shouldn''t happen, as the move is
marked with a ''before => exec[ "makefile-$name"
]'' clause).

Thijs

On 04/01/07, Thijs Oppermann <thijso+puppet@gmail.com>
wrote:
> Ok, on the second run I still get stuff as in the previous reply, but
> it does hang on the "old" error... And looking at the source of
the
> supposedly custom Makefile.PL I see that even though puppet tells me
> it substituted my custom version, it''s still the original one. And
> that explains the hang, because it wants user input, which it
won''t
> get from puppet of course...
>
> So now the question becomes why it doesn''t replace the file, while
it
> says it does... (and the errors from the previous reply of course...)
>
> Thijs
>
> On 04/01/07, Thijs Oppermann <thijso+puppet@gmail.com> wrote:
> > Hmm... running with --trace gave me some weird behaviour. Now it fails
> > to do some basic package installs, such as postfix, mailx and make,
> > while others go fine. The error these give is:
> >
> > Failed to retrieve current state: Failed to handle dpkg-query output
> >
> > The first couple of lines of trace for this error are thus:
> >
> > /usr/local/lib/site_ruby/1.8/puppet/provider/package/dpkg.rb:80:in
`query''
> > /usr/local/lib/site_ruby/1.8/puppet/type/package.rb:396:in
`retrieve''
> > /usr/local/lib/site_ruby/1.8/puppet/metatype/evaluation.rb:22:in
`evaluate''
> > /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:42:in
`apply''
> > /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:207:in
`eval_resource''
> > /usr/local/lib/site_ruby/1.8/puppet/util.rb:397:in `thinmark''
> >
> > And then the source of the error I wanted to look at gets skipped as
> > it has the make package as a requirement.
> >
> > Don''t know if this relates, but I did a ''svn
up'' on both the master as
> > the client before trying this.
> >
> > I''ll try the manual setup thing to test further...
> >
> > Thijs
> >
> >
> > P.S. the full trace is manageable, so here it is:
> >
> > notice: /base/enam/eserver/Package[libcgi-perl]/ensure: created
> > /usr/local/lib/site_ruby/1.8/puppet/provider/package/dpkg.rb:80:in
`query''
> > /usr/local/lib/site_ruby/1.8/puppet/type/package.rb:396:in
`retrieve''
> > /usr/local/lib/site_ruby/1.8/puppet/metatype/evaluation.rb:22:in
`evaluate''
> > /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:42:in
`apply''
> > /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:207:in
`eval_resource''
> > /usr/local/lib/site_ruby/1.8/puppet/util.rb:397:in `thinmark''
> > /usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
> > /usr/lib/ruby/1.8/benchmark.rb:307:in `realtime''
> > /usr/local/lib/site_ruby/1.8/puppet/util.rb:396:in `thinmark''
> > /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:206:in
`eval_resource''
> > /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:264:in
`evaluate''
> > /usr/local/lib/site_ruby/1.8/puppet/util.rb:397:in `thinmark''
> > /usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
> > /usr/lib/ruby/1.8/benchmark.rb:307:in `realtime''
> > /usr/local/lib/site_ruby/1.8/puppet/util.rb:396:in `thinmark''
> > /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:263:in
`evaluate''
> > /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:257:in
`evaluate''
> > /usr/local/lib/site_ruby/1.8/puppet/client/master.rb:116:in
`apply''
> > /usr/local/lib/site_ruby/1.8/puppet/client/master.rb:332:in
`run''
> > /usr/local/lib/site_ruby/1.8/puppet/util.rb:212:in
`benchmark''
> > /usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
> > /usr/lib/ruby/1.8/benchmark.rb:307:in `realtime''
> > /usr/local/lib/site_ruby/1.8/puppet/util.rb:211:in
`benchmark''
> > /usr/local/lib/site_ruby/1.8/puppet/client/master.rb:331:in
`run''
> > /usr/lib/ruby/1.8/sync.rb:229:in `synchronize''
> > /usr/local/lib/site_ruby/1.8/puppet/client/master.rb:317:in
`run''
> > /usr/bin/puppetd:427
> > /usr/local/lib/site_ruby/1.8/puppet/provider/package/dpkg.rb:80:in
`query''
> > /usr/local/lib/site_ruby/1.8/puppet/type/package.rb:396:in
`retrieve''
> > /usr/local/lib/site_ruby/1.8/puppet/metatype/evaluation.rb:22:in
`evaluate''
> > /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:42:in
`apply''
> > /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:207:in
`eval_resource''
> > /usr/local/lib/site_ruby/1.8/puppet/util.rb:397:in `thinmark''
> > /usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
> > /usr/lib/ruby/1.8/benchmark.rb:307:in `realtime''
> > /usr/local/lib/site_ruby/1.8/puppet/util.rb:396:in `thinmark''
> > /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:206:in
`eval_resource''
> > /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:264:in
`evaluate''
> > /usr/local/lib/site_ruby/1.8/puppet/util.rb:397:in `thinmark''
> > /usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
> > /usr/lib/ruby/1.8/benchmark.rb:307:in `realtime''
> > /usr/local/lib/site_ruby/1.8/puppet/util.rb:396:in `thinmark''
> > /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:263:in
`evaluate''
> > /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:257:in
`evaluate''
> > /usr/local/lib/site_ruby/1.8/puppet/client/master.rb:116:in
`apply''
> > /usr/local/lib/site_ruby/1.8/puppet/client/master.rb:332:in
`run''
> > /usr/local/lib/site_ruby/1.8/puppet/util.rb:212:in
`benchmark''
> > /usr/lib/ruby/1.8/benchmark.rb:293:in `measure''
> > /usr/lib/ruby/1.8/benchmark.rb:307:in `realtime''
> > /usr/local/lib/site_ruby/1.8/puppet/util.rb:211:in
`benchmark''
> > /usr/local/lib/site_ruby/1.8/puppet/client/master.rb:331:in
`run''
> > /usr/lib/ruby/1.8/sync.rb:229:in `synchronize''
> > /usr/local/lib/site_ruby/1.8/puppet/client/master.rb:317:in
`run''
> > /usr/bin/puppetd:427
> > err:
//base/enam/eserver/apache2server[apache2eserver]/Package[libapache2-mod-perl2]:
> > Failed to retrieve current state: Failed to handle dpkg-query output
> >
> >
> >
> >
> > On 04/01/07, Thijs Oppermann <thijso+puppet@gmail.com> wrote:
> > > OK, will do, but I need to reset my test machine for this, and do
it
> > > all remotely. My internet connectivity is not very reliable at
the
> > > moment, so it may take some time. Thanks for looking at it.
> > >
> > > Thijs
> > >
> > > On 04/01/07, Luke Kanies <luke@madstop.com> wrote:
> > > > On Jan 3, 2007, at 3:09 AM, Thijs Oppermann wrote:
> > > >
> > > > > OK, after some more testing I can say that this error
does not occur
> > > > > with 0.19.3. Same procedure (clean node, first run:
0.19.3 finishes
> > > > > with all OK, latest svn checkout (yesterday) does not).
> > > >
> > > > Also, can you file this as a bug with
''minor'' as the milestone?
> > > >
> > > > Thanks.
> > > >
> > > >   --
> > > >   Seize opportunity by the beard, for it is bald behind.  --
> > > > Bulgarian Proverb
> > > >  
---------------------------------------------------------------------
> > > >   Luke Kanies | http://reductivelabs.com |
http://madstop.com
> > > >
> > > >
> > > > _______________________________________________
> > > > Puppet-users mailing list
> > > > Puppet-users@madstop.com
> > > > https://mail.madstop.com/mailman/listinfo/puppet-users
> > > >
> > >
> >
>

On Jan 3, 2007, at 10:35 PM, Thijs Oppermann wrote:
> Hmm... running with --trace gave me some weird behaviour. Now it fails
> to do some basic package installs, such as postfix, mailx and make,
> while others go fine. The error these give is:
>
> Failed to retrieve current state: Failed to handle dpkg-query output
>
> The first couple of lines of trace for this error are thus:
>
> /usr/local/lib/site_ruby/1.8/puppet/provider/package/dpkg.rb:80:in  
> `query''
> /usr/local/lib/site_ruby/1.8/puppet/type/package.rb:396:in
`retrieve''
> /usr/local/lib/site_ruby/1.8/puppet/metatype/evaluation.rb:22:in  
> `evaluate''
> /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:42:in `apply''
> /usr/local/lib/site_ruby/1.8/puppet/transaction.rb:207:in  
> `eval_resource''
> /usr/local/lib/site_ruby/1.8/puppet/util.rb:397:in `thinmark''
I just modified things so that this will just throw a warning instead  
of an exception, and it will actually print out the line it can''t  
deal with.  It still shouldn''t happen, but at least it won''t
be as
big a problem if it does.

  --
  I don''t want the world, I just want your half.
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

On Jan 3, 2007, at 10:51 PM, Thijs Oppermann wrote:
> Sorry for spamming the list.
This is what it takes to find these weird edge cases; I''m just  
thankful you''re willing to spend the time tracking it down.
> Another test, after removing all the files relating to the perl error
> (such as the tar.gz and the resulting directory), seems to point to a
> sequencing error, as the subsequent puppet run completed the ''perl
> Makefile.PL'' step (and then stops because make is missing due to
the
> other error).
>
> So it seems as if under some conditions the move of the makefile is
> done *before* the untar of the source package, which of course results
> in bad stuff happening (and which shouldn''t happen, as the move is
> marked with a ''before => exec[ "makefile-$name"
]'' clause).
Ok.  I''ll see if I can figure out what''s going on, and
possibly to
reproduce any of it.

Thanks.

  --
  Never try to tell everything you know. It may take too short a time.
          --Norman Ford
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

On Jan 3, 2007, at 10:51 PM, Thijs Oppermann wrote:
> So it seems as if under some conditions the move of the makefile is
> done *before* the untar of the source package, which of course results
> in bad stuff happening (and which shouldn''t happen, as the move is
> marked with a ''before => exec[ "makefile-$name"
]'' clause).
I just had another idea.  Could you run the broken configuration  
again, but with the option --graph?

This should create .dot files in your $vardir/graphs directory.   
These .dot files can be turned into images showing what your  
configuration looks like.  It should make any ordering problems  
pretty clear.  If the .dot files are successfully created, please  
send them to me.

  --
  "They called me mad, and I called them mad, and damn them, they
  outvoted me." -- Nathaniel Lee, on being consigned to a mental
  institution, circa 17th c.
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

Okay, I just got everything installed locally and tried to reproduce  
this problem.

However, I do not get the ''Stack level too deep'' warning.  I
don''t
have your replacement Makefile.PL, so I don''t know if that could  
somehow be the problem, but using the one that comes with  
''everything'', I get the hang you mentioned, and when I cancel
things
Puppet behaves correctly.  I''ve tried it with both the stand-alone  
''puppet'' executable and with ''puppetd''.

At this point, I think I''m going to have to release without solving  
this problem.  I expect to put out more frequent point releases in  
the near future as we solidify a few aspects of the recent code,  
especially the Rails stuff, so as soon as we can reliably reproduce  
this I''ll get a fix released.

By the way, your wgetfile definition could be a bit more efficient if  
you added a ''creates'' line that pointed to the destination --
that
way, if the file already exists then the wget command would not be  
run again.

  --
  The leader of Jamestown was "John Smith" (not his real name), under
  whose direction the colony engaged in a number of activities,
  primarily related to starving. -- Dave Barry, "Dave Barry Slept
Here"
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

Okay, thanks. The strange thing is that this happens on a completely
clean VM of a fully up to date Ubuntu server install. And, of course,
that with version 0.19.3 I have no problems at all. Anyway, I''ll keep
on tinkering and testing. Maybe I''ll come up with something. Thanks
for the help so far.

Thijs

On 05/01/07, Luke Kanies <luke@madstop.com> wrote:
> Okay, I just got everything installed locally and tried to reproduce
> this problem.
>
> However, I do not get the ''Stack level too deep'' warning.
I don''t
> have your replacement Makefile.PL, so I don''t know if that could
> somehow be the problem, but using the one that comes with
> ''everything'', I get the hang you mentioned, and when I
cancel things
> Puppet behaves correctly.  I''ve tried it with both the stand-alone
> ''puppet'' executable and with ''puppetd''.
>
> At this point, I think I''m going to have to release without
solving
> this problem.  I expect to put out more frequent point releases in
> the near future as we solidify a few aspects of the recent code,
> especially the Rails stuff, so as soon as we can reliably reproduce
> this I''ll get a fix released.
>
> By the way, your wgetfile definition could be a bit more efficient if
> you added a ''creates'' line that pointed to the
destination -- that
> way, if the file already exists then the wget command would not be
> run again.
>
>   --
>   The leader of Jamestown was "John Smith" (not his real name),
under
>   whose direction the colony engaged in a number of activities,
>   primarily related to starving. -- Dave Barry, "Dave Barry Slept
Here"
>   ---------------------------------------------------------------------
>   Luke Kanies | http://reductivelabs.com | http://madstop.com
>
>
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users
>

About the more efficient wgetfile def: the problem is that I (in some
cases) rename the file I get to the $name value. So if I then add a
creates line, it will never update to a new version once a new one is
released.

I think I could adapt the def to work around that, but it would be
easier if I can assign results of commands to variables I can use
later. I think this comes close to what you once said earlier you did
not want for puppet: it becomes more of a programming language?

In this case it would be nice if I could just save the file using wget
with its original filename in combination with the --no-clobber option
so if the file already exists wget just does nothing. Assigning the
full name of this just downloaded newest file to a variable in puppet
(by grepping the output of the wget command) would then make the
subsequent actions easier (untar, cd into dir, make, install, etc), as
I would know which of the versions that are there is the newest.

But anyway, as I said, there are probably ways to work around this
already in puppet. I''ll just have to put some thinking into it.. ;)

Thijs


On 05/01/07, Luke Kanies <luke@madstop.com> wrote:
> Okay, I just got everything installed locally and tried to reproduce
> this problem.
>
> However, I do not get the ''Stack level too deep'' warning.
I don''t
> have your replacement Makefile.PL, so I don''t know if that could
> somehow be the problem, but using the one that comes with
> ''everything'', I get the hang you mentioned, and when I
cancel things
> Puppet behaves correctly.  I''ve tried it with both the stand-alone
> ''puppet'' executable and with ''puppetd''.
>
> At this point, I think I''m going to have to release without
solving
> this problem.  I expect to put out more frequent point releases in
> the near future as we solidify a few aspects of the recent code,
> especially the Rails stuff, so as soon as we can reliably reproduce
> this I''ll get a fix released.
>
> By the way, your wgetfile definition could be a bit more efficient if
> you added a ''creates'' line that pointed to the
destination -- that
> way, if the file already exists then the wget command would not be
> run again.
>
>   --
>   The leader of Jamestown was "John Smith" (not his real name),
under
>   whose direction the colony engaged in a number of activities,
>   primarily related to starving. -- Dave Barry, "Dave Barry Slept
Here"
>   ---------------------------------------------------------------------
>   Luke Kanies | http://reductivelabs.com | http://madstop.com
>
>
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users
>

OK, after some more tinkering and testing (and using my brain this time),
I''ve found the problem. It was a missing require on the copy exec of
the
Makefile.PL. Apparently in previous versions (0.19.3 specifically) the untar
happened before the copy, regardless. In the newer version it didn''t,
which
resulted in a copy being tried to a non-existent directory. So the
Makefile.PL never got replaced, resulting in a make exec later in the
process that was waiting for input which of course never came.

So now it seems to work (using the released 0.22.0 version on both the
client and the server).

I did however try to add the suggested ''creates'' clause, but
that threw
errors like the following:

err: Could not create wgetfile-TermReadKey-index: ''creates''
files must be
fully qualified. at /etc/puppet/manifests/functions/wgetfile.pp:31
err: ''creates'' files must be fully qualified. at
/etc/puppet/manifests/functions/wgetfile.pp:31

Which I assume means that I can''t dynamically assign the filename that
I
want created using a variable. I was using the following when these errors
popped up:

    $createsfn = $outname ?
{

       true    =>
$name,

       default =>
$source

    }

and a line with:

   creates  => $createsfn

in the exec clause that downloaded the tar file.

Any suggestions how to do this the right way?

Thijs


On 06/01/07, Thijs Oppermann <thijso+puppet@gmail.com>
wrote:
>
> About the more efficient wgetfile def: the problem is that I (in some
> cases) rename the file I get to the $name value. So if I then add a
> creates line, it will never update to a new version once a new one is
> released.
>
> I think I could adapt the def to work around that, but it would be
> easier if I can assign results of commands to variables I can use
> later. I think this comes close to what you once said earlier you did
> not want for puppet: it becomes more of a programming language?
>
> In this case it would be nice if I could just save the file using wget
> with its original filename in combination with the --no-clobber option
> so if the file already exists wget just does nothing. Assigning the
> full name of this just downloaded newest file to a variable in puppet
> (by grepping the output of the wget command) would then make the
> subsequent actions easier (untar, cd into dir, make, install, etc), as
> I would know which of the versions that are there is the newest.
>
> But anyway, as I said, there are probably ways to work around this
> already in puppet. I''ll just have to put some thinking into it..
;)
>
> Thijs
>
>
> On 05/01/07, Luke Kanies <luke@madstop.com> wrote:
> > Okay, I just got everything installed locally and tried to reproduce
> > this problem.
> >
> > However, I do not get the ''Stack level too deep''
warning.  I don''t
> > have your replacement Makefile.PL, so I don''t know if that
could
> > somehow be the problem, but using the one that comes with
> > ''everything'', I get the hang you mentioned, and when
I cancel things
> > Puppet behaves correctly.  I''ve tried it with both the
stand-alone
> > ''puppet'' executable and with
''puppetd''.
> >
> > At this point, I think I''m going to have to release without
solving
> > this problem.  I expect to put out more frequent point releases in
> > the near future as we solidify a few aspects of the recent code,
> > especially the Rails stuff, so as soon as we can reliably reproduce
> > this I''ll get a fix released.
> >
> > By the way, your wgetfile definition could be a bit more efficient if
> > you added a ''creates'' line that pointed to the
destination -- that
> > way, if the file already exists then the wget command would not be
> > run again.
> >
> >   --
> >   The leader of Jamestown was "John Smith" (not his real
name), under
> >   whose direction the colony engaged in a number of activities,
> >   primarily related to starving. -- Dave Barry, "Dave Barry Slept
Here"
> >  
---------------------------------------------------------------------
> >   Luke Kanies | http://reductivelabs.com | http://madstop.com
> >
> >
> > _______________________________________________
> > Puppet-users mailing list
> > Puppet-users@madstop.com
> > https://mail.madstop.com/mailman/listinfo/puppet-users
> >
>

_______________________________________________
Puppet-users mailing list
Puppet-users@madstop.com
https://mail.madstop.com/mailman/listinfo/puppet-users

On Jan 15, 2007, at 12:58 AM, Thijs Oppermann wrote:
> OK, after some more tinkering and testing (and using my brain this  
> time), I''ve found the problem. It was a missing require on the
copy
> exec of the Makefile.PL. Apparently in previous versions (0.19.3  
> specifically) the untar happened before the copy, regardless. In  
> the newer version it didn''t, which resulted in a copy being tried
> to a non-existent directory. So the Makefile.PL never got replaced,  
> resulting in a make exec later in the process that was waiting for  
> input which of course never came.
Huh.  I did a bit of testing with hanging execs and couldn''t  
replicate any of the problems you had other than the hanging (which  
should be fixed in 0.22, with the timeouts).
> Any suggestions how to do this the right way?
Just use a fully-qualified path for the ''creates'' location,
i.e., a
filename starting with ''/''.  Please let me know if that
doesn''t work.

  --
  A diplomat is a man who can convince his wife she''d look stout in a  
fur
  coat.
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

Great, I read this the second after I delete the test virtual machine I was
using, which had the output of the failing puppetd run in a file on it.
;)

So I have no log file to back this up, but in the output it was clear that
without the specific ''require'' statement, the untar action
(creating the
destination directory) happened *after* the copy action. Adding the
''require'' fixed it.

And maybe you didn''t get the error because you weren''t also
doing all the
other stuff I do in my configs. I never really tried to isolate the problem,
so a lot of other stuff was happening at every run.

And I''ll look into fully-qualified path thing.

Thanks,
Thijs

On 15/01/07, Luke Kanies <luke@madstop.com> wrote:
>
> On Jan 15, 2007, at 12:58 AM, Thijs Oppermann wrote:
>
> > OK, after some more tinkering and testing (and using my brain this
> > time), I''ve found the problem. It was a missing require on
the copy
> > exec of the Makefile.PL. Apparently in previous versions (0.19.3
> > specifically) the untar happened before the copy, regardless. In
> > the newer version it didn''t, which resulted in a copy being
tried
> > to a non-existent directory. So the Makefile.PL never got replaced,
> > resulting in a make exec later in the process that was waiting for
> > input which of course never came.
>
> Huh.  I did a bit of testing with hanging execs and couldn''t
> replicate any of the problems you had other than the hanging (which
> should be fixed in 0.22, with the timeouts).
>
> > Any suggestions how to do this the right way?
>
> Just use a fully-qualified path for the ''creates''
location, i.e., a
> filename starting with ''/''.  Please let me know if that
doesn''t work.
>
>   --
>   A diplomat is a man who can convince his wife she''d look stout
in a
> fur
>   coat.
>   ---------------------------------------------------------------------
>   Luke Kanies | http://reductivelabs.com | http://madstop.com
>
>
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users
>

_______________________________________________
Puppet-users mailing list
Puppet-users@madstop.com
https://mail.madstop.com/mailman/listinfo/puppet-users

On Jan 15, 2007, at 11:31 AM, Thijs Oppermann wrote:
> Great, I read this the second after I delete the test virtual  
> machine I was using, which had the output of the failing puppetd  
> run in a file on it.
> ;)
>
> So I have no log file to back this up, but in the output it was  
> clear that without the specific ''require'' statement, the
untar
> action (creating the destination directory) happened *after* the  
> copy action. Adding the ''require'' fixed it.
Sorry; I didn''t address that.

Execution order is definitely not guaranteed unless you explicitly  
specify relationships.  Puppet does a topological sort based on  
relationships, and the algorithm can get essentially random results  
for resources that have no explicit relationships.
> And maybe you didn''t get the error because you weren''t
also doing
> all the other stuff I do in my configs. I never really tried to  
> isolate the problem, so a lot of other stuff was happening at every  
> run.
I did try the exact perl setup you had with the Everything2 package,  
but I''m sure there were other variables in play.

  --
  To my embarrassment I was born in bed with a lady.
              --Wilson Mizner
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com