Puppet users - Apr 2010 - How to bring about serialization in puppet ?

Hi,
    I''m trying to manage some hosts using a single puppetmaster. I
don''t know why but puppet on clients seems to be executing the recipes
hosted on puppetmaster in a random order which is breaking apart
dependencies and resulting in a failed run of puppet for first time.
If I do multiple run of puppet thru puppetd --test, everything gets
installed and configured properly. However it''s very rare that on
first run I can see puppet managing the configuration and installation
without fail.
   Is there any way I can bring about serial execution of puppet
recipes. What I mean by serial execution is that suppose I''ve a
site.pp with following content:

package {"ruby-devel": ensure=>latest}
package {"rubygems": ensure=>latest}
exec {"install-mysql-gem":
         command=>''gem install mysql'',
         path=>"/bin:/usr/bin:/usr/sbin:/sbin",
         require=>[ Package["ruby-devel"],
Package["rubygems"]]
}

Now in many cases puppet tries to execute Exec["install-mysql-gem"]
before Package["ruby-devel"] or Package["rubygems"] or both.

Is there a way that I can ensure that puppet renders the file in order
the script is written ?

BTW I''m using CentOS 5.4 with puppet-server-0.24.5-1.el5 and
puppet-0.24.5-1.el5.

Regards,
Mayank

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

> Is there a way that I can ensure that puppet renders the file in order
> the script is written ?
No. And that''s not a bug it''s (really!) a feature.

If you have dependencies and they matter, then define (via  
require,before,subscribe,notify) and hence _document_ them. If there  
are no dependencies, the order doesn''t matter.

cheers pete

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

and:
> Now in many cases puppet tries to execute
Exec["install-mysql-gem"]
> before Package["ruby-devel"] or Package["rubygems"] or
both.
no that shouldn''t be the case. if you define such a require, both  
packages are installed _before_ the exec is executed.

Are you really sure that it is the way you explain here?

cheers pete

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

> Now in many cases puppet tries to execute
Exec["install-mysql-gem"]
> before Package["ruby-devel"] or Package["rubygems"] or
both.
>
> Is there a way that I can ensure that puppet renders the file in order
> the script is written ?
Like Pete says, that is what ''require'' is supposed to do. Can
you run
"puppetd --verbose --debug --test" (when you expect puppet to fail)
and post the output?

Mike

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thu, Apr 15, 2010 at 6:10 PM, mike_puppet@fadedink.co.uk <
mike@fadedink.co.uk> wrote:
>
> > Now in many cases puppet tries to execute
Exec["install-mysql-gem"]
> > before Package["ruby-devel"] or
Package["rubygems"] or both.
> >
> > Is there a way that I can ensure that puppet renders the file in order
> > the script is written ?
>
> Like Pete says, that is what ''require'' is supposed to do.
Can you run
> "puppetd --verbose --debug --test" (when you expect puppet to
fail)
> and post the output?
>
> Mike
>
>
I''ll re run puppetd after I''m done with setting up some more
recipes later
during the day. Will post the output for sure in case I get into trouble.
Thanks for the heads up.

Regards,

--
Mayank

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Apr 15, 2010, at 1:30 AM, Mayank wrote:
> Hi,
>    I''m trying to manage some hosts using a single puppetmaster. I
> don''t know why but puppet on clients seems to be executing the
recipes
> hosted on puppetmaster in a random order which is breaking apart
> dependencies and resulting in a failed run of puppet for first time.
> If I do multiple run of puppet thru puppetd --test, everything gets
> installed and configured properly. However it''s very rare that on
> first run I can see puppet managing the configuration and installation
> without fail.
>   Is there any way I can bring about serial execution of puppet
> recipes. What I mean by serial execution is that suppose I''ve a
> site.pp with following content:
> 
> package {"ruby-devel": ensure=>latest}
> package {"rubygems": ensure=>latest}
> exec {"install-mysql-gem":
>         command=>''gem install mysql'',
>         path=>"/bin:/usr/bin:/usr/sbin:/sbin",
>         require=>[ Package["ruby-devel"],
Package["rubygems"]]
> }
> 
> Now in many cases puppet tries to execute
Exec["install-mysql-gem"]
> before Package["ruby-devel"] or Package["rubygems"] or
both.
> 
> Is there a way that I can ensure that puppet renders the file in order
> the script is written ?
> 
> BTW I''m using CentOS 5.4 with puppet-server-0.24.5-1.el5 and
> puppet-0.24.5-1.el5.
> 
> Regards,
> Mayank
I posted about this exact thing yesterday (Subj: require inside define), only
you actually got replies. :)

In my case, it seems as though there''s something deep in the heart of
Puppet that always causes ''realize''d resources (User, in this
case) to be promoted to the top of the task list.  The User line gets
eval''d before everything else regardless of how many
''require''s I put in front of it.

One difference, though, is that I''m using 0.25.4 from the EPEL repo.

-Jim

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Thu, Apr 15, 2010 at 8:44 AM, Jim Bala <usrbin@gmail.com>
wrote:
> On Apr 15, 2010, at 1:30 AM, Mayank wrote:
>
>> Hi,
>>    I''m trying to manage some hosts using a single
puppetmaster. I
>> don''t know why but puppet on clients seems to be executing the
recipes
>> hosted on puppetmaster in a random order which is breaking apart
>> dependencies and resulting in a failed run of puppet for first time.
>> If I do multiple run of puppet thru puppetd --test, everything gets
>> installed and configured properly. However it''s very rare that
on
>> first run I can see puppet managing the configuration and installation
>> without fail.
>>   Is there any way I can bring about serial execution of puppet
>> recipes. What I mean by serial execution is that suppose I''ve
a
>> site.pp with following content:
>>
>> package {"ruby-devel": ensure=>latest}
>> package {"rubygems": ensure=>latest}
>> exec {"install-mysql-gem":
>>         command=>''gem install mysql'',
>>         path=>"/bin:/usr/bin:/usr/sbin:/sbin",
>>         require=>[ Package["ruby-devel"],
Package["rubygems"]]
>> }
>>
>> Now in many cases puppet tries to execute
Exec["install-mysql-gem"]
>> before Package["ruby-devel"] or Package["rubygems"]
or both.
>>
>> Is there a way that I can ensure that puppet renders the file in order
>> the script is written ?
>>
>> BTW I''m using CentOS 5.4 with puppet-server-0.24.5-1.el5 and
>> puppet-0.24.5-1.el5.
Keep at it. You''ll get it all working with requires=> eventually. It
will take forever, your head will hurt, and by the time your done,
you''ll have a god-aweful dependency mess that will make you totally
afraid to touch any of it ever again, but you''ll get it eventually if
you keep at it! I did!

Doug.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Douglas Garstang <doug.garstang@gmail.com> writes:
> On Thu, Apr 15, 2010 at 8:44 AM, Jim Bala <usrbin@gmail.com> wrote:
>> On Apr 15, 2010, at 1:30 AM, Mayank wrote:
>>
>>> I''m trying to manage some hosts using a single
puppetmaster. I don''t know
>>> why but puppet on clients seems to be executing the recipes hosted
on
>>> puppetmaster in a random order which is breaking apart dependencies
and
>>> resulting in a failed run of puppet for first time.
[...]
>>> package {"ruby-devel": ensure=>latest}
>>> package {"rubygems": ensure=>latest}
>>> exec {"install-mysql-gem":
>>>         command=>''gem install mysql'',
>>>         path=>"/bin:/usr/bin:/usr/sbin:/sbin",
>>>         require=>[ Package["ruby-devel"],
Package["rubygems"]]
>>> }
>>>
>>> Now in many cases puppet tries to execute
Exec["install-mysql-gem"]
>>> before Package["ruby-devel"] or
Package["rubygems"] or both.
...if you specify the ''require'' in your
''exec'', and puppet tries to run it
before the packages are installed you have found a puppet bug.

I strongly suspect from ...
>>> Is there a way that I can ensure that puppet renders the file in
order the
>>> script is written ?
... this that you don''t have that dependency, however, and are looking
for a
shortcut way to specify the dependencies through ordering of the input file.

[...]
> Keep at it. You''ll get it all working with requires=>
eventually. It will
> take forever, your head will hurt, and by the time your done,
you''ll have a
> god-aweful dependency mess that will make you totally afraid to touch any
of
> it ever again, but you''ll get it eventually if you keep at it! I
did!
...and here I would have to say that, Doug, you are probably getting something
wrong somewhere if the dependencies in your system setup are that complex.[1]

Generally, your dependency chains shouldn''t be enormously complex or
deep in
puppet, and there are several tricks that make it easy to express complex
dependencies quickly.

One of the big ones is a Class dependency:

    # Shouldn''t I be using the OS package for this?
    exec { "install-mysql-gem":
        command => "/sbin/do-magic",
        require => Class["ruby"]
    }

Then, inside class ruby you install the dependent packages:

    class ruby { package ["ruby-devel", "rubygems"]: ensure
=> latest }

That ensures that *all* the stuff in class ''ruby'' is done
before the exec for
your gems.  Nice, short, and encapsulated.

Then, if you build appropriate classes you can have very short, specific
requirements that can express complex — and transparently changing —
requirements through your codebase.

(For example: using the ''class ruby'' structure you can easily
implement the
 Debian version of it, and the mysql gem thing doesn''t need change at
all.)

Regards,
        Daniel

Footnotes: 
[1]  ...that, or maybe you work somewhere that does have enormously complex
     dependency sequences across your network, and they cannot be simplified
     in any way including changing the network. ;)

-- 
✣ Daniel Pittman            ✉ daniel@rimspace.net            ☎ +61 401 155 707
               ♽ made with 100 percent post-consumer electrons

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Sat, Apr 17, 2010 at 6:21 PM, Douglas Garstang
<doug.garstang@gmail.com> wrote:
> On Thu, Apr 15, 2010 at 8:44 AM, Jim Bala <usrbin@gmail.com> wrote:
>> On Apr 15, 2010, at 1:30 AM, Mayank wrote:
>>
>>> Hi,
>>>    I''m trying to manage some hosts using a single
puppetmaster. I
>>> don''t know why but puppet on clients seems to be executing
the recipes
>>> hosted on puppetmaster in a random order which is breaking apart
>>> dependencies and resulting in a failed run of puppet for first
time.
>>> If I do multiple run of puppet thru puppetd --test, everything gets
>>> installed and configured properly. However it''s very rare
that on
>>> first run I can see puppet managing the configuration and
installation
>>> without fail.
>>>   Is there any way I can bring about serial execution of puppet
>>> recipes. What I mean by serial execution is that suppose
I''ve a
>>> site.pp with following content:
>>>
>>> package {"ruby-devel": ensure=>latest}
>>> package {"rubygems": ensure=>latest}
>>> exec {"install-mysql-gem":
>>>         command=>''gem install mysql'',
>>>         path=>"/bin:/usr/bin:/usr/sbin:/sbin",
>>>         require=>[ Package["ruby-devel"],
Package["rubygems"]]
>>> }
>>>
>>> Now in many cases puppet tries to execute
Exec["install-mysql-gem"]
>>> before Package["ruby-devel"] or
Package["rubygems"] or both.
>>>
>>> Is there a way that I can ensure that puppet renders the file in
order
>>> the script is written ?
>>>
>>> BTW I''m using CentOS 5.4 with puppet-server-0.24.5-1.el5
and
>>> puppet-0.24.5-1.el5.
>
> Keep at it. You''ll get it all working with requires=>
eventually. It
> will take forever, your head will hurt, and by the time your done,
> you''ll have a god-aweful dependency mess that will make you
totally
> afraid to touch any of it ever again, but you''ll get it eventually
if
> you keep at it! I did!
So we went there with the more complicated parts of our config, and
then I came back to sanity, with enforcing this really really simple
rule on all our commits. As intra-class require/before statements can
only refer to individual resources. (by definition), the rule is that:

Inter-class require/before statements can only refer to whole classes,
never ever individual resources within those classes.


I mean this is all good programming practice, but it''s not something
people always keep in mind when putting out an ops fire and checking
fixes in, but it really makes your dependency situation so much
cleaner. You have well defined interfaces between objects, and you''re
free to change internal behavior as you need to without being afraid
to touch stuff.

>
> Doug.
>
> --
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
>
>


-- 
nigel

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Sun, Apr 18, 2010 at 8:45 AM, Nigel Kersten <nigelk@google.com> wrote:
> On Sat, Apr 17, 2010 at 6:21 PM, Douglas Garstang
> <doug.garstang@gmail.com> wrote:
> > On Thu, Apr 15, 2010 at 8:44 AM, Jim Bala <usrbin@gmail.com>
wrote:
> >> On Apr 15, 2010, at 1:30 AM, Mayank wrote:
> >>
> >>> Hi,
> >>>    I''m trying to manage some hosts using a single
puppetmaster. I
> >>> don''t know why but puppet on clients seems to be
executing the recipes
> >>> hosted on puppetmaster in a random order which is breaking
apart
> >>> dependencies and resulting in a failed run of puppet for first
time.
> >>> If I do multiple run of puppet thru puppetd --test, everything
gets
> >>> installed and configured properly. However it''s very
rare that on
> >>> first run I can see puppet managing the configuration and
installation
> >>> without fail.
> >>>   Is there any way I can bring about serial execution of
puppet
> >>> recipes. What I mean by serial execution is that suppose
I''ve a
> >>> site.pp with following content:
> >>>
> >>> package {"ruby-devel": ensure=>latest}
> >>> package {"rubygems": ensure=>latest}
> >>> exec {"install-mysql-gem":
> >>>         command=>''gem install mysql'',
> >>>         path=>"/bin:/usr/bin:/usr/sbin:/sbin",
> >>>         require=>[ Package["ruby-devel"],
Package["rubygems"]]
> >>> }
> >>>
> >>> Now in many cases puppet tries to execute
Exec["install-mysql-gem"]
> >>> before Package["ruby-devel"] or
Package["rubygems"] or both.
> >>>
> >>> Is there a way that I can ensure that puppet renders the file
in order
> >>> the script is written ?
> >>>
> >>> BTW I''m using CentOS 5.4 with
puppet-server-0.24.5-1.el5 and
> >>> puppet-0.24.5-1.el5.
> >
> > Keep at it. You''ll get it all working with requires=>
eventually. It
> > will take forever, your head will hurt, and by the time your done,
> > you''ll have a god-aweful dependency mess that will make you
totally
> > afraid to touch any of it ever again, but you''ll get it
eventually if
> > you keep at it! I did!
>
> So we went there with the more complicated parts of our config, and
> then I came back to sanity, with enforcing this really really simple
> rule on all our commits. As intra-class require/before statements can
> only refer to individual resources. (by definition), the rule is that:
>
> Inter-class require/before statements can only refer to whole classes,
> never ever individual resources within those classes.
>
>
> I mean this is all good programming practice, but it''s not
something
> people always keep in mind when putting out an ops fire and checking
> fixes in, but it really makes your dependency situation so much
> cleaner. You have well defined interfaces between objects, and
you''re
> free to change internal behavior as you need to without being afraid
> to touch stuff.
>
>
> >
> > Doug.
> >
> >
> >
>
>
>
> --
> nigel
>
>
I can''t take chances with re running puppet as what I am trying to
accomplish is complete automation of a setup on EC2. I''ve a script
which
fires up new instances on EC2. These instances are built using custom AMI
(machine images) which have entry for puppetmaster in /etc/hosts and have
puppet installed on ''em. I am running puppetd --test thru rc.local and
since
my puppetmaster is configured to autosign, hence I need this first run of
puppetd --test to get the job done. As entire setup is automated so I
can''t
wait for multiple runs of puppet before everything gets installed properly.
I''ve tried to make a dependency tree which should work out for me and
will
be testing it in next 20 min .... Will revert back with any updates in case
something fails this time... I need this first run of puppet to run without
fail for me... Praying to God this time to make it a success :)

Regards,
-- 
Mayank

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Sun, Apr 18, 2010 at 1:47 PM, Mayank <mail2mayank@gmail.com> wrote:
> On Sun, Apr 18, 2010 at 8:45 AM, Nigel Kersten <nigelk@google.com>
wrote:
>
>> On Sat, Apr 17, 2010 at 6:21 PM, Douglas Garstang
>> <doug.garstang@gmail.com> wrote:
>> > On Thu, Apr 15, 2010 at 8:44 AM, Jim Bala <usrbin@gmail.com>
wrote:
>> >> On Apr 15, 2010, at 1:30 AM, Mayank wrote:
>> >>
>> >>> Hi,
>> >>>    I''m trying to manage some hosts using a single
puppetmaster. I
>> >>> don''t know why but puppet on clients seems to be
executing the recipes
>> >>> hosted on puppetmaster in a random order which is breaking
apart
>> >>> dependencies and resulting in a failed run of puppet for
first time.
>> >>> If I do multiple run of puppet thru puppetd --test,
everything gets
>> >>> installed and configured properly. However it''s
very rare that on
>> >>> first run I can see puppet managing the configuration and
installation
>> >>> without fail.
>> >>>   Is there any way I can bring about serial execution of
puppet
>> >>> recipes. What I mean by serial execution is that suppose
I''ve a
>> >>> site.pp with following content:
>> >>>
>> >>> package {"ruby-devel": ensure=>latest}
>> >>> package {"rubygems": ensure=>latest}
>> >>> exec {"install-mysql-gem":
>> >>>         command=>''gem install mysql'',
>> >>>        
path=>"/bin:/usr/bin:/usr/sbin:/sbin",
>> >>>         require=>[ Package["ruby-devel"],
Package["rubygems"]]
>> >>> }
>> >>>
>> >>> Now in many cases puppet tries to execute
Exec["install-mysql-gem"]
>> >>> before Package["ruby-devel"] or
Package["rubygems"] or both.
>> >>>
>> >>> Is there a way that I can ensure that puppet renders the
file in order
>> >>> the script is written ?
>> >>>
>> >>> BTW I''m using CentOS 5.4 with
puppet-server-0.24.5-1.el5 and
>> >>> puppet-0.24.5-1.el5.
>> >
>> > Keep at it. You''ll get it all working with requires=>
eventually. It
>> > will take forever, your head will hurt, and by the time your done,
>> > you''ll have a god-aweful dependency mess that will make
you totally
>> > afraid to touch any of it ever again, but you''ll get it
eventually if
>> > you keep at it! I did!
>>
>> So we went there with the more complicated parts of our config, and
>> then I came back to sanity, with enforcing this really really simple
>> rule on all our commits. As intra-class require/before statements can
>> only refer to individual resources. (by definition), the rule is that:
>>
>> Inter-class require/before statements can only refer to whole classes,
>> never ever individual resources within those classes.
>>
>>
>> I mean this is all good programming practice, but it''s not
something
>> people always keep in mind when putting out an ops fire and checking
>> fixes in, but it really makes your dependency situation so much
>> cleaner. You have well defined interfaces between objects, and
you''re
>> free to change internal behavior as you need to without being afraid
>> to touch stuff.
>>
>>
>> >
>> > Doug.
>> >
>> >
>> >
>>
>>
>>
>> --
>> nigel
>>
>>
> I can''t take chances with re running puppet as what I am trying to
> accomplish is complete automation of a setup on EC2. I''ve a script
which
> fires up new instances on EC2. These instances are built using custom AMI
> (machine images) which have entry for puppetmaster in /etc/hosts and have
> puppet installed on ''em. I am running puppetd --test thru rc.local
and since
> my puppetmaster is configured to autosign, hence I need this first run of
> puppetd --test to get the job done. As entire setup is automated so I
can''t
> wait for multiple runs of puppet before everything gets installed properly.
> I''ve tried to make a dependency tree which should work out for me
and will
> be testing it in next 20 min .... Will revert back with any updates in case
> something fails this time... I need this first run of puppet to run without
> fail for me... Praying to God this time to make it a success :)
>
> Regards,
> --
> Mayank
>
Found an error:

Following is my puppet recipe:

class hadoopbase{
        exec {"rubygems-install":
                command => ''wget
http://rubyforge.org/frs/download.php/69365/rubygems-1.3.6.tgz && tar
-zxvf
rubygems-1.3.6.tgz && cd rubygems-1.3.6 && ruby setup.rb
&& cd .. && rm -rf
rubygems-1.3.6* && gem update --system'',
                path =>
"/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin",
        }
        exec {"updateyum":
                command => ''yum update -y'',
                path =>
"/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin",
        }
        package {"ruby-devel":
                        ensure => latest,
                        require => Exec["rubygems-install"],
        }
        package {"coreutils": ensure => latest}
        package {"subversion": ensure => latest}
        package {"GeoIP-devel": ensure => latest}
        exec {"downloadGeolite":
                command => ''cd /root && wget
http://10.214.42.7/GeoLiteCity.dat'',
                path =>
"/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin",
        }
        exec {"right_aws_gem_install":
                command => ''gem install --source
http://gems.rubyforge.org--version "1.10.0" right_aws'',
                path => "/bin:/usr/bin:/sbin:/usr/sbin",
                require => Package["ruby-devel"],
        }
        exec {"addressable_install":
                command => ''gem install --source
http://gems.rubyforge.org--version "2.1.1" addressable'',
                path => "/bin:/usr/bin:/sbin:/usr/sbin",
                require => Package["ruby-devel"],
        }
        exec {"mime_types":
                 command => ''gem install --source
http://gems.rubyforge.org--version "1.16" mime-types'',
                path => "/bin:/usr/bin:/sbin:/usr/sbin",
                require => Package["ruby-devel"],
        }
        file {"/tmp/net-geoip":
                source => "puppet:///files/hadoop/gems/net-geoip",
                recurse => true,
                ensure => present,
        }
        exec { "net-geoip-install":
                command => ''cd /tmp/net-geoip && gem build
net-geoip.gemspec
&& gem install net-geoip-0.0.76.gem'',
                path =>
"/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin",
                require => [ Package["ruby-devel"],
Package["GeoIP-devel"],
File["/tmp/net-geoip"] ],
        }
}

During first run of puppet thru rc.local I got following error in
/var/log/messages

Apr 18 04:11:55 ip-10-250-107-162 puppetd[1812]:
(//hadooprole/hadoopbase/Package[ruby-devel]/ensure) change from absent to
latest failed: Could not update: Execution of ''/usr/bin/yum -d 0 -e 0
-y
install ruby-devel'' returned 1: Error: Missing Dependency: ruby-libs
1.8.6.110-2.fc8 is needed by package ruby-devel#012 at
/etc/puppet/manifests/classes/hadoopbase.pp:13

However when I checked for ruby-libs package I found:

[root@ip-10-250-107-162 ~]# rpm -qa | grep ruby
ruby-1.8.6.114-1.fc8
ruby-irb-1.8.6.114-1.fc8
rubygems-0.9.4-1.fc8
ruby-libs-1.8.6.114-1.fc8
ruby-rdoc-1.8.6.114-1.fc8

Then I did re-run of puppet thru puppetd --test and it ran without fail and
now I''ve this

[root@ip-10-250-107-162 ~]# rpm -qa | grep ruby
ruby-libs-1.8.6.287-2.fc8
ruby-rdoc-1.8.6.287-2.fc8
ruby-shadow-1.4.1-10.fc8
ruby-1.8.6.287-2.fc8
ruby-devel-1.8.6.287-2.fc8
rubygems-1.2.0-2.fc8
ruby-irb-1.8.6.287-2.fc8

Any ideas as to why ruby-devel was not able to install properly during first
run ?
-- 
Mayank

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Apr 18, 2010, at 1:17 AM, Mayank wrote:
> On Sun, Apr 18, 2010 at 8:45 AM, Nigel Kersten <nigelk@google.com>
wrote:
> On Sat, Apr 17, 2010 at 6:21 PM, Douglas Garstang
> <doug.garstang@gmail.com> wrote:
> > On Thu, Apr 15, 2010 at 8:44 AM, Jim Bala <usrbin@gmail.com>
wrote:
> >> On Apr 15, 2010, at 1:30 AM, Mayank wrote:
> >>
> >>> Hi,
> >>>    I''m trying to manage some hosts using a single
puppetmaster. I
> >>> don''t know why but puppet on clients seems to be
executing the recipes
> >>> hosted on puppetmaster in a random order which is breaking
apart
> >>> dependencies and resulting in a failed run of puppet for first
time.
> >>> If I do multiple run of puppet thru puppetd --test, everything
gets
> >>> installed and configured properly. However it''s very
rare that on
> >>> first run I can see puppet managing the configuration and
installation
> >>> without fail.
> >>>   Is there any way I can bring about serial execution of
puppet
> >>> recipes. What I mean by serial execution is that suppose
I''ve a
> >>> site.pp with following content:
> >>>
> >>> package {"ruby-devel": ensure=>latest}
> >>> package {"rubygems": ensure=>latest}
> >>> exec {"install-mysql-gem":
> >>>         command=>''gem install mysql'',
> >>>         path=>"/bin:/usr/bin:/usr/sbin:/sbin",
> >>>         require=>[ Package["ruby-devel"],
Package["rubygems"]]
> >>> }
> >>>
> >>> Now in many cases puppet tries to execute
Exec["install-mysql-gem"]
> >>> before Package["ruby-devel"] or
Package["rubygems"] or both.
> >>>
> >>> Is there a way that I can ensure that puppet renders the file
in order
> >>> the script is written ?
> >>>
> >>> BTW I''m using CentOS 5.4 with
puppet-server-0.24.5-1.el5 and
> >>> puppet-0.24.5-1.el5.
> >
> > Keep at it. You''ll get it all working with requires=>
eventually. It
> > will take forever, your head will hurt, and by the time your done,
> > you''ll have a god-aweful dependency mess that will make you
totally
> > afraid to touch any of it ever again, but you''ll get it
eventually if
> > you keep at it! I did!
> 
> So we went there with the more complicated parts of our config, and
> then I came back to sanity, with enforcing this really really simple
> rule on all our commits. As intra-class require/before statements can
> only refer to individual resources. (by definition), the rule is that:
> 
> Inter-class require/before statements can only refer to whole classes,
> never ever individual resources within those classes.
> 
> 
> I mean this is all good programming practice, but it''s not
something
> people always keep in mind when putting out an ops fire and checking
> fixes in, but it really makes your dependency situation so much
> cleaner. You have well defined interfaces between objects, and
you''re
> free to change internal behavior as you need to without being afraid
> to touch stuff.
> 
> 
> >
> > Doug.
> >
> > 
> >
> 
> 
> 
> --
> nigel
> 
> 
> I can''t take chances with re running puppet as what I am trying to
accomplish is complete automation of a setup on EC2. I''ve a script
which fires up new instances on EC2. These instances are built using custom AMI
(machine images) which have entry for puppetmaster in /etc/hosts and have puppet
installed on ''em. I am running puppetd --test thru rc.local and since
my puppetmaster is configured to autosign, hence I need this first run of
puppetd --test to get the job done. As entire setup is automated so I
can''t wait for multiple runs of puppet before everything gets installed
properly. I''ve tried to make a dependency tree which should work out
for me and will be testing it in next 20 min .... Will revert back with any
updates in case something fails this time... I need this first run of puppet to
run without fail for me... Praying to God this time to make it a success :)
If you need it to work that bad, you could probably put more than one run in
your initialization script.  Also, if you send the second log somewhere, you can
improve it over time.  You''ll probably want to make it runs with the
cache though if you''re worried about performance.  Remember that
"--test" removes caching.  I use "puppetd --onetime --verbose
--no-daemonize".
-Patrick Mohr

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 18/04/2010, Douglas Garstang <doug.garstang@gmail.com>
wrote:
> On Thu, Apr 15, 2010 at 8:44 AM, Jim Bala <usrbin@gmail.com> wrote:
>> On Apr 15, 2010, at 1:30 AM, Mayank wrote:
>>
>>> Hi,
>>>    I''m trying to manage some hosts using a single
puppetmaster. I
>>> don''t know why but puppet on clients seems to be executing
the recipes
>>> hosted on puppetmaster in a random order which is breaking apart
>>> dependencies and resulting in a failed run of puppet for first
time.
>>> If I do multiple run of puppet thru puppetd --test, everything gets
>>> installed and configured properly. However it''s very rare
that on
>>> first run I can see puppet managing the configuration and
installation
>>> without fail.
>>>   Is there any way I can bring about serial execution of puppet
>>> recipes. What I mean by serial execution is that suppose
I''ve a
>>> site.pp with following content:
>>>
>>> package {"ruby-devel": ensure=>latest}
>>> package {"rubygems": ensure=>latest}
>>> exec {"install-mysql-gem":
>>>         command=>''gem install mysql'',
>>>         path=>"/bin:/usr/bin:/usr/sbin:/sbin",
>>>         require=>[ Package["ruby-devel"],
Package["rubygems"]]
>>> }
>>>
>>> Now in many cases puppet tries to execute
Exec["install-mysql-gem"]
>>> before Package["ruby-devel"] or
Package["rubygems"] or both.
>>>
>>> Is there a way that I can ensure that puppet renders the file in
order
>>> the script is written ?
>>>
>>> BTW I''m using CentOS 5.4 with puppet-server-0.24.5-1.el5
and
>>> puppet-0.24.5-1.el5.
>
> Keep at it. You''ll get it all working with requires=>
eventually. It
> will take forever, your head will hurt, and by the time your done,
> you''ll have a god-aweful dependency mess that will make you
totally
> afraid to touch any of it ever again, but you''ll get it eventually
if
> you keep at it! I did!
>
> Doug.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
-- 
Sent from my mobile device

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Mayank <mail2mayank@gmail.com> writes:
> On Sun, Apr 18, 2010 at 1:47 PM, Mayank <mail2mayank@gmail.com>
wrote:
>> On Sun, Apr 18, 2010 at 8:45 AM, Nigel Kersten
<nigelk@google.com> wrote:
[...]
> Found an error:
>
> Following is my puppet recipe:
[...]
> During first run of puppet thru rc.local I got following error in
/var/log/messages
>
> Apr 18 04:11:55 ip-10-250-107-162 puppetd[1812]:
(//hadooprole/hadoopbase/Package
> [ruby-devel]/ensure) change from absent to latest failed: Could not update:
Execution of
> ''/usr/bin/yum -d 0 -e 0 -y install ruby-devel'' returned
1: Error: Missing Dependency:
> ruby-libs = 1.8.6.110-2.fc8 is needed by package ruby-devel#012 at
/etc/puppet/manifests
> /classes/hadoopbase.pp:13
>
> However when I checked for ruby-libs package I found:
[... that rpm knows about it ...]
> Then I did re-run of puppet thru puppetd --test and it ran without fail and
> now I''ve this
[...]
> Any ideas as to why ruby-devel was not able to install properly during
first
> run?
Nope.  I can tell you right now that it isn''t a puppet problem,
though.[1]

The problem is that puppet executed ...

    /usr/bin/yum -d 0 -e 0 -y install ruby-devel

... and yum returned an error, claiming that it couldn''t find a
dependency of
the package.

Fix your yum setup and puppet will start working.  (Maybe you need something
to run a ''yum update'' or something like that?)

        Daniel

Footnotes: 
[1]  Technically, it may be puppet *related* problem, since the way puppet
     invokes the yum command, or the scheduling of it, may cause the failure
     by way of some side-effect.

-- 
✣ Daniel Pittman            ✉ daniel@rimspace.net            ☎ +61 401 155 707
               ♽ made with 100 percent post-consumer electrons

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Apr 18, 2010, at 1:55 AM, Daniel Pittman wrote:
>> Any ideas as to why ruby-devel was not able to install properly during
first
>> run?
> 
> Nope.  I can tell you right now that it isn''t a puppet problem,
though.[1]
> 
> The problem is that puppet executed ...
> 
>    /usr/bin/yum -d 0 -e 0 -y install ruby-devel
> 
> ... and yum returned an error, claiming that it couldn''t find a
dependency of
> the package.
> 
> Fix your yum setup and puppet will start working.  (Maybe you need
something
> to run a ''yum update'' or something like that?)
> 
>        Daniel
> 
> Footnotes: 
> [1]  Technically, it may be puppet *related* problem, since the way puppet
>     invokes the yum command, or the scheduling of it, may cause the failure
>     by way of some side-effect.
As a temporary workaround, try adding an entry to install ruby-libs between
ruby-gems-install and ruby-devel.  I say temporary because this looks like a bug
that will cause you other problems down the road if you don''t find the
root cause.

Here''s one way to diagnose this.  Try running puppet on a new system in
debug mode.  Then run those same commands yourself on a new system replacing -d
0 -e 0 with -d 8 -e 1.  Hopefully someone will have an easier idea.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Sun, Apr 18, 2010 at 2:39 PM, Patrick <kc7zzv@gmail.com> wrote:
>
> On Apr 18, 2010, at 1:55 AM, Daniel Pittman wrote:
>
> >> Any ideas as to why ruby-devel was not able to install properly
during
> first
> >> run?
> >
> > Nope.  I can tell you right now that it isn''t a puppet
problem,
> though.[1]
> >
> > The problem is that puppet executed ...
> >
> >    /usr/bin/yum -d 0 -e 0 -y install ruby-devel
> >
> > ... and yum returned an error, claiming that it couldn''t find
a
> dependency of
> > the package.
> >
> > Fix your yum setup and puppet will start working.  (Maybe you need
> something
> > to run a ''yum update'' or something like that?)
> >
> >        Daniel
> >
> > Footnotes:
> > [1]  Technically, it may be puppet *related* problem, since the way
> puppet
> >     invokes the yum command, or the scheduling of it, may cause the
> failure
> >     by way of some side-effect.
>
> As a temporary workaround, try adding an entry to install ruby-libs between
> ruby-gems-install and ruby-devel.  I say temporary because this looks like
a
> bug that will cause you other problems down the road if you don''t
find the
> root cause.
>
> Here''s one way to diagnose this.  Try running puppet on a new
system in
> debug mode.  Then run those same commands yourself on a new system
replacing
> -d 0 -e 0 with -d 8 -e 1.  Hopefully someone will have an easier idea.
>
>
>
Hi,
    I changed my puppet recipe a bit and here it is :

class hadoopbase{
        exec {"rubygems-install":
                command => ''wget
http://rubyforge.org/frs/download.php/69365/rubygems-1.3.6.tgz && tar
-zxvf
rubygems-1.3.6.tgz && cd rubygems-1.3.6 && ruby setup.rb
&& cd .. && rm -rf
rubygems-1.3.6* && gem update --system'',
                path =>
"/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin",
        }
        exec {"updateyum":
                command => ''yum update -y'',
                path =>
"/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin",
        }
        package {"ruby-devel":
                        ensure => latest,
                        require => [ Exec["updateyum"],
Exec["rubygems-install"] ],
        }
        package {"coreutils":
                        ensure => latest,
                        require => Exec["updateyum"],
        }
        package {"subversion":
                        ensure => latest,
                        require => Exec["updateyum"],
        }
        package {"GeoIP-devel":
                        ensure => latest,
                        require => Exec["updateyum"],
        }
        exec {"downloadGeolite":
                command => ''cd /root && wget
http://10.214.42.7/GeoLiteCity.dat'',
                path =>
"/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin",
        }
        exec {"right_aws_gem_install":
                command => ''gem install --source
http://gems.rubyforge.org--version "1.10.0" right_aws'',
                path => "/bin:/usr/bin:/sbin:/usr/sbin",
                require => Package["ruby-devel"],
        }
        exec {"addressable_install":
                command => ''gem install --source
http://gems.rubyforge.org--version "2.1.1" addressable'',
                path => "/bin:/usr/bin:/sbin:/usr/sbin",
                require => Package["ruby-devel"],
        }
        exec {"mime_types":
                 command => ''gem install --source
http://gems.rubyforge.org--version "1.16" mime-types'',
                path => "/bin:/usr/bin:/sbin:/usr/sbin",
                require => Package["ruby-devel"],
        }
        file {"/tmp/net-geoip":
                source => "puppet:///files/hadoop/gems/net-geoip",
                recurse => true,
                ensure => present,
                require => [ Package["ruby-devel"],
Package["GeoIP-devel"] ]
        }
        exec { "net-geoip-install":
                command => ''cd /tmp/net-geoip && gem build
net-geoip.gemspec
&& gem install net-geoip-0.0.76.gem'',
                path =>
"/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin",
                require => File["/tmp/net-geoip"],
        }
}

Now I am able to get puppet work for me in very first run. I had to do a
require => Exec["updateyum"] for every package installation in
order to
ensure that the command yum update -y ran before package is being installed.
However this brings me to my initial question. Since I''d mentioned
exec{"updateyum": ....} before any package statement so had puppet
some way
of ensuring serial execution of script I would have been saved from
mentioning it every time in package statements.

-- 
Mayank

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
> Now I am able to get puppet work for me in very first run. I had to do a
> require => Exec["updateyum"] for every package installation in
order to
> ensure that the command yum update -y ran before package is being
installed.
> However this brings me to my initial question. Since I''d mentioned
> exec{"updateyum": ....} before any package statement so had
puppet some way
> of ensuring serial execution of script I would have been saved from
> mentioning it every time in package statements.
you can shorten that by either wrap your package installation with a
define, or define at the top:

Package{ require => Exec["updateyum"] }

which will introduce a global dependency on the Package resource on that
exec. Note: afair if you set another dependency you will have to reset
it. But it looks like this will be once for your setup.

cheers pete
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkvK/6oACgkQbwltcAfKi38bOgCeJQ90OrRpX2KDbG+CbpUI6/yn
34UAoLJsRRztfPQmBdneIzdKNBb3Ouwf
=UIeP
-----END PGP SIGNATURE-----

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Sun, Apr 18, 2010 at 6:18 PM, Peter Meier <peter.meier@immerda.ch>
wrote:
>
>
> > Now I am able to get puppet work for me in very first run. I had to do
a
> > require => Exec["updateyum"] for every package
installation in order to
> > ensure that the command yum update -y ran before package is being
> installed.
> > However this brings me to my initial question. Since I''d
mentioned
> > exec{"updateyum": ....} before any package statement so had
puppet some
> way
> > of ensuring serial execution of script I would have been saved from
> > mentioning it every time in package statements.
>
> you can shorten that by either wrap your package installation with a
> define, or define at the top:
>
> Package{ require => Exec["updateyum"] }
>
> which will introduce a global dependency on the Package resource on that
> exec. Note: afair if you set another dependency you will have to reset
> it. But it looks like this will be once for your setup.
>
> cheers pete
>
Thanks I''ll wrap it up with a define and check it out again. Thanks to
everyone for valuable advises. I guess I''ll be creating some better
recipes
in coming days thanks to these advises :)

Regards,
--
Mayank

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Mayank <mail2mayank@gmail.com> writes:
> On Sun, Apr 18, 2010 at 6:18 PM, Peter Meier <peter.meier@immerda.ch>
wrote:
>> > Now I am able to get puppet work for me in very first run. I had
to do a
>> > require => Exec["updateyum"] for every package
installation in order to
>> > ensure that the command yum update -y ran before package is being
>> > installed.  However this brings me to my initial question. Since
I''d
>> > mentioned exec{"updateyum": ....} before any package
statement so had
>> > puppet some way of ensuring serial execution of script I would
have been
>> > saved from mentioning it every time in package statements.
>> 
>> you can shorten that by either wrap your package installation with a
define,
>> or define at the top:
>> 
>> Package{ require => Exec["updateyum"] }
>> 
>> which will introduce a global dependency on the Package resource on
that
>> exec. Note: afair if you set another dependency you will have to reset
>> it. But it looks like this will be once for your setup.
>
> Thanks I''ll wrap it up with a define and check it out again.
Thanks to
> everyone for valuable advises. I guess I''ll be creating some
better recipes
> in coming days thanks to these advises :)
I would strongly advise that you use the global dependency, since that avoids
the need to have a non-standard define that acts almost like the standard
package resource.

If you have packages that you require before the package manager update, just
declare them with an explicit empty ''require'' attribute and
you will override
the global dependency.

        Daniel

-- 
✣ Daniel Pittman            ✉ daniel@rimspace.net            ☎ +61 401 155 707
               ♽ made with 100 percent post-consumer electrons

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.