Puppet users - Sep 2007 - Re: #786: exported resources not refreshed

Hello all,
what exactly is the "design decision" that is needed here?
We currently have the problem that 0.23.2-3 (debian package) does not 
work when it comes to collecting exported resources. The resources are 
saved in the database, so picking them up is the problem (db is Mysql5.x).

This is a major problem for us, since we are using this a lot for our 
monitoring services. I currently do have the time to write a nagios 
module (and already started doing so). Since virtual resources do not 
work it is impossible for me to finish the module.

I have tried the most recent version of puppet from git, it does not 
work. I have tried applying the patches, mentioned here: 
http://mail.madstop.com/pipermail/puppet-users/2007-August/004252.html
Same result.

It would be nice if someone with the proper insight could describe the 
actual problem.

- do I need puppetmaster AND puppetd from git to make virtual resources 
work again?
- will this be problematic if used in a production environment?
- what and where are the relevant files/classes regarding the collecting 
  of virtual resources.

Thanks for the effort,
udo.


puppet wrote:
> #786: exported resources not refreshed
>
-----------------------------------------------------+----------------------
>      Reporter:  vvidic                               |           Owner: 
luke
>          Type:  defect                               |          Status: 
new
>      Priority:  normal                               |       Milestone: 
elmo
>     Component:  server                               |         Version: 
0.23.2
>      Severity:  normal                               |      Resolution:
>      Keywords:  puppetmaster, rails, stored configs  |           Stage: 
Needs design decision
>      Approval:  Unnecessary                          |           Patch: 
None
>    Complexity:  Unknown                              |   Compatibility: 
Unknown
> Specification:  Unnecessary                          |  
>
-----------------------------------------------------+----------------------
> Changes (by michael):
> 
>   * stage:  Unreviewed => Needs design decision
>   * milestone:  => elmo
>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tuesday 11 September 2007, Udo Waechter wrote:
> This is a major problem for us, since we are using this a lot for our
> monitoring services. I currently do have the time to write a nagios
> module (and already started doing so). Since virtual resources do not
> work it is impossible for me to finish the module.
Please take a look at 
http://git.black.co.at/?p=manifests.git;a=blob;f=manifests/classes/nagios.pp;h=77da63bf241532075bd75be2669b9611f4615880;hb=HEAD

Where I already have quite a bit of nagios infrastructure, only not yet 
converted to a module.


Regards, David
- -- 
The primary freedom of open source is not the freedom from cost, but the free-
dom to shape software to do what you want. This freedom is /never/ exercised
without cost, but is available /at all/ only by accepting the very different
costs associated with open source, costs not in money, but in time and effort.
- -- http://www.schierer.org/~luke/log/20070710-1129/on-forks-and-forking
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFG5lif/Pp1N6Uzh0URAgT4AJ9fg9yCkkGfHtK7i/RGqm5XnuSnpwCfVam3
+NVDn/FoTWSWil0LXTvv2F8=YWzH
-----END PGP SIGNATURE-----

Hey David,
thanks for the lnk. I was looking for this. I had in mind that I already 
saw this some while ago.
I will use the stuff you wrote to write the module, unfortunately this 
won''t be possible until the collection of virtual resources does not 
work again.

Bye,
udo.

David Schmitt wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Tuesday 11 September 2007, Udo Waechter wrote:
>> This is a major problem for us, since we are using this a lot for our
>> monitoring services. I currently do have the time to write a nagios
>> module (and already started doing so). Since virtual resources do not
>> work it is impossible for me to finish the module.
> 
> Please take a look at 
>
http://git.black.co.at/?p=manifests.git;a=blob;f=manifests/classes/nagios.pp;h=77da63bf241532075bd75be2669b9611f4615880;hb=HEAD
> 
> Where I already have quite a bit of nagios infrastructure, only not yet 
> converted to a module.

On Sep 11, 2007, at 2:49 AM, Udo Waechter wrote:
> Hello all,
> what exactly is the "design decision" that is needed here?
> We currently have the problem that 0.23.2-3 (debian package) does not
> work when it comes to collecting exported resources. The resources are
> saved in the database, so picking them up is the problem (db is  
> Mysql5.x).
The decision needed is how best to indicate to the client that  
exported resources have changed and thus the configuration has  
changed and thus it should recompile.

Considering that I''m in the middle of refactoring how the client gets  
its configuration, including how that configuration is refreshed,  
it''s awkward to attempt to make this decision right now.  I really am  
trying though, I promise.
> This is a major problem for us, since we are using this a lot for our
> monitoring services. I currently do have the time to write a nagios
> module (and already started doing so). Since virtual resources do not
> work it is impossible for me to finish the module.
If this is a major problem for you, then use --ignorecache to disable  
caching on the client.  This will force a recompile every time, thus  
ensuring that you always get the latest exported resources.
> I have tried the most recent version of puppet from git, it does not
> work. I have tried applying the patches, mentioned here:
> http://mail.madstop.com/pipermail/puppet-users/2007-August/004252.html
> Same result.
I assume by ''does not work'' you mean ''exhibits the
same problem'', not
that it exhibits some new problem.
> It would be nice if someone with the proper insight could describe the
> actual problem.
The problem is that the client does not realize that there is a new  
configuration, so it does not ask for a new configuration.  The  
client caches its configuration, and will prefer to use that  
configuration instead of compiling a new one.  It compares its  
compile date against the server''s claimed configuration date (which  
is based on the latest timestamp of all files used to compile the  
configuration), and if its compile date is more recent, then it does  
not recompile its configuration.

The reason it''s not refreshing is that the server''s timestamp
is not
taking into account whether exported resources have been updated in  
the database.

I''m beginning to think that it''s unreasonable to even attempt
to use
the configuration cache.
> - do I need puppetmaster AND puppetd from git to make virtual  
> resources
> work again?
I wouldn''t think so.
> - will this be problematic if used in a production environment?
I don''t think I can answer that question.  If it''s not working
in
test, and you require in production, then it seems it''ll be  
problematic in production.
> - what and where are the relevant files/classes regarding the  
> collecting
>   of virtual resources.
I''m not sure it will help in this case, because your problem (AFAICT)  
is that the client is just not recompiling its configuration, so it''s  
not a question of export or collect, it''s a question of whether the  
client should recompile.

That being said, the lib/puppet/parser/collector.rb file does all of  
the actual collection.

  --
  The leader of Jamestown was "John Smith" (not his real name), under
  whose direction the colony engaged in a number of activities,
  primarily related to starving. -- Dave Barry, "Dave Barry Slept
Here"
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

Hello Luke,
thanks for the verbose answer. I was gone for a while, thus my late 
reaction.

Luke Kanies wrote:
> On Sep 11, 2007, at 2:49 AM, Udo Waechter wrote: 
>> This is a major problem for us, since we are using this a lot for our
>> monitoring services. I currently do have the time to write a nagios
>> module (and already started doing so). Since virtual resources do not
>> work it is impossible for me to finish the module.
> 
> If this is a major problem for you, then use --ignorecache to disable  
> caching on the client.  This will force a recompile every time, thus  
> ensuring that you always get the latest exported resources.
This does not work, sorry. If I use --ignorecache, the exported 
resources are still not imported on the host that should have them.
The client says "Ignoring cache".

Should the client recognise a new virtual resource even if this was 
saved a while ago?
If /etc/nagios2/conf.d/XYZ does not exist, it should appear? If I delete 
it afterwards, it should appear again? I would say, yes.
>> I have tried the most recent version of puppet from git, it does not
>> work. I have tried applying the patches, mentioned here:
>> http://mail.madstop.com/pipermail/puppet-users/2007-August/004252.html
>> Same result.
> I assume by ''does not work'' you mean ''exhibits
the same problem'', not
> that it exhibits some new problem.
Yes.

Thanks again for the answers, although the problem still persist :(

Bye,
udo.

On Sep 17, 2007, at 5:29 AM, Udo Waechter wrote:
>
> Luke Kanies wrote:
>> If this is a major problem for you, then use --ignorecache to disable
>> caching on the client.  This will force a recompile every time, thus
>> ensuring that you always get the latest exported resources.
> This does not work, sorry. If I use --ignorecache, the exported
> resources are still not imported on the host that should have them.
> The client says "Ignoring cache".
So it''s apparently not a caching issue.
> Should the client recognise a new virtual resource even if this was
> saved a while ago?
> If /etc/nagios2/conf.d/XYZ does not exist, it should appear? If I  
> delete
> it afterwards, it should appear again? I would say, yes.
Virtual resources only exist in the parser; assuming you''ve realized  
the resource, it should behave like any other.
>> I assume by ''does not work'' you mean
''exhibits the same problem'', not
>> that it exhibits some new problem.
> Yes.
>
> Thanks again for the answers, although the problem still persist :(
Is there currently an open ticket that describes this problem  
sufficiently?  If not, will you open one?  If so, can you let me know  
if it''s not marked for fixing in misspiggy?

  --
  DOS is the _only_ operating system -- and I''m using that term
  loosely -- which [exhibits this behavior].
             --Aeleen Frisch, "Essential System Administration"
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

Hello.

Luke Kanies wrote:
> Is there currently an open ticket that describes this problem  
> sufficiently?  If not, will you open one?  If so, can you let me know  
> if it''s not marked for fixing in misspiggy?
I thought that ticket #786 would be the one which describes this 
problem. I honestly do not know how to describe this problem further.
There are other tickets which provide patches that should solve this:

http://mail.madstop.com/pipermail/puppet-users/2007-August/004252.html

I think there are enough tickets dealing with this problem by now. I do 
not want to sit here and wait for the problem to be solved, since most 
our remaining tasks regarding our configuration at the institute 
involves modules that use exported modules. I really would like to help.

Unfortunately I do not know very much about debugging Puppet(master) and 
or Ruby. Where can I find documentation on this?
How do you developers debug puppet? I do not mean the "--debug"
option,
I mean a real debugger with stepping through the code and all that.

Thanks,
udo.

On Tue, Sep 25, 2007 at 09:53:26AM +0200, Udo Waechter
wrote:
> I think there are enough tickets dealing with this problem by now. I do 
> not want to sit here and wait for the problem to be solved, since most 
> our remaining tasks regarding our configuration at the institute 
> involves modules that use exported modules. I really would like to help.
What kind of a resource are you trying to export? Can you create a small
example showing the problem?
> Unfortunately I do not know very much about debugging Puppet(master) and 
> or Ruby. Where can I find documentation on this?
> How do you developers debug puppet? I do not mean the "--debug"
option,
> I mean a real debugger with stepping through the code and all that.
I usually run both puppetmaster and puppetd in foreground to get the
error message. Then I try to find the message in the source and add a
few more printouts to understand what happens. To debug exported
resources it is also useful to monitor SQL queries.

-- 
Valentin

Hello.
Valentin Vidic wrote:
> On Tue, Sep 25, 2007 at 09:53:26AM +0200, Udo Waechter wrote:
>> I think there are enough tickets dealing with this problem by now. I do
>> not want to sit here and wait for the problem to be solved, since most 
>> our remaining tasks regarding our configuration at the institute 
>> involves modules that use exported modules. I really would like to
help.
> 
> What kind of a resource are you trying to export? Can you create a small
> example showing the problem?
I want to export a file type. Here is a example from the nagios-module:

define nagios_contact (
                 $contact_name,
                 $contact_alias,
                 $service_notification_period = "24x7",
                 $host_notification_period = "24x7",
                 $service_notification_options = "w,u,c,r",
                 $host_notification_options = "d,u,r",
                 $service_notification_commands = "notify-by-email",
                 $host_notification_commands = "host-notify-by-email",
                 $email= "",
                 $pager = "",
                 $contact_groups = "")
{
   info ( "Storing nagios contact: ${contact_name}" )
   @@file { "nagios_contact_${contact_name}":
     path=>"${NAGIOSCONFDIR}/contact_${contact_name}",
     ensure => present,
     content => template("nagios/contact_template.erb"),
     tag => "nagios",

   }

}

Then, in another class I want to read that file:

class nagios::host{

         $NAGIOSCONFDIR="/etc/nagios2/conf.d"
         info ( "Storing nagios configurations in: ${NAGIOSCONFDIR}" )
         file { $NAGIOSCONFDIR:
               ensure => directory,
               require => Package["nagios2"],
         }

         package { ["nagios2",
                 "nagios-images",
                 "nagios-nrpe-plugin",
                 "nagios-plugins",
                 "nagios-plugins-basic",
                 "nagios-plugins-standard",
                 "nagios2-common",
                 "nagiosgrapher"]: ensure => present }

	#collect all nagios_ definitions
         #File <<| title == nagios_ |>> #this does not work :(
         File <<||>>
}

Similar things are done in my munin-module, same thing happens there, 
namely nothing.
> 
>> Unfortunately I do not know very much about debugging Puppet(master)
and
>> or Ruby. Where can I find documentation on this?
>> How do you developers debug puppet? I do not mean the
"--debug" option,
>> I mean a real debugger with stepping through the code and all that.
> 
> I usually run both puppetmaster and puppetd in foreground to get the
> error message. Then I try to find the message in the source and add a
> few more printouts to understand what happens. To debug exported
> resources it is also useful to monitor SQL queries.
Well that is somewhat problematic, time consuming and not really that 
what I meant. I would like to see a description where someone has done 
debugging in lets say Eclipse. The ruby debugger there is pretty nice 
and well working. Maybe I''ll start to set up an environment to do this.

Have a nice day,
udo.

On Tue, Sep 25, 2007 at 10:45:48AM +0200, Udo Waechter
wrote:
> Hello.
> Valentin Vidic wrote:
> > On Tue, Sep 25, 2007 at 09:53:26AM +0200, Udo Waechter wrote:
> >> Unfortunately I do not know very much about debugging
Puppet(master) and
> >> or Ruby. Where can I find documentation on this?
> >> How do you developers debug puppet? I do not mean the
"--debug" option,
> >> I mean a real debugger with stepping through the code and all
that.
> > 
> > I usually run both puppetmaster and puppetd in foreground to get the
> > error message. Then I try to find the message in the source and add a
> > few more printouts to understand what happens. To debug exported
> > resources it is also useful to monitor SQL queries.
>
> Well that is somewhat problematic, time consuming
Don''t knock it ''til you''ve tried it.
> and not really that 
> what I meant. I would like to see a description where someone has done 
> debugging in lets say Eclipse.
Now if you want to talk about problematic and time consuming...
> The ruby debugger there is pretty nice 
> and well working. Maybe I''ll start to set up an environment to do
this.
Depends on which "ruby debugger" you''re talking about.  The
more-or-less
inbuilt one is pretty half-assed, IMO, and I don''t know too many people
who
are particularly keen on it.  Now, ruby-debug is a far more interesting
proposition, as far as an interactive debugger goes.

However, much of the time it''s still a pest to work with.  Sure, you
can set
breakpoints and examine your variables wherever you want, but here''s a
scary
thing -- you can do much the same thing with a couple of puts statements. 
Interactive debuggers are far more useful in compiled languages where it
takes you 10 minutes to go from save to run, but in Ruby it''s far less
useful, because it''s so damned easy to just make a quick change and
re-run.
Hell, in probably 99% of cases you''re far better off writing some more
unit
tests to illustrate what you''re trying to accomplish, which will (a)
improve
test coverage, which is a forever benefit, and (b) identify where the
problem is pretty quickly as your tests narrow down the places where the
issue *can''t* be.

I''ve fired up a debugger (and considered it a win) in Ruby exactly
once, and
that was a fairly esoteric case, that involved actually running a bunch of
hand-written code to see exactly what was needed inside the deep state of my
program.

As to my debugging environment: I''ve got a directory that contains a
dummy
puppet.conf and all of the usual suspect directories -- $confdir, $vardir,
etc.  That''s what I''d call the "harness" out of
which everything else runs.
When I''m coding, I''ve got an XTerm, running screen, with
about 6 tabs.  One with a puppetmasterd --debug --trace, another in which I
run puppetd --debug --trace, a third that''s probably editing
puppet.conf or
some other part of my harness (a manifest, for instance), then several tabs
containing various chunks of code that I''m working on.  I might have an
irb
tab and/or an ri tab, for experimentation and looking up classes and
methods, if I need it.

GUIs?  They''re just things that keep my xterm windows arranged nicely
on
screen, aren''t they?

- Matt

Hi,

Udo Waechter wrote:
> Unfortunately I do not know very much about debugging Puppet(master) and 
> or Ruby. Where can I find documentation on this?
> How do you developers debug puppet? I do not mean the "--debug"
option,
> I mean a real debugger with stepping through the code and all that.
FWIW, the development (beta) version of the Netbeans IDE (6.0) has Ruby
support including API docs and a graphical debugger. The full version is
a massive 172MB, but the Ruby-only version weighs in at only 19MB:

http://bits.netbeans.org/download/6_0/beta1/latest/

It''s a Java app and I''ve made it run successfully on both
Windows and
Linux. It has a very low learning curve compared to Eclipse.

hth,

-- Bob

Hello Matt and thanks for the long answer.

Matt Palmer wrote:
> On Tue, Sep 25, 2007 at 10:45:48AM +0200, Udo Waechter wrote:
>> Hello.
>> Valentin Vidic wrote:
>>> On Tue, Sep 25, 2007 at 09:53:26AM +0200, Udo Waechter wrote:
>>>> Unfortunately I do not know very much about debugging
Puppet(master) and
>>>> or Ruby. Where can I find documentation on this?
>>>> How do you developers debug puppet? I do not mean the
"--debug" option,
>>>> I mean a real debugger with stepping through the code and all
that.
>>> I usually run both puppetmaster and puppetd in foreground to get
the
>>> error message. Then I try to find the message in the source and add
a
>>> few more printouts to understand what happens. To debug exported
>>> resources it is also useful to monitor SQL queries.
>> Well that is somewhat problematic, time consuming 
> Don''t knock it ''til you''ve tried it.
I have been doing such things for some years now. Especially in ugly php 
due to the lack of something like a debugger.
I have done a lot of coding in Java - my mother programming language if 
i may say so - and in C even with ++.
I''ve learnt to use a debugger and i''ve learnt to use eclipse.
I do not
want to start a war about what is better and why.
All I want to say is, that I stopped using php, because it was such a 
pain in the ass to find out what was going wrong in my programs. I like 
to program Java and C(++) because of their great debugging possibilities  .
I would also like to do this with ruby, since it is also pretty nicely 
debuggable.
> However, much of the time it''s still a pest to work with.  Sure,
you can set
> breakpoints and examine your variables wherever you want, but
here''s a scary
> thing -- you can do much the same thing with a couple of puts statements. 
> Interactive debuggers are far more useful in compiled languages where it
> takes you 10 minutes to go from save to run, but in Ruby it''s far
less
> useful, because it''s so damned easy to just make a quick change
and re-run.
> Hell, in probably 99% of cases you''re far better off writing some
more unit
> tests to illustrate what you''re trying to accomplish, which will
(a) improve
> test coverage, which is a forever benefit, and (b) identify where the
> problem is pretty quickly as your tests narrow down the places where the
> issue *can''t* be.
The problem is another: If I do not know ruby very well, if I do not 
know the software very well, then i am simply not able to find out where 
something goes wrong. Puppet is nicely structured, yes. There are not 
many very long classes. Nevertheless, if I want to find out where 
something happens in the code, I first have to find out where it 
happens, right?
As soon as I have found that I can start writing tests and puts. For me 
it is easier to step through the code, than to first read a lot of code 
and guess where stuff might be. This way I also learn to know the 
software better.
> As to my debugging environment: I''ve got a directory that contains
a dummy
> puppet.conf and all of the usual suspect directories -- $confdir, $vardir,
> etc.  That''s what I''d call the "harness" out of
which everything else runs.
> When I''m coding, I''ve got an XTerm, running screen, with
> about 6 tabs.  One with a puppetmasterd --debug --trace, another in which I
> run puppetd --debug --trace, a third that''s probably editing
puppet.conf or
> some other part of my harness (a manifest, for instance), then several tabs
> containing various chunks of code that I''m working on.  I might
have an irb
> tab and/or an ri tab, for experimentation and looking up classes and
> methods, if I need it.
Well this seems to reseble my debugging setup, except that i have more 
xterms next to each other.
I also do have a xen machine, where i run a puppetclient-dev and a 
puppetmaster-dev where things are tested and developed.

Nevertheless, I do not really like it (except for the thing with the 
dev-hosts).
I think it takes me to much time to find out where certain things happen 
under puppet''s hood. Thats why I try to come up with a graphical 
debugging solution.
> 
> GUIs?  They''re just things that keep my xterm windows arranged
nicely on
> screen, aren''t they?
Well, no and yes. This is a dangerous terrain we are walking on, right?

If you want to keep your xterms nicely arranged, try wmii 
http://www.suckless.org/wiki/wmii (do print out the key shortcuts before 
you start using it, or you won''t be able todo very much)


Have a nice day,
udo.

On Sep 25, 2007, at 2:53 AM, Udo Waechter wrote:
> Unfortunately I do not know very much about debugging Puppet 
> (master) and
> or Ruby. Where can I find documentation on this?
> How do you developers debug puppet? I do not mean the "--debug"  
> option,
> I mean a real debugger with stepping through the code and all that.
Sorry to have been so silent on the thread...

I pretty much never use a debugger.  By "pretty much never", I mean  
never.

I largely rely on some combination of unit tests, print statements,  
and running the code in irb.

For debugging this, I''d modify the unit test (or create one, if it  
doesn''t exist) to capture the problem, and then keep running it as a  
I tried for the fix.

You can run individual tests by executing the test file with -n <test  
name>; e.g., for the exported test (which is unfortunately very  
long), you''d do:

cd ~/git/puppet/test
cd rails
./collection.rb -n test_collect_exported

I generally end up using prints and logs to inspect the details.

You can also add ''-d'' to the test to get all of the logging.

  --
  If there is anything the nonconformist hates worse than a conformist,
  it''s another nonconformist who doesn''t conform to the
prevailing
  standard of nonconformity.    --Bill Vaughan
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

On Tue, Sep 25, 2007 at 10:45:48AM +0200, Udo Waechter
wrote:
> Similar things are done in my munin-module, same thing happens there, 
> namely nothing.
I just tried this and it worked. File first gets exported to the mysql
database:

mysql> select * from resources where exported = 1 and
restype=''file'';
+-----+----------------------+---------+---------+----------------+----------+------+---------------------+
| id  | title                | restype | host_id | source_file_id | exported |
line | updated_at          |
+-----+----------------------+---------+---------+----------------+----------+------+---------------------+
| 171 | nagios_contact_admin | file    |       2 |             21 |        1 |  
20 | 2007-09-28 14:05:11 |
+-----+----------------------+---------+---------+----------------+----------+------+---------------------+
1 row in set (0.00 sec)


After that the nagios::host picks it up and creates the file. The only
problem is that the file gets created in / rather than /etc/nagios2/conf.d,
but that is because NAGIOSCONFDIR is not defined here:

   @@file { "nagios_contact_${contact_name}":
     path=>"${NAGIOSCONFDIR}/contact_${contact_name}",
     ensure => present,
     content => template("nagios/contact_template.erb"),
     tag => "nagios",
   }

-- 
Valentin

Hi Valentin,
sorry I did not post the whole class.
The database entry is created in my case also. It does not appear on the 
  nagios::host. I am using puppet(master) in debian version 0.23.2-10
In this version network/handler/master.rb is revision 2616 which has 
trouble with exporting resources. I think it was fixed in revision 7xxx

It seems that I will have to update my clients (at least the one who 
need to collect exported resources) to at least that revision.
Hmmm, I don''t like that thought. I guess there won''t be
another solution
for now.

One last questions: would it suffice to update the clients? Or do I also 
need the puppetmaster to be updated to the revision which fixes the 
issues with collecting resources.

btw: Valentin, which version of puppet and master are you using?

Thanks for the help,
udo.

Valentin Vidic wrote:
> On Tue, Sep 25, 2007 at 10:45:48AM +0200, Udo Waechter wrote:
>> Similar things are done in my munin-module, same thing happens there, 
>> namely nothing.
> 
> I just tried this and it worked. File first gets exported to the mysql
> database:
> 
> mysql> select * from resources where exported = 1 and
restype=''file'';
>
+-----+----------------------+---------+---------+----------------+----------+------+---------------------+
> | id  | title                | restype | host_id | source_file_id |
exported | line | updated_at          |
>
+-----+----------------------+---------+---------+----------------+----------+------+---------------------+
> | 171 | nagios_contact_admin | file    |       2 |             21 |       
1 |   20 | 2007-09-28 14:05:11 |
>
+-----+----------------------+---------+---------+----------------+----------+------+---------------------+
> 1 row in set (0.00 sec)
> 
> 
> After that the nagios::host picks it up and creates the file. The only
> problem is that the file gets created in / rather than /etc/nagios2/conf.d,
> but that is because NAGIOSCONFDIR is not defined here:
> 
>    @@file { "nagios_contact_${contact_name}":
>      path=>"${NAGIOSCONFDIR}/contact_${contact_name}",
>      ensure => present,
>      content => template("nagios/contact_template.erb"),
>      tag => "nagios",
>    }
>

On Sat, Sep 29, 2007 at 02:38:08PM +0200, Udo Waechter
wrote:
> It seems that I will have to update my clients (at least the one who 
> need to collect exported resources) to at least that revision.
> Hmmm, I don''t like that thought. I guess there won''t be
another solution
> for now.
The patches I sent were all on the puppetmaster side so maybe an update
of the clients is not needed.
> btw: Valentin, which version of puppet and master are you using?
I''m using Debian packages v0.23.2-1 with the previously mentioned
patches applied.

-- 
Valentin

On Sep 29, 2007, at 7:38 AM, Udo Waechter wrote:
> It seems that I will have to update my clients (at least the one who
> need to collect exported resources) to at least that revision.
> Hmmm, I don''t like that thought. I guess there won''t be
another
> solution
> for now.
>
> One last questions: would it suffice to update the clients? Or do I  
> also
> need the puppetmaster to be updated to the revision which fixes the
> issues with collecting resources.
You should not need to update the clients -- they don''t know anything  
about exported resources vs. normal resources.

  --
  Hegel was right when he said that we learn from history that man can
  never learn anything from history.      -- George Bernard Shaw
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

On Sat, 29 Sep 2007 14:38:08 +0200, Udo Waechter wrote:
> Hi Valentin,
> sorry I did not post the whole class. The database entry is created in
> my case also. It does not appear on the
>   nagios::host. I am using puppet(master) in debian version 0.23.2-10
> In this version network/handler/master.rb is revision 2616 which has
> trouble with exporting resources. I think it was fixed in revision 7xxx
I''m having this same problem, exported resources are not collected, and
I''m running the same versions and --ignorecache doesn''t do it
for me, and
I was not clear on what the solution was?

What was the solution to this problem? Is the solution to downgrade 
puppetmaster to 0.23.2-1 and apply the patches mentioned here:
http://mail.madstop.com/pipermail/puppet-users/2007-August/004252.html
?

This thread sort of devolved into a obtuse debate about if Eclipse was a 
good debugger or if a GUI was worth anything other than an xterm 
arranger. Worthy subjects to take place over rounds of beer, but I''d
like
to know the solution to my problem before I have any.

(OB ruby debugger comment: http://unroller.rubyforge.org)

micah

On Sat, 06 Oct 2007 23:26:52 +0000, micah wrote:
> I''m having this same problem, exported resources are not
collected, and
> I''m running the same versions and --ignorecache doesn''t
do it for me,
> and I was not clear on what the solution was?
The solution for me was to apply 

http://reductivelabs.com/trac/puppet/attachment/ticket/786/freshness.diff

Hello Micah,
now things become really, really weird for me:

I have puppetmaster and puppet version 0.23.2-10 on Debian Etch.
Like you mentioned below, I have applied the freshness.diff patch still 
no change for me.

Did you use 0.23.2-1?

Here comes the really strange part:
As I mentioned some posts earlier I want to write this nagios module, 
and am already using the munin module.

Here is what happenes:

The host which should get the munin-client-defintions gets only its own, 
none of the other! The same thing happens with the nagios::host. It gets 
its own munin-definition but none of the nagios definitions.
This is soemwhat clear, since both modules use "File
<<||>>" to import
exported resources. Thus, all exported ressources show up on all 
importing clients.

As for debugging: I put some "Puppet.info" lines in 
network/handler/master.rb in lines 69,90 and 92 but they never show up 
on the console of the puppetmaster.
It seems to me that the freshnes function is actually never called.
Not even on those runs when the above mentioned definitions are created 
on the clients!

What is going on there?

As I also mentioned earlier, the munin module worked fine for a long 
while, then simply stopped working. I did not change anything with this 
module, thus I am pretty sure that the munin-client definitions should 
show up again, as soon as I delete them from the munin::host

Can someone please tell me under which circumstances the
''freshness''
function is actually called? Shouldn''t this be on each run?

I have used ''--ignorecache'' on both of the clients and master.
I have also reinstalled puppet and -master on the puppetmaster-machine 
before applying the patch above.

Thanks for the answers and sorry for all the trouble.
Udo.


micah wrote:
> On Sat, 06 Oct 2007 23:26:52 +0000, micah wrote:
> 
>> I''m having this same problem, exported resources are not
collected, and
>> I''m running the same versions and --ignorecache
doesn''t do it for me,
>> and I was not clear on what the solution was?
> 
> The solution for me was to apply 
> 
> http://reductivelabs.com/trac/puppet/attachment/ticket/786/freshness.diff
> 
> 
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users
>

On Oct 8, 2007, at 7:13 AM, Udo Waechter wrote:
> Can someone please tell me under which circumstances the
''freshness''
> function is actually called? Shouldn''t this be on each run?
''freshness'' is called on the server by the client every time
the
client runs.  It''s used by the client to determine if the client  
should recompile.

Is there any chance that your times are out of sync between client  
and server, or maybe aren''t sync''ed on timezones?  The current
release uses local time for each machine, so the client could think  
that its version is more recent based on timezone differences.  Yes,  
this is a bug, and it''s fixed in git.

I''m sorry this is so difficult to debug; you should clearly be seeing  
''freshness'' being checked on every run for every client, and
if
you''re not, then something really, really weird is going on.

  --
  The Number 1 Sign You Have Nothing to Do at Work...
     The 4th Division of Paperclips has overrun the Pushpin Infantry
     and General White-Out has called for a new skirmish.
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

Luke Kanies wrote:
> ''freshness'' is called on the server by the client every
time the
> client runs.  It''s used by the client to determine if the client  
> should recompile.
hm. i haven''t read anything prior to this reply itt, so my apologies if
my message is redundant.

i have noticed times when certain changes to my manifests do not get 
propagated to the client. i have even experienced outright errors from 
puppetd. removing the localconfig.yaml and state .yaml (can''t remember 
its name) resolved the issue.

-scott

On Oct 9, 2007, at 12:04 AM, Scott Smith wrote:
> i have noticed times when certain changes to my manifests do not get
> propagated to the client. i have even experienced outright errors from
> puppetd. removing the localconfig.yaml and state .yaml (can''t
remember
> its name) resolved the issue.
It''s always difficult to get this exactly right -- knowing when any  
data required by the configuration has changed.

The bug that makes freshness timezone-dependent is especially bad.

This should all be both clearer and more correct in the next release.

  --
  "They called me mad, and I called them mad, and damn them, they
  outvoted me." -- Nathaniel Lee, on being consigned to a mental
  institution, circa 17th c.
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

Hi Luke,
On 09.10.2007, at 06:08, Luke Kanies wrote:
> On Oct 8, 2007, at 7:13 AM, Udo Waechter wrote:
>
>> Can someone please tell me under which circumstances the
''freshness''
>> function is actually called? Shouldn''t this be on each run?
>
> ''freshness'' is called on the server by the client every
time the
> client runs.  It''s used by the client to determine if the client
> should recompile.
>
Well that''s what I thought too, but still it does not happen.
> Is there any chance that your times are out of sync between client
> and server, or maybe aren''t sync''ed on timezones?  The
current
> release uses local time for each machine, so the client could think
> that its version is more recent based on timezone differences.  Yes,
> this is a bug, and it''s fixed in git.
>
The rerource importing clients had a libc6 from debian-unstable for  
some reason unknown to me. Their times and timezones are and were in  
sync though. We use our local ntp-server throughout the whole network  
and according to our munin, times are never out of sync. At least not  
more than some microseconds.
> I''m sorry this is so difficult to debug; you should clearly be
seeing
> ''freshness'' being checked on every run for every client,
and if
> you''re not, then something really, really weird is going on.
I am sorry, but this does not happen. Almost never. It happened once  
or twice for one or two hosts. Unfortunately I can not find out from  
which of the hosts this came, since most of the clients run at the  
same time. Thus the syslog entries are mixed up.

The strange thing is now that only the first output from the first  
line of the freshness function shows up.
Either of both outputs form the lines

88 		if defined? @interpreter
89                 bla = [exportdate.to_i, @interpreter.parsedate].max
90                 Puppet.info "=================EXPORTDATE IS: "+bla
91                 return bla
92             else
93                 Puppet.info "===========NO EXPORTDATE============="
94                 return 0
95             end

do not show up, which can not be!

Something is really really weird here....
Bye,
udo.

-- 
---[ Institute of Cognitive Science @ University of Osnabrueck
---[ Albrechtstrasse 28, D-49076 Osnabrueck, 969-3362
---[ Eyes: http://www.zoide.net/
---[ Ears: http://www.auriculabovinari.de/

On 09.10.2007, at 07:04, Scott Smith wrote:
> Luke Kanies wrote:
>> ''freshness'' is called on the server by the client
every time the
>> client runs.  It''s used by the client to determine if the
client
>> should recompile.
>
> hm. i haven''t read anything prior to this reply itt, so my  
> apologies if
> my message is redundant.
>
> i have noticed times when certain changes to my manifests do not get
> propagated to the client. i have even experienced outright errors from
> puppetd. removing the localconfig.yaml and state .yaml (can''t
remember
> its name) resolved the issue.
nope, deleting the .yaml files from the state-dir did not help.

udo.
-- 
---[ Institute of Cognitive Science @ University of Osnabrueck
---[ Albrechtstrasse 28, D-49076 Osnabrueck, 969-3362
---[ Eyes: http://www.zoide.net/
---[ Ears: http://www.auriculabovinari.de/

On Oct 9, 2007, at 2:01 PM, udo waechter wrote:
> Hi Luke,
> On 09.10.2007, at 06:08, Luke Kanies wrote:
>>
>> ''freshness'' is called on the server by the client
every time the
>> client runs.  It''s used by the client to determine if the
client
>> should recompile.
>>
> Well that''s what I thought too, but still it does not happen.
Then you need to debug why it''s not getting called on the clients.

At this point, I think the only possible solution is for you to ping  
me on irc and go through interactive debugging.  It''s clear that  
sending emails around isn''t working.
> I am sorry, but this does not happen. Almost never. It happened once
> or twice for one or two hosts. Unfortunately I can not find out from
> which of the hosts this came, since most of the clients run at the
> same time. Thus the syslog entries are mixed up.
>
> The strange thing is now that only the first output from the first
> line of the freshness function shows up.
> Either of both outputs form the lines
>
> 88 		if defined? @interpreter
> 89                 bla = [exportdate.to_i, @interpreter.parsedate].max
> 90                 Puppet.info "=================EXPORTDATE IS:
"+bla
> 91                 return bla
> 92             else
> 93                 Puppet.info "===========NO
EXPORTDATE============="
> 94                 return 0
> 95             end
>
> do not show up, which can not be!
Like I said, time to debug on the clients, then.

Really, though, it''s time to do this interactively, because something  
retarded is happening, something that both of us are missing somehow.

  --
  I went to a restaurant that serves "breakfast at anytime".  So I
  ordered French Toast during the Renaissance.     -- Stephen Wright
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com