[Puppet Users] Re: Exported/Collect resource not working?

Brice Figureau
2008-07-31T11:50:12Z

Hi,

It''s not in my habit to reply to myself, but I clearly think collected
resources are not working anymore in 0.24.x (including 0.24.5).

On Wed, 2008-07-30 at 17:39 +0200, Brice Figureau wrote:
> Hum, what''s wrong?
> Any idea or work-around?

I tried on a complete verbatim environment using debian 0.24.5-2 (both
puppetmaster and puppets) with the following simpler site.pp:

define deftest ()
{
   warning("DEFTEST ${fqdn} on ${name}")
   file { "${name}":
     content => "test"
   }
}

node "xen1.internal" {
  $part = "xen1"

  @@deftest {
     "/tmp/${part}":
       tag => "test"
  }
  Deftest <<| tag==''test'' |>>
}

node "xen2.internal" {
  $part = "xen2"

  @@deftest {
    "/tmp/${part}":
       tag => "test"
  }
  Deftest <<| tag==''test'' |>>
}

Note that as soon as I get rid of deftest define and use directly the
File resource, it is working as intended.

Then running puppetd on xen1.internal, no /tmp/xen1 is created although
the puppetmaster says:
info: Caching node for xen1.internal
debug: Creating default schedules
debug:
/Settings[/etc/puppet/puppet.conf]/Settings[puppetmasterd]/File[/etc/puppet/manifests/site.pp]:
Autorequiring File[/etc/puppet/manifests]
debug: Finishing transaction 23957021037080 with 0 changes
debug: Adding code to main on line 82 in file /etc/puppet/manifests/site.pp
debug: Scope(Node[xen1.internal]): Collected 1 Deftest resource in 0.01 seconds
warning: Scope(Deftest[/tmp/xen1]): DEFTEST xen1.internal on /tmp/xen1
debug: Scope(Node[xen1.internal]): Collected 0 Deftest resources in 0.00 seconds
debug: File[/tmp/xen1]: Adding default for backup
debug: File[/tmp/xen1]: Adding default for ignore
info: Stored catalog for xen1.internal in 1.05 seconds

And on xen1.internal after retrieving the plugins, it just says:
info: Loading fact netmask
info: Loading fact ldap_auth_bind_user
info: Loading fact configured_ntp_servers
debug: Retrieved facts in 9.27 seconds
debug: Retrieving catalog
debug: Calling puppetmaster.getconfig
debug: Retrieved catalog in 1.65 seconds
debug: Creating default schedules
info: Caching catalog at /var/lib/puppet/state/localconfig.yaml
notice: Starting catalog run
debug: Finishing transaction -614176518 with 0 changes
debug: Storing state
info: Creating state file /var/lib/puppet/state/state.yaml
debug: Stored state in 0.06 seconds
notice: Finished catalog run in 0.09 seconds

And the database contains the collected resource:
+-------+-----------+---------+---------+----------------+----------+------+---------------------+
| id    | title     | restype | host_id | source_file_id | exported | line |
updated_at          |
+-------+-----------+---------+---------+----------------+----------+------+---------------------+
| 51558 | /tmp/xen1 | File    |      68 |              1 |        1 |   58 |
2008-07-31 10:19:40 |
| 51560 | /tmp/xen1 | Deftest |      68 |              1 |        1 |   69 |
2008-07-31 10:19:40 |
+-------+-----------+---------+---------+----------------+----------+------+---------------------+

Now running puppet on xen2 produce the collected /tmp/xen1 file:
info: Loading fact configured_ntp_servers
info: Loading fact ldap_auth_bind_user
info: Loading fact netmask
info: Retrieving plugins
info: Loading fact configured_ntp_servers
info: Loading fact ldap_auth_bind_user
info: Loading fact netmask
info: Caching catalog at /var/lib/puppet/state/localconfig.yaml
notice: Starting catalog run
warning: //Node[xen2.internal]/Deftest[/tmp/xen1]/File[/tmp/xen1]/checksum: File
/tmp/xen1 does not exist -- cannot checksum
notice: //Node[xen2.internal]/Deftest[/tmp/xen1]/File[/tmp/xen1]/checksum:
defined ''checksum'' as
''{md5}3d09baddc21a365b7da5ae4d0aa5cb95''
notice: //Node[xen2.internal]/Deftest[/tmp/xen1]/File[/tmp/xen1]/content:
created file with contents {md5}3d09baddc21a365b7da5ae4d0aa5cb95
notice: Finished catalog run in 1.17 seconds

And on the puppetmaster:
info: Expiring the node cache of xen2.internal
info: Not using expired node for xen2.internal from cache; expired at
Thu Jul 31 10:24:20 +0200 2008
info: Caching node for xen2.internal
debug: Scope(Node[xen2.internal]): Collected 2 Deftest resources in 0.01
seconds
warning: Scope(Deftest[/tmp/xen2]): DEFTEST xen2.internal on /tmp/xen2
warning: Scope(Deftest[/tmp/xen1]): DEFTEST xen2.internal on /tmp/xen1
debug: Scope(Node[xen2.internal]): Collected 0 Deftest resources in 0.00
seconds
debug: File[/tmp/xen2]: Adding default for backup
debug: File[/tmp/xen2]: Adding default for ignore
debug: File[/tmp/xen1]: Adding default for backup
debug: File[/tmp/xen1]: Adding default for ignore
debug: File[/var/local]: Adding default for backup
debug: File[/var/local]: Adding default for ignore
info: Stored catalog for xen2.internal in 0.43 seconds
notice: Compiled catalog for xen2.internal in 0.54 seconds

Which if I''m not mistaken means that both resources are known by the
puppetmaster.

in the database:
+-------+-----------+---------+---------+----------------+----------+------+---------------------+
| id    | title     | restype | host_id | source_file_id | exported | line |
updated_at          |
+-------+-----------+---------+---------+----------------+----------+------+---------------------+
| 51563 | /tmp/xen2 | File    |      69 |              1 |        1 |   58 |
2008-07-31 10:25:21 |
| 51565 | /tmp/xen2 | Deftest |      69 |              1 |        1 |   79 |
2008-07-31 10:25:21 |
+-------+-----------+---------+---------+----------------+----------+------+---------------------+

Now back to xen1, running puppet again produce the /tmp/xen2 file but
still not the xen1 file. With this simpler manifest I wasn''t able to
reproduce the original error I reported (ie the duplicate resource), but
I think it is just another view of the same issue.

I filed this issue under bug #1472:
http://reductivelabs.com/redmine/issues/show/1472

Does anybody uses exported/collected resource successfully ?
Does any one who know puppet internals can point me to the code that
handle collected resource? I''d like to throw an eye on that and
instrument the code to see if I can find the problem.

Thanks,