Puppet users - Jan 2007 - scope issue after upgrade

I just upgraded to 0.22.0 from 0.18.4 and I''m running into what I think
are problems due to changes in scope rules.

In my site.pp manifest, I have the following:
import "classes/*"

node nfsserver {
   include nfsserver
}

node webserver {
   include webserver
}

class cluster1_sites {
	website {
		"example.com": ...;
		"example2.com": ...;
	}
}

node www01,www02 inherits webserver {
   include cluster1_sites
}

node fs01 inherits nfsserver {
   include cluster1_sites
}

The idea is that new websites can be added to the cluster1_sites class,
and the right changes will be made to the web servers and nfs servers.

So, classes/webserver.pp contains:
class webserver {
	service { lighttpd: ensure => running }
}

and classes/website.pp contains:
define website() {
   case tagged(webserver) {
      true: {
			file { "/config/path/$name": ..., notify => Service[lighttpd] }
		}
	}
   case tagged(nfsserver) {
      true: {
			...
		}
	}
}

The problem seems to be that the lighttpd service can no longer be
found:
err: Could not apply complete configuration: Could not retrieve dependency
''Service[lighttpd]'' at
/etc/puppet/manifests/classes/website.pp:26

Any ideas on how I get this working again?

Thanks,
Christian

On Jan 17, 2007, at 3:47 PM, Christian G. Warden wrote:
> I just upgraded to 0.22.0 from 0.18.4 and I''m running into what I
> think
> are problems due to changes in scope rules.
>
<snip>
> So, classes/webserver.pp contains:
> class webserver {
> 	service { lighttpd: ensure => running }
> }
>
> and classes/website.pp contains:
> define website() {
>    case tagged(webserver) {
>       true: {
> 			file { "/config/path/$name": ..., notify =>
Service[lighttpd] }
> 		}
> 	}
>    case tagged(nfsserver) {
>       true: {
> 			...
> 		}
> 	}
> }
Unless I''m just missing something, it seems to me that you just need  
to add "include webserver" at the top of website.pp.

-Blake

On Wed, Jan 17, 2007 at 05:19:49PM -0800, Blake Barnett
wrote:
> 
> On Jan 17, 2007, at 3:47 PM, Christian G. Warden wrote:
> 
> > I just upgraded to 0.22.0 from 0.18.4 and I''m running into
what I
> > think
> > are problems due to changes in scope rules.
> >
> <snip>
> 
> > So, classes/webserver.pp contains:
> > class webserver {
> > 	service { lighttpd: ensure => running }
> > }
> >
> > and classes/website.pp contains:
> > define website() {
> >    case tagged(webserver) {
> >       true: {
> > 			file { "/config/path/$name": ..., notify =>
Service[lighttpd] }
> > 		}
> > 	}
> >    case tagged(nfsserver) {
> >       true: {
> > 			...
> > 		}
> > 	}
> > }
> 
> Unless I''m just missing something, it seems to me that you just
need
> to add "include webserver" at the top of website.pp.
Not every node that contains websites is a webserver: the nfsservers,
for example.  I tried putting "include webserver" within "case
tagged(webserver)", but that didn''t solve the problem.

Christian

On Jan 17, 2007, at 5:28 PM, Christian G. Warden wrote:
> On Wed, Jan 17, 2007 at 05:19:49PM -0800, Blake Barnett wrote:
>>
>>
>> Unless I''m just missing something, it seems to me that you
just need
>> to add "include webserver" at the top of website.pp.
>
> Not every node that contains websites is a webserver: the nfsservers,
> for example.  I tried putting "include webserver" within
"case
> tagged(webserver)", but that didn''t solve the problem.
I see.  In that case you could define the service with a conditional  
in website.pp, I''m not sure if it''s possible with includes...
I''m
sure Luke has a better suggestion too.  :)

-Blake

On Jan 18, 2007, at 10:47 AM, Christian G. Warden wrote:
[snip]
> and classes/website.pp contains:
> define website() {
>    case tagged(webserver) {
>       true: {
> 			file { "/config/path/$name": ..., notify =>
Service[lighttpd] }
> 		}
> 	}
>    case tagged(nfsserver) {
>       true: {
> 			...
> 		}
> 	}
> }
>
> The problem seems to be that the lighttpd service can no longer be
> found:
> err: Could not apply complete configuration: Could not retrieve  
> dependency ''Service[lighttpd]'' at
/etc/puppet/manifests/classes/
> website.pp:26
>
> Any ideas on how I get this working again?
Currently, tags are set in file order, so if you''re using tags to  
determine if a class is set, then you need to make sure that the  
classes are included before the tags are tested.  In this case, your  
file order looks like it should be right, since parent node code gets  
evaluated before child nodes, so I don''t know what the problem is.

However, I''d redesign your classes a bit -- the webserver and  
nfsserver nodes are redundant.  I''d get rid of them, and just include  
those classes before the ''cluster_sites'' class in the node
definitions.

Note also in 0.22 (0.20+, actually, I think) you can do ''if tagged 
(nfsserver) { ... } else { ... }''.

  --
  I have lost friends, some by death... others through sheer  
inability to
  cross the street.     -- Virginia Woolf
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

On Thu, Jan 18, 2007 at 07:58:03PM +1100, Luke Kanies
wrote:
> On Jan 18, 2007, at 10:47 AM, Christian G. Warden wrote:
> [snip]
> > and classes/website.pp contains:
> > define website() {
> >    case tagged(webserver) {
> >       true: {
> > 			file { "/config/path/$name": ..., notify =>
Service[lighttpd] }
> > 		}
> > 	}
> >    case tagged(nfsserver) {
> >       true: {
> > 			...
> > 		}
> > 	}
> > }
> >
> > The problem seems to be that the lighttpd service can no longer be
> > found:
> > err: Could not apply complete configuration: Could not retrieve  
> > dependency ''Service[lighttpd]'' at
/etc/puppet/manifests/classes/
> > website.pp:26
> >
> > Any ideas on how I get this working again?
> 
> Currently, tags are set in file order, so if you''re using tags to
> determine if a class is set, then you need to make sure that the  
> classes are included before the tags are tested.  In this case, your  
> file order looks like it should be right, since parent node code gets  
> evaluated before child nodes, so I don''t know what the problem is.
> 
> However, I''d redesign your classes a bit -- the webserver and  
> nfsserver nodes are redundant.  I''d get rid of them, and just
include
> those classes before the ''cluster_sites'' class in the
node definitions.
Replacing 

node webserver {
	include webserver
}
node www01 inherits webserver {
}

with

node www01 {
	include webserver
}

seems to fix the problem, but the two should be equivalent, right?
 
> Note also in 0.22 (0.20+, actually, I think) you can do ''if tagged
> (nfsserver) { ... } else { ... }''.
I have a note in my manifest to change this after 0.19, but I couldn''t
find the new syntax in the docs.  Thanks.

Christian

On Thu, Jan 18, 2007 at 10:10:23AM -0800, Christian G. Warden
wrote:
> On Thu, Jan 18, 2007 at 07:58:03PM +1100, Luke Kanies wrote:
[...]
> > However, I''d redesign your classes a bit -- the webserver and
> > nfsserver nodes are redundant.  I''d get rid of them, and just
include
> > those classes before the ''cluster_sites'' class in
the node definitions.
> 
> Replacing 
> 
> node webserver {
> 	include webserver
> }
> node www01 inherits webserver {
> }
> 
> with
> 
> node www01 {
> 	include webserver
> }
> 
> seems to fix the problem, but the two should be equivalent, right?
Now I''m getting really bizarre behavior.  I have two nodes with the
same
configuration behaving differently.
node www01,www03 {
	include webserver
	include cluster_sites
}

Running pupppetd --test on www03 works fine, but on www01 I get:
err: Could not find type "remotefile" in file
/etc/puppet/manifests/classes/server.pp at line 16

where remotefile is defined in classes/util.pp and site.pp imports
classes/*, and webserver inherits server.

How can I go about debugging this?

Thanks,
Christian

On Jan 19, 2007, at 5:10 AM, Christian G. Warden wrote:
> Replacing
>
> node webserver {
> 	include webserver
> }
> node www01 inherits webserver {
> }
>
> with
>
> node www01 {
> 	include webserver
> }
>
> seems to fix the problem, but the two should be equivalent, right?
Well, mostly.  The thing you''re changing is that you''re able
to be
more specific in terms of ordering.

  --
  Hanlon''s razor:
     Never attribute to malice that which can adequately be explained by
     stupidity.
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

On Jan 19, 2007, at 6:34 AM, Christian G. Warden wrote:
> Now I''m getting really bizarre behavior.  I have two nodes with
the
> same
> configuration behaving differently.
> node www01,www03 {
> 	include webserver
> 	include cluster_sites
> }
>
> Running pupppetd --test on www03 works fine, but on www01 I get:
> err: Could not find type "remotefile" in file
/etc/puppet/manifests/
> classes/server.pp at line 16
>
> where remotefile is defined in classes/util.pp and site.pp imports
> classes/*, and webserver inherits server.
Um, I agree, that''s bizarre.
> How can I go about debugging this?
I''m somewhat stumped.  It can''t be import order, because
definitions
are set up at parse time, not compile time.

Can you pastie your site.pp file?

  --
  Experience is that marvelous thing that enables you recognize a
  mistake when you make it again.                  -- F. P. Jones
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

On Fri, Jan 19, 2007 at 09:26:48AM +1100, Luke Kanies
wrote:
> On Jan 19, 2007, at 6:34 AM, Christian G. Warden wrote:
> 
> > Now I''m getting really bizarre behavior.  I have two nodes
with the
> > same
> > configuration behaving differently.
> > node www01,www03 {
> > 	include webserver
> > 	include cluster_sites
> > }
> >
> > Running pupppetd --test on www03 works fine, but on www01 I get:
> > err: Could not find type "remotefile" in file
/etc/puppet/manifests/
> > classes/server.pp at line 16
> >
> > where remotefile is defined in classes/util.pp and site.pp imports
> > classes/*, and webserver inherits server.
> 
> Um, I agree, that''s bizarre.
> 
> > How can I go about debugging this?
> 
> I''m somewhat stumped.  It can''t be import order, because
definitions
> are set up at parse time, not compile time.
> 
> Can you pastie your site.pp file?
Here''s my (slightly sanitized) site.pp. 


$server = "puppet.example.com"

import "classes/*"

node default {
	include server
}

node webserver {
	include webserver
}

node nfsserver {
	include nfsserver
}

node managementserver {
	include server
	include ssl
	include kerberos-client
	include autofs
}

node dbserver {
	include server
	include kerberos-client
	include autofs
}

node loadbalancer {
	include loadbalancer
}

node db1,db2 inherits dbserver {
}

# FIXME: each virtual_ip must be an sslIP for one site even if the site
# doesn''t require ssl
node lb1,lb2 inherits loadbalancer {
	lbservice {
		1: virtual_ips => ["192.168.1.129"], real_ips =>
["192.168.2.185", "192.168.2.186"], ports => [25, 80,
443];
		2: virtual_ips => ["192.168.1.130"], real_ips =>
["192.168.2.188"], ports => [25, 80, 443];
	}
}

class www01-www02_sites {
	website {
		"www.example2.com":
			aliases => ["example2.com", "test.example2.com",
"www.example2.com.vip1.example.com"],
			concurrent => 10,
			username => www0001,
			project => absport,
			cvstag => "absport_beta-061215-2",
			sslIP => "192.168.1.129",
			dispatcher => true,
			sslCA => "godaddy";
		"www.example.com":
			aliases => ["example.com", "test.example.com",
"www.example.com.vip1.example.com"],
			username => www0002,
			project => example,
#			sslIP => "192.168.1.129",
			rewriteOnce => [
				''"^/t/" =>
"/common/track_open.php"'',
				''"^/pages/([^/]+)$" =>
"/common/page.php?ref=$1"'',
				''"^/example/press/item/([^/]+)$" =>
"/example/press/item.php?id=$1"'',
				''"^/example/clients/success/([^/]+)$" =>
"/example/clients/success.php?id=$1"'',
				''"^/example/clients/gallery/([^/]+)$" =>
"/example/clients/gallery.php?id=$1"''
			],
			rewriteRepeat => [
			
''"^/((?!404.php|example/|include/|images/|content/|donation/|common/|admin|mailer/).*)"
=> "/example/$1"'',
				''"^(.*/([^/.]+))/?$" =>
"$1/index.php"''
			],
			cvstag => "example-mysql5_dev-061208-1";
	}
}

class www03-sites {
	website {
		"mailer.example.com":
			aliases => ["mailer.example.com.vip2.example.com"],
			concurrent => 10,
			username => www0003,
			sslIP => "192.168.1.130",
			# sslCA => "godaddy";
	}
}

# Mercury site servers
node www01,www02,www03 {
	include webserver
	include www01-www02_sites
}

# Hermes server
#node www03 {
#	include webserver
#	include www03-sites
#}

node fs01,fs02 inherits nfsserver {
	include www01-www02_sites
	include www03-sites
}

node manage1 inherits managementserver {
	include www01-www02_sites
	include www03-sites
	file { "/usr/local/scripts/ssl_cert_expiration_check.php":
		source =>
"/etc/puppet/files/apps/lighttpd/ssl_cert_expiration_check.php",
		mode => 744,
	}
	cron { "ssl_cert_expiration_check":
		command => "/usr/local/scripts/ssl_cert_expiration_check.php",
		user => root,
		minute => 17,
		hour => 2
	}
}

On Fri, Jan 19, 2007 at 09:26:48AM +1100, Luke Kanies
wrote:
> On Jan 19, 2007, at 6:34 AM, Christian G. Warden wrote:
> 
> > Now I''m getting really bizarre behavior.  I have two nodes
with the
> > same
> > configuration behaving differently.
> > node www01,www03 {
> > 	include webserver
> > 	include cluster_sites
> > }
> >
> > Running pupppetd --test on www03 works fine, but on www01 I get:
> > err: Could not find type "remotefile" in file
/etc/puppet/manifests/
> > classes/server.pp at line 16
> >
> > where remotefile is defined in classes/util.pp and site.pp imports
> > classes/*, and webserver inherits server.
> 
> Um, I agree, that''s bizarre.
> 
> > How can I go about debugging this?
> 
> I''m somewhat stumped.  It can''t be import order, because
definitions
> are set up at parse time, not compile time.
With Luke''s invaluable help on #puppet, we figured out that I was
running 0.20 on the client that was failing.  I was using puppet to
manage my sources.list (Debian package repository list), which I had
updated to pull in 0.22, but puppet never got upgraded since it was
failing.

Christian, feeling silly

On Jan 22, 2007, at 3:56 PM, Christian G. Warden wrote:
>
> With Luke''s invaluable help on #puppet, we figured out that I was
> running 0.20 on the client that was failing.  I was using puppet to
> manage my sources.list (Debian package repository list), which I had
> updated to pull in 0.22, but puppet never got upgraded since it was
> failing.
Specifically the problem here was that a dependency on an instance of  
remotefile was not being resolved.  The configuration was being  
compiled fine, it just could not be instantiated on the client  
because of this dependency lookup problem.

In retrospect, I''ve opened a ticket to fix this error message so
it''s
clear that it''s a config-time failure, not a compile-time failure.

  --
  Only wimps use tape backup: _real_ men just upload their important
  stuff on ftp, and let the rest of the world mirror it.
                  --Linus Torvalds on linux-kernel
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com