Puppet users - Nov 2009 - Serious issue with 0.25.1, at wits end

Hi guys,

I''m attempting to move forward with our upgrade to 0.25.1, but today I
hit a show-stopper.

When running 0.25.1 behind Passenger I''m seeing the following
behavior. I''ll start up apache+passenger+puppetmasterd fresh, and run
puppetd -t on a client. The *next* client I run will attempt to fetch
the *previous* client''s catalog.

This doesn''t happen when not running behind Passenger.

I''m sure I''m screwing up my passenger config somehow, but for
the life
of me I can''t figure out how. Config files below, any ideas would be
HIGHLY appreciated.

--Paul

##########
apache2.conf
##########
# apache2.conf -- version for puppet-controlled installations
#
# this config file merely sets some important basic configs and
# includes other files; all the truly important stuff is configured in
# one of the include directories.

# ServerRoot: The top of the directory tree under which the server''s
# configuration, error, and log files are kept.
#
# NOTE!  If you intend to place this on an NFS (or otherwise network)
# mounted filesystem then please read the LockFile documentation (available
# at <URL:http://httpd.apache.org/docs-2.1/mod/mpm_common.html#lockfile>);
# you will save yourself a lot of trouble.
#
# Do NOT add a slash at the end of the directory path.
ServerRoot "/etc/apache2"

# AccessFileName: The name of the file to look for in each directory
# for additional configuration directives.  See also the AllowOverride
# directive.
AccessFileName .htaccess

# The following lines prevent .htaccess and .htpasswd files from being
# viewed by Web clients.
<Files ~ "^\.ht">
    Order allow,deny
    Deny from all
</Files>

# DefaultType is the default MIME type the server will use for a
# document if it cannot otherwise determine one, such as from filename
# extensions.  If your server contains mostly text or HTML documents,
# "text/plain" is a good value.  If most of your content is binary,
# such as applications or images, you may want to use
# "application/octet-stream" instead to keep browsers from trying to
# display binary files as though they are text.
DefaultType text/plain

# PidFile: The file in which the server should record its process
# identification number when it starts.  This needs to be set in
# /etc/apache2/envvars
PidFile ${APACHE_PID_FILE}

# These need to be set in /etc/apache2/envvars
User ${APACHE_RUN_USER}
Group ${APACHE_RUN_GROUP}

# HostnameLookups: Log the names of clients or just their IP addresses
# e.g., www.apache.org (on) or 204.62.129.132 (off).
# The default is off because it''d be overall better for the net if
people
# had to knowingly turn this feature on, since enabling it means that
# each client request will result in AT LEAST one lookup request to the
# nameserver.
HostnameLookups Off

# Log errors to the right place
ErrorLog /var/log/apache2/error.log

# Include module configuration:
Include /etc/apache2/mods-enabled/*.load
Include /etc/apache2/mods-enabled/*.conf

# Include generic snippets of statements
Include /etc/apache2/conf.d/*.conf

# Include the virtual host configurations:
Include /etc/apache2/sites-enabled/

##########
conf.d/500-puppetmasterd.conf
##########
PassengerRoot /var/lib/gems/1.8/gems/passenger-2.2.5
PassengerRuby /usr/bin/ruby1.8

PassengerHighPerformance on
PassengerMaxPoolSize 16
PassengerPoolIdleTime 300
PassengerStatThrottleRate 120
PassengerUseGlobalQueue on
RackAutoDetect Off
RailsAutoDetect Off

Listen 8140

<VirtualHost *:8140>
        SSLEngine on
        SSLProtocol -ALL +SSLv3 +TLSv1
        SSLCipherSuite ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:-LOW:-SSLv2:-EXP

        SSLCertificateFile      /var/lib/puppet/ssl/certs/puppet.pem
        SSLCertificateKeyFile   /var/lib/puppet/ssl/private_keys/puppet.pem
        SSLCertificateChainFile /var/lib/puppet/ssl/ca/ca_crt.pem
        SSLCACertificateFile    /var/lib/puppet/ssl/ca/ca_crt.pem
        # If Apache complains about invalid signatures on the CRL, you
can try disabling
        # CRL checking by commenting the next line, but this is not recommended.
        SSLCARevocationFile     /var/lib/puppet/ssl/ca/ca_crl.pem
        SSLVerifyClient optional
        SSLVerifyDepth  1
        SSLOptions +StdEnvVars

        DocumentRoot /etc/puppet/rack/public/
        RackBaseURI /
        <Directory /etc/puppet/rack/>
                Options None
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>
</VirtualHost>

##########
/etc/puppet/puppet.conf
##########
[main]
logdir = /var/log/puppet
report = true
rundir = /var/run/puppet
ssldir = /var/lib/puppet/ssl
vardir = /var/lib/puppet
factpath = /var/lib/puppet/lib/facter
pluginsync = true
reports = store, log

[puppetd]
server = s0005.digg.internal
runinterval = 600
splay = true
environment = production

[puppetmasterd]
autosign = true
ssl_client_header = SSL_CLIENT_S_DN
ssl_client_verify_header = SSL_CLIENT_VERIFY
manifest = /srv/puppet/production/manifests/site.pp
modulepath = /srv/puppet/production/modules
certname = puppet
certdnsnames = puppet:puppet.digg.internal:s0005.digg.internal

##########
/etc/puppet/rack/config.ru
##########
$0 = "puppetmasterd"
require ''puppet''

# if you want debugging:
#ARGV << "--debug"

ARGV << "--rack"
require ''puppet/application/puppetmasterd''
run Puppet::Application[:puppetmasterd].run

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Hi Paul,

On Wed, 2009-11-04 at 17:36 -0800, Paul Lathrop wrote:
> Hi guys,
> 
> I''m attempting to move forward with our upgrade to 0.25.1, but
today I
> hit a show-stopper.
> 
> When running 0.25.1 behind Passenger I''m seeing the following
> behavior. I''ll start up apache+passenger+puppetmasterd fresh, and
run
> puppetd -t on a client. The *next* client I run will attempt to fetch
> the *previous* client''s catalog.
> 
> This doesn''t happen when not running behind Passenger.
> 
> I''m sure I''m screwing up my passenger config somehow, but
for the life
> of me I can''t figure out how. Config files below, any ideas would
be
> HIGHLY appreciated.
I''ve seen the same issue lately on #puppet, brought by different
people.
The issue is a combination of Puppet 0.25 + Passenger 2.2.5.

Reverting to Passenger 2.2.2 _or_ Puppet 0.24.8 fixes the issue.

I suggest you install passenger 2.2.2 while we find what the issue is.
In a nutshell the problem is that Passenger ENV seems to be Write Once
in the life of a ruby Interpreter. So the second host that connects gets
the CN of the first one.

There is a bug open on the passenger project, which tells us that it has
been fixed in 2.2.5, so I think this is a combination of puppet and
passenger that creates the issue.

I think Christian Hofstaedtler can answer better than I can, so I''m
CCing him.
-- 
Brice Figureau
Follow the latest Puppet Community evolutions on www.planetpuppet.org!


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Hi guys,

* Brice Figureau <brice-puppet@daysofwonder.com> [091105
10:39]:
> Hi Paul,
> 
> On Wed, 2009-11-04 at 17:36 -0800, Paul Lathrop wrote:
> > [puppet 0.25.1 + passenger not working]
> 
> I''ve seen the same issue lately on #puppet, brought by different
people.
> The issue is a combination of Puppet 0.25 + Passenger 2.2.5.
> 
> Reverting to Passenger 2.2.2 _or_ Puppet 0.24.8 fixes the issue.
> 
> I suggest you install passenger 2.2.2 while we find what the issue is.
> In a nutshell the problem is that Passenger ENV seems to be Write Once
> in the life of a ruby Interpreter. So the second host that connects gets
> the CN of the first one.
> 
> [..]
I''ve now been able to reproduce this with Passenger 2.2.5 and
current Puppet HEAD. The surprising solution is to remove the work
around code for Passenger 2.2.4, which falls back to using ENV for
stuff which ain''t in request.env.

I''ve posted a patch to puppet-dev [1] which actually does this.
If you could test it, it''d be great.

Christian


[1] Message-ID: <1257416944-2114-2-git-send-email-ch+git@zeha.at>

-- 
christian hofstaedtler

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Thu, Nov 5, 2009 at 2:30 AM, Christian Hofstaedtler <ch+git@zeha.at>
wrote:
> I''ve now been able to reproduce this with Passenger 2.2.5 and
> current Puppet HEAD. The surprising solution is to remove the work
> around code for Passenger 2.2.4, which falls back to using ENV for
> stuff which ain''t in request.env.
>
> I''ve posted a patch to puppet-dev [1] which actually does this.
> If you could test it, it''d be great.
Tested and confirmed, this works great. Thanks, Christian!

--Paul

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Nov 6, 9:36 pm, Paul Lathrop <paul.lath...@gmail.com>
wrote:
> On Thu, Nov 5, 2009 at 2:30 AM, Christian Hofstaedtler
<ch+...@zeha.at> wrote:
> > I''ve now been able to reproduce this withPassenger2.2.5 and
> > current Puppet HEAD. The surprising solution is to remove the work
> > around code forPassenger2.2.4, which falls back to using ENV for
> > stuff which ain''t in request.env.
>
> > I''ve posted a patch to puppet-dev [1] which actually does
this.
> > If you could test it, it''d be great.
>
> Tested and confirmed, this works great. Thanks, Christian!
Thanks for testing this.

Christian

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---