thr3ads.net - Puppet users - [Puppet Users] kindly have a look at this issue please, related to multiple puppet masters [Jan 2013]

If this information is useful, please help other people find it:
Share via:

bin.zhou2007@gmail.com

2013-Jan-16 10:39 UTC

[Puppet Users] kindly have a look at this issue please, related to multiple puppet masters

hi,  could everyone kindly have a look at this issue below please. Thanks a 
lot!


It works while I was using apache+passenger+puppet-master on the server 
side,  I used ''puppet agent -t'' on the client side, and it was
successfully
synchronized saying that:

"sudo puppet agent -t 
Notice: Ignoring --listen on onetime run
Info: Retrieving plugin
Info: Caching catalog for agent.xxxx.net
Info: Applying configuration version ''1358322483''"


But unfortunately it cannot work if I am trying to use apache as a load 
balancer, and two virtual hosts as the puppet backend servers who actually 
serves the requests from puppet agents.

Here below is the *access log of balancer*:

10.16.27.31 - - [16/Jan/2013:16:54:21 +0800] "GET 
/production/node/agent.xxxx.net? HTTP/1.1" 403 113 "-"
"-"

10.16.27.31 - - [16/Jan/2013:16:54:23 +0800] "GET 
/production/file_metadatas/plugins?&links=manage&recurse=true&checksum_type=md5&ignore=---+%0A++-+%22.svn%22%0A++-+CVS%0A++-+%22.git%22
HTTP/1.1" 403 105 "-" "-"

10.16.27.31 - - [16/Jan/2013:16:54:25 +0800] "GET 
/production/file_metadata/plugins? HTTP/1.1" 403 103 "-"
"-"

10.16.27.31 - - [16/Jan/2013:16:54:26 +0800] "POST 
/production/catalog/agent.xxxx.net HTTP/1.1" 403 116 "-"
"-"

10.16.27.31 - - [16/Jan/2013:16:54:26 +0800] "PUT 
/production/report/agent.xxxx.net HTTP/1.1" 502 560 "-"
"-"


Here below is the* error log of balancer*:

[Wed Jan 16 16:54:26 2013] [error] [client 10.16.27.31] (20014)Internal 
error: proxy: error reading status line from remote server 127.0.0.1

[Wed Jan 16 16:54:26 2013] [error] [client 10.16.27.31] proxy: Error 
reading from remote server returned by /production/report/agent.xxxx.net


Here below is what* /var/log/messages* said:

Jan 16 16:54:23 master puppet-master[22191]: Starting Puppet master version 
3.0.2

Jan 16 16:54:23 master puppet-master[22255]: Denying access: Forbidden 
request: master.xxxx.net(127.0.0.1) access to /node/agent.xxxx.net [find] 
at :99

Jan 16 16:54:23 master puppet-master[22255]: Forbidden request: 
master.xxxx.net(127.0.0.1) access to /node/agent.xxxx.net [find] at :99

Jan 16 16:54:25 master puppet-master[22273]: Starting Puppet master version 
3.0.2

Jan 16 16:54:25 master puppet-master[22325]: Denying access: Forbidden 
request: master.xxxx.net(127.0.0.1) access to /file_metadata/plugins 
[search] at :99

Jan 16 16:54:25 master puppet-master[22325]: Forbidden request: 
master.xxxx.net(127.0.0.1) access to /file_metadata/plugins [search] at :99

Jan 16 16:54:25 master puppet-master[22255]: Denying access: Forbidden 
request: master.xxxx.net(127.0.0.1) access to /file_metadata/plugins [find] 
at :99

Jan 16 16:54:25 master puppet-master[22255]: Forbidden request: 
master.xxxx.net(127.0.0.1) access to /file_metadata/plugins [find] at :99

Jan 16 16:54:26 master puppet-master[22325]: Denying access: Forbidden 
request: master.xxxx.net(127.0.0.1) access to /catalog/agent.xxxx.net 
[find] at :99

Jan 16 16:54:26 master puppet-master[22325]: Forbidden request: 
master.xxxx.net(127.0.0.1) access to /catalog/agent.xxxx.net [find] at :99

Jan 16 16:54:26 master puppet-master[22255]: Denying access: Forbidden 
request: master.xxxx.net(127.0.0.1) access to /report/agent.xxxx.net [save] 
at :99

Jan 16 16:54:26 master puppet-master[22255]: Forbidden request: 
master.xxxx.net(127.0.0.1) access to /report/agent.xxxx.net [save] at :99

Jan 16 17:41:02 master ntpd[1660]: synchronized to 10.16.13.14, stratum 2


Here below is what one of the worker said:   (*
puppetmaster_worker_access_18140.log*)

127.0.0.1 - - [16/Jan/2013:16:54:21 +0800] "GET 
/production/node/agent.xxxx.net? HTTP/1.1" 403 113 "-"
"-"

127.0.0.1 - - [16/Jan/2013:16:54:25 +0800] "GET 
/production/file_metadata/plugins? HTTP/1.1" 403 103 "-"
"-"

127.0.0.1 - - [16/Jan/2013:16:54:26 +0800] "PUT 
/production/report/agent.xxxx.net HTTP/1.1" 403 - "-"
"-"


(* puppetmaster_worker_error_18140.log*)

[Wed Jan 16 16:54:26 2013] [error] [client 127.0.0.1] (104)Connection reset 
by peer: ap_content_length_filter: apr_bucket_read() failed


[root@master httpd]# *less puppetmaster_worker_access_18141.log*

127.0.0.1 - - [16/Jan/2013:16:54:23 +0800] "GET 
/production/file_metadatas/plugins?&links=manage&recurse=true&checksum_type=md5&ignore=---+%0A++-+%22.svn%22%0A++-+CVS%0A++-+%22.git%22
HTTP/1.1" 403 105 "-" "-"

127.0.0.1 - - [16/Jan/2013:16:54:26 +0800] "POST 
/production/catalog/agent.xxxx.net HTTP/1.1" 403 116 "-"
"-"


*
*

*Here below come all related configurations:*

*
*

[root@master conf.d]# *cat passenger.conf*

LoadModule passenger_module 
/usr/lib64/ruby/gems/1.8/gems/passenger-3.0.17/ext/apache2/mod_passenger.so

PassengerRoot /usr/lib64/ruby/gems/1.8/gems/passenger-3.0.17

PassengerRuby /usr/bin/ruby

# And the passenger performance tuning settings:

PassengerHighPerformance On

PassengerUseGlobalQueue On

# Set this to about 1.5 times the number of CPU cores in your master:

PassengerMaxPoolSize 3

# Recycle master processes after they service 1000 requests

PassengerMaxRequests 1000

# Stop processes if they sit idle for 10 minutes

PassengerPoolIdleTime 600

 

[root@master conf.d]# *cat puppetmaster.conf*

<Proxy balancer://puppetmaster>

        BalancerMember http://127.0.0.1:18140

        BalancerMember http://127.0.0.1:18141

</Proxy>

 

Listen 8140

<VirtualHost *:8140>

    SSLEngine On

    

    # Only allow high security cryptography. Alter if needed for 
compatibility.

    SSLProtocol             All -SSLv2

    SSLCipherSuite          HIGH:!ADH:RC4+RSA:-MEDIUM:-LOW:-EXP

    SSLCertificateFile      /var/lib/puppet/ssl/certs/master.xxxx.net.pem

    SSLCertificateKeyFile   
/var/lib/puppet/ssl/private_keys/master.xxxx.net.pem

    SSLCertificateChainFile /var/lib/puppet/ssl/ca/ca_crt.pem

    SSLCACertificateFile    /var/lib/puppet/ssl/ca/ca_crt.pem

    SSLCARevocationFile     /var/lib/puppet/ssl/ca/ca_crl.pem

    SSLVerifyClient         optional

    SSLVerifyDepth          1

    SSLOptions              +StdEnvVars +ExportCertData

    

    # These request headers are used to pass the client certificate

    # authentication information on to the puppet master process

    RequestHeader set X-SSL-Subject %{SSL_CLIENT_S_DN}e

    RequestHeader set X-Client-DN %{SSL_CLIENT_S_DN}e

    RequestHeader set X-Client-Verify %{SSL_CLIENT_VERIFY}e

 

        <Location />

                SetHandler balancer-manager

                Order allow,deny

                Allow from all

        </Location>

 

        ProxyPass / balancer://puppetmaster/

        ProxyPassReverse / balancer://puppetmaster/

        ProxyPreserveHost On

 

        ErrorLog /var/log/httpd/balancer_error.log

        CustomLog /var/log/httpd/balancer_access.log combined

        CustomLog /var/log/httpd/balancer_ssl_requests.log "%t %h 
%{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"

 

</VirtualHost>


[root@master conf.d]# *cat puppetmaster_worker_18140.conf *

 

Listen 18140

<VirtualHost 127.0.0.1:18140>

    SSLEngine Off

 

        # Obtain Authentication Information from Client Request Headers

        SetEnvIf X-Client-Verify "(.*)" SSL_CLIENT_VERIFY=$1  

        SetEnvIf X-SSL-Client-DN "(.*)" SSL_CLIENT_S_DN=$1  

 

    RackAutoDetect On

    DocumentRoot /usr/share/puppet/rack/puppetmasterd_18140/public/

    <Directory /usr/share/puppet/rack/puppetmasterd_18140/>

        Options None

        AllowOverride None

        Order Allow,Deny

        Allow from All

        ## This relaxes Apache security settings.

        #AllowOverride all

        ## MultiViews must be turned off.

        #Options -MultiViews

    </Directory>

 

        ErrorLog /var/log/httpd/puppetmaster_worker_error_18140.log

        CustomLog /var/log/httpd/puppetmaster_worker_access_18140.log 
combined

 

</VirtualHost>

 

[root@master conf.d]# *cat puppetmaster_worker_18141.conf  *

 

Listen 18141

<VirtualHost 127.0.0.1:18141>

    SSLEngine Off

 

        # Obtain Authentication Information from Client Request Headers

        SetEnvIf X-Client-Verify "(.*)" SSL_CLIENT_VERIFY=$1  

        SetEnvIf X-SSL-Client-DN "(.*)" SSL_CLIENT_S_DN=$1  

 

    RackAutoDetect On

    DocumentRoot /usr/share/puppet/rack/puppetmasterd_18141/public/

    <Directory /usr/share/puppet/rack/puppetmasterd_18141/>

        Options None

        AllowOverride None

        Order Allow,Deny

        Allow from All

        ## This relaxes Apache security settings.

        #AllowOverride all

        ## MultiViews must be turned off.

        #Options -MultiViews

    </Directory>

 

        ErrorLog /var/log/httpd/puppetmaster_worker_error_18141.log

        CustomLog /var/log/httpd/puppetmaster_worker_access_18141.log 
combined

 

</VirtualHost>

 

 

 








-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/JIg1s-iLKPoJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

bin.zhou2007@gmail.com

2013-Jan-18 07:31 UTC

head link

[Puppet Users] Re: kindly have a look at this issue please, related to multiple puppet masters

Could somebody kindly have a look at this please, thanks a lot.

在 2013年1月16日星期三UTC+8下午6时39分06秒，bin.zh...@gmail.com写道：>
> hi,  could everyone kindly have a look at this issue below please. Thanks 
> a lot!
>
>
> It works while I was using apache+passenger+puppet-master on the server 
> side,  I used ''puppet agent -t'' on the client side, and
it was successfully
> synchronized saying that:
>
> "sudo puppet agent -t 
> Notice: Ignoring --listen on onetime run
> Info: Retrieving plugin
> Info: Caching catalog for agent.xxxx.net
> Info: Applying configuration version ''1358322483''"
>
>
> But unfortunately it cannot work if I am trying to use apache as a load 
> balancer, and two virtual hosts as the puppet backend servers who actually 
> serves the requests from puppet agents.
>
> Here below is the *access log of balancer*:
>
> 10.16.27.31 - - [16/Jan/2013:16:54:21 +0800] "GET /production/node/
> agent.xxxx.net? HTTP/1.1" 403 113 "-" "-"
>
> 10.16.27.31 - - [16/Jan/2013:16:54:23 +0800] "GET 
>
/production/file_metadatas/plugins?&links=manage&recurse=true&checksum_type=md5&ignore=---+%0A++-+%22.svn%22%0A++-+CVS%0A++-+%22.git%22
> HTTP/1.1" 403 105 "-" "-"
>
> 10.16.27.31 - - [16/Jan/2013:16:54:25 +0800] "GET 
> /production/file_metadata/plugins? HTTP/1.1" 403 103 "-"
"-"
>
> 10.16.27.31 - - [16/Jan/2013:16:54:26 +0800] "POST
/production/catalog/
> agent.xxxx.net HTTP/1.1" 403 116 "-" "-"
>
> 10.16.27.31 - - [16/Jan/2013:16:54:26 +0800] "PUT /production/report/
> agent.xxxx.net HTTP/1.1" 502 560 "-" "-"
>
>
> Here below is the* error log of balancer*:
>
> [Wed Jan 16 16:54:26 2013] [error] [client 10.16.27.31] (20014)Internal 
> error: proxy: error reading status line from remote server 127.0.0.1
>
> [Wed Jan 16 16:54:26 2013] [error] [client 10.16.27.31] proxy: Error 
> reading from remote server returned by /production/report/agent.xxxx.net
>
>
> Here below is what* /var/log/messages* said:
>
> Jan 16 16:54:23 master puppet-master[22191]: Starting Puppet master 
> version 3.0.2
>
> Jan 16 16:54:23 master puppet-master[22255]: Denying access: Forbidden 
> request: master.xxxx.net(127.0.0.1) access to /node/agent.xxxx.net [find] 
> at :99
>
> Jan 16 16:54:23 master puppet-master[22255]: Forbidden request: 
> master.xxxx.net(127.0.0.1) access to /node/agent.xxxx.net [find] at :99
>
> Jan 16 16:54:25 master puppet-master[22273]: Starting Puppet master 
> version 3.0.2
>
> Jan 16 16:54:25 master puppet-master[22325]: Denying access: Forbidden 
> request: master.xxxx.net(127.0.0.1) access to /file_metadata/plugins 
> [search] at :99
>
> Jan 16 16:54:25 master puppet-master[22325]: Forbidden request: 
> master.xxxx.net(127.0.0.1) access to /file_metadata/plugins [search] at 
> :99
>
> Jan 16 16:54:25 master puppet-master[22255]: Denying access: Forbidden 
> request: master.xxxx.net(127.0.0.1) access to /file_metadata/plugins 
> [find] at :99
>
> Jan 16 16:54:25 master puppet-master[22255]: Forbidden request: 
> master.xxxx.net(127.0.0.1) access to /file_metadata/plugins [find] at :99
>
> Jan 16 16:54:26 master puppet-master[22325]: Denying access: Forbidden 
> request: master.xxxx.net(127.0.0.1) access to /catalog/agent.xxxx.net[find]
at :99
>
> Jan 16 16:54:26 master puppet-master[22325]: Forbidden request: 
> master.xxxx.net(127.0.0.1) access to /catalog/agent.xxxx.net [find] at :99
>
> Jan 16 16:54:26 master puppet-master[22255]: Denying access: Forbidden 
> request: master.xxxx.net(127.0.0.1) access to /report/agent.xxxx.net[save]
at :99
>
> Jan 16 16:54:26 master puppet-master[22255]: Forbidden request: 
> master.xxxx.net(127.0.0.1) access to /report/agent.xxxx.net [save] at :99
>
> Jan 16 17:41:02 master ntpd[1660]: synchronized to 10.16.13.14, stratum 2
>
>
> Here below is what one of the worker said:   (*
> puppetmaster_worker_access_18140.log*)
>
> 127.0.0.1 - - [16/Jan/2013:16:54:21 +0800] "GET /production/node/
> agent.xxxx.net? HTTP/1.1" 403 113 "-" "-"
>
> 127.0.0.1 - - [16/Jan/2013:16:54:25 +0800] "GET 
> /production/file_metadata/plugins? HTTP/1.1" 403 103 "-"
"-"
>
> 127.0.0.1 - - [16/Jan/2013:16:54:26 +0800] "PUT /production/report/
> agent.xxxx.net HTTP/1.1" 403 - "-" "-"
>
>
> (* puppetmaster_worker_error_18140.log*)
>
> [Wed Jan 16 16:54:26 2013] [error] [client 127.0.0.1] (104)Connection 
> reset by peer: ap_content_length_filter: apr_bucket_read() failed
>
>
> [root@master httpd]# *less puppetmaster_worker_access_18141.log*
>
> 127.0.0.1 - - [16/Jan/2013:16:54:23 +0800] "GET 
>
/production/file_metadatas/plugins?&links=manage&recurse=true&checksum_type=md5&ignore=---+%0A++-+%22.svn%22%0A++-+CVS%0A++-+%22.git%22
> HTTP/1.1" 403 105 "-" "-"
>
> 127.0.0.1 - - [16/Jan/2013:16:54:26 +0800] "POST /production/catalog/
> agent.xxxx.net HTTP/1.1" 403 116 "-" "-"
>
>
> *
> *
>
> *Here below come all related configurations:*
>
> *
> *
>
> [root@master conf.d]# *cat passenger.conf*
>
> LoadModule passenger_module 
> /usr/lib64/ruby/gems/1.8/gems/passenger-3.0.17/ext/apache2/mod_passenger.so
>
> PassengerRoot /usr/lib64/ruby/gems/1.8/gems/passenger-3.0.17
>
> PassengerRuby /usr/bin/ruby
>
> # And the passenger performance tuning settings:
>
> PassengerHighPerformance On
>
> PassengerUseGlobalQueue On
>
> # Set this to about 1.5 times the number of CPU cores in your master:
>
> PassengerMaxPoolSize 3
>
> # Recycle master processes after they service 1000 requests
>
> PassengerMaxRequests 1000
>
> # Stop processes if they sit idle for 10 minutes
>
> PassengerPoolIdleTime 600
>
>  
>
> [root@master conf.d]# *cat puppetmaster.conf*
>
> <Proxy balancer://puppetmaster>
>
>         BalancerMember http://127.0.0.1:18140
>
>         BalancerMember http://127.0.0.1:18141
>
> </Proxy>
>
>  
>
> Listen 8140
>
> <VirtualHost *:8140>
>
>     SSLEngine On
>
>     
>
>     # Only allow high security cryptography. Alter if needed for 
> compatibility.
>
>     SSLProtocol             All -SSLv2
>
>     SSLCipherSuite          HIGH:!ADH:RC4+RSA:-MEDIUM:-LOW:-EXP
>
>     SSLCertificateFile      /var/lib/puppet/ssl/certs/master.xxxx.net.pem
>
>     SSLCertificateKeyFile   
> /var/lib/puppet/ssl/private_keys/master.xxxx.net.pem
>
>     SSLCertificateChainFile /var/lib/puppet/ssl/ca/ca_crt.pem
>
>     SSLCACertificateFile    /var/lib/puppet/ssl/ca/ca_crt.pem
>
>     SSLCARevocationFile     /var/lib/puppet/ssl/ca/ca_crl.pem
>
>     SSLVerifyClient         optional
>
>     SSLVerifyDepth          1
>
>     SSLOptions              +StdEnvVars +ExportCertData
>
>     
>
>     # These request headers are used to pass the client certificate
>
>     # authentication information on to the puppet master process
>
>     RequestHeader set X-SSL-Subject %{SSL_CLIENT_S_DN}e
>
>     RequestHeader set X-Client-DN %{SSL_CLIENT_S_DN}e
>
>     RequestHeader set X-Client-Verify %{SSL_CLIENT_VERIFY}e
>
>  
>
>         <Location />
>
>                 SetHandler balancer-manager
>
>                 Order allow,deny
>
>                 Allow from all
>
>         </Location>
>
>  
>
>         ProxyPass / balancer://puppetmaster/
>
>         ProxyPassReverse / balancer://puppetmaster/
>
>         ProxyPreserveHost On
>
>  
>
>         ErrorLog /var/log/httpd/balancer_error.log
>
>         CustomLog /var/log/httpd/balancer_access.log combined
>
>         CustomLog /var/log/httpd/balancer_ssl_requests.log "%t %h 
> %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
>
>  
>
> </VirtualHost>
>
>
> [root@master conf.d]# *cat puppetmaster_worker_18140.conf *
>
>  
>
> Listen 18140
>
> <VirtualHost 127.0.0.1:18140>
>
>     SSLEngine Off
>
>  
>
>         # Obtain Authentication Information from Client Request Headers
>
>         SetEnvIf X-Client-Verify "(.*)" SSL_CLIENT_VERIFY=$1  
>
>         SetEnvIf X-SSL-Client-DN "(.*)" SSL_CLIENT_S_DN=$1  
>
>  
>
>     RackAutoDetect On
>
>     DocumentRoot /usr/share/puppet/rack/puppetmasterd_18140/public/
>
>     <Directory /usr/share/puppet/rack/puppetmasterd_18140/>
>
>         Options None
>
>         AllowOverride None
>
>         Order Allow,Deny
>
>         Allow from All
>
>         ## This relaxes Apache security settings.
>
>         #AllowOverride all
>
>         ## MultiViews must be turned off.
>
>         #Options -MultiViews
>
>     </Directory>
>
>  
>
>         ErrorLog /var/log/httpd/puppetmaster_worker_error_18140.log
>
>         CustomLog /var/log/httpd/puppetmaster_worker_access_18140.log 
> combined
>
>  
>
> </VirtualHost>
>
>  
>
> [root@master conf.d]# *cat puppetmaster_worker_18141.conf  *
>
>  
>
> Listen 18141
>
> <VirtualHost 127.0.0.1:18141>
>
>     SSLEngine Off
>
>  
>
>         # Obtain Authentication Information from Client Request Headers
>
>         SetEnvIf X-Client-Verify "(.*)" SSL_CLIENT_VERIFY=$1  
>
>         SetEnvIf X-SSL-Client-DN "(.*)" SSL_CLIENT_S_DN=$1  
>
>  
>
>     RackAutoDetect On
>
>     DocumentRoot /usr/share/puppet/rack/puppetmasterd_18141/public/
>
>     <Directory /usr/share/puppet/rack/puppetmasterd_18141/>
>
>         Options None
>
>         AllowOverride None
>
>         Order Allow,Deny
>
>         Allow from All
>
>         ## This relaxes Apache security settings.
>
>         #AllowOverride all
>
>         ## MultiViews must be turned off.
>
>         #Options -MultiViews
>
>     </Directory>
>
>  
>
>         ErrorLog /var/log/httpd/puppetmaster_worker_error_18141.log
>
>         CustomLog /var/log/httpd/puppetmaster_worker_access_18141.log 
> combined
>
>  
>
> </VirtualHost>
>
>  
>
>  
>
>  
>
>
>
>
>
>
>
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/LlVuhAAtOL0J.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Puppet users - Jan 2013 - kindly have a look at this issue please, related to multiple puppet masters

[Puppet Users] kindly have a look at this issue please, related to multiple puppet masters

[Puppet Users] Re: kindly have a look at this issue please, related to multiple puppet masters