[Puppet Users] nginx - mongrel - fair proxy balancer

Peter Meier
2010-08-18T12:55:21Z

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi

I recently came across
http://www.brainspl.at/articles/2007/11/09/a-fair-proxy-balancer-for-nginx-and-mongrel
and wanted to share the experience with it:

I tested it on my nginx-mongrel setup and it improved the speed of my
client runs heavily. If you have a look at how the improved balancer is
working then this balancer is what you should use for an
nginx->puppetmaster setup:

Puppet clients are asking masters for:

* catalogs (rather long and big requests)
* file metadata (rather small and fast requests)
* file sources (can be offloaded to nginx or apache thanks to the new
rest interface and on 2.6 it''s also "faster")

Furthermore, they PUT "reports" on the master.

I think these points cover more or less all relevant parts of the
communication between client and master.

During a pupet client run, there are first a bunch of file
metadata/sources (plugins/facts) request, then the catalog request, then
a lot of file metadata requests and probably some file source requests.
At the end the reports (if configured) are PUT on the server.

The default load balancer of nginx is just round-robin, which means more
or less that the the first requests goes to the first backend server,
the second to the second and so on. This is fine in most cases. However
as catalogs requests can lost for more than a couple dozen seconds the
mongrel distance dealing with the catalog request is busy compiling and
not responding to any other requests. This means that if one client is
asking for a catalog every 4th probably small and fast request will be
stuck in the queue for the busy instance. If there are currently 2
clients asking for catalogs every second request gets already stuck.
etc. This means that clients asking for metadata are slowed down,
because their requests get stuck in the busy queue.

The fair proxy scheduler tries to keep track of the busy instances and
does redirect requests to other (probably more free) instances, hence
the requests are served much faster. For further discussion have a look
at http://www.ruby-forum.com/topic/132604

Before using the fair proxy scheduler I had a lot of clients that were
just sitting there and waiting for something. If your ran them with
debug, you could see that they were often waiting for file metadata
requests. Using the fair proxy scheduler I could more or less get rid
off these waiting requests and the client runs pass now much faster and
without getting stucked.

I have added a paragraph about the fair proxy scheduler to the wiki page
about the nginx-mongrel setup. If you have any comments or further
notes, just add them.

cheers pete
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkxr2DgACgkQbwltcAfKi3+E/QCgmvAIFroqFdA4eTyJDV2BW/ZZ
zD0AnAwvrkR1yniZh9x3Xg+IpK20JEx5
=5hKX
-----END PGP SIGNATURE-----

--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at