Puppet users - Jun 2009 - POLL: Migrating web changes across load-balanced servers

Hi folks,

I''m polling the group to see what some basic techniques are that
people use to migrate web sites to load-balanced servers.

In the past, IRC folks have mentioned rsysnc as the way.

I''m specifically curious if people are:

1) Letting puppet MD5 a website via a file{} tag?  Could that be done?
2) Using version control somehow?  I''ve been toying with creating a
mercurial plugin somehow.  If you push some changes to a production
machine, you can configure mercurial to auto-update.

What''s some ideas?  Any pointers?

TIA
Mike

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Hi Mike,

I''m not sure if your after puppet-specific methods, or just general
systems for this?

When we set up our load balancing and  DR, we preferred to keep it
simple, the methods used are outlined below.

1) Site code - This is managed but subversion, we maintain separate
repositories for testing and production environments - when a change is
ready to go live it is merged into the stable codebase. 
    -    To then update the production servers, a number of methods are
available
        1) For minor updates, there is a nightly cronjob that updates
all the live servers, and DR setup
        2) For critical fixes. I usually use cssh to trigger the update
immediately, alternatively we have an authenticating email daemon that
can be used to trigger the update cron at any time.

    - For user contributed files (ie image/video uploads, pdf
attachments etc) We use a centralised SAN store for the upload
locations. All uploads are also written to a temporary location and then
rsync''d to the DR location every 15 minutes.

    - For SQL - we use MySQL replication, in a Master-Master setup
between live and DR locations, with each master then having slaves to
balance the load

Cheers

Tim

-----Original Message-----
From: drmikecrowe <drmikecrowe@gmail.com>
Reply-to: puppet-users@googlegroups.com
To: Puppet Users <puppet-users@googlegroups.com>
Subject: [Puppet Users] POLL: Migrating web changes across load-balanced
servers
Date: Sun, 14 Jun 2009 14:05:24 -0700 (PDT)


Hi folks,

I''m polling the group to see what some basic techniques are that
people use to migrate web sites to load-balanced servers.

In the past, IRC folks have mentioned rsysnc as the way.

I''m specifically curious if people are:

1) Letting puppet MD5 a website via a file{} tag?  Could that be done?
2) Using version control somehow?  I''ve been toying with creating a
mercurial plugin somehow.  If you push some changes to a production
machine, you can configure mercurial to auto-update.

What''s some ideas?  Any pointers?

TIA
Mike



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Hi Tim,

That''s very close to where I am now, except we use mercurial instead
of svn.  :)

However, I''m more searching for non-same hosts (i.e. no SAN). 
I''m
basically asking the question of puppet managed ( file
{ source=>"puppet://.... } ) vs. another scheme.

My hunch is I do NOT want puppet managing 1000''s of files and
propagating them.  Maybe have puppet monitor a version file, and when
that changes, execute an rsync to bring the directories into sync.

Thanks for the response!
Mike

On Jun 14, 7:06 pm, Tim Allingham <deser...@gmail.com>
wrote:
> Hi Mike,
>
> I''m not sure if your after puppet-specific methods, or just
general
> systems for this?
>
> When we set up our load balancing and  DR, we preferred to keep it
> simple, the methods used are outlined below.
>
> 1) Site code - This is managed but subversion, we maintain separate
> repositories for testing and production environments - when a change is
> ready to go live it is merged into the stable codebase.
>     -    To then update the production servers, a number of methods are
> available
>         1) For minor updates, there is a nightly cronjob that updates
> all the live servers, and DR setup
>         2) For critical fixes. I usually use cssh to trigger the update
> immediately, alternatively we have an authenticating email daemon that
> can be used to trigger the update cron at any time.
>
>     - For user contributed files (ie image/video uploads, pdf
> attachments etc) We use a centralised SAN store for the upload
> locations. All uploads are also written to a temporary location and then
> rsync''d to the DR location every 15 minutes.
>
>     - For SQL - we use MySQL replication, in a Master-Master setup
> between live and DR locations, with each master then having slaves to
> balance the load
>
> Cheers
>
> Tim
>
>
>
> -----Original Message-----
> From: drmikecrowe <drmikecr...@gmail.com>
>
> Reply-to: puppet-users@googlegroups.com
> To: Puppet Users <puppet-users@googlegroups.com>
> Subject: [Puppet Users] POLL: Migrating web changes across load-balanced
> servers
> Date: Sun, 14 Jun 2009 14:05:24 -0700 (PDT)
>
> Hi folks,
>
> I''m polling the group to see what some basic techniques are that
> people use to migrate web sites to load-balanced servers.
>
> In the past, IRC folks have mentioned rsysnc as the way.
>
> I''m specifically curious if people are:
>
> 1) Letting puppet MD5 a website via a file{} tag?  Could that be done?
> 2) Using version control somehow?  I''ve been toying with creating
a
> mercurial plugin somehow.  If you push some changes to a production
> machine, you can configure mercurial to auto-update.
>
> What''s some ideas?  Any pointers?
>
> TIA
> Mike
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---

On Mon, Jun 15, 2009 at 08:41:31AM -0700, drmikecrowe
wrote:
> 
> Hi Tim,
> 
> That''s very close to where I am now, except we use mercurial
instead
> of svn.  :)
> 
> However, I''m more searching for non-same hosts (i.e. no SAN). 
I''m
> basically asking the question of puppet managed ( file
> { source=>"puppet://.... } ) vs. another scheme.
> 
> My hunch is I do NOT want puppet managing 1000''s of files and
> propagating them.  Maybe have puppet monitor a version file, and when
> that changes, execute an rsync to bring the directories into sync.
Mike,

I sync a shell script which has a version number in it which I
increment.  When the script gets deployed it gets executed, 
(refreshonly => true in the exec) which runs rsync to deploy the site.  It 
also does other things, like using IPtables to block inbound connections 
whilst the rsync runs, so the load balancers don''t send any traffic to
the node
during the update.

Regards,

-- 
Sam Bashton - Bashton Ltd, Manchester, England
Linux Consultancy / High Availability Systems
www.bashton.com / 0161-424-9600 / DDI 0161-424-9611

On Mon, 15 Jun 2009 08:41:31 -0700 (PDT)
drmikecrowe <drmikecrowe@gmail.com> wrote:
> 
> Hi Tim,
> 
> That''s very close to where I am now, except we use mercurial
instead
> of svn.  :)
> 
> However, I''m more searching for non-same hosts (i.e. no SAN). 
I''m
> basically asking the question of puppet managed ( file
> { source=>"puppet://.... } ) vs. another scheme.
> 
> My hunch is I do NOT want puppet managing 1000''s of files and
> propagating them.  Maybe have puppet monitor a version file, and when
> that changes, execute an rsync to bring the directories into sync.
I have similar needs but have not found any satisfying solution yet.

Maybe this feature might prove useful, once it is implemented:
http://projects.reductivelabs.com/issues/184

I imagine this would allow us to do something like this:

file { "/var/www/whatever/":
  ensure => "r123",
  source => "svn://myproject.com/svn/repository/"
}

Marc



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---