Puppet users - Jan 2012 - proper way to mirror apt.puppetlabs.com?

I am wanting to set up a local mirror of apt.puppetlabs.com, and am a
bit confused by the layout as well as the announcement that was made
in this thread:

http://groups.google.com/group/puppet-users/browse_thread/thread/0786824ea9531fe3/f53c9006843ab8dc

I''m not sure exactly what I should be pulling, as
rsync://apt.puppetlabs.com/packages/apt
gets everything, but it seems like a bit of a mess. The top-level
pool/ directory is older than the top-level dists/ directory, but that
seems to be what is recommended in the above thread. If you look just
at the subdirectory ubuntu/ everything is in sync (all updated on Jan
3rd, 2012), however there are a ton of date-specific directories
listed under http://apt.puppetlabs.com/ubuntu/dists/ that seem to have
duplicate information.

I''m hoping that someone will know the proper way to rsync this mirror,
so things are compatible going forward. Thanks!

--
Aaron Bull Schaefer

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

The recommendation for mirroring debian repositories is generally to use
something like apt-mirror or apt-cacher-ng.  Is there a reason you
can''t do
this?

These work by checking the index (e.g.
http://apt.puppetlabs.com/ubuntu/dists/lucid/main/binary-amd64/Packages)
and pulling the packages.  You can''t walk a HTTP repository, HTTP
isn''t
smart like that.

On 6 January 2012 01:46, Aaron Schaefer <aaronschaefer@gmail.com> wrote:
> I am wanting to set up a local mirror of apt.puppetlabs.com, and am a
> bit confused by the layout as well as the announcement that was made
> in this thread:
>
>
>
http://groups.google.com/group/puppet-users/browse_thread/thread/0786824ea9531fe3/f53c9006843ab8dc
>
> I''m not sure exactly what I should be pulling, as rsync://
> apt.puppetlabs.com/packages/apt
> gets everything, but it seems like a bit of a mess. The top-level
> pool/ directory is older than the top-level dists/ directory, but that
> seems to be what is recommended in the above thread. If you look just
> at the subdirectory ubuntu/ everything is in sync (all updated on Jan
> 3rd, 2012), however there are a ton of date-specific directories
> listed under http://apt.puppetlabs.com/ubuntu/dists/ that seem to have
> duplicate information.
>
> I''m hoping that someone will know the proper way to rsync this
mirror,
> so things are compatible going forward. Thanks!
>
> --
> Aaron Bull Schaefer
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jan 5, 6:19 pm, Adam Gibbins <a...@adamgibbins.com>
wrote:
> The recommendation for mirroring debian repositories is generally to use
> something like apt-mirror or apt-cacher-ng.  Is there a reason you
can''t do
> this?

Our local mirror server is running CentOS, so I didn''t think apt-
mirror was an option, but it looks like we''re using debmirror for
Ubuntu repositories already, so I''ll modify our script to add
apt.puppetlabs.com in there as well. Thanks for pointing me in the
right direction...I had just set up rsyncing for yum.puppetlabs.com
without issue and was perplexed by the apt-side of things.

--
Aaron Bull Schaefer

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Jan 5, 9:42 pm, Aaron Schaefer <aaronschae...@gmail.com>
wrote:
> On Jan 5, 6:19 pm, Adam Gibbins <a...@adamgibbins.com> wrote:
>
> > The recommendation for mirroring debian repositories is generally to
use
> > something like apt-mirror or apt-cacher-ng.  Is there a reason you
can''t do
> > this?
>
> Our local mirror server is running CentOS, so I didn''t think apt-
> mirror was an option, but it looks like we''re using debmirror for
> Ubuntu repositories already, so I''ll modify our script to add
> apt.puppetlabs.com in there as well. Thanks for pointing me in the
> right direction...I had just set up rsyncing for yum.puppetlabs.com
> without issue and was perplexed by the apt-side of things.
The rsync instructions elsewhere in the group and on google don''t seem
to work for me, and nmapping yum.puppetlabs.com shows only 80/443
open.  What''s the correct way to mirror the yum repo these days?

Thanks!

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

I use debmirror for apt, reposync for yum (followed by createrepo).
Both work really well (debmirror took a little to get going under
CentOS, but now it works like a charm).

Matt

On Thu, Feb 2, 2012 at 6:42 PM, unixdoes <unixdoes@gmail.com>
wrote:
> On Jan 5, 9:42 pm, Aaron Schaefer <aaronschae...@gmail.com> wrote:
>> On Jan 5, 6:19 pm, Adam Gibbins <a...@adamgibbins.com> wrote:
>>
>> > The recommendation for mirroring debian repositories is generally
to use
>> > something like apt-mirror or apt-cacher-ng.  Is there a reason you
can''t do
>> > this?
>>
>> Our local mirror server is running CentOS, so I didn''t think
apt-
>> mirror was an option, but it looks like we''re using debmirror
for
>> Ubuntu repositories already, so I''ll modify our script to add
>> apt.puppetlabs.com in there as well. Thanks for pointing me in the
>> right direction...I had just set up rsyncing for yum.puppetlabs.com
>> without issue and was perplexed by the apt-side of things.
>
> The rsync instructions elsewhere in the group and on google don''t
seem
> to work for me, and nmapping yum.puppetlabs.com shows only 80/443
> open.  What''s the correct way to mirror the yum repo these days?
>
> Thanks!
>
> --
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
>


-- 
Matthew Nicholson

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

When the announcement was made, the rsync configuration was not changed, so the
excludes were not in sync with reality.  I have just reviewed the settings on
the rsync server and tested better excludes.  The following command is now
working like I expect.

rsync -av --copy-links --del rsync://apt.puppetlabs.com/packages/apt/ apt/

Let me know if this helps you.

-- 
Zach Leslie :: Puppet Labs :: 503.208.9791

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.