CentOS - Feb 2018 - Any alternatives for the horrible reposync

Hi,
I'm currently trying to mirror a couple of yum repositories and the only
tool that seems to be available for this is reposync.
Unfortunately reposync for some inexplicable reason seems to use the yum
config of the local system as a basis for its work which makes no sense
and creates all kinds of problems where cache directories and metadata
gets mixed up.
Are there any alternatives? Some repos support rsync but not all of them
so I'm looking for something that works for all repos.

Regards,
  Dennis

On Tue, 27 Feb 2018, Dennis Jacobfeuerborn wrote:
> Hi,
> I'm currently trying to mirror a couple of yum repositories and the
only
> tool that seems to be available for this is reposync.
> Unfortunately reposync for some inexplicable reason seems to use the yum
> config of the local system as a basis for its work which makes no sense
> and creates all kinds of problems where cache directories and metadata
> gets mixed up.
> Are there any alternatives? Some repos support rsync but not all of them
> so I'm looking for something that works for all repos.
reposync uses whatever config file you point it at with "-c", only
using
/etc/yum.conf if you don't bother.

jh

On Tue, Feb 27, 2018 at 12:11:29PM +0100, Dennis Jacobfeuerborn
wrote:
> I'm currently trying to mirror a couple of yum repositories and the
only
> tool that seems to be available for this is reposync.
> Unfortunately reposync for some inexplicable reason seems to use the yum
> config of the local system as a basis for its work which makes no sense
> and creates all kinds of problems where cache directories and metadata
> gets mixed up.
> Are there any alternatives? Some repos support rsync but not all of them
> so I'm looking for something that works for all repos.
You might try "dnf reposync".


-- 
Matthew Miller
<mattdm at fedoraproject.org>
Fedora Project Leader

Try rsync where possible, lftp (mirror feature) where not (that's what I
do).

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

----- Original Message -----
> From: "Dennis Jacobfeuerborn" <dennisml at conversis.de>
> To: "CentOS mailing list" <centos at centos.org>
> Sent: Tuesday, 27 February, 2018 11:11:29
> Subject: [CentOS] Any alternatives for the horrible reposync
> Hi,
> I'm currently trying to mirror a couple of yum repositories and the
only
> tool that seems to be available for this is reposync.
> Unfortunately reposync for some inexplicable reason seems to use the yum
> config of the local system as a basis for its work which makes no sense
> and creates all kinds of problems where cache directories and metadata
> gets mixed up.
> Are there any alternatives? Some repos support rsync but not all of them
> so I'm looking for something that works for all repos.
> 
> Regards,
>  Dennis
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> https://lists.centos.org/mailman/listinfo/centos

On 27 February 2018 at 06:11, Dennis Jacobfeuerborn
<dennisml at conversis.de> wrote:
> Hi,
> I'm currently trying to mirror a couple of yum repositories and the
only
> tool that seems to be available for this is reposync.
> Unfortunately reposync for some inexplicable reason seems to use the yum
> config of the local system as a basis for its work which makes no sense
> and creates all kinds of problems where cache directories and metadata
> gets mixed up.
> Are there any alternatives? Some repos support rsync but not all of them
> so I'm looking for something that works for all repos.
>
It is not 'inexplicable'. reposync was primarily built for a user to
sync down the repositories they are using to be local.. so using
yum.conf makes sense. The fact that it can be used for a lot of other
things is built into various configs which the man page covers. As
John Hodrien mentioned, you can use the -C flag to point it to a
different config file. This is the way to use it if you are wanting to
download other files and data. Tools like cobbler wrap the reposync in
this fashion.



> Regards,
>   Dennis
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> https://lists.centos.org/mailman/listinfo/centos


-- 
Stephen J Smoogen.

On 27.02.2018 16:45, Stephen John Smoogen wrote:
> On 27 February 2018 at 06:11, Dennis Jacobfeuerborn
> <dennisml at conversis.de> wrote:
>> Hi,
>> I'm currently trying to mirror a couple of yum repositories and the
only
>> tool that seems to be available for this is reposync.
>> Unfortunately reposync for some inexplicable reason seems to use the
yum
>> config of the local system as a basis for its work which makes no sense
>> and creates all kinds of problems where cache directories and metadata
>> gets mixed up.
>> Are there any alternatives? Some repos support rsync but not all of
them
>> so I'm looking for something that works for all repos.
>>
> 
> It is not 'inexplicable'. reposync was primarily built for a user
to
> sync down the repositories they are using to be local.. so using
> yum.conf makes sense. The fact that it can be used for a lot of other
> things is built into various configs which the man page covers. As
> John Hodrien mentioned, you can use the -C flag to point it to a
> different config file. This is the way to use it if you are wanting to
> download other files and data. Tools like cobbler wrap the reposync in
> this fashion.
I've been trying to use a custom config file and the -t option to
somehow separate the operation of reposync from the systems repositories
but this does not seem to work.
When I tried to copy a repository reposync reported that it already had
a more current repomd.xml than the one offered by the repository.
An investigation with strace revealed that reposync still was checking
/var/cache/yum for cached files even though i tried both -t and
explicitly creating a new directory and using --cachedir.
Reposync looks into that cache directory and doesn't find the repomd.xml
which is great but then also checks /var/cache/yum where it finds a
repomd.xml and uses that.

What I mean by inexplicable is that it would make more sense to make
reposync a generic tool for syncing yum repos and then simply provide
the option to use /etc/yum.conf rather then to hard-code (as is
apparently the case) these system specific behaviors.

For now what seems to work is using a unique repo name in the config
file to make it impossible for reposync to find a matching file in
/var/cache/yum but that's more of a hack than a fix for the issue.

Regards,
  Dennis