CentOS - Mar 2021 - proper recipe for local mirror

Hi! What is the proper solution for a local mirror for centos 7 (or oven 
8) repositories?

trying with rsync, while the packages are downloaded but the repodata is 
not and is actually impossible to be used by clients. while i try to 
recreate the repodata with
createrepo -C -d --workers=4 ${repo}
where ${repo} is the sync destination where it contains 
%arch/{Packages,repodata} directories

the clients throw errors like:
<LOCAL_MIRROR>/mirrors/centos/7/updates/x86_64/repodata/51d63b04c8bb74c5d9f09ae65f763f319a69492d9a8fac5ea0bf9b63bf347c5c-primary.sqlite.bz2:
[Errno 14] HTTP Error 404 - Not Found

What am i missing?
Thanks a lot!
Adrian

On Mar 4, 2021, at 7:04 AM, Adrian Sevcenco <Adrian.Sevcenco at
spacescience.ro> wrote:
> 
> What is the proper solution for a local mirror for centos 7 (or oven 8)
repositories?
https://www.osradar.com/how-to-create-centos-8-local-repository-mirrors-with-rsync-nginx/

> Hi! What is the proper solution for a local mirror for centos 7 (or oven
> 8) repositories?
>
> trying with rsync, while the packages are downloaded but the repodata is
> not and is actually impossible to be used by clients. while i try to
Hi,

The script below is how I sync CentOS to a local mirror which is then used
directly by the clients, no createrepo needed.

Please note that the exclude list is something to adjust to your needs to
save space and traffic. Also the $MIRRORS list should be changed to use
best mirrors for your situation.

Hope this helps,
Simon

---%<---
#!/bin/bash

RELEASES="{7,8}"
MIRRORS="rsync://mirrors.kernel.org rsync://mirror.xxxxxxx.xxx"
TOUT=60

LOCALDIR=$(readlink -f $(dirname $0))
MODULE=${LOCALDIR##*/}
RETVAL=1

# find a usable mirror
echo "list of mirrors \`${MIRRORS}'"
for TESTURL in $MIRRORS; do
  echo "testing mirror \`${TESTURL}/${MODULE}'"
  if rsync -qn ${TESTURL}/${MODULE}/ .; then
    # this mirror _should_ work ...
    echo "using mirror \`${TESTURL}/${MODULE}'"
    RSYNCURLS=$(eval echo ${TESTURL}/${MODULE}/RPM-GPG-KEY-CentOS-\*
${TESTURL}/${MODULE}/${RELEASES})
    echo "syncing \`${RSYNCURLS}'"
    # ... but it can still fail here
    if rsync -rtv --timeout=$TOUT --copy-links --delete --delete-excluded
--progress --no-motd \
      --exclude "aarch*/" \
      --exclude "alpha/" \
      --exclude "ia64/" \
      --exclude "ppc*/" \
      --exclude "s390*/" \
      --exclude "SRPMS/" \
      --exclude "debug/" \
      --exclude "*~" \
      --exclude ".*info" \
      --exclude "kickstart/" \
      --exclude "?/apt/" \
      --exclude "?/atomic/" \
      --exclude "?/build/" \
      --exclude "?/cloud/" \
      --exclude "?/docs/" \
      --exclude "?/dotnet/" \
      --exclude "?/isos/" \
      --exclude "?/nfv/" \
      --exclude "?/opstools/" \
      --exclude "?/os/" \
      --exclude "?/paas/" \
      --exclude "?/rt/" \
      --exclude "?/virt/" \
      "$@" $RSYNCURLS ${LOCALDIR}/
    then
      RETVAL=0
      break
    else
      echo "mirror \`${TESTURL}/${MODULE}' failed"
      RETVAL=1
    fi
  fi
done

exit $RETVAL
---%<---