freebsd stable - Jan 2021 - How to handle the pack files now we have switched to git?

So, for me the switch to git went very smoothly. I havent moved to 
etcupdate yet, but will probably do that soon. Hopwever I did hit one 
issue. What I do is to build on a single machine, and then send that to 
a number of places using rsync.

But what seems to happen wuth git is that it has a big pack file of 
objects, and the name of the opack file is the SHA1 of whatever is 
inside it. So if something chnages then the filename chnages - and thus 
rsync tries to move the entire lot all over again, even if the change is 
tiny.

Does anyone have a solution to this ? Currently I am telling rsync to 
ignore .git when it sends the source, but am not entirely happy with that.

But apart from that, git works fine ;)

-pete.

Van: Pete French <petefrench at ingresso.co.uk>
Datum: dinsdag, 5 januari 2021 12:08
Aan: FreeBSD Stable Mailing List <freebsd-stable at freebsd.org>
Onderwerp: How to handle the pack files now we have switched to
git?
> 
> So, for me the switch to git went very smoothly. I havent moved to
etcupdate yet, but will probably do that soon. Hopwever I did hit one issue.
What I do is to build on a single machine, and then send that to a number of
places using rsync.
> 
> But what seems to happen wuth git is that it has a big pack file of
objects, and the name of the opack file is the SHA1 of whatever is inside it. So
if something chnages then the filename chnages - and thus rsync tries to move
the entire lot all over again, even if the change is tiny.
> 
> Does anyone have a solution to this ? Currently I am telling rsync to
ignore .git when it sends the source, but am not entirely happy with that.
> 
> But apart from that, git works fine ;)
> 
> -pete.
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at
freebsd.org"
> 
> 
> 

Why are you not happy with ignoring .git? It sounds like a pretty reasonable
thing to do for your setup.
Otherwise (if you keep the .git dir) there is nothing you can do about this
(AFAIK) and you will rsync quite some data now and then.

Regards,
Ronald.

On 2021-01-05 03:08, Pete French wrote:
> So, for me the switch to git went very smoothly. I havent moved to
etcupdate
> yet,
> but will probably do that soon. Hopwever I did hit one issue. What I do is 
> to
> build on a single machine, and then send that to a number of places using 
> rsync.
> 
> But what seems to happen wuth git is that it has a big pack file of
objects,
> and
> the name of the opack file is the SHA1 of whatever is inside it. So if 
> something
> chnages then the filename chnages - and thus rsync tries to move the entire
> lot
> all over again, even if the change is tiny.
> 
> Does anyone have a solution to this ? Currently I am telling rsync to
ignore
> .git
> when it sends the source, but am not entirely happy with that.
> 
> But apart from that, git works fine ;)
Brilliant minds think alike. I use nearly the same routine. ;-)
Have been doing so on svn(1) as well. FWIW the only difference between 
passing
the archive with vs without the .git. Is that getting the details of what rev
the source is at will not be available without the .git. Same on svn.
For me this is only a minor inconvenience. To help overcome this in the past.
I issued a 'svn info /usr/src >./SRC-REV'. Then simply passed that
file along
to the receiving host(s).
OTOH it won't be (easily) possible to "up" the repo(s) from the
receiving
hosts
w/o the .git.

HTH

--Chris
> 
> -pete.
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at
freebsd.org"

On Tue, 2021-01-05 at 11:08:48 +0000, Pete French wrote:
>So, for me the switch to git went very smoothly. I havent moved to
>etcupdate yet, but will probably do that soon. Hopwever I did hit one
>issue. What I do is to build on a single machine, and then send that to
>a number of places using rsync.
>
>But what seems to happen wuth git is that it has a big pack file of
>objects, and the name of the opack file is the SHA1 of whatever is
>inside it. So if something chnages then the filename chnages - and thus
>rsync tries to move the entire lot all over again, even if the change is
>tiny.
That's not entirely correct. The packfile will only be rewritten if
a) enough other stuff has accumulated
b) you force a repack.

You are fighting against git's GC mechanism a bit here, but you can 
still make it work. On your source of truth host, do the following:

1. git gc --aggressive
2. look at .git/objects/pack, there should be a single big pack
3. touch .git/objects/pack/pack-<whateverhash>.keep  (or was it
.pack.keep?)
4. rsync --del to all other hosts

Now future git gc runs will not delete that big pack, you'll only get 
churn in the new, much smaller, packs.

hth
Uli