dovecot - Oct 2020 - Version controlled (git) Maildir generated by Dovecot

Hi David, 

Please find answers bellow.

Kind regards,
Adam

---------- P?vodn? e-mail ----------
Od: David Myers <david.myers.24j74 at gmail.com>
Komu: Adam <adam.ranek at seznam.cz>, Dovecot Mailing List <dovecot at
dovecot.org>
Datum: 7. 10. 2020 8:44:28
P?edm?t: Re: Version controlled (git) Maildir generated by Dovecot
Hello Adam, and the dovecot list
> Just a question, I hate to pollute the thread, so feel free to push these
> questions into a new thread if deemed necessary. So as you can guess
I'm a bit
> of a newb here, so rather obvious questions are about to arrive....
> 
> As you are using GIT for your archive (which is a cool idea by the way)
I'm
> sure you are well aware that not all files types play nicely with version
> control, my question therefore is : How do you plan to handle attachments ?
I use git for everything including for example LibreOffice / Word documents. Git
works just fine with binary files. You can't use text tools like "git
diff" but... it works.
> Also, although I appreciate the idea of using git, emails generally
don't
> 'change', but I guess that also depends on how you are storing them
(single
> email with links to previous / next ... etc, or as a single big file for
each
> specific thread). Although this is hitting my limits of understanding for
how
> dovecot works, so I probably need educating on this (a pointer to the docs
would
> be good).
As I mentioned in the first e-mail, I configured Dovecot to use Maildir format
-> each e-mail is a single text file. Mail body + attachment(s) are in the
same file, attachment(s) are Base64 encoded.
> You seem concerned regarding the files that you are ignoring that you will
need
> to 'recreate them', so why not do a complete git add . prior to
adding them into
> the git ignore, then you have an initial state for those files too.
But I don't want to store files that can be regenerated. I don't want to
backup stuff, that doesn't have information value.
> Final thought, what advantage do you envisage by using git as opposed to
simply
> using a filter to select the files over a certain age, and place them into
a
> zipped TAR archive ? Although I guess you could eventually zip the git
archive
> too, and in the interim it would remain searchable by your users mail
clients
> whilt in git.
I like to use git ;-). Tar will work just fine.

In this use case the only real benefit of git is that it never forgets. Unless I
delete whole .git directory, I can make a mistake, delete some e-mails (files),
commit changes and rollback. I can't rollback if I delete tar archive.
> Thanks in advance, and apologies once again for polluting your question
with my
> own.
David

Hello Adam,

thanks for the reply. Sounds fair enough to me. I hadn't thought about that
last benefit of git. I haven't deleted anything off my pc for years ...
still got HDD from 15 years ago with 'something' on them ?

Sorry this is going to go off topic somewhat .... (ok, I've just read it
again... its gone off topic a lot... again, apologies for that)

One more question, if you are using exclusively LibreOffice, I understand
it has a mode where it will separate the file into its constituent flat,
text XML files (style, contents, formatting etc), all of which can then be
stored in git with all the advantages that privides, no binary files
needed. Do you use this functionality ? I haven't done this so I don't
know
how it impacts the work flow for a user, or how it will integrate into a
git workflow, but would be interested to hear a user experience. I just use
the inbuilt 'versioning' that is available within libreoffice (much
better
than multiple copies of the same file with just a few changes).

Hopefully my last set of 'novice questions' ;)

thanks in advance.

David

On Wed, Oct 7, 2020 at 10:41 AM Adam <adam.ranek at seznam.cz> wrote:
> Hi David,
>
> Please find answers bellow.
>
> Kind regards,
> Adam
>
> ---------- P?vodn? e-mail ----------
> Od: David Myers <david.myers.24j74 at gmail.com>
> Komu: Adam <adam.ranek at seznam.cz>, Dovecot Mailing List <
> dovecot at dovecot.org>
> Datum: 7. 10. 2020 8:44:28
> P?edm?t: Re: Version controlled (git) Maildir generated by Dovecot
> Hello Adam, and the dovecot list
>
> > Just a question, I hate to pollute the thread, so feel free to push
these
> > questions into a new thread if deemed necessary. So as you can guess
I'm
> a bit
> > of a newb here, so rather obvious questions are about to arrive....
> >
> > As you are using GIT for your archive (which is a cool idea by the
way)
> I'm
> > sure you are well aware that not all files types play nicely with
version
> > control, my question therefore is : How do you plan to handle
> attachments ?
>
> I use git for everything including for example LibreOffice / Word
> documents. Git works just fine with binary files. You can't use text
tools
> like "git diff" but... it works.
>
> > Also, although I appreciate the idea of using git, emails generally
don't
> > 'change', but I guess that also depends on how you are storing
them
> (single
> > email with links to previous / next ... etc, or as a single big file
for
> each
> > specific thread). Although this is hitting my limits of understanding
> for how
> > dovecot works, so I probably need educating on this (a pointer to the
> docs would
> > be good).
>
> As I mentioned in the first e-mail, I configured Dovecot to use Maildir
> format -> each e-mail is a single text file. Mail body + attachment(s)
are
> in the same file, attachment(s) are Base64 encoded.
>
> > You seem concerned regarding the files that you are ignoring that you
> will need
> > to 'recreate them', so why not do a complete git add . prior
to adding
> them into
> > the git ignore, then you have an initial state for those files too.
>
> But I don't want to store files that can be regenerated. I don't
want to
> backup stuff, that doesn't have information value.
>
> > Final thought, what advantage do you envisage by using git as opposed
to
> simply
> > using a filter to select the files over a certain age, and place them
> into a
> > zipped TAR archive ? Although I guess you could eventually zip the git
> archive
> > too, and in the interim it would remain searchable by your users mail
> clients
> > whilt in git.
>
> I like to use git ;-). Tar will work just fine.
>
> In this use case the only real benefit of git is that it never forgets.
> Unless I delete whole .git directory, I can make a mistake, delete some
> e-mails (files), commit changes and rollback. I can't rollback if I
delete
> tar archive.
>
> > Thanks in advance, and apologies once again for polluting your
question
> with my
> > own.
>
> David
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20201007/882ee895/attachment.html>

Hi David,

I've never heard about such feature in LibreOffice. Thank you for letting me
know.

I don't really see myself using the feature since I'd have to remember
it. I'm used to store all sort of stuff / binary files in git. My rule of
thumb is that if the file is bellow 10M, just add/commit it. Is it proper way of
using git? No. Does it work? Yes ;-).

I think that the difference between us is that I'm used to use git for
everything and you aren't which means neither you or I are correct ;-).

Kind regards,
Adam

---------- P?vodn? e-mail ----------
Od: David Myers <david.myers.24j74 at gmail.com>
Komu: Adam <adam.ranek at seznam.cz>
Datum: 7. 10. 2020 10:05:28
P?edm?t: Re: Version controlled (git) Maildir generated by
Dovecot
> Hello Adam,
> 
> thanks for the reply. Sounds fair enough to me. I hadn't thought about
that last benefit of git. I haven't deleted anything off my pc for years ...
still got HDD from 15 years ago with 'something' on them ?
> 
> Sorry this is going to go off topic somewhat .... (ok, I've just read
it again... its gone off topic a lot... again, apologies for that)
> 
> 
> One more question, if you are using exclusively LibreOffice, I understand
it
> has a mode where it will separate the file into its constituent flat, text
XML
> files (style, contents, formatting etc), all of which can then be stored in
git
> with all the advantages that privides, no binary files needed. Do you use
this
> functionality ? I haven't done this so I don't know how it impacts
the work flow
> for a user, or how it will integrate into a git workflow, but would be
> interested to hear a user experience. I just use the inbuilt
'versioning' that
> is available within libreoffice (much better than multiple copies of the
same
> file with just a few changes).
> 
> Hopefully my last set of 'novice questions' ;)
> 
> thanks in advance.
> 
> David
> 
> On Wed, Oct 7, 2020 at 10:41 AM Adam <adam.ranek at seznam.cz> wrote:
> Hi David, 
> 
> Please find answers bellow.
> 
> Kind regards,
> Adam
> 
> ---------- P?vodn? e-mail ----------
> Od: David Myers <david.myers.24j74 at gmail.com>
> Komu: Adam <adam.ranek at seznam.cz>, Dovecot Mailing List
<dovecot at dovecot.org>
> Datum: 7. 10. 2020 8:44:28
> P?edm?t: Re: Version controlled (git) Maildir generated by Dovecot
> Hello Adam, and the dovecot list
> 
> > Just a question, I hate to pollute the thread, so feel free to push
these
> > questions into a new thread if deemed necessary. So as you can guess
I'm a bit
> > of a newb here, so rather obvious questions are about to arrive....
> > 
> > As you are using GIT for your archive (which is a cool idea by the
way) I'm
> > sure you are well aware that not all files types play nicely with
version
> > control, my question therefore is : How do you plan to handle
attachments ?
> 
> I use git for everything including for example LibreOffice / Word
documents. Git works just fine with binary files. You can't use text tools
like "git diff" but... it works.
> 
> > Also, although I appreciate the idea of using git, emails generally
don't
> > 'change', but I guess that also depends on how you are storing
them (single
> > email with links to previous / next ... etc, or as a single big file
for each
> > specific thread). Although this is hitting my limits of understanding
for how
> > dovecot works, so I probably need educating on this (a pointer to the
docs would
> > be good).
> 
> As I mentioned in the first e-mail, I configured Dovecot to use Maildir
format -> each e-mail is a single text file. Mail body + attachment(s) are in
the same file, attachment(s) are Base64 encoded.
> 
> > You seem concerned regarding the files that you are ignoring that you
will need
> > to 'recreate them', so why not do a complete git add . prior
to adding them into
> > the git ignore, then you have an initial state for those files too.
> 
> But I don't want to store files that can be regenerated. I don't
want to backup stuff, that doesn't have information value.
> 
> > Final thought, what advantage do you envisage by using git as opposed
to simply
> > using a filter to select the files over a certain age, and place them
into a
> > zipped TAR archive ? Although I guess you could eventually zip the git
archive
> > too, and in the interim it would remain searchable by your users mail
clients
> > whilt in git.
> 
> I like to use git ;-). Tar will work just fine.
> 
> In this use case the only real benefit of git is that it never forgets.
Unless I delete whole .git directory, I can make a mistake, delete some e-mails
(files), commit changes and rollback. I can't rollback if I delete tar
archive.
> 
> > Thanks in advance, and apologies once again for polluting your
question with my
> > own.