dovecot - Apr 2022 - Bad Signature - Both Roundcube and Squirrelmail webmail cannot search for anything + cannot open many emails because there are more than 200, 000 emails in my Inbox

> My advice for anyone who wants to be able to keep and search very large
> monolithic mailstores would be: synchronise (e.g. via
> https://isync.sourceforge.io/ or fetchmail or getmail or rsync or Unison
> or whatever) those mailstores from the server onto your local
> filesystem; use maildir on your local filesystem; and use either Mutt's
> "limiting" functions, or notmuch's index/search functions,
for
> searching/browsing.
Wouldn't it be very tedious and time consuming to sync mailboxes from
the server onto our local filesystems?

Regards,

Mr. Turritopsis Dohrnii Teo En Ming
Targeted Individual in Singapore
20 Apr 2022 Wednesday



On Wed, 20 Apr 2022 at 14:21, Sam Kuper <sampablokuper at posteo.net>
wrote:
>
> On Tue, Apr 19, 2022 at 07:26:10PM -0600, Shawn Heisey wrote:
> > I would bet that if you accessed a gmail folder with 5 million
> > messages in it using IMAP, you would have similar problems with it to
> > those that have been described here in this thread.  IMAP is a
> > beautiful protocol, but I don't think it was designed for handling
> > that many messages.
>
> This.  Sadly, Mark Crispin (author of IMAP and IMAP2) is no longer with
> us to confirm.
>
> Even at just ~100B for each message's headers, your IMAP2 client would
> likely need at least ~500MB free RAM to load 5 million mails.
>
> By the time Mark stopped working on UW IMAP (the reference IMAP
> implementation, aka Panda IMAP), c.2010, even top-of-the-range
> smartphones typically had only ~512MB RAM total, and top-of-the-range
> ThinkPads had max ~4GiB (which was the upper limit of what 32-bit
> operating systems, still popular then, could handle).
>
> When IMAP2 was invented, c.1988-1990, RAM like that was basically
> supercomputer territory.
>
> Had Mark intended or expected IMAP2 users to have had supercomputers at
> their disposal, and to be searching such large volumes of mail over the
> protocol, I suspect he would have designed the protocol differently: for
> raw efficiency over human readability.
>
> My advice for anyone who wants to be able to keep and search very large
> monolithic mailstores would be: synchronise (e.g. via
> https://isync.sourceforge.io/ or fetchmail or getmail or rsync or Unison
> or whatever) those mailstores from the server onto your local
> filesystem; use maildir on your local filesystem; and use either Mutt's
> "limiting" functions, or notmuch's index/search functions,
for
> searching/browsing.
>
> Good luck in your quest!
>
> Sam

On Wed, Apr 20, 2022 at 09:55:02PM +0800, Turritopsis Dohrnii Teo En Ming
wrote:
>> My advice for anyone who wants to be able to keep and search very large
>> monolithic mailstores would be: synchronise (e.g. via
>> https://isync.sourceforge.io/ or fetchmail or getmail or rsync or
Unison
>> or whatever) those mailstores from the server onto your local
>> filesystem; use maildir on your local filesystem; and use either
Mutt's
>> "limiting" functions, or notmuch's index/search
functions, for
>> searching/browsing.
> 
> Wouldn't it be very tedious and time consuming to sync mailboxes from
> the server onto our local filesystems?
Setting it up can be tedious depending on your needs.

If your mailserver is self-hosted, giving you direct access to the
filesystem, that gives you more options (rsync, Unison, Dovecot dsync,
etc).

Even if not, it's still possible using isync, fetchmail, or whatever
(see URL above).

Once the initial sync is performed, though, subsequent syncs should only
need to transmit the difference between the local and remote mailstores,
and therefore should usually be fast and, optionally, automatic.

Sam