dovecot - Apr 2022 - Bad Signature - Both Roundcube and Squirrelmail webmail cannot search for anything + cannot open many emails because there are more than 200, 000 emails in my Inbox

On 4/19/2022 5:12 PM, Turritopsis Dohrnii Teo En Ming
wrote:
> It is not true that no email system was ever designed for 200,000 or
> more emails.
>
> If I did not remember wrongly, more than 10 years ago, I used to have
> a Gmail email account, with 4-5 MILLION email messages in the Inbox.
> There was no problem searching even though I had 4-5 MILLION email
> messages in the Inbox.
If you were using either a Google app or the gmail.com website to access 
that inbox, then it doesn't compare.? I really doubt that either of 
those uses the IMAP protocol.? They would be using something proprietary 
that is highly optimized for the way that Google stores data and 
leverages the enormous amounts of computing power that they maintain.

I would bet that if you accessed a gmail folder with 5 million messages 
in it using IMAP, you would have similar problems with it to those that 
have been described here in this thread.? IMAP is a beautiful protocol, 
but I don't think it was designed for handling that many messages.

Thanks,
Shawn

On Tue, Apr 19, 2022 at 07:26:10PM -0600, Shawn Heisey
wrote:
> I would bet that if you accessed a gmail folder with 5 million
> messages in it using IMAP, you would have similar problems with it to
> those that have been described here in this thread.? IMAP is a
> beautiful protocol, but I don't think it was designed for handling
> that many messages.
This.  Sadly, Mark Crispin (author of IMAP and IMAP2) is no longer with
us to confirm.

Even at just ~100B for each message's headers, your IMAP2 client would
likely need at least ~500MB free RAM to load 5 million mails.

By the time Mark stopped working on UW IMAP (the reference IMAP
implementation, aka Panda IMAP), c.2010, even top-of-the-range
smartphones typically had only ~512MB RAM total, and top-of-the-range
ThinkPads had max ~4GiB (which was the upper limit of what 32-bit
operating systems, still popular then, could handle).

When IMAP2 was invented, c.1988-1990, RAM like that was basically
supercomputer territory.

Had Mark intended or expected IMAP2 users to have had supercomputers at
their disposal, and to be searching such large volumes of mail over the
protocol, I suspect he would have designed the protocol differently: for
raw efficiency over human readability.

My advice for anyone who wants to be able to keep and search very large
monolithic mailstores would be: synchronise (e.g. via
https://isync.sourceforge.io/ or fetchmail or getmail or rsync or Unison
or whatever) those mailstores from the server onto your local
filesystem; use maildir on your local filesystem; and use either Mutt's
"limiting" functions, or notmuch's index/search functions, for
searching/browsing.

Good luck in your quest!

Sam

I believe Gmail is using IMAP. The instructions for configuring Gmail
email accounts in Outlook specifically mention IMAP server hostname:

imap.gmail.com

TCP Port 993, SSL

Regards,

Mr. Turritopsis Dohrnii Teo En Ming
Targeted Individual in Singapore
20 Apr 2022 Wednesday


On Wed, 20 Apr 2022 at 09:26, Shawn Heisey <elyograg at elyograg.org>
wrote:
>
> On 4/19/2022 5:12 PM, Turritopsis Dohrnii Teo En Ming wrote:
> > It is not true that no email system was ever designed for 200,000 or
> > more emails.
> >
> > If I did not remember wrongly, more than 10 years ago, I used to have
> > a Gmail email account, with 4-5 MILLION email messages in the Inbox.
> > There was no problem searching even though I had 4-5 MILLION email
> > messages in the Inbox.
>
> If you were using either a Google app or the gmail.com website to access
> that inbox, then it doesn't compare.  I really doubt that either of
> those uses the IMAP protocol.  They would be using something proprietary
> that is highly optimized for the way that Google stores data and
> leverages the enormous amounts of computing power that they maintain.
>
> I would bet that if you accessed a gmail folder with 5 million messages
> in it using IMAP, you would have similar problems with it to those that
> have been described here in this thread.  IMAP is a beautiful protocol,
> but I don't think it was designed for handling that many messages.
>
> Thanks,
> Shawn
>