Just ideas. Removing or deleting a single message from near the beginning of a large flat file takes an inordinate amount of time because the remainder of the flat file has to be rewritten all the way from the point of the deleted message to the end of the file and then truncated. On January 30, 2022 6:30:44 PM AKST, Sam Kuper <sampablokuper at posteo.net> wrote:>On Sun, Jan 30, 2022 at 06:17:49PM -0900, justina colmena ~biz wrote: >> On January 30, 2022 5:46:53 PM AKST, dovecot at ptld.com wrote: >>> Storing mail in a db... at the end of the day isn't it still just a >>> file (.db file) on the drive? >>> >>> Aren't you just adding bloat and complexity vs just storing the mail >>> directly (maildir format) to a file on the drive? [...] >> >> You'll get better indexing and fast full text search by storing your >> emails in a database rather than a flat file, hopefully after decoding >> any attachments. Especially for spam scoring, analysis, and >> classification. Much better performance deleting or moving specific >> messages, too. > >Do you have evidence to back up these claims, specifically re: mail >servers? > >Like-for-like benchmarks, for instance? > >Thanks, > >Sam > >-- >A: When it messes up the order in which people normally read text. >Q: When is top-posting a bad thing? > >() ASCII ribbon campaign. Please avoid HTML emails & proprietary >/\ file formats. (Why? See e.g. https://v.gd/jrmGbS ). Thank you.-- Sent from my Android device with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20220130/73eeca40/attachment.htm>
On Sun, Jan 30, 2022 at 07:49:56PM -0900, justina colmena ~biz wrote:> On January 30, 2022 6:30:44 PM AKST, Sam Kuper wrote: >> On Sun, Jan 30, 2022 at 06:17:49PM -0900, justina colmena ~biz wrote: >>> On January 30, 2022 5:46:53 PM AKST, dovecot at ptld.com wrote: >>>> Storing mail in a db... at the end of the day isn't it still just a >>>> file (.db file) on the drive? >>>> >>>> Aren't you just adding bloat and complexity vs just storing the >>>> mail directly (maildir format) to a file on the drive? [...] >>> >>> You'll get better indexing and fast full text search by storing your >>> emails in a database rather than a flat file, hopefully after >>> decoding any attachments. Especially for spam scoring, analysis, and >>> classification. Much better performance deleting or moving specific >>> messages, too. >> >> Do you have evidence to back up these claims, specifically re: mail >> servers? >> >> Like-for-like benchmarks, for instance? > > Just ideas.OK, no then.> Removing or deleting a single message from near the beginning of a > large flat file takes an inordinate amount of time because the > remainder of the flat file has to be rewritten all the way from the > point of the deleted message to the end of the file and then > truncated.You might want to look up what Maildir is before making bold but apparently unfounded claims about it. Maildir is not a "large flat file". It is a set of conventions that amount to a database specification, in the traditional sense of the word "database": a system for storing data. (Not a relational database.) DJB developed Maildir to gain performance and reliability improvements over mbox files. Unlike Maildirs, mbox files *are* "large flat files". Best wishes, Sam -- A: When it messes up the order in which people normally read text. Q: When is top-posting a bad thing? () ASCII ribbon campaign. Please avoid HTML emails & proprietary /\ file formats. (Why? See e.g. https://v.gd/jrmGbS ). Thank you.
> > Just ideas.Maybe an idea to participate on a Microsoft forum? They like to use db's for email, and they are removing everything what is nice in order to push people into their cloud. So lots to change for the better there. It's so crappy that I recently wrote Bill Gates that he should not whine so much about the environment, because if he used only half of his profits to optimize code/designs in ms products, this would result in a significant reduction in energy use. Think of what global effect that has. FYI T-mobile (and the commercial version of dovecot?) is working on storing emails in object storage, that is the future.
On 2022-01-31 05:49, justina colmena ~biz wrote:> Just ideas. > > Removing or deleting a single message from near the beginning of a > large flat file takes an inordinate amount of time because the > remainder of the flat file has to be rewritten all the way from the > point of the deleted message to the end of the file and then > truncated.maybe why postfix only support maildir++ all other formats is properitary formats