dovecot - Apr 2016 - expunging messages doesn't update maildirsize

This might be something that everybody thinks of as a well-known fact, 
but I've been searching around quite a bit without finding anything 
definitive. Maybe I just have the wrong idea of what's supposed to be 
going on.

I'm using maildir+ quotas. For arriving mail, it works as I expect: new 
lines are appended to maildirsize to reflect the size of the new 
messages. But I also expected that mail that is deleted (and expunged, 
yes totally gone from the disk drive, verified from the command line) 
would lead to some kind of update to maildirsize or maybe even a 
complete recalculation. But that does not happen on my system (dovecot 
2.2.9 on a Ubuntu variant).

Now, I know that I can run "doveadm" from a cron job or whatever to
have
the maildirsize info recomputed. I also know that the maildirsize file 
will be recreated when it gets larger than 4k. That's all fine. But I 
don't see why I should have to have this out-of-band cron job, and I 
don't see why my users should have to wait if their maildirsize file 
doesn't tip over the magical 4k mark (which is a lot of individual 
messages being recorded).

Is it by design that dovecot doesn't update maildirsize immediately when 
messages are expunged? If yes, why?

On 04/19/2016 07:17 PM, WJCarpenter wrote:
> Is it by design that dovecot doesn't update maildirsize immediately 
> when messages are expunged? If yes, why?
Replying to my own question since I figured it out. Local config glitch, 
not a dovecot bug or design fault.

I should have mentioned that I don't use dovecot's local delivery agent.
Updates to maildirsize for newly arriving messages is done by exim.

Boring details follow. They become interesting details if you are having 
the same problem. :-)

I use the SQL backend for keeping track of user data, including quota 
values. Consequently, I have configured 3 very similar (but not 
identical) SQL queries for that user data. One is in my exim config, and 
two (password_query and user_query) are in my dovecot-sql.conf.

Those of you familiar with dovecot's SQL queries will know that the 
password_query can also return values that would ordinarily be looked up 
via user_query, the motivation being that user_query can often be 
skipped completely. The dovecot 2.2.9 release broke that userdb prefetch 
feature (it was fixed soon after, but I am using my distribution's 
dovecot version [Ubuntu 14.04], which is stuck at 2.2.9). Even though 
userdb prefetch doesn't work and both passdb and userdb queries are both 
done every time, my user_query is just a clone of my password_query in 
anticipation of the day when I can get rid of user_query.

It's the cloning of the query where things went wrong. In the 
password_query, the prefetched user quota value is returned as 
"userdb_quota". In the user_query, it's returned as
"quota". When I
cloned the query, I neglected to change "userdb_quota" to
"quota" in my
select list. (That's a bummer, because I did correctly change 
"userdb_home" to "home" in my select list, so I must've
known I should
do it.)

The net is that dovecot did the password_query and got the correct value 
for the user quota. It then ignored that (due to the prefetch bug) and 
did the user_query. The user_query reported the user quota with the 
wrong field name, and dovecot figured there was no quota for that user 
(which means /every/ user).

Once I fixed my 7-character configuration mistake, it started working 
correctly.

For the sake of brevity, I've left out the parts where I went down a 
bunch of blind alleys before finding the correct solution. :-)