dovecot - Apr 2009 - Quota calculation problem with 1.2 beta 4

Two of my users are reporting that their quota is not being
recalculated ALTHOUGH they deleted lots and lots of mails.

I'm using today's hg checkout of 1.2 beta 4

-- 
Ralf Hildebrandt
Postfix - Einrichtung, Betrieb und Wartung       Tel. +49 (0)30-450 570-155
http://www.computerbeschimpfung.de
Wen du nicht mit K?nnen beeindrucken kannst, den verwirre mit
Schwachsinn!
-------------- next part --------------
# 1.2.beta4: /usr/local/etc/dovecot.conf
# OS: Linux 2.6.29 i686 Debian squeeze/sid 
protocols: imap imaps pop3s
ssl_cert_file: /etc/ssl/certs/postamt.pem
ssl_key_file: /etc/ssl/private/postamt.key
disable_plaintext_auth: no
version_ignore: yes
login_dir: /usr/local/var/run/dovecot/login
login_executable(default): /usr/local/libexec/dovecot/imap-login -D
login_executable(imap): /usr/local/libexec/dovecot/imap-login -D
login_executable(pop3): /usr/local/libexec/dovecot/pop3-login -D
login_process_per_connection: no
login_process_size: 0
login_processes_count: 64
login_max_processes_count: 2048
login_max_connections: 512
max_mail_processes: 2048
mail_max_userip_connections: 2048
verbose_proctitle: yes
mail_location: maildir:~/Maildir
fsync_disable: yes
maildir_copy_preserve_filename: yes
mail_drop_priv_before_exec: yes
mail_executable(default): /usr/local/libexec/dovecot/imap
mail_executable(imap): /usr/local/libexec/dovecot/imap
mail_executable(pop3): /usr/local/libexec/dovecot/pop3
mail_plugins(default): quota imap_quota trash mail_log fts fts_squat zlib
autocreate
mail_plugins(imap): quota imap_quota trash mail_log fts fts_squat zlib
autocreate
mail_plugins(pop3): quota mail_log
mail_plugin_dir(default): /usr/local/lib/dovecot/imap
mail_plugin_dir(imap): /usr/local/lib/dovecot/imap
mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3
mail_log_max_lines_per_sec: 0
imap_client_workarounds(default): outlook-idle netscape-eoh
imap_client_workarounds(imap): outlook-idle netscape-eoh
imap_client_workarounds(pop3): 
pop3_lock_session(default): no
pop3_lock_session(imap): no
pop3_lock_session(pop3): yes
pop3_uidl_format(default): %08Xu%08Xv
pop3_uidl_format(imap): %08Xu%08Xv
pop3_uidl_format(pop3): %v-%u
pop3_client_workarounds(default): 
pop3_client_workarounds(imap): 
pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh
namespace:
  type: private
  separator: .
  prefix: INBOX.
  inbox: yes
  list: yes
  subscriptions: yes
auth default:
  mechanisms: plain login
  cache_size: 64
  master_user_separator: *
  debug: yes
  debug_passwords: yes
  passdb:
    driver: passwd-file
    args: /usr/local/etc/dovecot.masterusers
    pass: yes
    master: yes
  passdb:
    driver: shadow
  passdb:
    driver: pam
    args: cache_key=%u dovecot
  userdb:
    driver: passwd
  socket:
    type: listen
    client:
      path: /var/spool/postfix/private/auth
      mode: 432
      user: postfix
      group: postfix
plugin:
  fts: squat
  quota: maildir
  quota_rule: Trash:storage=2048M
  quota_warning: storage=90%% /usr/local/scripts/quota-warning 90
  quota_warning2: storage=95%% /usr/local/scripts/quota-warning 95
  quota_warning3: storage=99%% /usr/local/scripts/quota-warning 99
  trash: /usr/local/etc/dovecot-trash.conf
  autocreate: Trash
  autocreate2: spam
  autocreate3: Sent
  autocreate4: Drafts
  autosubscribe: Trash
  autosubscribe2: spam
  autosubscribe3: Sent
  autosubscribe4: Drafts

Ralf Hildebrandt schrieb:
> Two of my users are reporting that their quota is not being
> recalculated ALTHOUGH they deleted lots and lots of mails.
> 
> I'm using today's hg checkout of 1.2 beta 4
> 
> 
Hi Ralf, did they report so , or is it real,
sometimes users are failing with expunge options of their client

-- 
Best Regards

MfG Robert Schetterer

Germany/Munich/Bavaria

On Wed, 2009-04-01 at 11:28 +0200, Ralf Hildebrandt
wrote:
> Two of my users are reporting that their quota is not being
> recalculated ALTHOUGH they deleted lots and lots of mails.
Does maildirsize file show any such deletions? Like lots of lines /
large numbers in lines beginning with "-"?

I did a few tests and it seemed to work correctly. Also the Maildir++
quota code hasn't really changed since v1.1 (quota configuration
structures changed somewhat though).

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20090401/3e02e777/attachment-0002.bin>

On Wed, 1 Apr 2009, Wolfgang.Friebel at desy.de wrote:
> On Wed, 1 Apr 2009, Timo Sirainen wrote:
>
>>> but the IMAP command reported correctly what I also found in the
>>> maildirsize file (in userdb the configured limit was 1000M, his
actual
>>> INBOX size was below 600M):
>> 
>> Are you calling deliver with -d parameter? If not, the userdb lookup is
>> not done in that case. If you are, set mail_debug=yes and paste the
logs
>> when this happens.
>> 
>
> I did not call deliver with the -d parameter. I just added that right now
Sorry for the trouble. I did recheck that I already had called deliver 
using -d ${recipient}. That seems not to be the problem.

I made several experiments with quota settings and userdb while mail_debug 
was on. Here are my findings:

After changing /etc/userdb and modifying my 
userdb_quota_rule=*:storage=200M with an editor the IMAP GETQUOTAROOT does 
immediately report the changed value. The ~user/Maildir/maildirsize 
content does however not change and according to the maillog output the 
quota calculation is done based on the contents of that file.

If I do delete that file after changing the contents of the users INBOX 
the file gets regenerated with the contents from the quota rule (i.e. 
900MB, see below) and not with the value from the userdb (200M)

If then I do another IMAP GETQUOTAROOT the correct value is reported again 
(200M) and the contents of ~user/Maildir/maildirsize is changed to that 
value as well.

This is however not the only finding. I manipulated further my userdb 
quota and did set it to 90MB. Shortly after that another user (Quota 
900MB) received a mail. The debug output for that mail delivery:

dovecot: deliver(user2): Quota warning: bytes=89653 2480 (95%)...

indicates that my settings were used instead for quota calculation. The 
maildirsize content for that user was however still at 900MB, not 90MB.

I am now puzzled what should happen when and whether I am looking to the 
correct indicators for problems. I could make the log file available, I 
would however prefer to generate a logfile where I get told what to try in 
what order to get more information from that logfile.

It could be that the bug is somewhere in my config, which I do post again 
for reference (excerpt), but I cannot see what might be wrong with it.

protocol imap {
   mail_plugins = quota imap_quota zlib acl imap_acl
}

protocol lda {
   mail_plugins = sieve quota acl
}
auth default {
   mechanisms = gssapi gss-spnego login plain
   passdb pam {
   }
   userdb passwd-file {
        args = /etc/userdb
   }
   user = root
}
plugin {
   quota = maildir:Quota
   quota_rule = *:storage=900MB
   quota_rule2 = Trash:ignore
}


-- 
Wolfgang Friebel                   Deutsches Elektronen-Synchrotron DESY
Phone/Fax:  +49 33762 77372/216    Platanenallee 6
Mail: Wolfgang.Friebel AT desy.de  D-15738 Zeuthen  Germany

* Timo Sirainen <tss at iki.fi>:
> On Wed, 2009-04-01 at 11:28 +0200, Ralf Hildebrandt wrote:
> > Two of my users are reporting that their quota is not being
> > recalculated ALTHOUGH they deleted lots and lots of mails.
> 
> Does maildirsize file show any such deletions? Like lots of lines /
> large numbers in lines beginning with "-"?
> 
> I did a few tests and it seemed to work correctly. Also the Maildir++
> quota code hasn't really changed since v1.1 (quota configuration
> structures changed somewhat though).
It seems my users need to log out and back in again to get the correct
quota info displayed within Squirrel Mail

-- 
Ralf Hildebrandt
Postfix - Einrichtung, Betrieb und Wartung       Tel. +49 (0)30-450 570-155
http://www.computerbeschimpfung.de
Recipient user name postmaster (postmaster at domain.de) not unique. 
Several matches found in Domino Directory. 
Oder auch: Viele Postmaster verderben den Server!