dovecot - Apr 2014 - Why does it appear that dovecot is deleting messages after migration?

Hi, y'all, long-time Mac OS X Server user Bill here with a thorny question
for the mailing list since nobody at discussions.apple.com can quite put their
finger on the answer. While well-aquainted with other aspects of Server, Dovecot
is a bit of a mystery still to me. So here it goes:

Background

I'm struggling to migrate from MacOS X 10.6 to 10.9. I tried twice using the
"live server in target disk mode" method, but after things went
horribly awry (meaning I was left with no working services on the new server), I
gave up on that and have been working to slowly bring services over, one at a
time.

I've got both Dovecot and PostFix accepting connections from my test
machine, and I am able to connect with IMAP to the new server.

What I Do...

First, I turn both mail servers off. Then I tar up the
/var/spool/imap/dovecot/mail directory on the old server, move it to the new
server, untar it in /Library/Server/Mail/Data/ (where it belongs), and change
ownership to _dovecot:mail, all per many other suggestions found in the Apple
forums. (All files appear to be in maildir format, that I can tell.)

I then turn the mail service back on on the new server.

What Dovecot Does... (or appears to do)

If I quickly connect to the new server with my IMAP client (MacOS X Mail.app)
and create a "fresh user" in Mail.app, message folders appear
correctly. But it's only a matter of time before Dovecot, or something, runs
"doveadm index -u (usernames) (mailboxes)" on all of the mailboxes. As
it does, it deletes thousands of messages, leaving, for example, the same 17 in
INBOX and 3695 in "Deleted Items Archive". And 0 in others, and other
seemingly-random numbers of messages in still yet others. I can watch it
marching through the mailboxes, one at a time, if I "ps -ax" enough
times.

What I've Done to Counteract This...

Nothing I've done so far has been able to do anything about it. Re-untar the
archive, and it just does it all again. Try using dsync, and it just deletes the
messages from the un-tar'd archive. In other words, there are thousands of
messages in the archive which don't seem to meet Dovecot's requirement
for being in that particular folder, so it deletes them. The "cur"
directory in a mailbox directory may start out with >21000 messages in it,
but when it's done, it'll have deleted the same messages from
"cur" every time, leaving me with a fraction of what I started out
with. I even used the 65-migrate_mailboxes.pl script (supplied as part of OS X
10.9 Server), and the same thing happens. (The migrate script does a lot more
than just what I've noted above, but most of what it does is responsible for
settings and configuration migration. Very little of the script is responsible
for moving the data, and that part of the script ends up being a bunch of
"cp" followed by a "chown"--essentially what I'm doing.)

The Questions

What is going on? Failing a fix, am I trying to migrate the wrong way? Is there
a better way?

Random Thoughts and Observations

I've changed hostnames. The old one was shr-xs.mydomain.net, and the new one
is shr-mini.mydomain.net. Is it unhappy with that for some reason?

In one of the directories where messages are deleted, there were a zillion files
named like this:

1396718896.M324523P43630.shr-xs.mydomain.net,S=1209,W=1238:2,Sac

There were also a bunch of files named like this:

1211416234.cyrus.457,S=893:2,Sac

When it's all said and done, and Dovecot has done its thing, care to guess
which ones are left? Only these:

1211416234.cyrus.457,S=893:2,Sac
 
Just to see if I could influence which files get deleted, I deleted all of the
dovecot* files in the mailbox directory. And Dovecot recreated them faithfully,
but still deleted all but the ones which said "cyrus" in the
filenames.

I changed the hostname to match shr-xs.mydomain.net, and that doesn't seem
to have affected things a bit, though I don't have enough data to support
that I really succeeded in changing the hostname for all services (specifically
for Dovecot).

Is Dovecot really filtering out messages that just don't seem to
"belong" in the folder?

Thanks, y'all. I've puzzled on this one for about eight hours and it now
hurts my brain. I hope somebody else has an answer sans brain hurts.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mon, 28 Apr 2014, Bill Eccles wrote:
> correctly. But it's only a matter of time before Dovecot, or something,
> runs "doveadm index -u (usernames) (mailboxes)" on all of the
mailboxes.
> As it does, it deletes thousands of messages, leaving, for example, the 
> same 17 in INBOX and 3695 in "Deleted Items Archive". And 0 in
others,
after you untar the files and before the automagic kicks in, can you run 
doveadm index -u (usernames) (mailboxes)
via dtruss to make sure that command is deleting the messages?

Did you've enabled:

+ mail_debug=yes enables all kinds of mail related debug logging, such as 
showing where Dovecot is looking for mails.

+ http://wiki2.dovecot.org/Plugins/MailLog
Maybe your client does that?

- -- 
Steffen Kaiser
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQEVAwUBU15MSHz1H7kL/d9rAQIt2Af/QABYgpGglUhKT2iSuOk+JkBDgkuPZgd3
uYgUxR3psNZz6Kel8RuuyTp3B1G0UhJuku2d1E9Q2yBQQ4sOUgbI1+mUKNeSraiS
mWDBbqx0/2T8J/o0UMzdsVcNwav7ul//MuoefZlZD/hRI45VYK91WbJJcp8Fv9Jb
x9IJmRKA5x2MMbSyLLSy2VrM5Cq4hdLRj5atanWDj20uA0iw9xb5A4gaVwsByKC1
itwjvUkUn+cLjQ4gBQYOha9hyi6TfThHk/OWqajJqcgjBU+5p0BQUYfjOcZhVjfD
sDwik+dC+2WikotzhnIEpBSGrVlHE8R/avpCiJGPioRTe+MpJjSI9A==Hpxw
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mon, 28 Apr 2014, Bill Eccles wrote:
> correctly. But it's only a matter of time before Dovecot, or something,
> runs "doveadm index -u (usernames) (mailboxes)" on all of the
mailboxes.
> As it does, it deletes thousands of messages, leaving, for example, the 
> same 17 in INBOX and 3695 in "Deleted Items Archive". And 0 in
others,
after you untar the files and before the automagic kicks in, can you run 
doveadm index -u (usernames) (mailboxes)
via dtruss to make sure that command is deleting the messages?

Did you've enabled:

+ mail_debug=yes enables all kinds of mail related debug logging, such as 
showing where Dovecot is looking for mails.

+ http://wiki2.dovecot.org/Plugins/MailLog
Maybe your client does that?

- -- 
Steffen Kaiser
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQEVAwUBU15MSHz1H7kL/d9rAQIt2Af/QABYgpGglUhKT2iSuOk+JkBDgkuPZgd3
uYgUxR3psNZz6Kel8RuuyTp3B1G0UhJuku2d1E9Q2yBQQ4sOUgbI1+mUKNeSraiS
mWDBbqx0/2T8J/o0UMzdsVcNwav7ul//MuoefZlZD/hRI45VYK91WbJJcp8Fv9Jb
x9IJmRKA5x2MMbSyLLSy2VrM5Cq4hdLRj5atanWDj20uA0iw9xb5A4gaVwsByKC1
itwjvUkUn+cLjQ4gBQYOha9hyi6TfThHk/OWqajJqcgjBU+5p0BQUYfjOcZhVjfD
sDwik+dC+2WikotzhnIEpBSGrVlHE8R/avpCiJGPioRTe+MpJjSI9A==Hpxw
-----END PGP SIGNATURE-----

On Apr 28, 2014, at 8:40 AM, Steffen Kaiser <skdovecot at
smail.inf.fh-brs.de> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Mon, 28 Apr 2014, Bill Eccles wrote:
> 
>> correctly. But it's only a matter of time before Dovecot, or
something, runs "doveadm index -u (usernames) (mailboxes)" on all of
the mailboxes. As it does, it deletes thousands of messages, leaving, for
example, the same 17 in INBOX and 3695 in "Deleted Items Archive". And
0 in others,
> 
> after you untar the files and before the automagic kicks in, can you run
doveadm index -u (usernames) (mailboxes)
> via dtruss to make sure that command is deleting the messages?
> 
> Did you've enabled:
> 
> + mail_debug=yes enables all kinds of mail related debug logging, such as
showing where Dovecot is looking for mails.
> 
> + http://wiki2.dovecot.org/Plugins/MailLog
> Maybe your client does that?
> 
> - -- Steffen Kaiser
Steffen--

Dtruss showed nothing unusual, and I'll bet you expected that. But this
morning, as I was untarring the tarball, I noticed these two processes show up
in the ps -ax list:

28445 ??         0:14.98 find . -name *.shr-xs.mydomain.net* -print0
28446 ??         0:00.00 xargs -0 rm

Since the mail services were off, hence Dovecot has no processes listed in ps
-ax, I'm sure that I'm blaming Dovecot for something it ain't doing.

(dig dig dig... um...)

Ah... well, don't I feel foolish. It's my own sa-learn script at fault!

My script attempts to clean out the spam/ham folders like this:

cd /var/spool/imap/dovecot/mail/public/.Learn\ as\ Spam\ \(Bad\ E-mail\)/cur/
find . -name '*.shr-xs.mydomain.net*' -print0 | xargs -0 rm

and given that these two directories don't exist, it ends up running these
commands from /. which cleans the entire hard drive of all items matching the
pattern above.

Is there a better way to clean these directories out using native Dovecot
commands (so I don't do this again!)?

Thanks for your help!
Bill