dovecot - May 2004 - Small change to make dovecot pop3 uw-imap migration friendly

Hi,

    today I've finished migration from uw-imap daemons to shiny and fast
dovecot. With thanks of dovecot my mail server load average drops by factor
of ten even with ancient unix mailboxes. ;-)

    So, the only thing I've discovered is what POP3 uidls are different than
ones used in the uw-imap. Luckily, the difference are only in the format
string used in uidl response. So, I've made the following change in the code

--- dovecot-0.99.10.4.orig/src/pop3/commands.c 2003-05-28 15:17:15.000000000
+0400
+++ dovecot-0.99.10.4/src/pop3/commands.c 2004-05-27 14:06:48.000000000
+0400
@@ -374,7 +374,7 @@

  while ((mail = client->mailbox->fetch_next(ctx)) != NULL) {
   client_send_line(client, message == 0 ?
-     "%u %u.%u" : "+OK %u %u.%u",
+     "%u %08x%08x" : "+OK %u %08x%08x",
      mail->seq, client->uidvalidity, mail->uid);
   found = TRUE;
  }

    and got everything just like in the old uw-imap pop3. I'm current
running RH9, but thinking of FC2 which uses dovecot. So, I've rebuild the
FC2 rpm for RH9 including this patch. Pop3 users, which do not remove their
mails from the server are really happy - they do not receive old mails
twice.

    Probably this small change could be applied to the 1.0, or used as a
config option.

    Any suggestions are greatly appriciated.

Thanks,
    Oleg.

On 27.5.2004, at 16:29, Oleg I. Vdovikin wrote:
> Hi,
>
>     today I've finished migration from uw-imap daemons to shiny and 
> fast
> dovecot. With thanks of dovecot my mail server load average drops by 
> factor
> of ten even with ancient unix mailboxes. ;-)
Get 0.99.10.5, it fixes possible mbox corruption.
>     So, the only thing I've discovered is what POP3 uidls are 
> different than
> ones used in the uw-imap. Luckily, the difference are only in the 
> format
> string used in uidl response. So, I've made the following change in 
> the code
Actually I just wrote a similiar patch a few days ago .. :) 
http://dovecot.org/patches/pop3-uidl-uwimap.patch it's against .10.5 
which changed a bit.
>     Probably this small change could be applied to the 1.0, or used as 
> a
> config option.
I was thinking about making this fully configurable, ie. a config 
option which would accept printf-like string. "%v.%u"
"%08xv%08xu" or
something. Or maybe it's not worth it, I'd have to look at other POP3 
servers to see what different kinds of UIDs they use..
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20040527/375b3abb/attachment-0001.bin>

> On 27.5.2004, at 16:29, Oleg I. Vdovikin wrote:
>
>> Hi,
>>
>>     today I've finished migration from uw-imap daemons to shiny and
>> fast
>> dovecot. With thanks of dovecot my mail server load average drops by
>> factor
>> of ten even with ancient unix mailboxes. ;-)
>
> Get 0.99.10.5, it fixes possible mbox corruption.
Thanks, will do.
>
>>     So, the only thing I've discovered is what POP3 uidls are
>> different than
>> ones used in the uw-imap. Luckily, the difference are only in the
>> format
>> string used in uidl response. So, I've made the following change in
>> the code
>
> Actually I just wrote a similiar patch a few days ago .. :)
> http://dovecot.org/patches/pop3-uidl-uwimap.patch it's against .10.5
> which changed a bit.
Perfect. ;-)
>
>>     Probably this small change could be applied to the 1.0, or used as
>> a
>> config option.
>
> I was thinking about making this fully configurable, ie. a config
> option which would accept printf-like string. "%v.%u"
"%08xv%08xu" or
> something. Or maybe it's not worth it, I'd have to look at other
POP3
> servers to see what different kinds of UIDs they use..
Yes, this sounds reasonable. Probably you will need to add more format
specifiers to catch everything. ;-)

Thanks,
Oleg.

On Thu, May 27, 2004 at 05:57:40PM +0300, Timo Sirainen
wrote:
> >    Probably this small change could be applied to the 1.0, or used as 
> >a
> >config option.
> 
> I was thinking about making this fully configurable, ie. a config 
> option which would accept printf-like string. "%v.%u"
"%08xv%08xu" or
> something. Or maybe it's not worth it, I'd have to look at other
POP3
> servers to see what different kinds of UIDs they use..
Suggestion: if you have any way to maintain per-message state information,
keep the UID in there. You can prime it using an algorithm of your choice.
Ideally you'd allow its value to be taken from a header (e.g. X-POP3-UIDL)
if present.

The reason for this: when migrating a mailbox from any other POP3 server,
you can take whatever UIDL the old POP3 server gives and attach it to the
message in Dovecot.

courier-imap *almost* allows me to do this. It keeps a state file
(courierpop3dsizelist), containing one line per message - it has the exact
message size as per RFC1939, and a sequence number. The UIDL entry is then
sequenced from UID${uidvalidity}.${sequence}

What I'd prefer is that instead of a sequence number, the exact UIDL string
is stored there. Then I could migrate from *any* POP3 server and preserve
UIDLs.

(I'm keeping an eye on Dovecot... courier-imap has served me extremely well
in production environments and under heavy load, but from what I've seen on
this list, Dovecot is very good at producing diagnostic error messages when
things go wrong, which courier is dreadful at... I can't deploy Dovecot
until it supports Maildir++ quotas though)

Regards,

Brian.

Timo Sirainen <tss at iki.fi> writes:
> On 27.5.2004, at 16:29, Oleg I. Vdovikin wrote:
>
>>     So, the only thing I've discovered is what POP3 uidls are
>> different than ones used in the uw-imap. Luckily, the difference are
>> only in the format string used in uidl response. So, I've made the
>> following change in the code
>
> Actually I just wrote a similiar patch a few days ago .. :)
> http://dovecot.org/patches/pop3-uidl-uwimap.patch it's against .10.5
> which changed a bit.
[...]
> I was thinking about making this fully configurable, ie. a config option
> which would accept printf-like string. "%v.%u"
"%08xv%08xu" or
> something. Or maybe it's not worth it, I'd have to look at other
POP3
> servers to see what different kinds of UIDs they use..
Question: will this apply to all UIDs or just to those for new mail? I
can't tell from the code fragment.

There are two requirements:

1. major: a UID that any client may have seen _MUST NOT_ change.

This means that a formatting string MUST ONLY have an impact on newly
arriving mail.

2. minor: those who migrate from UWImap would like to use UWImap UIDs
   for existing mail.

This could be achieved by a little program that runs once per mailbox
after the initial switch.

And note that the client must not care about the UIDL format as long as
it's legal, so anything beyond "importing" the UWIMAP UIDL is
bogus.

-- 
Matthias Andree

Encrypted mail welcome: my GnuPG key ID is 0x052E7D95