dovecot - Jul 2008 - POP3 locking

I'm trying to understand what dovecot's behavior will be in a given 
scenario to see if it will work as a (hopefully better) replacement for 
the home-grown pop3 server we currently use.

User A is on a slow dialup connection, and uses, oh, let's say Outlook, 
since that's where most of our problems seem to spring from.  User A 
uses outlook to download his mail over pop3.  Half way through 
downloading a particularly large message, Outlook decides that the whole 
process isn't happening fast enough, so it opens another connection to 
the server and tries to download (potentially) the same message again.

Now, our current server establishes a lock on the mailbox, and when that 
second connection happens it simply responds that there's no messages to 
collect... we used to have it terminate the connection, but this brings 
up confusing error messages on User A's client, and they call up and 
complain -- not an ideal situation.

I've got a dovecot installation on my testing server, and I think I've 
set everything up correctly; it authenticates from my LDAP database, 
finds the home directory, lists the mail, all that... I believe 
somewhere it there it uses fcntl to establish an advisory lock on the 
mailbox (oh, we're using Maildir, by the way).

I'm not sure where, though, or how long the lock persists for, or what 
its actual purpose is, because my tests show me that I can log the same 
users in with two concurrent sessions and get the same message twice. 
It's probably a fault in my understanding of how locking in user 
sessions of a pop3 server should work, but that doesn't sound right to me.

Can someone explain if what I'm seeing is correct, or what problem it 
may be symptomatic of?  Does dovecot even do concurrent-user-session 
locking?  So far I've enjoyed playing with dovecot -- far more than the 
current server we use.  I'd like to use it as our primary pop3 server, 
but preventing users from having concurrent sessions and getting the 
same message more than once, and keeping them from seeing errors because 
of their client trying to open multiple sessions is important.

Oh, and I'm user dovecot 1.0.beta3, since I don't really have the option
of upgrading the OS on my mail servers any time soon.

Adrian

Adrian Overbury wrote:
> I'm trying to understand what dovecot's behavior will be in a given
> scenario to see if it will work as a (hopefully better) replacement 
> for the home-grown pop3 server we currently use.
>
> User A is on a slow dialup connection, and uses, oh, let's say 
> Outlook, since that's where most of our problems seem to spring from.  
> User A uses outlook to download his mail over pop3.  Half way through 
> downloading a particularly large message, Outlook decides that the 
> whole process isn't happening fast enough, so it opens another 
> connection to the server and tries to download (potentially) the same 
> message again.
I can't say I have ever seen this happen with Outlook to be honest?  
Usually when this happens it means the user has added their one account 
in Outlook more than once?  Which version do you see this happen in?

Dovecot can be made to do exclusive locks, or the default is to share 
the mailbox.  I don't recall the config option, but check a recent conf file
> Oh, and I'm user dovecot 1.0.beta3, since I don't really have the 
> option of upgrading the OS on my mail servers any time soon.
Now that's just silly talk.  Upgrade to a more recent version anyway

If you post your OS type then I'm sure someone will help with packages - 
otherwise compile from source

Ed W

On Wed, 30 Jul 2008, Adrian Overbury wrote:
> Now, our current server establishes a lock on the mailbox, and when that  
> second connection happens it simply responds that there's no messages
to
> collect... we used to have it terminate the connection, but this brings  
> up confusing error messages on User A's client, and they call up and  
> complain -- not an ideal situation.
Why do you do that? Is "-ERR unable to lock maildrop" in response to
PASS or APOP not an option? One computer (laptop) accessing the same
time as a PC accesses the box will purge its copy of the UIDL and
refetch ALL messages next time...
> I'm not sure where, though, or how long the lock persists for, or what
> its actual purpose is, because my tests show me that I can log the same  
> users in with two concurrent sessions and get the same message twice.  
> It's probably a fault in my understanding of how locking in user  
> sessions of a pop3 server should work, but that doesn't sound right to 
> me.
Well... according to RFC-1939, the session should be locked when
entering TRANSACTION state. Purpose is to avoid two processes stomping
over each other in the later UPDATE state.
> Can someone explain if what I'm seeing is correct, or what problem it  
> may be symptomatic of?  Does dovecot even do concurrent-user-session  
> locking?  So far I've enjoyed playing with dovecot -- far more than the
> current server we use.  I'd like to use it as our primary pop3 server,
> but preventing users from having concurrent sessions and getting the  
> same message more than once, and keeping them from seeing errors because  
> of their client trying to open multiple sessions is important.
Same here - I just did a trivial test (Dovecot 1.1 from Mercurial repo,
two POP3+SSL sessions at the same time, off of Maildir): log in both,
delete a message in one and QUIT (so it passes through UPDATE state),
LIST in the remaining session still lists the deleted message, but
retrieving it results in an error. Probably confusing for clients ;-)

HOWEVER, if the same message is downloaded twice, that's a client bug.
It can use UIDL and avoid scheduling the same message twice for
fetching.

Outlook is a piece of sh.. WRT mail capabilities, with Outlook 2003
still not fixing MIME + quoting bugs that Microsoft is aware of since
the mid-1990s.

I have some hope that I'll be able to migrate to Kontact (KMail +
organizer + stuff) next year. Certainly not my favourite but given
there's no recent Windows version of Evolution...
> Oh, and I'm user dovecot 1.0.beta3, since I don't really have the
option
> of upgrading the OS on my mail servers any time soon.
I'd stay away from such a version, but use a recent 1.0.X or 1.1.X - if
someone is looking after your server regularly (once or twice a week or
so), compiling it yourself shouldn't be an issue.

HTH

-- 
Matthias Andree

On Wed, Jul 30, 2008 at 06:05:27PM +0200, Kai Schaetzl
wrote:
> 
> but it seems to be off by default. From my dovecot.conf (1.07):
> 
> #pop3_lock_session = no
> 
> I think it should be on by default.

The problem is if you have an MUA/MDA retrieving lots of messages
over a potentially slow connection, the mailbox can remain locked
for a very long time for a read-only operation.

The problem that requires doing it is basically a client problem.
Namely, some clients may get confused if in the middle of a series
of RETR commands, they get a mailbox locked reply. Thus, the option
exists to accomodate potentially broken clients and has potentially
negative effects so I think the default is appropriate.

-- 
Mark Sapiro mark at msapiro net       The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan