dovecot - Jan 2006 - 1.0 beta1 released

As promised, here it is finally.

v1.0 is getting nearer and I want people to start looking at Dovecot's
sources. Getting some more help with development would be nice, but I'd
also like to hear if there are any hidden bugs in the code. So, one week
from now I'll start offering 1000 EUR for the first person to point out
a remotely exploitable security hole in Dovecot. See
http://dovecot.org/security.html for more information.

Changes since 1.0alpha5:

UPGRADE WARNING: Unlike earlier Dovecot versions, 1.0 beta1 now requires
Diffie-Hellman parameters to have been generated before SSL/TLS is used.
These are generated when Dovecot is started for the first time and it
may take several minutes. During this time SSL/TLS connections aren't
working.

(I decided to leave it this way at least for now. I don't think any new
Dovecot admins will try to log in with SSL within the first 5 minutes of
the installation?)

Features:

 - DSA SSL connections provide now forward secrecy (which causes the DH
parameter issue mentioned above)
 - Support for password protected SSL private keys
 - LDAP: Added authentication bind support. Patch by J.M. Maurer
 - LDAP: Added fast authentication binding. Patch by Geff
<boing@boing.com>
 - PAM: Changed -session parameter to session=yes
 - kqueue support. Patch by Vaclav Haisman
 - dbox updates. Kind of works now, but not recommended for more than
testing yet. There still are known bugs.
 - maildir_copy_with_hardlinks=yes works again
 - Sometimes Dovecot kept telling about keyword changes over and over
again even if there weren't any.
 - Beginnings of fallbacking to in-memory indexes when write fails with
"out of disk space" error. Not perfect yet, but helps in some cases.
 - When IDLEing, send a small notification every 2 minutes so that
NATs/firewalls don't close the connection.
 - Added gdbhelper binary to help debugging

Performance improvements:

 - Don't fdatasync() cache file. Not that important and gives better
performance.
 - mmap_disable=yes mode doesn't keep rewriting index file now
constantly, so it uses much less disk I/O.
 - Some other minor index file improvements

Bugfixes:

 - UID STORE command wasn't returning UIDs in replies
 - Fixes for various problems with IDLE command.
 - THREAD command crashed if no search matches were found
 - Some fixes to handling LIST command with namespaces. Helps Pine.
 - Fixed a situation in mbox when it could get the process to infinite
loop
 - Fixed one mbox assert crash
 - Use long line wrapping for X-IMAPbase, X-IMAP and X-Keywords headers
for UW-IMAP compatibility
 - dovecot --exec-mail wasn't cleaning existing environment variables
properly

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url :
http://dovecot.org/pipermail/dovecot/attachments/20060116/effd4750/attachment.pgp

That's silly.  The first thing I try with any new install is test it
out.  The idea of generating the DH key during install is useful.


On Mon, 16 Jan 2006 22:52:30 +0200
Timo Sirainen <tss@iki.fi> wrote:

: (I decided to leave it this way at least for now. I don't think any
: new Dovecot admins will try to log in with SSL within the first 5
: minutes of the installation?)

-- 
All the taxes paid over a lifetime by the average American are spent by
the government in less than a second.
		-- Jim Fiebig

Timo Sirainen wrote:
> As promised, here it is finally.
Not to nudge about it, but I wonder if you looked at my mail of 
yesterday regarding PAM_RHOST...  I now downloaded the beta and still 
had to apply that patch to get the client host to be passed to PAM as 
PAM_RHOST on FreeBSD 5.4.

Great news beside that.  I think one of the bugs I previously noticed 
with IDLE (which I found difficult to point at since it manifested 
itself with Thunderbird which opens several connections to the server) 
is gone.

   Cheers,
   -- Tom

-- 
   Tom Alsberg - hacker (being the best description fitting this space)
   Web page:	http://www.cs.huji.ac.il/~alsbergt/
DISCLAIMER:  The above message does not even necessarily represent what
my fingers have typed on the keyboard, save anything further.

Timo Sirainen <tss@iki.fi> writes:
> As promised, here it is finally.
>
> v1.0 is getting nearer and I want people to start looking at Dovecot's
> sources. Getting some more help with development would be nice, but I'd
Thanks for your work!

I looked at the .spec "source" with rpmbuild on SUSE Linux 10.0 x86,
which came up with these autocheck errors - it appears as though the
.spec file could use some updating WRT newer plugins.

RPM build errors:
    File listed twice: /usr/libexec/dovecot
    Installed (but unpackaged) file(s) found:
   /usr/lib/dovecot/imap/lib01_zlib_plugin.a
   /usr/lib/dovecot/imap/lib01_zlib_plugin.la
   /usr/lib/dovecot/imap/lib01_zlib_plugin.so
   /usr/lib/dovecot/imap/lib02_imap_quota_plugin.a
   /usr/lib/dovecot/imap/lib02_imap_quota_plugin.la
   /usr/lib/dovecot/imap/lib02_imap_quota_plugin.so
   /usr/lib/dovecot/lib01_quota_plugin.a
   /usr/lib/dovecot/lib01_quota_plugin.la
   /usr/lib/dovecot/lib01_quota_plugin.so
   /usr/lib/dovecot/lib02_trash_plugin.a
   /usr/lib/dovecot/lib02_trash_plugin.la
   /usr/lib/dovecot/lib02_trash_plugin.so

The attached patch fixes the immediate packaging errors, but does not
yet add a SUSE-compatible init script (SUSE don't have /sbin/service or
daemon or /etc/init.d/functions) - I don't have time for that now.

Note that SUSE also move /usr/libexec/* to /usr/lib for LSB compliance.

-- 
Matthias Andree
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dovecot.spec.in-1.0.beta1.patch
Type: text/x-patch
Size: 827 bytes
Desc: update dovecot.spec[.in] for 1.0.beta1
Url :
http://dovecot.org/pipermail/dovecot/attachments/20060117/e67a2108/dovecot.spec.in-1.0.beta1.bin

Le 16.01.2006 21:52, Timo Sirainen a ?crit :

I have this make install failure :

Making install in quota
make[3]: Entering directory
`/home/nico/sources/dovecot/dovecot-1.0.beta1/src/plugins/quota'
make[4]: Entering directory
`/home/nico/sources/dovecot/dovecot-1.0.beta1/src/plugins/quota'
mkdir -p -- /tmp/test/lib/dovecot/imap /tmp/test/lib/dovecot/lda
mkdir: `/tmp/test/lib/dovecot/imap' exists but is not a directory

Regards,
-- 
Nico
Retournons ? nos moutons.
	-+- Fran?ois Rabelais (1494?-1553), Tiers livre (chap. 34) -+-

Timo Sirainen wrote:
> v1.0 is getting nearer and I want people to start looking at Dovecot's
Duh! Just when I finally got the time to add SQLite support. It works, but
I have to clean some things up.
Do you think it could still go into 1.0? There are only a few minor
changes to the rest of the code, so it's very unlikely that something got
broken.
> sources. Getting some more help with development would be nice, but I'd
I'll try to fix the epoll+ssl problems, which should be resolved before
1.0, I think. I already took a short look at it, but I have a hard time
understanding what the code is doing and there are no comments.

Hi, while I have been debugging some kevent() related problems I have 
stumbled upon this in imap process:

(gdb) bt
#0  0x00000000 in ?? ()
#1  0x0808befd in notify_callback (context=0x80fa040) at 
index-mailbox-check.c:65
#2  0x080c1ede in event_callback (context=0x80f3160) at 
ioloop-notify-kqueue.c:46
#3  0x080c2aff in io_loop_handler_run (ioloop=0x80ed200) at 
ioloop-kqueue.c:184
#4  0x080c1c02 in io_loop_run (ioloop=0x80ed200) at ioloop.c:235
#5  0x080649ab in main (argc=1, argv=0xbfbfea58, envp=0xbfbfea60) at 
main.c:238
(gdb) frame 1
#1  0x0808befd in notify_callback (context=0x80fa040) at 
index-mailbox-check.c:65
65                      ibox->notify_callback(&ibox->box, 
ibox->notify_context);
(gdb) l
60              ibox->notify_last_check = ioloop_time;
61              if ((unsigned int)(ioloop_time - ibox->notify_last_sent)
>62                  ibox->min_notify_interval) {
63                      ibox->notify_last_sent = ioloop_time;
64                      ibox->notify_pending = FALSE;
65                      ibox->notify_callback(&ibox->box, 
ibox->notify_context);
66              } else {
67                      ibox->notify_pending = TRUE;
68              }
69      }
(gdb) inspect ibox
$10 = (struct index_mailbox *) 0x80fa040
(gdb) inspect *ibox
$11 = {box = {name = 0x80fa160 "INBOX", storage = 0x80eba40, v = {
      is_readonly = 0x808e4b8 <index_storage_is_readonly>,
      allow_new_keywords = 0x808e4e8 <index_storage_allow_new_keywords>,
      close = 0x8066e38 <maildir_storage_close>,
      get_status = 0x808d974 <index_storage_get_status>,
      sync_init = 0x8069148 <maildir_storage_sync_init>,
      sync_next = 0x808eb74 <index_mailbox_sync_next>,
      sync_deinit = 0x808ed3c <index_mailbox_sync_deinit>,
      notify_changes = 0x8066e88 <maildir_notify_changes>,
      transaction_begin = 0x80692d8 <maildir_transaction_begin>,
      transaction_commit = 0x8069310 <maildir_transaction_commit>,
      transaction_rollback = 0x80693ec <maildir_transaction_rollback>,
      keywords_create = 0x808e634 <index_keywords_create>,
      keywords_free = 0x808e658 <index_keywords_free>,
      get_uids = 0x8087fc0 <index_storage_get_uids>, mail_alloc = 
0x8089a6c <index_mail_alloc>,
      header_lookup_init = 0x808bae8 <index_header_lookup_init>,
      header_lookup_deinit = 0x808bd48 <index_header_lookup_deinit>,
      search_get_sorting = 0x808d524 <index_storage_search_get_sorting>,
      search_init = 0x808d538 <index_storage_search_init>,
      search_deinit = 0x808d604 <index_storage_search_deinit>,
      search_next = 0x808d7dc <index_storage_search_next>,
      save_init = 0x806e04c <maildir_save_init>,
      save_continue = 0x806e370 <maildir_save_continue>,
      save_finish = 0x806e40c <maildir_save_finish>,
      save_cancel = 0x806e6a4 <maildir_save_cancel>, copy = 0x806ba0c 
<maildir_copy>,
      is_inconsistent = 0x808e518 <index_storage_is_inconsistent>}, pool 
= 0x80fa010,
    module_contexts = {buffer = 0x80fa168, element_size = 4}}, storage = 
0x80eba40,
  index = 0x80f4400, view = 0x80f4a00, cache = 0x80f4600, mail_vfuncs = 
0x80ded60,
  is_recent = 0x8066054 <maildir_is_recent>, md5hdr_ext_idx = 2, 
notify_to = 0x0,
  notify_files = 0x0, notify_ios = 0x0, notify_last_check = 1137544337,
!!!!!!!!
  notify_last_sent = 1137544337, min_notify_interval = 0, 
notify_callback = 0,
!!!!!!!!
  notify_context = 0x0, next_lock_notify = 1137544335,
  last_notify_type = MAILBOX_LOCK_NOTIFY_NONE, commit_log_file_seq = 0,
  commit_log_file_offset = 0, keyword_names = 0x80f446c, cache_fields = 
0x80f4900,
  recent_flags = 0x80e13a0, recent_flags_start_seq = 130, 
recent_flags_count = 1,
  synced_recent_count = 1, sync_last_check = 1137544332, readonly = 0, 
keep_recent = 0,
  recent_flags_synced = 1, sent_diskspace_warning = 0, 
sent_readonly_flags_warning = 0,
  notify_pending = 0, mail_read_mmaped = 0}

The notify_callback field is NULL and the process SIGSEGVs. I know very 
little about IMAP protocol and I think this could be just a case of bad 
handling of invalid input. The IMAP session I had follows:

[...]
a1 OK Logged in.
a2 SELECT INBOX
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk 
\*)] Flags permitted.
* 131 EXISTS
* 1 RECENT
* OK [UNSEEN 99] First unseen.
* OK [UIDVALIDITY 1137196697] UIDs valid
* OK [UIDNEXT 134] Predicted next UID
a2 OK [READ-WRITE] Select completed.
a3 IDLE
+ idling
* 132 EXISTS
* 2 RECENT
a4 DONE
a3 BAD Expected DONE.


Vaclav Haisman

Packages for FC4:

i386:
http://fedora.ivazquez.net/yum/4/i386/RPMS.alternatives/dovecot-1.0-0.iva.2.beta1.i386.rpm
http://fedora.ivazquez.net/yum/4/i386/SRPMS.alternatives/dovecot-1.0-0.iva.2.beta1.src.rpm

ppc:
http://fedora.ivazquez.net/yum/4/ppc/RPMS.alternatives/dovecot-1.0-0.iva.2.beta1.ppc.rpm
http://fedora.ivazquez.net/yum/4/ppc/SRPMS.alternatives/dovecot-1.0-0.iva.2.beta1.src.rpm

x86_64:
http://fedora.ivazquez.net/yum/4/x86_64/RPMS.alternatives/dovecot-1.0-0.iva.2.beta1.x86_64.rpm
http://fedora.ivazquez.net/yum/4/x86_64/SRPMS.alternatives/dovecot-1.0-0.iva.2.beta1.src.rpm

-- 
Ignacio Vazquez-Abrams <ivazquez@ivazquez.net>
http://fedora.ivazquez.net/

gpg --keyserver hkp://subkeys.pgp.net --recv-key 38028b72
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url :
http://dovecot.org/pipermail/dovecot/attachments/20060119/eecef4e7/attachment-0001.pgp