Scott Goodwin
2010-Nov-03 19:53 UTC
[Dovecot] emails getting mangled when dragging from Exchange account to IMAP shared folders
I'm having the most frustrating issue, and I'm at a loss for what is happening. I'm not even sure it is with Dovecot, but that's why I'm posting this here... if it isn't a dovecot issue, maybe someone can get me headed in the right direction? I've posted relevant dovecot and config info at the bottom of this post. Here's the scenario: We recently migrated our email from Postfix+Dovecot+Amavis+SA to Exchange (not my choice, unfortunately). Now, users have to use Outlook (as opposed to before, when they could choose between Outlook and Thunderbird), and have the new Exchange account set up, as well as their old IMAP account, which they still use for shared project folders. So, emails come in to Exchange, then they drag the emails from the Exchange mailbox to the shared branch under their IMAP account, and into the appropriate job folder. Emails to one specific user in my organization (let's call him Roy at Ocean.org) is having his emails mangled when dragging certain items from his Exchange mailbox to the IMAP account. I can't reproduce the issue with any other user. Also, this only seems to happens with emails from one other specific domain (let's call them tornado.org). So, in summary, user1 at Tornado.orgsend an email to Roy at Ocean.organd Sue at Ocean.org. Both drag the email from their Exchange box into any folder (shared or otherwise) in the IMAP tree. Sue's copy makes it in just fine, while Roy's copy gets mangled. Here's the headers of the resultant email after Roy has dragged it into the shared folder: =======================================From: "User1" <User1 at tornado.org> To: "Roy", "Sue" References: <B3457C1920D7A2438CCC19EF2A527293372B11 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A527293372D9A at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272933A2C3D at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272933A310E at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272933A3683 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272933A3B97 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272933D103B at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272933D192F at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272933D1B12 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272933FD341 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272933FD8F8 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272933FD9A0 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272933FDFDD at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272934C8927 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272934C8B46 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272934C8D76 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272934C8D7A at abc027.abc.local> <B3457C1920 ======================================= And here's the email body: ======================================= D7A2438CCC19EF2A5272934C8D95 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272934C8DB6 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272934C8F0D at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272935008B5 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A527293500C27 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272935011BC at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A527293525B1D at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A527293525D81 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A527293526139 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A527293526416 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272935524A6 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272935524C9 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A527293589E1C at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272935BF6A9 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272935E7414 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272935E75E2 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272935E75E8 at abc027.abc.local> <B3457C1920 D7A2438CCC19EF2A5272935E75F4 at abc027.abc.local> <B3457C1920D7A2438CCC19EF2A5272935E800E at abc027.abc.local> Subject: RE: Some Subject Date: Tue, 26 Oct 2010 15:11:01 -0700 Message-ID: <B3457C1920D7A2438CCC19EF2A52729364D0A0 at abc027.abc.local> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0118_01CB7755.52354760" X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcrhmOPG84jptRwERxazrnj4Ne2YAQAACMtAAAAjcPAAAAu+gAAAE1hAAAAZWCAAABMaEAAADGzwACO3fdAACN0HsAEpVUrgAPt2jvAAAcrbcABrFxCQAAAXwjAAABGv8AAAI35QAYbDMHAAAPjDYAADNREgAA5XR4AAHoOFcAFtPWTAAIubCCAALwf/MAADBptQACzN6nAAAC61kAAAEN+AAACfkjAAO7MG4AAHLrMQALmKX6AAAUlCcAAAq9SgAAuoo+AAABqqMABXq+FwAD5jAAABYqZYoAOISoLgAAAd//AAAKVPYAApG8DwAAt6dsAAKxpd8AEy9g6gAADrZVAAAE4WgAAlXdkQAPY4tNAAAWY+kAFsS/RgAFtG7qAAAGJT8AAAPLUwAAA2xsAAACamEAAC4ufAADaZyfAAKizZ0AAAZ9EAAAWRHAAAAOgkgAAAUUOgAAQWBBAAij+IkAAE2k1wAAPgAgAANGrwsAACt+wgAC0EW9AAAEDpoAA12EJgAAAyAtAAAyATYAAgyfBQAKS/7/AAAEPWEAABNsTAAAAZujAALLkhkAAwwG5AAAMVt8AAAE3iwAEhzNqgAA8qbEAAIqDaMAA0UTmgAEE0hxAAlsz9IAABAdrAAFIwRcAAZxd5oAAOziPQAJEu45AAOaFlIAAeeCygADwtnbAD9zXHwAApeNOAACk/pfAAABxl0AAAWgGAAAB7d4AACGbyQACNOVmwABF68KAAKuKysAAsiR5gADDcY1AAOMAK8ACO561QABCk+7AAI5IqsAAJGVEgAFjaIkAANWSCQAACuaAwAAAKUQAABugScAAAbmTQAADspwAAzNO+MAAiFH8wAAEA4mAAMv6E0AEtz2wQADWsQHAAO4pvkACIRgpAAAApFcAAABO4UABmP4/QAAEJ/lABNOByMAAF28awAACwfDA Content-Language: en-us X-OlkEid: C664F3259FA9393D4124CD409B5CAE330696EE82 content-class: urn:content-classes:message This is a multi-part message in MIME format. ------=_NextPart_000_0118_01CB7755.52354760 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit <SNIP> ========================================================================== Obviously, it's breaking the message in the References header, and I've tracked down the following byte sequence at the breaking point: 0D:09:0A:09 ...which is CR,TAB,CR,LF. What puzzles me is why Roy's gets broken here, when the others' do not; they're the exact same email, so you'd think they would all get broken. I can't figure out A) where that byte sequence is coming from (our Exchange server, Tornado.net's exchange server, or one of the two of our external spam filtering service... they use BigFish.com, and we use MxLogic), or B) if that's even the issue. I've rebuilt Roy's entire machine, reconfigured Outlook from scratch, still happening. When he logs onto another machine and sets up his Exchange account on another outlook, it still happens. Our Exchange administrator (I don't have access to Exchange) says there is nothing wrong with his Exchange account, for what it's worth. I don't expect anyone to have the solution here, since there are many variables, and there is other info that would probably be needed. What I DO want is to be pointed in the right direction on how I might track this down. For example, what is the path that this mangled email would take in this case? Even with full logging in Dovecot and postfix, I can't see any info on the path that this email follows in our system, and where it might be getting mangled. Does it actually go through the same path as an email that was actually sent to the IMAP account, or is it different when it is dragged in from another account in the same mail client? Is this more likely a problem with Postfix, or Sieve, or....? (I've disabled Amavis and SA to get them out of the picture, and same problem). Here's the dovecot config info; please let me know if you want any other config info, and thanks ahead of time! # dovecot --version 1.1.19 # dovecot -n # 1.1.19: /etc/dovecot/dovecot.conf # OS: Linux 2.6.28-hardened-r9 x86_64 Gentoo Base System release 1.12.13 protocols: imaps imap managesieve listen(default): 127.0.0.1:143 listen(imap): 127.0.0.1:143 listen(managesieve): 127.0.0.1:2000 ssl_listen(default): *:993 ssl_listen(imap): *:993 ssl_listen(managesieve): ssl_cert_file: /etc/ssl/dovecot/imapd.crt ssl_key_file: /etc/ssl/dovecot/imapd.key login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(managesieve): /usr/libexec/dovecot/managesieve-login login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(managesieve): no valid_chroot_dirs: /var/mail first_valid_uid: 206 last_valid_uid: 206 first_valid_gid: 206 last_valid_gid: 206 maildir_copy_preserve_filename: yes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(managesieve): /usr/libexec/dovecot/managesieve mail_plugins(default): acl mail_plugins(imap): acl mail_plugins(managesieve): mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(managesieve): /usr/lib64/dovecot/managesieve imap_client_workarounds(default): outlook-idle delay-newmail imap_client_workarounds(imap): outlook-idle delay-newmail imap_client_workarounds(managesieve): sieve_storage(default): sieve_storage(imap): sieve_storage(managesieve): ~ sieve(default): sieve(imap): sieve(managesieve): ~/.dovecot.sieve namespace: type: private separator: . location: maildir:~/Maildir inbox: yes list: yes subscriptions: yes namespace: type: public separator: . prefix: shared. location: maildir:/var/mail/shared list: yes lda: postmaster_address: postmaster at jensenmaritime.com hostname: mail.jensenmaritime.com mail_plugins: cmusieve mail_plugin_dir: /usr/lib/dovecot/lda sieve_global_path: /var/mail/.dovecot.sieve sieve: ~/.dovecot.sieve sendmail_path: /usr/lib/sendmail auth_socket_path: /var/run/dovecot/deliver-auth auth lda: default_realm: jensenmaritime.com user: postmaster passdb: driver: ldap args: /etc/dovecot/lda-ldap.conf userdb: driver: static args: allow_all_users=yes user=%Lu uid=206 gid=206 home=/var/mail/%Lu socket: type: listen master: path: /var/run/dovecot/deliver-auth mode: 384 user: vmail group: vmail auth default: mechanisms: PLAIN LOGIN default_realm: jensenmaritime.com user: postmaster passdb: driver: ldap args: /etc/dovecot/default-ldap.conf userdb: driver: static args: user=%Lu uid=206 gid=206 home=/var/mail/%Lu socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix plugin: acl: vfile # grep -v '^ *\(#.*\)\?$' /etc/dovecot/dovecot-ldap.conf uris = ldap://127.0.0.1:389/ ldap_version = 3 base = ou=users,ou=accounts,dc=jensenmaritime,dc=com scope = onelevel auth_bind = yes pass_filter (&(objectClass=CourierMailAccount)(!(disableimap=*))(mail=%Lu)) pass_attrs = auth_username_format=%Lu # uname -a Linux milne 2.6.28-hardened-r9 #4 SMP Mon Jun 7 10:50:28 PDT 2010 x86_64 Intel(R) Xeon(R) CPU E5405 @ 2.00GHz GenuineIntel GNU/Linux (Using Gentoo 64-bit Linux)
Timo Sirainen
2010-Nov-03 20:32 UTC
[Dovecot] emails getting mangled when dragging from Exchange account to IMAP shared folders
On 3.11.2010, at 19.53, Scott Goodwin wrote:> Emails to one specific user in my organization (let's call him Roy at Ocean.org) > is having his emails mangled when dragging certain items from his Exchange > mailbox to the IMAP account.Is the client using SSL? If not, looking at the actual IMAP traffic would be helpful. Best would be to look it at the Outlook machine, but I don't know what Windows tools exist for that. You could also enable Dovecot's rawlog, which works also with SSL enabled. If in the output you see the same corruption then it's caused by Outlook (or some antivirus/firewall in the middle). http://wiki.dovecot.org/Debugging/Rawlog