thr3ads.net - dovecot - FreeBSD ZFS maildir to mdbox [Mar 2015]

If this information is useful, please help other people find it:
Share via:

Roland van Laar

2015-Mar-17 22:09 UTC

FreeBSD ZFS maildir to mdbox

Hello,

I'm converting my mailbox from Maildir to mdbox..
The Maildir is from an 1.2 server.

The new server is a virtual FreeBSD ZFS server.
The conversion is not working, I'm getting either segfaults or the sync 
just quits with errors.
I also did set with mmap_disable=yes. This didn't resolve the problem.
How can I fix this?

The same conversion on a virtual ubuntu 14.04 box works with mentions of 
"filename has the wrong S value" and Corrupted index errors.

The FreeBSD errors and dovecot -n are included below:

dsync(vagrant): Error: 
read(./Maildir/cur/1296038598.29562.mail.micite.net,S=20542:2,Sa) 
failed: Cached message size smaller than expected (20542 < 20640, 
box=INBOX, UID=475)
dsync(vagrant): Error: Maildir filename has wrong S value, renamed the 
file from ./Maildir/cur/1296038598.29562.mail.micite.net,S=20542:2,Sa to 
./Maildir/cur/1296038598.29562.mail.micite.net,S=20640:2,Sa
dsync(vagrant): Error: Corrupted index cache file 
./Maildir/dovecot.index.cache: Broken physical size for mail UID 475
dsync(vagrant): Error: write(/home/vagrant/mdbox/storage/m.12) failed: 
Invalid argument
dsync(vagrant): Error: copy: 
i_stream_read(./Maildir/cur/1296038598.29562.mail.micite.net,S=20542:2,Sa) 
failed: Cached message size smaller than expected (20542 < 20640, 
box=INBOX, UID=475)
dsync(vagrant): Error: 
read(./Maildir/cur/1296038598.29562.mail.micite.net,S=20542:2,Sa) 
failed: Cached message size smaller than expected (20542 < 20640, 
box=INBOX, UID=475) (uid=475, box=INBOX)
dsync(vagrant): Error: Mailbox INBOX: read(msg input) failed: Cached 
message size smaller than expected (20542 < 20640, box=INBOX, UID=475)
dsync(vagrant): Error: 
read(./Maildir/cur/1296038598.29562.mail.micite.net,S=20542:2,Sa) 
failed: Cached message size smaller than expected (20542 < 20640, 
box=INBOX, UID=475) (uid=475, box=INBOX)
dsync(vagrant): Panic: file mail-index-transaction-update.c: line 964 
(mail_index_update_ext): assertion failed: (seq > 0 && (seq <= 
mail_index_view_get_messages_count(t->view) || seq <= t->last_new_seq))
Abort (core dumped)

(gdb) core doveadm.core
Core was generated by `doveadm'.
Program terminated with signal 6, Aborted.
#0  0x00000008013f8a1a in ?? ()
(gdb) bt
#0  0x00000008013f8a1a in ?? ()
#1  0x00000008013f7149 in ?? ()
#2  0x0000000000000000 in ?? ()


or with no core dump:

% doveadm sync maildir:./Maildir
dsync(vagrant): Error: 
read(./Maildir/cur/1296466748.64525.mail.micite.net,S=7629:2,Sa) failed: 
Cached message size smaller than expected (7629 < 7694, box=INBOX, UID=899)
dsync(vagrant): Error: Maildir filename has wrong S value, renamed the 
file from ./Maildir/cur/1296466748.64525.mail.micite.net,S=7629:2,Sa to 
./Maildir/cur/1296466748.64525.mail.micite.net,S=7694:2,Sa
dsync(vagrant): Error: Corrupted index cache file 
./Maildir/dovecot.index.cache: Broken physical size for mail UID 899
dsync(vagrant): Error: write(/home/vagrant/mdbox/storage/m.11) failed: 
Invalid argument
dsync(vagrant): Error: copy: 
i_stream_read(./Maildir/cur/1296466748.64525.mail.micite.net,S=7629:2,Sa)
failed:
Cached message size smaller than expected (7629 < 7694, box=INBOX, UID=899)
dsync(vagrant): Error: 
read(./Maildir/cur/1296466748.64525.mail.micite.net,S=7629:2,Sa) failed: 
Cached message size smaller than expected (7629 < 7694, box=INBOX, 
UID=899) (uid=899, box=INBOX)
dsync(vagrant): Error: Mailbox INBOX: read(msg input) failed: Cached 
message size smaller than expected (7629 < 7694, box=INBOX, UID=899)
dsync(vagrant): Error: 
read(./Maildir/cur/1296466748.64525.mail.micite.net,S=7629:2,Sa) failed: 
Cached message size smaller than expected (7629 < 7694, box=INBOX, 
UID=899) (uid=899, box=INBOX)



% dovecot -n
# 2.2.16: /usr/local/etc/dovecot/dovecot.conf
# OS: FreeBSD 10.1-RELEASE-p6 amd64
mail_location = mdbox:~/mdbox
namespace {
   inbox = yes
   location    prefix    separator = .
}
passdb {
   args = /usr/local/etc/dovecot/conf.d/dovecot-sql.conf.ext
   driver = sql
}
protocols = lmtp imap
service auth {
   unix_listener /var/spool/postfix/private/auth {
     group = postfix
     mode = 0660
     user = postfix
   }
}
service lmtp {
   unix_listener /var/spool/postfix/private/dovecot-lmtp {
     group = postfix
     mode = 0600
     user = postfix
   }
   user = vmail
}
ssl = required
ssl_cert = </etc/ssl/ec/ec.crt
ssl_key = </etc/ssl/ec/ec.key
userdb {
   args = /usr/local/etc/dovecot/conf.d/dovecot-sql.conf.ext
   driver = sql
}
protocol lmtp {
   postmaster_address = postmaster at micite.net
}

Toni Mattila

2015-Mar-18 07:26 UTC

head link

FreeBSD ZFS maildir to mdbox

Hi,

On 18-Mar-15 00:09, Roland van Laar wrote:> I'm converting my mailbox from Maildir to mdbox..
> The Maildir is from an 1.2 server.
> The same conversion on a virtual ubuntu 14.04 box works with mentions of
> "filename has the wrong S value" and Corrupted index errors.
You should fix the Maildir files first to have correct S= (size) on 
them. Older maildrops and qmail likes to create wrong sizes and newer 
dovecots rely on that S= to be correct.

You can use http://www.dovecot.org/tools/maildir-size-fix.pl or similar 
script to fix your existing maildirs.

Best Regards,
Toni

Roland van Laar

2015-Mar-21 11:25 UTC

head link

FreeBSD ZFS maildir to mdbox

On 18-03-15 08:26, Toni Mattila wrote:> Hi,
>
> On 18-Mar-15 00:09, Roland van Laar wrote:
>> I'm converting my mailbox from Maildir to mdbox..
>> The Maildir is from an 1.2 server.
>> The same conversion on a virtual ubuntu 14.04 box works with mentions
of
>> "filename has the wrong S value" and Corrupted index errors.
>
> You should fix the Maildir files first to have correct S= (size) on 
> them. Older maildrops and qmail likes to create wrong sizes and newer 
> dovecots rely on that S= to be correct.
>
> You can use http://www.dovecot.org/tools/maildir-size-fix.pl or 
> similar script to fix your existing maildirs.That fixed it for me, the migration from maildir to mdbox is working now.

I had trouble running the script at first, I added some extra 
information about the different kind of commandline options.
Feel free to include them in the script on dovecot.org.

LINE: 233
if (scalar @ARGV == 0) {
     print STDERR "Usage: maildir-size-fix.pl /path/to/Maildir\n";
   print STDERR "-c :Check if the files are compressed. Use the 
uncompressed size for S=size.\n";
   print STDERR "-f :If S=size already exists, verify that it is 
correct.\n";
   print STDERR "-n :If filename doesn't already have a S=size, add
it.\n";
   print STDERR "-p :If UIDLs are based on filename and no P<uidl>
entry
already
   \t exist for a message, write a P<original filename> entry so it
doesn't
   \t change when renaming a file.\n";
   print STDERR "-r :Recursively scan the maildir for
subdirectories.\n";
   print STDERR "-v :Verbose logging.\n";
   exit 1
}>
> Best Regards,
> Toni
>

Apparently Analagous Threads

Search for more apparently analagous threads

dovecot - Mar 2015 - FreeBSD ZFS maildir to mdbox

FreeBSD ZFS maildir to mdbox

FreeBSD ZFS maildir to mdbox

FreeBSD ZFS maildir to mdbox

Apparently Analagous Threads