Dan Price
2007-Mar-05 09:21 UTC
[Dovecot] new crashes: is the index/mail cache endian neutral?
Part of our migration plan takes our users from one endianness to
another (big to little). Will the index and mail cache files survive? I'm
seeing some new core dumps as the first test user is migrated, which makes me
think... not. What about if we want to build dovecot 64 bit in the future?
Will that cause problems too?
Stack trace is below the log messages. I've edited the user's name out
and
replaced it with foobar, and cleansed out a few sensitive things.
-dp
dovecot: Mar 01 08:46:49 Info: imap-login: Login: user=<foobar>,
method=PLAIN, r
ip=WWW.XXX.YYY.ZZ, lip=AAA.BBB.CC.DD, TLS
dovecot: Mar 01 08:46:49 Info: IMAP(foobar): Effective uid=88442, gid=10, home=/
home/foobar
dovecot: Mar 01 08:46:49 Info: IMAP(foobar): mbox: data=/home/foobar:INBOX=/var/
mail/foobar:INDEX=/home/foobar/Mail/imapd-indices
dovecot: Mar 01 08:46:49 Info: IMAP(foobar): mbox: root=/home/foobar, index=/hom
e/foobar/Mail/imapd-indices, inbox=/var/mail/foobar
dovecot: Mar 01 08:46:50 Error: IMAP(foobar): Corrupted index file /home/foobar/
Mail/imapd-indices/.imap/INBOX/dovecot.index: messages_count too large (0 >
0)
dovecot: Mar 01 08:46:50 Error: IMAP(foobar): Corrupted index file /home/foobar/
Mail/imapd-indices/.imap/INBOX/dovecot.index: messages_count too large (0 >
0)
dovecot: Mar 01 08:46:50 Error: IMAP(foobar): Corrupted index cache file /home/f
oobar/Mail/imapd-indices/.imap/INBOX/dovecot.index.cache: field header points ou
tside file
dovecot: Mar 01 08:46:51 Error: IMAP(foobar): Corrupted transaction log file /ho
me/foobar/Mail/imapd-indices/.imap/INBOX/dovecot.index.log: unknown record type
0x10
dovecot: Mar 01 08:46:51 Warning: IMAP(foobar): fscking index file /home/foobar/
Mail/imapd-indices/.imap/INBOX/dovecot.index
dovecot: Mar 01 08:46:51 Error: IMAP(foobar): Unexpected transaction log desync
with index /home/foobar/Mail/imapd-indices/.imap/INBOX/dovecot.index
dovecot: Mar 01 08:46:51 Info: IMAP(foobar): Disconnected: Mailbox is in inconsi
stent state, please relogin.
dovecot: Mar 01 08:48:06 Info: imap-login: Login: user=<foobar>,
method=PLAIN, r
ip=WWW.XXX.YYY.ZZ, lip=AAA.BBB.CC.DD, TLS
dovecot: Mar 01 08:48:06 Info: IMAP(foobar): Effective uid=88442, gid=10, home=/
home/foobar
dovecot: Mar 01 08:48:06 Info: IMAP(foobar): mbox: data=/home/foobar:INBOX=/var/
mail/foobar:INDEX=/home/foobar/Mail/imapd-indices
dovecot: Mar 01 08:48:06 Info: IMAP(foobar): mbox: root=/home/foobar, index=/hom
e/foobar/Mail/imapd-indices, inbox=/var/mail/foobar
dovecot: Mar 01 09:07:09 Error: IMAP(foobar): Corrupted index file /home/foobar/
Mail/imapd-indices/Mail/.imap/zfs/dovecot.index: messages_count too large (27853
45536 > 1)
dovecot: Mar 01 09:07:10 Error: IMAP(foobar): Corrupted index cache file /home/f
oobar/Mail/imapd-indices/Mail/.imap/zfs/dovecot.index.cache: field header points
outside file
dovecot: Mar 01 09:20:44 Error: IMAP(foobar): Corrupted index file /home/foobar/
Mail/imapd-indices/Mail/.imap/NDMP/dovecot.index: messages_count too large (5200
93696 > 3)
dovecot: Mar 01 09:20:44 Error: IMAP(foobar): Corrupted transaction log file /ho
me/foobar/Mail/imapd-indices/Mail/.imap/NDMP/dovecot.index.log: start_offset (61
44) > file size (1768)
dovecot: Mar 01 09:20:44 Warning: IMAP(foobar): fscking index file /home/foobar/
Mail/imapd-indices/Mail/.imap/NDMP/dovecot.index
dovecot: Mar 01 09:20:44 Error: child 111112 (imap) killed with signal 11
-dp
#0 0x0809eaa1 in mail_index_lock_exclusive (index=0x8117c70,
lock_id_r=0x8047490) at mail-index-lock.c:326
ret = 0
__PRETTY_FUNCTION__ = "mail_index_lock_exclusive"
#1 0x0809e15e in mail_index_fsck (index=0x8117c70) at mail-index-fsck.c:169
error = 0x80e9498 ""
lock_id = 135173240
file_seq = 16777216
file_offset = 6144
ret = 134509928
lock_log = true
#2 0x080a5c49 in mail_transaction_log_file_set_corrupted (file=0x8101c30,
fmt=0x80d51a8 "start_offset (%llu) > file size (%llu)")
at mail-transaction-log.c:64
No locals.
#3 0x080a785e in mail_transaction_log_file_map (file=0x8101c30,
start_offset=6144, end_offset=18446744073709551615)
at mail-transaction-log.c:1313
index = (struct mail_index *) 0x8117c70
st = {st_dev = 47777187, st_pad1 = {0, 0, 0}, st_ino = 15761,
st_mode = 33152, st_nlink = 1, st_uid = 88442, st_gid = 10,
st_rdev = 4294967295, st_pad2 = {0, 0}, st_size = 1768, st_atim = {
tv_sec = 1172769644, tv_nsec = 151892829}, st_mtim = {tv_sec = 1172522367,
tv_nsec = 0}, st_ctim = {tv_sec = 1172732240, tv_nsec = 566025559},
st_blksize = 2048, st_blocks = 4,
st_fstype = "lofs", '\0' <repeats 11 times>, st_pad4 =
{0, 0, 0, 0, 0, 0, 0,
0}}
ret = 1768
use_mmap = 1
__PRETTY_FUNCTION__ = "mail_transaction_log_file_map"
#4 0x080a7bca in mail_transaction_log_sync_lock (log=0x8101bd8,
file_seq_r=0x0, file_offset_r=0x0) at mail-transaction-log.c:1373
__PRETTY_FUNCTION__ = "mail_transaction_log_sync_lock"
#5 0x0809dad3 in mail_index_open (index=0x8117c70, flags=17,
lock_method=MAIL_INDEX_LOCK_FCNTL) at mail-index.c:1423
i = 0
ret = 135294216
#6 0x08096241 in index_storage_mailbox_init (ibox=0x81164b8, name=0x0,
flags=0, move_to_memory=false) at index-storage.c:360
storage = (struct mail_storage *) 0x80f62a0
index_flags = 17
lock_method = MAIL_INDEX_LOCK_FCNTL
ret = 0
__PRETTY_FUNCTION__ = "index_storage_mailbox_init"
#7 0x0807cbfc in mbox_alloc (storage=0x80f62a0, index=0x0,
name=0x80f68b0 "Mail/NDMP", path=0x80e92f0
"/home/foobar/Mail/NDMP",
flags=14) at mbox-storage.c:546
mbox = (struct mbox_mailbox *) 0x81164b8
pool = 0x1
#8 0x0807cd7c in mbox_open (storage=0x80f62a0, name=0x80f68b0
"Mail/NDMP",
flags=14) at mbox-storage.c:662
mbox = (struct mbox_mailbox *) 0x80e9310
index = (struct mail_index *) 0x0
path = 0x80e92f0 "/home/foobar/Mail/NDMP"
index_dir = 0x80e9310
"/home/foobar/Mail/imapd-indices/Mail/.imap/NDMP"
#9 0x0807cf67 in mbox_mailbox_open (_storage=0x80f62a0,
name=0x80f68b0 "Mail/NDMP", input=0x0, flags=14) at
mbox-storage.c:773
path = 0x80e92d8 "/home/foobar/Mail/NDMP"
st = {st_dev = 47777187, st_pad1 = {0, 0, 0}, st_ino = 15330,
st_mode = 33152, st_nlink = 1, st_uid = 88442, st_gid = 10,
st_rdev = 4294967295, st_pad2 = {0, 0}, st_size = 1189083, st_atim = {
tv_sec = 1172732807, tv_nsec = 577163661}, st_mtim = {tv_sec = 1172679608,
tv_nsec = 0}, st_ctim = {tv_sec = 1172731930, tv_nsec = 691932658},
st_blksize = 131072, st_blocks = 2224,
st_fstype = "lofs", '\0' <repeats 11 times>, st_pad4 =
{0, 0, 0, 0, 0, 0, 0,
0}}
#10 0x080acbf6 in mailbox_open (storage=0x0, name=0x80f68b0
"Mail/NDMP",
input=0x0, flags=14) at mail-storage.c:365
No locals.
#11 0x08067f63 in cmd_copy (cmd=0x80f65cc) at cmd-copy.c:83
client = (struct client *) 0x80f6588
storage = (struct mail_storage *) 0x80f62a0
destbox = (struct mailbox *) 0x8
t = (struct mailbox_transaction_context *) 0x806bec0
search_arg = (struct mail_search_arg *) 0x80f8800
messageset = 0x80f68a8 "542506"
mailbox = 0x80f68b0 "Mail/NDMP"
sync_flags = 0
ret = 134511032
#12 0x0806affd in cmd_uid (cmd=0x80f65cc) at cmd-uid.c:19
cmd_name = 0x80f68a0 "copy"
#13 0x0806b9ce in client_handle_input (cmd=0x80f65cc) at client.c:331
client = (struct client *) 0x80f6588
__PRETTY_FUNCTION__ = "client_handle_input"
#14 0x0806b962 in client_handle_input (cmd=0x80f65cc) at client.c:388
client = (struct client *) 0x80f6588
__PRETTY_FUNCTION__ = "client_handle_input"
#15 0x0806bb10 in _client_input (context=0x80f6588) at client.c:428
client = (struct client *) 0x80f6588
cmd = (struct client_command_context *) 0x80f65cc
ret = 2
#16 0x080bdc35 in io_loop_handler_run (ioloop=0x80f3d80) at ioloop-poll.c:199
ctx = (struct ioloop_handler_context *) 0x80f3db8
pollfd = (struct pollfd *) 0x2
tv = {tv_sec = 0, tv_usec = 617423}
io = (struct io *) 0x8105460
t_id = 2
msecs = 0
ret = 0
call = true
--
Daniel Price - Solaris Kernel Engineering - dp at eng.sun.com - blogs.sun.com/dp
Timo Sirainen
2007-Mar-06 14:29 UTC
[Dovecot] new crashes: is the index/mail cache endian neutral?
On Mon, 2007-03-05 at 01:21 -0800, Dan Price wrote:> Part of our migration plan takes our users from one endianness to > another (big to little). Will the index and mail cache files survive?It should rebuild the index files silently and automatically.> I'm > seeing some new core dumps as the first test user is migrated, which makes me > think... not.Looks like the endianess check was done a bit too late. This should fix it: http://dovecot.org/list/dovecot-cvs/2007-March/007927.html> What about if we want to build dovecot 64 bit in the future? > Will that cause problems too?No, the index files will survive that change. If sizeof(off_t) changes then dovecot.index.cache file gets rebuilt, but the default is to use 64bit offsets also with 32bit machines (at least with Linux and Solaris).> dovecot: Mar 01 09:20:44 Warning: IMAP(foobar): fscking index file /home/foobar/ > Mail/imapd-indices/Mail/.imap/NDMP/dovecot.index > dovecot: Mar 01 09:20:44 Error: child 111112 (imap) killed with signal 11http://dovecot.org/list/dovecot-cvs/2007-March/007929.html should fix this. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: <http://dovecot.org/pipermail/dovecot/attachments/20070306/492f86fa/attachment.bin>