Kenneth Porter
2009-May-25 02:05 UTC
[Dovecot] Converting mbox files with dots in names to Maildir
I tried to run convert-tool from dovecot-1.2-0.rc2 on a tree of mbox files with some containing dots in the filenames and got this failure: # /usr/libexec/dovecot/convert-tool ken /home/ken mbox:/home/ken/mail maildir:/home/ken/Maildir Error: Mailbox conversion: Couldn't create mailbox Lists.WINE.patches: Mailbox already exists Error: Internal failure *** glibc detected *** /usr/libexec/dovecot/convert-tool: double free or corruption (fasttop): 0x08ce6cc8 *** The directory in question contains these files: # ls Lists/WINE/ announce bugs cvs devel patches patches.20050130 Will I need to go through the (large) file tree and rename all files with dots in the names? If so, anyone have a script that will do that? (Changing the dot to dash would be fine with me.)
Kenneth Porter
2009-May-25 19:17 UTC
[Dovecot] Converting mbox files with dots in names to Maildir
Looks like I need to use the optional convert-tool argument "alt_hierarchy_char=-" to get the desired effect. Looking at the source, this is used when the folder name is converted from source format to destination format and the name contains the hierarchy separator from the destination format. Alas, if you don't provide this argument, convert-tool (and presumably the convert plugin, which shares this code) will just crash with a double-free when it encounters such a name.
Timo Sirainen
2009-May-26 01:50 UTC
[Dovecot] Converting mbox files with dots in names to Maildir
On Sun, 2009-05-24 at 19:05 -0700, Kenneth Porter wrote:> # /usr/libexec/dovecot/convert-tool ken /home/ken mbox:/home/ken/mail > maildir:/home/ken/Maildir > Error: Mailbox conversion: Couldn't create mailbox Lists.WINE.patches: > Mailbox already exists > Error: Internal failure > *** glibc detected *** /usr/libexec/dovecot/convert-tool: double free or > corruption (fasttop): 0x08ce6cc8 ***Can you run this via valgrind and show me its output? It's done simply by: valgrind convert-tool ... -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: <http://dovecot.org/pipermail/dovecot/attachments/20090525/bc08711f/attachment-0002.bin>
Kenneth Porter
2009-May-26 04:24 UTC
[Dovecot] Converting mbox files with dots in names to Maildir
Attached is valgrind run. -------------- next part -------------- ==6804== Memcheck, a memory error detector. ==6804== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et al. ==6804== Using LibVEX rev 1658, a library for dynamic binary translation. ==6804== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP. ==6804== Using valgrind-3.2.1, a dynamic binary instrumentation framework. ==6804== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al. ==6804== For more details, rerun with: -v ==6804== Info: Successfully converted ==6804== Invalid read of size 4 ==6804== at 0x80889F2: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x808D4E4: mail_user_deinit_base (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A195: main (in /usr/libexec/dovecot/convert-tool) ==6804== Address 0x401AFF8 is 48 bytes inside a block of size 52 free'd ==6804== at 0x4004FDA: free (vg_replace_malloc.c:233) ==6804== by 0x808896C: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A836: convert_storage (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A177: main (in /usr/libexec/dovecot/convert-tool) ==6804== ==6804== Invalid read of size 4 ==6804== at 0x80889F5: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x808D4E4: mail_user_deinit_base (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A195: main (in /usr/libexec/dovecot/convert-tool) ==6804== Address 0x401AFC8 is 0 bytes inside a block of size 52 free'd ==6804== at 0x4004FDA: free (vg_replace_malloc.c:233) ==6804== by 0x808896C: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A836: convert_storage (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A177: main (in /usr/libexec/dovecot/convert-tool) ==6804== ==6804== Invalid read of size 4 ==6804== at 0x8088929: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x808D4E4: mail_user_deinit_base (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A195: main (in /usr/libexec/dovecot/convert-tool) ==6804== Address 0x401AFF0 is 40 bytes inside a block of size 52 free'd ==6804== at 0x4004FDA: free (vg_replace_malloc.c:233) ==6804== by 0x808896C: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A836: convert_storage (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A177: main (in /usr/libexec/dovecot/convert-tool) ==6804== ==6804== Invalid read of size 4 ==6804== at 0x808892C: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x808D4E4: mail_user_deinit_base (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A195: main (in /usr/libexec/dovecot/convert-tool) ==6804== Address 0x401AFEC is 36 bytes inside a block of size 52 free'd ==6804== at 0x4004FDA: free (vg_replace_malloc.c:233) ==6804== by 0x808896C: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A836: convert_storage (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A177: main (in /usr/libexec/dovecot/convert-tool) ==6804== ==6804== Invalid read of size 4 ==6804== at 0x8088946: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x808D4E4: mail_user_deinit_base (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A195: main (in /usr/libexec/dovecot/convert-tool) ==6804== Address 0x401AFDC is 20 bytes inside a block of size 52 free'd ==6804== at 0x4004FDA: free (vg_replace_malloc.c:233) ==6804== by 0x808896C: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A836: convert_storage (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A177: main (in /usr/libexec/dovecot/convert-tool) ==6804== ==6804== Invalid write of size 4 ==6804== at 0x808895A: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x808D4E4: mail_user_deinit_base (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A195: main (in /usr/libexec/dovecot/convert-tool) ==6804== Address 0x401AFDC is 20 bytes inside a block of size 52 free'd ==6804== at 0x4004FDA: free (vg_replace_malloc.c:233) ==6804== by 0x808896C: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A836: convert_storage (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A177: main (in /usr/libexec/dovecot/convert-tool) ==6804== ==6804== Invalid free() / delete / delete[] ==6804== at 0x4004FDA: free (vg_replace_malloc.c:233) ==6804== by 0x808896C: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x808D4E4: mail_user_deinit_base (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A195: main (in /usr/libexec/dovecot/convert-tool) ==6804== Address 0x401AFC8 is 0 bytes inside a block of size 52 free'd ==6804== at 0x4004FDA: free (vg_replace_malloc.c:233) ==6804== by 0x808896C: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A836: convert_storage (in /usr/libexec/dovecot/convert-tool) ==6804== by 0x804A177: main (in /usr/libexec/dovecot/convert-tool) ==6804== ==6804== ERROR SUMMARY: 7 errors from 7 contexts (suppressed: 12 from 1) ==6804== malloc/free: in use at exit: 1,845 bytes in 4 blocks. ==6804== malloc/free: 831 allocs, 828 frees, 557,458 bytes allocated. ==6804== For counts of detected errors, rerun with: -v ==6804== searching for pointers to 4 not-freed blocks. ==6804== checked 50,380 bytes. ==6804== ==6804== LEAK SUMMARY: ==6804== definitely lost: 1,845 bytes in 4 blocks. ==6804== possibly lost: 0 bytes in 0 blocks. ==6804== still reachable: 0 bytes in 0 blocks. ==6804== suppressed: 0 bytes in 0 blocks. ==6804== Use --leak-check=full to see details of leaked memory. [root at sphincter ken]# ls mail-converted Maildir realmail [root at sphincter ken]# mv mail-converted mail [root at sphincter ken]# rm -Rf Maildir/ [root at sphincter ken]# valgrind /usr/libexec/dovecot/convert-tool ken /home/ken mbox:/home/ken/mail maildir:/home/ken/Maildir alt_hierarchy_char=- > /tmp/valgrind.log ==6814== Memcheck, a memory error detector. ==6814== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et al. ==6814== Using LibVEX rev 1658, a library for dynamic binary translation. ==6814== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP. ==6814== Using valgrind-3.2.1, a dynamic binary instrumentation framework. ==6814== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al. ==6814== For more details, rerun with: -v ==6814== Info: Successfully converted ==6814== Invalid read of size 4 ==6814== at 0x80889F2: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x808D4E4: mail_user_deinit_base (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A195: main (in /usr/libexec/dovecot/convert-tool) ==6814== Address 0x401AFF8 is 48 bytes inside a block of size 52 free'd ==6814== at 0x4004FDA: free (vg_replace_malloc.c:233) ==6814== by 0x808896C: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A836: convert_storage (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A177: main (in /usr/libexec/dovecot/convert-tool) ==6814== ==6814== Invalid read of size 4 ==6814== at 0x80889F5: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x808D4E4: mail_user_deinit_base (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A195: main (in /usr/libexec/dovecot/convert-tool) ==6814== Address 0x401AFC8 is 0 bytes inside a block of size 52 free'd ==6814== at 0x4004FDA: free (vg_replace_malloc.c:233) ==6814== by 0x808896C: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A836: convert_storage (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A177: main (in /usr/libexec/dovecot/convert-tool) ==6814== ==6814== Invalid read of size 4 ==6814== at 0x8088929: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x808D4E4: mail_user_deinit_base (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A195: main (in /usr/libexec/dovecot/convert-tool) ==6814== Address 0x401AFF0 is 40 bytes inside a block of size 52 free'd ==6814== at 0x4004FDA: free (vg_replace_malloc.c:233) ==6814== by 0x808896C: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A836: convert_storage (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A177: main (in /usr/libexec/dovecot/convert-tool) ==6814== ==6814== Invalid read of size 4 ==6814== at 0x808892C: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x808D4E4: mail_user_deinit_base (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A195: main (in /usr/libexec/dovecot/convert-tool) ==6814== Address 0x401AFEC is 36 bytes inside a block of size 52 free'd ==6814== at 0x4004FDA: free (vg_replace_malloc.c:233) ==6814== by 0x808896C: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A836: convert_storage (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A177: main (in /usr/libexec/dovecot/convert-tool) ==6814== ==6814== Invalid read of size 4 ==6814== at 0x8088946: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x808D4E4: mail_user_deinit_base (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A195: main (in /usr/libexec/dovecot/convert-tool) ==6814== Address 0x401AFDC is 20 bytes inside a block of size 52 free'd ==6814== at 0x4004FDA: free (vg_replace_malloc.c:233) ==6814== by 0x808896C: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A836: convert_storage (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A177: main (in /usr/libexec/dovecot/convert-tool) ==6814== ==6814== Invalid write of size 4 ==6814== at 0x808895A: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x808D4E4: mail_user_deinit_base (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A195: main (in /usr/libexec/dovecot/convert-tool) ==6814== Address 0x401AFDC is 20 bytes inside a block of size 52 free'd ==6814== at 0x4004FDA: free (vg_replace_malloc.c:233) ==6814== by 0x808896C: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A836: convert_storage (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A177: main (in /usr/libexec/dovecot/convert-tool) ==6814== ==6814== Invalid free() / delete / delete[] ==6814== at 0x4004FDA: free (vg_replace_malloc.c:233) ==6814== by 0x808896C: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x808D4E4: mail_user_deinit_base (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A195: main (in /usr/libexec/dovecot/convert-tool) ==6814== Address 0x401AFC8 is 0 bytes inside a block of size 52 free'd ==6814== at 0x4004FDA: free (vg_replace_malloc.c:233) ==6814== by 0x808896C: mail_namespace_free (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x8088A0C: mail_namespaces_deinit (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A836: convert_storage (in /usr/libexec/dovecot/convert-tool) ==6814== by 0x804A177: main (in /usr/libexec/dovecot/convert-tool) ==6814== ==6814== ERROR SUMMARY: 7 errors from 7 contexts (suppressed: 12 from 1) ==6814== malloc/free: in use at exit: 1,845 bytes in 4 blocks. ==6814== malloc/free: 831 allocs, 828 frees, 557,458 bytes allocated. ==6814== For counts of detected errors, rerun with: -v ==6814== searching for pointers to 4 not-freed blocks. ==6814== checked 50,380 bytes. ==6814== ==6814== LEAK SUMMARY: ==6814== definitely lost: 1,845 bytes in 4 blocks. ==6814== possibly lost: 0 bytes in 0 blocks. ==6814== still reachable: 0 bytes in 0 blocks. ==6814== suppressed: 0 bytes in 0 blocks. ==6814== Use --leak-check=full to see details of leaked memory.