similar to: Single Instance Storage after the fact

Having had a week with the system I'm now inclined to believe that the issues I've seen are not necessarily related to SIS and zlib but may be due to replication instead. After experiencing these issues and looking at historical mailing list posts I wrote a basic script (as was suggested as a possible diagnostic approach for SIS before SIS was even implemented!) which pulled all the

I'm running Dovecot 2.3.4 with Single Instance Storage (SIS) and zlib and I am frequently seeing errors where files are missing in the attachments directory even though the zipped file in the hash directory is actually there.? This appears not to have been an issue in Dovecot 2.3.1 Requested output of the server below. The error looks like this in the log Dec 18 08:58:58 dovecot01

http://hg.dovecot.org/dovecot-2.0-sis contains the code for it. Otherwise it's the latest (as of writing this) dovecot-2.0 hg tree. Please test if you're interested in SIS. :) Once there's at least some testing, I'll probably add this to v2.0.x since very little of this new code is used when SIS is disabled (which is the default of course). SIS works pretty much like explained in

Hello We are using dovecot 2.0.16 with mdbox+zlib. We are now testing SiS (Single Instance Storage) and I have 5 questions: 1? Is possible to "dedup" existing mdboxes? 2? Are attachments compressed with zlib if mdboxes already use zlib? 3? I have plenty CPU to spare, should I use a low value of mail_attachment_min_size , like 16KB ? 4? Can I "undo" SiS if I have

Ok, didn't find much on this... The bottom of /etc/dovecot/conf.d/10-mail.conf has the following settings: #mail_attachment_dir = #mail_attachment_min_size = 128k #mail_attachment_fs = sis posix #mail_attachment_hash = %{sha1} all of which are apparently the defaults according to doveconf -d output - so no need to set them explicitly, unless I want to change them, right? So, enable the

This single-instance storage is going to encourage bad habits! I just found myself preparing an e-mail that needs to be sent to multiple recipients - including several within my own organization - yet it's not necessary for everyone (in-house) to get the large attachment. As I was getting ready to split the message into two parts for distribution I remembered - I can just send it to

I have read most of the doc on the dovecot website, and couldn't find any info on the single instance storage feature, so I'm posting my questions here. - Are these 3 parameters the only one necessary for single instance storage? I cannot find any doc on this feature on the website; is there anything specific I need to know about them? (the last one isn't exactly self-explanatory). -

It's apparent I have some level of corruption in my mailboxes - something to do with the combination of mdbox, sis, and zlib. I should note that I know I have a mix of compressed and raw messages - if that makes any difference in identifying the problems. I think I need to somehow regenerate the mailboxes with issues. Is there a suggestion for how to accomplish this? Should I perhaps use

Since changeset 8a63f621bd2e I'm one step closer to the perfect working setup. I'm sharing one GID per domain, all it's alias names and accounts. So I can use: mail_attachment_dir = /srv/mail/.SiS/%{gid} In order to avoid permission trouble I've set fs ACL on the .SiS dir: ,--[ getfacl srv/mail/.SiS ]-- | # file: srv/mail/.SiS | # owner: root | # group: root | user::rwx |

I've been running continous dsync backups of our Maildirs for a few weeks now, with the destination dsync server using mdbox and SIS. The idea was that the destination server would act as a warm copy of all our active users data. The active servers are using Maildir, and has: $ df -h /usr/local/atmail/users/ Filesystem Size Used Avail Use% Mounted on /dev/atmailusers

Hi, I am in the middle of setting up a dovecot imap server to replace an aging cyrus imapd. The new server is using mdbox storage for better filesystem performance and is running dovecot-2.2.32. To migrate email over and keep UIDs etc. intact I used the backup feature of doveadm. During testing this generally worked fine. So thanks for that functionality, it really makes migration much

Hi everyone, I am currently trying to setup our next dovecot server (2.2.18) with SIS enabled, but I have enconutered 1 problem and 1 question: *1) Same attachment, different filenames* I have a feeling a misconfigured something (or forgot to do something) because SIS almost works, but not quite since 3 identical attachment take 3x the space (see end of email for my SIS config). For 3 the

Hi, A couple of months ago I had a problem with Single Attachment Storage after infrastructure migration; All mailboxes were rsynced to another filesystem, and that may have broken Single Attachment Storage. Many, many (if not all) mailboxes show the below logs on dovecot: imap(foo at bar.com): Error: read(attachments-connector(zlib(/dovecotdir/mail/

Now I'm seeing a bunch of these: 02:28 bubba dovecot: dsync-local(user at mydomain.com): Error: bzlib.read(/var/mail/mydomain.com/user/mdbox/storage/m.20): corrupted data at 24123649 Mar 21 11:02:28 bubba dovecot: dsync-local(user at mydomain.com): Error: dsync(oldserver.host): read(zlib(/var/mail/mydomain.com/user/mdbox/storage/m.20)) failed:

Just a note to myself and whoever else cares, should be added to wiki once it has its own page about this: With single-dbox messages can be copied with hard linking. This means that there can be multiple files that point to the same attachment file. The attachment is now deleted only once the mail file's link count drops to zero, so this works fine.. ..until someone goes and starts manually

I'm getting close - I believe I have one-way replication processing now. It's amazing watching the mails magically appear in the new server! I'll tender some updates to the wiki in the hopes it will help others. At this time, I have two errors I haven't been able to eliminate with config changes: dovecot: doveadm: Error: dsync-remote(user at mydomain.com): Warning:

Hi, I'm trying to use dsync to convert sdbox + sis mailboxes to mbox (mbox is chosen here to "re-attach" the attachments to their original place) # dsync -Dv -u $LOGIN -o "mail_location=sdbox:/zfspool/clone_srv_mail/$LOGIN" -o "mail_attachment_dir=/zfspool/clone_srv_attachments" backup

Hi all, I'm receiving a reproducible dsync backup panic for some of my users. The error message is as follows: dsync-remote(user@<removed>): Panic: file dsync-brain-mailbox-tree-sync.c: line 111 (dsync_brain_mailbox_tree_sync_change): assertion failed: (brain->no_backup_overwrite) dsync-remote(user@<removed>): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x69b70)

Hello, I am trying to build dovecot-2.1.15 on Solaris 10 update 10 with Solaris Studio 12.3 and get this error: /bin/bash ../../libtool --tag=CC --mode=link /opt/SUNWspro/bin/cc -fast -xipo=2 -I/usr/sfw/include -I/usr/sfw/include -no-undefined -o doveadm doveadm-mail.o doveadm-mail-altmove.o doveadm-mail-expunge.o doveadm-mail-fetch.o doveadm-mail-import.o doveadm-mail-index.o

Now that v2.0.0 is only waiting for people to report bugs (and me to figure out how to fix them), I've finally had time to start doing what I actually came here (Portugal Telecom/SAPO) to do. :) The idea is to have dbox and mdbox support saving attachments (or MIME parts in general) to separate files, which with some magic gives a possibility to do single instance attachment storage. Comments