Webert de Souza Lima
2016-Sep-29 12:33 UTC
doveadm backup fails (compromised single attachment storage)
Hi, A couple of months ago I had a problem with Single Attachment Storage after infrastructure migration; All mailboxes were rsynced to another filesystem, and that may have broken Single Attachment Storage. Many, many (if not all) mailboxes show the below logs on dovecot: imap(foo at bar.com): Error: read(attachments-connector(zlib(/dovecotdir/mail/ bar.com/foo/mailboxes/INBOX/dbox-Mails/u.26426))) failed: read(/dovecotdir/attach/ bar.com/de/86/de8673894d6fb3f4460e3c26436eefa9a73517fa0f000452f553822367220761502e1d0ce220eee5aa9acf232df0adebf40cce90b57d2e60e1eb9c9ef21671fa-b0d3411772c14957536100009331bd36-43cea6154b3275573b0800009331bd36-26426[base64:19 b/l]) failed: open(/dovecotdir/attach/ bar.com/de/86/de8673894d6fb3f4460e3c26436eefa9a73517fa0f000452f553822367220761502e1d0ce220eee5aa9acf232df0adebf40cce90b57d2e60e1eb9c9ef21671fa-b0d3411772c14957536100009331bd36-43cea6154b3275573b0800009331bd36-26426) failed: No such file or directory When that happens, the MUA keeps syncing forever. Now, I need to migrate all mailboxes (again) to another dovecot instance (with no SAS), which works perfectly for new users but when I try to migrate users from my current dovecot server for this new server, I get such errors again, and I can't migrate: 2016-09-29T12:20:50.995934059Z Sep 29 12:20:50 dsync-server(foo at bar.com): Error: dsync(cf7d091311eb): read(attachments-connector(zlib(/dovecotdir/mdbox/bar.com/foo/storage/m.1))) failed: read(/dovecotdir/attach/ bar.com/0c/df/0cdf86b1920938fe3a043f87e2ee9e63dda276bd5b9fba687e4a0c63d181c3b6ebdb96a9517f048c963db71404ad5d14e896e2e67b7abb0c9e107aed5c15ecf1-430ea904dff46757ba1700009331bd36[base64:18 b/l]) failed: open(/dovecotdir/attach/ bar.com/0c/df/0cdf86b1920938fe3a043f87e2ee9e63dda276bd5b9fba687e4a0c63d181c3b6ebdb96a9517f048c963db71404ad5d14e896e2e67b7abb0c9e107aed5c15ecf1-430ea904dff46757ba1700009331bd36) failed: No such file or directory (last sent=mail, last recv=mail_request (EOL)) Is there a way to fix the attachments problem? (I know I can't recover such files, that's Ok) Is there a way to migrate (dsync backup) ignoring such problems? Thanks in advance.
Webert de Souza Lima
2016-Sep-30 13:18 UTC
doveadm backup fails (compromised single attachment storage)
by SAS I meant SIAS (Single Instance Attachment Storage). On Thu, Sep 29, 2016 at 9:33 AM Webert de Souza Lima <webert.boss at gmail.com> wrote:> Hi, > > A couple of months ago I had a problem with Single Attachment Storage > after infrastructure migration; > > All mailboxes were rsynced to another filesystem, and that may have broken > Single Attachment Storage. Many, many (if not all) mailboxes show the below > logs on dovecot: > > imap(foo at bar.com): Error: > read(attachments-connector(zlib(/dovecotdir/mail/ > bar.com/foo/mailboxes/INBOX/dbox-Mails/u.26426))) failed: > read(/dovecotdir/attach/ > bar.com/de/86/de8673894d6fb3f4460e3c26436eefa9a73517fa0f000452f553822367220761502e1d0ce220eee5aa9acf232df0adebf40cce90b57d2e60e1eb9c9ef21671fa-b0d3411772c14957536100009331bd36-43cea6154b3275573b0800009331bd36-26426[base64:19 > <http://bar.com/de/86/de8673894d6fb3f4460e3c26436eefa9a73517fa0f000452f553822367220761502e1d0ce220eee5aa9acf232df0adebf40cce90b57d2e60e1eb9c9ef21671fa-b0d3411772c14957536100009331bd36-43cea6154b3275573b0800009331bd36-26426%5Bbase64:19> > b/l]) failed: open(/dovecotdir/attach/ > bar.com/de/86/de8673894d6fb3f4460e3c26436eefa9a73517fa0f000452f553822367220761502e1d0ce220eee5aa9acf232df0adebf40cce90b57d2e60e1eb9c9ef21671fa-b0d3411772c14957536100009331bd36-43cea6154b3275573b0800009331bd36-26426) > failed: No such file or directory > > > When that happens, the MUA keeps syncing forever. > > Now, I need to migrate all mailboxes (again) to another dovecot instance > (with no SAS), which works perfectly for new users but when I try to > migrate users from my current dovecot server for this new server, I get > such errors again, and I can't migrate: > > 2016-09-29T12:20:50.995934059Z Sep 29 12:20:50 dsync-server(foo at bar.com): > Error: dsync(cf7d091311eb): > read(attachments-connector(zlib(/dovecotdir/mdbox/bar.com/foo/storage/m.1))) > failed: read(/dovecotdir/attach/ > bar.com/0c/df/0cdf86b1920938fe3a043f87e2ee9e63dda276bd5b9fba687e4a0c63d181c3b6ebdb96a9517f048c963db71404ad5d14e896e2e67b7abb0c9e107aed5c15ecf1-430ea904dff46757ba1700009331bd36[base64:18 > <http://bar.com/0c/df/0cdf86b1920938fe3a043f87e2ee9e63dda276bd5b9fba687e4a0c63d181c3b6ebdb96a9517f048c963db71404ad5d14e896e2e67b7abb0c9e107aed5c15ecf1-430ea904dff46757ba1700009331bd36%5Bbase64:18> > b/l]) failed: open(/dovecotdir/attach/ > bar.com/0c/df/0cdf86b1920938fe3a043f87e2ee9e63dda276bd5b9fba687e4a0c63d181c3b6ebdb96a9517f048c963db71404ad5d14e896e2e67b7abb0c9e107aed5c15ecf1-430ea904dff46757ba1700009331bd36) > failed: No such file or directory (last sent=mail, last recv=mail_request > (EOL)) > > Is there a way to fix the attachments problem? (I know I can't recover > such files, that's Ok) > Is there a way to migrate (dsync backup) ignoring such problems? > > Thanks in advance. >
Webert de Souza Lima
2016-Oct-03 18:29 UTC
doveadm backup fails (compromised single attachment storage)
Since no one seems to know if mailboxes can be "fixed", is possible to run dsync backup ignoring errors? There is no such documentation. When the describe errors occur, sync is interrupted. On Fri, Sep 30, 2016 at 10:18 AM Webert de Souza Lima <webert.boss at gmail.com> wrote:> by SAS I meant SIAS (Single Instance Attachment Storage). > > On Thu, Sep 29, 2016 at 9:33 AM Webert de Souza Lima < > webert.boss at gmail.com> wrote: > >> Hi, >> >> A couple of months ago I had a problem with Single Attachment Storage >> after infrastructure migration; >> >> All mailboxes were rsynced to another filesystem, and that may have >> broken Single Attachment Storage. Many, many (if not all) mailboxes show >> the below logs on dovecot: >> >> imap(foo at bar.com): Error: >> read(attachments-connector(zlib(/dovecotdir/mail/ >> bar.com/foo/mailboxes/INBOX/dbox-Mails/u.26426))) failed: >> read(/dovecotdir/attach/ >> bar.com/de/86/de8673894d6fb3f4460e3c26436eefa9a73517fa0f000452f553822367220761502e1d0ce220eee5aa9acf232df0adebf40cce90b57d2e60e1eb9c9ef21671fa-b0d3411772c14957536100009331bd36-43cea6154b3275573b0800009331bd36-26426[base64:19 >> <http://bar.com/de/86/de8673894d6fb3f4460e3c26436eefa9a73517fa0f000452f553822367220761502e1d0ce220eee5aa9acf232df0adebf40cce90b57d2e60e1eb9c9ef21671fa-b0d3411772c14957536100009331bd36-43cea6154b3275573b0800009331bd36-26426%5Bbase64:19> >> b/l]) failed: open(/dovecotdir/attach/ >> bar.com/de/86/de8673894d6fb3f4460e3c26436eefa9a73517fa0f000452f553822367220761502e1d0ce220eee5aa9acf232df0adebf40cce90b57d2e60e1eb9c9ef21671fa-b0d3411772c14957536100009331bd36-43cea6154b3275573b0800009331bd36-26426) >> failed: No such file or directory >> >> >> When that happens, the MUA keeps syncing forever. >> >> Now, I need to migrate all mailboxes (again) to another dovecot instance >> (with no SAS), which works perfectly for new users but when I try to >> migrate users from my current dovecot server for this new server, I get >> such errors again, and I can't migrate: >> >> 2016-09-29T12:20:50.995934059Z Sep 29 12:20:50 dsync-server(foo at bar.com): >> Error: dsync(cf7d091311eb): >> read(attachments-connector(zlib(/dovecotdir/mdbox/ >> bar.com/foo/storage/m.1))) failed: read(/dovecotdir/attach/ >> bar.com/0c/df/0cdf86b1920938fe3a043f87e2ee9e63dda276bd5b9fba687e4a0c63d181c3b6ebdb96a9517f048c963db71404ad5d14e896e2e67b7abb0c9e107aed5c15ecf1-430ea904dff46757ba1700009331bd36[base64:18 >> <http://bar.com/0c/df/0cdf86b1920938fe3a043f87e2ee9e63dda276bd5b9fba687e4a0c63d181c3b6ebdb96a9517f048c963db71404ad5d14e896e2e67b7abb0c9e107aed5c15ecf1-430ea904dff46757ba1700009331bd36%5Bbase64:18> >> b/l]) failed: open(/dovecotdir/attach/ >> bar.com/0c/df/0cdf86b1920938fe3a043f87e2ee9e63dda276bd5b9fba687e4a0c63d181c3b6ebdb96a9517f048c963db71404ad5d14e896e2e67b7abb0c9e107aed5c15ecf1-430ea904dff46757ba1700009331bd36) >> failed: No such file or directory (last sent=mail, last recv=mail_request >> (EOL)) >> >> Is there a way to fix the attachments problem? (I know I can't recover >> such files, that's Ok) >> Is there a way to migrate (dsync backup) ignoring such problems? >> >> Thanks in advance. >> >