It has never been safe, not even in 2.2.10, to access mail from multiple backends concurrently. You have just gotten lucky. You must use director (or similar solution) to ensure all sessions end up in same server. Aki> On 02/03/2021 09:26 Gonzalo Palacios Goicolea <gonzalo.palacios at externo.uam.es> wrote: > > > Hi Aki > We're using NFS v3. Should we use v4 to avoid this problem? > > Regarding concurrent access, when a user uses more than one device it can go to any of the servers. It's the same config we had with dovecot-2.2.10 that never gave us this problem. > So, if the issue is related to concurrent access, is it any configuration we can apply to solve it? Or the only solution is director? > Thanks again > > El 02/03/2021 a las 7:27, Aki Tuomi escribi?: > > > > On 01/03/2021 14:59 Gonzalo Palacios Goicolea <gonzalo.palacios at externo.uam.es> wrote: > > > > > > > > > Hi All, > > > > > > We recently moved from RHEL6 to RHEL7 and updated dovecot from dovecot-2.2.10-1_14.el6.x86_64 to dovecot-2.2.36-8.el7.x86_64. We use NFS for storing the mail, and 5 servers running at the same time. > > > Before updating dovecot we stopped the service and deleted all dovecot.index* files > > > Now we're experiencing some problems. For example, when deleting an email it is note deleted and appears again duplicated. These are the logs: > > > > > > Feb 28 08:50:42 server5 dovecot: imap(user at domain): Error: Index path/Maildir/dovecot.index: Lost log for seq=5 offset=228: Missing middle file seq=5 (between 5..4294967295, we have seqs 4): Log is locked - newer log can't exist (initial_mapped=1, reason=Index mapped) > > > Feb 28 08:50:42 server5 dovecot: imap(user at domain): Warning: fscking index file path/Maildir/dovecot.index > > > Feb 28 08:50:42 server5 dovecot: imap(user at domain): Error: Failed to map transaction log path/Maildir/dovecot.index.log at sync_offset=10328 after locking: pread() failed: Stale file handle > > > Feb 28 08:50:42 server5 dovecot: imap(user at domain): Error: Index path/Maildir/dovecot.index: Lost log for seq=5 offset=228: Missing middle file seq=5 (between 5..4294967295, we have seqs 4): Requested newer log than exists - still after NFS flush: Log inode is unchanged (initial_mapped=1, reason=Index mapped) > > > Feb 28 08:50:45 server5 dovecot: imap(user at domain): Error: Failed to map transaction log path/Maildir/dovecot.index.log at sync_offset=10328 after locking: pread() failed: Stale file handle > > > Feb 28 08:50:45 server5 dovecot: imap(user at domain): Error: Failed to map view for path/Maildir/dovecot.index: Missing middle file seq=5 (between 5..5, we have seqs 4): Requested newer log than exists - still after NFS flush: Log inode is unchanged > > > > > > Usually restarting the email client solves the problem for a while. We've tried deleting the indexes but it happens again some time later. > > > > > > This is dovecot config: > > > > > > dovecot -n > > > # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf > > > # Pigeonhole version 0.4.24 (124e06aa) > > > # OS: Linux 3.10.0-1160.6.1.el7.x86_64 x86_64 Red Hat Enterprise Linux Server release 7.9 (Maipo) > > > # Hostname: server > > > auth_cache_negative_ttl = 5 mins > > > auth_cache_size = 500 M > > > auth_cache_ttl = 5 mins > > > auth_failure_delay = 5 secs > > > default_vsz_limit = 1 G > > > disable_plaintext_auth = no > > > hostname = server > > > imap_logout_format = in(bytes read)=%i out(bytes sent)=%o > > > listen = * > > > lock_method = dotlock > > > mail_fsync = always > > > mail_location = maildir:%h/Maildir > > > mail_nfs_index = yes > > > mail_nfs_storage = yes > > > mail_plugins = " quota mail_log notify" > > > managesieve_notify_capability = mailto > > > managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapflags notify > > > mbox_write_locks = fcntl > > > mmap_disable = yes > > > namespace { > > > location = maildir:%h/Maildir/_backup > > > prefix = _backup. > > > separator = . > > > type = private > > > } > > > namespace inbox { > > > inbox = yes > > > location > > > mailbox Drafts { > > > special_use = \Drafts > > > } > > > mailbox Junk { > > > special_use = \Junk > > > } > > > mailbox Sent { > > > special_use = \Sent > > > } > > > mailbox "Sent Messages" { > > > special_use = \Sent > > > } > > > mailbox Trash { > > > special_use = \Trash > > > } > > > prefix > > > } > > > passdb { > > > args = /etc/dovecot/dovecot-ldap.conf.ext > > > driver = ldap > > > } > > > plugin { > > > mail_log_events = delete undelete expunge mailbox_delete mailbox_rename mailbox_create save > > > mail_log_fields = uid box msgid from subject > > > quota = maildir:_backup:ns> > > quota_exceeded_message = El espacio asignado para el almacenamiento del correo del destinatario ha superado su capacidad total. Por ello, el correo que usted intenta enviar no se puede entregar al destinatario. > > > quota_warning = storage=70%% quota-warning 70 %u > > > quota_warning1 = storage=90%% quota-warning 90 %u > > > quota_warning2 = storage=95%% quota-warning 95 %u > > > quota_warning3 = storage=99%% quota-warning 99 %u > > > sieve = ~/.dovecot.sieve > > > sieve_dir = ~/Maildir/sieve > > > sieve_extensions = +notify +imapflags > > > } > > > postmaster_address = postmaster at domain > > > protocols = imap pop3 sieve > > > service auth { > > > unix_listener auth-userdb { > > > group = vmail > > > mode = 0666 > > > user = vmail > > > } > > > } > > > service imap-login { > > > inet_listener imap { > > > port = 143 > > > } > > > process_min_avail = 20 > > > } > > > service imap { > > > process_limit = 2048 > > > } > > > service managesieve-login { > > > inet_listener sieve { > > > port = 4190 > > > } > > > inet_listener sieve_deprecated { > > > port = 2000 > > > } > > > process_min_avail = 20 > > > } > > > service pop3-login { > > > inet_listener pop3 { > > > port = 110 > > > } > > > process_min_avail = 20 > > > } > > > service pop3 { > > > process_limit = 2048 > > > } > > > service quota-warning { > > > executable = script /etc/dovecot/scripts_notification_quota/quota-warning.sh > > > unix_listener quota-warning { > > > user = vmail > > > } > > > user = vmail > > > } > > > ssl = no > > > ssl_cert = </etc/pki/dovecot/certs/dovecot.pem > > > ssl_key = # hidden, use -P to show it > > > submission_host = smtp.domain:25 > > > syslog_facility = local1 > > > userdb { > > > args = /etc/dovecot/dovecot-ldap.conf.ext > > > driver = ldap > > > } > > > protocol lda { > > > mail_plugins = " quota mail_log notify sieve" > > > } > > > protocol imap { > > > mail_max_userip_connections = 10000 > > > mail_plugins = " quota mail_log notify imap_quota" > > > } > > > protocol pop3 { > > > mail_max_userip_connections = 10 > > > } > > > > > > Can you please help me? > > > Thanks in advance and best regards > > > > > > -- > > > > > > Gonzalo Palacios Goicolea > > > > > Hi! > > > > Are you ensuring that the same user is not concurrently being accessed by multiple backends. All mail sessions for a user, including mail delivery. > > > > Are you using NFSv3 or NFSv4? > > > > Aki > > > -- > > Gonzalo Palacios Goicolea > U.T de Infraestructura de Equipos Centrales > Tecnolog?as de la Informaci?n > Universidad Aut?noma de Madrid ? Campus de Cantoblanco > c/ Francisco Tom?s y Valiente, n? 11, Edificio B de la EPS ? TI-208 - 28049, Madrid > Tel.: 91497 2368 ? gonzalo.palacios at externo.uam.es - www.uam.es > > LinkedIn | Twitter | Facebook | Youtube > ?Eres de la Aut?noma? Hazte AlumniUAM > Antes de imprimir este correo piense si es necesario.Cuidemos el medioambiente.
Gonzalo Palacios Goicolea
2021-Mar-02 07:56 UTC
Failed to map transaction log path - Stale file handle
Hi Aki, Thanks for your quick answer. I've read it's recommended to use different servers for director and backends. So, for a 5 backend cluster, how many director servers are recommended? I guess that at least 2, but if it's better to use more than 2, please, let me know. Best regards Gonzalo El 02/03/2021 a las 8:35, Aki Tuomi escribi?:> It has never been safe, not even in 2.2.10, to access mail from multiple backends concurrently. You have just gotten lucky. > > You must use director (or similar solution) to ensure all sessions end up in same server. > > Aki > >> On 02/03/2021 09:26 Gonzalo Palacios Goicolea <gonzalo.palacios at externo.uam.es> wrote: >> >> >> Hi Aki >> We're using NFS v3. Should we use v4 to avoid this problem? >> >> Regarding concurrent access, when a user uses more than one device it can go to any of the servers. It's the same config we had with dovecot-2.2.10 that never gave us this problem. >> So, if the issue is related to concurrent access, is it any configuration we can apply to solve it? Or the only solution is director? >> Thanks again >> >> El 02/03/2021 a las 7:27, Aki Tuomi escribi?: >> >>>> On 01/03/2021 14:59 Gonzalo Palacios Goicolea <gonzalo.palacios at externo.uam.es> wrote: >>>> >>>> >>>> Hi All, >>>> >>>> We recently moved from RHEL6 to RHEL7 and updated dovecot from dovecot-2.2.10-1_14.el6.x86_64 to dovecot-2.2.36-8.el7.x86_64. We use NFS for storing the mail, and 5 servers running at the same time. >>>> Before updating dovecot we stopped the service and deleted all dovecot.index* files >>>> Now we're experiencing some problems. For example, when deleting an email it is note deleted and appears again duplicated. These are the logs: >>>> >>>> Feb 28 08:50:42 server5 dovecot: imap(user at domain): Error: Index path/Maildir/dovecot.index: Lost log for seq=5 offset=228: Missing middle file seq=5 (between 5..4294967295, we have seqs 4): Log is locked - newer log can't exist (initial_mapped=1, reason=Index mapped) >>>> Feb 28 08:50:42 server5 dovecot: imap(user at domain): Warning: fscking index file path/Maildir/dovecot.index >>>> Feb 28 08:50:42 server5 dovecot: imap(user at domain): Error: Failed to map transaction log path/Maildir/dovecot.index.log at sync_offset=10328 after locking: pread() failed: Stale file handle >>>> Feb 28 08:50:42 server5 dovecot: imap(user at domain): Error: Index path/Maildir/dovecot.index: Lost log for seq=5 offset=228: Missing middle file seq=5 (between 5..4294967295, we have seqs 4): Requested newer log than exists - still after NFS flush: Log inode is unchanged (initial_mapped=1, reason=Index mapped) >>>> Feb 28 08:50:45 server5 dovecot: imap(user at domain): Error: Failed to map transaction log path/Maildir/dovecot.index.log at sync_offset=10328 after locking: pread() failed: Stale file handle >>>> Feb 28 08:50:45 server5 dovecot: imap(user at domain): Error: Failed to map view for path/Maildir/dovecot.index: Missing middle file seq=5 (between 5..5, we have seqs 4): Requested newer log than exists - still after NFS flush: Log inode is unchanged >>>> >>>> Usually restarting the email client solves the problem for a while. We've tried deleting the indexes but it happens again some time later. >>>> >>>> This is dovecot config: >>>> >>>> dovecot -n >>>> # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf >>>> # Pigeonhole version 0.4.24 (124e06aa) >>>> # OS: Linux 3.10.0-1160.6.1.el7.x86_64 x86_64 Red Hat Enterprise Linux Server release 7.9 (Maipo) >>>> # Hostname: server >>>> auth_cache_negative_ttl = 5 mins >>>> auth_cache_size = 500 M >>>> auth_cache_ttl = 5 mins >>>> auth_failure_delay = 5 secs >>>> default_vsz_limit = 1 G >>>> disable_plaintext_auth = no >>>> hostname = server >>>> imap_logout_format = in(bytes read)=%i out(bytes sent)=%o >>>> listen = * >>>> lock_method = dotlock >>>> mail_fsync = always >>>> mail_location = maildir:%h/Maildir >>>> mail_nfs_index = yes >>>> mail_nfs_storage = yes >>>> mail_plugins = " quota mail_log notify" >>>> managesieve_notify_capability = mailto >>>> managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapflags notify >>>> mbox_write_locks = fcntl >>>> mmap_disable = yes >>>> namespace { >>>> location = maildir:%h/Maildir/_backup >>>> prefix = _backup. >>>> separator = . >>>> type = private >>>> } >>>> namespace inbox { >>>> inbox = yes >>>> location >>>> mailbox Drafts { >>>> special_use = \Drafts >>>> } >>>> mailbox Junk { >>>> special_use = \Junk >>>> } >>>> mailbox Sent { >>>> special_use = \Sent >>>> } >>>> mailbox "Sent Messages" { >>>> special_use = \Sent >>>> } >>>> mailbox Trash { >>>> special_use = \Trash >>>> } >>>> prefix >>>> } >>>> passdb { >>>> args = /etc/dovecot/dovecot-ldap.conf.ext >>>> driver = ldap >>>> } >>>> plugin { >>>> mail_log_events = delete undelete expunge mailbox_delete mailbox_rename mailbox_create save >>>> mail_log_fields = uid box msgid from subject >>>> quota = maildir:_backup:ns>>>> quota_exceeded_message = El espacio asignado para el almacenamiento del correo del destinatario ha superado su capacidad total. Por ello, el correo que usted intenta enviar no se puede entregar al destinatario. >>>> quota_warning = storage=70%% quota-warning 70 %u >>>> quota_warning1 = storage=90%% quota-warning 90 %u >>>> quota_warning2 = storage=95%% quota-warning 95 %u >>>> quota_warning3 = storage=99%% quota-warning 99 %u >>>> sieve = ~/.dovecot.sieve >>>> sieve_dir = ~/Maildir/sieve >>>> sieve_extensions = +notify +imapflags >>>> } >>>> postmaster_address = postmaster at domain >>>> protocols = imap pop3 sieve >>>> service auth { >>>> unix_listener auth-userdb { >>>> group = vmail >>>> mode = 0666 >>>> user = vmail >>>> } >>>> } >>>> service imap-login { >>>> inet_listener imap { >>>> port = 143 >>>> } >>>> process_min_avail = 20 >>>> } >>>> service imap { >>>> process_limit = 2048 >>>> } >>>> service managesieve-login { >>>> inet_listener sieve { >>>> port = 4190 >>>> } >>>> inet_listener sieve_deprecated { >>>> port = 2000 >>>> } >>>> process_min_avail = 20 >>>> } >>>> service pop3-login { >>>> inet_listener pop3 { >>>> port = 110 >>>> } >>>> process_min_avail = 20 >>>> } >>>> service pop3 { >>>> process_limit = 2048 >>>> } >>>> service quota-warning { >>>> executable = script /etc/dovecot/scripts_notification_quota/quota-warning.sh >>>> unix_listener quota-warning { >>>> user = vmail >>>> } >>>> user = vmail >>>> } >>>> ssl = no >>>> ssl_cert = </etc/pki/dovecot/certs/dovecot.pem >>>> ssl_key = # hidden, use -P to show it >>>> submission_host = smtp.domain:25 >>>> syslog_facility = local1 >>>> userdb { >>>> args = /etc/dovecot/dovecot-ldap.conf.ext >>>> driver = ldap >>>> } >>>> protocol lda { >>>> mail_plugins = " quota mail_log notify sieve" >>>> } >>>> protocol imap { >>>> mail_max_userip_connections = 10000 >>>> mail_plugins = " quota mail_log notify imap_quota" >>>> } >>>> protocol pop3 { >>>> mail_max_userip_connections = 10 >>>> } >>>> >>>> Can you please help me? >>>> Thanks in advance and best regards >>>> >>>> -- >>>> >>>> Gonzalo Palacios Goicolea >>>> >>> Hi! >>> >>> Are you ensuring that the same user is not concurrently being accessed by multiple backends. All mail sessions for a user, including mail delivery. >>> >>> Are you using NFSv3 or NFSv4? >>> >>> Aki >>> >> -- >> >> Gonzalo Palacios Goicolea >> U.T de Infraestructura de Equipos Centrales >> Tecnolog?as de la Informaci?n >> Universidad Aut?noma de Madrid ? Campus de Cantoblanco >> >> Antes de imprimir este correo piense si es necesario.Cuidemos el medioambiente.-- *Gonzalo Palacios Goicolea* U.T de Infraestructura de Equipos Centrales /Tecnolog?as de la Informaci?n/ Universidad Aut?noma de Madrid ? Campus de Cantoblanco <3D"https://alumni.uam.es/"> Antes de imprimir este correo piense si es necesario.Cuidemos el medioambiente. -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20210302/f0336510/attachment-0001.html>