thr3ads.net - dovecot - replicator: Panic: data stack: Out of memory when allocating 268435496 bytes [Jan 2023]

If this information is useful, please help other people find it:
Share via:

Gerben Wierda

2023-Jan-06 10:32 UTC

replicator: Panic: data stack: Out of memory when allocating 268435496 bytes

> On 6 Jan 2023, at 08:53, Aki Tuomi <aki.tuomi at open-xchange.com>
wrote:
> 
> 
> 
> On January 6, 2023 3:56:39 AM GMT+02:00, Gerben Wierda <gerben.wierda at
rna.nl> wrote:
>> One step further in my quest to create a replacement mail server.
>> 
>> I now have my old mail server (2.3.19.1, macOS + MacPorts) and my new
(2.3.20, Alpine Linux, Docker, apk package). When I turn on replication it
works, but, after a while I see:
>> 
>> Jan 06 00:50:31 replicator: Panic: data stack: Out of memory when
allocating 268435496 bytes
>> Jan 06 00:50:32 replicator: Fatal: master: service(replicator): child
133 killed with signal 6 (core dumped)
>> Jan 06 00:50:32 lmtp(pid 195 user sysbh): Warning: replication(sysbh):
Sync failure:
>> Jan 06 00:50:32 lmtp(pid 195 user sysbh): Warning: replication(sysbh):
Remote sent invalid input: -
>> 
>> I've removed synchronous operation for now (found a message on the
net suggesting that) but is this known and what does it mean?
>> 
>> Gerben Wierda (LinkedIn
<https://www.linkedin.com/in/gerbenwierda>)
>> R&A IT Strategy <https://ea.rna.nl/> (main site)
>> Book: Chess and the Art of Enterprise Architecture
<https://ea.rna.nl/the-book/>
>> Book: Mastering ArchiMate
<https://ea.rna.nl/the-book-edition-iii/>
>> 
> 
> Dovecot default memory limit is 256M. You should probably set
> 
> service replicator {
>  vsz_limit = 2G
> }
> 
> because replicator might have to use more memory, especially for larger
indexes.
> 
> Aki
That is a good tip as well. 

I had followed this bit of experience from someone else:
https://marc.info/?l=dovecot&m=164438199727640
<https://marc.info/?l=dovecot&m=164438199727640>, haven't seen any
err message since. But that might be because they are in sync now and both sides
are aware. Can I trigger full replication again so I can test?

Gerben
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20230106/35b28c7c/attachment.htm>

Paul Kudla

2023-Jan-06 15:49 UTC

head link

replicator: Panic: data stack: Out of memory when allocating 268435496 bytes

i ran into this as well

here is the full config for mine with replication

# cat dovecot.conf
# 2.3.14 (cee3cbc0d): /usr/local/etc/dovecot/dovecot.conf
# OS: FreeBSD 12.1-RELEASE amd64
# Hostname: mail18.scom.ca

auth_debug = no
auth_debug_passwords = no

default_process_limit = 16384

mail_debug = no

#lock_method = dotlock
#mail_max_lock_timeout = 300s

#mbox_read_locks = dotlock
#mbox_write_locks = dotlock

mmap_disable = yes
dotlock_use_excl = no
mail_fsync = always
mail_nfs_storage = no
mail_nfs_index = no

auth_mechanisms = plain login
auth_verbose = yes
base_dir = /data/dovecot/run/
debug_log_path = syslog
disable_plaintext_auth = no
dsync_features = empty-header-workaround

info_log_path = syslog
login_greeting = SCOM.CA Internet Services Inc. - Dovecot ready
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c


mail_location = maildir:~/

mail_plugins = " virtual notify replication fts fts_lucene "
mail_prefetch_count = 20

protocols = imap pop3 lmtp sieve


protocol lmtp {
   mail_plugins = $mail_plugins sieve
   postmaster_address }

service lmtp {
   process_limit=1000
   vsz_limit = 512m
   client_limit=1
    unix_listener /usr/home/postfix.local/private/dovecot-lmtp {
      group = postfix
      mode = 0600
      user = postfix
   }
}

protocol lda {
   mail_plugins = $mail_plugins sieve
}

service lda {
   process_limit=1000
   vsz_limit = 512m
}

service imap {
   process_limit=4096
   vsz_limit = 2g
   client_limit=1
}

service pop3 {
   process_limit=1000
   vsz_limit = 512m
   client_limit=1
}

namespace inbox {
   inbox = yes
   location    mailbox Drafts {
     auto = subscribe
     special_use = \Drafts
   }
   mailbox Sent {
     auto = subscribe
     special_use = \Sent
   }
   mailbox Trash {
     auto = subscribe
     special_use = \Trash
   }
   prefix    separator = /
}

passdb {
   args = /usr/local/etc/dovecot/dovecot-pgsql.conf
   driver = sql
}

doveadm_port = 12345
doveadm_password = secretxxxx

service doveadm {
   process_limit = 0
   process_min_avail = 0
   idle_kill = 0
   client_limit = 1
   user = vmail
   inet_listener {
     port = 12345
   }
}

service config {
   unix_listener config {
     user = vmail
     }
}

dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u
#dsync_remote_cmd = doveadm sync -d -u%u

replication_dsync_parameters = -d -N -l 300 -U

plugin {
   mail_log_events = delete undelete expunge copy mailbox_delete 
mailbox_rename
   mail_log_fields = uid, box, msgid, from, subject, size, vsize, flags
   push_notification_driver = dlog

   sieve = file:~/sieve;active=~/sieve/.dovecot.sieve
   #sieve = ~/.dovecot.sieve
   sieve_duplicate_default_period = 1h
   sieve_duplicate_max_period = 1h
   sieve_extensions = +duplicate +notify +imapflags +vacation-seconds
   sieve_global_dir = /usr/local/etc/dovecot/sieve
   sieve_before = /usr/local/etc/dovecot/sieve/duplicates.sieve


   mail_replica = tcp:10.221.0.19:12345
   #mail_replica = remote:vmail at 10.221.0.19
   #replication_sync_timeout = 2

   fts = lucene
   fts_lucene = whitespace_chars=@.
         fts_autoindex = yes
         fts_languages = en
}

#sieve_extensions = vnd.dovecot.duplicate

#sieve_plugins = vnd.dovecot.duplicate

service anvil {
   process_limit = 1
   client_limit=5000
   vsz_limit = 512m
   unix_listener anvil {
     group = vmail
     mode = 0666
   }
}

service indexer-worker {
         vsz_limit = 2g
}



service auth {
    process_limit = 1
    client_limit=5000
    vsz_limit = 1g

    unix_listener auth-userdb {
       mode = 0660
       user = vmail
       group = vmail
    }
    unix_listener /var/spool/postfix/private/auth {
       mode = 0666
    }

}

service stats {
   process_limit = 1000
   vsz_limit = 1g
   unix_listener stats-reader {
     group = vmail
     mode = 0666
   }
   unix_listener stats-writer {
     group = vmail
     mode = 0666
   }
}
userdb {
   args = /usr/local/etc/dovecot/dovecot-pgsql.conf
   driver = sql

}

protocol imap {
   mail_max_userip_connections = 50
   mail_plugins = $mail_plugins notify replication
}

protocol pop3 {
   mail_max_userip_connections = 50
   mail_plugins = $mail_plugins notify replication
}

protocol imaps {
   mail_max_userip_connections = 25
   mail_plugins = $mail_plugins notify replication
}

protocol pop3s {
   mail_max_userip_connections = 25
   mail_plugins = $mail_plugins notify replication
}


service managesieve-login {
   process_limit = 1000
   vsz_limit = 1g
   inet_listener sieve {
     port = 4190
   }
}

verbose_proctitle = yes

replication_max_conns = 100

replication_full_sync_interval = 1d

service replicator {
   client_limit = 0
   drop_priv_before_exec = no
   idle_kill = 4294967295s
   process_limit = 1
   process_min_avail = 0
   service_count = 0
   vsz_limit = 8g
         unix_listener replicator-doveadm {
     mode = 0600
     user = vmail
   }
   vsz_limit = 8192M
}


service aggregator {
   process_limit = 1000
   #vsz_limit = 1g
   fifo_listener replication-notify-fifo {
     user = vmail
     group = vmail
     mode = 0666
   }

}

service pop3-login {
   process_limit = 1000
   client_limit = 100
   vsz_limit = 512m
  }


service imap-urlauth-login {
   process_limit = 1000
   client_limit = 1000
   vsz_limit = 1g
}


service imap-login {
   process_limit=1000
   client_limit = 1000
   vsz_limit = 1g
  }


protocol sieve {
   managesieve_implementation_string = Dovecot Pigeonhole
   managesieve_max_line_length = 65536
}




#Addition ssl config
!include sni.conf

with sni cert support (examples)

# cat sni.conf
#sni.conf
ssl = yes
verbose_ssl = yes
ssl_dh =</usr/local/etc/dovecot/dh-4096.pem
ssl_prefer_server_ciphers = yes
#ssl_min_protocol = TLSv1.2

#Default *.scom.ca
ssl_key =</usr/local/etc/dovecot/scom.pem
ssl_cert =</usr/local/etc/dovecot/scom.pem
ssl_ca =</usr/local/etc/dovecot/scom.pem

local_name .scom.ca {
ssl_key =</usr/local/etc/dovecot/scom.pem
ssl_cert =</usr/local/etc/dovecot/scom.pem
ssl_ca =</usr/local/etc/dovecot/scom.pem

}

local_name mail.clancyca.com {
   ssl_key =</usr/local/etc/dovecot/cert/mail.clancyca.com
   ssl_cert =</usr/local/etc/dovecot/cert/mail.clancyca.com
   ssl_ca =</usr/local/etc/dovecot/cert/mail.clancyca.com
}

local_name mail.paulkudla.net {
   ssl_key =</usr/local/etc/dovecot/cert/mail.paulkudla.net
   ssl_cert =</usr/local/etc/dovecot/cert/mail.paulkudla.net
   ssl_ca =</usr/local/etc/dovecot/cert/mail.paulkudla.net
}

local_name mail.ekst.ca {
   ssl_key =</usr/local/etc/dovecot/cert/mail.ekst.ca
   ssl_cert =</usr/local/etc/dovecot/cert/mail.ekst.ca
   ssl_ca =</usr/local/etc/dovecot/cert/mail.ekst.ca
}

local_name mail.hamletdevelopments.ca {
   ssl_key =</usr/local/etc/dovecot/cert/mail.hamletdevelopments.ca
   ssl_cert =</usr/local/etc/dovecot/cert/mail.hamletdevelopments.ca
   ssl_ca =</usr/local/etc/dovecot/cert/mail.hamletdevelopments.ca
}

pg sql support supporting replication

# cat dovecot-pgsql.conf
driver = pgsql
connect = host=localhost port=5433 dbname=scom_billing user=pgsql
passworddefault_pass_scheme = PLAIN

password_query = SELECT username as user, password FROM email_users 
WHERE username = '%u' and password <> 'alias' and status =
True and
destination = '%u'

user_query = SELECT home, uid, gid FROM email_users WHERE username = 
'%u' and password <> 'alias' and status = True and
destination = '%u'

#iterate_query = SELECT user, password FROM email_users WHERE username = 
'%u' and password <> 'alias' and status = True and
destination = '%u'

iterate_query = SELECT "username" as user, domain FROM email_users
WHERE
status = True and alias_flag = False





Happy Friday !!!
Thanks - paul

Paul Kudla


Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3

Toronto 416.642.7266
Main?1.866.411.7266
Fax?1.888.892.7266
Email?paul at scom.ca

On 1/6/2023 5:32 AM, Gerben Wierda wrote:> 
>> On 6 Jan 2023, at 08:53, Aki Tuomi <aki.tuomi at open-xchange.com 
>> <mailto:aki.tuomi at open-xchange.com>> wrote:
>>
>>
>>
>> On January 6, 2023 3:56:39 AM GMT+02:00, Gerben Wierda 
>> <gerben.wierda at rna.nl <mailto:gerben.wierda at rna.nl>>
wrote:
>>> One step further in my quest to create a replacement mail server.
>>>
>>> I now have my old mail server (2.3.19.1, macOS + MacPorts) and my
new
>>> (2.3.20, Alpine Linux, Docker, apk package). When I turn on 
>>> replication it works, but, after a while I see:
>>>
>>> Jan 06 00:50:31 replicator: Panic: data stack: Out of memory when 
>>> allocating 268435496 bytes
>>> Jan 06 00:50:32 replicator: Fatal: master: service(replicator):
child
>>> 133 killed with signal 6 (core dumped)
>>> Jan 06 00:50:32 lmtp(pid 195 user sysbh): Warning: 
>>> replication(sysbh): Sync failure:
>>> Jan 06 00:50:32 lmtp(pid 195 user sysbh): Warning: 
>>> replication(sysbh): Remote sent invalid input: -
>>>
>>> I've removed synchronous operation for now (found a message on
the
>>> net suggesting that) but is this known and what does it mean?
>>>
>>> Gerben Wierda (LinkedIn
<https://www.linkedin.com/in/gerbenwierda
>>> <https://www.linkedin.com/in/gerbenwierda>>)
>>> R&A IT Strategy <https://ea.rna.nl/
<https://ea.rna.nl/>> (main site)
>>> Book: Chess and the Art of Enterprise?Architecture 
>>> <https://ea.rna.nl/the-book/
<https://ea.rna.nl/the-book/>>
>>> Book: Mastering ArchiMate
<https://ea.rna.nl/the-book-edition-iii/
>>> <https://ea.rna.nl/the-book-edition-iii/>>
>>>
>>
>> Dovecot default memory limit is 256M. You should probably set
>>
>> service replicator {
>> ?vsz_limit = 2G
>> }
>>
>> because replicator might have to use more memory, especially for 
>> larger indexes.
>>
>> Aki
> 
> That is a good tip as well.
> 
> I had followed this bit of experience from someone else: 
> https://marc.info/?l=dovecot&m=164438199727640 
> <https://marc.info/?l=dovecot&m=164438199727640>, haven't
seen any err
> message since. But that might be because they are in sync now and both 
> sides are aware. Can I trigger full replication again so I can test?
> 
> Gerben
> 
> -- 
> This message has been scanned for viruses and
> dangerous content by *MailScanner* <http://www.mailscanner.info/>,
and is
> believed to be clean.

dovecot - Jan 2023 - replicator: Panic: data stack: Out of memory when allocating 268435496 bytes

replicator: Panic: data stack: Out of memory when allocating 268435496 bytes

replicator: Panic: data stack: Out of memory when allocating 268435496 bytes