Aki Tuomi
2023-Jan-06 07:53 UTC
Re: replicator: Panic: data stack: Out of memory when allocating 268435496 bytes
On January 6, 2023 3:56:39 AM GMT+02:00, Gerben Wierda <gerben.wierda at rna.nl> wrote:>One step further in my quest to create a replacement mail server. > >I now have my old mail server (2.3.19.1, macOS + MacPorts) and my new (2.3.20, Alpine Linux, Docker, apk package). When I turn on replication it works, but, after a while I see: > >Jan 06 00:50:31 replicator: Panic: data stack: Out of memory when allocating 268435496 bytes >Jan 06 00:50:32 replicator: Fatal: master: service(replicator): child 133 killed with signal 6 (core dumped) >Jan 06 00:50:32 lmtp(pid 195 user sysbh): Warning: replication(sysbh): Sync failure: >Jan 06 00:50:32 lmtp(pid 195 user sysbh): Warning: replication(sysbh): Remote sent invalid input: - > >I've removed synchronous operation for now (found a message on the net suggesting that) but is this known and what does it mean? > >Gerben Wierda (LinkedIn <https://www.linkedin.com/in/gerbenwierda>) >R&A IT Strategy <https://ea.rna.nl/> (main site) >Book: Chess and the Art of Enterprise?Architecture <https://ea.rna.nl/the-book/> >Book: Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/> >Dovecot default memory limit is 256M. You should probably set service replicator { vsz_limit = 2G } because replicator might have to use more memory, especially for larger indexes. Aki
justina colmena ~biz
2023-Jan-06 10:21 UTC
replicator: Panic: data stack: Out of memory when allocating 268435496 bytes
On Thursday, January 5, 2023 10:53:13 PM AKST Aki Tuomi wrote:> On January 6, 2023 3:56:39 AM GMT+02:00, Gerben Wierda<gerben.wierda at rna.nl> wrote:> >Jan 06 00:50:31 replicator: Panic: data stack: Out of memory when > >allocating 268435496 bytes Jan 06 00:50:32 replicator: Fatal: master: > ... > service replicator { > vsz_limit = 2G > } > > because replicator might have to use more memory, especially for larger > indexes. > > AkiThat's probably as good a short-term fix as any, but a longer term fix will probably require effectively "going on a diet," losing weight, cracking down on memory leaks, matching up every malloc() and free() and getting leaner and meaner with the memory allocation and Big-O time & space complexity of algorithms. -- https://justina.abeja.colmena.biz/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: This is a digitally signed message part. URL: <https://dovecot.org/pipermail/dovecot/attachments/20230106/d0f581c7/attachment.sig>
Gerben Wierda
2023-Jan-06 10:32 UTC
replicator: Panic: data stack: Out of memory when allocating 268435496 bytes
> On 6 Jan 2023, at 08:53, Aki Tuomi <aki.tuomi at open-xchange.com> wrote: > > > > On January 6, 2023 3:56:39 AM GMT+02:00, Gerben Wierda <gerben.wierda at rna.nl> wrote: >> One step further in my quest to create a replacement mail server. >> >> I now have my old mail server (2.3.19.1, macOS + MacPorts) and my new (2.3.20, Alpine Linux, Docker, apk package). When I turn on replication it works, but, after a while I see: >> >> Jan 06 00:50:31 replicator: Panic: data stack: Out of memory when allocating 268435496 bytes >> Jan 06 00:50:32 replicator: Fatal: master: service(replicator): child 133 killed with signal 6 (core dumped) >> Jan 06 00:50:32 lmtp(pid 195 user sysbh): Warning: replication(sysbh): Sync failure: >> Jan 06 00:50:32 lmtp(pid 195 user sysbh): Warning: replication(sysbh): Remote sent invalid input: - >> >> I've removed synchronous operation for now (found a message on the net suggesting that) but is this known and what does it mean? >> >> Gerben Wierda (LinkedIn <https://www.linkedin.com/in/gerbenwierda>) >> R&A IT Strategy <https://ea.rna.nl/> (main site) >> Book: Chess and the Art of Enterprise Architecture <https://ea.rna.nl/the-book/> >> Book: Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/> >> > > Dovecot default memory limit is 256M. You should probably set > > service replicator { > vsz_limit = 2G > } > > because replicator might have to use more memory, especially for larger indexes. > > AkiThat is a good tip as well. I had followed this bit of experience from someone else: https://marc.info/?l=dovecot&m=164438199727640 <https://marc.info/?l=dovecot&m=164438199727640>, haven't seen any err message since. But that might be because they are in sync now and both sides are aware. Can I trigger full replication again so I can test? Gerben -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20230106/35b28c7c/attachment.htm>
Gerben Wierda
2023-Jan-17 22:45 UTC
replicator: Panic: data stack: Out of memory when allocating 268435496 bytes
> On 6 Jan 2023, at 08:53, Aki Tuomi <aki.tuomi at open-xchange.com> wrote: > > > > On January 6, 2023 3:56:39 AM GMT+02:00, Gerben Wierda <gerben.wierda at rna.nl> wrote: >> One step further in my quest to create a replacement mail server. >> >> I now have my old mail server (2.3.19.1, macOS + MacPorts) and my new (2.3.20, Alpine Linux, Docker, apk package). When I turn on replication it works, but, after a while I see: >> >> Jan 06 00:50:31 replicator: Panic: data stack: Out of memory when allocating 268435496 bytes >> Jan 06 00:50:32 replicator: Fatal: master: service(replicator): child 133 killed with signal 6 (core dumped) >> Jan 06 00:50:32 lmtp(pid 195 user sysbh): Warning: replication(sysbh): Sync failure: >> Jan 06 00:50:32 lmtp(pid 195 user sysbh): Warning: replication(sysbh): Remote sent invalid input: - >> >> I've removed synchronous operation for now (found a message on the net suggesting that) but is this known and what does it mean? >> >> Gerben Wierda (LinkedIn <https://www.linkedin.com/in/gerbenwierda>) >> R&A IT Strategy <https://ea.rna.nl/> (main site) >> Book: Chess and the Art of Enterprise Architecture <https://ea.rna.nl/the-book/> >> Book: Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/> >> > > Dovecot default memory limit is 256M. You should probably set > > service replicator { > vsz_limit = 2G > } > > because replicator might have to use more memory, especially for larger indexes. > > AkiComing back to this one. doveadm config says everything is vsz_limit = 18446744073709551615 B Isn't that more than 2G already (nd certainly more than 256M?)? I also tried putting default_vsz_limit = 2G in dovecot.conf but that doesn't change anything either. What am I missing? -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20230117/a4cdce79/attachment-0001.htm>