You could also add it as setting for the fts_xapian plugin parameters? Aki> On 04/11/2020 08:42 Joan Moreau <jom at grosjo.net> wrote: > > > For machines with low memory, I would like to detect how much ram remains available before starting indexing a mail, so I can commit everything on disk before the ram is exhausted (and break the process) > I tried to put a "fake" allocation to test if it fails, (so it can fail separately, and I can "if ram remaining is above X") but the is really not clean > > > > On 2020-11-04 06:28, Aki Tuomi wrote: > > > > > > > On 04/11/2020 05:19 Joan Moreau <jom at grosjo.net> wrote: > > > > > > > > > Hello > > > I am looking for help around memory management > > > 1 - How to get the current value of "vsz_limit" from inside a plugin (namely https://github.com/grosjo/fts-xapian/ ) , especially for indexer-worker > > > 2 - Is there a macro or function in dovecot to get the remaining free memory from this vsz value ? > > > Thank you > > > > > > > > > > Hi Joan, > > > > I don't think there is a feasible way to access this setting as of now. Is there a reason you need this? We usually recommend setting vsz_limit=0 for indexer-worker. > > > > Aki > >
yes, will do so. It would be nice however to be able to access the actual dovecot config from the plugin side On 2020-11-04 06:46, Aki Tuomi wrote:> You could also add it as setting for the fts_xapian plugin parameters? > > Aki > > On 04/11/2020 08:42 Joan Moreau <jom at grosjo.net> wrote: > > For machines with low memory, I would like to detect how much ram > remains available before starting indexing a mail, so I can commit > everything on disk before the ram is exhausted (and break the process) > I tried to put a "fake" allocation to test if it fails, (so it can fail > separately, and I can "if ram remaining is above X") but the is really > not clean > > On 2020-11-04 06:28, Aki Tuomi wrote: > > On 04/11/2020 05:19 Joan Moreau <jom at grosjo.net> wrote: > > Hello > I am looking for help around memory management > 1 - How to get the current value of "vsz_limit" from inside a plugin > (namely https://github.com/grosjo/fts-xapian/ ) , especially for > indexer-worker > 2 - Is there a macro or function in dovecot to get the remaining free > memory from this vsz value ? > Thank you > > Hi Joan, > > I don't think there is a feasible way to access this setting as of now. > Is there a reason you need this? We usually recommend setting > vsz_limit=0 for indexer-worker. > > Aki-------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20201106/ceaf8ebb/attachment.html>
Seems I had forgotten that you can use src/lib/restrict.h, in particular, restrict_get_process_size() to figure out the limit. You can combine this with getrusage to find out current usage. Aki> On 06/11/2020 13:26 Joan Moreau <jom at grosjo.net> wrote: > > > yes, will do so. > It would be nice however to be able to access the actual dovecot config from the plugin side > > > > On 2020-11-04 06:46, Aki Tuomi wrote: > > You could also add it as setting for the fts_xapian plugin parameters? > > > > Aki > > > > > > > On 04/11/2020 08:42 Joan Moreau <jom at grosjo.net> wrote: > > > > > > > > > For machines with low memory, I would like to detect how much ram remains available before starting indexing a mail, so I can commit everything on disk before the ram is exhausted (and break the process) > > > I tried to put a "fake" allocation to test if it fails, (so it can fail separately, and I can "if ram remaining is above X") but the is really not clean > > > > > > > > > > > > On 2020-11-04 06:28, Aki Tuomi wrote: > > > > > > > > > > > > > > > > > On 04/11/2020 05:19 Joan Moreau <jom at grosjo.net> wrote: > > > > > > > > > > > > > > > Hello > > > > > I am looking for help around memory management > > > > > 1 - How to get the current value of "vsz_limit" from inside a plugin (namely https://github.com/grosjo/fts-xapian/ ) , especially for indexer-worker > > > > > 2 - Is there a macro or function in dovecot to get the remaining free memory from this vsz value ? > > > > > Thank you > > > > > > > > > > > > > > > > > > Hi Joan, > > > > > > > > I don't think there is a feasible way to access this setting as of now. Is there a reason you need this? We usually recommend setting vsz_limit=0 for indexer-worker. > > > > > > > > Aki > > > >