dovecot - Jan 2023 - How do you scale dovecot for good performance with Roundcube webmailer in front? (hitting limits without exhausting resources)

Good day,

I am currently setting up/debugging a webmailer-only setup using Roundcube
(latest version) with Dovecot (2.3.20, latest as of now).

Generally speaking the setup performs OK, I wrote a JMeter Roundcube script to
test the Roundcube. This includes actions like listing mails and fetching them.
I am hitting a somehow dubious limit of 600 req/s though. (This includes all
kinds of HTTP calls to Roundcube, not only the ones triggering IMAP). When the
dovecot becomes unavailable, the performance without mails jumps to 7700 req/s.
Testing is done with 4 JMeter-servers and one client.?

My setup is a Dovecot with MySQL (Percona XtraDB) backend. Mails are encrypted
with mail_crypt and EC keys. Authentication is done in parallel
(auth_cache_verify_with_workers=yes). Where possible min_available_processes
have been set equal to the threads available on the Dovecot machine (64).

Hardware is a 64 thread Xeon CPU at 2.10 GHz, 96 GB RAM, SSDs as backing storage
IOPS read 40000/write 13000.

What settings do you recommend and how was your experience with Roundcube and
its performance in general (what should be possible with that kind of HW?)?

Best regards and thanks in advance!

????
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20230126/6d3a8592/attachment.htm>

are you running (squirrelmail's) imapproxy on the roundcube machine? it 
keeps user imap connections active to dovecot, meaning that roundcube 
doesn't have to constantly log back in via imap for each operation.

that might help somewhat.

(we just use the debian package version of imapproxy - it seems the 
www.imapproxy.org website is down right now)

On 2023-01-26 10:18, Malte Schmidt wrote:
> Good day,
> 
> I am currently setting up/debugging a webmailer-only setup using 
> Roundcube (latest version) with Dovecot (2.3.20, latest as of now).
> 
> Generally speaking the setup performs OK, I wrote a JMeter Roundcube 
> script to test the Roundcube. This includes actions like listing mails 
> and fetching them. I am hitting a somehow dubious limit of 600 req/s 
> though. (This includes all kinds of HTTP calls to Roundcube, not only 
> the ones triggering IMAP). When the dovecot becomes unavailable, the 
> performance without mails jumps to 7700 req/s. Testing is done with 4 
> JMeter-servers and one client.
> 
> My setup is a Dovecot with MySQL (Percona XtraDB) backend. Mails are 
> encrypted with mail_crypt and EC keys. Authentication is done in 
> parallel (auth_cache_verify_with_workers=yes). Where possible 
> min_available_processes have been set equal to the threads available on 
> the Dovecot machine (64).
> 
> Hardware is a 64 thread Xeon CPU at 2.10 GHz, 96 GB RAM, SSDs as backing 
> storage IOPS read 40000/write 13000.
> 
> What settings do you recommend and how was your experience with 
> Roundcube and its performance in general (what should be possible with 
> that kind of HW?)?
> 
> Best regards and thanks in advance!
> 
> ????

> 
> Generally speaking the setup performs OK, I wrote a JMeter Roundcube script
to
> test the Roundcube. This includes actions like listing mails and fetching
> them. I am hitting a somehow dubious limit of 600 req/s though. (This
includes
> all kinds of HTTP calls to Roundcube, not only the ones triggering IMAP).
When
> the dovecot becomes unavailable, the performance without mails jumps to
7700
> req/s. Testing is done with 4 JMeter-servers and one client.
> 
Should you not be testing also dovecot performance directly? So you can see what
is it's maximum. I can't imagine http interfaces are the bottle neck.