dovecot - May 2009 - Possibly dumb questions about DC and user/system limits

Because of user access growth, the number of processes associated with 
IMAP has increased and I thought to change some of the configured 
parms.  dovecot -n yields:

# 1.1.14: /usr/local/etc/dovecot.conf
# OS: AIX 1 005A928C4C00  
listen: *:143
ssl_listen: *:993
disable_plaintext_auth: no
verbose_ssl: yes
login_dir: /var/run/dovecot/login
login_executable: /usr/local/libexec/dovecot/imap-login
login_processes_count: 12
login_max_processes_count: 774
max_mail_processes: 1280
mail_max_userip_connections: 12
verbose_proctitle: yes
first_valid_uid: 200
mail_location: mbox:~/mail:INBOX=/var/spool/mail/%u:INDEX=/var/dcindx/%u
mbox_write_locks: fcntl
mbox_dirty_syncs: no
auth default:
  passdb:
    driver: pam
  userdb:
    driver: passwd

The only changes was that max_mail_processes went from 1024 to 1280.  
Now I get a error message when I start DC:

Warning: fd limit 2000 is lower than what Dovecot can use under full load (more 
than 2054). Either grow the limit or change login_max_processes_count and 
max_mail_processes settings

So I changed the no_size and no_size_hard to 3500 and 4000 respectively in both
dovecot and root

AIX defines nosize:* *Sets the soft limit for the number of file descriptors a
user process may have open at one time.

an lsuser dovecot returns:

dovecot id=417 pgrp=dovecot groups=dovecot shell=/bin/false daemon=true
admin=false ... fsize=2097151 cpu=-1 data=262144 stack=65536 core=2097151
rss=65536 nofiles=3500 nofiles_hard=4000

I kill dovecot and all children and restart it....same error message

What am I missing?  Does the machine have to be rebooted for the no_size 
to be updated?


-- 
==== Once upon a time, the Internet was a friendly, 
neighbors-helping-neighbors small town, and no one locked their doors. 
Now it's like an apartment in Bed-Stuy: you need three heavy duty 
pick-proof locks, one of those braces that goes from the lock to the 
floor, and bars on the windows.... ==== Stewart Dean, Unix System Admin, 
Bard College, New York 12504 sdean at bard.edu voice: 845-758-7475, fax: 
845-758-7035

On Thu, 2009-05-14 at 14:17 -0400, Stewart Dean wrote:
> Warning: fd limit 2000 is lower than what Dovecot can use under full load
(more
> than 2054). Either grow the limit or change login_max_processes_count and 
> max_mail_processes settings
..
> I kill dovecot and all children and restart it....same error message
> 
> What am I missing?  Does the machine have to be rebooted for the no_size 
> to be updated?
I've no idea about AIX, but it sounds similar to how Linux distros set
them up. They probably affect only the logging in users' processes and
not services that start up.

Anyway you probably still have "ulimit" command in AIX shells too? So
before starting up Dovecot, make sure "ulimit -n" returns enough
files.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20090514/30ab7b30/attachment-0002.bin>

On May 14, 2009, at 2:17 PM, Stewart Dean wrote:
>
> Warning: fd limit 2000 is lower than what Dovecot can use under full  
> load (more than 2054). Either grow the limit or change  
> login_max_processes_count and max_mail_processes settings
>
> So I changed the no_size and no_size_hard to 3500 and 4000  
> respectively in both dovecot and root
>
> AIX defines nosize:* *Sets the soft limit for the number of file  
> descriptors a user process may have open at one time.
>
> an lsuser dovecot returns:
>
> dovecot id=417 pgrp=dovecot groups=dovecot shell=/bin/false  
> daemon=true admin=false ... fsize=2097151 cpu=-1 data=262144  
> stack=65536 core=2097151 rss=65536 nofiles=3500 nofiles_hard=4000
> I kill dovecot and all children and restart it....same error message
>
> What am I missing?  Does the machine have to be rebooted for the  
> no_size to be updated?
>
AIX should not have to be rebooted for this to happen. For debug, I  
would change the shell to a real shell; change the fsize to 600 or  
something small; logout and then login(as root and then su or just as  
the user) and try to make large files and verify limits. Ping me off  
list for more debugging unless people consider this on topic.

-Jonathan


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2541 bytes
Desc: not available
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20090515/38cc1f47/attachment-0002.bin>

On May 14, 2009, at 2:17 PM, Stewart Dean wrote:
>
>
> The only changes was that max_mail_processes went from 1024 to  
> 1280.  Now I get a error message when I start DC:
>
> Warning: fd limit 2000 is lower than what Dovecot can use under full  
> load (more than 2054). Either grow the limit or change  
> login_max_processes_count and max_mail_processes settings
>
> So I changed the no_size and no_size_hard to 3500 and 4000  
> respectively in both dovecot and root
>
> AIX defines nosize:* *Sets the soft limit for the number of file  
> descriptors a user process may have open at one time.
>
I found a way to recreate the problem on my side. Since I start  
dovecot from inetd, I must verify my shell has the proper ulimits then  
do stopsrc -s inetd;startsrc -s inetd to pick up the new ulimits. I  
didn't notice how you started dovecot. Maybe this helps.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2541 bytes
Desc: not available
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20090515/48f6d468/attachment-0002.bin>

Maybe it's a D'uh for everyone else, but the reason I wasn't getting
anywhere when I increased the no_size and no_size_hard (and confirmed 
that it had changed using the lsuser cmd) was that a) it needed to be 
done against root, not dovecot and b) I had to exit root and re-login 
for that to be part of root environment instance.  Indeed, before going 
out and coming back, ulimit -n did still show 2000.  So lsuser shows the 
configuration and ulimit shows the current environment.  All better 
until the next time.

I was also interested to learn, while I was looking under rocks here and 
there, that in AIX with a 64 bit kernel there is total number of files 
that can be in use by all users of ~3,355,236.  Wouldn't it be nasty to 
bump into that!

Jonathan Siegle wrote:
>
> On May 14, 2009, at 2:17 PM, Stewart Dean wrote:
>
>>
>>
>> The only changes was that max_mail_processes went from 1024 to 1280.  
>> Now I get a error message when I start DC:
>>
>> Warning: fd limit 2000 is lower than what Dovecot can use under full 
>> load (more than 2054). Either grow the limit or change 
>> login_max_processes_count and max_mail_processes settings
>>
>> So I changed the no_size and no_size_hard to 3500 and 4000 
>> respectively in both dovecot and root
>>
>> AIX defines nosize:* *Sets the soft limit for the number of file 
>> descriptors a user process may have open at one time.
>>
>
> I found a way to recreate the problem on my side. Since I start 
> dovecot from inetd, I must verify my shell has the proper ulimits then 
> do stopsrc -s inetd;startsrc -s inetd to pick up the new ulimits. I 
> didn't notice how you started dovecot. Maybe this helps.
>
>
-- 
==== Once upon a time, the Internet was a friendly, 
neighbors-helping-neighbors small town, and no one locked their doors. 
Now it's like an apartment in Bed-Stuy: you need three heavy duty 
pick-proof locks, one of those braces that goes from the lock to the 
floor, and bars on the windows.... ==== Stewart Dean, Unix System Admin, 
Bard College, New York 12504 sdean at bard.edu voice: 845-758-7475, fax: 
845-758-7035