CentOS - Feb 2008 - Huge mailq

Where should we start on preventing this type of problem?

[root at server34 mqueue]# find | wc -l
185259

-jason

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
-                                                               -
- Jason Pyeron                      PD Inc. http://www.pdinc.us -
- Principal Consultant              10 West 24th Street #100    -
- +1 (443) 269-1555 x333            Baltimore, Maryland 21218   -
-                                                               -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

This message is for the designated recipient only and may contain
privileged, proprietary, or otherwise private information. If you
have received it in error, purge the message from your system and
notify the sender immediately.  Any other use of the email by you
is prohibited.

On Thu, Feb 21, 2008 at 9:05 AM, Jason Pyeron <jpyeron at pdinc.us>
wrote:
> Where should we start on preventing this type of problem?
>
> [root at server34 mqueue]# find | wc -l
> 185259
>
Short answer: We can't.

Long answer: Need to know a lot more about the problem before one can
even try to fix a problem like this:

1) What are the emails from? Single program? Single user? Multiple issues etc
2) Where are the emails coming from?
3) Who are the emails are going to?
4) What kind of network setting is it.
5) What kind of server is this? [Mail server, mail gateway, personal
desktop, etc]

These are just starting questions..


-- 
Stephen J Smoogen. -- CSIRT/Linux System Administrator
How far that little candle throws his beams! So shines a good deed
in a naughty world. = Shakespeare. "The Merchant of Venice"

Let's start with an oops, that messages was sent on Thursday.

The queue spiked at 320k messages, then we discovered a name server was
down, causing an nfs authentication problem, which caused homedirs to not be
deliverable, causing the queue to reach DOS levels, causing the spamfilter
to switch off, causing mail not to be delivered...
 
With a little careful loading (beefed up the VM config) we were able to
deliver all but 25k in 17 hours. I guess this request was one of the 25k.
> -----Original Message-----
> From: lists-centos 
> [mailto:replies-lists-a1z2-centos at listmail.innovate.net] 
> Sent: Monday, February 25, 2008 20:20
> To: Jason Pyeron
> Subject: Re: [CentOS] Huge mailq
> 
> you should start by looking at your maillog to see what's going on.
> you should also check to see if you have a sendmail/postfix (whatever
> your mta - i'm assuming sendmail because postfix holds its outbound
> queue differently) queue process running.
> 
sendmail
> entries in the maillog will help you figure out if legit mail is
> backing up, and hints as to why, or if you're a spam
"gateway". if
> the latter, you need to look into the source. e.g., are you an open
> relay or do you have web pages that spammers are using.
> 
> it's possible that you have a lot of orphaned df* (data) files, but a
> rather large number.
> 
> this is just a start ...
> 
> [by the way, you don't need a "find" to feed to a
"wc". a "wc -l qf*"
> in the mqueue directory would do the trick.]
> 
Too many thingies on the cmd line error.
>   - Rick
> 
> 
> 
> ------------ Original Message ------------
> > Date: Thursday, February 21, 2008 11:05:45 AM -0500
> > From: Jason Pyeron <jpyeron at pdinc.us>
> > To: 'CentOS mailing list' <centos at centos.org>
> > Subject: [CentOS] Huge mailq
> > 
> > Where should we start on preventing this type of problem?
> > 
> > [root at server34 mqueue]# find | wc -l
> > 185259
> > 
> > -jason
> > 
> ---------- End Original Message ----------
> 


-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
-                                                               -
- Jason Pyeron                      PD Inc. http://www.pdinc.us -
- Principal Consultant              10 West 24th Street #100    -
- +1 (443) 269-1555 x333            Baltimore, Maryland 21218   -
-                                                               -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

This message is for the designated recipient only and may contain
privileged, proprietary, or otherwise private information. If you
have received it in error, purge the message from your system and
notify the sender immediately.  Any other use of the email by you
is prohibited.

Jason Pyeron wrote:
> Where should we start on preventing this type of problem?
>
> [root at server34 mqueue]# find | wc -l
> 185259
/etc/init.d/sendmail stop
chkconfig --level 2345 sendmail off
find /var/spool/mqueue -type f -exec rm -f {} \;

That'll empty out your queue and you won't have to worry about
it building up again, pesky thing!

:)

nate