Shorewall users - Aug 2004 - Dropped/Corrupted Log Entries?

Hello!

We''re seeing a problem that I think is more of a Netfilter or kernel
issue than anything to do with Shorewall, but I thought I would ask
about it here, too, to see if anyone has suggestions. What we''re seeing
is that, when we do a very fast port scan of a system on the other side
of a firewall with Shorewall installed and various common rules, a
majority of the denied connection attempts are not showing up in the
logs even though they should, and depending on circumstances and
versions, anywhere from 1% to 100% of the accepted connections are also
not being logged.

At the same time, we also notice that some corrupted Netfilter log
entries are appearing in our default system log, each typically the last
3/4 or so of a legtimate-looking log entry. There are only a few of
these -- they don''t make up the balance of the missing entires
mentioned
above -- but it does indicate that the scan is overwhelming the ability
of the system to correctly manage log data.

We tried rebuilding the 2.4.26 kernel with a larger value for
CONFIG_LOG_BUF_SHIFT, but this did not seem to have an effect. Has
anyone else seen anything like this? Do you have any suggestions for how
we might tune the system to get more complete logs? I suppose there is a
trade-off here between completeness and DoS, but we can tune that with
explicit log rate limiting. We''re more concerned now about the apparent
intrinsic limits, and the fact that the kernel seems to be broken to the
degree that corrupted entries are written.

Details of the configuration below.

Thanks!

Tim

------------------------------------

We''ve tested this on two different systems to see if there has been a
change over time:

    Linux 2.4.21, Shorewall 1.4.6b
    Linux 2.4.26, Shorewall 2.0.6

and find that the number of events sucessfully recorded in the logs are
about the same, though curiously, with the older system, there seems to
be a bias toward recording the accepted connections. There also seems to
be a relationship between the number of accepted connections logged and
the total number of Netfilter rules on the system (more rules, fewer
connections logged), but this has not been extensively tested. We do the
scan from a host on "loc" to a host on "net".

The shorewall.conf file has:

    LOGRATE    LOGBURST
The rules file has:

    ACCEPT   net                    $FW           tcp     ssh       -
    ACCEPT:info dmz                 net           tcp
telnet,ftp,http,https,smtp -
    ACCEPT:info dmz                 net           tcp     domain,pop3,imap -
    ACCEPT   dmz                    net           udp     domain    -
    ACCEPT:info net                 loc           tcp
ftp,http,https,smtp -
    ACCEPT:info net                 loc           tcp     domain,pop3,imap -
    ACCEPT:info net                 loc           udp     domain    -
    ACCEPT:info net                 dmz           tcp
ftp,http,https,smtp -
    ACCEPT:info net                 dmz           tcp     domain,pop3,imap -
    ACCEPT:info net                 dmz           udp     domain    -
    ACCEPT:info loc                 net           tcp
telnet,ftp,http,https,smtp -
    ACCEPT:info loc                 net           tcp     domain,pop3,smtp -
    ACCEPT:info loc                 net           udp     domain    -

The policy file has:

    $FW             all             ACCEPT          -
    loc             dmz             ACCEPT          -
    net             net             DROP            info
    net             all             DROP            info
    all             all             REJECT          info

Tim Burress wrote:
> Hello!
> 
> We''re seeing a problem that I think is more of a Netfilter or
kernel
> issue than anything to do with Shorewall, but I thought I would ask
> about it here, too, to see if anyone has suggestions. What we''re
seeing
> is that, when we do a very fast port scan of a system on the other side
> of a firewall with Shorewall installed and various common rules, a
> majority of the denied connection attempts are not showing up in the
> logs even though they should, and depending on circumstances and
> versions, anywhere from 1% to 100% of the accepted connections are also
> not being logged.
> 
I''ve made the point many times before that netfilter logging is not a 
facility for maintaining an audit trail of all connection attempts 
because it will drop messages if subjected to a high enough load. You 
can try adding "-" before the log name in those /etc/syslog.conf logs 
where netfilter messages are being directed; that will speed up writing 
those messages to disk and may provide some relief.

Also be sure that you are running the latest syslogd/klogd available for 
your distribution.

-Tom
-- 
Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net