First, a solution to the problem is being posted herein, so I will not make
an
attempt to document the entire configuration, but rather only the important
relevent portions of the configuration - unless someone requests more to
test.
Second, a disclaimer. I am using:
$ sudo shorewall version
3.0.4
I can''t help this. I am using Mandriva Corporate Server 4.0 expressly
because
it allows me to keep a stable system with a 5-year security update plan.
They
rarely update packages until it becomes too hard to backport security
patches.
Third, even though this is an old version, I would like to ask this question
in case it is relevant to newer versions, because resolving this has taken a
very long time.
Scenario:
- The server needs to be able to use HTTPS to contact the net zone to
perform
an important maintenance function.
- "ULOG" is used as the firewall log level to keep firewall messages
from
cluttering the system logs, and, use of "info" does not change the
results
(but does identify yet another bug).
- Some traffic is always logged even if permitted because volume should be
low
and because it helps an administrator get a feel for what the users may be
doing that might be a big higher risk.
Problem:
- The rules file is correctly configured to allow $FW to net tcp traffic on
port 443 (HTTPS).
- Attempts to use https from $FW to net fail WITH NO LOG EVENTS, but the log
does work because other log/drop/reject events do show up. "Identically
configured" port 80 (HTTP) allow from $FW to net does not fail.
- The relevant parts of the configuration are:
policy
loc all CONTINUE
net all CONTINUE
$FW all REJECT $LOG
rules
LOG:$LOG:HTTPSout $FW all tcp 443 - - - -
LOG:$LOG:WEBout $FW all tcp 80 - - - -
ACCEPT $FW all tcp 80 - - - :root
ACCEPT $FW net tcp 443 - - - :root
Example:
lynx http://www.mandriva.com
Result: Success
lynx: https://download.mandriva.com
Result: Failed, no log entries, except HTTPSout LOG messages.
Diagnostics:
$ shorewall restart
...
WARNING: Log Prefix shortened to "Shorewall:fw2net:LOG:HTTPSout"
Rule "LOG:ULOG:HTTPSout fw net tcp 443 - - - -" added.
...
Rule "ACCEPT fw net tcp 80 - - - :root" added.
...
Rule "ACCEPT fw net tcp 443 - - - :root" added.
...
Activating Rules...
Processing /etc/shorewall/start ...
Shorewall Restarted
Processing /etc/shorewall/started ...
An excerpt from the log shows:
ULOG style log entry
Aug 17 10:05:06 dnd Shorewall:fw2net:LOG:HTTPSout IN= OUT=eth0
MACSRC=192.168.128.7 DST=212.85.147.126 LEN=60 TOS=00 PREC=0x00 TTL=64 ID=9834
DF PROTO=TCP SPT=51040 DPT=443 SEQ=2207294271 ACK=0 WINDOW=5840 SYN URGP=0
info style log entry
Aug 17 11:47:42 dnd kernel: Shorewall:fw2net:LOG:HTTPSoutIN= OUT=eth0
SRC=192.168.128.7 DST=212.85.147.126 LEN=60 TOS=0x00 PREC=0x00 TTL=64
ID=59981 DF PROTO=TCP SPT=52385 DPT=443 WINDOW=5840 RES=0x00 SYN URGP=0
No other log entries exist in this time frame, yet the traffic is
blocked. NOTE that there is an anomaly in the "info" log entry at
"Shorewall:fw2net:LOG:HTTPSoutIN=". There should be a space
between the tag and the IN= field.
Resolution:
Spotted reference in "Problems Corrected in 3.4.1" to:
3) Log messages specifying a log tag had two spaces appended to the
log prefix. This could cause mysterious "log-prefix truncated"
messages.
I had been ignoring the above warning because it didn''t make sense,
and
anyway, I didn''t care that a log prefix was shortened as long as it
appeared in the log, and the logging was working fine.
I decided to take the troubleshooter stance that if something is broken,
you view any anomaly, no matter how innocuous, as a potential indicator
for the problem. I renamed the log rule for HTTPS as follows:
LOG:$LOG:443out $FW all tcp 443 - - - -
With this change, no warnings about Log Prefix truncation is emitted on
a shorewall restart.
HTTPS traffic from fw to net now works correctly.
Summary:
This is a flaw. I cannot say yet whether this appears in later versions
of shorewall, but I also do not see any indications that such an issue has
been fixed, though my search may not have been conclusive.
The firewall should not block any traffic without logging the block if
it has been configured to log all drops and rejects!!!
Reconfiguring the logging to "info" instead of "ULOG" does
not affect this
issue. It occurs no matter what logger is used.
Also note that the "info" style log entry is flawed. A space should
be
found between the log tag and the IN= field.
---
Kevin R. Bulgrien
Design and Development Engineer
VertexRSI
http://www.gdsatcom.com/
General Dynamics SATCOM Technologies Tel: 903-295-1480
x24109
1915 Harrison Road 903-381-4109
Longview, TX 75604-5438 Fax: 903-295-1479
This email message is for the sole use of the intended recipient(s) and may
contain General Dynamics SATCOM Technologies confidential or privileged
information. Any unauthorized review, use, disclosure or distribution is
prohibited. If you are not an intended recipient, please contact the sender by
reply email and destroy all copies of the original message.
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
Kevin, Bulgrien, Kevin wrote:> - The relevant parts of the configuration are: > > policy > > loc all CONTINUE > net all CONTINUE > $FW all REJECT $LOG > > rules > > LOG:$LOG:HTTPSout $FW all tcp 443 - - - - > LOG:$LOG:WEBout $FW all tcp 80 - - - - > > ACCEPT $FW all tcp 80 - - - :root > ACCEPT $FW net tcp 443 - - - :root > > Example: > > lynx http://www.mandriva.com > > Result: Success > > lynx: https://download.mandriva.com > > Result: Failed, no log entries, except HTTPSout LOG messages.I need to see the output of "shorewall dump" to see what, if anything, Shorewall did wrong.> > No other log entries exist in this time frame, yet the traffic is > blocked. NOTE that there is an anomaly in the "info" log entry at > "Shorewall:fw2net:LOG:HTTPSoutIN=". There should be a space > between the tag and the IN= field.That anomaly still exists in the current code. It will be corrected in Shorewall 4.0.3.> > Resolution: > > Spotted reference in "Problems Corrected in 3.4.1" to: > > 3) Log messages specifying a log tag had two spaces appended to the > log prefix. This could cause mysterious "log-prefix truncated" > messages. > > I had been ignoring the above warning because it didn''t make sense, and > anyway, I didn''t care that a log prefix was shortened as long as it > appeared in the log, and the logging was working fine. > > I decided to take the troubleshooter stance that if something is broken, > you view any anomaly, no matter how innocuous, as a potential indicator > for the problem. I renamed the log rule for HTTPS as follows: > > LOG:$LOG:443out $FW all tcp 443 - - - - > > With this change, no warnings about Log Prefix truncation is emitted on > a shorewall restart. > > HTTPS traffic from fw to net now works correctly. > > Summary: > > This is a flaw. I cannot say yet whether this appears in later versions > of shorewall, but I also do not see any indications that such an issue has > been fixed, though my search may not have been conclusive.Given the way that shorewall 3.0.4 worked, I find this problem report and the resolution to be a complete mystery. I have confirmed that (other than the missing space anomaly) that the problem doesn''t occur in Shorewall 4.0.3 with either the Perl or Shell compilers. But then, given that I can''t understand how it could have happened in 3.0.4, I''m not sure that there isn''t something else going on in your configuration that causes the problem and might still cause the problem in later releases (even though log rule generation was rewritten in 3.2.0). So if you could re-create the bug then capture the output of "shorewall dump", I will be interested to take a look at it. Thanks, -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
> > Resolution: > > > > Spotted reference in "Problems Corrected in 3.4.1" to: > > > > 3) Log messages specifying a log tag had two spaces appended to the > > log prefix. This could cause mysterious "log-prefix truncated" > > messages. > > > > I had been ignoring the above warning because it didn''t make sense,and> > anyway, I didn''t care that a log prefix was shortened as long as it > > appeared in the log, and the logging was working fine. > > > > I decided to take the troubleshooter stance that if something isbroken,> > you view any anomaly, no matter how innocuous, as a potentialindicator> > for the problem. I renamed the log rule for HTTPS as follows: > > > > LOG:$LOG:443out $FW all tcp 443 - - - - > > > > With this change, no warnings about Log Prefix truncation is emittedon> > a shorewall restart. > > > > HTTPS traffic from fw to net now works correctly. > > > > Summary: > > > > This is a flaw. I cannot say yet whether this appears in laterversions> > of shorewall, but I also do not see any indications that such an issuehas> > been fixed, though my search may not have been conclusive. > > Given the way that shorewall 3.0.4 worked, I find this problem report and > the resolution to be a complete mystery. I have confirmed that (other than > the missing space anomaly) that the problem doesn''t occur in Shorewall4.0.3> with either the Perl or Shell compilers. But then, given that I can''t > understand how it could have happened in 3.0.4, I''m not sure that there > isn''t something else going on in your configuration that causes theproblem> and might still cause the problem in later releases (even though log rule > generation was rewritten in 3.2.0). > > So if you could re-create the bug then capture the output of "shorewall > dump", I will be interested to take a look at it. > > Thanks, > -TomI have hosed myself. I didn''t tar up the configuration once it started working, and I tried to reproduce the problem and now cannot get it back working, so indications are that the solution is a bit more than changing the LOG tag... The sad thing is I was very sure I had only changed/added log items and https rule to $FW all instead of $FW net when it started working. I have not yet figured out how to fix it, so anyway, if nothing else, it is an opportunity to log as a known issue. The attached configuration is broken. Failed attempts are SRC=192.168.128.7 DST=212.85.147.126 To try to make sure it wasn''t a destination fault, after the failure, I did shorewall clear and retried. The https link succeeded, after which I immediately did a start. The https link attempt again fails even with the LOG rule not triggering the truncation warning. --- Kevin R. Bulgrien Design and Development Engineer VertexRSI This email message is for the sole use of the intended recipient(s) and may contain General Dynamics SATCOM Technologies confidential or privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not an intended recipient, please contact the sender by reply email and destroy all copies of the original message. ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
> I have hosed myself. I didn''t tar up the configuration once > it started > working, and I tried to reproduce the problem and now cannot > get it back > working, so indications are that the solution is a bit more > than changing > the LOG tag... The sad thing is I was very sure I had only > changed/added > log items and https rule to $FW all instead of $FW net when it started > working. I have not yet figured out how to fix it, so > anyway, if nothing > else, it is an opportunity to log as a known issue. > > The attached configuration is broken. Failed attempts are > > SRC=192.168.128.7 DST=212.85.147.126 > > To try to make sure it wasn''t a destination fault, after the > failure, I > did shorewall clear and retried. The https link succeeded, > after which > I immediately did a start. The https link attempt again > fails even with > the LOG rule not triggering the truncation warning.It turns out I did do something else to the configuration to get it to work. I had added two more LOG entries to debug the situation. It wasn''t the HTTPSout log rule rename to 443out that fixed it. Apparently what fixed it was adding two more log rules. LOG:$LOG:NETout $FW net tcp - - - - - LOG:$LOG:NETin net all tcp - - - - With the NETout and NETin rules added, the https configuration works for either instance of the https log tag, but if I comment out the the NETin log rule, things break again. This is very weird. Do you want the rules file too? --- Kevin R. Bulgrien Design and Development Engineer VertexRSI This email message is for the sole use of the intended recipient(s) and may contain General Dynamics SATCOM Technologies confidential or privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not an intended recipient, please contact the sender by reply email and destroy all copies of the original message. ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
Bulgrien, Kevin wrote:>> I have hosed myself. I didn''t tar up the configuration once >> it started >> working, and I tried to reproduce the problem and now cannot >> get it back >> working, so indications are that the solution is a bit more >> than changing >> the LOG tag... The sad thing is I was very sure I had only >> changed/added >> log items and https rule to $FW all instead of $FW net when it started >> working. I have not yet figured out how to fix it, so >> anyway, if nothing >> else, it is an opportunity to log as a known issue. >> >> The attached configuration is broken. Failed attempts are >> >> SRC=192.168.128.7 DST=212.85.147.126 >> >> To try to make sure it wasn''t a destination fault, after the >> failure, I >> did shorewall clear and retried. The https link succeeded, >> after which >> I immediately did a start. The https link attempt again >> fails even with >> the LOG rule not triggering the truncation warning. > > It turns out I did do something else to the configuration to > get it to work. I had added two more LOG entries to debug the > situation. It wasn''t the HTTPSout log rule rename to 443out > that fixed it. Apparently what fixed it was adding two more > log rules. > > LOG:$LOG:NETout $FW net tcp - - - - > - > LOG:$LOG:NETin net all tcp - - - - > > > With the NETout and NETin rules added, the https configuration > works for either instance of the https log tag, but if I > comment out the the NETin log rule, things break again. > > This is very weird. Do you want the rules file too?I understand what''s wrong with the ruleset; it has not rule(s) for handling net->fw traffic. By adding the ''net->all'' logging rule, you forced the net2fw chain to be created. I would like a tarball with the entire /etc/shorewall so that I can be sure that the problem is fixed in the current releases. Thanks, -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
> > It turns out I did do something else to the configuration to > > get it to work. I had added two more LOG entries to debug the > > situation. It wasn''t the HTTPSout log rule rename to 443out > > that fixed it. Apparently what fixed it was adding two more > > log rules. > > > > LOG:$LOG:NETout $FW net tcp - - --> > - > > LOG:$LOG:NETin net all tcp - - --> > > > > > With the NETout and NETin rules added, the https configuration > > works for either instance of the https log tag, but if I > > comment out the the NETin log rule, things break again. > > > > This is very weird. Do you want the rules file too? > > I understand what''s wrong with the ruleset; it has not rule(s) for > handling net->fw traffic. By adding the ''net->all'' logging rule, you > forced the net2fw chain to be created. > > I would like a tarball with the entire /etc/shorewall so that I can be > sure that the problem is fixed in the current releases. > > Thanks, > -TomSo I could remove the LOG rules just by adding any valid net->fw rule even if it is a DROP or REJECT rule that is redundant with a policy? I have sent etc_shorewall_net2fw.tar.bz2 to you directly instead of over the list. --- Kevin R. Bulgrien Design and Development Engineer VertexRSI This email message is for the sole use of the intended recipient(s) and may contain General Dynamics SATCOM Technologies confidential or privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not an intended recipient, please contact the sender by reply email and destroy all copies of the original message. ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
Bulgrien, Kevin wrote:> > So I could remove the LOG rules just by adding any valid net->fw rule > even if it is a DROP or REJECT rule that is redundant with a policy?That would be my guess.> > I have sent etc_shorewall_net2fw.tar.bz2 to you directly instead of > over the list.-Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
Tom Eastep wrote:> Bulgrien, Kevin wrote: > >> So I could remove the LOG rules just by adding any valid net->fw rule >> even if it is a DROP or REJECT rule that is redundant with a policy? > > That would be my guess. > >> I have sent etc_shorewall_net2fw.tar.bz2 to you directly instead of >> over the list.Getting forgetful in my old age. I neglected to say... Thanks! -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
Hmm...
Your message did not reach some or all of the intended recipients.
Subject: Re: [Shorewall-users] Bug? LOG Actions may break ACCEPT
rules
Sent: 8/17/2007 3:03 PM
The following recipient(s) could not be reached:
''teastep@shorewall.net'' on 8/17/2007 3:01 PM
This message could not be sent. Try sending the message again
later, or contact your network administrator. The item is currently being
scanned for viruses. Try again in a few moments. Error is
[0x80040119-0x00050a-0x000546].
What''s the right way to get it to you, do I wait, or sent on list.
The bz2 is about 32K.
---
Kevin R. Bulgrien
Design and Development Engineer
VertexRSI
This email message is for the sole use of the intended recipient(s) and may
contain General Dynamics SATCOM Technologies confidential or privileged
information. Any unauthorized review, use, disclosure or distribution is
prohibited. If you are not an intended recipient, please contact the sender by
reply email and destroy all copies of the original message.
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
Bulgrien, Kevin wrote:> Hmm... > > Your message did not reach some or all of the intended recipients. > > Subject: Re: [Shorewall-users] Bug? LOG Actions may break ACCEPT > rules > Sent: 8/17/2007 3:03 PM > > The following recipient(s) could not be reached: > > ''teastep@shorewall.net'' on 8/17/2007 3:01 PM > This message could not be sent. Try sending the message again > later, or contact your network administrator. The item is currently being > scanned for viruses. Try again in a few moments. Error is > [0x80040119-0x00050a-0x000546]. > > What''s the right way to get it to you, do I wait, or sent on list. > The bz2 is about 32K.Hmmm -- no one else has reported problems sending to me. I didn''t even see any connection attempt from General Dynamics in my log. Try sending it to teastep@avvanta.com -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
> > -----Original Message----- > > > > So I could remove the LOG rules just by adding any valid net->fw rule > > even if it is a DROP or REJECT rule that is redundant with a policy? > > That would be my guess.Tom, Yes, replacing the LOG rules... # # HTTPS fw to net was fixed by adding these two rules. If the NETin rule # is commented out, HTTPS from fw to net will be broken. LOG:$LOG:NETout $FW net tcp - - - - - LOG:$LOG:NETin net all tcp - - - - - # With... # # Workaround: fw->net will not work with out this: # DROP net $FW icmp 8 - - - - # Does the trick. Thank-you for all you do to help. Shorewall is the only firewall I''ve ever used... and it is on quite a few systems I run at work and at home, so you''re efforts are greatly appreciated. --- Kevin R. Bulgrien Design and Development Engineer VertexRSI This email message is for the sole use of the intended recipient(s) and may contain General Dynamics SATCOM Technologies confidential or privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not an intended recipient, please contact the sender by reply email and destroy all copies of the original message. ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
Bulgrien, Kevin wrote:> > I have sent etc_shorewall_net2fw.tar.bz2 to you directly instead of > over the list. >Your net->all policy is CONTINUE. Since ''net'' is not a sub-zone of any other zone, rules from net->fw fall off the end of the world. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
Tom Eastep wrote:> Bulgrien, Kevin wrote: > >> I have sent etc_shorewall_net2fw.tar.bz2 to you directly instead of >> over the list. >> > > Your net->all policy is CONTINUE. Since ''net'' is not a sub-zone of any other > zone, rules from net->fw fall off the end of the world.In other words, if you change your net->all policy to DROP or REJECT, then you won''t need to extra rule. I''ve added a patch to 4.0.3 (Shell and Perl) to avoid this problem in the future Thanks, -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
On Fri, 2007-08-17 at 15:19 -0700, Tom Eastep wrote:> Tom Eastep wrote: > > Bulgrien, Kevin wrote: > > > >> I have sent etc_shorewall_net2fw.tar.bz2 to you directly instead of > >> over the list. > >> > > > > Your net->all policy is CONTINUE. Since ''net'' is not a sub-zone of any other > > zone, rules from net->fw fall off the end of the world. > > In other words, if you change your net->all policy to DROP or REJECT, then > you won''t need to extra rule. > > I''ve added a patch to 4.0.3 (Shell and Perl) to avoid this problem in the futureThe patch also causes Shorewall-perl to warn when a CONTINUE policy is between orphan zones (zones with no parent zones): Compiling /home/teastep/Kevin/shorewall/policy... WARNING: CONTINUE policy between orphan zones (net, fw) : /home/teastep/Kevin/shorewall/policy (line 106) WARNING: CONTINUE policy between orphan zones (net, dnd) : /home/teastep/Kevin/shorewall/policy (line 106) -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/