Hia folks, I have setup my own firewall using the Shoreline Firewall. Now - I have identified that users using the IE 6 or 7 get dropped quite fast by my system. As a little Explanation: Firewall is a Lex-Mini ITX System, 3 ethernet 100/10, 1 USB Wlan, 1 USB ADSL Modem connected. I''m using Busybox to boot the system, e.g. to copy the Root-FS from a CF-Card to a Ram-FS, before the system initialises. Shoreline Firewall is setup to block everything - and logging is done using the ulogd extension to a remote Mysql Database inside the Service Network (Some people also know this as DMZ - but this term is wrong for this). The Ulog Daemon will store the tagged packets by shorewall as beeing rejected, dropped or accepted, depending on the per interface policy I have set up. Now - as I don''t like people to scan my systems - I have written a little Daemon that watches the mysql-Ulog DB for dropped packets, initialises a counter - and every site that produces a certain number of dropped packets - get''s marked as Blacklisted, and the firewall adds this site in a matter of 1/minute to the shorewall dynamic blacklist. Now - the Internet Explorer 6 or 7 produces strange packets - that the shorewall system dropps immediatly - this users using the IE 6 or 7 get blacklisted fast... IMHO - I don''t mind blocking people using M$ Software out of my site - but I''m interested as to why the system drops these packets... And the point is - I don''t know why this happens ... OK - here is a clean entry: oob_time_sec 2007-05-16 17:46:58 oob_time_usec 15754 oob_prefix Shorewall:world_dnat:DNAT: oob_in ppp0 ip_saddr 83.171.189.186 ip_daddr 212.114.251.235 ip_protocol 6 ip_ttl 126 ip_totlen 52 ip_ihl 5 ip_csum 2876 ip_id 4036 ip_fragoff 16384 tcp_sport 49506 tcp_dport 80 tcp_seq 3710813843 tcp_window 8192 tcp_syn 1 And now the one the firewall has dropped: oob_time_sec 2007-05-16 17:47:01 oob_time_usec 130066 oob_prefix Shorewall:world2fw:DROP: oob_in ppp0 ip_saddr 83.171.189.186 ip_daddr 212.114.251.235 ip_protocol 6 ip_ttl 126 ip_totlen 52 ip_ihl 5 ip_csum 2833 ip_id 4079 ip_fragoff 16384 tcp_sport 49516 tcp_dport 80 tcp_seq 4235568596 tcp_window 8192 tcp_syn 1 It seems that this packet is not identified as having to be forwarded to the Webserver behind - but as the destination would be the firewall itself. Which is IMHO odd ... I have asked a friend to access my site and captured the requests. The IE request produced a Dropped packet, the Firefox not... So i could send these in if anyone is interested (I attached the IE-capture to this Mail). You can identify the requests by checking the sourceports in the pcap-file (using ethereal or wireshark)... Have you seen this before ? Note that this happens only with Microsoft Internet Explorer since version 6 and later ... All other browsers have no issues ... I am using Shorewall 3.2.4-1 on an Ubuntu 6.06.1 LTS based system. Thx for your Time ;) PS: I prefere not to send the shorewall dump to the list - can be missused by people accessing the archives - as the entire network infrastructure can be read out of it. Anyone who is si kind to help me - let me know - I''ll send you the dump. Thx. -- ------------------------------------------------------------------------ | Joerg Mertin : smurphy@solsys.org (Home)| | in Forchheim/Germany : smurphy@linux.de (Alt1)| | Stardust''s LiNUX System : | | Web: http://www.solsys.org | ------------------------------------------------------------------------ PGP Fingerprint: AF0F FB75 997B 025F 4538 5AD6 9888 5D97 170B 8B7A ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/
Joerg Mertin wrote:> > It seems that this packet is not identified as having to be forwarded to > the Webserver behind - but as the destination would be the firewall > itself. Which is IMHO odd ...It most likely means that, for some reason, the packets are not matching either the DNAT rule or an existing conntrack entry.> I have asked a friend to access my site and captured the requests. The IE > request produced a Dropped packet, the Firefox not... So i could send > these in if anyone is interested (I attached the IE-capture to this Mail). > You can identify the requests by checking the sourceports in the pcap-file > (using ethereal or wireshark)...Afraid not -- either your mailer or the list software has mangled the pcap file so that neither tcpdump nor Wireshark will have anything to do with it.> > Have you seen this before ? Note that this happens only with Microsoft > Internet Explorer since version 6 and later ... > All other browsers have no issues ...No such problems have been reported here in the past.> > > I am using Shorewall 3.2.4-1 on an Ubuntu 6.06.1 LTS based system. > > Thx for your Time ;) > PS: I prefere not to send the shorewall dump to the list - can be missused > by people accessing the archives - as the entire network infrastructure > can be read out of it. Anyone who is si kind to help me - let me know - > I''ll send you the dump.I''m doubtful that a dump will tell us anything. You are logging to a SQL database and the log dump section of the dump comprises one of the most important diagnostic tools that we have. What I suspect that we''ll see is some packets matching your log rule and the following drop rule and nothing else. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/
Joerg Mertin
2007-May-16 20:48 UTC
Re: Shorewall Dropping packets from IE 6 and higher ...
Hi there, <quote who="Tom Eastep"> [...]> It most likely means that, for some reason, the packets are not matching > either the DNAT rule or an existing conntrack entry.Well - the funny thing is that Firefox does not have that beheaviour - and I don''t know why. Could it be a portscanning-detection that activates ? My firewall isn''t very fast - but 533MHz CPU for a 6mBit line should IMHO be fast enough ... [...]> Afraid not -- either your mailer or the list software has mangled the pcap > file so that neither tcpdump nor Wireshark will have anything to do with > it.Hmm. Bad. I uploaded them to my Website. Check out: http://www.solsys.org/linux/shorewall_dump.txt http://www.solsys.org/linux/mozilla.pcap http://www.solsys.org/linux/ie.pcap [...]> No such problems have been reported here in the past.I never saw that beheaviour in my computer Life before either. 15years :( [...]> I''m doubtful that a dump will tell us anything. You are logging to a SQL > database and the log dump section of the dump comprises one of the most > important diagnostic tools that we have. What I suspect that we''ll see isHmm. I could deactivate the ulog-d and let shorewall write to a file. What exact directive should I give it to provide usable informations ?> some packets matching your log rule and the following drop rule and > nothing > else.Exact. But why does the system identify the packet not beeing part of a generic communication ? The 3-Way handshak took place - and only std requests get through after on... and I don''t see a difference between DNAT''ed packets and the DROP''ed packets ... Thanks & Cheers Joerg -- ------------------------------------------------------------------------ | Joerg Mertin : smurphy@solsys.org (Home)| | in Forchheim/Germany : smurphy@linux.de (Alt1)| | Stardust''s LiNUX System : | | Web: http://www.solsys.org | ------------------------------------------------------------------------ PGP Fingerprint: AF0F FB75 997B 025F 4538 5AD6 9888 5D97 170B 8B7A ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/
Joerg Mertin wrote:> Hi there, > > <quote who="Tom Eastep"> > [...] >> It most likely means that, for some reason, the packets are not matching >> either the DNAT rule or an existing conntrack entry. > > Well - the funny thing is that Firefox does not have that beheaviour - and > I don''t know why. Could it be a portscanning-detection that activates ?Shorewall has no portscanning-detection.> http://www.solsys.org/linux/shorewall_dump.txtYou have a limit on your DNAT rule!!!!!!!!!!!! Anything in excess of the limit will be rejected in the world2fw chain. IE tends to open more parallel connections than Mozilla. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/
Joerg Mertin
2007-May-17 09:38 UTC
Re: Shorewall Dropping packets from IE 6 and higher ...
Damn, I knew it ... I had set this up because of some people doing too many requests on ssh etc. - and because of lazyness I added all port-numbers under one rule - and didn''t take it out any more... Thx for the Hint. I didn''t see the forest because of the trees anymore ... Cheers Joerg <quote who="Tom Eastep">> Joerg Mertin wrote: >> Hi there, >> >> <quote who="Tom Eastep"> >> [...] >>> It most likely means that, for some reason, the packets are not >>> matching >>> either the DNAT rule or an existing conntrack entry. >> >> Well - the funny thing is that Firefox does not have that beheaviour - >> and >> I don''t know why. Could it be a portscanning-detection that activates ? > > Shorewall has no portscanning-detection. > > >> http://www.solsys.org/linux/shorewall_dump.txt > > You have a limit on your DNAT rule!!!!!!!!!!!! Anything in excess of the > limit will be rejected in the world2fw chain. IE tends to open more > parallel > connections than Mozilla. > > -Tom > -- > Tom Eastep \ Nothing is foolproof to a sufficiently talented fool > Shoreline, \ http://shorewall.net > Washington USA \ teastep@shorewall.net > PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/_______________________________________________ > Shorewall-users mailing list > Shorewall-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/shorewall-users >-- ------------------------------------------------------------------------ | Joerg Mertin : smurphy@solsys.org (Home)| | in Forchheim/Germany : smurphy@linux.de (Alt1)| | Stardust''s LiNUX System : | | Web: http://www.solsys.org | ------------------------------------------------------------------------ PGP Fingerprint: AF0F FB75 997B 025F 4538 5AD6 9888 5D97 170B 8B7A ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/
Joerg Mertin
2007-May-17 09:54 UTC
Re: Shorewall Dropping packets from IE 6 and higher ...
It seems that the IE is issuing really way more requests than I assumed. I did put the Value to 20/sec:50 which should make it OK. If people really want to be faster in accessing my site - well - they''ll be locked out. It was actually a limit I had set for the Bot-Harvesters and Brute-Force attacks issuing on a regular base. However - I added 2 more techniques to stop these - a harvester Trap: https://stargate.solsys.org/harvester/index.html which is very easy to setup - and also acting as Limiting factor for Bots not respecting the robot.txt file, and for ssh I wrote a script that looks for failed logins and locks probing sites immediatly - if not whitelisted. So - I now can configure the connection rate to be more relaxed. Thx again ... I knew it had something to do with the connection rate - but I completely had ignored the connection-rate settings in the DNAT rule ... Cheers Joerg <quote who="Joerg Mertin">> Damn, I knew it ... I had set this up because of some people doing too > many requests on ssh etc. - and because of lazyness I added all > port-numbers under one rule - and didn''t take it out any more... > > Thx for the Hint. I didn''t see the forest because of the trees anymore ... > > Cheers > Joerg > > <quote who="Tom Eastep"> >> Joerg Mertin wrote: >>> Hi there, >>> >>> <quote who="Tom Eastep"> >>> [...] >>>> It most likely means that, for some reason, the packets are not >>>> matching >>>> either the DNAT rule or an existing conntrack entry. >>> >>> Well - the funny thing is that Firefox does not have that beheaviour - >>> and >>> I don''t know why. Could it be a portscanning-detection that activates ? >> >> Shorewall has no portscanning-detection. >> >> >>> http://www.solsys.org/linux/shorewall_dump.txt >> >> You have a limit on your DNAT rule!!!!!!!!!!!! Anything in excess of the >> limit will be rejected in the world2fw chain. IE tends to open more >> parallel >> connections than Mozilla. >> >> -Tom >> -- >> Tom Eastep \ Nothing is foolproof to a sufficiently talented fool >> Shoreline, \ http://shorewall.net >> Washington USA \ teastep@shorewall.net >> PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key >> >> ------------------------------------------------------------------------- >> This SF.net email is sponsored by DB2 Express >> Download DB2 Express C - the FREE version of DB2 express and take >> control of your XML. No limits. Just data. Click to get it now. >> http://sourceforge.net/powerbar/db2/_______________________________________________ >> Shorewall-users mailing list >> Shorewall-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/shorewall-users >> > > > -- > ------------------------------------------------------------------------ > | Joerg Mertin : smurphy@solsys.org (Home)| > | in Forchheim/Germany : smurphy@linux.de (Alt1)| > | Stardust''s LiNUX System : | > | Web: http://www.solsys.org | > ------------------------------------------------------------------------ > PGP Fingerprint: AF0F FB75 997B 025F 4538 5AD6 9888 5D97 170B 8B7A > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > _______________________________________________ > Shorewall-users mailing list > Shorewall-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/shorewall-users >-- ------------------------------------------------------------------------ | Joerg Mertin : smurphy@solsys.org (Home)| | in Forchheim/Germany : smurphy@linux.de (Alt1)| | Stardust''s LiNUX System : | | Web: http://www.solsys.org | ------------------------------------------------------------------------ PGP Fingerprint: AF0F FB75 997B 025F 4538 5AD6 9888 5D97 170B 8B7A ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/
Joerg Mertin wrote:> It seems that the IE is issuing really way more requests than I assumed. > I did put the Value to 20/sec:50 which should make it OK. If people really > want to be faster in accessing my site - well - they''ll be locked out. >The problem with using LIMIT/BURST is that if some people really want fast access then *other* people may be locked out. LIMIT/BURST limits the total number of connections *from all clients*. If you want to Limit the per-client connection rate, you need to use the ''Limit'' action. With DNAT that means that you must configure separate DNAT- and Limit rules. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/
Joerg Mertin
2007-May-17 20:37 UTC
Re: Shorewall Dropping packets from IE 6 and higher ...
Hmmm... had that setup - however my connection then was limited to rather low speed. From a 6MBit connection I had suddenly only 2 to 3mbit available. Using the tcclasses config-file. I don''t want to limit people in speed. However, I have identified some port-scanning that were quite heavy sometimes back - and I just wanted to limit the number of new connections per time-frame per host... Any better idea of doing that ? Thanks Joerg <quote who="Tom Eastep"> [...]> with using LIMIT/BURST is that if some people really want fast > access then *other* people may be locked out. LIMIT/BURST limits the total > number of connections *from all clients*. > > If you want to Limit the per-client connection rate, you need to use the > ''Limit'' action. With DNAT that means that you must configure separate > DNAT- > and Limit rules.-- ------------------------------------------------------------------------ | Joerg Mertin : smurphy@solsys.org (Home)| | in Forchheim/Germany : smurphy@linux.de (Alt1)| | Stardust''s LiNUX System : | | Web: http://www.solsys.org | ------------------------------------------------------------------------ PGP Fingerprint: AF0F FB75 997B 025F 4538 5AD6 9888 5D97 170B 8B7A ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/
Joerg Mertin wrote:> Hmmm... > > had that setup - however my connection then was limited to rather low > speed. From a 6MBit connection I had suddenly only 2 to 3mbit available. > Using the tcclasses config-file.What does tcclasses have to do with limiting the connection rate of individual hosts?> > I don''t want to limit people in speed. However, I have identified some > port-scanning that were quite heavy sometimes back - and I just wanted to > limit the number of new connections per time-frame per host... > Any better idea of doing that ?I already told you. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/
Joerg Mertin
2007-May-18 20:47 UTC
Re: Shorewall Dropping packets from IE 6 and higher ...
<quote who="Tom Eastep"> [...]> If you want to Limit the per-client connection rate, you need to use the > ''Limit'' action. With DNAT that means that you must configure separate > DNAT- > and Limit rules.Is there a possibility to tell the system to unblock the IP after a certain time ? or will I have to csript this ? Cheers Joerg -- ------------------------------------------------------------------------ | Joerg Mertin : smurphy@solsys.org (Home)| | in Forchheim/Germany : smurphy@linux.de (Alt1)| | Stardust''s LiNUX System : | | Web: http://www.solsys.org | ------------------------------------------------------------------------ PGP Fingerprint: AF0F FB75 997B 025F 4538 5AD6 9888 5D97 170B 8B7A ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/
Joerg Mertin wrote:> <quote who="Tom Eastep"> > [...] >> If you want to Limit the per-client connection rate, you need to use the >> ''Limit'' action. With DNAT that means that you must configure separate >> DNAT- >> and Limit rules. > > Is there a possibility to tell the system to unblock the IP after a > certain time ? or will I have to csript this ? >Please read the documentation for Limit -- http://www.shorewall.net/PortKnocking.html#Limit You need not script the unblocking. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/