Hello everyone, I recently got shorewall and squid working pretty well with some excellent help from the documentation and additional help from Tom. Now I am working on a couple of anomolies and am hoping that someone has some experience with this issue. Here is the situation: I am having a problem with some remote web sites. I have followed the instructions on setting up squid on the same server as shorewall using transparent proxy. 1. Most sites work fine with either no proxy settings (transparent) or with browsers pointing to the squid proxy running on 8080. 2. Most https sites won''t work unless clients use the proxy. (This seems weird as I am not trying to transparently proxy https) 3. At least one https site (https://banking.wellsfargo.com) won''t work either way. 4. At least two http sites won''t work either way. (http://www.wellsfargo.com and http://www.ci.tulare.ca.us) I have solved several of these types of problems by modifying /etc/shorewall/rules and /etc/squid.conf allowing the required traffic. However the wellsfargo problem does not seem to trip any of my shorewall policies as far as squid goes. It also does not seem to trip any of the squid security acls. I will simply get an ERROR (110) Connection Timed Out message from squid but no other clues in the logs. Everything works fine if I bypass the proxy and disable the shorewall REDIRECT statements Thanks in advance for considering my problem, if I am asking a FAQ, I apoligize in advance but could not find any good clues regarding my specific problem. One more thing to consider. We have always used the default PTR record that my ISP set for our firewalls public IP address and never put an A record for the firewall. I have changed the A record to point to the firewall but the PTR record has not been updated with the correct hostname yet. Could that be my problem? It seems most webservers don''t care but maybe these two do? I''m confused.... :-) In /etc/squid.conf I have changed the following from the defaults: http_port 172.30.2.7:8080 httpd_accel_host virtual httpd_accel_with_proxy on httpd_accel_uses_host_header on In /etc/shorewall/rules #Allow firewall to retrieve http pages ACCEPT fw net tcp http,https ACCEPT fw all tcp 8080 # Allow firewall to retrieve mail from www.pintolake.com:8383 via squid ACCEPT fw all tcp 8383 ACCEPT fw all tcp 8443 # Redirect Outgoing Web Traffic to local Squid Proxy REDIRECT loc 8080 tcp www - !172.30.0.0/16 REDIRECT loc1 8080 tcp www - !172.30.0.0/16 REDIRECT loc2 8080 tcp www - !172.30.0.0/16 REDIRECT loc3 8080 tcp www - !172.30.0.0/16 REDIRECT loc4 8080 tcp www - !172.30.0.0/16 # Allow local networks to request Pages from the internet ACCEPT loc fw tcp http,https ACCEPT loc1 fw tcp http,https ACCEPT loc2 fw tcp http,https ACCEPT loc3 fw tcp http,https ACCEPT loc4 fw tcp http,https # Allow firewall to do ftp ACCEPT fw net tcp ftp #
Joel Staker wrote:> > I am having a problem with some remote web sites. I have followed the > instructions on setting up squid on the same server as shorewall using > transparent proxy. > > 1. Most sites work fine with either no proxy settings (transparent) or > with browsers pointing to the squid proxy running on 8080. > 2. Most https sites won''t work unless clients use the proxy. (This > seems weird as I am not trying to transparently proxy https)Which thankfully is impossible anyway...> 3. At least one https site (https://banking.wellsfargo.com) won''t work > either way. > 4. At least two http sites won''t work either way. > (http://www.wellsfargo.com and http://www.ci.tulare.ca.us)I have a transparent Squid Proxy (see http://shorewall.net/myfiles.htm) and all of the above http sites work ok when transparently proxied and the Wells HTTPS site also works fine (I don''t proxy HTTPS).> > I have solved several of these types of problems by modifying > /etc/shorewall/rules and /etc/squid.conf allowing the required traffic. > However the wellsfargo problem does not seem to trip any of my shorewall > policies as far as squid goes. It also does not seem to trip any of the > squid security acls. > > I will simply get an ERROR (110) Connection Timed Out message from > squid but no other clues in the logs.These cases are often best diagnosed using ethereal (or tcpdump if you don''t have a graphical environment on your firewall).> > Everything works fine if I bypass the proxy and disable the shorewall > REDIRECT statements >What does "bypass the proxy" mean? Does that simply mean that you remove the REDIRECT statements or are you doing something else? There is one thing that I don''t understand: Nothing I see here should have any effect on https from the local zones to the net. So if "...bypass the proxy and disable the shorewall REDIRECT statements" suddenly makes such traffic work then we are witnessing something from the Twilight Zone.> > One more thing to consider. We have always used the default PTR record > that my ISP set for our firewalls public IP address and never put an A > record for the firewall. I have changed the A record to point to the > firewall but the PTR record has not been updated with the correct > hostname yet. Could that be my problem? It seems most webservers don''t > care but maybe these two do? > > I''m confused.... :-)Me too.> > In /etc/squid.conf I have changed the following from the defaults: > > http_port 172.30.2.7:8080 > httpd_accel_host virtual > httpd_accel_with_proxy on > httpd_accel_uses_host_header onWhat about httpd_accel_port (I don''t recall if that defaults to 8080)? -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net
Joel Staker wrote:> One more thing to consider. We have always used the default PTR record > that my ISP set for our firewalls public IP address and never put an A > record for the firewall. I have changed the A record to point to the > firewall but the PTR record has not been updated with the correct > hostname yet. Could that be my problem? It seems most webservers don''t > care but maybe these two do?Since you are using RFC 1918 addresses in your local network, I assume that this situation also exists if you aren''t using Squid at all -- right? -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net
Tom Eastep wrote:> > What about httpd_accel_port (I don''t recall if that defaults to 8080)? >Make that "...defaults to 80". -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net
>>> teastep@shorewall.net 05/24/04 04:53PM >>>Joel Staker wrote:> > I am having a problem with some remote web sites. I have followedthe> instructions on setting up squid on the same server as shorewallusing> transparent proxy. > > 1. Most sites work fine with either no proxy settings (transparent)or> with browsers pointing to the squid proxy running on 8080. > 2. Most https sites won''t work unless clients use the proxy. (This > seems weird as I am not trying to transparently proxy https)Which thankfully is impossible anyway... As you might have guessed I am an idiot... What I meant to say was if the client is not configured to use the proxy *and* the REDIRECT statements are commented out it works, if the redirect statements are working then some https sites fail. As far as I can tell everything should work whether the redirects are in effect or not. This does not seem to be the case for me.> 3. At least one https site (https://banking.wellsfargo.com) won''twork> either way. > 4. At least two http sites won''t work either way. > (http://www.wellsfargo.com and http://www.ci.tulare.ca.us)I have a transparent Squid Proxy (see http://shorewall.net/myfiles.htm) and all of the above http sites work ok when transparently proxied and the Wells HTTPS site also works fine (I don''t proxy HTTPS). This leads me to believe that, in theory, what I am trying to do should be working just fine and maybe the problem is a DNS, RDNS problem that only a few sites care about.> > I have solved several of these types of problems by modifying > /etc/shorewall/rules and /etc/squid.conf allowing the requiredtraffic.> However the wellsfargo problem does not seem to trip any of myshorewall> policies as far as squid goes. It also does not seem to trip any ofthe> squid security acls. > > I will simply get an ERROR (110) Connection Timed Out message from > squid but no other clues in the logs.These cases are often best diagnosed using ethereal (or tcpdump if you don''t have a graphical environment on your firewall). I have a Dell Poweredge 2650 with Dual Xeons, this thing has just been begging for more to do. <Grin> I''ll take a look at the ethereal results and get back to you.> > Everything works fine if I bypass the proxy and disable theshorewall> REDIRECT statements >What does "bypass the proxy" mean? Does that simply mean that you remove the REDIRECT statements or are you doing something else? I am also telling the clients not to use the proxy. So, when squid is totally out of the picture everything works great. This tells me it is not a DNS RDNS issue. See? I told you I am an idiot... There is one thing that I don''t understand: Nothing I see here should have any effect on https from the local zones to the net. So if "...bypass the proxy and disable the shorewall REDIRECT statements" suddenly makes such traffic work then we are witnessing something from the Twilight Zone. Well I can mess up a perfectly good configuration file with the best of em. But indeed with things configured pretty much like your example files mentioned above, None of my clients can get to https://banking.wellsfargo.com however the same client will have *no problems* getting to https://www.wamu.com or any other https site that I''ve tested.> > One more thing to consider. We have always used the default PTRrecord> that my ISP set for our firewalls public IP address and never put anA> record for the firewall. I have changed the A record to point tothe> firewall but the PTR record has not been updated with the correct > hostname yet. Could that be my problem? It seems most webserversdon''t> care but maybe these two do? > > I''m confused.... :-)Me too.> > In /etc/squid.conf I have changed the following from the defaults: > > http_port 172.30.2.7:8080 > httpd_accel_host virtual > httpd_accel_with_proxy on > httpd_accel_uses_host_header onWhat about httpd_accel_port (I don''t recall if that defaults to 8080)? It defaults to 80, however I was under the impression that I would only set that if I was accelerating internal servers? -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net _______________________________________________ Shorewall-users mailing list Post: Shorewall-users@lists.shorewall.net Subscribe/Unsubscribe: https://lists.shorewall.net/mailman/listinfo/shorewall-users Support: http://www.shorewall.net/support.htm FAQ: http://www.shorewall.net/FAQ.htm
>>> teastep@shorewall.net 05/24/04 05:10PM >>>Joel Staker wrote:> One more thing to consider. We have always used the default PTRrecord> that my ISP set for our firewalls public IP address and never put anA> record for the firewall. I have changed the A record to point tothe> firewall but the PTR record has not been updated with the correct > hostname yet. Could that be my problem? It seems most webserversdon''t> care but maybe these two do?Since you are using RFC 1918 addresses in your local network, I assume that this situation also exists if you aren''t using Squid at all -- right? Correct, I am grasping at straws here pretty much. Thanks for keeping me sane. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net _______________________________________________ Shorewall-users mailing list Post: Shorewall-users@lists.shorewall.net Subscribe/Unsubscribe: https://lists.shorewall.net/mailman/listinfo/shorewall-users Support: http://www.shorewall.net/support.htm FAQ: http://www.shorewall.net/FAQ.htm
Joel Staker wrote:> >>3. At least one https site (https://banking.wellsfargo.com) won''t > > work > >>either way. >>4. At least two http sites won''t work either way. >>(http://www.wellsfargo.com and http://www.ci.tulare.ca.us) > > > I have a transparent Squid Proxy (see http://shorewall.net/myfiles.htm) > > and all of the above http sites work ok when transparently proxied and > > the Wells HTTPS site also works fine (I don''t proxy HTTPS). > > This leads me to believe that, in theory, what I am trying to do should > be working just fine and maybe the problem is a DNS, RDNS problem that > only a few sites care about.FWIS, https://banking.wellsfargo.com doesn''t work through my Squid proxy either... -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net
Well, at least I am not crazy. :-) Thanks Tom.>>> teastep@shorewall.net 05/24/04 05:34PM >>>Joel Staker wrote:> >>3. At least one https site (https://banking.wellsfargo.com) won''t > > work > >>either way. >>4. At least two http sites won''t work either way. >>(http://www.wellsfargo.com and http://www.ci.tulare.ca.us) > > > I have a transparent Squid Proxy (seehttp://shorewall.net/myfiles.htm)> > and all of the above http sites work ok when transparently proxiedand> > the Wells HTTPS site also works fine (I don''t proxy HTTPS). > > This leads me to believe that, in theory, what I am trying to doshould> be working just fine and maybe the problem is a DNS, RDNS problemthat> only a few sites care about.FWIS, https://banking.wellsfargo.com doesn''t work through my Squid proxy either... -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net _______________________________________________ Shorewall-users mailing list Post: Shorewall-users@lists.shorewall.net Subscribe/Unsubscribe: https://lists.shorewall.net/mailman/listinfo/shorewall-users Support: http://www.shorewall.net/support.htm FAQ: http://www.shorewall.net/FAQ.htm
Joel Staker wrote:> Well, at least I am not crazy. :-) Thanks Tom. >But then, I think it''s rather silly to use transparent proxying for HTTP and manual for HTTPS. If you have to configure everyone''s browser anyway, why not just make them both manual? Also -- what version of Squid are you running. I''ve had odd problems with other versions of Squid but the one I''m running (squid-2.5.STABLE3-93 on SuSE) has been pretty trouble free. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net
Tom Eastep wrote:> Joel Staker wrote: > >> Well, at least I am not crazy. :-) Thanks Tom. >> > > But then, I think it''s rather silly to use transparent proxying for HTTP > and manual for HTTPS. If you have to configure everyone''s browser > anyway, why not just make them both manual?And conversely -- what does proxying HTTPS buy you? Not much AFAIKT unless you are into logging all of your users'' surfing. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net
Tom Eastep wrote:> Tom Eastep wrote: > >> Joel Staker wrote: >> >>> Well, at least I am not crazy. :-) Thanks Tom. >>> >> >> But then, I think it''s rather silly to use transparent proxying for >> HTTP and manual for HTTPS. If you have to configure everyone''s browser >> anyway, why not just make them both manual? > > > And conversely -- what does proxying HTTPS buy you? Not much AFAIKT > unless you are into logging all of your users'' surfing. >And I now have https://banking.wellsfargo.com working with Squid. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net
Joel Staker wrote:> Hello everyone, > > I recently got shorewall and squid working pretty well with some > excellent help from the documentation and additional help from Tom. Now > I am working on a couple of anomolies and am hoping that someone has > some experience with this issue. > > Here is the situation: > > I am having a problem with some remote web sites. I have followed the > instructions on setting up squid on the same server as shorewall using > transparent proxy. > > 1. Most sites work fine with either no proxy settings (transparent) or > with browsers pointing to the squid proxy running on 8080. > 2. Most https sites won''t work unless clients use the proxy. (This > seems weird as I am not trying to transparently proxy https) > 3. At least one https site (https://banking.wellsfargo.com) won''t work > either way. > 4. At least two http sites won''t work either way. > (http://www.wellsfargo.com and http://www.ci.tulare.ca.us) > > I have solved several of these types of problems by modifying > /etc/shorewall/rules and /etc/squid.conf allowing the required traffic. > However the wellsfargo problem does not seem to trip any of my shorewall > policies as far as squid goes. It also does not seem to trip any of the > squid security acls. > > I will simply get an ERROR (110) Connection Timed Out message from > squid but no other clues in the logs. > > Everything works fine if I bypass the proxy and disable the shorewall > REDIRECT statements > > Thanks in advance for considering my problem, if I am asking a FAQ, I > apoligize in advance but could not find any good clues regarding my > specific problem. > > One more thing to consider. We have always used the default PTR record > that my ISP set for our firewalls public IP address and never put an A > record for the firewall. I have changed the A record to point to the > firewall but the PTR record has not been updated with the correct > hostname yet. Could that be my problem? It seems most webservers don''t > care but maybe these two do? > > I''m confused.... :-) > > In /etc/squid.conf I have changed the following from the defaults: > > http_port 172.30.2.7:8080 > httpd_accel_host virtual > httpd_accel_with_proxy on > httpd_accel_uses_host_header on > > In /etc/shorewall/rulesI didn''t see rules allowing port 8080 from loc*->fw... -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net
Tom Eastep wrote:> Joel Staker wrote:>> >> In /etc/shorewall/rules > > > I didn''t see rules allowing port 8080 from loc*->fw... >And I apologize for the terrible job of cropping the quote in my last post... -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net
Tom Eastep wrote:>> >> In /etc/shorewall/rules > > I didn''t see rules allowing port 8080 from loc*->fw... >Nevermind -- the REDIRECT rules will add those rules as a by-product... -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net
On Mon, 24 May 2004 17:15:06 -0700 Tom Eastep <teastep@shorewall.net> wrote:> Tom Eastep wrote: > Make that "...defaults to 80".Right: httpd_accel_host virtual httpd_accel_port 80 httpd_accel_with_proxy on httpd_accel_uses_host_header on http_port 192.168.10.1:3128 Sounds a bit like he''s got it set up to allow people to use the 8080 proxy port to use the proxy though. I don''t think Squid allows this when it is working as a transparent proxy (but I could be wrong). -- Paul Slinski -o) Network Administrator /\ Global IQX, Inc. _\_v The information transmitted is intended only for the addressee and may contain confidential, proprietary and/or privileged material. Any unauthorized review, distribution or other use of or the taking of any action in reliance upon this information is prohibited. If you received this in error, please contact the sender and delete or destroy this message and any copies.
Paul Slinski wrote:> On Mon, 24 May 2004 17:15:06 -0700 > Tom Eastep <teastep@shorewall.net> wrote: > > >>Tom Eastep wrote: >>Make that "...defaults to 80". > > > Right: > > httpd_accel_host virtual > httpd_accel_port 80 > httpd_accel_with_proxy on > httpd_accel_uses_host_header on > http_port 192.168.10.1:3128 > > Sounds a bit like he''s got it set up to allow people to use the 8080 > proxy port to use the proxy though. I don''t think Squid allows this > when it is working as a transparent proxy (but I could be wrong). >Squid does allow manual proxy through the transparent port(s). -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net