Hello folks. I had a spontaneous reboot of my OpenWRT and Shorewall-lite 4.0.5 running firewall this evening. I must have taxed it''s puny little 32MB of memory. What is interesting is that on reboot, when a shorewall start was done, it appears that a packet leaked through before the SNAT rules were in place. Consider these two entries from the conntrack table: udp 17 3114 src=10.75.22.3 dst=66.51.110.210 sport=5060 dport=5060 packets=2520 bytes=1317666 [UNREPLIED] src=66.51.110.210 dst=10.75.22.3 sport=5060 dport=5060 packets=0 bytes=0 mark=0 use=1 udp 17 3597 src=10.75.22.3 dst=66.51.127.173 sport=5060 dport=5060 packets=144 bytes=85513 src=66.51.127.173 dst=67.193.45.68 sport=5060 dport=5060 packets=203 bytes=79918 [ASSURED] mark=64 use=1 Notice the second entry has the SNAT recorded in the table while the first does not. I can only imagine that this is because the packet hit netfilter before the SNAT rules were put into place. So I guess the question is, even with say, a default policy for all chains in netfilter of DROP, would an entry still make it into the conntrack table? I tend to think it would not. So perhaps this packet made it through in that short time window when networking is up, netfilter is loaded but before shorewall has done it''s thing. I wonder if anything can be done to mitigate this? Even if one were to simply do: # modprobe <netfilter modules> # for chain in FORWARD INPUT OUTPUT; do> iptables -P $chain DROP > donethere is still a small window of opportunity there. Is there no way to eliminate this possibility? Perhaps a way to load the netfilter modules in a "default DROP policy" mode? Alternatively, I wonder, is it possible to load the filtering modules first, set a default DROP policy and then install the nat table rules, then load the conntrack modules (perhaps the conntrack modules need loading before you can set up the nat table -- I don''t think the order of these two is critical since the default policy is still DROP at this point) and then the filter table policy. Or some other combination thereof which eliminates the race? b. ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
Brian J. Murrell wrote:> I wonder if anything can be done to mitigate this? Even if one were to > simply do:Sure -- same solution that has always been available. Start Shorewall before you start networking. Of course you can''t use any of Shorewall''s features that rely on detecting the current network configuration... -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
Tom Eastep wrote:> Brian J. Murrell wrote: > >> I wonder if anything can be done to mitigate this? Even if one were to >> simply do: > > Sure -- same solution that has always been available. Start Shorewall > before you start networking. > > Of course you can''t use any of Shorewall''s features that rely on > detecting the current network configuration... >And, of course, you can''t use traffic shaping or Multi-ISP. But then that''s another reason why we''re having the other discussion about separating Shorewall from routing and traffic shaping. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
On Sat, 2008-03-29 at 22:39 -0700, Tom Eastep wrote:> > Sure -- same solution that has always been available. Start Shorewall before > you start networking.Yes, but...> Of course you can''t use any of Shorewall''s features that rely on detecting > the current network configuration...Right. I was thinking/hoping for a solution that was more elegant. Such as I suggested perhaps... preventing packets from hitting the conntrack state engine before the nat rules were set up. Is it technically impossible to do that with iptables? I''ve never really played much with module loading order and dependencies of iptables. b. ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
Brian J. Murrell wrote:> On Sat, 2008-03-29 at 22:39 -0700, Tom Eastep wrote: >> Sure -- same solution that has always been available. Start Shorewall before >> you start networking. > > Yes, but... > >> Of course you can''t use any of Shorewall''s features that rely on detecting >> the current network configuration... > > Right. I was thinking/hoping for a solution that was more elegant. > Such as I suggested perhaps... preventing packets from hitting the > conntrack state engine before the nat rules were set up. > > Is it technically impossible to do that with iptables? I''ve never > really played much with module loading order and dependencies of > iptables.As long as you bring up networking before Shorewall, there is not a thing that Shorewall can do to solve this problem. So you get to solve this one yourself. Check out the ''raw'' table and the NOTRACK target; you might be able to avoid tracking any traffic until the Shorewall-generated script runs if you configure appropriate NOTRACK rules before bringing up networking. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
Tom Eastep wrote:> Brian J. Murrell wrote: >> On Sat, 2008-03-29 at 22:39 -0700, Tom Eastep wrote: >>> Sure -- same solution that has always been available. Start Shorewall >>> before you start networking. >> >> Yes, but... >> >>> Of course you can''t use any of Shorewall''s features that rely on >>> detecting the current network configuration... >> >> Right. I was thinking/hoping for a solution that was more elegant. >> Such as I suggested perhaps... preventing packets from hitting the >> conntrack state engine before the nat rules were set up. >> >> Is it technically impossible to do that with iptables? I''ve never >> really played much with module loading order and dependencies of >> iptables. > > As long as you bring up networking before Shorewall, there is not a > thing that Shorewall can do to solve this problem. So you get to solve > this one yourself. > > Check out the ''raw'' table and the NOTRACK target; you might be able to > avoid tracking any traffic until the Shorewall-generated script runs if > you configure appropriate NOTRACK rules before bringing up networking. > > -TomSince modprobe is run twice for a (re)start, once for detecting capabilities, then again with the compiled script without ill effects to shorewall, could you not load x_tables, ip_tables, and iptable_filter (in that order, did I forget any?). That should be enough to have the filter table loaded, then use the posted script before you start the network. Shorewall is going to flush the tables anyway right? Just a thought, Jerry ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
If ip_forward is never enabled until shorewall has started, then no packets will ever pass through the system. You''re then left with just local stuff on the firewall itself, which shouldn''t really be an issue (since you shouldn''t be running anything at that point). This should be the default behaviour, so I''d be looking into why that didn''t happen. ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
Andrew Suffield wrote:> If ip_forward is never enabled until shorewall has started, then no > packets will ever pass through the system. You''re then left with just > local stuff on the firewall itself, which shouldn''t really be an issue > (since you shouldn''t be running anything at that point). > > This should be the default behaviour, so I''d be looking into why that > didn''t happen.Well, the ip_forward flag is set after the loading of the modules, but before the loading of the ruleset. I know, small window but it is made worse with the limited resources that Brain has available. So using your logic, the setting of the flag should be the last thing done to ensure that this doesn''t happen? Jerry ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
On Sun, Mar 30, 2008 at 03:12:51AM -0500, Jerry Vonau wrote:> Andrew Suffield wrote: > > If ip_forward is never enabled until shorewall has started, then no > > packets will ever pass through the system. You''re then left with just > > local stuff on the firewall itself, which shouldn''t really be an issue > > (since you shouldn''t be running anything at that point). > > > > This should be the default behaviour, so I''d be looking into why that > > didn''t happen. > > Well, the ip_forward flag is set after the loading of the modules, but > before the loading of the ruleset. I know, small window but it is made > worse with the limited resources that Brain has available. So using your > logic, the setting of the flag should be the last thing done to ensure > that this doesn''t happen?I cannot imagine any reason why you might want it to happen before loading the firewall rules. ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
Andrew Suffield wrote:> On Sun, Mar 30, 2008 at 03:12:51AM -0500, Jerry Vonau wrote: >> Andrew Suffield wrote: >>> If ip_forward is never enabled until shorewall has started, then no >>> packets will ever pass through the system. You''re then left with just >>> local stuff on the firewall itself, which shouldn''t really be an issue >>> (since you shouldn''t be running anything at that point). >>> >>> This should be the default behaviour, so I''d be looking into why that >>> didn''t happen. >> Well, the ip_forward flag is set after the loading of the modules, but >> before the loading of the ruleset. I know, small window but it is made >> worse with the limited resources that Brain has available. So using your >> logic, the setting of the flag should be the last thing done to ensure >> that this doesn''t happen? > > I cannot imagine any reason why you might want it to happen before > loading the firewall rules.A quick snip: Setting up rules for DHCP... Setting up ARP filtering... Setting up Route Filtering... Setting up Martian Logging... Setting up Accept Source Routing... IP Forwarding Enabled Setting up SYN Flood Protection... Setting up Rules... Should this be fixed then? Jerry ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
Jerry Vonau wrote:> Andrew Suffield wrote: >> On Sun, Mar 30, 2008 at 03:12:51AM -0500, Jerry Vonau wrote: >>> Andrew Suffield wrote: >>>> If ip_forward is never enabled until shorewall has started, then no >>>> packets will ever pass through the system. You''re then left with just >>>> local stuff on the firewall itself, which shouldn''t really be an issue >>>> (since you shouldn''t be running anything at that point). >>>> >>>> This should be the default behaviour, so I''d be looking into why that >>>> didn''t happen. >>> Well, the ip_forward flag is set after the loading of the modules, but >>> before the loading of the ruleset. I know, small window but it is made >>> worse with the limited resources that Brain has available. So using your >>> logic, the setting of the flag should be the last thing done to ensure >>> that this doesn''t happen? >> I cannot imagine any reason why you might want it to happen before >> loading the firewall rules. > > A quick snip: > > Setting up rules for DHCP... > Setting up ARP filtering... > Setting up Route Filtering... > Setting up Martian Logging... > Setting up Accept Source Routing... > IP Forwarding Enabled > Setting up SYN Flood Protection... > Setting up Rules... > > Should this be fixed then?Probably. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
On Sat, 2008-03-29 at 23:00 -0700, Tom Eastep wrote:> > As long as you bring up networking before Shorewall, there is not a thing > that Shorewall can do to solve this problemI think you are confusing two different races. Yes, a race exists in that window of time when networking is brought up but before shorewall gets rules installed. There is another window within the above window which starts with netfilter seeing packets and keeping conntrack state for them. I suspect that this happens not as soon as networking is up, but sometime after that when the conntrack modules are loaded. I also suspect that it happens before the NAT rules are applied. b. ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
On Sun, 2008-03-30 at 08:06 +0100, Andrew Suffield wrote:> If ip_forward is never enabled until shorewall has started, then no > packets will ever pass through the system.Interestingly simple solution -- for forwarded packets.> You''re then left with just > local stuff on the firewall itself, which shouldn''t really be an issue > (since you shouldn''t be running anything at that point).Anything running locally should not need (S)NAT anyway. With the exception of the "other interfaces source address" SNAT that is done for MultiISP configurations, but even then, perhaps that situation is minimal.> This should be the default behaviour, so I''d be looking into why that > didn''t happen.Indeed. Just to be clear, Shorewall will set "net.ipv4.ip_forward=1" during it''s set up, so /etc/sysctl should have it =0, yes? b. ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
Tom Eastep wrote:> Jerry Vonau wrote: >> A quick snip: >> >> Setting up rules for DHCP... >> Setting up ARP filtering... >> Setting up Route Filtering... >> Setting up Martian Logging... >> Setting up Accept Source Routing... >> IP Forwarding Enabled >> Setting up SYN Flood Protection... >> Setting up Rules... >> >> Should this be fixed then? > > Probably. >teastep@wifiursa:~/Configs/gateway> shorewall -v reload gateway Compiling... Processing /home/teastep/Configs/gateway/params ... ... Restarting Shorewall Lite.... Initializing... Processing /home/teastep/Configs/gateway/init ... Setting up ARP filtering... Setting up Route Filtering... Setting up Martian Logging... Setting up Accept Source Routing... Setting up Proxy ARP... Null Routing the RFC 1918 subnets Adding Providers... Setting up Traffic Control... Adding TC Filters Preparing iptables-restore input... Running /usr/sbin/iptables-restore... Setting up dynamic rules... IP Forwarding Enabled Processing /home/teastep/Configs/gateway/start ... Processing /home/teastep/Configs/gateway/started ... done. System gateway reloaded teastep@wifiursa:~/Configs/gateway> Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
On Sun, 2008-03-30 at 08:11 -0700, Tom Eastep wrote:> > teastep@wifiursa:~/Configs/gateway> shorewall -v reload gateway > Compiling... > Processing /home/teastep/Configs/gateway/params ... > ... > Restarting Shorewall Lite.... > Initializing... > Processing /home/teastep/Configs/gateway/init ... > Setting up ARP filtering... > Setting up Route Filtering... > Setting up Martian Logging... > Setting up Accept Source Routing... > Setting up Proxy ARP... > Null Routing the RFC 1918 subnets > Adding Providers... > Setting up Traffic Control... > Adding TC Filters > Preparing iptables-restore input... > Running /usr/sbin/iptables-restore... > Setting up dynamic rules... > IP Forwarding Enabled > Processing /home/teastep/Configs/gateway/start ... > Processing /home/teastep/Configs/gateway/started ... > done. > System gateway reloaded > teastep@wifiursa:~/Configs/gateway>Just for posterity: root@gw:~# shorewall-lite -v restart Restarting Shorewall Lite.... Initializing... Processing /etc/shorewall/gw/init ... Setting up ARP filtering... Setting up Route Filtering... Setting up Martian Logging... Setting up Accept Source Routing... IP Forwarding Enabled Setting up Proxy ARP... Shorewall-generated routing tables and routing rules removed Adding Providers... Provider CGCO (1) Added Provider IGS (2) Added Default route ''nexthop via 67.193.44.1 dev eth0.1 weight 1 nexthop via 192.168.200.1 dev ppp0 weight 1'' Added Setting up Traffic Control... Preparing iptables-restore input... Running /usr/sbin/iptables-restore... Setting up dynamic rules... Processing /etc/shorewall/gw/start ... Processing /etc/shorewall/gw/started ... done. root@gw:~# shorewall-lite version 4.0.5 Lite b. ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
Tom Eastep wrote:> Running /usr/sbin/iptables-restore... > Setting up dynamic rules... > IP Forwarding Enabled > Processing /home/teastep/Configs/gateway/start ... > Processing /home/teastep/Configs/gateway/started ... > done. > System gateway reloaded > teastep@wifiursa:~/Configs/gateway>Here''s the patch to Shorewall-perl. Shorewall-shell patch forthcoming... -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
On Sun, Mar 30, 2008 at 11:06:46AM -0400, Brian J. Murrell wrote:> > You''re then left with just > > local stuff on the firewall itself, which shouldn''t really be an issue > > (since you shouldn''t be running anything at that point). > > Anything running locally should not need (S)NAT anyway.Well, I was thinking more about filtering rules. But the point in the boot process when shorewall starts should be before anything else that uses the network, aside from the fundamental things like dhcp and ping. People who start it later do so at their own risk. ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
Tom Eastep wrote:> Tom Eastep wrote: > >> Running /usr/sbin/iptables-restore... >> Setting up dynamic rules... >> IP Forwarding Enabled >> Processing /home/teastep/Configs/gateway/start ... >> Processing /home/teastep/Configs/gateway/started ... >> done. >> System gateway reloaded >> teastep@wifiursa:~/Configs/gateway> > > Here''s the patch to Shorewall-perl. Shorewall-shell patch forthcoming...Here are both patches (forgot to attach Perl patch the last time). -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
On Sun, 2008-03-30 at 16:30 +0100, Andrew Suffield wrote:> > Well, I was thinking more about filtering rules. But the point in the > boot process when shorewall starts should be before anything else that > uses the network, aside from the fundamental things like dhcp and > ping.Indeed, include PPP ip-up in that list. The thing is that a DHCP "lease acquired" script or a PPP ip-up script might want to go out to the Internet and update a DDNS service with the new address. As one example. Certainly, this could be done in a two stage process. Instead of doing the DDNS updates it could simply stash the data for a process later on in boot to do the updates.> People who start it later do so at their own risk.Or start trying to work with the network before Shorewall has done it''s thing, but yes, agreed. b. ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It''s the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace