Shorewall users - Oct 2011 - two interfaces with private Ip (rfc1918) on both side and dhcp issue

Hello all,

I''m using shorewall on a linux machine that has two interfaces, eth0 
being connected on the internal network (10.10.10.0/24) and eth1 being 
connected to the external network.
On eth0 the IP is statically configured to 10.10.10.254 and there is a 
dhcp server running for the machines in the private network.
On eth1, the IP is dynamically assigned by my ISP modem that acts as a 
bridge.

I have thus followed the "two interfaces" example which works just
fine
and I have this in the /etc/shorewall/interfaces file :

net    eth1    detect    dhcp
loc    eth0    detect    dhcp

and the following in the /etc/shorewall/masq file :

eth1    10.10.10.0/24

and obviously a few rules in the /etc/shorewall/rules file (port 
forwarding).
Everything is running ok and when eth1 gets plugged in, the kernel 
detects it and tells the dhcp client to get an ip address as can be seen 
here:

Oct 24 22:34:15 server kernel: e100: eth1 NIC Link is Up 100 Mbps Full 
Duplex
Oct 24 22:34:16 server dhclient: DHCPDISCOVER on eth1 to 255.255.255.255 
port 67 interval 12
Oct 24 22:34:16 server dhclient: DHCPOFFER from 192.0.2.254
Oct 24 22:34:16 server dhclient: DHCPREQUEST on eth1 to 255.255.255.255 
port 67
Oct 24 22:34:16 server dhclient: DHCPACK from 192.0.2.254
Oct 24 22:34:16 server dhclient: bound to 192.0.2.189 -- renewal in 
236700 seconds.
Oct 24 22:34:16 server ifplugd(eth1)[2223]: client: Determining IP 
information for eth1... done.
Oct 24 22:34:16 server ifplugd(eth1)[2223]: client: 192.0.2.189
Oct 24 22:34:16 server ifplugd(eth1)[2223]: Program executed successfully.

The problem that I''m having is that the ISP is gradually changing from 
the "bridge" mode to a "routed" mode which means that
instead of getting
a public IP address from the modem, I now get a private IP in the 
192.168.1.0/24
And in this new "routed" mode, the eth1 interface does not get an IP 
address, the dhcp response is filtered out. I have had a look in the log 
and what I get is this:

Oct 24 22:33:27 server kernel: e100: eth1 NIC Link is Up 100 Mbps Full 
Duplex
Oct 24 22:33:29 server dhclient: DHCPREQUEST on eth1 to 255.255.255.255 
port 67
Oct 24 22:33:29 server dhclient: DHCPNAK from 192.168.1.254
Oct 24 22:33:30 server dhclient: DHCPDISCOVER on eth1 to 255.255.255.255 
port 67 interval 5
Oct 24 22:33:30 server dhclient: DHCPOFFER from 192.168.1.254
Oct 24 22:33:30 server kernel: Shorewall:net2loc:DROP:IN=eth1 OUT=eth0 
SRC=192.168.1.254 DST=192.168.1.17 LEN=576 TOS=0x00 PREC=0x00 TTL=63 
ID=0 DF PROTO=UDP SPT=67 DPT=68 LEN=556
Oct 24 22:33:30 server dhclient: DHCPREQUEST on eth1 to 255.255.255.255 
port 67

Apparently the response appears to Shorewall as if it is directed to the 
"loc" zone despite this zone not being on the 192.168.1.0/24 subnet
I tried adding the following rules to /etc/shorewall/rules but to no avail:

accept    net    $FW    udp    67
accept    net    $FW    udp    68
accept    net    loc    udp    67
accept    net    loc    udp    68

The ISP''s modem configuration also allows to force all DHCP responses
to
have their source address as the broadcast address, but this does not 
help, it looks as if it''s even worse:

Oct 24 22:29:43 server kernel: e100: eth1 NIC Link is Up 100 Mbps Full 
Duplex
Oct 24 22:29:47 server dhclient: DHCPREQUEST on eth1 to 255.255.255.255 
port 67
Oct 24 22:29:47 server kernel: martian source 255.255.255.255 from 
192.168.1.254, on dev eth1
Oct 24 22:29:47 server kernel: ll header: 
ff:ff:ff:ff:ff:ff:f4:ca:e5:46:db:64:08:00
Oct 24 22:29:47 server dhclient: DHCPNAK from 192.168.1.254
Oct 24 22:29:48 server dhclient: DHCPDISCOVER on eth1 to 255.255.255.255 
port 67 interval 6
Oct 24 22:29:48 server dhclient: DHCPOFFER from 192.168.1.254
Oct 24 22:29:48 server dhclient: DHCPREQUEST on eth1 to 255.255.255.255 
port 67
Oct 24 22:29:48 server kernel: martian source 255.255.255.255 from 
192.168.1.254, on dev eth1
Oct 24 22:29:48 server kernel: ll header: 
ff:ff:ff:ff:ff:ff:f4:ca:e5:46:db:64:08:00

I must be missing something obvious in my configuration, but when I 
first activated the "routed" mode on the modem, I naively thought that
it would work "out of the box" for the outgoing connections because of
the DHCP configuration for eth1. I knew I would have to add some port 
forwarding rules in the ISP modem, but that could be done later on, 
provided the "outgoing" connections were working.

Do you have any idea what I have missed?
Anything I should try?
Do you need more information?

Thanks in advance for your help

Olivier

------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn 
about Cisco certifications, training, and career opportunities. 
http://p.sf.net/sfu/cisco-dev2dev

On Tue, 2011-10-25 at 11:05 +0200, OBones wrote:
> 
> I must be missing something obvious in my configuration, but when I 
> first activated the "routed" mode on the modem, I naively thought
that
> it would work "out of the box" for the outgoing connections
because of
> the DHCP configuration for eth1. I knew I would have to add some port 
> forwarding rules in the ISP modem, but that could be done later on, 
> provided the "outgoing" connections were working.
> 
> Do you have any idea what I have missed?
> Anything I should try?
> Do you need more information?
Please post the output of these two commands:

- ip addr ls
- ip route ls

Thanks,
-Tom
-- 
Tom Eastep        \ When I die, I want to go like my Grandfather who
Shoreline,         \ died peacefully in his sleep. Not screaming like
Washington, USA     \ all of the passengers in his car
http://shorewall.net \________________________________________________



------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn 
about Cisco certifications, training, and career opportunities. 
http://p.sf.net/sfu/cisco-dev2dev

Tom Eastep wrote:
> On Tue, 2011-10-25 at 11:05 +0200, OBones wrote:
>
>> Do you need more information?
> Please post the output of these two commands:
>
> - ip addr ls
> - ip route ls
Sure enough, here are the outputs:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
     inet 127.0.0.1/8 scope host lo
     inet6 ::1/128 scope host
        valid_lft forever preferred_lft forever
2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast 
state UP qlen 1000
     link/ether 00:90:27:ca:ba:e8 brd ff:ff:ff:ff:ff:ff
     inet 192.0.2.189/24 brd 192.0.2.255 scope global eth1
     inet6 fe80::290:27ff:feca:bae8/64 scope link
        valid_lft forever preferred_lft forever
3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast 
state UNKNOWN qlen 1000
     link/ether 00:14:85:c0:ea:2b brd ff:ff:ff:ff:ff:ff
     inet 10.10.10.254/24 brd 10.10.10.255 scope global eth0
     inet6 fe80::214:85ff:fec0:ea2b/64 scope link
        valid_lft forever preferred_lft forever


192.0.2.0/24 dev eth1  proto kernel  scope link  src 192.0.2.189  metric 10
10.10.10.0/24 dev eth0  proto kernel  scope link  src 10.10.10.254  
metric 10
169.254.0.0/16 dev eth1  scope link  metric 10
169.254.0.0/16 dev eth0  scope link  metric 10
127.0.0.0/8 dev lo  scope link
default via 192.0.2.254 dev eth1  metric 10
default via 10.10.10.254 dev eth0  metric 10

Hope this helps

Regards
Olivier

------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn 
about Cisco certifications, training, and career opportunities. 
http://p.sf.net/sfu/cisco-dev2dev

On Tue, 2011-10-25 at 15:13 +0200, OBones wrote:
> 
> 192.0.2.0/24 dev eth1  proto kernel  scope link  src 192.0.2.189  metric 10
> 10.10.10.0/24 dev eth0  proto kernel  scope link  src 10.10.10.254  
> metric 10
> 169.254.0.0/16 dev eth1  scope link  metric 10
> 169.254.0.0/16 dev eth0  scope link  metric 10
> 127.0.0.0/8 dev lo  scope link
> default via 192.0.2.254 dev eth1  metric 10
> default via 10.10.10.254 dev eth0  metric 10
Get rid of the default route out of eth0 -- a simple configuration like
yours should have exactly one default route - out of the ''net''
interface.

-Tom
-- 
Tom Eastep        \ When I die, I want to go like my Grandfather who
Shoreline,         \ died peacefully in his sleep. Not screaming like
Washington, USA     \ all of the passengers in his car
http://shorewall.net \________________________________________________



------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn 
about Cisco certifications, training, and career opportunities. 
http://p.sf.net/sfu/cisco-dev2dev

Tom Eastep wrote:
> On Tue, 2011-10-25 at 15:13 +0200, OBones wrote:
>
>> 192.0.2.0/24 dev eth1  proto kernel  scope link  src 192.0.2.189 
metric 10
>> 10.10.10.0/24 dev eth0  proto kernel  scope link  src 10.10.10.254
>> metric 10
>> 169.254.0.0/16 dev eth1  scope link  metric 10
>> 169.254.0.0/16 dev eth0  scope link  metric 10
>> 127.0.0.0/8 dev lo  scope link
>> default via 192.0.2.254 dev eth1  metric 10
>> default via 10.10.10.254 dev eth0  metric 10
> Get rid of the default route out of eth0 -- a simple configuration like
> yours should have exactly one default route - out of the
''net''
> interface.
Thanks, I removed the default route on eth0. For reference to others, 
this was done with the following command:

route del default gw 10.10.10.254 eth0

this worked, the route is no longer there.
Not being near the firewall at the moment, I can''t test the
"routed"
mode for the modem just yet but will try tonight and keep everyone posted.

However, I have one more question:
This default route is not something I remember having configured, to me 
it gets added every time I reboot the firewall. I looked around and 
found that it could come from /etc/sysconfig/network-scripts/ifcfg-eth0
Can you confirm that I need to remove the "GATEWAY=10.10.10.254" line 
from /etc/sysconfig/network-scripts/ifcfg-eth0 in order to prevent the 
default route from being added on every boot?
I know I could issue the above "route del" command in
/etc/rc.d/rc.local
and then restart shorewall, but I would like a nicer solution if there 
is one.

Regards
Olivier

------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn 
about Cisco certifications, training, and career opportunities. 
http://p.sf.net/sfu/cisco-dev2dev

On Tue, 2011-10-25 at 16:01 +0200, OBones wrote:
> Tom Eastep wrote:
> > Get rid of the default route out of eth0 -- a simple configuration
like
> > yours should have exactly one default route - out of the
''net''
> > interface.
> Thanks, I removed the default route on eth0. For reference to others, 
> this was done with the following command:
> 
> route del default gw 10.10.10.254 eth0
> 
> this worked, the route is no longer there.
> Not being near the firewall at the moment, I can''t test the
"routed"
> mode for the modem just yet but will try tonight and keep everyone posted.
> 
> However, I have one more question:
> This default route is not something I remember having configured, to me 
> it gets added every time I reboot the firewall. I looked around and 
> found that it could come from /etc/sysconfig/network-scripts/ifcfg-eth0
> Can you confirm that I need to remove the "GATEWAY=10.10.10.254"
line
> from /etc/sysconfig/network-scripts/ifcfg-eth0 in order to prevent the 
> default route from being added on every boot?
That''s correct. You probably added that default route when you
configured the device using your distribution''s GUI. 

-Tom
-- 
Tom Eastep        \ When I die, I want to go like my Grandfather who
Shoreline,         \ died peacefully in his sleep. Not screaming like
Washington, USA     \ all of the passengers in his car
http://shorewall.net \________________________________________________



------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn 
about Cisco certifications, training, and career opportunities. 
http://p.sf.net/sfu/cisco-dev2dev

Tom Eastep wrote:
> On Tue, 2011-10-25 at 16:01 +0200, OBones wrote:
>> Tom Eastep wrote:
>>> Get rid of the default route out of eth0 -- a simple configuration
like
>>> yours should have exactly one default route - out of the
''net''
>>> interface.
>> this worked, the route is no longer there.
>> Not being near the firewall at the moment, I can''t test the
"routed"
>> mode for the modem just yet but will try tonight and keep everyone
posted.
I was able to test that yesterday evening and while the DHCP reply was 
no longer filtered out (no shorewall drop rule logged), the dhcp client 
was not able to process the response and so the interface did not get a 
valid IP address.
I then looked in the configuration for the dhcp client, and despite this 
not being related to shorewall, I''m posting the solution here for a 
reference to others in the same situation.
The problem came from a bad configuration in /etc/dhclient-eth1.conf 
which contained this :

interface "eth1" {
     send dhcp-lease-time 2592000;
     supersede dhcp-server-identifier 255.255.255.255;
}

It''s the second line that made the problem happen when the modem is in 
the "routed" mode.
Once I commented out that directive (and restarted the dhcp daemon), the 
interface got its address just fine and outgoing connections worked 
straight away. Adding forwarding rules in the modem was the next step 
and once done, I was back to working conditions.
>> Can you confirm that I need to remove the
"GATEWAY=10.10.10.254" line
>> from /etc/sysconfig/network-scripts/ifcfg-eth0 in order to prevent the
>> default route from being added on every boot?
> That''s correct. You probably added that default route when you
> configured the device using your distribution''s GUI.
That must have been it, I have removed it now and it worked just fine.

Once again, many thanks for your help.

Regards
Olivier

------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn 
about Cisco certifications, training, and career opportunities. 
http://p.sf.net/sfu/cisco-dev2dev