Shorewall users - Nov 2005 - Shorewall with multiple providers - basically three-in-one revisited

I have almost finished setting up everything I want with multiple providers.
Right now I''m using fwmarks to send traffic from 192.168.0.254 through
eth1,
traffic from 192.168.0.253 through eth2, and all other traffic through eth3.
There is only one thing that I still want to be able to do:
I have DNATs set up so people from outside can connect to servers running on
192.168.0.254 and 192.168.0.253.  However, when someone from the inside
attempts to connect to the IP addresses of eth1 or eth2, there is no policy
in place that will foreword the connection request to 192.168.0.254 or
192.168.0.253.  What I would like to happen is for requests from
192.168.0.2-192.168.0.252 to actually travel out through eth3, back in
through eth1 or eth2, and then be forwarded to the appropriate internal
server.  Basically, I don''t want there to be any way for traffic
originating
on 192.168.0.2-192.168.0.252 which is destined for eth1 or eth2 to shortcut
through the firewall without actually traveling over the internet.  I just
don''t know where to define this type of action or how to define it.  If
that''s not possible, an alternative I would like to implement would be
to
re-write the source and destination for packets originating from 
192.168.0.2-252 so that it appears that they originated from eth3''s IP 
address and then foreword them to 192.168.0.253 or 254.  If anyone has any 
ideas on how I might go about implementing either of these solutions, I 
would much appreciate your help.

Thanks,
Russel Riley


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click

----- Original Message ----- 
> I have almost finished setting up everything I want with multiple
providers.
> Right now I''m using fwmarks to send traffic from 192.168.0.254
through eth1,
> traffic from 192.168.0.253 through eth2, and all other traffic through
eth3.
> There is only one thing that I still want to be able to do:
> I have DNATs set up so people from outside can connect to servers running
on
> 192.168.0.254 and 192.168.0.253.  However, when someone from the inside
> attempts to connect to the IP addresses of eth1 or eth2, there is no policy
> in place that will foreword the connection request to 192.168.0.254 or
> 192.168.0.253.  What I would like to happen is for requests from
> 192.168.0.2-192.168.0.252 to actually travel out through eth3, back in
> through eth1 or eth2, and then be forwarded to the appropriate internal
> server.  Basically, I don''t want there to be any way for traffic
originating
> on 192.168.0.2-192.168.0.252 which is destined for eth1 or eth2 to shortcut
> through the firewall without actually traveling over the internet.  I just
> don''t know where to define this type of action or how to define
it.  If
> that''s not possible, an alternative I would like to implement
would be to
> re-write the source and destination for packets originating from 
> 192.168.0.2-252 so that it appears that they originated from
eth3''s IP
> address and then foreword them to 192.168.0.253 or 254.  If anyone has any 
> ideas on how I might go about implementing either of these solutions, I 
> would much appreciate your help.
> 
> Thanks,
> Russel Riley
> 
Russel:
Sounds like your just missing a small piece of the picture somewhere, could you
post your config files, just so we know where your current config is at so far.

Jerry





-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_idv28&alloc_id845&op=click

Now, first I think you should tell us WHY you can''t have the local
traffic short-cutting through the firewall -- maybe we can think of
another way to achieve your ultimate goal.

But the simplest way to do what you suggest, I think, would be to:

1) Put the two servers on a separate subnet, such as 192.168.2.253 & 254.
and
2) Have two firewalls, one for the 192.168.2.xxx subnet, and one for
the 192.168.0.xxx one.


Rune

On 11/21/05, Russel <rusabus@hotmail.com> wrote:
> I have almost finished setting up everything I want with multiple
providers.
> Right now I''m using fwmarks to send traffic from 192.168.0.254
through eth1,
> traffic from 192.168.0.253 through eth2, and all other traffic through
eth3.
> There is only one thing that I still want to be able to do:
> I have DNATs set up so people from outside can connect to servers running
on
> 192.168.0.254 and 192.168.0.253.  However, when someone from the inside
> attempts to connect to the IP addresses of eth1 or eth2, there is no policy
> in place that will foreword the connection request to 192.168.0.254 or
> 192.168.0.253.  What I would like to happen is for requests from
> 192.168.0.2-192.168.0.252 to actually travel out through eth3, back in
> through eth1 or eth2, and then be forwarded to the appropriate internal
> server.  Basically, I don''t want there to be any way for traffic
originating
> on 192.168.0.2-192.168.0.252 which is destined for eth1 or eth2 to shortcut
> through the firewall without actually traveling over the internet.  I just
> don''t know where to define this type of action or how to define
it.  If
> that''s not possible, an alternative I would like to implement
would be to
> re-write the source and destination for packets originating from
> 192.168.0.2-252 so that it appears that they originated from
eth3''s IP
> address and then foreword them to 192.168.0.253 or 254.  If anyone has any
> ideas on how I might go about implementing either of these solutions, I
> would much appreciate your help.
>
> Thanks,
> Russel Riley
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
> Register for a JBoss Training Course.  Free Certification Exam
> for All Training Attendees Through End of 2005. For more info visit:
> http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
> _______________________________________________
> Shorewall-users mailing list
> Shorewall-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/shorewall-users
>

-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_idv28&alloc_id845&op=click

On Monday 21 November 2005 10:46, Rune Kock wrote:
> Now, first I think you should tell us WHY you can''t have the local
> traffic short-cutting through the firewall -- maybe we can think of
> another way to achieve your ultimate goal.
>
> But the simplest way to do what you suggest, I think, would be to:
>
> 1) Put the two servers on a separate subnet, such as 192.168.2.253 &
254.
> and
> 2) Have two firewalls, one for the 192.168.2.xxx subnet, and one for
> the 192.168.0.xxx one.
Split DNS also comes to mind.

-Tom
-- 
Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net
PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key

>> I have almost finished setting up everything I want with multiple
providers.
>> Right now I''m using fwmarks to send traffic from 192.168.0.254
through
>> eth1, traffic from 192.168.0.253 through eth2, and all other traffic
through eth3.
>> There is only one thing that I still want to be able to do:
>> I have DNATs set up so people from outside can connect to servers 
>> running on
>> 192.168.0.254 and 192.168.0.253.  However, when someone from the 
>> inside attempts to connect to the IP addresses of eth1 or eth2, there 
>> is no policy in place that will foreword the connection request to 
>> 192.168.0.254 or 192.168.0.253.  What I would like to happen is for 
>> requests from
>> 192.168.0.2-192.168.0.252 to actually travel out through eth3, back in 
>> through eth1 or eth2, and then be forwarded to the appropriate 
>> internal server.  Basically, I don''t want there to be any way
for
>> traffic originating on 192.168.0.2-192.168.0.252 which is destined for 
>> eth1 or eth2 to shortcut through the firewall without actually 
>> traveling over the internet.  I just don''t know where to
define this
>> type of action or how to define it.  If that''s not possible,
an
>> alternative I would like to implement would be to re-write the source 
>> and destination for packets originating from
>> 192.168.0.2-252 so that it appears that they originated from
eth3''s IP
>> address and then foreword them to 192.168.0.253 or 254.  If anyone has 
>> any ideas on how I might go about implementing either of these 
>> solutions, I would much appreciate your help.
>> 
>> Thanks,
>> Russel Riley
>> 
>Russel:
>Sounds like your just missing a small piece of the picture somewhere, could
>you post your config files, just so we know where your current config is at
>so far.
>Jerry
Jerry:
I have attached my configuration files
>Now, first I think you should tell us WHY you can''t have the local
traffic
>short-cutting through the firewall -- maybe we can think of another way to
>achieve your ultimate goal.
Rune, here''s why: I host large LAN parties (sometimes up to 170 people,
but
usually fewer than 20).  Mostly I do the smaller parties from my home.  Many
times the guests like to play Starcraft on Battle.net.  In Starcraft,
Battle.net does not allow multiple users to join the same game without
having separate external IP addresses.  So, whenever players want to play
Starcraft, I have to run new network cables, bypassing my firewall, reset my
cable modem, restart my Linux networking services, restart shorewall, have
my guests release and renew their IP leases, and then troubleshoot for 10
minutes before they can join the same game.  It''s all a big hassle
(especially to the people playing other games who lose their connections for
10-20 minutes) not to mention a security risk (because I don''t want to
spend
20 minutes checking each person''s computer to see if they have a
personal
firewall, etc.)  It may be possible to allow traffic to shortcut through the
firewall, but I don''t know what battle.net expects the traffic to do,
so I
don''t know if that would even work.  With multiple providers, I can
just
configure my DHCP server so that certain players each get assigned to a
different provider (and hence a different IP address) and then join the same
game.  I know it''s all a bit silly to go to such trouble for a game,
but
it''s much easier than the alternative described above.

I also thought of using multiple firewall/routers, but I don''t have the
resources for two more machines.  Hence the "basically three-in-one."

Any help is much appreciated
Thanks
Russel Riley

-- 
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.362 / Virus Database: 267.13.4/176 - Release Date: 11/20/2005

----- Original Message ----- 
> >> I have DNATs set up so people from outside can connect to servers 
> >> running on
> >> 192.168.0.254 and 192.168.0.253.  However, when someone from the 
> >> inside attempts to connect to the IP addresses of eth1 or eth2,
there
> >> is no policy in place that will foreword the connection request to
> >> 192.168.0.254 or 192.168.0.253.  What I would like to happen is
for
> >> requests from
> >> 192.168.0.2-192.168.0.252 to actually travel out through eth3,
back in
> >> through eth1 or eth2, and then be forwarded to the appropriate 
> >> internal server.  Basically, I don''t want there to be any
way for
> >> traffic originating on 192.168.0.2-192.168.0.252 which is destined
for
> >> eth1 or eth2 to shortcut through the firewall without actually 
> >> traveling over the internet.  I just don''t know where to
define this
> >> type of action or how to define it.  If that''s not
possible, an
> >> alternative I would like to implement would be to re-write the
source
> >> and destination for packets originating from
> >> 192.168.0.2-252 so that it appears that they originated from
eth3''s IP
> >> address and then foreword them to 192.168.0.253 or 254.  If anyone
has
> >> any ideas on how I might go about implementing either of these 
> >> solutions, I would much appreciate your help.
If your receiving inbound connections, then "track" as an option in
the providers file is a
good idea. The "routeback" on eth1,2,3 in interfaces is not needed,
but maybe needed
on eth0, more on that in a bit.

When you using multi-isp it is a good idea to use snat in the masq file, by
adding
the desired external ip to the address column, replace <pub-ip> with the
required info.
eth1   192.168.0.254/32 <pub-ip>

Your dnat rules might need to have the original destination column used, which
is the external ip of that connection.   
DNAT net:eth1        loc:192.168.0.254     tcp 6112:6119 <pub-ip>

To make like easier when a <pub-ip> gets changed by dhcp, use the params
file to define a
variable and then use the variable where ever you would use that ip.

Your issue is kind of like http://www.shorewall.net/FAQ.htm#faq1d, without the
dmz zone,
and http://www.shorewall.net/FAQ.htm#faq2. Might need dnat rules for your
loc2loc traffic,
along with playing around with the masq file, but start with the above, before
we go further.
Can machines in 192.168.0.2-252, ping the external ip address of eth1 and eth2?
Please
summit a "shorewall dump", that give a pile more info that a
"status"

Jerry











-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_idv28&alloc_id845&op=click

Hi Russel

Okay, if I understand you correctly:

You have gotten yourself two extra internet lines that are only used
to allow two extra people to play on Battle.net.  Can''t they just play
a LAN-game without going through Battle.net?

Is it possible to get extra public IP-addresses on your primary line instead?

What you called "servers" in your original post are really two
game-computers (clients, not servers) that need to be separated from
the rest(?)
> I also thought of using multiple firewall/routers, but I don''t
have the
> resources for two more machines.  Hence the "basically
three-in-one."
I still think that extra routers would make your configuration a lot
simpler.  How about going to the local junk-yard and find some old
Pentiums?  Just about any computer with 32MiB ram should run a command
line Debian.  Or you might not even need Linux-routers on the
secondary lines -- $30 discount routers might do.



Rune

On 11/22/05, Russel <rusabus@hotmail.com> wrote:
> >> I have almost finished setting up everything I want with multiple
> providers.
> >> Right now I''m using fwmarks to send traffic from
192.168.0.254 through
> >> eth1, traffic from 192.168.0.253 through eth2, and all other
traffic
> through eth3.
> >> There is only one thing that I still want to be able to do:
> >> I have DNATs set up so people from outside can connect to servers
> >> running on
> >> 192.168.0.254 and 192.168.0.253.  However, when someone from the
> >> inside attempts to connect to the IP addresses of eth1 or eth2,
there
> >> is no policy in place that will foreword the connection request to
> >> 192.168.0.254 or 192.168.0.253.  What I would like to happen is
for
> >> requests from
> >> 192.168.0.2-192.168.0.252 to actually travel out through eth3,
back in
> >> through eth1 or eth2, and then be forwarded to the appropriate
> >> internal server.  Basically, I don''t want there to be any
way for
> >> traffic originating on 192.168.0.2-192.168.0.252 which is destined
for
> >> eth1 or eth2 to shortcut through the firewall without actually
> >> traveling over the internet.  I just don''t know where to
define this
> >> type of action or how to define it.  If that''s not
possible, an
> >> alternative I would like to implement would be to re-write the
source
> >> and destination for packets originating from
> >> 192.168.0.2-252 so that it appears that they originated from
eth3''s IP
> >> address and then foreword them to 192.168.0.253 or 254.  If anyone
has
> >> any ideas on how I might go about implementing either of these
> >> solutions, I would much appreciate your help.
> >>
> >> Thanks,
> >> Russel Riley
> >>
> >Russel:
> >Sounds like your just missing a small piece of the picture somewhere,
could
>
> >you post your config files, just so we know where your current config
is at
> >so far.
>
> >Jerry
>
> Jerry:
> I have attached my configuration files
>
> >Now, first I think you should tell us WHY you can''t have the
local traffic
> >short-cutting through the firewall -- maybe we can think of another way
to
> >achieve your ultimate goal.
>
> Rune, here''s why: I host large LAN parties (sometimes up to 170
people, but
> usually fewer than 20).  Mostly I do the smaller parties from my home. 
Many
> times the guests like to play Starcraft on Battle.net.  In Starcraft,
> Battle.net does not allow multiple users to join the same game without
> having separate external IP addresses.  So, whenever players want to play
> Starcraft, I have to run new network cables, bypassing my firewall, reset
my
> cable modem, restart my Linux networking services, restart shorewall, have
> my guests release and renew their IP leases, and then troubleshoot for 10
> minutes before they can join the same game.  It''s all a big hassle
> (especially to the people playing other games who lose their connections
for
> 10-20 minutes) not to mention a security risk (because I don''t
want to spend
> 20 minutes checking each person''s computer to see if they have a
personal
> firewall, etc.)  It may be possible to allow traffic to shortcut through
the
> firewall, but I don''t know what battle.net expects the traffic to
do, so I
> don''t know if that would even work.  With multiple providers, I
can just
> configure my DHCP server so that certain players each get assigned to a
> different provider (and hence a different IP address) and then join the
same
> game.  I know it''s all a bit silly to go to such trouble for a
game, but
> it''s much easier than the alternative described above.
>
> I also thought of using multiple firewall/routers, but I don''t
have the
> resources for two more machines.  Hence the "basically
three-in-one."
>
> Any help is much appreciated
> Thanks
> Russel Riley
>
> --
> No virus found in this outgoing message.
> Checked by AVG Free Edition.
> Version: 7.1.362 / Virus Database: 267.13.4/176 - Release Date: 11/20/2005
>
>
>
>
>

-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_idv28&alloc_id845&op=click

Hi,

I just subscribed to this list, so forgive me if the question has altready
been asked and answered. I did not look at the list archives, but
before posting this question I read the relevant documentation, the
FAQ, the "providers" file format, followed the advice about setting
MARK_IN_FORWARD_CHAIN=Yes in shorewall.conf. I also upgraded
iptables to the latest development release available (I''m using FC4)
and verified that the required patch is included. My kernel is:
2.6.14-1.1637_FC4smp .

The machine acts as a firewall with multiple interfaces; they are assigned
this way:

eth0    main interface to Internet (ADSL, router, multiple IP addresses)
eth1    local network
eth2    dmz
eth3    crossover cable to another server, for backup
eth4    secondary interface to Internet (ADSL, router, multiple IP addresses)

The secondary interface should act as a backup when the primary fails,
and can also be useful for load balancing for outgoing connections.
On the primary there are two additional IP addresses with static NAT
active to two loc and dmz machines; they are active onty on eth0.
The firewall acts also as the MX for some domains, and I plan to
publish its IP address on eth4 as a secondary MX.

The providers file contains these lines:

ISP1    1       1       main            eth0            85.xxx.xxx.xxx   
track,balance
eth1,eth2
ISP2    2       2       main            eth4            80.xxx.xxx.xxx   
track,balance
eth1,eth2

I verified that the firewall is reachable from outside through the addresses on
both eth0 and eth4, therefore the above lines seem to work as expected.

The problem arises when I test the failover. I try to disconnect eth0 or eth4,
and
I expect to have any new connection routed to the interface remaining active.
Sometimes (rarely) this works, and a traceroute correctly shows that a route
through the cative router is selected. Most of the times, however, it looks like
the previously used route is maintained, and the remote host is unreachable
until the disconnected cable is reconnected.

My question is: did I misunderstand when thinking that this setup can also be
used for failover in case of failure on one of the external interfaces? Actually
I did not find any document stating that this was possible, but I assumed that
it was obviously possible.

Thaks a lot for any help

Elio



-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click

----- Original Message ----- 
> The secondary interface should act as a backup when the primary fails,
> and can also be useful for load balancing for outgoing connections.
><snip> 
> The problem arises when I test the failover. I try to disconnect eth0 or
eth4, and
> I expect to have any new connection routed to the interface remaining
active.
> Sometimes (rarely) this works, and a traceroute correctly shows that a
route
> through the cative router is selected. Most of the times, however, it looks
like
> the previously used route is maintained, and the remote host is unreachable
> until the disconnected cable is reconnected.
> 
> My question is: did I misunderstand when thinking that this setup can also
be
> used for failover in case of failure on one of the external interfaces?
Actually
> I did not find any document stating that this was possible, but I assumed
that
> it was obviously possible.
You misunderstood, failover is not automatic out of the box. There are ways 
to monitor and change the config or speedup the trying of the alternate gateway.
Check the mail archives, Friday, September 30, 2005
Re: [Shorewall-users] shorewall + Squid + Two ISP setup
Where I talk about some proc settings that you can play with, and John posted 
an monitoring script. 

Jerry





-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_idv28&alloc_id845&op=click

Hi Elio,


Julian Anastasov''s patches http://www.ssi.bg/~ja/#routes seems to have
something to do with this.  He calls it dead gateway detection.

But his descriptions are hard to follow.  I''m afraid I don''t
really know
what the patches actually do.

Anyway, it''s not easy for the kernel to know whether an internet link
is
dead -- in most cases, the link to the ADSL-modem will still be up.  So
maybe you need some kind of daemon to ping a very stable server somewhere on
the net, and then take the interface down if no replies arrive?


Rune

On 11/22/05, Elio Tondo <elio@tondo.it> wrote:
>
> Hi,
>
> I just subscribed to this list, so forgive me if the question has altready
> been asked and answered. I did not look at the list archives, but
> before posting this question I read the relevant documentation, the
> FAQ, the "providers" file format, followed the advice about
setting
> MARK_IN_FORWARD_CHAIN=Yes in shorewall.conf. I also upgraded
> iptables to the latest development release available (I''m using
FC4)
> and verified that the required patch is included. My kernel is:
> 2.6.14-1.1637_FC4smp .
>
> The machine acts as a firewall with multiple interfaces; they are assigned
> this way:
>
> eth0    main interface to Internet (ADSL, router, multiple IP addresses)
> eth1    local network
> eth2    dmz
> eth3    crossover cable to another server, for backup
> eth4    secondary interface to Internet (ADSL, router, multiple IP
> addresses)
>
> The secondary interface should act as a backup when the primary fails,
> and can also be useful for load balancing for outgoing connections.
> On the primary there are two additional IP addresses with static NAT
> active to two loc and dmz machines; they are active onty on eth0.
> The firewall acts also as the MX for some domains, and I plan to
> publish its IP address on eth4 as a secondary MX.
>
> The providers file contains these lines:
>
> ISP1    1       1       main            eth0            85.xxx.xxx.xxx
>     track,balance
> eth1,eth2
> ISP2    2       2       main            eth4            80.xxx.xxx.xxx
>     track,balance
> eth1,eth2
>
> I verified that the firewall is reachable from outside through the
> addresses on
> both eth0 and eth4, therefore the above lines seem to work as expected.
>
> The problem arises when I test the failover. I try to disconnect eth0 or
> eth4, and
> I expect to have any new connection routed to the interface remaining
> active.
> Sometimes (rarely) this works, and a traceroute correctly shows that a
> route
> through the cative router is selected. Most of the times, however, it
> looks like
> the previously used route is maintained, and the remote host is
> unreachable
> until the disconnected cable is reconnected.
>
> My question is: did I misunderstand when thinking that this setup can also
> be
> used for failover in case of failure on one of the external interfaces?
> Actually
> I did not find any document stating that this was possible, but I assumed
> that
> it was obviously possible.
>
> Thaks a lot for any help
>
> Elio
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
> Register for a JBoss Training Course.  Free Certification Exam
> for All Training Attendees Through End of 2005. For more info visit:
> http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
> _______________________________________________
> Shorewall-users mailing list
> Shorewall-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/shorewall-users
>

> If your receiving inbound connections, then "track" as an option
in the
> providers file is a 
> good idea. The "routeback" on eth1,2,3 in interfaces is not
needed, but
> maybe needed 
> on eth0, more on that in a bit.
I have removed the routeback from eth1,2,3.  However, the Shorewall
documentation states "Packet marking for traffic control purposes may not
be
done in the PREROUTING table for connections involving providers with
''track'' specified (see below)."  I am using the
prerouting table, and
shorewall fails to start with the track option specified.  Here is the error
message: 
Processing /etc/shorewall/providers...
   Provider CPE1 1 1 main eth1 detect track,balance eth0,eth4 Added
   Provider CPE2 2 2 main eth2 detect track,balance eth0,eth4 Added
   Provider CPE3 3 3 main eth3 detect track,balance eth0,eth4 Added
   Default route  nexthop via 72.24.8.1 dev eth1 weight 1 nexthop via
24.119.1.1 dev eth2 weight 1 nexthop via 24.117.144.1 dev eth3 weight 1
Added.
iptables: No chain/target/match by that name
   ERROR: Command "/sbin/iptables -t mangle -A PREROUTING -m connmark !
--mark 0 -j CONNMARK --restore-mark" Failed

Do I need to change some kernel options or is this expected?
> When you using multi-isp it is a good idea to use snat in the masq file, 
> by adding
> the desired external ip to the address column, replace <pub-ip> with
the
> required info.
> eth1   192.168.0.254/32 <pub-ip>
Done.
> Your dnat rules might need to have the original destination column used, 
> which is the external ip of that connection.   
> DNAT net:eth1        loc:192.168.0.254     tcp 6112:6119 <pub-ip>
Done.
> To make like easier when a <pub-ip> gets changed by dhcp, use the
params
> file to define a variable and then use the variable where ever you would 
> use that ip.
Done.
> Your issue is kind of like http://www.shorewall.net/FAQ.htm#faq1d, without
> the dmz zone, and http://www.shorewall.net/FAQ.htm#faq2. Might need dnat 
> rules for your loc2loc traffic, along with playing around with the masq 
> file, but start with the above, before we go further. 
I have tried some DNAT rules similar to what is shown in these FAQs, but
with no success.  They''re commented out now, but still in the rules
file.
> Can machines in 192.168.0.2-252, ping the external ip address of eth1 and
> eth2? 
Yes, all machines in the loc zone can ping all interfaces on the firewall,
but I have an "ACCEPT loc fw icmp 8" rule to allow that.  If I remove
that
rule, I can''t ping any firewall interfaces.
> Please summit a "shorewall dump", that give a pile more info that
a
> "status"
A shorewall dump is attached, along with all updated configuration files.
It seems to me that all I need to make things work should be some DNAT
rules, but I just haven''t been able to nail down the syntax.

Thanks,
Russel Riley





-- 
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.362 / Virus Database: 267.13.5/177 - Release Date: 11/21/2005

Your error is because your kernel or iptables does not support
connection marking (connmark).  IIRC, this was added to the kernel in
version 2.6.12, it was a patch in the netfilter patch-o-matic before
that.  Of course, even if you have 2.6.12, that option might have been
disabled in the compilation.

As to iptables, I would recommend 1.3.2 or later, as Tom found a bug
in 1.3.1 -- I think that iptables detects at compile time whether the
kernel supports connmark, so you may need to recompile that as well.


From your dump:

Shorewall has detected the following iptables/netfilter capabilities:
   NAT: Available
   Packet Mangling: Available
   Multi-port Match: Available
   Extended Multi-port Match: Not available
   Connection Tracking Match: Available
   Packet Type Match: Available
   Policy Match: Not available
   Physdev Match: Not available
   IP range Match: Not available
   Recent Match: Available
   Owner Match: Available
   Ipset Match: Not available
   CONNMARK Target: Not available
   Connmark Match: Not available
   Raw Table: Not available
   CLASSIFY Target: Not available


Rune


On 11/23/05, Russel <rusabus@hotmail.com> wrote:
> > If your receiving inbound connections, then "track" as an
option in the
> > providers file is a
> > good idea. The "routeback" on eth1,2,3 in interfaces is not
needed, but
> > maybe needed
> > on eth0, more on that in a bit.
>
> I have removed the routeback from eth1,2,3.  However, the Shorewall
> documentation states "Packet marking for traffic control purposes may
not be
> done in the PREROUTING table for connections involving providers with
> ''track'' specified (see below)."  I am using the
prerouting table, and
> shorewall fails to start with the track option specified.  Here is the
error
> message:
> Processing /etc/shorewall/providers...
>    Provider CPE1 1 1 main eth1 detect track,balance eth0,eth4 Added
>    Provider CPE2 2 2 main eth2 detect track,balance eth0,eth4 Added
>    Provider CPE3 3 3 main eth3 detect track,balance eth0,eth4 Added
>    Default route  nexthop via 72.24.8.1 dev eth1 weight 1 nexthop via
> 24.119.1.1 dev eth2 weight 1 nexthop via 24.117.144.1 dev eth3 weight 1
> Added.
> iptables: No chain/target/match by that name
>    ERROR: Command "/sbin/iptables -t mangle -A PREROUTING -m connmark
!
> --mark 0 -j CONNMARK --restore-mark" Failed
>
> Do I need to change some kernel options or is this expected?
>
> > When you using multi-isp it is a good idea to use snat in the masq
file,
> > by adding
> > the desired external ip to the address column, replace <pub-ip>
with the
> > required info.
> > eth1   192.168.0.254/32 <pub-ip>
>
> Done.
>
> > Your dnat rules might need to have the original destination column
used,
> > which is the external ip of that connection.
> > DNAT net:eth1        loc:192.168.0.254     tcp 6112:6119
<pub-ip>
>
> Done.
>
> > To make like easier when a <pub-ip> gets changed by dhcp, use
the params
> > file to define a variable and then use the variable where ever you
would
> > use that ip.
>
> Done.
>
> > Your issue is kind of like http://www.shorewall.net/FAQ.htm#faq1d,
without
> > the dmz zone, and http://www.shorewall.net/FAQ.htm#faq2. Might need
dnat
> > rules for your loc2loc traffic, along with playing around with the
masq
> > file, but start with the above, before we go further.
>
> I have tried some DNAT rules similar to what is shown in these FAQs, but
> with no success.  They''re commented out now, but still in the
rules file.
>
> > Can machines in 192.168.0.2-252, ping the external ip address of eth1
and
> > eth2?
>
> Yes, all machines in the loc zone can ping all interfaces on the firewall,
> but I have an "ACCEPT loc fw icmp 8" rule to allow that.  If I
remove that
> rule, I can''t ping any firewall interfaces.
>
> > Please summit a "shorewall dump", that give a pile more info
that a
> > "status"
>
> A shorewall dump is attached, along with all updated configuration files.
> It seems to me that all I need to make things work should be some DNAT
> rules, but I just haven''t been able to nail down the syntax.
>
> Thanks,
> Russel Riley
>
>
>
>
>
> --
> No virus found in this outgoing message.
> Checked by AVG Free Edition.
> Version: 7.1.362 / Virus Database: 267.13.5/177 - Release Date: 11/21/2005
>
>
>
>
>

-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_idv28&alloc_id845&op=click

----- Original Message ----- 
From: "Russel" <rusabus@hotmail.com>
To: <shorewall-users@lists.sourceforge.net>
Sent: Wednesday, November 23, 2005 00:42
Subject: [Shorewall-users] Shorewall with multiple providers - basically
three-in-one revisited

> > If your receiving inbound connections, then "track" as an
option in the
> > providers file is a 
> > good idea. The "routeback" on eth1,2,3 in interfaces is not
needed, but
> > maybe needed 
> > on eth0, more on that in a bit.
> 
> I have removed the routeback from eth1,2,3.  However, the Shorewall
> documentation states "Packet marking for traffic control purposes may
not be
> done in the PREROUTING table for connections involving providers with
> ''track'' specified (see below)."  I am using the
prerouting table, and
> shorewall fails to start with the track option specified.  Here is the
error
> message: 
> Processing /etc/shorewall/providers...
>    Provider CPE1 1 1 main eth1 detect track,balance eth0,eth4 Added
>    Provider CPE2 2 2 main eth2 detect track,balance eth0,eth4 Added
>    Provider CPE3 3 3 main eth3 detect track,balance eth0,eth4 Added
>    Default route  nexthop via 72.24.8.1 dev eth1 weight 1 nexthop via
> 24.119.1.1 dev eth2 weight 1 nexthop via 24.117.144.1 dev eth3 weight 1
> Added.
> iptables: No chain/target/match by that name
>    ERROR: Command "/sbin/iptables -t mangle -A PREROUTING -m connmark
!
> --mark 0 -j CONNMARK --restore-mark" Failed
> 
> Do I need to change some kernel options or is this expected?
> 
Guess you missed the "Use of this feature requires that your kernel 
and iptables support CONNMARK target and conntrack match support." 
part of the page.

Try:  shorewall show capabilities

You have a kernel problem: 
   CONNMARK Target: Not available  <<<<<
   Connmark Match: Not available      <<<<<

What version? and did you roll your own? if so your missing some options.

Jerry







-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_idv28&alloc_id845&op=click

This issue is now resolved, and I want to say thanks to everyone who helped.
I can now have three players in Starcraft on battle.net through one server.
Everything now works exactly as I had hoped.  The shorewall FAQ at this
location provided a basis for the configuration that finally worked:
http://www.shorewall.net/FAQ.htm#faq2 (thanks Jerry).  This was basically
adapted to match what I am doing.  Instead of making all traffic appear as
if it originated on the firewall, I made modifications to make it appear as
if traffic originated on the external interface that corresponded to each
internal IP address.  I had to modify a lot of the lines but the same basic
concept is there.  Here is what I did:
In masq I added these lines:
# From 253 to 254
eth0:192.168.0.254	192.168.0.253/32	$ETH2_IP tcp 6112:6119
eth0:192.168.0.254	192.168.0.253/32	$ETH2_IP udp 6112:6119
# From 253 to 252
eth0:192.168.0.252      192.168.0.253/32  $ETH2_IP tcp 6112:6119
eth0:192.168.0.252      192.168.0.253/32  $ETH2_IP udp 6112:6119
# From 254 to 253
eth0:192.168.0.253      192.168.0.254/32  $ETH1_IP tcp 6112:6119
eth0:192.168.0.253      192.168.0.254/32  $ETH1_IP udp 6112:6119
# From 254 to 252
eth0:192.168.0.252	192.168.0.254/32	$ETH1_IP tcp 6112:6119
eth0:192.168.0.252      192.168.0.254/32	$ETH1_IP udp 6112:6119
# From 252 to 254
eth0:192.168.0.254      192.168.0.252/32  $ETH3_IP tcp 6112:6119
eth0:192.168.0.254      192.168.0.252/32  $ETH3_IP udp 6112:6119
# From 252 to 253
eth0:192.168.0.253      192.168.0.252/32  $ETH3_IP tcp 6112:6119
eth0:192.168.0.253      192.168.0.252/32  $ETH3_IP udp 6112:6119

In rules I added these DNATs to handle all traffic between local hosts:
DNAT	loc:192.168.0.254 loc:192.168.0.253	tcp	6112:6119 - $ETH3_IP
DNAT	loc:192.168.0.254 loc:192.168.0.253	udp	6112:6119 - $ETH3_IP
DNAT  loc:192.168.0.254 loc:192.168.0.252 tcp   6112:6119 - $ETH2_IP
DNAT  loc:192.168.0.254 loc:192.168.0.252 udp   6112:6119 - $ETH2_IP
DNAT	loc:192.168.0.253 loc:192.168.0.254	tcp	6112:6119 - $ETH1_IP
DNAT	loc:192.168.0.253 loc:192.168.0.254	udp	6112:6119 - $ETH1_IP
DNAT  loc:192.168.0.253 loc:192.168.0.252 tcp   6112:6119 - $ETH3_IP
DNAT  loc:192.168.0.253 loc:192.168.0.252 udp   6112:6119 - $ETH3_IP
DNAT  loc:192.168.0.252 loc:192.168.0.254 tcp   6112:6119 - $ETH1_IP
DNAT  loc:192.168.0.252 loc:192.168.0.254 udp   6112:6119 - $ETH1_IP
DNAT  loc:192.168.0.252 loc:192.168.0.253 tcp   6112:6119 - $ETH2_IP
DNAT  loc:192.168.0.252 loc:192.168.0.253 udp   6112:6119 - $ETH2_IP

These DNATs allow for players on the internet to connect:
DNAT net:eth1 loc:192.168.0.254 tcp	6112:6119 - $ETH1_IP
DNAT net:eth1 loc:192.168.0.254 udp	6112:6119 - $ETH1_IP
DNAT net:eth2 loc:192.168.0.253 tcp	6112:6119 - $ETH2_IP
DNAT net:eth2 loc:192.168.0.253 udp 6112:6119 - $ETH2_IP
DNAT net:eth3 loc:192.168.0.252 tcp 6112:6119 - $ETH3_IP
DNAT net:eth3 loc:192.168.0.252 udp 6112:6119 - $ETH3_IP

In params I added these lines to define $ETHX_IP:
ETH1_IP=`find_first_interface_address eth1`
ETH2_IP=`find_first_interface_address eth2`
ETH3_IP=`find_first_interface_address eth3`

That''s the entire configuration that was specific to battle.net
hosting.
Traffic between internal clients doesn''t travel out the default
gateway, it
just shortcuts through the firewall and gets some modification along the way
so that the responses shortcut back as if they had gone through the gateway.

Again, thanks to everyone who helped me get this working.
Sincerely,
Russel Riley

-- 
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.362 / Virus Database: 267.13.7/182 - Release Date: 11/24/2005
 


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click

Russel:

Just to round out your config, could you post the rest of the config files?
Kind of need to have a complete set, for a full picture, might help someone 
else in the future. Glad you got it to work, for you. :-)

Thanks,
Jerry

----- Original Message ----- 
Subject: [Shorewall-users] Shorewall with multiple providers - basically
three-in-one *RESOLVED*





-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id865&op=click

> Russel:
>
> Just to round out your config, could you post the rest of the config 
> files?  Kind of need to have a complete set, for a full picture, might 
> help someone else in the future. Glad you got it to work, for you. :-)
>
> Thanks,
> Jerry
The config files are attached.

Thanks again,
Russel

-- 
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.362 / Virus Database: 267.13.8/183 - Release Date: 11/25/2005