Folks: I have a cluster of systems running OpenAFS client, behind a Linux system running shorewall, that need to communicate with a AFS servers in the internet zone. Masquerading get me most of the way, and works for every other service I need. AFS clients use UDP from port 7001 to communicate to servers, and get responses from these servers. Initially, everything works. But AFS servers also send callbacks to the clients (using whatever ports the server has received packets from), about every 5-10 minutes. The problem is that the NAT mappings time out in 3 minutes, and iptables has no way to increase this. One solution would be to write a connection tracking module, and I am pursuing that, but a faster way to get this working might be to have explicit rules in the firewall. Basically, what I want is, for each system in the cluster, to assign packets coming from UDP port 7001 to a different port on the firewall, and forward packets coming to that port to port 7001 on the appropriate server. DNAT rules, of course, can do this port forwarding. I use rules such as: DNAT net loc:10.0.1.1:7001 udp 17001 DNAT net loc:10.0.1.2:7001 udp 17002 But as far as I can tell, both because it doesn''t work, from my albeit incomplete understanding of how iptables works, and from what I see using ethereal, packets sent from 10.0.1.1:7001 are not automatically being translated to coming from firewall port 17001, but are using regularly generated NAT mappings like port 1825. I think what I need is an additional rule that will SNAT this one port. But I seem to not be able to find how to SNAT just one port from a server. How can I get this to work? Or where should I be reading to get a clue? -- R. Lindsay Todd email: toddr@rpi.edu Senior Systems Programmer phone: 518-276-2605 Rensselaer Polytechnic Institute fax: 518-276-2809 Troy, NY 12180-3590 WWW: http://www.rpi.edu/~toddr The views, opinions, and judgments expressed in this message are solely those of the author. The message contents have not been reviewed or approved by Rensselaer.
On Thu, 2003-10-02 at 11:59, R. Lindsay Todd wrote:> Folks: I have a cluster of systems running OpenAFS client, behind a > Linux system running shorewall, that need to communicate with a AFS > servers in the internet zone. Masquerading get me most of the way, and > works for every other service I need. > > AFS clients use UDP from port 7001 to communicate to servers, and get > responses from these servers. Initially, everything works. But AFS > servers also send callbacks to the clients (using whatever ports the > server has received packets from), about every 5-10 minutes. The > problem is that the NAT mappings time out in 3 minutes, and iptables has > no way to increase this.And there is no mechanism to send "keep-alives"?> > One solution would be to write a connection tracking module, and I am > pursuing that, but a faster way to get this working might be to have > explicit rules in the firewall. Basically, what I want is, for each > system in the cluster, to assign packets coming from UDP port 7001 to a > different port on the firewall, and forward packets coming to that port > to port 7001 on the appropriate server. > > DNAT rules, of course, can do this port forwarding. I use rules such as: > > DNAT net loc:10.0.1.1:7001 udp 17001 > DNAT net loc:10.0.1.2:7001 udp 17002 > > But as far as I can tell, both because it doesn''t work, from my albeit > incomplete understanding of how iptables works, and from what I see > using ethereal, packets sent from 10.0.1.1:7001 are not automatically > being translated to coming from firewall port 17001, but are using > regularly generated NAT mappings like port 1825.That is of course because DNAT rules have no effect on connections that are initiated in the opposite direction.> I think what I need is > an additional rule that will SNAT this one port. But I seem to not be > able to find how to SNAT just one port from a server. > > How can I get this to work? Or where should I be reading to get a clue?You need to hand-craft your own SNAT rules in /etc/shorewall/start -- you can insert them into the front of the <externalif>_masq chain. See http://shorewall.net/shorewall_extension_scripts.htm. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net
--- "R. Lindsay Todd" <toddr@rpi.edu> wrote:> AFS clients use UDP from port 7001 to communicate to servers, and get > responses from these servers. Initially, everything works. But AFS > servers also send callbacks to the clients (using whatever ports the > server has received packets from), about every 5-10 minutes. The > problem is that the NAT mappings time out in 3 minutes, and iptables has > no way to increase this.So it sounds as though if the AFS clients sit idle for more than 3 minutes iptables flushes the UDP mapping? I wasn''t aware of the specific iptables "mapping time" threshhold. Although I don''t know much about AFS, the first obvious choice to me (correct me if I''m wrong)would be to keep the connection alive, (because this seems like the problem to me) by at the very least, sending a ping packet every (minute and 1/2)from the clients to the servers until you can find out another way through AFS dev to figure out how to keep the connection up through an AFS dev app. If I were you I would try and contact someone/engineer through AFS (hopefully thier support is good)and let them know what your perdicment is. This way your fixing the problem through thier software and not ignoring it by coming up with chinsee work-arounds like I suggested. Just my thoughts. JBanks __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com
Folks:
Thanks much for the help. As Tom said, I needed to add rules in
the "start" script, which I did:
run_iptables -t nat -I eth0_masq -o eth0 -j SNAT -p udp -s $NODEADDR \
--sport 7001 --to-source $MYIP:$NODEPORT
Since this was a script, I was able to use a loop over each node. Also,
I created similar dnat entries directly in PREROUTING, rather than split
information between this script and DNAT rules. (I had no other dnat
entries, so had to use PREROUTING.)
To clarify a few comments people have made: Linux iptables times
out UDP mapping after about 180 seconds of being idle. You can hack the
appropriate kernel module, of course, but usually this is appropriate.
AFS is old software, written before NAT was used. It is a
stateful distributed file system using UDP for lower overhead, and
callbacks, initiated by the server, of 5 minutes or more to also reduce
overhead. Some folks have used a "natkeep" program to keep UDP
mappings
open -- this periodically sents packets with short TTL toward the
servers, solely to keep these mappings open. But this program is not
overly portable. There was some discussion on openafs-devel this past
month about the possibility of adding a keepalive to the client, only to
be used when behind a NAT firewall. I suspect that something like that
will eventually happen, but needed to get this going immediately.
Thanks again for the help!
/Lindsay
--
R. Lindsay Todd email: toddr@rpi.edu
Senior Systems Programmer phone: 518-276-2605
Rensselaer Polytechnic Institute fax: 518-276-2809
Troy, NY 12180-3590 WWW: http://www.rpi.edu/~toddr
The views, opinions, and judgments expressed in this message are
solely those of the author. The message contents have not been
reviewed or approved by Rensselaer.