Syslinux - Jul 2003 - tftp-hpa server with multiple network interfaces

Hello,

A host of mine which acts as a gateway for a cluster also acts as a dhcp
and tftp server for the cluster nodes. Recently, for reasons that are
not really important we changed the interfaces so that eth0 is conencted
to our production network and the eth1 network is connected to the
cluster's internal network. It was the opposite before. Normally, we
want most of the services running on this gateway to only be availble to
the internal network. For example, the dhcp and tftp server should only
respond to the internal network (cluster nodes). Making the dhcp server
do this was simple enough. We are having some trouble getting the tftp
server to do the same. In fact we cannot get the tftp server to respond
with anything other than timeouts on the internal network. These
timeouts occur from PXE boot clients (dhcp works for them) and using
command line tftp from an already booted client. tftp from the gateway
(tftp server) works. We are running tftp-hpa from xinetd whose config
file is included below.

service tftp
{
        socket_type     = dgram
        protocol        = udp
        wait            = yes
        user            = root
        server          = /usr/sbin/in.tftpd
        server_args     = -v -v -p -s /tftpboot
        only_from       = 192.168.0.0/24
        bind            = 192.168.0.1
        disable         = no
}

Here is the client tftp session:

tftp> connect 192.168.0.1
tftp> binary
tftp> get pxegrub
Transfer timed out.
  
tftp>

The trace shows a connection from one fo the cluster nodes
(192.168.0.112) to the server (192.168.0.1). The interesting thing is
the replies come from both 128.97.176.200 (external interface) and the
internal. Could this be the problem? The gate is running NAT for the
internal network could this be causing a problem? When the internal
network was connected to eth0 previously everything worked fine.

Here is the ethereal trace from the server:

    No. Time        Source                Destination           Protocol
Info
    147 13.640616   192.168.0.112         192.168.0.1           TFTP    
Read Request, File: pxegrub, Transfer type: octet
    148 13.678526   128.97.176.200        192.168.0.112         UDP     
Source port: 33184  Destination port: 32802
    149 13.679517   192.168.0.112         192.168.0.1           TFTP    
Acknowledgement, Block: 1
    150 13.679707   192.168.0.1           192.168.0.112         ICMP    
Destination unreachable
    151 14.674531   128.97.176.200        192.168.0.112         UDP     
Source port: 33184  Destination port: 32802
    152 14.675565   192.168.0.112         192.168.0.1           TFTP    
Acknowledgement, Block: 1
    153 14.675733   192.168.0.1           192.168.0.112         ICMP    
Destination unreachable
    154 16.674600   128.97.176.200        192.168.0.112         UDP     
Source port: 33184  Destination port: 32802
    155 16.675623   192.168.0.112         192.168.0.1           TFTP    
Acknowledgement, Block: 1
    156 16.675743   192.168.0.1           192.168.0.112         ICMP    
Destination unreachable
    157 20.674559   128.97.176.200        192.168.0.112         UDP     
Source port: 33184  Destination port: 32802
    158 20.675576   192.168.0.112         192.168.0.1           TFTP    
Acknowledgement, Block: 1
    159 20.675720   192.168.0.1           192.168.0.112         ICMP    
Destination unreachable
    160 21.902096   Cisco_8e:10:4e        CDP/VTP               CDP     
Cisco Discovery Protocol
    161 25.673873   192.168.0.112         192.168.0.1           TFTP    
Acknowledgement, Block: 1
    162 25.674020   192.168.0.1           192.168.0.112         ICMP    
Destination unreachable
    163 28.674494   128.97.176.200        192.168.0.112         UDP     
Source port: 33184  Destination port: 32802
    164 28.675470   192.168.0.112         192.168.0.1           TFTP    
Acknowledgement, Block: 1
    165 28.675648   192.168.0.1           192.168.0.112         ICMP    
Destination unreachable
    166 33.673652   192.168.0.112         192.168.0.1           TFTP    
Acknowledgement, Block: 1
    167 33.673813   192.168.0.1           192.168.0.112         ICMP    
Destination unreachable
    168 38.673568   192.168.0.112         192.168.0.1           TFTP    
Acknowledgement, Block: 1
    169 38.673749   192.168.0.1           192.168.0.112         ICMP    
Destination unreachable
    170 43.673485   192.168.0.112         192.168.0.1           TFTP    
Acknowledgement, Block: 1
    171 43.673693   192.168.0.1           192.168.0.112         ICMP    
Destination unreachable
    172 44.674570   128.97.176.200        192.168.0.112         UDP     
Source port: 33184  Destination port: 32802
    173 44.675578   192.168.0.112         192.168.0.1           TFTP    
Acknowledgement, Block: 1
    174 44.675744   192.168.0.1           192.168.0.112         ICMP    
Destination unreachable

Could the problem be that we are not binding tftpd to the right
interface? xinetd is starting the server is just appears that the
responses are not going out over the right interface.

Any help or pointers would be very much appreciated!

Thanks,
Scott Friedman
UCLA

Scott A. Friedman wrote:
>         server_args     = -v -v -p -s /tftpboot
>         only_from       = 192.168.0.0/24
>         bind            = 192.168.0.1
>         disable         = no
This doesn't really work for tftp -- tftpd basically manages its own
connections (mostly) and
> 
> The trace shows a connection from one fo the cluster nodes
> (192.168.0.112) to the server (192.168.0.1). The interesting thing is
> the replies come from both 128.97.176.200 (external interface) and the
> internal. Could this be the problem? The gate is running NAT for the
> internal network could this be causing a problem? When the internal
> network was connected to eth0 previously everything worked fine.
> 
This indicates a routing table or filter table setup problem.

I'd recommend trying to reduce the number of unknowns by running tftpd
in standalone mode for the time being.

in.tftpd -l -a 192.168.0.1 -v -v -p -s /tftpboot

	-hpa

Comments below...

On Thu, 2003-07-10 at 13:41, H. Peter Anvin wrote:
> > 
> > The trace shows a connection from one fo the cluster nodes
> > (192.168.0.112) to the server (192.168.0.1). The interesting thing is
> > the replies come from both 128.97.176.200 (external interface) and the
> > internal. Could this be the problem? The gate is running NAT for the
> > internal network could this be causing a problem? When the internal
> > network was connected to eth0 previously everything worked fine.
> > 
> 
> This indicates a routing table or filter table setup problem.
> 
Yes, this was it, thanks. When we switched the interfaces we forgot to
update the NAT iptable

Scott