thr3ads.net - LARTC - Rip problems [Sep 2002]

If this information is useful, please help other people find it:
Share via:

Joseph Watson

2002-Sep-28 17:46 UTC

Rip problems

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello,

	I am running a portmaster for dialup customers.  This portmaster has a T1 
connected to it, and is the gateway to the internet.  Behind it I have a 
class C network divided up into 4 subnets, netmask 255.255.255.192 or /26.  
One of the subnets is behind my linuxbox (the 192 network).  The portmaster 
is running rip(1), so I installed routed on the linux box, and configured it 
to share its information.

ie.
EXPORT_GATEWAY="no"
SILENT="no"

  When I start routed, the appropriate routes show up in the portmaster after 
about a 30 seconds, and all works good for about 2 1/2 minutes.  Then the 
portmaster sets the Metric to 16 for the route to my subnet behind the 
firewall, and routing quits working.  If I restart routed, we will repeat the 
process.  If I stop routed during the 2 1/2 mins, it will immediately set the 
Met to 16.  This tells me that they are communicating because when I shut 
routed down the metric is set to 16.  But why does this happen exactly at 2 
1/2 min??  I am quite confused?

Any suggestions would be great.

- -- 
Regards

Joseph Watson
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE9lesTABydhMNsDgMRAuZzAKCDKkKGviW6PYX4YxDyTfyzikNhHQCeLiSz
+cONh6fjWlgOY4eirljT0CI=vgXh
-----END PGP SIGNATURE-----

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Joseph Watson

2002-Sep-28 19:10 UTC

head link

Re: Rip problems

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello,

I have identified the source of the problem, but not the cause of it. Here is a
output from
tcpdump that shows the rip traffic from the routed process on the linuxbox. 
Again I have
configured routed as follows:

EXPORT_GATEWAY="no"
SILENT="no"

and it gets its information from the network setup on the localmachine.

in the following output, all xx.xx.xx. represent the same class C address.

[root@dwwireless httpd]# tcpdump -ni eth0 udp port 520 and src host xx.xx.xx.52
tcpdump: listening on eth0
12:33:16.427345 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 2]:
{192.168.1.0}(1) {xx.xx.xx.0}(1) (DF)
12:33:21.658707 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 2]:
{192.168.1.0}(16) {xx.xx.xx.0}(16) (DF)  <----- restarted routed ------
12:33:22.359761 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-req 24 (DF)
12:33:52.358588 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 3]:
{xx.xx.xx.192}(1) {192.168.1.0}(1)[|rip] (DF)
12:34:22.359529 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 3]:
{xx.xx.xx.192}(1) {192.168.1.0}(1)[|rip] (DF)
12:34:52.360541 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 3]:
{xx.xx.xx.192}(1) {192.168.1.0}(1)[|rip] (DF)
12:35:22.361625 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 3]:
{xx.xx.xx.192}(1) {192.168.1.0}(1)[|rip] (DF)
12:35:52.362543 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 3]:
{xx.xx.xx.192}(1) {192.168.1.0}(1)[|rip] (DF)
12:36:22.365285 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 3]:
{xx.xx.xx.192}(16) {192.168.1.0}(1)[|rip] (DF)
12:36:52.364554 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 3]:
{xx.xx.xx.192}(16) {192.168.1.0}(1)[|rip] (DF)
12:37:22.365592 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 2]:
{192.168.1.0}(1) {xx.xx.xx.0}(1) (DF)
12:37:52.366556 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 2]:
{192.168.1.0}(1) {xx.xx.xx.0}(1) (DF)
12:38:22.367531 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 2]:
{192.168.1.0}(1) {xx.xx.xx.0}(1) (DF)
12:38:52.368547 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 2]:
{192.168.1.0}(1) {xx.xx.xx.0}(1) (DF)
12:39:22.369516 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 2]:
{192.168.1.0}(1) {xx.xx.xx.0}(1) (DF)
12:39:52.370524 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 2]:
{192.168.1.0}(1) {xx.xx.xx.0}(1) (DF)
12:40:22.371571 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 2]:
{192.168.1.0}(1) {xx.xx.xx.0}(1) (DF)
12:40:52.372525 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 2]:
{192.168.1.0}(1) {xx.xx.xx.0}(1) (DF)
12:41:22.373521 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 2]:
{192.168.1.0}(1) {xx.xx.xx.0}(1) (DF)
12:41:52.374513 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 2]:
{192.168.1.0}(1) {xx.xx.xx.0}(1) (DF)
12:42:22.375511 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 2]:
{192.168.1.0}(1) {xx.xx.xx.0}(1) (DF)
12:42:28.221015 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 2]:
{192.168.1.0}(16) {xx.xx.xx.0}(16) (DF)  <---- restarted routed -------
12:42:28.937439 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-req 24 (DF)
12:42:59.066864 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 3]:
{xx.xx.xx.192}(1) {192.168.1.0}(1)[|rip] (DF)
12:43:29.067806 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 3]:
{xx.xx.xx.192}(1) {192.168.1.0}(1)[|rip] (DF)
12:43:59.068802 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 3]:
{xx.xx.xx.192}(1) {192.168.1.0}(1)[|rip] (DF)
12:44:29.069811 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 3]:
{xx.xx.xx.192}(1) {192.168.1.0}(1)[|rip] (DF)
12:44:59.070798 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 3]:
{xx.xx.xx.192}(1) {192.168.1.0}(1)[|rip] (DF)
12:45:29.073590 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 3]:
{xx.xx.xx.192}(16) {192.168.1.0}(1)[|rip] (DF)
12:45:59.072795 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 3]:
{xx.xx.xx.192}(16) {192.168.1.0}(1)[|rip] (DF)
12:46:29.073861 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 2]:
{192.168.1.0}(1) {xx.xx.xx.0}(1) (DF)
12:46:59.074794 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 2]:
{192.168.1.0}(1) {xx.xx.xx.0}(1) (DF)
12:47:28.741356 xx.xx.xx.52.520 > xx.xx.xx.62.520:  RIPv1-resp [items 13]:
{xx.xx.xx.65}(2) {xx.xx.xx.67}(2)[|rip] (DF)
12:47:29.075708 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 2]:
{192.168.1.0}(1) {xx.xx.xx.0}(1) (DF)
12:47:59.076795 xx.xx.xx.52.520 > xx.xx.xx.63.520:  RIPv1-resp [items 2]:
{192.168.1.0}(1) {xx.xx.xx.0}(1) (DF)

As you can see, routed is expiring the route xx.xx.xx.192 network, and adding
one to the hole class C block.
This is obviously wrong, and I don''t know why it is changing its
routing information in mid swing???

Any Ideas?

- --
Regards

Joseph Watson
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE9lf6UABydhMNsDgMRAlT4AKDaZwYQHEzVA/VQGuadgeUlwcptsQCgxOyU
LWz1POL9TDTO+LELeMkFS3s=yRZe
-----END PGP SIGNATURE-----

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Joseph Watson

2002-Sep-28 19:46 UTC

head link

Re: Rip problems

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Saturday 28 September 2002 03:39 pm, you wrote:> >Any Ideas?
>
> I''ve not been following closely, but could this be a RIPv1 vs
RIPv2 thing ?
>
> (As I understand things, RIPv1 did not support CIDR, but RIPv2 does).
Im not sure, but it worked for the first few minutes and then routed started 
changing the routing tables, and I don''t know where this info was
coming
from???

I think I am going to try gated.

- -- 
Regards

Joseph Watson
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE9lgcdABydhMNsDgMRAkvBAJ9YOpwPcYrY8rIYzXbcDEyYK/PVkgCg2xiG
bnfpmXr42tUrqFNr8WAMdSI=+cEd
-----END PGP SIGNATURE-----

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Ramin Alidousti

2002-Sep-29 03:43 UTC

head link

Re: Rip problems

On Sat, Sep 28, 2002 at 03:46:37PM -0400, Joseph Watson wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Saturday 28 September 2002 03:39 pm, you wrote:
> > >Any Ideas?
> >
> > I''ve not been following closely, but could this be a RIPv1 vs
RIPv2 thing ?
> >
> > (As I understand things, RIPv1 did not support CIDR, but RIPv2 does).
Yes, it''s correct, RIPv1 is classful.
> Im not sure, but it worked for the first few minutes and then routed
started
> changing the routing tables, and I don''t know where this info was
coming
> from???
Can you explain what exactly was working in the first few minutes? Did you
actually see a /26 route on the portmaster? There are a few timers in RIP.
The 30 sec you mentioned reminds me of the the update advertisement timer
and the 3 min interval is the holddown timer. What is the topology and where
does the remaining block (C - /26) reside? You said that you have a class C.
Is it a class C or a /24? iaw what is the natural class of your netblock?
Do you also run RIP between the portmaster and that other part? This is
important because if you do you''ll end up having two different routes
to the
same natural class of your netblock which is not what you want.
> I think I am going to try gated.
A better choice is zebra if you''re already familiar with cisco syntax.
Does your portmaster support OSPF?

Ramin
> 
> - -- 
> Regards
> 
> Joseph Watson
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.0.7 (GNU/Linux)
> 
> iD8DBQE9lgcdABydhMNsDgMRAkvBAJ9YOpwPcYrY8rIYzXbcDEyYK/PVkgCg2xiG
> bnfpmXr42tUrqFNr8WAMdSI> =+cEd
> -----END PGP SIGNATURE-----
> 
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Joseph Watson

2002-Sep-29 05:16 UTC

head link

Re: Rip problems

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Saturday 28 September 2002 11:43 pm, Ramin Alidousti
wrote:> Can you explain what exactly was working in the first few minutes? Did you
> actually see a /26 route on the portmaster? There are a few timers in RIP.
> The 30 sec you mentioned reminds me of the the update advertisement timer
> and the 3 min interval is the holddown timer. What is the topology and
> where does the remaining block (C - /26) reside? You said that you have a
> class C. Is it a class C or a /24? iaw what is the natural class of your
> netblock? Do you also run RIP between the portmaster and that other part?
> This is important because if you do you''ll end up having two
different
> routes to the same natural class of your netblock which is not what you
> want.
>
> > I think I am going to try gated.
>
> A better choice is zebra if you''re already familiar with cisco
syntax.
> Does your portmaster support OSPF?
>
> Ramin

I will try to be as complete as possible :)

I have a /24, ie xx.xx.xx.0 - xx.xx.xx.255
This is split into 4 subnets ie /26
The xx.xx.xx.0/26 subnet is used on the network that the portmaster is plugged
into.
I also have a second Portmaster on this network, and xx.xx.xx.63/26 is asigned 
to the modem pool split between the portmasters.  The routing is taken care of 
by rip on the two portmasters.

Now on the same xx.xx.xx.0/26 network, I have a linux box.  Behind this 
Linux Box is where I have the xx.xx.xx.128/26 network.

xx.xx.xx.192/26 is not being used now.

                       ,---modems in the xx.xx.xx.64/26 ---,
                ,-----''------.                      
,-----''-------.
- ---T1--CSU/DSU--| portmaster |---xx.xx.xx.0/26-------| portmaster2 |
                `------------`        |              `-------------`  
                               ,--------------.
                               |  Linux Box   |
                               `--------------`
                                      |
                                xx.xx.xx.128/26

Now, I am monitoring the routing tables in the portmaster connected to the 
T1.  Since it is the gateway this is what matters for incoming traffic.  When
I start routed on the linuxbox, after 30 seconds it broadcasts its info, and
the portmaster updated its routes with a xx.xx.xx.128/26: I can then access 
clients on the xx.xx.xx.128/26 network behind the linux box.  But after 
3min''s, routed start brodcasting the route to the 128/26 network with a
metric of 16.  And as expected, I can''t access the 128/26 network
anymore.
Then it started broadcasting a route to the xx.xx.xx.0/24 network????  Where 
does it get this from??  What I can''t figure out is why routed changes
what
it is broadcasting??  

So it is something to do with routed on the linux box!  I monitored all rip 
traffic with tcpdump (a previous post has this info) and nothing else is 
telling it to change the routing.  The only other rip divices on the network 
are the portmasters, and they are taking care of the xx.xx.xx.64/26 network 
that the modems are on.

Gated talks about routes from the local interface config timing out, and 
there is config options to prevent this.  Maybe this is what I am seeing
with routed??

I have added a static route to the portmaster until I can figure this out.

Thanks for the help
- -- 
Regards

Joseph Watson
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE9lozxABydhMNsDgMRAmJ7AKC2cS/bNkAbaJIQBojSTpnU1yRqsgCffk+D
yCXIDHSYEUNPVm1qrkek4mg=mzJ/
-----END PGP SIGNATURE-----

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Ramin Alidousti

2002-Sep-29 18:50 UTC

head link

Re: Rip problems

> I will try to be as complete as possible :)
> 
> I have a /24, ie xx.xx.xx.0 - xx.xx.xx.255
> This is split into 4 subnets ie /26
> The xx.xx.xx.0/26 subnet is used on the network that the portmaster
> is plugged into. I also have a second Portmaster on this network, and
> xx.xx.xx.63/26 is asigned to the modem pool split between the portmasters.
> The routing is taken care of by rip on the two portmasters.
*) Are we talking RIPv1 or v2?
> Now on the same xx.xx.xx.0/26 network, I have a linux box.  Behind this 
> Linux Box is where I have the xx.xx.xx.128/26 network.
> 
> xx.xx.xx.192/26 is not being used now.
> 
>                        ,---modems in the xx.xx.xx.64/26 ---,
>                 ,-----''------.                      
,-----''-------.
> - ---T1--CSU/DSU--| portmaster |---xx.xx.xx.0/26-------| portmaster2 |
>                 `------------`        |              `-------------`  
>                                ,--------------.
>                                |  Linux Box   |
>                                `--------------`
>                                       |
>                                 xx.xx.xx.128/26
> 
> Now, I am monitoring the routing tables in the portmaster connected to the 
> T1.  Since it is the gateway this is what matters for incoming traffic. 
When
> I start routed on the linuxbox, after 30 seconds it broadcasts its info,
and
> the portmaster updated its routes with a xx.xx.xx.128/26: I can then access
*) So you actually see xx.xx.xx.128/26 with the next hop the linux
box''s
   interface on xx.xx.xx.0/26? This means that you''re not running
RIPv1.
> clients on the xx.xx.xx.128/26 network behind the linux box.  But after 
> 3min''s, routed start brodcasting the route to the 128/26 network
with a
> metric of 16.
This is the mechanism to announce the unreachability of the route.

*) Are you running RIP among the two portmasters and the linux box?
> And as expected, I can''t access the 128/26 network anymore.
> Then it started broadcasting a route to the xx.xx.xx.0/24 network???? 
Where
> does it get this from??  What I can''t figure out is why routed
changes what
> it is broadcasting??  
Somehow, the linux box receives the route for the whole /24. At this point
it poisons the previous /26 and starts announcing the /24.
> 
> So it is something to do with routed on the linux box!  I monitored all rip
> traffic with tcpdump (a previous post has this info) and nothing else is 
> telling it to change the routing.
Your previous posting only showed the RIP packets from the linux box. Can
you capture all the RIP packets?

> The only other rip divices on the network are the portmasters, and they are
> taking care of the xx.xx.xx.64/26 network that the modems are on.
Exactly. I didn''t see any routing information exchange between the
linux
box and the second portmaster, though.
> 
> Gated talks about routes from the local interface config timing out, and 
I don''t understand what you mean here.
> there is config options to prevent this.  Maybe this is what I am seeing
> with routed??
> 
> I have added a static route to the portmaster until I can figure this out.
Yes. For your setup static route is good enough and possibly the easiest.
But just as an exercise, I''d like to know what the problem is.

Ramin
> 
> Thanks for the help
> - -- 
> Regards
> 
> Joseph Watson
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.0.7 (GNU/Linux)
> 
> iD8DBQE9lozxABydhMNsDgMRAmJ7AKC2cS/bNkAbaJIQBojSTpnU1yRqsgCffk+D
> yCXIDHSYEUNPVm1qrkek4mg> =mzJ/
> -----END PGP SIGNATURE-----_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Joseph Watson

2002-Sep-30 04:39 UTC

head link

Re: Rip problems

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sunday 29 September 2002 02:50 pm, Ramin Alidousti
wrote:> > I have added a static route to the portmaster until I can figure this
> > out.
>
> Yes. For your setup static route is good enough and possibly the easiest.
> But just as an exercise, I''d like to know what the problem is.
>
> Ramin
Well, doesn''t this just figure.  I tried to recreate the problem ....
and it
no longer exists??  

I removed the static routes from the portmaster and rebooted it.  I then 
started routed on the linux box and it is 2 hours now and working perfectly!

I have a hunch that I know what the problem was though.  There was a problem 
with the pool on one portmaster, and it was spilling over into the 128/26 
network.  I corrected the problem and reset all ports so that everyone had to 
log back on, and nobody was assigned an address from the 128/26 network, but 
I did not reboot it.  It must have been trying to keep possesion of the 
128/26 network for its own use???

Thanks much.

- -- 
Regards

Joseph Watson

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE9l9VzABydhMNsDgMRAik5AJ9kfUIQCZslxZ0BqyVQvoDJ8iKruACgrQ9B
mBbmniv0KBPZR2csFkxEGqo=fNmW
-----END PGP SIGNATURE-----

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

James Sneeringer

2002-Sep-30 16:42 UTC

head link

Re: Rip problems

On Sat, Sep 28, 2002 at 01:46:37PM -0400, Joseph Watson wrote:
| EXPORT_GATEWAY="no"
| SILENT="no"

This should cause the equivalent of "routed -s" to be run.  The
"-s" tells
routed to send routing updates.  Check with "ps ax".  You can get
further
debugging out of it with "-d" and "-t".

|   When I start routed, the appropriate routes show up in the portmaster after 
| about a 30 seconds, and all works good for about 2 1/2 minutes.  Then the 
| portmaster sets the Metric to 16 for the route to my subnet behind the 
| firewall, and routing quits working.

PortMasters do this when they think they need to remove the route from the
routing table.  They set the "O" flag (for obsolete, I guess) and set
the
metric to 16 (because 16 is the largest metric permitted by RIPv1).  The
route will eventually disappear from the table unless another update is
received.

| If I restart routed, we will repeat the 
| process.  If I stop routed during the 2 1/2 mins, it will immediately set the 
| Met to 16.  This tells me that they are communicating because when I shut 
| routed down the metric is set to 16.  But why does this happen exactly at 2 
| 1/2 min??  I am quite confused?

It sounds like routed isn''t sending routing updates.  RIPv1 sends the
whole
routing table every 30 seconds to the broadcast address (which is why it
takes about 30 seconds for the PortMaster to see the routes).  My guess is
it''s only sending out the initial announcement, and when the PM
doesn''t see
subsequent announcements for a couple minutes, it drops the routes.

If possible, consider using OSPF instead.  RIPv1 is quite obsolete and
generally useless on subnetted networks like yours.  PortMasters have done
OSPF since ComOS 3.5, and you can implement it on Linux with zebra or gated.
For further PortMaster-specific help, consider subscribing to the
portmaster-users@portmasters.com list.  See http://www.portmasters.com/
for more info.

-James

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

LARTC - Sep 2002 - Rip problems

Rip problems

Re: Rip problems

Re: Rip problems

Re: Rip problems

Re: Rip problems

Re: Rip problems

Re: Rip problems

Re: Rip problems