asterisk users - Apr 2005 - Motherboard failure with 2 Digium TE405P cards

Hello,

I have spend a long time trying to figure out exactly what is the problem
with one of my Asterisk servers, it is the only one at any of our locations
that has two Digium quad T1 cards in it with 7 T1s connected to it. Most of
the rest of our Asterisk servers run identical hardware except that they
only have a single TE405P board in them. Here's what seems to happen to this
system starting 6 months ago:

Take brand new Asus motherboard with P4 processor, 2GB RAM, SATA or SCSI
drives and two TE405P Digium quad T1 boards. Hook up one local and one long
distance T1, hook up 4 crossover PRIs to other telco equipment, hook up one
channelbank.

The system will run perfectly for about 5 weeks, then randomly the channel
bank users will notice a weird audio cracking sound and the system will
crash. Upon investigation the second TE405P card will have it's lights all
off and on reboot they will not go back on again. After frantically
switching the PCI slot that the lights-out card was in to a free slot the
card works again and everything is happy again, but now no digium card will
work in the other slot again. Another 5 weeks or so passes and again one of
the Digium quad cards stops working. At this point I swap out the entire
system(including quad cards) with another system that has been running for 6
months with no problem and put the malfunctioning system in production with
a single quad card(which now has been running fine 4 months later) and after
6 weeks it happens to the new system. The whole process repeats itself and I
am now on my 3rd set of completely different components serving in this
role(even with different brands of components) and my first PCI slot just
failed last week. We need to have the capability to handle 7 T1s on this
machine and it is not over-heated or overloaded from a system load
standpoint. We also have $200 550W Enermax power supplies in these servers
that have never failed us before.

So here's the question, do two Digium TE405P boards draw too much power or
do something else that would harm a brand new motherboard over time?

Does anyone else out there run two quad board in production? if so what
hardware do you use?

I'm just looking for some user feedback before I contact Digium hardware
support on this.

Thanks,

MATT---

Check the telco equipment you are plugging into (PBXes) with the
crossovers.. Unless they are all on the same power grid and protected
I would blame them. my two cents...

On 4/18/05, mattf <mattf@vicimarketing.com> wrote:
> Hello,
> 
> I have spend a long time trying to figure out exactly what is the problem
> with one of my Asterisk servers, it is the only one at any of our locations
> that has two Digium quad T1 cards in it with 7 T1s connected to it. Most of
> the rest of our Asterisk servers run identical hardware except that they
> only have a single TE405P board in them. Here's what seems to happen to
this
> system starting 6 months ago:
> 
> Take brand new Asus motherboard with P4 processor, 2GB RAM, SATA or SCSI
> drives and two TE405P Digium quad T1 boards. Hook up one local and one long
> distance T1, hook up 4 crossover PRIs to other telco equipment, hook up one
> channelbank.
> 
> The system will run perfectly for about 5 weeks, then randomly the channel
> bank users will notice a weird audio cracking sound and the system will
> crash. Upon investigation the second TE405P card will have it's lights
all
> off and on reboot they will not go back on again. After frantically
> switching the PCI slot that the lights-out card was in to a free slot the
> card works again and everything is happy again, but now no digium card will
> work in the other slot again. Another 5 weeks or so passes and again one of
> the Digium quad cards stops working. At this point I swap out the entire
> system(including quad cards) with another system that has been running for
6
> months with no problem and put the malfunctioning system in production with
> a single quad card(which now has been running fine 4 months later) and
after
> 6 weeks it happens to the new system. The whole process repeats itself and
I
> am now on my 3rd set of completely different components serving in this
> role(even with different brands of components) and my first PCI slot just
> failed last week. We need to have the capability to handle 7 T1s on this
> machine and it is not over-heated or overloaded from a system load
> standpoint. We also have $200 550W Enermax power supplies in these servers
> that have never failed us before.
> 
> So here's the question, do two Digium TE405P boards draw too much power
or
> do something else that would harm a brand new motherboard over time?
> 
> Does anyone else out there run two quad board in production? if so what
> hardware do you use?
> 
> I'm just looking for some user feedback before I contact Digium
hardware
> support on this.
> 
> Thanks,
> 
> MATT---
> _______________________________________________
> Asterisk-Users mailing list
> Asterisk-Users@lists.digium.com
> http://lists.digium.com/mailman/listinfo/asterisk-users
> To UNSUBSCRIBE or update options visit:
>    http://lists.digium.com/mailman/listinfo/asterisk-users
> 

-- 
Andrew Latham

http://www.lathama.com
lathama@gmail.com
lathama@yahoo.com
lathama@lathama.com
If any of the above are not working,
we have bigger problems than my email.

Just from long term experience it might be a heat problem.

Check the really basic stuff first.

The air flow might not be adequate for the box. Make sure your ribbon
cables and such are not blocking flow. 

Two cards might draw too much power, causing the power supply to
overheat causing everything to overheat. 

Don't add more fans, put in better/more efficient fans or a better power
supply.

After six months are you getting dust build up on the fans or vents?
More dust traps more heat which cause more power to be needed to run
fans and convert AC/DC which causes more heat, and so on. But you are
reporting a five week breakdown.

Put a recording thermometer in your boxes. It could be the cooling is
not running as expected in the room. Do you own the room?

I once had a room where the janitor would shut the air-conditioning off
at night because he knew nobody was in there. Then he would turn it back
on in the morning before I got there. The machine was dead, but the room
was ice cold. That took three weeks and a lot of IBM repair guys later
to discover. I only found it because I checked the room on a weekend and
it was 90+ in there.

Don't over tax the air-conditioners. I once had a room where the company
insisted on keeping it at 60 because things kept over heating, every
time there was an over heating they pushed the thermostat lower. Turns
out the air-conditioner was turning itself off because it was
overheating from the demand. Then after a few hours off it would
comeback on, cool and overheat itself because it was unable to keep the
room as cold as a meat locker. 

Even better, another time someone brought in a portable cooler to keep a
room/closet with a switch in it like an icebox. They vented the heat
from the portable cooler out of the room into the dropped ceiling via a
20 foot exhaust hose. By stuffing the exhaust hose into the plenum the
hose was accidentally pointed at the thermostat of the HVAC thermostat
for the entire office. So with 90+ degree air pointed at the office HVAC
thermostat the office HVAC thought it was 90 inside and kept the place
so cold we could barely work there during the winter.

Moral of the story, sometimes it ain't anything you are doing.

Race "the Tyrant" Vanderdecken

-----Original Message----- 
From: asterisk-users-bounces@lists.digium.com
[mailto:asterisk-users-bounces@lists.digium.com] On Behalf Of mattf
Sent: Monday, April 18, 2005 10:35 AM
To: 'asterisk-users@lists.digium.com'
Subject: [Asterisk-Users] Motherboard failure with 2 Digium TE405P cards

Hello,

I have spend a long time trying to figure out exactly what is the
problem
with one of my Asterisk servers, it is the only one at any of our
locations
that has two Digium quad T1 cards in it with 7 T1s connected to it. Most
of
the rest of our Asterisk servers run identical hardware except that they
only have a single TE405P board in them. Here's what seems to happen to
this
system starting 6 months ago:

Take brand new Asus motherboard with P4 processor, 2GB RAM, SATA or SCSI
drives and two TE405P Digium quad T1 boards. Hook up one local and one
long
distance T1, hook up 4 crossover PRIs to other telco equipment, hook up
one
channelbank.

The system will run perfectly for about 5 weeks, then randomly the
channel
bank users will notice a weird audio cracking sound and the system will
crash. Upon investigation the second TE405P card will have it's lights
all
off and on reboot they will not go back on again. After frantically
switching the PCI slot that the lights-out card was in to a free slot
the
card works again and everything is happy again, but now no digium card
will
work in the other slot again. Another 5 weeks or so passes and again one
of
the Digium quad cards stops working. At this point I swap out the entire
system(including quad cards) with another system that has been running
for 6
months with no problem and put the malfunctioning system in production
with
a single quad card(which now has been running fine 4 months later) and
after
6 weeks it happens to the new system. The whole process repeats itself
and I
am now on my 3rd set of completely different components serving in this
role(even with different brands of components) and my first PCI slot
just
failed last week. We need to have the capability to handle 7 T1s on this
machine and it is not over-heated or overloaded from a system load
standpoint. We also have $200 550W Enermax power supplies in these
servers
that have never failed us before.

So here's the question, do two Digium TE405P boards draw too much power
or
do something else that would harm a brand new motherboard over time?

Does anyone else out there run two quad board in production? if so what
hardware do you use?

I'm just looking for some user feedback before I contact Digium hardware
support on this.

Thanks,

MATT---
_______________________________________________
Asterisk-Users mailing list
Asterisk-Users@lists.digium.com
http://lists.digium.com/mailman/listinfo/asterisk-users
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-users