Evan Bursey <ehbursey at lbl.gov> wrote:> I've just started caring for a dual Opteron machine. It
> has a 3Ware 9500S-12 RAID controller, a Tyan S2882G3NR
> motherboard, 10 IBM 400GB disk drives, and a 650 watt power
> supply.
Ouch, that's pushing it bad. I use a 550W with an old
dual-P3 (sub-20W/each) and 3Ware Escalade 7800 with 8 x PATA
at home.
Most of my other installations that have 8-12 drives are
typically 6U or pedestal with 850W or 1000W 2+1 2xSSI power
supplies.
> (actually, I did get some joy out of recompiling the
kernel.> Its been several years since I last compiled a kernel and
> the process has changed for the better).
Actually, you shouldn't have to rebuild the kernel.
You just:
1. Run make in the 3Ware source directory (the Makefile
3Ware includes does the job)
2. Copy the new 3w-9xxx.ko module in the appropriate
/lib/modules/x.x.x-x directory
3. Run "depmod -a" to rebuild dependencies
4. Run "mkinitrd /boot/x.x.x-x-new.img x.x.x-x" to build a
new initrd
5. Edit /etc/grub.conf to add a new entry with the new
initrd
> I've been in contact with 3Ware. From the error logs I
> supplied to them, it looks like the issue is an
insufficient> power supply.
Would not surprise me one bit. Although the 3Ware can stage
spin-ups at boot, if you have enough power usage, the drives
can still stall due to lack of power in full seek.
> Their rule of thumb is 300W for the motherboard
You'll easily need 400+W with dual-Opterons. I'm sure your
voltage is dropping on the +12V due to inadequate current.
> and 35 W for each disk drive.
Yep, sounds typical at 3A at 12V ~ 36W.
> That means that I would need (at a theoretical minimum) a
> 650 W power supply - which I have.
I'd say you're way over budget.
> I only notice the problem in SMP mode, not in uniprocessor
> mode.
Correct because even though the CPU is plugged in, it's
probably idle and not sucking up any juice.
> That suggests to me that (1) I have a
> boarderline-sufficient power supply that can support one
> but not two CPUs
I concur, it's definitely power. Unless you're staging your
drive startup, I'm surprised you don't have an issue at boot.
> or (2) there is a problem with the combination of
> Opteron/3Ware 9500S-12 hardware and the x86_64 SMP
> kernel. I suppose there are even worse possibilities,
> but I'd rather not think about those right now.
> What do you think?
Definitely power. You are way over budget.
You should either consider a 2+1 PS design (3 units, 2
required to run), or use a separate power supply for the
drives. What kind of enclosure do you have currently?
> Does anyone have any experience with this sort of hardware
> on x86_64 kernels?
I don't think it's a kernel issue. I think the x86-64 kernel
is making far greater use of your Opteron's capabilities. I
mean, there are literally units that get "turned on" when the
x86-64 kernel is running -- things the i686/Athlon kernel
does not enable at all.
> Does the 3w_9xxx driver work well on a x86_64 machine?
> Does anyone have a suggestion as to what capacity power
> supply I should have?
Yes, 850W, easily. Power PC & Cooling has a non-redundant
SSI one for just over $400. I know there are a few other
vendors that have one for about $300.
I'd be interested in what your enclosure is right now.
--
Bryan J. Smith | Sent from Yahoo Mail
mailto:b.j.smith at ieee.org | (please excuse any
http://thebs413.blogspot.com/ | missing headers)