CentOS - Jul 2005 - Good, concurrent I/O design in a server -- WAS: SPARC platforms

From: Peter Arremann <loony at loonybin.org>
> When have I ever argued about that chipset design is better? :-)
> Opteron is a great example that you can do it better...
> Where did you even get the idea I am fixated on that???
http://lists.centos.org/pipermail/centos/2005-June/007341.html
 "<even more anal>Except the iommu, those are limitations of
  chipset, bus and whatever, not EM64T.</even more anal>"

It very much has to do with how EM64T differs from AMD64.
GARTs and I/O MMUs are even more capable than the Opteron.
The issue has not yet been forced on Intel _because_ they still
use a "single point of contention" design.  All CPUs share one
bus, all memory shares another, etc... into a "hub" (with
exception of a proprietary NUMA solution, but all I/O still goes
through that MCH "hub" on even those designs).

http://lists.centos.org/pipermail/centos/2005-June/007662.html
 "Lets be realistic here... all I/O on one CPU only makes a difference for
  the most IO intensive apps (talking several hundret MB/sec or more)..."
  avoid boards that attach all memory to one cpu - unless they cost ..
  so much less that you can add another gig or two in memory...
  I/O on the other hand, while not ideal, will have much lesser effect
  on your performance."

This is very much an issue.  If I have an OS that has processor
affinity for I/O, like Solaris, on a partial mesh platform like most
SPARCs (non-"i" versions), or the newer Opteron (with multiple
AMD8131/8132 or even the nVidia Pro 2200+2050 combo), then
I'm going to be able to handle _concurrent_ I/O for independent
(and sometimes even inter-dependent) processors.

You think of "raw throughput" like 8.4GBps FSB can easily handle
a 0.5GBps transfer, but what you don't realize that the 0.5GBps
transfer holds up the _entire_ 8.4GBps bus for _just_ 0.5GBps.
I.e., it takes 1 second for a 0.5GBps burst from the I/O, which
means you do _not_ "get the other 7.9GBps" while it's tied up.

Now on workstations, not too much of an issue (except for
when you need all:  video, NIC, storage).  And maybe not a
web server where your Internet pipe is only 100Mbps.  But if
you are a major communications carrier, or even just a heavily
trumped file server, near-line server (with both in-band and
out-of-band connects), possibly multiple DB, etc..., it really
_does_ make a _lot_ of different.

http://lists.centos.org/pipermail/centos/2005-June/007668.html  
 "The board: Asus K8N-DL"
A board recommendation you ahd to quickly backtracked on after
a post by myself.  You even said:  
http://lists.centos.org/pipermail/centos/2005-June/007674.html
 "But give me some credit - after all it was you who threw out the
  $300 price mark and besides that, the original poster said he
  had no real IO requirements ... Completely agreed - spend the
  extra money if you want to build a serious server...  "
Which made me scratch my head, because there were other $300
boards with 8-16x the PCI I/O capability because they had
an AMD8131.

And that's just to start.

Which is what these threads are all about, how to build high
performance servers, which may not be the same as for yourself.
The hot-plug thread, that's just the total icing on the cake
because you can't possibly see why someone might want an
old NUMA/SBUS platform.

You can assume I'm "wrong," or go out-of-your-way to talk about
it.  But that's _not_ while I'm posting.  I'm posting so people
are aware that there _are_ applications where you _might_ want
such capabilities.  Your continued insistence in asserting there
is no application whatsoever is what *I* have a "problem" with.

It's not about "wrong" -- it's about what people _might_ need.


--
Bryan J. Smith   mailto:b.j.smith at ieee.org