So after toying around with some stuff a few months back I got bogged down and set this project aside for a while. Time to revisit. <BR><BR> Looking around there still is not a good "these cards/motherboards" work list. the HCL is hardly ever updated, and its far more geared towards business use than hobbyist/home use. So bearing all of that in mind I will need the following things: <BR><BR> 1. At least 2 gigE nics. <BR> 2. At least 6 SATAII ports (and at least 6 drive slots) <BR> 3. Reasonable price (probably going to build it myself.) <BR> I''m not worried about hotswapping. I want to make sure the box is going to be upgradeable with decent priced parts for a while (so PCI is out). My current fileserver is 6 years old. Out of space (or close enough) and starting to become a little less stable than I would like. So without getting into all of the gory details, the things I am stuck on are the following: <BR><BR> 1. What consumer level motherboards (not a $400 server board, I dont need that) and/or chipsets does opensolaris support at this point ? I dont want "if you compile this 3 week old version with these four changes it might work". I want "this works". I use solaris at work on all sorts of Sun hardware, but of course I cant afford Sun hardware for my house. <BR><BR> 2. What consumer level SATAII chipsets work. 4-ports onboard is fine for now since I can always add a card later. I will need at least four ports to start. pci-e cards are highly preferred since pci-x is expensive and going to become rarer. (mark my words) <BR><BR> So I was hoping that this board would work: <a href="http://www.gigabyte.com.tw/Products/Motherboard/Products_Overview.aspx?ClassValue=Motherboard&ProductID=2287&ProductName=GA-M57SLI-S4">HERE</a> <BR><BR> I''m open to suggestions. I''d prefer to use solaris and zfs, but if it cannot be easily done I will stick with Linux. This message posted from opensolaris.org
On January 18, 2007 6:27:14 PM -0800 "." <clockwork at sigsys.org> wrote:> Looking around there still is not a good "these cards/motherboards" work > list.You must have just missed the "What SATA controllers are people using for ZFS?" thread. Not a list, but you can probably find similar components to what was recommended. I would be hestitant myself to use any other SATA card than what was recommended, however motherboard choice is probably fairly wide open. -frank
Couldn''t this be considered a compatibility list that we can trust for OpenSolaris and ZFS? http://www.sun.com/io_technologies/ I''ve been looking at it for the past few days. I am looking for eSATA support options - more details below. Only 2 devices on the list show support for eSATA, both are PCI-X. One uses Infiniband, one uses an eSATA multiplier cable. I wish PCI Express would get in there. That would open up my options more... I really don''t want to be limited only to internal SATA; I want to use these 5 drive eSATA-driven enclosures, like one of the below: http://www.norcotek.com/item_detail.php?categoryid=8&modelno=ds-500 http://fwdepot.com/thestore/product_info.php/products_id/1586 http://fwdepot.com/thestore/product_info.php/products_id/1325 http://fwdepot.com/thestore/product_info.php/products_id/1578 (a 10 drive one [2 eSATA ports used]) This would effectively, if I understand it right, allow for 20 drives per controller card (4 ports x 5 drives apiece) However, I don''t think OpenSolaris/Solaris support these unless the Addonics eSATA PCI-X adapter supports them. I have not figured that one out yet. All I know is I want ZFS. This would be for home usage, I want something as small and quiet as possible. Otherwise I would look into getting 3u type drive shelves (which would be noisy, etc.) I have a couple other friends as well all interested in this same idea. Just waiting around for confirmation that these combinations work, or for the hardware support to grow... Anyone have any additional thoughts? I looked at the SATA thread already, didn''t help me much though. On 1/18/07, Frank Cusack <fcusack at fcusack.com> wrote:> You must have just missed the "What SATA controllers are people using > for ZFS?" thread. Not a list, but you can probably find similar components > to what was recommended. I would be hestitant myself to use any other > SATA card than what was recommended, however motherboard choice is probably > fairly wide open.
2007/1/18, . <clockwork at sigsys.org>:> 2. What consumer level SATAII chipsets work. 4-ports onboard is fine for now since I can always add a card later. I will need at least four ports to start. pci-e cards are highly preferred since pci-x is expensive and going to become rarer. (mark my words)Something worth mentioning is that ACHI SATA controllers will be supported in the next Nevada build. As such, I would probably look at Intel boards instead. Nvidia SATA support has been long awaited, but there is no clear indication of when, if ever it will arrive. It will still work, but with no NCQ or hot swap. Chris
On 1/18/07, . <clockwork at sigsys.org> wrote:> Looking around there still is not a good "these cards/motherboards" work list. the HCL is hardly ever updated, and its far more geared towards business use than hobbyist/home use.Yes, this is true. This list is the best resource I have found so far, and I have been half heartedly looking for the last three months or so. So to help you and and see if I can get some answers myself, I will post the system I am currently looking at. I have picked these parts up from mentions on the list: ASUS M2NPV-VM ( http://www.newegg.com/Product/Product.asp?item=N82E16813131014 ) AMD Sempron 64 2800+ ( http://www.newegg.com/Product/Product.asp?item=N82E16819104245 ) SYBA SD-SATA-4P PCI SATA Controller Card ( http://www.newegg.com/product/Product.asp?item=N82E16815124020 ) The sata card is only a SATA 1 card, but do I care? Do the ports on the motherboard work? (I think I saw somewhere they do in PATA compatibility mode.) As you can see I am going for bottom of the line, but it is just a box to play around with, and a gigE nas box if things work out well. So list, what do you think?
On Thu, 18 Jan 2007, . wrote:> Looking around there still is not a good "these cards/motherboards" work > list. the HCL is hardly ever updated, and its far more geared towards > business use than hobbyist/home use. So bearing all of that in mind I > will need the following things: <BR><BR > > 1. At least 2 gigE nics. <BR> > 2. At least 6 SATAII ports (and at least 6 drive slots) <BR> > 3. Reasonable price (probably going to build it myself.) <BR>I can give you another working recipe - one that may not meet your needs exactly, but one that will work nicely. This particular recipe was not a design based on a clean-sheet-of-paper - but rather, my Sun Ultra 20 motherboard died and I wanted a replacement system that would provide a classic *workstation* function while also providing (extra SATA disk drive bays for) ZFS-based storage and allow re-use of the (heavily modified) Ultra-20 parts. This is the resulting system which is *highly* satisfactory/stable as a desktop driving two LCD panels[0], while providing ZFS filespace and software development facilities and currently running 6 zones under Solaris Update 3: Motherboard: Asus A8N32-SLI Delux NB: AMD 939-pin socket CPU: AMD x4200 X2 dual-core CPU with 1Mb L2 cache per core [1] RAM: 2 * Corsair TWINX2048-3200C2PT 2Gb kit (4Gb total) Case: Antec P180 mid-tower (comes with no PSU) ZFS disks: 2 * Seagate 500GB SATA ST3500641AS (on sale @ Frys) Boot disk: Western Digital 74Gb SATA WDC WD740GD-00FLA1 [1] PSU: SS-500HT (500W) [1] Graphics: 7800GT [1][3] CDROM/DVD: Plextor 716A slot-loader [1] (the only one to work with the Ultra-20 front panel) Add ins: - Gigabyte I-ram (GC-RAMDISK) with 4*Kingston KVR400X74C3A/1G DIMMs [1][2] - extra Antec TriCool 120mm 3-speed fan (front panel) The motherboard was setup using the Asus automatic overclock BIOS function set to +10% and the x4200 appears as an x4400 [4] Notes: a) Upside: The Antec P180 provides a compartment (at the bottom of the case) which includes 4-bay SATA disk drive bay in front of a 1" * 120mm fan which holds 4 * SATA drives mounted on silicon vibration dampers and the Power Supply Unit (PSU). This compartment has airflow which is separated from the main motherboard area of the case. There is extra space between the disk drives and cooling is excellent. b) Downside: The PSU wiring harness, coming upwards from the PSU/disk drive compartment, blocks off access to some of the neighbouring motherboard expansion slots. c) Upside: There are lots of disk drive bays - aside from the ones mentioned in the bottom PSU compartment. d) I found that 4 of the built-in (motherboard) SATA ports worked with no effort. The other two SATA ports did not work on first try - but no effort was expended trying to make them work. Currently the 4 working SATA ports are assigned: 1) WD740 boot drive 2) Seagate 500Gb drive (ZFS mirror) 3) Seagate 500Gb drive (ZFS mirror) 4) Gigabyte I-ram e) Downside: The build time for this box was longer that usual. Perhaps because of the extra "head scratching" time required to figure out the unusual case layout (power supply at the bottom and all wiring going vertically upwards) and the time it took to route the wiring and cable-tie everything neatly. If you decide to try this recipe, snag a 939-pin Model 165 Opteron dual-core processor (before the supply dries up completely). These are known to overclock easily and reliably to 2.6GHz+. There are many (now old) articles describing how to do this and the Asus A8N32-SLI Delux is known as a motherboard that is easy/reliable to overclock. Checkout newegg.com and zipzoomfly.com for the CPU. Hurry before they are gone!! NB: I know that the 939-pin AMD family is already EOL - but these components are rock solid and high performance. But buy the long term config your want *now*, since it will be impossible to upgrade this system later. Email me offlist if you have any questions. [0] Dell 3007WFP 30" panel running at 2560x1600 and iiyama AU5131DT running at 1600x1200 running under Twinview using the Sun/Nvidia driver version *9746 driver (twinview config provided by nvidia-config). [1] moved from the (modified) Ultra-20 [2] used to provide test zones that can be deployed pretty quickly and can provide fast swap space, if necessary. [3] BFG GeForce 7800GT OC (The OC indicates that its over-clocked) [4] # psrinfo -v Status of virtual processor 0 as of: 01/18/2007 23:16:43 on-line since 01/01/2007 21:15:39. The i386 processor operates at 2420 MHz, and has an i387 compatible floating point processor. Status of virtual processor 1 as of: 01/18/2007 23:16:43 on-line since 01/01/2007 21:15:45. The i386 processor operates at 2420 MHz, and has an i387 compatible floating point processor. Regards, Al Hopper Logical Approach Inc, Plano, TX. al at logical-approach.com Voice: 972.379.2133 Fax: 972.379.2134 Timezone: US CDT OpenSolaris.Org Community Advisory Board (CAB) Member - Apr 2005 OpenSolaris Governing Board (OGB) Member - Feb 2006
David,
That would be nice, and reasonably cheap. I hope it works. The only
issue I have with it is the lack of 3.5 spots, only having 3 such slots
really limits the ability to grow the space over time.
>> "You must have just missed the "What SATA controllers are
people using
for ZFS?" thread."
No. I just didnt see anything that fit my needs on the list, tons of PCI-X
cards (I''m not buying an overpriced server mobo) and some
"possibly" working
controllers on the motherboards.
>> "ACHI SATA ... probably look at Intel boards instead."
Whats ACHI ? I didnt see anything useful on google or wikipedia ... is it a
chipset ? The issue I take with intel is there chips are either grossly
power hungry/hot (anything pre-pentium M) or ungodly expensive (core, core
2). They dont have anything that competes with a 65W AM2 athlon64.
>> "... A8N32-SLI"
That might work. I dont see a real huge win for using an opteron since this
box wont be doing much at all besides fileserving/streaming media. (Heck I
wish I could do it on the crappy old 32-bit hardware I have laying around,
but alas ... no worky). 4 SATA ports on board would allow me to do a
mirrored root. Add in a mythical pci-e card with at least 4 ports and I have
what I need.
Does anybody know of a guaranteed working pci-e 2 port card ? Must be pci-e.
On 1/18/07, David <akda5id+zfs at gmail.com>
wrote:>
> On 1/18/07, . <clockwork at sigsys.org> wrote:
> > Looking around there still is not a good "these
cards/motherboards" work
> list. the HCL is hardly ever updated, and its far more geared towards
> business use than hobbyist/home use.
>
> Yes, this is true. This list is the best resource I have found so far,
> and I have been half heartedly looking for the last three months or
> so. So to help you and and see if I can get some answers myself, I
> will post the system I am currently looking at. I have picked these
> parts up from mentions on the list:
>
> ASUS M2NPV-VM (
> http://www.newegg.com/Product/Product.asp?item=N82E16813131014 )
>
> AMD Sempron 64 2800+ (
> http://www.newegg.com/Product/Product.asp?item=N82E16819104245 )
>
> SYBA SD-SATA-4P PCI SATA Controller Card (
> http://www.newegg.com/product/Product.asp?item=N82E16815124020 )
>
> The sata card is only a SATA 1 card, but do I care? Do the ports on
> the motherboard work? (I think I saw somewhere they do in PATA
> compatibility mode.)
>
> As you can see I am going for bottom of the line, but it is just a box
> to play around with, and a gigE nas box if things work out well.
>
> So list, what do you think?
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20070119/9f93848b/attachment.html>
>However, I don''t think OpenSolaris/Solaris support these unless the >Addonics eSATA PCI-X adapter supports them. I have not figured that >one out yet. All I know is I want ZFS.I''m not using the multiplier, but I am using the 4 port Addonics eSATA PCI-X card in a PCI slot, btw - eSATA == SATA with a different connector and better shielding. I have an internal SATA card that I''ve been using with 42" SATA cables to external drives in a power chassis. Works just fine. I am doing the same with the Addonics and an eSATA to SATA adapter. It gives me a longer reach. This message posted from opensolaris.org
2007/1/19, clockwork at sigsys.org <clockwork at sigsys.org>:> >> "ACHI SATA ... probably look at Intel boards instead." > > Whats ACHI ? I didnt see anything useful on google or wikipedia ... is it a > chipset ? The issue I take with intel is there chips are either grossly > power hungry/hot (anything pre-pentium M) or ungodly expensive (core, core > 2). They dont have anything that competes with a 65W AM2 athlon64.Oops, I seem to have transposed some characters while typing that. It is AHCI: Advanced Host Controller Interface. Many hardware vendors are standardizing on this specification for SATA interfaces. The most common ones are found in the Intel ICH6R, ICH7R, and ICH8R south bridges, but others from VIA, Nvidia, SiS and Jmicron are planned or available. See http://www.opensolaris.org/os/community/device_drivers/projects/AHCI The driver is fairly new and does not support much at present, but it should be a safe bet in the future. As for PCIe cards, I think the only options are the the two port SiI3132 and Jmicron based cards. Sorry about the typo. Chris
> On 1/18/07, . <clockwork at sigsys.org> wrote: > > SYBA SD-SATA-4P PCI SATA Controller Card ( > http://www.newegg.com/product/Product.asp?item=N82E168 > 15124020 ) > >>From my home ZFS server setup, I had tried two Syba SD-SATA2-2E2I PCI-X SATA II Controller Cards without any luck; both cards'' BIOS'' wouldn''t recgonize my SATA drives.http://www.newegg.com/Product/Product.asp?Item=N82E16816124003R Although the Syba tech support was reasonable, they only helped conclude that BOTH of the cards I had received were defective - yep, that was 0 and 2. Perhaps the other model listed above works better, I don''t know. In the end, I just stuck with my onboard SATA I/O which is only SATA 150, but was still fast enough for my network. For my ZFS home server, I was able to find an excellent Sun W1100z on eBay ~$360 that came with a 2.4GHz Opteron, 1GB ECC RAM, Quardro FX 500, and 80Gb IDE drive - perfect for the basis of my ZFS server build-up. To that I''ve added two Seagate Barracude ES ST3400620NS 400Gb SATA II drives for about $360 that now run my 400Gb whole-disk ZFS mirror. Overall I''m in for $720 for a great server/workstation with 400Gb of redundant storage, not bad, and performance is much better, with much greater functionality, than most $1,000+ home NAS systems I''ve seen. At first I thought I''d toss the IDE drive, but I ended up just put the OS on it with all my data and zones on the ZFS mirror...at least until ZFS root. One of these days I''ll get around to finishing the blog about it: http://classiarius.com/W1100z%20Home%20ZFS%20Server/W1100z%20Home%20ZFS%20Server.html HTH. Wes W. This message posted from opensolaris.org
As a followup, the system I''m trying to use this on is a dual PII 400 with 512MB. Real low budget. 2 500 GB drives with 2 120 GB in a RAIDZ. The idea is that I can get 2 more 500 GB drives later to get full capacity. I tested going from a 20GB to a 120GB and that worked well. I''m finding the throughput just isn''t there. < 2MB/s compared to 20MB/s on a similar Linux system. Anyone else going this low budget? This message posted from opensolaris.org
Tom Buskey wrote:> As a followup, the system I''m trying to use this on is a dual PII 400 with 512MB. Real low budget. > > 2 500 GB drives with 2 120 GB in a RAIDZ. The idea is that I can get 2 more 500 GB drives later to get full capacity. I tested going from a 20GB to a 120GB and that worked well. > > I''m finding the throughput just isn''t there. < 2MB/s compared to 20MB/s on a similar Linux system. > > Anyone else going this low budget? > > > This message posted from opensolaris.orgHmm... that''s lower than I would have expected. Something is likely wrong. These machines do have very limited memory bandwidth, so the checksumming will cost more here than on faster CPUs. How fast can you DD from the raw device to /dev/null? - Bart -- Bart Smaalders Solaris Kernel Performance barts at cyber.eng.sun.com http://blogs.sun.com/barts
On Wed, Jan 31, 2007 at 05:01:19AM -0800, Tom Buskey wrote:> As a followup, the system I''m trying to use this on is a dual PII 400 > with 512MB. Real low budget. > > 2 500 GB drives with 2 120 GB in a RAIDZ. The idea is that I can get > 2 more 500 GB drives later to get full capacity. I tested going from > a 20GB to a 120GB and that worked well. > > I''m finding the throughput just isn''t there. < 2MB/s compared to > 20MB/s on a similar Linux system. > > Anyone else going this low budget?There are many folks (including myself) that have done similar super-low budget setups. Which SATA controller are you using? If it''s the SI3112, it has a documented problem when you try to use both SATA channels simultaneously - it gets less than 2MB/s, compared to about 50MB/s on each drive individually. I''m not sure if the SI3114 has similar problems, or not. This may not be your problem, but I know the SI3112 was popular on machines in that timeframe. --Bill
That''s good to know.
It''s a new Addonics 4 port card. Specifically:
ADS3GX4R5-E RAID5/JBOD 4-port ext. SATA II PCI-X
prtconf -v output:
pci1095,7124, instance #0
Driver properties:
name=''sata'' type=int items=1 dev=none
.....
name=''compatible'' type=string items=7
value=''pci1095,3124.1095.7124.2'' +
''pci1095,3124.1095.7124''
+ ''pci1095,7124'' + ''pci1095,3124.2 '' +
''pci1095,3124'' + ''pciclass,010400'' +
''pciclass,0104''
name=''model'' type=string items=1
value=''RAID controller''
.....
name=''vendor-id'' type=int items=1
value=00001095
name=''device-id'' type=int items=1
value=00003124
So it''s based on the 3124 chip right?
This message posted from opensolaris.org
Hello Tom,
Wednesday, January 31, 2007, 2:01:19 PM, you wrote:
TB> As a followup, the system I''m trying to use this on is a dual
PII
TB> 400 with 512MB. Real low budget.
TB> 2 500 GB drives with 2 120 GB in a RAIDZ. The idea is that I can
TB> get 2 more 500 GB drives later to get full capacity. I tested
TB> going from a 20GB to a 120GB and that worked well.
TB> I''m finding the throughput just isn''t there. <
2MB/s compared to
TB> 20MB/s on a similar Linux system.
TB> Anyone else going this low budget?
I''ve setup on Celeron 500MHz, Asus P3B-F with 256MB of RAM on S10U3.
Two 250GB ATA disk in a mirror and another 18GB ATA disk for a system.
Throughput is about 25-29MB/s. Works as expected.
--
Best regards,
Robert mailto:rmilkowski at task.gda.pl
http://milek.blogspot.com
Urk! Where is this documented? And - is it something you can do nothing about, or are we ultimately trying to address it somewhere / somehow? Thanks!! Nathan. Bill Moore wrote:> On Wed, Jan 31, 2007 at 05:01:19AM -0800, Tom Buskey wrote: >> As a followup, the system I''m trying to use this on is a dual PII 400 >> with 512MB. Real low budget. >> >> 2 500 GB drives with 2 120 GB in a RAIDZ. The idea is that I can get >> 2 more 500 GB drives later to get full capacity. I tested going from >> a 20GB to a 120GB and that worked well. >> >> I''m finding the throughput just isn''t there. < 2MB/s compared to >> 20MB/s on a similar Linux system. >> >> Anyone else going this low budget? > > There are many folks (including myself) that have done similar super-low > budget setups. Which SATA controller are you using? If it''s the > SI3112, it has a documented problem when you try to use both SATA > channels simultaneously - it gets less than 2MB/s, compared to about > 50MB/s on each drive individually. I''m not sure if the SI3114 has > similar problems, or not. This may not be your problem, but I know the > SI3112 was popular on machines in that timeframe. > > > --Bill > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
> Tom Buskey wrote: > > As a followup, the system I''m trying to use this on > is a dual PII 400 with 512MB. Real low budget.> > Hmm... that''s lower than I would have expected. > Something is > ikely wrong. These machines do have very limited > memory> How fast can you DD from the raw device to /dev/null?Roughly 230Mb/s> > > - Bart > > > -- > Bart Smaalders Solaris Kernel Performance > barts at cyber.eng.sun.com http://blogs.sun.com/barts > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discu > ss >This message posted from opensolaris.org
Tom Buskey wrote:>> Tom Buskey wrote: >>> As a followup, the system I''m trying to use this on >> is a dual PII 400 with 512MB. Real low budget. > >> Hmm... that''s lower than I would have expected. >> Something is >> ikely wrong. These machines do have very limited >> memory > >> How fast can you DD from the raw device to /dev/null? > > Roughly 230Mb/s > >>Do you mean ~28MB/sec? Something is definitely bogus. What happens when you do dd from both drives at once? - Bart -- Bart Smaalders Solaris Kernel Performance barts at cyber.eng.sun.com http://blogs.sun.com/barts
Sorry, that''s dd from /dev/zero to /dev/null I think there''s an issue with my SATA card On 2/7/07, Bart Smaalders <bart.smaalders at sun.com> wrote:> > Tom Buskey wrote: > >> Tom Buskey wrote: > >>> As a followup, the system I''m trying to use this on > >> is a dual PII 400 with 512MB. Real low budget. > > > >> Hmm... that''s lower than I would have expected. > >> Something is > >> ikely wrong. These machines do have very limited > >> memory > > > >> How fast can you DD from the raw device to /dev/null? > > > > Roughly 230Mb/s > > > >> > > Do you mean ~28MB/sec? > > Something is definitely bogus. What happens when you do dd > from both drives at once? > > - Bart > > -- > Bart Smaalders Solaris Kernel Performance > barts at cyber.eng.sun.com http://blogs.sun.com/barts >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20070207/601b9a17/attachment.html>