Hi I''m running OpenSolaris 148 on a few boxes, and newer boxes are getting installed as we speak. What would you suggest for a good SLOG device? It seems some new PCI-E-based ones are hitting the market, but will those require special drivers? Cost is obviously alsoo an issue here.... Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 roy at karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer p? norsk.
On Tue, Mar 01, 2011 at 08:03:42AM -0800, Roy Sigurd Karlsbakk wrote:> Hi > > I''m running OpenSolaris 148 on a few boxes, and newer boxes are > getting installed as we speak. What would you suggest for a good SLOG > device? It seems some new PCI-E-based ones are hitting the market, > but will those require special drivers? Cost is obviously alsoo an > issue here.... > > Vennlige hilsener / Best regards > > royWhat type of workload are you looking to handle? We''ve had good luck with pairs of Intel X-25E''s for VM datastore duty. We also have a DDRrive X1 which is probably the best option out there currently and will handle workloads the X-25E''s can''t. I believe a lot of folks here use the Vertex SLC-based SF-15 SSD''s also. Ray
I''d back that. X25E''s are great but also look at the STECH ZeusIOPS as well as the new Intel''s. --- W. A. Khushil Dep - khushil.dep at gmail.com - 07905374843 Windows - Linux - Solaris - ZFS - XenServer - FreeBSD - C/C++ - PHP/Perl - LAMP - Nexenta - Development - Consulting & Contracting http://www.khushil.com/ - http://www.facebook.com/GlobalOverlord On 1 March 2011 16:08, Ray Van Dolson <rvandolson at esri.com> wrote:> On Tue, Mar 01, 2011 at 08:03:42AM -0800, Roy Sigurd Karlsbakk wrote: > > Hi > > > > I''m running OpenSolaris 148 on a few boxes, and newer boxes are > > getting installed as we speak. What would you suggest for a good SLOG > > device? It seems some new PCI-E-based ones are hitting the market, > > but will those require special drivers? Cost is obviously alsoo an > > issue here.... > > > > Vennlige hilsener / Best regards > > > > roy > > What type of workload are you looking to handle? We''ve had good luck > with pairs of Intel X-25E''s for VM datastore duty. > > We also have a DDRrive X1 which is probably the best option out there > currently and will handle workloads the X-25E''s can''t. > > I believe a lot of folks here use the Vertex SLC-based SF-15 SSD''s > also. > > Ray > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20110301/726586dd/attachment.html>
The PCIe based ones are good (typically they are quite fast), but check the following first: a) do you need an SLOG at all? Some workloads (asynchronous ones) will never benefit from an SLOG. b) form factor. at least one manufacturer uses a PCIe card which is not compliant with the PCIe form-factor and will not fit in many cases -- especially typical 1U boxes. c) driver support. d) do they really just go straight to ram/flash, or do they have an on-device SAS or SATA bus? Some PCIe devices just stick a small flash device on a SAS or SATA controller. I suspect that those devices won''t see a lot of benefit relative to an external drive (although they could theoretically drive that private SAS/SATA bus at much higher rates than an external bus -- but I''ve not checked into it.) The other thing with PCIe based devices is that they consume an IO slot, which may be precious to you depending on your system board and other I/O needs. - Garrett On Tue, 2011-03-01 at 17:03 +0100, Roy Sigurd Karlsbakk wrote:> Hi > > I''m running OpenSolaris 148 on a few boxes, and newer boxes are getting installed as we speak. What would you suggest for a good SLOG device? It seems some new PCI-E-based ones are hitting the market, but will those require special drivers? Cost is obviously alsoo an issue here.... > > Vennlige hilsener / Best regards > > roy > -- > Roy Sigurd Karlsbakk > (+47) 97542685 > roy at karlsbakk.net > http://blogg.karlsbakk.net/ > -- > I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer p? norsk. > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
> a) do you need an SLOG at all? Some workloads (asynchronous ones) will > never benefit from an SLOG.We''re planning to use this box for CIFS/NFS, so we''ll need an SLOG to speed things up.> b) form factor. at least one manufacturer uses a PCIe card which is > not compliant with the PCIe form-factor and will not fit in many cases > -- especially typical 1U boxes.The box is 4U with some 7 8x PCIe slots, so I think it should do fine> c) driver support.That was why I asked here in the first place...> d) do they really just go straight to ram/flash, or do they have an > on-device SAS or SATA bus? Some PCIe devices just stick a small flash > device on a SAS or SATA controller. I suspect that those devices won''t > see a lot of benefit relative to an external drive (although they > could theoretically drive that private SAS/SATA bus at much higher rates > than an external bus -- but I''ve not checked into it.) > > The other thing with PCIe based devices is that they consume an IO > slot, > which may be precious to you depending on your system board and other > I/O needs.As I mentioned above, we have sufficient slots. As for the SATA/SAS onboard controller, that was the reason I asked here in the first place. So - do anyone know a good device for this? X25-E is rather old now, so there should be better ones available...... Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 roy at karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer p? norsk.
On Tue, Mar 01, 2011 at 09:56:35AM -0800, Roy Sigurd Karlsbakk wrote:> > a) do you need an SLOG at all? Some workloads (asynchronous ones) will > > never benefit from an SLOG. > > We''re planning to use this box for CIFS/NFS, so we''ll need an SLOG to > speed things up. > > > b) form factor. at least one manufacturer uses a PCIe card which is > > not compliant with the PCIe form-factor and will not fit in many cases > > -- especially typical 1U boxes. > > The box is 4U with some 7 8x PCIe slots, so I think it should do fine > > > c) driver support. > > That was why I asked here in the first place... > > > d) do they really just go straight to ram/flash, or do they have an > > on-device SAS or SATA bus? Some PCIe devices just stick a small flash > > device on a SAS or SATA controller. I suspect that those devices won''t > > see a lot of benefit relative to an external drive (although they > > could theoretically drive that private SAS/SATA bus at much higher rates > > than an external bus -- but I''ve not checked into it.) > > > > The other thing with PCIe based devices is that they consume an IO > > slot, > > which may be precious to you depending on your system board and other > > I/O needs. > > As I mentioned above, we have sufficient slots. As for the SATA/SAS > onboard controller, that was the reason I asked here in the first > place. > > So - do anyone know a good device for this? X25-E is rather old now, > so there should be better ones available......I think the OCZ Vertex 2 EX (SLC) is fairly highly regarded: http://www.ocztechnology.com/ocz-vertex-2-ex-series-sata-ii-2-5-ssd.html Note that if you''re using an LSI backplane (probably are if you''re using SuperMicro hardware), they have tended to certify only against the X-25E. Other drives should work fine, but just an FYI. This page (maybe a little dated, I''m not sure) has some pretty good info: http://www.nexenta.org/projects/site/wiki/About_suggested_NAS_SAN_Hardware> > Vennlige hilsener / Best regards > > royRay
On Tue, March 1, 2011 11:11, Khushil Dep wrote:> I''d back that. X25E''s are great but also look at the STECH ZeusIOPS as > well as the new Intel''s.STEC''s products are not available to retail customers, only OEMs. (Unless something has changed recently, in which case a link would be useful.)
Patrick O''Sullivan
2011-Mar-01 18:57 UTC
[zfs-discuss] [OpenIndiana-discuss] Good SLOG devices?
Next gen spec sheets suggest the X25-E will get a "Power Safe Write Cache," something it does not have today. See: http://www.anandtech.com/Show/Index/3965?cPage=5&all=False&sort=0&page=1&slug=intels-3rd-generation-x25m-ssd-specs-revealed (Article is about X25-M, scroll down for X25-E info.) On Tuesday, March 1, 2011, Roy Sigurd Karlsbakk <roy at karlsbakk.net> wrote:>> a) do you need an SLOG at all? Some workloads (asynchronous ones) will >> never benefit from an SLOG. > > We''re planning to use this box for CIFS/NFS, so we''ll need an SLOG to speed things up. > >> b) form factor. at least one manufacturer uses a PCIe card which is >> not compliant with the PCIe form-factor and will not fit in many cases >> -- especially typical 1U boxes. > > The box is 4U with some 7 8x PCIe slots, so I think it should do fine > >> c) driver support. > > That was why I asked here in the first place... > >> d) do they really just go straight to ram/flash, or do they have an >> on-device SAS or SATA bus? Some PCIe devices just stick a small flash >> device on a SAS or SATA controller. I suspect that those devices won''t >> see a lot of benefit relative to an external drive (although they >> could theoretically drive that private SAS/SATA bus at much higher rates >> than an external bus -- but I''ve not checked into it.) >> >> The other thing with PCIe based devices is that they consume an IO >> slot, >> which may be precious to you depending on your system board and other >> I/O needs. > > As I mentioned above, we have sufficient slots. As for the SATA/SAS onboard controller, that was the reason I asked here in the first place. > > So - do anyone know a good device for this? X25-E is rather old now, so there should be better ones available...... > > Vennlige hilsener / Best regards > > roy > -- > Roy Sigurd Karlsbakk > (+47) 97542685 > roy at karlsbakk.net > http://blogg.karlsbakk.net/ > -- > I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer p? norsk. > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >
Roy Sigurd Karlsbakk
2011-Mar-01 19:24 UTC
[zfs-discuss] [OpenIndiana-discuss] Good SLOG devices?
> Next gen spec sheets suggest the X25-E will get a "Power Safe Write > Cache," something it does not have today. > > See: > http://www.anandtech.com/Show/Index/3965?cPage=5&all=False&sort=0&page=1&slug=intels-3rd-generation-x25m-ssd-specs-revealed > > (Article is about X25-M, scroll down for X25-E info.)5k IOPS for write isn''t very good for SLOGs, comparing to those 50k+ drives available..... Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 roy at karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer p? norsk.
David,
STEC/DataON ZeusRAM(Z4RZF3D-8UC-DNS) SSD now available for users in channel.
It is 8GB DDR3 RAM based SAS SSD protected by supercapacitor and NVRAM 16GB.
It is designed for ZFS ZIL with low latency
http://dataonstorage.com/zeusram
Rocky
-----Original Message-----
From: zfs-discuss-bounces at opensolaris.org
[mailto:zfs-discuss-bounces at opensolaris.org] On Behalf Of David Magda
Sent: Tuesday, March 01, 2011 10:30 AM
To: Khushil Dep
Cc: zfs-discuss at opensolaris.org
Subject: Re: [zfs-discuss] Good SLOG devices?
On Tue, March 1, 2011 11:11, Khushil Dep wrote:> I''d back that. X25E''s are great but also look at the
STECH ZeusIOPS as
> well as the new Intel''s.
STEC''s products are not available to retail customers, only OEMs.
(Unless
something has changed recently, in which case a link would be useful.)
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
And I''d just add ? amke sure you are running a recent enough release of ZFS to support importation of a pool without the SLOG device being available, just in case export/recovery of the pool need to be attempted elsewhere than the traditional server. Of course they also limit deployments in poor man cluster scenarios and or HA configs such as NexentaStor. Craig On 1 Mar 2011, at 16:35, Garrett D''Amore wrote:> The PCIe based ones are good (typically they are quite fast), but check > the following first: > > a) do you need an SLOG at all? Some workloads (asynchronous ones) will > never benefit from an SLOG. > > b) form factor. at least one manufacturer uses a PCIe card which is > not compliant with the PCIe form-factor and will not fit in many cases > -- especially typical 1U boxes. > > c) driver support. > > d) do they really just go straight to ram/flash, or do they have an > on-device SAS or SATA bus? Some PCIe devices just stick a small flash > device on a SAS or SATA controller. I suspect that those devices won''t > see a lot of benefit relative to an external drive (although they could > theoretically drive that private SAS/SATA bus at much higher rates than > an external bus -- but I''ve not checked into it.) > > The other thing with PCIe based devices is that they consume an IO slot, > which may be precious to you depending on your system board and other > I/O needs. > > - Garrett > > On Tue, 2011-03-01 at 17:03 +0100, Roy Sigurd Karlsbakk wrote: >> Hi >> >> I''m running OpenSolaris 148 on a few boxes, and newer boxes are getting installed as we speak. What would you suggest for a good SLOG device? It seems some new PCI-E-based ones are hitting the market, but will those require special drivers? Cost is obviously alsoo an issue here.... >> >> Vennlige hilsener / Best regards >> >> roy >> -- >> Roy Sigurd Karlsbakk >> (+47) 97542685 >> roy at karlsbakk.net >> http://blogg.karlsbakk.net/ >> -- >> I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer p? norsk. >> _______________________________________________ >> zfs-discuss mailing list >> zfs-discuss at opensolaris.org >> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss-- Craig Morgan Cinnabar Solutions Ltd t: +44 (0)791 338 3190 f: +44 (0)870 705 1726 e: craig at cinnabar-solutions.com w: www.cinnabar-solutions.com
On Tue, March 1, 2011 16:32, Rocky Shek wrote:> David, > > STEC/DataON ZeusRAM(Z4RZF3D-8UC-DNS) SSD now available for users in > channel. > > It is 8GB DDR3 RAM based SAS SSD protected by supercapacitor and NVRAM > 16GB. > > It is designed for ZFS ZIL with low latency > > http://dataonstorage.com/zeusramSays "call for price". I know what that means, it means "If you have to ask, you can''t afford it." -- David Dyer-Bennet, dd-b at dd-b.net; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info
On Tue, March 1, 2011 10:35, Garrett D''Amore wrote:> a) do you need an SLOG at all? Some workloads (asynchronous ones) will > never benefit from an SLOG.I''ve been fighting the urge to maybe do something about ZIL (which is what we''re talking about here, right?). My load is CIFS, not NFS (so not synchronous, right?), but there are a couple of areas that are significant to me where I do decent-size (100MB to 1GB) sequential writes (to newly-created files). On the other hand, when those writes seem to me to be going slowly, the disk access lights aren''t mostly on, suggesting that the disk may not be what''s holding me up. I can test that by saving to local disk and comparing times, also maybe running zpool iostat. This is a home system, lightly used; the performance issue is me sitting waiting while big Photoshop files save. So of some interest to me personally, and not at ALL like what performance issues on NAS usually look like. It''s on a UPS, so I''m not terribly worried about losses on power failure; and I''d just lose my work since the last save, generally, at worst. I might not believe the disk access lights on the box (Chenbro chassis, with two 4-drive hot-swap bays for the data disks; driven off the motherboard SATA plus a Supermicro 8-port SAS controller with SAS-to-SATA cables). In doing a drive upgrade just recently, I got rather confusing results with the lights, perhaps the controller or the drive model made a difference in when the activity lights came on. The VDEVs in the pool are mirror pairs. It''s been expanded twice by adding VDEVs and once by replacing devices in one VDEV. So the load is probably fairly unevenly spread across them just now. My desktop connects to this server over gigabit ethernet (through one switch; the boxes sit next to each other on a shelf over my desk). I''ll do more research before spending money. But as a question of general theory, should a decent separate intent log device help for a single-user sequential write sequence in the 100MB to 1GB size range? -- David Dyer-Bennet, dd-b at dd-b.net; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info
On 3/2/11 9:42 AM, "David Dyer-Bennet" <dd-b at dd-b.net> wrote:>Says "call for price". I know what that means, it means "If you have to >ask, you can''t afford it.I called. It''s $3k -- not a fit for my archive servers, but an interesting idea for a database server I''m building.... Probably not a great product for the home hobbyist, though. :^) -- Dave Pooser, ACSA Manager of Information Services Alford Media http://www.alfordmedia.com
On Wed, Mar 2 at 9:58, David Dyer-Bennet wrote:> >On Tue, March 1, 2011 10:35, Garrett D''Amore wrote: > >> a) do you need an SLOG at all? Some workloads (asynchronous ones) will >> never benefit from an SLOG. > >I''ve been fighting the urge to maybe do something about ZIL (which is what >we''re talking about here, right?). My load is CIFS, not NFS (so not >synchronous, right?), but there are a couple of areas that are significant >to me where I do decent-size (100MB to 1GB) sequential writes (to >newly-created files). On the other hand, when those writes seem to me to >be going slowly, the disk access lights aren''t mostly on, suggesting that >the disk may not be what''s holding me up. I can test that by saving to >local disk and comparing times, also maybe running zpool iostat. > >This is a home system, lightly used; the performance issue is me sitting >waiting while big Photoshop files save. So of some interest to me >personally, and not at ALL like what performance issues on NAS usually >look like. It''s on a UPS, so I''m not terribly worried about losses on >power failure; and I''d just lose my work since the last save, generally, >at worst. > >I might not believe the disk access lights on the box (Chenbro chassis, >with two 4-drive hot-swap bays for the data disks; driven off the >motherboard SATA plus a Supermicro 8-port SAS controller with SAS-to-SATA >cables). In doing a drive upgrade just recently, I got rather confusing >results with the lights, perhaps the controller or the drive model made a >difference in when the activity lights came on. > >The VDEVs in the pool are mirror pairs. It''s been expanded twice by >adding VDEVs and once by replacing devices in one VDEV. So the load is >probably fairly unevenly spread across them just now. My desktop connects >to this server over gigabit ethernet (through one switch; the boxes sit >next to each other on a shelf over my desk). > >I''ll do more research before spending money. But as a question of general >theory, should a decent separate intent log device help for a single-user >sequential write sequence in the 100MB to 1GB size range?ZIL, as I understand it, is only for small synchronous writes, the opposite of your workload. If you don''t have a SLOG, the ZIL is embedded in your pool anyway. Above a certain size, the writes go straight to the pool''s final storage location. I''d be curious if you''re getting errors in your SMB stream, or maybe your server is set to hold onto too much data before flushing (default is 45 seconds, and there''s been reports of the system not always force-flushing early when the buffers fill) I''ve heard reports of short-stroking the amount of time it accumulates write data resulting in improved performance in some workloads. --eric -- Eric D. Mudama edmudama at bounceswoosh.org