I recently got a new SSD (ocz vertex LE 50gb) It seems to work really well as a ZIL performance wise. My question is, how safe is it? I know it doesn''t have a supercap so lets'' say dataloss occurs....is it just dataloss or is it pool loss? also, does the fact that i have a UPS matter? the numbers i''m seeing are really nice....these are some nfs tar times before zil: real 2m21.498s user 0m5.756s sys 0m8.690s real 2m23.870s user 0m5.756s sys 0m8.739s and these are the same ones after. real 0m32.739s user 0m5.708s sys 0m8.515s real 0m35.580s user 0m5.707s sys 0m8.526s I also sliced it....i have 16 gb ram so i used a 9 gb slice for zil and the rest for L2ARC this is for a single 10 drive raidz2 vdev so far....i''m really impressed with the performance gains -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100524/377153a9/attachment.html>
On Mon, May 24, 2010 at 05:48:56PM -0400, Thomas Burgess wrote:> I recently got a new SSD (ocz vertex LE 50gb) > > It seems to work really well as a ZIL performance wise. My question is, how > safe is it? I know it doesn''t have a supercap so lets'' say dataloss > occurs....is it just dataloss or is it pool loss?Just dataloss.> also, does the fact that i have a UPS matter?Relative to power loss, yes. Nico --
On 5/24/2010 2:48 PM, Thomas Burgess wrote:> I recently got a new SSD (ocz vertex LE 50gb) > > It seems to work really well as a ZIL performance wise. My question > is, how safe is it? I know it doesn''t have a supercap so lets'' say > dataloss occurs....is it just dataloss or is it pool loss? >ZFS is always consistent on-disk, by design. Loss of the ZIL will result in loss of the data in the ZIL which hasn''t been flushed out to the hard drives, but otherwise, the data on the hard drives is consistent and uncorrupted.> > also, does the fact that i have a UPS matter? >It avoids the scenario of losing data in your ZIL due to power loss (and, of course, the rest of your system). So, yes, if you actually care about your system, I''d recommend at least a minimal UPS to allow for quick shutdown after a power loss.> > the numbers i''m seeing are really nice....these are some nfs tar times > before zil: > > > real 2m21.498s > > user 0m5.756s > > sys 0m8.690s > > > real 2m23.870s > > user 0m5.756s > > sys 0m8.739s > > > > and these are the same ones after. > > > > > real 0m32.739s > > user 0m5.708s > > sys 0m8.515s > > > > real 0m35.580s > > user 0m5.707s > > sys 0m8.526s > > > > > I also sliced it....i have 16 gb ram so i used a 9 gb slice for zil > and the rest for L2ARC > > > > this is for a single 10 drive raidz2 vdev so far....i''m really > impressed with the performance gains >That''s going to pretty much be the best-case use for the ZIL - NFS writes being synchronous. Of course, using the rest of the SSD for L2ARC is likely to be almost (if not more) helpful for performance for a wider variety of actions. -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100524/0f20195c/attachment.html>
On 5/24/2010 2:48 PM, Thomas Burgess wrote:> I recently got a new SSD (ocz vertex LE 50gb)Not familiar with that model....> > It seems to work really well as a ZIL performance wise. My question > is, how safe is it? I know it doesn''t have a supercap so lets'' say > dataloss occurs....is it just dataloss or is it pool loss?It should just be dataloss -- you''d potentially lose any filesystem changes made since the last time data was committed to the actual storage device (not the ZIL.) It still sounds like losing the ZIL is a bad idea if you really care about data integrity data though.> > > also, does the fact that i have a UPS matter? >It might! If you can guarantee that the actual drive won''t lose power then you have much more security. *But*, if the drive is powered from the computer''s power supply or internal bus, then you could have an issue -- power loss to the drive can occur in this situation when the system reboots, even though the mains still had power available to them all the time. (Hmmm... do the typical SATA power plugs from computer power supplies have power removed when the system takes a full reset? I would not *think* so, but with software controlled power supplies I''m not 100% certain. It might be an interesting test to measure the power from some common power supplies -- as applied to the SATA power connector -- during a system panic or full reset (reboot -p for example). Its possible that power is supplied continuously to the peripherals even in the event of this kind of failure. DDRdrive X1 solves this with an external power brick. I''m not sure what other storage devices do here. The supercap solution is nice because it doesn''t need the external power, but can supply power to the media even when the host removes power from the device itself.> > the numbers i''m seeing are really nice....these are some nfs tar times > before zil:Wow. That''s a pretty huge improvement. :-) - Garrett (newly of Nexenta)> > > real 2m21.498s > > user 0m5.756s > > sys 0m8.690s > > > real 2m23.870s > > user 0m5.756s > > sys 0m8.739s > > > > and these are the same ones after. > > > > > real 0m32.739s > > user 0m5.708s > > sys 0m8.515s > > > > real 0m35.580s > > user 0m5.707s > > sys 0m8.526s > > > > > I also sliced it....i have 16 gb ram so i used a 9 gb slice for zil > and the rest for L2ARC > > > > this is for a single 10 drive raidz2 vdev so far....i''m really > impressed with the performance gains > > > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100524/9d962a60/attachment.html>
> > > ZFS is always consistent on-disk, by design. Loss of the ZIL will result > in loss of the data in the ZIL which hasn''t been flushed out to the hard > drives, but otherwise, the data on the hard drives is consistent and > uncorrupted. > > > > This is what i thought. I have read this list on and off for awhile nowbut i''m not a guru....I see a lot of stuff about the intel ssd and disabling the write cache....so i just wasn''t sure...This is good news.> > It avoids the scenario of losing data in your ZIL due to power loss (and, > of course, the rest of your system). So, yes, if you actually care about > your system, I''d recommend at least a minimal UPS to allow for quick > shutdown after a power loss. > > > yes, i have a nice little UPS. I''ve tested it a few times and it seems towork well. It gives me about 20 minutes of power and can even send commands via a script to shut down the system before the battery goes dry.> That''s going to pretty much be the best-case use for the ZIL - NFS writes > being synchronous. Of course, using the rest of the SSD for L2ARC is likely > to be almost (if not more) helpful for performance for a wider variety of > actions. > > > yes, i have another machine without a zil (i bought a kingston 64 gb ssd onsale and intended to try it as a zil but ultimately decided to just use it as l2arc because of the performance numbers...) but the l2arc helps a ton for my uses. I did slice this ssd...i used 9 gb for zil and the rest for l2arc (about 36 gb) I''m really impressed with this ssd....for only 160 dollars (180 - 20 mail in rebate) it''s a killer deal. it can do 235 MB/s sustained writes and has soemthing like 15,000 iops> -- > Erik Trimble > Java System Support > Mailstop: usca22-123 > Phone: x17195 > Santa Clara, CA > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100524/0b84ba6a/attachment.html>
> > > Not familiar with that model.... > >It''s a sandforce sf-1500 model but without a supercap....heres some info on it: Maximum Performance - Max Read: up to 270MB/s - Max Write: up to 250MB/s - Sustained Write: up to 235MB/s - Random Write 4k: 15,000 IOPS - Max 4k IOPS: 50,000 per http://www.ocztechnology.com/products/solid-state-drives/2-5--sata-ii/performance-enterprise-solid-state-drives/ocz-vertex-limited-edition-sata-ii-2-5--ssd.html> > > Wow. That''s a pretty huge improvement. :-) > > - Garrett (newly of Nexenta) > > >yes, i love it. I''m really impressed with this ssd for the money....160 usd (180 - 20 rebate) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100524/64e193f7/attachment.html>
> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- > bounces at opensolaris.org] On Behalf Of Nicolas Williams > > > I recently got a new SSD (ocz vertex LE 50gb) > > > > It seems to work really well as a ZIL performance wise. > > I know it doesn''t have a supercap so lets'' say dataloss > > occurs....is it just dataloss or is it pool loss? > > Just dataloss.WRONG! The correct answer depends on your version of solaris/opensolaris. More specifically, it depends on the zpool version. The latest fully updated sol10 and the latest opensolaris release (2009.06) only go up to zpool 14 or 15. But in zpool 19 is when a ZIL loss doesn''t permanently offline the whole pool. I know this is available in the developer builds. The best answer to this, I think, is in the ZFS Best Practices Guide: (uggh, it''s down right now, so I can''t paste the link) If you have zpool <19, and you lose an unmirrored ZIL, then you lose your pool. Also, as a configurable option apparently, I know on my systems, it also meant I needed to power cycle. If you have zpool >=19, and you lose an unmirrored ZIL, then performance will be degraded, but everything continues to work as normal. Apparently the most common mode of failure for SSD''s is also failure to read. To make it worse, a ZIL is only read after system crash, which means the possibility of having a failed SSD undetected must be taken into consideration. If you do discover a failed ZIL after crash, with zpool <19 your pool is lost. But with zpool >=19 only the unplayed writes are lost. With zpool >=19, your pool will be intact, but you would lose up to 30sec of writes that occurred just before the crash.
On Tue, May 25, 2010 at 11:27 AM, Edward Ned Harvey <solaris2 at nedharvey.com>wrote:> > From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- > > bounces at opensolaris.org] On Behalf Of Nicolas Williams > > > > > I recently got a new SSD (ocz vertex LE 50gb) > > > > > > It seems to work really well as a ZIL performance wise. > > > I know it doesn''t have a supercap so lets'' say dataloss > > > occurs....is it just dataloss or is it pool loss? > > > > Just dataloss. > > WRONG! > > The correct answer depends on your version of solaris/opensolaris. More > specifically, it depends on the zpool version. The latest fully updated > sol10 and the latest opensolaris release (2009.06) only go up to zpool 14 > or > 15. But in zpool 19 is when a ZIL loss doesn''t permanently offline the > whole pool. I know this is available in the developer builds. > > The best answer to this, I think, is in the ZFS Best Practices Guide: > (uggh, it''s down right now, so I can''t paste the link) > > If you have zpool <19, and you lose an unmirrored ZIL, then you lose your > pool. Also, as a configurable option apparently, I know on my systems, it > also meant I needed to power cycle. > > If you have zpool >=19, and you lose an unmirrored ZIL, then performance > will be degraded, but everything continues to work as normal. > > Apparently the most common mode of failure for SSD''s is also failure to > read. To make it worse, a ZIL is only read after system crash, which means > the possibility of having a failed SSD undetected must be taken into > consideration. If you do discover a failed ZIL after crash, with zpool <19 > your pool is lost. But with zpool >=19 only the unplayed writes are lost. > With zpool >=19, your pool will be intact, but you would lose up to 30sec > of > writes that occurred just before the crash. > > > I didn''t ask about losing my zil.I asked about power loss taking out my pool. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100525/c127d917/attachment.html>
> From: Thomas Burgess [mailto:wonslung at gmail.com] > > Just dataloss. > WRONG! > > I didn''t ask about losing my zil. > > I asked about power loss taking out my pool.As I recall:> I recently got a new SSD (ocz vertex LE 50gb) > > It seems to work really well as a ZIL performance wise. My question > is, how safe is it? I know it doesn''t have a supercap so lets'' say > dataloss occurs....is it just dataloss or is it pool loss?At least to me, this was not clearly "not asking about losing zil" and was not clearly "asking about power loss." Sorry for answering the question you thought you didn''t ask. I would suggest clarifying your question, by saying instead: "so lets'' say *power*loss occurs" Then it would have been clear what you were asking. Since this is a SSD you''re talking about, unless you have enabled nonvolatile write cache on that disk (which you should never do), and the disk incorrectly handles cache flush commands (which it should never do), then the supercap is irrelevant. All ZIL writes are to be done synchronously. If you have a power loss, you don''t lose your pool, and you also don''t lose any writes in the ZIL. You do, however, lose any async writes that were not yet flushed to disk. There is no way to prevent that, regardless of ZIL configuration.
On Mon, 24 May 2010, Thomas Burgess wrote:> > It''s a sandforce sf-1500 model but without a supercap....heres some info on it: > > Maximum Performance > > * Max Read: up to 270MB/s > * Max Write: up to 250MB/s > * Sustained Write: up to 235MB/s > * Random Write 4k: 15,000 IOPS > * Max 4k IOPS: 50,000Isn''t there a serious problem with these specifications? It seems that the minimum assured performance values (and the median) are much more interesting than some "maximum" performance value which might only be reached during a brief instant of the device lifetime under extremely ideal circumstances. It seems that toilet paper may of much more practical use than these specifications. In fact, I reject them as being specifications at all. The Apollo reentry vehicle was able to reach amazing speeds, but only for a single use. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
> > > At least to me, this was not clearly "not asking about losing zil" and was > not clearly "asking about power loss." Sorry for answering the question > you > thought you didn''t ask. >I was only responding to your response of WRONG!!! The guy wasn''t wrong in regards to my questions. I''m sorry for not making THAT more clear in my post.> > I would suggest clarifying your question, by saying instead: "so lets'' say > *power*loss occurs" Then it would have been clear what you were asking. > >I''m pretty sure i did ask about power loss....or at least it was implied by my point about the UPS. You''re right, i probably should have been a little more clear.> Since this is a SSD you''re talking about, unless you have enabled > nonvolatile write cache on that disk (which you should never do), and the > disk incorrectly handles cache flush commands (which it should never do), > then the supercap is irrelevant. All ZIL writes are to be done > synchronously. > > This SSD doesn''t use nonvolatile write cache (at least i don''t think itdoes, it''s a SF-1500 based ssd) I might be wrong about this, but i thought one of the biggest things about the sandforce was that it doesn''t use DRAM> If you have a power loss, you don''t lose your pool, and you also don''t lose > any writes in the ZIL. You do, however, lose any async writes that were > not > yet flushed to disk. There is no way to prevent that, regardless of ZIL > configuration. >Yes, I know that i lose async writes....i just wasn''t sure if that resulted in an issue...I might be somewhat confused to how the ZIL works but i thought the point of the ZIL was to "pretend" a write actually happened when it may not have actually been flushed to disk yet...in this case, a write to the zil might not make it to disk....i just didn''t know if this could result in a loss of a pool due to some sort of corruption of the uberblock or something.....I''m not entirely up to speed on the voodoo that is ZFS. I wasn''t trying to be rude, sorry if it came off like that. I am aware of the issue regarding removing the ZIL on non-dev versions of opensolaris....i am on b134 so that doesnt'' apply to me. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100525/65e88820/attachment.html>
On Tue, May 25, 2010 at 12:38 PM, Bob Friesenhahn < bfriesen at simple.dallas.tx.us> wrote:> On Mon, 24 May 2010, Thomas Burgess wrote: > >> >> It''s a sandforce sf-1500 model but without a supercap....heres some info >> on it: >> >> Maximum Performance >> >> * Max Read: up to 270MB/s >> * Max Write: up to 250MB/s >> * Sustained Write: up to 235MB/s >> * Random Write 4k: 15,000 IOPS >> * Max 4k IOPS: 50,000 >> > > Isn''t there a serious problem with these specifications? It seems that the > minimum assured performance values (and the median) are much more > interesting than some "maximum" performance value which might only be > reached during a brief instant of the device lifetime under extremely ideal > circumstances. It seems that toilet paper may of much more practical use > than these specifications. In fact, I reject them as being specifications > at all. > > The Apollo reentry vehicle was able to reach amazing speeds, but only for a > single use. > > Bob >What exactly do you mean? Every review i''ve read about this device has been great. Every review i''ve read about the sandforce controllers has been good to....are you saying they have shorter lifetimes? Everything i''ve read has made them sound like they should last longer than typical ssds because they write less actual data....> -- > Bob Friesenhahn > bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ > GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100525/74b71abe/attachment.html>
Also, let me note, it came with a 3 year warranty so I expect it to last at least 3 years...but if it doesn''t, i''ll just return it under the warranty. On Tue, May 25, 2010 at 1:26 PM, Thomas Burgess <wonslung at gmail.com> wrote:> > > On Tue, May 25, 2010 at 12:38 PM, Bob Friesenhahn < > bfriesen at simple.dallas.tx.us> wrote: > >> On Mon, 24 May 2010, Thomas Burgess wrote: >> >>> >>> It''s a sandforce sf-1500 model but without a supercap....heres some info >>> on it: >>> >>> Maximum Performance >>> >>> * Max Read: up to 270MB/s >>> * Max Write: up to 250MB/s >>> * Sustained Write: up to 235MB/s >>> * Random Write 4k: 15,000 IOPS >>> * Max 4k IOPS: 50,000 >>> >> >> Isn''t there a serious problem with these specifications? It seems that >> the minimum assured performance values (and the median) are much more >> interesting than some "maximum" performance value which might only be >> reached during a brief instant of the device lifetime under extremely ideal >> circumstances. It seems that toilet paper may of much more practical use >> than these specifications. In fact, I reject them as being specifications >> at all. >> >> The Apollo reentry vehicle was able to reach amazing speeds, but only for >> a single use. >> >> Bob >> > What exactly do you mean? > Every review i''ve read about this device has been great. Every review i''ve > read about the sandforce controllers has been good to....are you saying they > have shorter lifetimes? Everything i''ve read has made them sound like they > should last longer than typical ssds because they write less actual data.... > > > > >> -- >> Bob Friesenhahn >> bfriesen at simple.dallas.tx.us, >> http://www.simplesystems.org/users/bfriesen/ >> GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ >> > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100525/4c237797/attachment.html>
On Tue, 25 May 2010, Thomas Burgess wrote:> > The Apollo reentry vehicle was able to reach amazing speeds, but only for a single use. > > What exactly do you mean??What I mean is what I said. A set of specifications which are all written as "maximums" (i.e. peak) is pretty useless. Perhaps if you were talking about a maximum ambient temperature specification or maximum allowed elevation, then a maximum specification makes sense. Perhaps the device is fine (I have no idea) but these posted specifications are virtually useless. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
> From: Thomas Burgess [mailto:wonslung at gmail.com] > > I might be somewhat confused to how the ZIL > works but i thought the point of the ZIL was to "pretend" a write > actually happened when it may not have actually been flushed to disk > yet...No. How the ZIL works is like this: Whenever a process issues a sync write, the process blocks until the OS acknowledges the write has been committed to nonvolatile storage. Assuming you have a dedicated log device, the OS immediately commits this data to the log device, and unblocks the process. Then, the data is able to float around in RAM with all the async write requests, getting optimized for disk performance and so forth. The OS might aggregate up to 30 secs of small writes into a single larger sequential transaction for the primary storage devices. If there''s an unfortunate event such as system crash during the meantime, then upon the next bootup, the OS will notice data in the ZIL log, which was intended for a TXG, which never made its way out to primary storage. Therefore, the OS replays the log, and commits those writes now. All the async writes that were still in RAM were lost, but the sync writes were not. The dedicated log device helps sync writes approach the performance of async writes. Nothing beats the performance of async writes. If you disable ZIL, then sync writes are handled the same as async writes. Both in terms of performance, and risk.
>> >> Since this is a SSD you''re talking about, unless you have enabled >> nonvolatile write cache on that disk (which you should never do), and the >> disk incorrectly handles cache flush commands (which it should never do), >> then the supercap is irrelevant. ?All ZIL writes are to be done >> synchronously. >> > This SSD doesn''t use nonvolatile write cache (at least i don''t think it > does, it''s a SF-1500 based ssd) > I might be wrong about this, but i thought one of the biggest things about > the sandforce was that it doesn''t use DRAM >I am not an expert but reading through lots of materials in the subject searching for some affordable ZIL solution I come to this: The sandforce controllers use ram for caching, but they are using limited cache on-die and does not require _external_ (to the controller) DRAM on board. Read this article about SandForce controllers (the info comes actually from SandForce as it is stated in the first paragraph): http://www.anandtech.com/show/3661/understanding-sandforces-sf1200-sf1500-not-all-drives-are-equal Especially this part: "As an enterprise class solution, the SF-1500 is designed to complete any writes in progress in the event of sudden power loss. The SF-1500 firmware is configured expecting the presence of the super cap we saw in the Vertex 2 Pro. As such it expects that it can write at full speed without any worries about power loss. In other words, it assumes you have power failure protection at the drive level." Also if you read it through you will see that different OEMs use the controllers differently, not always conforming to the guidelines provided by sandforce. So having an SF controller on board does not guarantee anything (like having a non-production ready firmware).>> >> If you have a power loss, you don''t lose your pool, and you also don''t >> lose >> any writes in the ZIL. ?You do, however, lose any async writes that were >> not >> yet flushed to disk. ?There is no way to prevent that, regardless of ZIL >> configuration. > > Yes, I know that i lose async writes....i just wasn''t sure if that resulted > in an issue...I might be somewhat confused to how the ZIL works but i > thought the point of the ZIL was to "pretend" a write actually happened when > it may not have actually been flushed to disk yet...in this case, a write to > the zil might not make it to disk....i just didn''t know if this could result > in a loss of a pool due to some sort of corruption of the uberblock or > something.....I''m not entirely up to speed on the voodoo that is ZFS. > >If your ZIL does use nonvolatile cache and does not honor flush requests then a powerloss is the same as loosing the ZIL altogether since it will not have the data saved for a playback.
On May 26, 2010, at 4:12 AM, Attila Mravik wrote:> If your ZIL does use nonvolatile cache and does not honor flush > requests then a powerloss is the same as loosing the ZIL altogether > since it will not have the data saved for a playback.This is not a correct statement. Those are two different failure modes and the recovery method is different. At this point it is worth reminding the forum that if you do not have a separate log device and you do use HDDs for the pool, then you are in the volatile cache + cache flush respected (we hope) scenario for the ZIL. -- richard -- Richard Elling richard at nexenta.com +1-760-896-4422 ZFS and NexentaStor training, Rotterdam, July 13-15, 2010 http://nexenta-rotterdam.eventbrite.com/