I've setup a server with FreeBSD 8.2 (prerelase) and patched zfs to ver. 28. The server has 11 disks each 2 TB in raidz2. The performance is very good and I've got approx. 117 MB/s on plain GB nics using iscsi. I'm mounting the FreeBSD-server from a couple of vmware esxi 4.1 servers using nfs, but when there is alot of i/o the server becomes unresponsive, easily triggered by installing ie. ms-sql. The server itself is up but is not reachable from the network. When I take the nic down and up again connection to the network is reestablished (ip-wise). A friend of mine has suggested that I disable the zil. The page http://wiki.freebsd.org/ZFSTuningGuide says 'Disabling ZIL is not recommended where data consistency is required (such as database servers) but will not result in file system corruption.' Has anyone tried to disable zil and achieved better performance and still maintain a consistent filesystem? -- regards Claus When lenity and cruelty play for a kingdom, the gentler gamester is the soonest winner. Shakespeare twitter.com/kometen
Hello, I'd be curious to know how much memory you have. Also, you mention both iSCSI and NFS in the post and also say that you are using ESX with NFS. Can you reconfirm that you're definitely using NFS and not iSCSI? This is a type of setup I've been investigating myself and I hope you succeed. Regards, Mark
On Mon, Mar 28, 2011 at 4:22 AM, Claus Guttesen <kometen@gmail.com> wrote:> I've setup a server with FreeBSD 8.2 (prerelase) and patched zfs to > ver. 28. The server has 11 disks each 2 TB in raidz2. The performance > is very good and I've got approx. 117 MB/s on plain GB nics using > iscsi. > > I'm mounting the FreeBSD-server from a couple of vmware esxi 4.1 > servers using nfs, but when there is alot of i/o the server becomes > unresponsive, easily triggered by installing ie. ms-sql. The server > itself is up but is not reachable from the network. When I take the > nic down and up again connection to the network is reestablished > (ip-wise). > > A friend of mine has suggested that I disable the zil. The page > http://wiki.freebsd.org/ZFSTuningGuide says 'Disabling ZIL is not > recommended where data consistency is required (such as database > servers) but will not result in file system corruption.' > > Has anyone tried to disable zil and achieved better performance and > still maintain a consistent filesystem?If your disk controller has a lot of cache on it, and a battery backup, then enabling the write cache and disabling the ZIL can be faster, without sacrifising consistency (the write cache on the controller acts like a ZIL). There's several threads on the zfs-discuss mailing list where this is discussed. However, the better solution, and the one most recommended for those using NFS with ZFS, is to install a small, write-optimised, SLC-based SSD to the system as a separate log (SLOG/ZIL) device. NFS is a very sync-heavy protocol, and having a super-fast ZIL sitting on a separate SSD will greatly improve things. -- Freddie Cash fjwcash@gmail.com
Quoting Claus Guttesen <kometen@gmail.com> (from Mon, 28 Mar 2011 13:22:00 +0200):> I've setup a server with FreeBSD 8.2 (prerelase) and patched zfs to > ver. 28. The server has 11 disks each 2 TB in raidz2. The performance > is very good and I've got approx. 117 MB/s on plain GB nics using > iscsi. > > I'm mounting the FreeBSD-server from a couple of vmware esxi 4.1 > servers using nfs, but when there is alot of i/o the server becomes > unresponsive, easily triggered by installing ie. ms-sql. The server > itself is up but is not reachable from the network. When I take the > nic down and up again connection to the network is reestablished > (ip-wise). > > A friend of mine has suggested that I disable the zil. The page > http://wiki.freebsd.org/ZFSTuningGuide says 'Disabling ZIL is not > recommended where data consistency is required (such as database > servers) but will not result in file system corruption.' > > Has anyone tried to disable zil and achieved better performance and > still maintain a consistent filesystem?The ZIL is not linked to NIC down/up events. It is a completely different topic. I suggest to find the real problem instead of doing some random tuning (which is not tuning in this case but foot-shooting). FYI: disabling the ZIL is someting to do if you are desperate, do not care about production incidents, and everything else (if the ZIL is the problem -- which most probably it isn't by reading your message -- a (maybe write optimized) SSD as a log device could be a solution) does not solve the issue. Bye, Alexander. -- I'll eat ANYTHING that's BRIGHT BLUE!! http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137
> > I'm mounting the FreeBSD-server from a couple of vmware esxi 4.1 > servers using nfs, but when there is alot of i/o the server becomes > unresponsive, easily triggered by installing ie. ms-sql. The server > itself is up but is not reachable from the network. When I take the > nic down and up again connection to the network is reestablished > (ip-wise). >Others have made good comments w.r.t. the zil, however if all IP activity on the server has stopped (and is fixed via "ifconfig XX down; ifconfig XX up") it sounds more like a network device driver issue to me? So, is it just NFS that wedges or all IP activity and does NFS come back to life after the "ifconfig XX up"? rick