Hi. Is the problem with ZFS supporting 4k sectors or is the problem mixing 512 byte and 4k sector disks in one pool, or something else? I have seen alot of discussion on the 4k issue but I haven''t understood what the actual problem ZFS has with 4k sectors is. It''s getting harder and harder to find large disks with 512 byte sectors so what should we do? TIA...
I think the problem is with disks that are 4k organised, but report their blocksize as 512. If the disk reports it''s blocksize correctly as 4096, then ZFS should not have a problem. At least my 2TB Seagate Barracuda disks seemed to report their blocksizes as 4096, and my zpools on those machines have ashift set to 12, which is correct, since 2?? = 4096 You cannot mix 512 and 4096 byte blocksize disks in one pool, at least not in a mirror. All disks in a single pool should have the same blocksize. There is a hacked version of zpool for OpenIndiana that has a blocksize option to the create subcommand. I don''t know if other OSes have similar fixes. Such a hacked zpool might not be useful for the root pool at install time, at least it would require some subterfuge to make the installation use the correct blocksize, since the process is a bit too far automated. On 2012-07-19 14:00, zfs-discuss-request at opensolaris.org wrote:> Message: 2 > Date: Wed, 18 Jul 2012 16:44:37 +0200 (CEST) > From: Dave U. Random<anonymous at anonymitaet-im-inter.net> > To:zfs-discuss at opensolaris.org > Subject: [zfs-discuss] Question on 4k sectors > Message-ID: > <07c48c7b7d3a7e568976704a375afc27 at anonymitaet-im-inter.net> > > Hi. Is the problem with ZFS supporting 4k sectors or is the problem mixing > 512 byte and 4k sector disks in one pool, or something else? I have seen > alot of discussion on the 4k issue but I haven''t understood what the actual > problem ZFS has with 4k sectors is. It''s getting harder and harder to find > large disks with 512 byte sectors so what should we do? TIA... >
On Thu, Jul 19, 2012 at 02:29:38PM +0200, Hans J. Albertsson wrote:> I think the problem is with disks that are 4k organised, but report > their blocksize as 512. > > If the disk reports it''s blocksize correctly as 4096, then ZFS should > not have a problem. > At least my 2TB Seagate Barracuda disks seemed to report their > blocksizes as 4096, and my zpools on those machines have ashift set to > 12, which is correct, since 2?? = 4096 > > You cannot mix 512 and 4096 byte blocksize disks in one pool, at least > not in a mirror. All disks in a single pool should have the same blocksize.Ashift is a vdev property. As far as I know you can have several vdevs with different ashifts in one pool. Besides, it is possible to mix 512 and 4096 byte blocksizes in a mirror vdev (and likely also in other vdev configurations), if and only if you start with the 4096 byte blocksize disk. That is, you can attach 512 byte blocksize disks to ashift=12 vdevs, but you can''t attach a 4096 byte blocksize disk to an ashift=9 vdev. And even that is not entirely true: some implementation of ZFS might choose to just warn about it or not care at all, while others will refuse to create or import such a pool. Hans -- %SYSTEM-F-ANARCHISM, The operating system has been overthrown
On Thu, Jul 19, 2012 at 5:29 AM, Hans J. Albertsson <hans.j.albertsson at branneriet.se> wrote:> I think the problem is with disks that are 4k organised, but report their > blocksize as 512. > > If the disk reports it''s blocksize correctly as 4096, then ZFS should not > have a problem. > At least my 2TB Seagate Barracuda disks seemed to report their blocksizes as > 4096, and my zpools on those machines have ashift set to 12, which is > correct, since 2?? = 4096 > > You cannot mix 512 and 4096 byte blocksize disks in one pool, at least not > in a mirror. All disks in a single pool should have the same blocksize. > > There is a hacked version of zpool for OpenIndiana that has a blocksize > option to the create subcommand. I don''t know if other OSes have similar > fixes.FreeBSD includes the gnop(8) command which can be used to create pseudo-devices that declare any size of sectors you want. This can be used to create ashift=12 vdevs on top of 512B, pseudo-512B, or 4K drives. # gnop -S 4096 da{0,1,2,3,4,5,6,7} # zpool create pool raidz2 da{0,1,2,3,4,5,6,7}.nop # zpool export pool # gnop destroy da{0,1,2,3,4,5,6,7}.nop # zpool import -d /dev pool -- Freddie Cash fjwcash at gmail.com
"Hans J. Albertsson" <hans.j.albertsson at branneriet.se> wrote:> I think the problem is with disks that are 4k organised, but report > their blocksize as 512. > > If the disk reports it''s blocksize correctly as 4096, then ZFS should > not have a problem. > At least my 2TB Seagate Barracuda disks seemed to report their > blocksizes as 4096, and my zpools on those machines have ashift set to > 12, which is correct, since 2?? = 4096Thanks, this is good to know. Is there any way, looking at manufacturers data sheets for drives, whether they report their blocksize correctly? From Seagate and WD that list the number of sectors, it''s trivial to determine what sectors the disk is using. But is this number what the disk is really organized in or is it the number the disk reports?! It is very confusing... So far we seem to rely on reports from people on the list, which is good for us but bad for guys who wasted money on drives that don''t work as they should (the drives that don''t report actual sector sector size correctly). Really, it would be so helpful to know which drives we can buy with confidence and which should be avoided...is there any way to know from the manufacturers web sites or do you have to actually buy one and see what it does? Thanks to everyone for the info.
2012-07-23 18:37, Anonymous wrote:> Really, it would be so helpful to know which drives we can buy with > confidence and which should be avoided...is there any way to know from the > manufacturers web sites or do you have to actually buy one and see what it > does? Thanks to everyone for the info.I think that vendors'' marking like 512e may give a clue on their support of "emulated 512-byte sectors", whatever they would mean by that for a specific model line. I believe you can roughly be certain that all 3Tb HDDs except Hitachi use 4Kb native sectors, and 4Tb disks are all 4Kb. If these disks don''t expose such sector sizing to the OS properly, you can work around that in several ways, including, as of recent illumos changes, an override config file for the SCSI driver. The main problem with "avoiding" 4kb drives seems to be just the cases where you want to replace a single disk in an older pool built with 512b-native sectored drives. For new pools (or rather new complete top-level VDEVs) this does not matter much, except that your overheads with small data blocks can get noticeably bigger. There were statements on this list that drives emulating 512b sectors (whether they announce it properly or not) are not all inherently evil - this emulation by itself may be of some concern regarding performance, but not one of reliability. Then again, firmware errors are possible in any part of the stack, of both older and newer models ;) HTH, //Jim
You wrote:> 2012-07-23 18:37, Anonymous wrote: > > Really, it would be so helpful to know which drives we can buy with > > confidence and which should be avoided...is there any way to know from the > > manufacturers web sites or do you have to actually buy one and see what it > > does? Thanks to everyone for the info. > > I think that vendors'' marking like 512e may give a clue on > their support of "emulated 512-byte sectors", whatever they > would mean by that for a specific model line.Yeah but buying through the mail it''s awfully difficult to see the vendor markings until it''s too late ;)> I believe you can roughly be certain that all 3Tb HDDs except > Hitachi use 4Kb native sectors, and 4Tb disks are all 4Kb. > If these disks don''t expose such sector sizing to the OS > properly, you can work around that in several ways, including, > as of recent illumos changes, an override config file for the > SCSI driver.The question was relative to some older boxes running S10 and not planning to upgrade the OS, keeping them alive as long as possible...> The main problem with "avoiding" 4kb drives seems to be just > the cases where you want to replace a single disk in an older > pool built with 512b-native sectored drives.Right, that''s what we''re concerned with.> For new pools (or rather new complete top-level VDEVs) this does not > matter much, except that your overheads with small data blocks can get > noticeably bigger.Understood.> There were statements on this list that drives emulating 512b > sectors (whether they announce it properly or not) are not > all inherently evil - this emulation by itself may be of some > concern regarding performance, but not one of reliability. > Then again, firmware errors are possible in any part of the > stack, of both older and newer models ;)I haven''t seen any post suggesting 512b emulation didn''t have very adverse effects on performance. Given how touchy ZFS seems to be I don''t want to give him any excuses! Thanks for your post.
On Mon, 23 Jul 2012, Anonymous Remailer (austria) wrote:> > The question was relative to some older boxes running S10 and not planning > to upgrade the OS, keeping them alive as long as possible...Recent Solaris 10 kernel patches are addressing drives with 4k sectors. It seems that Solaris 10 will work with drives with 4k sectors so Solaris 10 users will not be stuck. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/