I just thought I''d mention this because it turned out to be the cause of disk I/O performance issues I''ve been having. For the arhchive/search engines of nothing else. Setup: Storage server (Linux): 1) ZFS zvol backed images, ZFS configured for 4KB sector disks 2) IETD iSCSI server dom0 (Linux): 3) Open-iSCSI client (Linux) domU is Windows. When 2) is set for 4096 byte sectors, the performance is terrible, and qemu-dm tops out at about 5MB/s when it hits the CPU limit. Reduce it to 512 bytes, re-partition accordingly to ensure the partition is 4KB aligned, and the performance goes up massively. Booting up domU now takes seconds rather than minutes, and doing the fresh-install-millions-of-updates update takes minutes rather than hours. Something in the stack really seems to grind to a halt when dealing with 4KB physical sectors at the moment. Nothing breaks or warns, it just runs really slowly. Gordan
On Wed, 2013-06-05 at 07:47 +0100, Gordan Bobic wrote:> I just thought I''d mention this because it turned out to be the cause of > disk I/O performance issues I''ve been having. For the arhchive/search > engines of nothing else. > > Setup: > > Storage server (Linux): > 1) ZFS zvol backed images, ZFS configured for 4KB sector disks > 2) IETD iSCSI server > > dom0 (Linux): > 3) Open-iSCSI client (Linux) > > domU is Windows.Does Windows know about 4k sectors, or does that end at #3? I can imagine that doing a read/modify/write for each 512 bytes written would kill performance. You mention qemu so I guess this is using emulated IDE -- I''d be very surprised if that supported anything other than 512b sectors, perhaps the upstream qemu version might but I doubt the Xen traditional fork does. I don''t know if it is possible to get PV drivers into the Windows installer super early but if it was (driver disk?) that would be useful, otherwise installing them ASAP after initial install would seem wise. The only other thing which springs to mind is enabling jumbo frames on the network path. Not sure how critical that is these days with GSO and GRO etc. Ian.
On Thu, 6 Jun 2013 09:58:51 +0100, Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Wed, 2013-06-05 at 07:47 +0100, Gordan Bobic wrote: >> I just thought I''d mention this because it turned out to be the >> cause of >> disk I/O performance issues I''ve been having. For the >> arhchive/search >> engines of nothing else. >> >> Setup: >> >> Storage server (Linux): >> 1) ZFS zvol backed images, ZFS configured for 4KB sector disks >> 2) IETD iSCSI server >> >> dom0 (Linux): >> 3) Open-iSCSI client (Linux) >> >> domU is Windows. > > Does Windows know about 4k sectors, or does that end at #3? I can > imagine that doing a read/modify/write for each 512 bytes written > would > kill performance.Good question. I don''t know, but it can see all the space on the volume, and there is no 512B emulation going on at any point up to and including point #3. The sectors, at lest to dom0, show up as 4096 bytes, both physical and logical.> You mention qemu so I guess this is using emulated IDE -- I''d be very > surprised if that supported anything other than 512b sectors, perhaps > the upstream qemu version might but I doubt the Xen traditional fork > does. I don''t know if it is possible to get PV drivers into the > Windows > installer super early but if it was (driver disk?) that would be > useful, > otherwise installing them ASAP after initial install would seem wise.I''m not sure if it "supports" anything other than 512B sectors, but it seemed to work without any errors/corruption, albeit very slowly. Installing PV drivers resulted in BSODs during booting. Ideally, I''m hoping to get the domU booting to an iSCSI share directly via the PCI-passthrough NIC, thus avoiding the need for PV drivers alltogether. Gordan
> > domU is Windows. > > Does Windows know about 4k sectors, or does that end at #3? I can > imagine that doing a read/modify/write for each 512 bytes written would > kill performance.http://support.microsoft.com/kb/2510009>=8/2012 can handle native 4kb sectors<8/2012 knows how to handle alignment for 512e but not 4kb sectors There was a patch recently for vbd to advertise physical sector size... without that I''d be a bit nervous about actually using even 512e in case windows makes incorrect assumptions about the underlying alignment. James