Libguestfs - Feb 2020 - Re: [RFC] lib: allow to specify physical/logical block size for disks

Am 10.02.2020 um 12:43 hat Richard W.M. Jones
geschrieben:
> On Sat, Feb 08, 2020 at 01:25:28AM +0200, Mykola Ivanets wrote:
> > From: Nikolay Ivanets <stenavin@gmail.com>
> > 
> > I faced with situation where libguestfs cannot recognize partitions on
a
> > disk image which was partitioned on a system with "4K
native" sector
> > size support.
> 
> Do you have a small test case for this?
> 
> > In order to fix the issue we need to allow users to specify desired
> > physical and/or logical block size per drive basis.
> 
> It seems like physical_block_size / logical_block_size in qemu are
> completely undocumented.  However I did some experiments with patching
> libguestfs and examining the qemu and parted code.  Here are my
> observations:
> 
> (1) Setting only physical_block_size = 4096 seems to do nothing.
The guest sees the physical_block_size and can try to keep its requests
aligned as an optimisation. But it doesn't actually make a semantic
difference as to how the content of the disk is accessed.
> (2) Setting only logical_block_size = 4096 is explicitly rejected by
> virtio-scsi:
> 
>
https://git.qemu.org/?p=qemu.git;a=blob;f=hw/scsi/scsi-disk.c;h=10d0794d60f196f177563aae00bed2181f5c1bb1;hb=HEAD#l2352
> 
> (A similar test exists for virtio-blk)
> 
> (3) Setting both physical_block_size = logical_block_size = 4096
> changes how parted partitions GPT disks.  The partition table is
> clearly using 4K sectors as you can see by examining the disk
> afterwards with hexdump.
This is what you want for emulating a 4k native disk.
> (4) Neither setting changes MBR partitioning by parted, although my
> interpretation of Wikipedia indicates that it should be possible to
> create a MBR disk with 4K sector size.  Maybe I'm doing something
> wrong, or parted just doesn't support this case.
I seem to remember that 4k native disks require GPT, but if you say you
read otherwise, I'm not 100% sure about this any more.
> So it appears that we should just have one blocksize control (maybe
> called "sectorsize"?) which sets both physical_block_size and
> logical_block_size to the same value.  It may also be worth enforcing
> that blocksize/sectorsize must be set to 512 or 4096 (which we can
> relax later if necessary).
A single option (to control logical_block_size) makes sense for
libguestfs. physical_block_size is only relevant for the appliance and
not for the resulting image, so it can be treated as an implementation
detail.

Kevin

пн, 10 лют. 2020 о 15:48 Kevin Wolf <kwolf@redhat.com>
пише:
>
> Am 10.02.2020 um 12:43 hat Richard W.M. Jones geschrieben:
> > On Sat, Feb 08, 2020 at 01:25:28AM +0200, Mykola Ivanets wrote:
> > > From: Nikolay Ivanets <stenavin@gmail.com>
> > >
> > > I faced with situation where libguestfs cannot recognize
partitions on a
> > > disk image which was partitioned on a system with "4K
native" sector
> > > size support.
> >
> > Do you have a small test case for this?
> >
> > > In order to fix the issue we need to allow users to specify
desired
> > > physical and/or logical block size per drive basis.
> >
> > It seems like physical_block_size / logical_block_size in qemu are
> > completely undocumented.  However I did some experiments with patching
> > libguestfs and examining the qemu and parted code.  Here are my
> > observations:
> >
> > (1) Setting only physical_block_size = 4096 seems to do nothing.
>
> The guest sees the physical_block_size and can try to keep its requests
> aligned as an optimisation. But it doesn't actually make a semantic
> difference as to how the content of the disk is accessed.
>
> > (2) Setting only logical_block_size = 4096 is explicitly rejected by
> > virtio-scsi:
> >
> >
https://git.qemu.org/?p=qemu.git;a=blob;f=hw/scsi/scsi-disk.c;h=10d0794d60f196f177563aae00bed2181f5c1bb1;hb=HEAD#l2352
> >
> > (A similar test exists for virtio-blk)
> >
> > (3) Setting both physical_block_size = logical_block_size = 4096
> > changes how parted partitions GPT disks.  The partition table is
> > clearly using 4K sectors as you can see by examining the disk
> > afterwards with hexdump.
>
> This is what you want for emulating a 4k native disk.
>
> > (4) Neither setting changes MBR partitioning by parted, although my
> > interpretation of Wikipedia indicates that it should be possible to
> > create a MBR disk with 4K sector size.  Maybe I'm doing something
> > wrong, or parted just doesn't support this case.
>
> I seem to remember that 4k native disks require GPT, but if you say you
> read otherwise, I'm not 100% sure about this any more.
>
> > So it appears that we should just have one blocksize control (maybe
> > called "sectorsize"?) which sets both physical_block_size
and
> > logical_block_size to the same value.  It may also be worth enforcing
> > that blocksize/sectorsize must be set to 512 or 4096 (which we can
> > relax later if necessary).
>
> A single option (to control logical_block_size) makes sense for
> libguestfs. physical_block_size is only relevant for the appliance and
> not for the resulting image, so it can be treated as an implementation
> detail.
>
> Kevin
>
So, can we summarize?

- in libguestfs we will expose the only parameter called 'blocksize'
- 512 and 4096 are the only allowed values for 'blocksize' for now
- we will reject libvirt XML with values for physical_* and
logical_block_size other then 512 or 4096
- importing disks configuration from libvirt XML we will use logical_block_size

Richard, are we fine with that?

--
  Mykola Ivanets

On Mon, Feb 10, 2020 at 04:15:40PM +0200, Nikolay Ivanets
wrote:
> пн, 10 лют. 2020 о 15:48 Kevin Wolf <kwolf@redhat.com> пише:
> >
> > Am 10.02.2020 um 12:43 hat Richard W.M. Jones geschrieben:
> > > On Sat, Feb 08, 2020 at 01:25:28AM +0200, Mykola Ivanets wrote:
> > > > From: Nikolay Ivanets <stenavin@gmail.com>
> > > >
> > > > I faced with situation where libguestfs cannot recognize
partitions on a
> > > > disk image which was partitioned on a system with "4K
native" sector
> > > > size support.
> > >
> > > Do you have a small test case for this?
> > >
> > > > In order to fix the issue we need to allow users to specify
desired
> > > > physical and/or logical block size per drive basis.
> > >
> > > It seems like physical_block_size / logical_block_size in qemu
are
> > > completely undocumented.  However I did some experiments with
patching
> > > libguestfs and examining the qemu and parted code.  Here are my
> > > observations:
> > >
> > > (1) Setting only physical_block_size = 4096 seems to do nothing.
> >
> > The guest sees the physical_block_size and can try to keep its
requests
> > aligned as an optimisation. But it doesn't actually make a
semantic
> > difference as to how the content of the disk is accessed.
> >
> > > (2) Setting only logical_block_size = 4096 is explicitly rejected
by
> > > virtio-scsi:
> > >
> > >
https://git.qemu.org/?p=qemu.git;a=blob;f=hw/scsi/scsi-disk.c;h=10d0794d60f196f177563aae00bed2181f5c1bb1;hb=HEAD#l2352
> > >
> > > (A similar test exists for virtio-blk)
> > >
> > > (3) Setting both physical_block_size = logical_block_size = 4096
> > > changes how parted partitions GPT disks.  The partition table is
> > > clearly using 4K sectors as you can see by examining the disk
> > > afterwards with hexdump.
> >
> > This is what you want for emulating a 4k native disk.
> >
> > > (4) Neither setting changes MBR partitioning by parted, although
my
> > > interpretation of Wikipedia indicates that it should be possible
to
> > > create a MBR disk with 4K sector size.  Maybe I'm doing
something
> > > wrong, or parted just doesn't support this case.
> >
> > I seem to remember that 4k native disks require GPT, but if you say
you
> > read otherwise, I'm not 100% sure about this any more.
> >
> > > So it appears that we should just have one blocksize control
(maybe
> > > called "sectorsize"?) which sets both
physical_block_size and
> > > logical_block_size to the same value.  It may also be worth
enforcing
> > > that blocksize/sectorsize must be set to 512 or 4096 (which we
can
> > > relax later if necessary).
> >
> > A single option (to control logical_block_size) makes sense for
> > libguestfs. physical_block_size is only relevant for the appliance and
> > not for the resulting image, so it can be treated as an implementation
> > detail.
> >
> > Kevin
> >
> 
> So, can we summarize?
> 
> - in libguestfs we will expose the only parameter called
'blocksize'
> - 512 and 4096 are the only allowed values for 'blocksize' for now
... and unset, which means 512.
> - we will reject libvirt XML with values for physical_* and
>   logical_block_size other then 512 or 4096
> - importing disks configuration from libvirt XML we will use
logical_block_size
> 
> Richard, are we fine with that?
Yup, looks good to me, thanks for investigating this issue.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/