Libguestfs - Feb 2020 - Re: [RFC] lib: allow to specify physical/logical block size for disks

пн, 10 лют. 2020 о 13:43 Richard W.M. Jones <rjones@redhat.com>
пише:
>
> On Sat, Feb 08, 2020 at 01:25:28AM +0200, Mykola Ivanets wrote:
> > From: Nikolay Ivanets <stenavin@gmail.com>
> >
> > I faced with situation where libguestfs cannot recognize partitions on
a
> > disk image which was partitioned on a system with "4K
native" sector
> > size support.
>
> Do you have a small test case for this?
We can easily create one with patched libguestfs and attach disk to
unpatched libguestfs.
> > In order to fix the issue we need to allow users to specify desired
> > physical and/or logical block size per drive basis.
>
> It seems like physical_block_size / logical_block_size in qemu are
> completely undocumented.  However I did some experiments with patching
> libguestfs and examining the qemu and parted code.  Here are my
> observations:
>
> (1) Setting only physical_block_size = 4096 seems to do nothing.
See my thoughts on this in previous email.
> (2) Setting only logical_block_size = 4096 is explicitly rejected by
> virtio-scsi:
>
>
https://git.qemu.org/?p=qemu.git;a=blob;f=hw/scsi/scsi-disk.c;h=10d0794d60f196f177563aae00bed2181f5c1bb1;hb=HEAD#l2352
>
> (A similar test exists for virtio-blk)
>
> (3) Setting both physical_block_size = logical_block_size = 4096
> changes how parted partitions GPT disks.  The partition table is
> clearly using 4K sectors as you can see by examining the disk
> afterwards with hexdump.
>
> (4) Neither setting changes MBR partitioning by parted, although my
> interpretation of Wikipedia indicates that it should be possible to
> create a MBR disk with 4K sector size.  Maybe I'm doing something
> wrong, or parted just doesn't support this case.
>
> So it appears that we should just have one blocksize control (maybe
> called "sectorsize"?) which sets both physical_block_size and
> logical_block_size to the same value.  It may also be worth enforcing
> that blocksize/sectorsize must be set to 512 or 4096 (which we can
> relax later if necessary).
If we stick with the only parameter, I think blocksize might be better name,
especially if we want to split this parameter somewhere latter.

Here are more precise restrictions:

Both values must be a power of 2 between 512 and 32768.
logical_block_size must be
less or equals to physical_block_size.

--
  Mykola Ivanets

On Mon, Feb 10, 2020 at 02:28:08PM +0200, Nikolay Ivanets
wrote:
> пн, 10 лют. 2020 о 13:43 Richard W.M. Jones <rjones@redhat.com> пише:
> >
> > On Sat, Feb 08, 2020 at 01:25:28AM +0200, Mykola Ivanets wrote:
> > > From: Nikolay Ivanets <stenavin@gmail.com>
> > >
> > > I faced with situation where libguestfs cannot recognize
partitions on a
> > > disk image which was partitioned on a system with "4K
native" sector
> > > size support.
> >
> > Do you have a small test case for this?
> 
> We can easily create one with patched libguestfs and attach disk to
> unpatched libguestfs.
> 
> > > In order to fix the issue we need to allow users to specify
desired
> > > physical and/or logical block size per drive basis.
> >
> > It seems like physical_block_size / logical_block_size in qemu are
> > completely undocumented.  However I did some experiments with patching
> > libguestfs and examining the qemu and parted code.  Here are my
> > observations:
> >
> > (1) Setting only physical_block_size = 4096 seems to do nothing.
> 
> See my thoughts on this in previous email.
> 
> > (2) Setting only logical_block_size = 4096 is explicitly rejected by
> > virtio-scsi:
> >
> >
https://git.qemu.org/?p=qemu.git;a=blob;f=hw/scsi/scsi-disk.c;h=10d0794d60f196f177563aae00bed2181f5c1bb1;hb=HEAD#l2352
> >
> > (A similar test exists for virtio-blk)
> >
> > (3) Setting both physical_block_size = logical_block_size = 4096
> > changes how parted partitions GPT disks.  The partition table is
> > clearly using 4K sectors as you can see by examining the disk
> > afterwards with hexdump.
> >
> > (4) Neither setting changes MBR partitioning by parted, although my
> > interpretation of Wikipedia indicates that it should be possible to
> > create a MBR disk with 4K sector size.  Maybe I'm doing something
> > wrong, or parted just doesn't support this case.
> >
> > So it appears that we should just have one blocksize control (maybe
> > called "sectorsize"?) which sets both physical_block_size
and
> > logical_block_size to the same value.  It may also be worth enforcing
> > that blocksize/sectorsize must be set to 512 or 4096 (which we can
> > relax later if necessary).
> 
> If we stick with the only parameter, I think blocksize might be better
name,
> especially if we want to split this parameter somewhere latter.
> 
> Here are more precise restrictions:
> 
> Both values must be a power of 2 between 512 and 32768.
> logical_block_size must be
> less or equals to physical_block_size.
Agreed, but note that we can relax restrictions later if we want, but
enforcing restrictions later is an ABI break.

The only disk format I'm aware of which uses !512 and !4K sectors are
CD ROMs (2K sector size), although libguestfs reads those without any
problems today.  Even if you consider NASes where 64K sectors are
normal, they still use 512 or 4K logical sectors (with lots of
horrible read-modify-write cycles).

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v

пн, 10 лют. 2020 о 15:02 Richard W.M. Jones <rjones@redhat.com>
пише:
>
> On Mon, Feb 10, 2020 at 02:28:08PM +0200, Nikolay Ivanets wrote:
> > пн, 10 лют. 2020 о 13:43 Richard W.M. Jones <rjones@redhat.com>
пише:
> > >
> > > On Sat, Feb 08, 2020 at 01:25:28AM +0200, Mykola Ivanets wrote:
> > > > From: Nikolay Ivanets <stenavin@gmail.com>
> > > >
> > > > I faced with situation where libguestfs cannot recognize
partitions on a
> > > > disk image which was partitioned on a system with "4K
native" sector
> > > > size support.
> > >
> > > Do you have a small test case for this?
> >
> > We can easily create one with patched libguestfs and attach disk to
> > unpatched libguestfs.
> >
> > > > In order to fix the issue we need to allow users to specify
desired
> > > > physical and/or logical block size per drive basis.
> > >
> > > It seems like physical_block_size / logical_block_size in qemu
are
> > > completely undocumented.  However I did some experiments with
patching
> > > libguestfs and examining the qemu and parted code.  Here are my
> > > observations:
> > >
> > > (1) Setting only physical_block_size = 4096 seems to do nothing.
> >
> > See my thoughts on this in previous email.
> >
> > > (2) Setting only logical_block_size = 4096 is explicitly rejected
by
> > > virtio-scsi:
> > >
> > >
https://git.qemu.org/?p=qemu.git;a=blob;f=hw/scsi/scsi-disk.c;h=10d0794d60f196f177563aae00bed2181f5c1bb1;hb=HEAD#l2352
> > >
> > > (A similar test exists for virtio-blk)
> > >
> > > (3) Setting both physical_block_size = logical_block_size = 4096
> > > changes how parted partitions GPT disks.  The partition table is
> > > clearly using 4K sectors as you can see by examining the disk
> > > afterwards with hexdump.
> > >
> > > (4) Neither setting changes MBR partitioning by parted, although
my
> > > interpretation of Wikipedia indicates that it should be possible
to
> > > create a MBR disk with 4K sector size.  Maybe I'm doing
something
> > > wrong, or parted just doesn't support this case.
> > >
> > > So it appears that we should just have one blocksize control
(maybe
> > > called "sectorsize"?) which sets both
physical_block_size and
> > > logical_block_size to the same value.  It may also be worth
enforcing
> > > that blocksize/sectorsize must be set to 512 or 4096 (which we
can
> > > relax later if necessary).
> >
> > If we stick with the only parameter, I think blocksize might be better
name,
> > especially if we want to split this parameter somewhere latter.
> >
> > Here are more precise restrictions:
> >
> > Both values must be a power of 2 between 512 and 32768.
> > logical_block_size must be
> > less or equals to physical_block_size.
>
> Agreed, but note that we can relax restrictions later if we want, but
> enforcing restrictions later is an ABI break.
>
> The only disk format I'm aware of which uses !512 and !4K sectors are
> CD ROMs (2K sector size), although libguestfs reads those without any
> problems today.  Even if you consider NASes where 64K sectors are
> normal, they still use 512 or 4K logical sectors (with lots of
> horrible read-modify-write cycles).
In this case we will reject libvirt XML with block size other then 512 and 4096.
I'm fine with that because other values are artificial.