Libguestfs - Dec 2015 - Re: packaging virtio-win

On 11/27/2015 12:16 PM, Roman Kagan wrote:
> On Thu, Nov 26, 2015 at 06:29:45PM -0600, Jeff Nelson wrote:
>> On Thu, Nov 26, 2015 at 08:12:41PM +0300, Roman Kagan wrote:
>>> Do I get it right that the way virtio-win is packaged for Fedora
and
>>> RHEL is driven by the scripts at
>>>
>>> https://github.com/crobinso/virtio-win-pkg-scripts
>>>
>>> and I should be submitting patches or pull requests to it?
>>
>> Yes, that's what I use to package the drivers for RHEL.
>>
>>
>>> Or should the layout be determined only by the build machinery at
>>>
>>> https://github.com/YanVugenfirer/kvm-guest-drivers-windows
>>>
>>> and I should be direct my submissions that way?
>>
>> I don't know, but if you work strictly with virtio-win-pkg-scripts
it
>> shouldn't matter.
> 
> Thanks a lot, I'll look into it early next week.
> 
Sorry I'm only getting to this thread now :/

I'd like to try and summarize the interrelated packaging changes that were
mentioned in this thread:

- The iso layout is not optimal for programmatic consumption
- The iso layout is not compatible with windows driver media autodetect
conventions
- The iso and RPM host contents do not match
- The reasoning behind the iso/RPM layout is not clear

To all this I say... preach, my brother :)

https://bugzilla.redhat.com/show_bug.cgi?id=1217658
https://bugzilla.redhat.com/show_bug.cgi?id=1167941

The ISO and RPM layout is largely the result of organic extensions and no one
ever tried to streamline things, so I fully support any work in that
direction. But the things to consider:

- There are many consumers of the iso layout as it is right now. Any changes
in the short to medium term need to be entirely additive so we can get the new
layout right without breaking every consumer. So keep the existing iso layout
of $drivername/$weird-os/$arch, but _add_ the $arch/$standard-os/* layout, and
avoid duplicates by hardlinking identical files (the scripts on github already
have code that handles the last bit). In fact it might be a good starting
point to write a script that takes the existing iso, extracts it, and
rearranges the contents to match the new format. Then I could help integrate
that with the scripts and my own testing.

- The layout should use the latest windows conventions for arch/osname. If
this means we have to diverge from the naming convention we currently use for
RPM host installed drivers, that's fine, since we can maintain back compat
there with symlinks. However I've tried searching for this info (googling
'windows driver cd layout/hierarchy/format') but I can't find
anything
definitive... anyone have a pointer? Especially WRT the current standard vs.
the pre-windows-7 standard that vadim pointed out.

- Anything regarding a separate iso with a separate layout to appease < win7
windows vintage should be handled separately, if at all IMO
> I guess there's no mailing list, so I'll have to do pull requests
on
> github, right?
Let's stick with email and virt-tools-list@redhat.com since there may be
lots
of discussion...

Thanks,
Cole

On Mon, 2015-11-30 at 17:56 -0500, Cole Robinson wrote:
> On 11/27/2015 12:16 PM, Roman Kagan wrote:
> > On Thu, Nov 26, 2015 at 06:29:45PM -0600, Jeff Nelson wrote:
> >> On Thu, Nov 26, 2015 at 08:12:41PM +0300, Roman Kagan wrote:
> >>> Do I get it right that the way virtio-win is packaged for
Fedora and
> >>> RHEL is driven by the scripts at
> >>>
> >>> https://github.com/crobinso/virtio-win-pkg-scripts
> >>>
> >>> and I should be submitting patches or pull requests to it?
> >>
> >> Yes, that's what I use to package the drivers for RHEL.
> >>
> >>
> >>> Or should the layout be determined only by the build machinery
at
> >>>
> >>> https://github.com/YanVugenfirer/kvm-guest-drivers-windows
> >>>
> >>> and I should be direct my submissions that way?
> >>
> >> I don't know, but if you work strictly with
virtio-win-pkg-scripts it
> >> shouldn't matter.
> > 
> > Thanks a lot, I'll look into it early next week.
> > 
> 
> Sorry I'm only getting to this thread now :/
> 
> I'd like to try and summarize the interrelated packaging changes that
were
> mentioned in this thread:
> 
> - The iso layout is not optimal for programmatic consumption
> - The iso layout is not compatible with windows driver media autodetect
> conventions
> - The iso and RPM host contents do not match
> - The reasoning behind the iso/RPM layout is not clear
> 
> To all this I say... preach, my brother :)
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1217658
> https://bugzilla.redhat.com/show_bug.cgi?id=1167941
> 
> The ISO and RPM layout is largely the result of organic extensions and no
one
> ever tried to streamline things, so I fully support any work in that
> direction. But the things to consider:
> 
> - There are many consumers of the iso layout as it is right now. Any
changes
> in the short to medium term need to be entirely additive so we can get the
new
> layout right without breaking every consumer. So keep the existing iso
layout
> of $drivername/$weird-os/$arch, but _add_ the $arch/$standard-os/* layout,
and
> avoid duplicates by hardlinking identical files (the scripts on github
already
> have code that handles the last bit). In fact it might be a good starting
> point to write a script that takes the existing iso, extracts it, and
> rearranges the contents to match the new format. Then I could help
integrate
> that with the scripts and my own testing.
> 
> - The layout should use the latest windows conventions for arch/osname. If
> this means we have to diverge from the naming convention we currently use
for
> RPM host installed drivers, that's fine, since we can maintain back
compat
> there with symlinks. However I've tried searching for this info
(googling
> 'windows driver cd layout/hierarchy/format') but I can't find
anything
> definitive... anyone have a pointer? Especially WRT the current standard
vs.
> the pre-windows-7 standard that vadim pointed out.
I don't remember all details, but there are some traces left in
bugzilla:
 
https://bugzilla.redhat.com/show_bug.cgi?id=708930
https://bugzilla.redhat.com/show_bug.cgi?id=700118#c31

Vadim.
> 
> - Anything regarding a separate iso with a separate layout to appease <
win7
> windows vintage should be handled separately, if at all IMO
> 
> > I guess there's no mailing list, so I'll have to do pull
requests on
> > github, right?
> 
> Let's stick with email and virt-tools-list@redhat.com since there may
be lots
> of discussion...
> 
> Thanks,
> Cole
>

On Mon, Nov 30, 2015 at 05:56:48PM -0500, Cole Robinson
wrote:
> - There are many consumers of the iso layout as it is right now. Any
> changes in the short to medium term need to be entirely additive so
> we can get the new layout right without breaking every consumer. So
> keep the existing iso layout of $drivername/$weird-os/$arch, but
> _add_ the $arch/$standard-os/* layout, and avoid duplicates by
> hardlinking identical files (the scripts on github already have code
> that handles the last bit).
May also be worth asking who the current consumers are.

Virt-v2v is one of them.

Are there others?  (GNOME Boxes?)

Virt-v2v uses a very simple scheme where it looks at each path element
for magic strings (eg. "/win2003/" or "/x86/") and from that
it works
out what Windows OS + arch the drivers in that directory are intended
for.  It's flexible enough that we can easily change it, but only if
we're told what the new convention will be.

https://github.com/libguestfs/libguestfs/blob/master/v2v/windows.ml#L132-L165

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/

On Mon, Nov 30, 2015 at 05:56:48PM -0500, Cole Robinson
wrote:
> https://bugzilla.redhat.com/show_bug.cgi?id=1167941
This one is restricted access so I can't see it.

The links and suggestions are very appreciated, thanks.
> > I guess there's no mailing list, so I'll have to do pull
requests on
> > github, right?
> 
> Let's stick with email and virt-tools-list at redhat.com since there
may be lots
> of discussion...
OK.  Then I'll start posting patches there once I have something ready.
I guess everybody interested is subscribed; if not please speak up so
that I cc you.

Thanks,
Roman.

On Tue, Dec 01, 2015 at 03:42:05PM +0300, Roman Kagan
wrote:
> On Mon, Nov 30, 2015 at 05:56:48PM -0500, Cole Robinson wrote:
> > https://bugzilla.redhat.com/show_bug.cgi?id=1167941
> 
> This one is restricted access so I can't see it.
I've opened it up - try again now.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

On 12/01/2015 04:49 AM, Richard W.M. Jones wrote:
> On Mon, Nov 30, 2015 at 05:56:48PM -0500, Cole Robinson wrote:
>> - There are many consumers of the iso layout as it is right now. Any
>> changes in the short to medium term need to be entirely additive so
>> we can get the new layout right without breaking every consumer. So
>> keep the existing iso layout of $drivername/$weird-os/$arch, but
>> _add_ the $arch/$standard-os/* layout, and avoid duplicates by
>> hardlinking identical files (the scripts on github already have code
>> that handles the last bit).
> 
> May also be worth asking who the current consumers are.
> 
> Virt-v2v is one of them.
> 
> Are there others?  (GNOME Boxes?)
> 
* ovirt, as part of building their own iso and guest installer
* spice-guest-tools.exe builds
* boxes/libosinfo get drivers from
https://zeenix.fedorapeople.org/drivers/win-tools/preinst/    which is just a
copy from a tested virtio-win release, but they control the filesystem layout.

Explicitly documenting this is one of the things on my todo list. But there
may be far more... who knows what RHEL customers are doing with the drivers.

- Cole

+Lev (oVirt guest installer)

----- Original Message -----
> From: "Cole Robinson" <crobinso@redhat.com>
> To: "Roman Kagan" <rkagan@virtuozzo.com>, "Jeff
Nelson" <jenelson@redhat.com>, "Richard W.M. Jones"
> <rjones@redhat.com>, "Denis Lunev"
<den@virtuozzo.com>, "Yan Vugenfirer"
<yvugenfi@redhat.com>, "Vadim Rozenfeld"
> <vrozenfe@redhat.com>, michen@redhat.com, "Amnon Ilan"
<ailan@redhat.com>, "Jin" <lijin@redhat.com>,
> libguestfs@redhat.com, "Alexander Burluka"
<aburluka@virtuozzo.com>
> Sent: Tuesday, December 1, 2015 12:56:48 AM
> Subject: Re: packaging virtio-win
> 
> On 11/27/2015 12:16 PM, Roman Kagan wrote:
> > On Thu, Nov 26, 2015 at 06:29:45PM -0600, Jeff Nelson wrote:
> >> On Thu, Nov 26, 2015 at 08:12:41PM +0300, Roman Kagan wrote:
> >>> Do I get it right that the way virtio-win is packaged for
Fedora and
> >>> RHEL is driven by the scripts at
> >>>
> >>> https://github.com/crobinso/virtio-win-pkg-scripts
> >>>
> >>> and I should be submitting patches or pull requests to it?
> >>
> >> Yes, that's what I use to package the drivers for RHEL.
> >>
> >>
> >>> Or should the layout be determined only by the build machinery
at
> >>>
> >>> https://github.com/YanVugenfirer/kvm-guest-drivers-windows
> >>>
> >>> and I should be direct my submissions that way?
> >>
> >> I don't know, but if you work strictly with
virtio-win-pkg-scripts it
> >> shouldn't matter.
> > 
> > Thanks a lot, I'll look into it early next week.
> > 
> 
> Sorry I'm only getting to this thread now :/
> 
> I'd like to try and summarize the interrelated packaging changes that
were
> mentioned in this thread:
> 
> - The iso layout is not optimal for programmatic consumption
> - The iso layout is not compatible with windows driver media autodetect
> conventions
> - The iso and RPM host contents do not match
> - The reasoning behind the iso/RPM layout is not clear
> 
> To all this I say... preach, my brother :)
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1217658
> https://bugzilla.redhat.com/show_bug.cgi?id=1167941
> 
> The ISO and RPM layout is largely the result of organic extensions and no
one
> ever tried to streamline things, so I fully support any work in that
> direction. But the things to consider:
> 
> - There are many consumers of the iso layout as it is right now. Any
changes
> in the short to medium term need to be entirely additive so we can get the
> new
> layout right without breaking every consumer. So keep the existing iso
layout
> of $drivername/$weird-os/$arch, but _add_ the $arch/$standard-os/* layout,
> and
> avoid duplicates by hardlinking identical files (the scripts on github
> already
> have code that handles the last bit). In fact it might be a good starting
> point to write a script that takes the existing iso, extracts it, and
> rearranges the contents to match the new format. Then I could help
integrate
> that with the scripts and my own testing.
> 
> - The layout should use the latest windows conventions for arch/osname. If
> this means we have to diverge from the naming convention we currently use
for
> RPM host installed drivers, that's fine, since we can maintain back
compat
> there with symlinks. However I've tried searching for this info
(googling
> 'windows driver cd layout/hierarchy/format') but I can't find
anything
> definitive... anyone have a pointer? Especially WRT the current standard
vs.
> the pre-windows-7 standard that vadim pointed out.
> 
> - Anything regarding a separate iso with a separate layout to appease <
win7
> windows vintage should be handled separately, if at all IMO
> 
> > I guess there's no mailing list, so I'll have to do pull
requests on
> > github, right?
> 
> Let's stick with email and virt-tools-list@redhat.com since there may
be lots
> of discussion...
> 
> Thanks,
> Cole
> 
>