Libguestfs - Nov 2015 - Re: packaging virtio-win (was: Re: [PATCH] v2v: virtio-win: include *.dll too)

On Thu, Nov 26, 2015 at 08:12:41PM +0300, Roman Kagan
wrote:
>On Tue, Nov 24, 2015 at 01:07:03PM +0000, Richard W.M. Jones wrote:
>> On Mon, Nov 23, 2015 at 08:26:10PM +0300, Roman Kagan wrote:
>> > Having given it some more thought and having discussed that with
>> > colleagues who do various things with the drivers, like generating
>> > unattended install helpers or just manually installing drivers
into
>> > guests, I'm now of the opinion that neither libguestfs nor any
other
>> > user of the drivers should look inside the driver files in order
to
>> > match them against the guest OS, because it's hard for
programs and
>> > unrealistic for humans.
>> >
>> > I tend to believe it should be the responsibility of the driver
packages
>> > to lay out the drivers in a directory structure that both humans
and
>> > programs could grok easily and that wouldn't change once
established; in
>> > a sense this would be both an API and a human interface.
>> >
>> > If I were to lay it out I'd group the stuff by guest OS, so
that for a
>> > particular guest a single entity -- a directory tree or a
pre-created
>> > ISO or floppy image -- would contain all that's needed for
this guest
>> > and nothing else.  This would allow separation of concerns, so
that
>> > actors who bring the stuff into the guest wouldn't need to
care about
>> > what's inside, and actors involved with its installation
wouldn't need
>> > to know how to find the right one.
>>
>> The good news is that this is what we do right now.
>>
>> > E.g. from this POV the way drivers are stored in RHEL7 virtio-win
>> > package on the *filesystem* looks OK, the way they are stored in
the
>> > *ISO image* in the same package doesn't.
>> >
>> > As for naming one can stick with what's being used in that
virtio-win
>> > rpm, i.e.
>> >
>> > {amd64,i386}/Win{XP,2003,2008,2008R2,2012,2012R2,7,8,8.1}
>> >
>> > or switch to the names passed to inf2cat in its /os parameter,
i.e.
>> >
>> >
XP_X86,Server2003_X86,...,8_X64,Server8_X64,6_3_X86,6_3_X64,Server6_3_X64
>> >
(https://msdn.microsoft.com/en-us/library/windows/hardware/ff547089(v=vs.85).aspx)
>> >
>> > Whatever convention is chosen it needs to be followed thereafter.
>>
>> Yup - very important that once we decide what the naming convention is
>> going to be, we stick with it.  Moving drivers around or using ad-hoc
>> naming schemes causes churn in virt-v2v and probably other projects
>> too.
>>
>> > The main problem with this proposal is that I don't know how
to
>> > implement it: there are multiple vendors of virtio-win packages,
and,
>> > for licensing reasons, they don't even share the source code,
so it's
>> > unclear who could own this convention and enforce it on others.
>>
>> I guess that virt-v2v will have to deal with this with more special
>> rules.
>>
>> > I'd appreciate any thoughts or suggestions on how to proceed
from that
>> > and whether it's worth it at all.
>
>Apparently I failed to inspire anybody who can affect the driver
>packaging layout :)  Let me try again.
>
>Do I get it right that the way virtio-win is packaged for Fedora and
>RHEL is driven by the scripts at
>
>https://github.com/crobinso/virtio-win-pkg-scripts
>
>and I should be submitting patches or pull requests to it?
Yes, that's what I use to package the drivers for RHEL.

>Or should the layout be determined only by the build machinery at
>
>https://github.com/YanVugenfirer/kvm-guest-drivers-windows
>
>and I should be direct my submissions that way?
I don't know, but if you work strictly with virtio-win-pkg-scripts it
shouldn't matter.

>Or any combination of the above?
>
>Also does anybody know this is done in SUSE?
No idea.
>Any help would be appreciated.
>
>Thanks,
>Roman.

-Jeff

On Thu, Nov 26, 2015 at 06:29:45PM -0600, Jeff Nelson
wrote:
> On Thu, Nov 26, 2015 at 08:12:41PM +0300, Roman Kagan wrote:
> >Do I get it right that the way virtio-win is packaged for Fedora and
> >RHEL is driven by the scripts at
> >
> >https://github.com/crobinso/virtio-win-pkg-scripts
> >
> >and I should be submitting patches or pull requests to it?
> 
> Yes, that's what I use to package the drivers for RHEL.
> 
> 
> >Or should the layout be determined only by the build machinery at
> >
> >https://github.com/YanVugenfirer/kvm-guest-drivers-windows
> >
> >and I should be direct my submissions that way?
> 
> I don't know, but if you work strictly with virtio-win-pkg-scripts it
> shouldn't matter.
Thanks a lot, I'll look into it early next week.

I guess there's no mailing list, so I'll have to do pull requests on
github, right?

Thanks,
Roman.

On Fri, Nov 27, 2015 at 08:16:46PM +0300, Roman Kagan
wrote:
> On Thu, Nov 26, 2015 at 06:29:45PM -0600, Jeff Nelson wrote:
> > On Thu, Nov 26, 2015 at 08:12:41PM +0300, Roman Kagan wrote:
> > >Do I get it right that the way virtio-win is packaged for Fedora
and
> > >RHEL is driven by the scripts at
> > >
> > >https://github.com/crobinso/virtio-win-pkg-scripts
> > >
> > >and I should be submitting patches or pull requests to it?
> > 
> > Yes, that's what I use to package the drivers for RHEL.
> > 
> > 
> > >Or should the layout be determined only by the build machinery at
> > >
> > >https://github.com/YanVugenfirer/kvm-guest-drivers-windows
> > >
> > >and I should be direct my submissions that way?
> > 
> > I don't know, but if you work strictly with virtio-win-pkg-scripts
it
> > shouldn't matter.
> 
> Thanks a lot, I'll look into it early next week.
> 
> I guess there's no mailing list, so I'll have to do pull requests
on
> github, right?
I guess for general discussion, the virt-tools mailing list is the
right one, but use pull requests on github for patches I suppose.

  https://www.redhat.com/mailman/listinfo/virt-tools-list

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top

On 11/27/2015 12:16 PM, Roman Kagan wrote:
> On Thu, Nov 26, 2015 at 06:29:45PM -0600, Jeff Nelson wrote:
>> On Thu, Nov 26, 2015 at 08:12:41PM +0300, Roman Kagan wrote:
>>> Do I get it right that the way virtio-win is packaged for Fedora
and
>>> RHEL is driven by the scripts at
>>>
>>> https://github.com/crobinso/virtio-win-pkg-scripts
>>>
>>> and I should be submitting patches or pull requests to it?
>>
>> Yes, that's what I use to package the drivers for RHEL.
>>
>>
>>> Or should the layout be determined only by the build machinery at
>>>
>>> https://github.com/YanVugenfirer/kvm-guest-drivers-windows
>>>
>>> and I should be direct my submissions that way?
>>
>> I don't know, but if you work strictly with virtio-win-pkg-scripts
it
>> shouldn't matter.
> 
> Thanks a lot, I'll look into it early next week.
> 
Sorry I'm only getting to this thread now :/

I'd like to try and summarize the interrelated packaging changes that were
mentioned in this thread:

- The iso layout is not optimal for programmatic consumption
- The iso layout is not compatible with windows driver media autodetect
conventions
- The iso and RPM host contents do not match
- The reasoning behind the iso/RPM layout is not clear

To all this I say... preach, my brother :)

https://bugzilla.redhat.com/show_bug.cgi?id=1217658
https://bugzilla.redhat.com/show_bug.cgi?id=1167941

The ISO and RPM layout is largely the result of organic extensions and no one
ever tried to streamline things, so I fully support any work in that
direction. But the things to consider:

- There are many consumers of the iso layout as it is right now. Any changes
in the short to medium term need to be entirely additive so we can get the new
layout right without breaking every consumer. So keep the existing iso layout
of $drivername/$weird-os/$arch, but _add_ the $arch/$standard-os/* layout, and
avoid duplicates by hardlinking identical files (the scripts on github already
have code that handles the last bit). In fact it might be a good starting
point to write a script that takes the existing iso, extracts it, and
rearranges the contents to match the new format. Then I could help integrate
that with the scripts and my own testing.

- The layout should use the latest windows conventions for arch/osname. If
this means we have to diverge from the naming convention we currently use for
RPM host installed drivers, that's fine, since we can maintain back compat
there with symlinks. However I've tried searching for this info (googling
'windows driver cd layout/hierarchy/format') but I can't find
anything
definitive... anyone have a pointer? Especially WRT the current standard vs.
the pre-windows-7 standard that vadim pointed out.

- Anything regarding a separate iso with a separate layout to appease < win7
windows vintage should be handled separately, if at all IMO
> I guess there's no mailing list, so I'll have to do pull requests
on
> github, right?
Let's stick with email and virt-tools-list@redhat.com since there may be
lots
of discussion...

Thanks,
Cole