Libguestfs - Nov 2015 - Re: [PATCH] v2v: virtio-win: include *.dll too

> On 06 Nov 2015, at 08:51, Vadim Rozenfeld <vrozenfe@redhat.com>
wrote:
> 
> On Thu, 2015-11-05 at 21:46 +0000, Richard W.M. Jones wrote:
>> On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld wrote:
>>> On Wed, 2015-10-28 at 14:53 +0300, Roman Kagan wrote:
>>>>  1) tell which devices can be configured
>>> Not sure that I fully understated your question. but if you are
going to
>>> create some sort of off-line automatic virtio-win drivers update
>>> utility, then it shouldn't be too complicated. Firs of all you
will need
>>> to obtain the Windows kernel version by reading the following
Registry
>>> key - "HKLM\SOFTWARE\Microsoft\Windows
NT\CurrentVersion". Let's say it
>>> 6.3, which means that it is Win8.1 or WS2012R2, parsing
"BuildLabEx"
>>> string from the same hive will give you information about the
platform
>>> bitness. Next you need to go through inf files, and find
"DriverVer"
>>> string, like this one . taken from from vioscsi.inf
>>> 
>>> DriverVer=08/01/2015,62.72.104.10800
>>> 
>>> This string contains build time and version stamps. The version
stamp
>>> looks as follow "62.72.104.10800"
>>> where 62 means a target Windows kernel version multiplied by 10. In
this
>>> case it is 6.2 which means Win8 or WS2012
>>> 72 - the target host platform version multiplied bu 10 (was
RHEL7.2)
>>> 104 just a magic number, but it can be changed, don't make any
>>> assumption based on this number.
>>> 10800 our internal build number (build 108) multiplied by 100
>> 
>> Hi Vadim,
>> 
>> I've almost finished implementing this, but I have a couple of
>> technical questions:
>> 
>> (1) How can we tell from the .inf file what architecture (x86 or
>> amd64) the driver is for?  There is a [Manufacturer] section which
>> looks hopeful, eg:
>> 
>>  [Manufacturer]
>>  %RHEL%   = RHEL,NTAMD64
>> 
>> but the contents are not very consistent across all the .inf files I
>> have access to.
> 
> Hi Richard,
> 
> It should be consistent. We do specify target architecture (amd64 or
> x86) when stamping inf/inx files. IIRC, qemupciserial.inf is the only
> exception, but it goes without binaries. In any case, if you are going
> to use existing distribution media, you can rely on the media layout -
> for example, if "amd64" is a part of your current path - you are
in
> 64-bit directory. 
>> 
>> (2) If we have a directory containing multiple drivers, can we assume
>> that if any .inf file found matches the guest OS, then all those
>> drivers can be installed?  So far this appears to be true.  
> Yes, we will never mix 32 and 64 bit drivers into the same directory.
> I swear:) 
> 
>> If not
>> then we have to start associating drivers with .inf files, which seems
>> a bit complicated.
>> 
>> (3) You write:
>> 
>>> If you found an inf file with the matching minor OS (6 in our case)
>>> version and matching or less but close minor version number (2 vs
3)
>>> then you are in the right directory.
>> 
>> When you say "close minor version number" is there a
particular
>> definition you have in mind?  eg. minor or minor-1 is OK?  Or do we
>> have to consider all .inf files in totality and choose the nearest
>> one?
> Ideally, we should try finding the exact match. If not, we have to find
> one with the same major version number and nearest minor number which is
> smaller than the target one. For example, let's say we are looking for
a
> driver for Win8.1/WS2012R2, we know that both of them have kernel
> version 6.3 . So, ideally we should to find inf file with the following
> driver version string:
> 
> DriverVer=XX/YY/ZZZZ,63.AA.BBB.CCCCC
> 
> if we cannot find an inf file with such criteria, we should try finding
> another one with the OS version 62 (which means that a driver was build
> for kernel 6.2 Win8/WS2012 and technically should be compatible with
> 6.3). So, we are looking for DriverVer which looks like this one:
> DriverVer=XX/YY/ZZZZ,62.AA.BBB.CCCCC
> 
> Still no luck - decrease the target OS version to 61 and repeat.
> 
> For WDF drivers like balloon, vioserial, and rng you can go as far as
> you can trying to find an appropriate driver. 
> 
> For miniport drivers like netkvm ( Yan, please confirm ), viostor and
> vioscsi I wouldn't go too far. For example drivers designed to fork in
> specific stacks at kernel version 5.1 (WinXP) can refuse working with
> kernel 6.2 (Win8).
While NDIS miniport designed for NDIS 5.1 (XP) still can run on latest Windows,
I wouldn’t do that. For NDIS 6.x (Windows 2008 and newer), we can use older
drivers on new OS.
But installing a driver for a newer OS on older OS might lead to driver
miss-behaviour.

> 
> However, Win10 will be a problem. At least as long as we do not release
> virtio-win drivers built for this particular OS version, and keep using 
> drivers designed for Win8/WS2012 and Win8.1/WS2012R2 platforms.
> Win10 has kernel version 10.0 (guys, who has Win10 RTM running - please
> confirm). It used to be 6.4 for RC and Beta versions, but IIRC
> Microsoft decided to change it to 10.0 for RTM and final release. 
> Microsoft promised that drivers designed for Win8/WS2012 and
> Win8.1/WS2012R2 platforms will continue working, at least for the
> nearest future.
> 
> Vadim.
> 
>> 
>> Rich.

On Sun, Nov 08, 2015 at 10:15:31AM +0200, Yan Vugenfirer
wrote:
> 
> > On 06 Nov 2015, at 08:51, Vadim Rozenfeld <vrozenfe@redhat.com>
wrote:
> > 
> > On Thu, 2015-11-05 at 21:46 +0000, Richard W.M. Jones wrote:
> >> On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld wrote:
> >>> On Wed, 2015-10-28 at 14:53 +0300, Roman Kagan wrote:
> >>>>  1) tell which devices can be configured
> >>> Not sure that I fully understated your question. but if you
are going to
> >>> create some sort of off-line automatic virtio-win drivers
update
> >>> utility, then it shouldn't be too complicated. Firs of all
you will need
> >>> to obtain the Windows kernel version by reading the following
Registry
> >>> key - "HKLM\SOFTWARE\Microsoft\Windows
NT\CurrentVersion". Let's say it
> >>> 6.3, which means that it is Win8.1 or WS2012R2, parsing
"BuildLabEx"
> >>> string from the same hive will give you information about the
platform
> >>> bitness. Next you need to go through inf files, and find
"DriverVer"
> >>> string, like this one . taken from from vioscsi.inf
> >>> 
> >>> DriverVer=08/01/2015,62.72.104.10800
> >>> 
> >>> This string contains build time and version stamps. The
version stamp
> >>> looks as follow "62.72.104.10800"
> >>> where 62 means a target Windows kernel version multiplied by
10. In this
> >>> case it is 6.2 which means Win8 or WS2012
> >>> 72 - the target host platform version multiplied bu 10 (was
RHEL7.2)
> >>> 104 just a magic number, but it can be changed, don't make
any
> >>> assumption based on this number.
> >>> 10800 our internal build number (build 108) multiplied by 100
> >> 
> >> Hi Vadim,
> >> 
> >> I've almost finished implementing this, but I have a couple of
> >> technical questions:
> >> 
> >> (1) How can we tell from the .inf file what architecture (x86 or
> >> amd64) the driver is for?  There is a [Manufacturer] section which
> >> looks hopeful, eg:
> >> 
> >>  [Manufacturer]
> >>  %RHEL%   = RHEL,NTAMD64
> >> 
> >> but the contents are not very consistent across all the .inf files
I
> >> have access to.
> > 
> > Hi Richard,
> > 
> > It should be consistent. We do specify target architecture (amd64 or
> > x86) when stamping inf/inx files. IIRC, qemupciserial.inf is the only
> > exception, but it goes without binaries. In any case, if you are going
> > to use existing distribution media, you can rely on the media layout -
> > for example, if "amd64" is a part of your current path - you
are in
> > 64-bit directory. 
> >> 
> >> (2) If we have a directory containing multiple drivers, can we
assume
> >> that if any .inf file found matches the guest OS, then all those
> >> drivers can be installed?  So far this appears to be true.  
> > Yes, we will never mix 32 and 64 bit drivers into the same directory.
> > I swear:) 
> > 
> >> If not
> >> then we have to start associating drivers with .inf files, which
seems
> >> a bit complicated.
> >> 
> >> (3) You write:
> >> 
> >>> If you found an inf file with the matching minor OS (6 in our
case)
> >>> version and matching or less but close minor version number (2
vs 3)
> >>> then you are in the right directory.
> >> 
> >> When you say "close minor version number" is there a
particular
> >> definition you have in mind?  eg. minor or minor-1 is OK?  Or do
we
> >> have to consider all .inf files in totality and choose the nearest
> >> one?
> > Ideally, we should try finding the exact match. If not, we have to
find
> > one with the same major version number and nearest minor number which
is
> > smaller than the target one. For example, let's say we are looking
for a
> > driver for Win8.1/WS2012R2, we know that both of them have kernel
> > version 6.3 . So, ideally we should to find inf file with the
following
> > driver version string:
> > 
> > DriverVer=XX/YY/ZZZZ,63.AA.BBB.CCCCC
> > 
> > if we cannot find an inf file with such criteria, we should try
finding
> > another one with the OS version 62 (which means that a driver was
build
> > for kernel 6.2 Win8/WS2012 and technically should be compatible with
> > 6.3). So, we are looking for DriverVer which looks like this one:
> > DriverVer=XX/YY/ZZZZ,62.AA.BBB.CCCCC
> > 
> > Still no luck - decrease the target OS version to 61 and repeat.
> > 
> > For WDF drivers like balloon, vioserial, and rng you can go as far as
> > you can trying to find an appropriate driver. 
> > 
> > For miniport drivers like netkvm ( Yan, please confirm ), viostor and
> > vioscsi I wouldn't go too far. For example drivers designed to
fork in
> > specific stacks at kernel version 5.1 (WinXP) can refuse working with
> > kernel 6.2 (Win8).
> 
> While NDIS miniport designed for NDIS 5.1 (XP) still can run on latest
Windows, I wouldn’t do that. For NDIS 6.x (Windows 2008 and newer), we can use
older drivers on new OS.
> But installing a driver for a newer OS on older OS might lead to driver
miss-behaviour.
I'm wondering how people who don't rely on virt-v2v are supposed to
discover the virtio drivers for their guests?  This includes both those
who do it programmatically and those who locate the appropriate drivers
manually.

I mean, both manual and programmatic procedures should resolve to the
same set of drivers, and ideally the procedure should be the same,
declared official and documented.

I see no problem implementing any particular algorithm, no matter how
complex, in v2v, but I'd like to make sure that my colleagues who, say,
write a tool for unattended deployment of Windows guests, or others, who
manually set up their Windows VMs, arrive at the same configuration.

I tend to think that your original idea of sorting that out on the
driver build side, and storing the resulting drivers grouped by
Windows arch/version following a well-defined and stable naming
convention is a more reliable and universal solution.

Thanks,
Roman.

On Mon, 2015-11-09 at 18:08 +0300, Roman Kagan wrote:
> On Sun, Nov 08, 2015 at 10:15:31AM +0200, Yan Vugenfirer wrote:
> > 
> > > On 06 Nov 2015, at 08:51, Vadim Rozenfeld
<vrozenfe@redhat.com> wrote:
> > > 
> > > On Thu, 2015-11-05 at 21:46 +0000, Richard W.M. Jones wrote:
> > >> On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld
wrote:
> > >>> On Wed, 2015-10-28 at 14:53 +0300, Roman Kagan wrote:
> > >>>>  1) tell which devices can be configured
> > >>> Not sure that I fully understated your question. but if
you are going to
> > >>> create some sort of off-line automatic virtio-win drivers
update
> > >>> utility, then it shouldn't be too complicated. Firs
of all you will need
> > >>> to obtain the Windows kernel version by reading the
following Registry
> > >>> key - "HKLM\SOFTWARE\Microsoft\Windows
NT\CurrentVersion". Let's say it
> > >>> 6.3, which means that it is Win8.1 or WS2012R2, parsing
"BuildLabEx"
> > >>> string from the same hive will give you information about
the platform
> > >>> bitness. Next you need to go through inf files, and find
"DriverVer"
> > >>> string, like this one . taken from from vioscsi.inf
> > >>> 
> > >>> DriverVer=08/01/2015,62.72.104.10800
> > >>> 
> > >>> This string contains build time and version stamps. The
version stamp
> > >>> looks as follow "62.72.104.10800"
> > >>> where 62 means a target Windows kernel version multiplied
by 10. In this
> > >>> case it is 6.2 which means Win8 or WS2012
> > >>> 72 - the target host platform version multiplied bu 10
(was RHEL7.2)
> > >>> 104 just a magic number, but it can be changed, don't
make any
> > >>> assumption based on this number.
> > >>> 10800 our internal build number (build 108) multiplied by
100
> > >> 
> > >> Hi Vadim,
> > >> 
> > >> I've almost finished implementing this, but I have a
couple of
> > >> technical questions:
> > >> 
> > >> (1) How can we tell from the .inf file what architecture (x86
or
> > >> amd64) the driver is for?  There is a [Manufacturer] section
which
> > >> looks hopeful, eg:
> > >> 
> > >>  [Manufacturer]
> > >>  %RHEL%   = RHEL,NTAMD64
> > >> 
> > >> but the contents are not very consistent across all the .inf
files I
> > >> have access to.
> > > 
> > > Hi Richard,
> > > 
> > > It should be consistent. We do specify target architecture (amd64
or
> > > x86) when stamping inf/inx files. IIRC, qemupciserial.inf is the
only
> > > exception, but it goes without binaries. In any case, if you are
going
> > > to use existing distribution media, you can rely on the media
layout -
> > > for example, if "amd64" is a part of your current path
- you are in
> > > 64-bit directory. 
> > >> 
> > >> (2) If we have a directory containing multiple drivers, can
we assume
> > >> that if any .inf file found matches the guest OS, then all
those
> > >> drivers can be installed?  So far this appears to be true.  
> > > Yes, we will never mix 32 and 64 bit drivers into the same
directory.
> > > I swear:) 
> > > 
> > >> If not
> > >> then we have to start associating drivers with .inf files,
which seems
> > >> a bit complicated.
> > >> 
> > >> (3) You write:
> > >> 
> > >>> If you found an inf file with the matching minor OS (6 in
our case)
> > >>> version and matching or less but close minor version
number (2 vs 3)
> > >>> then you are in the right directory.
> > >> 
> > >> When you say "close minor version number" is there
a particular
> > >> definition you have in mind?  eg. minor or minor-1 is OK?  Or
do we
> > >> have to consider all .inf files in totality and choose the
nearest
> > >> one?
> > > Ideally, we should try finding the exact match. If not, we have
to find
> > > one with the same major version number and nearest minor number
which is
> > > smaller than the target one. For example, let's say we are
looking for a
> > > driver for Win8.1/WS2012R2, we know that both of them have kernel
> > > version 6.3 . So, ideally we should to find inf file with the
following
> > > driver version string:
> > > 
> > > DriverVer=XX/YY/ZZZZ,63.AA.BBB.CCCCC
> > > 
> > > if we cannot find an inf file with such criteria, we should try
finding
> > > another one with the OS version 62 (which means that a driver was
build
> > > for kernel 6.2 Win8/WS2012 and technically should be compatible
with
> > > 6.3). So, we are looking for DriverVer which looks like this one:
> > > DriverVer=XX/YY/ZZZZ,62.AA.BBB.CCCCC
> > > 
> > > Still no luck - decrease the target OS version to 61 and repeat.
> > > 
> > > For WDF drivers like balloon, vioserial, and rng you can go as
far as
> > > you can trying to find an appropriate driver. 
> > > 
> > > For miniport drivers like netkvm ( Yan, please confirm ), viostor
and
> > > vioscsi I wouldn't go too far. For example drivers designed
to fork in
> > > specific stacks at kernel version 5.1 (WinXP) can refuse working
with
> > > kernel 6.2 (Win8).
> > 
> > While NDIS miniport designed for NDIS 5.1 (XP) still can run on latest
Windows, I wouldn’t do that. For NDIS 6.x (Windows 2008 and newer), we can use
older drivers on new OS.
> > But installing a driver for a newer OS on older OS might lead to
driver miss-behaviour.
> 
> I'm wondering how people who don't rely on virt-v2v are supposed to
> discover the virtio drivers for their guests?  This includes both those
> who do it programmatically and those who locate the appropriate drivers
> manually.
> 
> I mean, both manual and programmatic procedures should resolve to the
> same set of drivers, and ideally the procedure should be the same,
> declared official and documented.
> 
> I see no problem implementing any particular algorithm, no matter how
> complex, in v2v, but I'd like to make sure that my colleagues who, say,
> write a tool for unattended deployment of Windows guests, or others, who
> manually set up their Windows VMs, arrive at the same configuration.
> 
> I tend to think that your original idea of sorting that out on the
> driver build side, and storing the resulting drivers grouped by
> Windows arch/version following a well-defined and stable naming
> convention is a more reliable and universal solution.
> 
It definitely shouldn't be so complicated. Packaging drivers on
per-platform base can solve most of the problems. One single
iso image, containing both 32 and 64 bit drivers compatible to 
run on Win8/Win8.1/WS2012/WS2012R2, and exposing an appropriate 
media layout "understandable" by all these platforms, is  what 
we need. After all, the information provided n this thread, should be
enough for writing a script that can split our single iso image 
into several properly build ones. In this case, the only thing you 
should worry about, is that <os>\<architecture> naming pattern
is not the same among the various Windows platforms. IIRC there were
some changes around Vista/Longhorn when Microsoft did it.

Cheers,
Vadim.

> Thanks,
> Roman.