Libguestfs - Nov 2015 - Re: Fwd: [PATCH] v2v: virtio-win: include *.dll too

On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld
wrote:
> On Wed, 2015-10-28 at 14:53 +0300, Roman Kagan wrote:
> >   1) tell which devices can be configured
> Not sure that I fully understated your question. but if you are going to
> create some sort of off-line automatic virtio-win drivers update
> utility, then it shouldn't be too complicated. Firs of all you will
need
> to obtain the Windows kernel version by reading the following Registry
> key - "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion".
Let's say it
> 6.3, which means that it is Win8.1 or WS2012R2, parsing
"BuildLabEx"
> string from the same hive will give you information about the platform
> bitness. Next you need to go through inf files, and find
"DriverVer"
> string, like this one . taken from from vioscsi.inf
> 
> DriverVer=08/01/2015,62.72.104.10800
> 
> This string contains build time and version stamps. The version stamp
> looks as follow "62.72.104.10800"
> where 62 means a target Windows kernel version multiplied by 10. In this
> case it is 6.2 which means Win8 or WS2012
> 72 - the target host platform version multiplied bu 10 (was RHEL7.2)
> 104 just a magic number, but it can be changed, don't make any
> assumption based on this number.
> 10800 our internal build number (build 108) multiplied by 100
Hi Vadim,

I've almost finished implementing this, but I have a couple of
technical questions:

(1) How can we tell from the .inf file what architecture (x86 or
amd64) the driver is for?  There is a [Manufacturer] section which
looks hopeful, eg:

  [Manufacturer]
  %RHEL%   = RHEL,NTAMD64

but the contents are not very consistent across all the .inf files I
have access to.

(2) If we have a directory containing multiple drivers, can we assume
that if any .inf file found matches the guest OS, then all those
drivers can be installed?  So far this appears to be true.  If not
then we have to start associating drivers with .inf files, which seems
a bit complicated.

(3) You write:
> If you found an inf file with the matching minor OS (6 in our case)
> version and matching or less but close minor version number (2 vs 3)
> then you are in the right directory.
When you say "close minor version number" is there a particular
definition you have in mind?  eg. minor or minor-1 is OK?  Or do we
have to consider all .inf files in totality and choose the nearest
one?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html

On Thu, 2015-11-05 at 21:46 +0000, Richard W.M. Jones
wrote:
> On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld wrote:
> > On Wed, 2015-10-28 at 14:53 +0300, Roman Kagan wrote:
> > >   1) tell which devices can be configured
> > Not sure that I fully understated your question. but if you are going
to
> > create some sort of off-line automatic virtio-win drivers update
> > utility, then it shouldn't be too complicated. Firs of all you
will need
> > to obtain the Windows kernel version by reading the following Registry
> > key - "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion".
Let's say it
> > 6.3, which means that it is Win8.1 or WS2012R2, parsing
"BuildLabEx"
> > string from the same hive will give you information about the platform
> > bitness. Next you need to go through inf files, and find
"DriverVer"
> > string, like this one . taken from from vioscsi.inf
> > 
> > DriverVer=08/01/2015,62.72.104.10800
> > 
> > This string contains build time and version stamps. The version stamp
> > looks as follow "62.72.104.10800"
> > where 62 means a target Windows kernel version multiplied by 10. In
this
> > case it is 6.2 which means Win8 or WS2012
> > 72 - the target host platform version multiplied bu 10 (was RHEL7.2)
> > 104 just a magic number, but it can be changed, don't make any
> > assumption based on this number.
> > 10800 our internal build number (build 108) multiplied by 100
> 
> Hi Vadim,
> 
> I've almost finished implementing this, but I have a couple of
> technical questions:
> 
> (1) How can we tell from the .inf file what architecture (x86 or
> amd64) the driver is for?  There is a [Manufacturer] section which
> looks hopeful, eg:
> 
>   [Manufacturer]
>   %RHEL%   = RHEL,NTAMD64
> 
> but the contents are not very consistent across all the .inf files I
> have access to.
Hi Richard,

It should be consistent. We do specify target architecture (amd64 or
x86) when stamping inf/inx files. IIRC, qemupciserial.inf is the only
exception, but it goes without binaries. In any case, if you are going
to use existing distribution media, you can rely on the media layout -
for example, if "amd64" is a part of your current path - you are in
64-bit directory. 
> 
> (2) If we have a directory containing multiple drivers, can we assume
> that if any .inf file found matches the guest OS, then all those
> drivers can be installed?  So far this appears to be true.  
Yes, we will never mix 32 and 64 bit drivers into the same directory.
I swear:) 
> If not
> then we have to start associating drivers with .inf files, which seems
> a bit complicated.
> 
> (3) You write:
> 
> > If you found an inf file with the matching minor OS (6 in our case)
> > version and matching or less but close minor version number (2 vs 3)
> > then you are in the right directory.
> 
> When you say "close minor version number" is there a particular
> definition you have in mind?  eg. minor or minor-1 is OK?  Or do we
> have to consider all .inf files in totality and choose the nearest
> one?
Ideally, we should try finding the exact match. If not, we have to find
one with the same major version number and nearest minor number which is
smaller than the target one. For example, let's say we are looking for a
driver for Win8.1/WS2012R2, we know that both of them have kernel
version 6.3 . So, ideally we should to find inf file with the following
driver version string:

DriverVer=XX/YY/ZZZZ,63.AA.BBB.CCCCC

if we cannot find an inf file with such criteria, we should try finding
another one with the OS version 62 (which means that a driver was build
for kernel 6.2 Win8/WS2012 and technically should be compatible with
6.3). So, we are looking for DriverVer which looks like this one:
DriverVer=XX/YY/ZZZZ,62.AA.BBB.CCCCC

Still no luck - decrease the target OS version to 61 and repeat.

For WDF drivers like balloon, vioserial, and rng you can go as far as
you can trying to find an appropriate driver. 

For miniport drivers like netkvm ( Yan, please confirm ), viostor and
vioscsi I wouldn't go too far. For example drivers designed to fork in
specific stacks at kernel version 5.1 (WinXP) can refuse working with
kernel 6.2 (Win8).

However, Win10 will be a problem. At least as long as we do not release
virtio-win drivers built for this particular OS version, and keep using 
drivers designed for Win8/WS2012 and Win8.1/WS2012R2 platforms.
Win10 has kernel version 10.0 (guys, who has Win10 RTM running - please
confirm). It used to be 6.4 for RC and Beta versions, but IIRC
Microsoft decided to change it to 10.0 for RTM and final release. 
Microsoft promised that drivers designed for Win8/WS2012 and
Win8.1/WS2012R2 platforms will continue working, at least for the
nearest future.

Vadim.
  
> 
> Rich.
>

> On 06 Nov 2015, at 08:51, Vadim Rozenfeld <vrozenfe@redhat.com>
wrote:
> 
> On Thu, 2015-11-05 at 21:46 +0000, Richard W.M. Jones wrote:
>> On Thu, Oct 29, 2015 at 11:05:42AM +1100, Vadim Rozenfeld wrote:
>>> On Wed, 2015-10-28 at 14:53 +0300, Roman Kagan wrote:
>>>>  1) tell which devices can be configured
>>> Not sure that I fully understated your question. but if you are
going to
>>> create some sort of off-line automatic virtio-win drivers update
>>> utility, then it shouldn't be too complicated. Firs of all you
will need
>>> to obtain the Windows kernel version by reading the following
Registry
>>> key - "HKLM\SOFTWARE\Microsoft\Windows
NT\CurrentVersion". Let's say it
>>> 6.3, which means that it is Win8.1 or WS2012R2, parsing
"BuildLabEx"
>>> string from the same hive will give you information about the
platform
>>> bitness. Next you need to go through inf files, and find
"DriverVer"
>>> string, like this one . taken from from vioscsi.inf
>>> 
>>> DriverVer=08/01/2015,62.72.104.10800
>>> 
>>> This string contains build time and version stamps. The version
stamp
>>> looks as follow "62.72.104.10800"
>>> where 62 means a target Windows kernel version multiplied by 10. In
this
>>> case it is 6.2 which means Win8 or WS2012
>>> 72 - the target host platform version multiplied bu 10 (was
RHEL7.2)
>>> 104 just a magic number, but it can be changed, don't make any
>>> assumption based on this number.
>>> 10800 our internal build number (build 108) multiplied by 100
>> 
>> Hi Vadim,
>> 
>> I've almost finished implementing this, but I have a couple of
>> technical questions:
>> 
>> (1) How can we tell from the .inf file what architecture (x86 or
>> amd64) the driver is for?  There is a [Manufacturer] section which
>> looks hopeful, eg:
>> 
>>  [Manufacturer]
>>  %RHEL%   = RHEL,NTAMD64
>> 
>> but the contents are not very consistent across all the .inf files I
>> have access to.
> 
> Hi Richard,
> 
> It should be consistent. We do specify target architecture (amd64 or
> x86) when stamping inf/inx files. IIRC, qemupciserial.inf is the only
> exception, but it goes without binaries. In any case, if you are going
> to use existing distribution media, you can rely on the media layout -
> for example, if "amd64" is a part of your current path - you are
in
> 64-bit directory. 
>> 
>> (2) If we have a directory containing multiple drivers, can we assume
>> that if any .inf file found matches the guest OS, then all those
>> drivers can be installed?  So far this appears to be true.  
> Yes, we will never mix 32 and 64 bit drivers into the same directory.
> I swear:) 
> 
>> If not
>> then we have to start associating drivers with .inf files, which seems
>> a bit complicated.
>> 
>> (3) You write:
>> 
>>> If you found an inf file with the matching minor OS (6 in our case)
>>> version and matching or less but close minor version number (2 vs
3)
>>> then you are in the right directory.
>> 
>> When you say "close minor version number" is there a
particular
>> definition you have in mind?  eg. minor or minor-1 is OK?  Or do we
>> have to consider all .inf files in totality and choose the nearest
>> one?
> Ideally, we should try finding the exact match. If not, we have to find
> one with the same major version number and nearest minor number which is
> smaller than the target one. For example, let's say we are looking for
a
> driver for Win8.1/WS2012R2, we know that both of them have kernel
> version 6.3 . So, ideally we should to find inf file with the following
> driver version string:
> 
> DriverVer=XX/YY/ZZZZ,63.AA.BBB.CCCCC
> 
> if we cannot find an inf file with such criteria, we should try finding
> another one with the OS version 62 (which means that a driver was build
> for kernel 6.2 Win8/WS2012 and technically should be compatible with
> 6.3). So, we are looking for DriverVer which looks like this one:
> DriverVer=XX/YY/ZZZZ,62.AA.BBB.CCCCC
> 
> Still no luck - decrease the target OS version to 61 and repeat.
> 
> For WDF drivers like balloon, vioserial, and rng you can go as far as
> you can trying to find an appropriate driver. 
> 
> For miniport drivers like netkvm ( Yan, please confirm ), viostor and
> vioscsi I wouldn't go too far. For example drivers designed to fork in
> specific stacks at kernel version 5.1 (WinXP) can refuse working with
> kernel 6.2 (Win8).
While NDIS miniport designed for NDIS 5.1 (XP) still can run on latest Windows,
I wouldn’t do that. For NDIS 6.x (Windows 2008 and newer), we can use older
drivers on new OS.
But installing a driver for a newer OS on older OS might lead to driver
miss-behaviour.

> 
> However, Win10 will be a problem. At least as long as we do not release
> virtio-win drivers built for this particular OS version, and keep using 
> drivers designed for Win8/WS2012 and Win8.1/WS2012R2 platforms.
> Win10 has kernel version 10.0 (guys, who has Win10 RTM running - please
> confirm). It used to be 6.4 for RC and Beta versions, but IIRC
> Microsoft decided to change it to 10.0 for RTM and final release. 
> Microsoft promised that drivers designed for Win8/WS2012 and
> Win8.1/WS2012R2 platforms will continue working, at least for the
> nearest future.
> 
> Vadim.
> 
>> 
>> Rich.

I think one - maybe final? - problem.  How can I tell the difference
between drivers for "client" versions of Windows (eg. Windows 7)
and server versions of Windows (eg. Windows 2008 Server)?

It seems in many or most cases the drivers are identical, eg:

$ md5sum viostor/2k12/amd64/* viostor/w8/amd64/*
bbe250c13bf891fd7292ccab9908a63a  viostor/2k12/amd64/viostor.cat
c2fdfc6a47fdc9dd25242441cdf1a68e  viostor/2k12/amd64/viostor.inf
5ca2cadca2214b90196755dab06676f4  viostor/2k12/amd64/viostor.pdb
80c5ea60a69f37e965cb277383abd067  viostor/2k12/amd64/viostor.sys
bbe250c13bf891fd7292ccab9908a63a  viostor/w8/amd64/viostor.cat
c2fdfc6a47fdc9dd25242441cdf1a68e  viostor/w8/amd64/viostor.inf
5ca2cadca2214b90196755dab06676f4  viostor/w8/amd64/viostor.pdb
80c5ea60a69f37e965cb277383abd067  viostor/w8/amd64/viostor.sys

$ md5sum Balloon/2k12/amd64/* Balloon/w8/amd64/*
d4104f08e674ab9def49d086d960349d  Balloon/2k12/amd64/balloon.cat
0325b7e0552e1567292eb6c1068fee18  Balloon/2k12/amd64/balloon.inf
ac2116f04fd678d0e264f34e3bf90d1e  Balloon/2k12/amd64/balloon.pdb
c975501dd5b43ade3717722b81acbc1f  Balloon/2k12/amd64/balloon.sys
56f63afdd992a5395a4d8da6a52e559e  Balloon/2k12/amd64/blnsvr.exe
c539f2056fb196da377d14180fda195c  Balloon/2k12/amd64/blnsvr.pdb
d10864c1730172780c2d4be633b9220a  Balloon/2k12/amd64/WdfCoInstaller01011.dll
d4104f08e674ab9def49d086d960349d  Balloon/w8/amd64/balloon.cat
0325b7e0552e1567292eb6c1068fee18  Balloon/w8/amd64/balloon.inf
ac2116f04fd678d0e264f34e3bf90d1e  Balloon/w8/amd64/balloon.pdb
c975501dd5b43ade3717722b81acbc1f  Balloon/w8/amd64/balloon.sys
56f63afdd992a5395a4d8da6a52e559e  Balloon/w8/amd64/blnsvr.exe
c539f2056fb196da377d14180fda195c  Balloon/w8/amd64/blnsvr.pdb
d10864c1730172780c2d4be633b9220a  Balloon/w8/amd64/WdfCoInstaller01011.dll

(completely identical)

$ md5sum viostor/2k8/amd64/* viostor/w7/amd64/*d1132fbd27c98f25ea71a6eb4037b2f8 
viostor/2k8/amd64/viostor.cat
defccbe48aca1cd97fee4d7b16053b72  viostor/2k8/amd64/viostor.inf
6da6120368c097806e4ac7fc6b940810  viostor/2k8/amd64/viostor.pdb
63b05edfb75d9dc60e82ab3c0bfee701  viostor/2k8/amd64/viostor.sys
af0cca02244fcfedf3209aef56610464  viostor/w7/amd64/viostor.cat
defccbe48aca1cd97fee4d7b16053b72  viostor/w7/amd64/viostor.inf
6da6120368c097806e4ac7fc6b940810  viostor/w7/amd64/viostor.pdb
63b05edfb75d9dc60e82ab3c0bfee701  viostor/w7/amd64/viostor.sys

(almost identical, *.cat files are different)

In any case the *.inf files don't seem to have any distinguishing
feature which would allow us to check this.

Maybe this doesn't matter?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/