Xen devel - Dec 2009 - Possible regression in "passthough: add no_wb option for pci conf write"

> passthough: add no_wb option for pci conf write
> 
> Current pt_pci_write_config always writes back to real pci conf
> space. However, in the case of MSI address and data registers,
> if guest changes the affinity of the interrupt, stale data will
> be written to these registers. This is particularly a problem
> if Xen uses per-CPU vector, where the interrupt in question fails
> to work. This patch fixes this by adding an option to disable the
> write back of certain controls.
This patch seems to cause the creation of passing a domain
to fail	when passing through either an intel 82576 or 82572EI.
I didn''t try any others.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Sun, 2009-12-06 at 13:55 +0800, Simon Horman wrote:
> > passthough: add no_wb option for pci conf write
> > 
> > Current pt_pci_write_config always writes back to real pci conf
> > space. However, in the case of MSI address and data registers,
> > if guest changes the affinity of the interrupt, stale data will
> > be written to these registers. This is particularly a problem
> > if Xen uses per-CPU vector, where the interrupt in question fails
> > to work. This patch fixes this by adding an option to disable the
> > write back of certain controls.
> 
> This patch seems to cause the creation of passing a domain
> to fail	when passing through either an intel 82576 or 82572EI.
> I didn''t try any others.
> 
Hi Simon,
	Do you notice anything unusual in the ioemu and hypervisor logs?

Thanks,
Qing

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, Dec 07, 2009 at 10:16:40AM +0800, Qing He wrote:
> On Sun, 2009-12-06 at 13:55 +0800, Simon Horman wrote:
> > > passthough: add no_wb option for pci conf write
> > > 
> > > Current pt_pci_write_config always writes back to real pci conf
> > > space. However, in the case of MSI address and data registers,
> > > if guest changes the affinity of the interrupt, stale data will
> > > be written to these registers. This is particularly a problem
> > > if Xen uses per-CPU vector, where the interrupt in question fails
> > > to work. This patch fixes this by adding an option to disable the
> > > write back of certain controls.
> > 
> > This patch seems to cause the creation of passing a domain
> > to fail	when passing through either an intel 82576 or 82572EI.
> > I didn''t try any others.
> > 
> 
> Hi Simon,
> 	Do you notice anything unusual in the ioemu and hypervisor logs?
I couldn''t see anything of particular note. I''ll try and poke
a bit harder.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, 2009-12-07 at 10:55 +0800, Simon Horman wrote:
> On Mon, Dec 07, 2009 at 10:16:40AM +0800, Qing He wrote:
> > On Sun, 2009-12-06 at 13:55 +0800, Simon Horman wrote:
> > > > passthough: add no_wb option for pci conf write
> > > > 
> > > > Current pt_pci_write_config always writes back to real pci
conf
> > > > space. However, in the case of MSI address and data
registers,
> > > > if guest changes the affinity of the interrupt, stale data
will
> > > > be written to these registers. This is particularly a
problem
> > > > if Xen uses per-CPU vector, where the interrupt in question
fails
> > > > to work. This patch fixes this by adding an option to
disable the
> > > > write back of certain controls.
> > > 
> > > This patch seems to cause the creation of passing a domain
> > > to fail	when passing through either an intel 82576 or 82572EI.
> > > I didn''t try any others.
> > > 
> > 
> > Hi Simon,
> > 	Do you notice anything unusual in the ioemu and hypervisor logs?
> 
> I couldn''t see anything of particular note. I''ll try and
poke a bit harder.
> 
Thank you, I''ll also try at my side. What''s your
configuration? On what
changeset are you experiencing this?

Thanks,
Qing

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, Dec 07, 2009 at 10:57:50AM +0800, Qing He wrote:
> On Mon, 2009-12-07 at 10:55 +0800, Simon Horman wrote:
> > On Mon, Dec 07, 2009 at 10:16:40AM +0800, Qing He wrote:
> > > On Sun, 2009-12-06 at 13:55 +0800, Simon Horman wrote:
> > > > > passthough: add no_wb option for pci conf write
> > > > > 
> > > > > Current pt_pci_write_config always writes back to real
pci conf
> > > > > space. However, in the case of MSI address and data
registers,
> > > > > if guest changes the affinity of the interrupt, stale
data will
> > > > > be written to these registers. This is particularly a
problem
> > > > > if Xen uses per-CPU vector, where the interrupt in
question fails
> > > > > to work. This patch fixes this by adding an option to
disable the
> > > > > write back of certain controls.
> > > > 
> > > > This patch seems to cause the creation of passing a domain
> > > > to fail	when passing through either an intel 82576 or
82572EI.
> > > > I didn''t try any others.
> > > > 
> > > 
> > > Hi Simon,
> > > 	Do you notice anything unusual in the ioemu and hypervisor logs?
> > 
> > I couldn''t see anything of particular note. I''ll try
and poke a bit harder.
> > 
> 
> Thank you, I''ll also try at my side. What''s your
configuration? On what
> changeset are you experiencing this?
Hi,

I have attached logs from xend and qemu-dm. Unfortunately they
don''t seem to shed much light on the problem other than that
qemu-dm exits prematurely.

The "bad" logs are using staging/xen-unstable.hg:

	changeset:   20587:763bc7e6b3b4
	tag:         tip
	user:        Keir Fraser <keir.fraser@citrix.com>
	date:        Sat Dec 05 12:32:34 2009 +0000
	summary:     x86_32: Fix build after 20575:0930d17589a6

And staging/qemu-xen-unstable.git:

	commit e2b98415256cb264bc25e6df539ec0dc9d1b85b0
	Author: Ian Jackson <ian.jackson@eu.citrix.com>
	Date:   Fri Nov 6 18:11:50 2009 +0000

	passthough: add no_wb option for pci conf write

	Current pt_pci_write_config always writes back to real pci conf
	space. However, in the case of MSI address and data registers,
	if guest changes the affinity of the interrupt, stale data will
	be written to these registers. This is particularly a problem
	if Xen uses per-CPU vector, where the interrupt in question fails
	to work. This patch fixes this by adding an option to disable the
	write back of certain controls.

	Signed-off-by: Qing He <qing.he@intel.com>

the "good" logs use the same version of staging/xen-unstable.hg and
the previous commit of staging/qemu-xen-unstable.git:

	commit de6c06808e300b35368178a281660ae8acf64f66
	Author: Ian Jackson <ian.jackson@eu.citrix.com>
	Date:   Fri Nov 6 18:10:44 2009 +0000

	Enlarge the size of the global mmio_space mmio[].

	With the Multi-Function passthrough, we''re actually able to assign
	more than 32 functions to guest, so we should enlarge the MAX_MMIO.
	1024 should be big enough.

	Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>

Dom0 is linux-xen 2.6.18.8 + by igb backport

	URL: http://hg.vergenet.net/xen/linux-2.6.18-xen-igb/

	changeset:   1123:f4deded581e6
	tag:         tip
	parent:      1121:4b89768ad33b
	parent:      1122:f1a207ccb493
	user:        Simon Horman <horms@verge.net.au>
	date:        Mon Aug 31 09:54:00 2009 +1000
	summary:     Merge with http://xenbits.xensource.com/linux-2.6.18-xen.hg






_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, 2009-12-07 at 11:43 +0800, Simon Horman wrote:
> [2009-12-07 12:32:03 3360] DEBUG (DevController:139) Waiting for devices
tap2.
> [2009-12-07 12:32:03 3360] DEBUG (DevController:139) Waiting for devices
tap.
> [2009-12-07 12:32:03 3360] DEBUG (DevController:139) Waiting for devices
vtpm.
> [2009-12-07 12:32:03 3360] INFO (XendDomain:1206) Domain hvm (8) unpaused.
> [2009-12-07 12:32:03 3360] WARNING (image:544) domain hvm: device model
failure: pid 8472: died due to signal 11; see /var/log/xen/qemu-dm-hvm.log
> [2009-12-07 12:32:03 3360] WARNING (XendDomainInfo:1983) Domain has
crashed: name=hvm id=8.
> [2009-12-07 12:32:03 3360] ERROR (XendDomainInfo:2117) VM hvm restarting
too fast (Elapsed time: 0.161757 seconds). Refusing to restart to avoid loops.
> [2009-12-07 12:32:03 3360] DEBUG (XendDomainInfo:2961)
XendDomainInfo.destroy: domid=8
>From the `bad'' log, it seems that the device model actually
crashes,
by receiving a SIGSEGV. So you are obseversing that ioemu window
disappear at boot time?

Thanks,
Qing

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, Dec 07, 2009 at 01:29:28PM +0800, Qing He wrote:
> On Mon, 2009-12-07 at 11:43 +0800, Simon Horman wrote:
> > [2009-12-07 12:32:03 3360] DEBUG (DevController:139) Waiting for
devices tap2.
> > [2009-12-07 12:32:03 3360] DEBUG (DevController:139) Waiting for
devices tap.
> > [2009-12-07 12:32:03 3360] DEBUG (DevController:139) Waiting for
devices vtpm.
> > [2009-12-07 12:32:03 3360] INFO (XendDomain:1206) Domain hvm (8)
unpaused.
> > [2009-12-07 12:32:03 3360] WARNING (image:544) domain hvm: device
model failure: pid 8472: died due to signal 11; see /var/log/xen/qemu-dm-hvm.log
> > [2009-12-07 12:32:03 3360] WARNING (XendDomainInfo:1983) Domain has
crashed: name=hvm id=8.
> > [2009-12-07 12:32:03 3360] ERROR (XendDomainInfo:2117) VM hvm
restarting too fast (Elapsed time: 0.161757 seconds). Refusing to restart to
avoid loops.
> > [2009-12-07 12:32:03 3360] DEBUG (XendDomainInfo:2961)
XendDomainInfo.destroy: domid=8
> 
> >From the `bad'' log, it seems that the device model actually
crashes,
> by receiving a SIGSEGV.
Yes, I think that is the case.
> So you are obseversing that ioemu window
> disappear at boot time?
I''m not sure what you mean by "ioemu window".

I start the domain with xm -c ..., and after a few seconds it reports that
it can''t connect to the console.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, 2009-12-07 at 13:29 +0800, Qing He wrote:
> On Mon, 2009-12-07 at 11:43 +0800, Simon Horman wrote:
> > [2009-12-07 12:32:03 3360] DEBUG (DevController:139) Waiting for
devices tap2.
> > [2009-12-07 12:32:03 3360] DEBUG (DevController:139) Waiting for
devices tap.
> > [2009-12-07 12:32:03 3360] DEBUG (DevController:139) Waiting for
devices vtpm.
> > [2009-12-07 12:32:03 3360] INFO (XendDomain:1206) Domain hvm (8)
unpaused.
> > [2009-12-07 12:32:03 3360] WARNING (image:544) domain hvm: device
model failure: pid 8472: died due to signal 11; see /var/log/xen/qemu-dm-hvm.log
> > [2009-12-07 12:32:03 3360] WARNING (XendDomainInfo:1983) Domain has
crashed: name=hvm id=8.
> > [2009-12-07 12:32:03 3360] ERROR (XendDomainInfo:2117) VM hvm
restarting too fast (Elapsed time: 0.161757 seconds). Refusing to restart to
avoid loops.
> > [2009-12-07 12:32:03 3360] DEBUG (XendDomainInfo:2961)
XendDomainInfo.destroy: domid=8
> 
> >From the `bad'' log, it seems that the device model actually
crashes,
> by receiving a SIGSEGV. So you are obseversing that ioemu window
> disappear at boot time?
Ok, I understand the problem, if pt_find_reg_grp() fails and
returns NULL, it will jump to out:, but at this time reg is
still NULL (pt_find_reg() is not reached) which leads to a
NULL dereference.

I''ll fix it, thanks for reporting.

Thanks,
Qing
> 
> Thanks,
> Qing
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, 2009-12-07 at 13:46 +0800, Simon Horman wrote:
> > So you are obseversing that ioemu window
> > disappear at boot time?
> 
> I''m not sure what you mean by "ioemu window".
> 
Never mind, I meant vnc or sdl console of ioemu.

Can you try if the following patch fixes the problem?

---
diff --git a/hw/pass-through.c b/hw/pass-through.c
index b125d72..f28f6c7 100644
--- a/hw/pass-through.c
+++ b/hw/pass-through.c
@@ -1556,7 +1556,7 @@ static void pt_pci_write_config(PCIDevice *d, uint32_t
address, uint32_t val,
     val >>= ((address & 3) << 3);
 
 out:
-    if (!reg->no_wb) {
+    if (!(reg && reg->no_wb)) {  /* unknown regs are passed through
*/
         ret = pci_write_block(pci_dev, address, (uint8_t *)&val, len);
 
         if (!ret)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, Dec 07, 2009 at 01:57:28PM +0800, Qing He wrote:
> On Mon, 2009-12-07 at 13:46 +0800, Simon Horman wrote:
> > > So you are obseversing that ioemu window
> > > disappear at boot time?
> > 
> > I''m not sure what you mean by "ioemu window".
> > 
> Never mind, I meant vnc or sdl console of ioemu.
> 
> Can you try if the following patch fixes the problem?
Thanks, that seems to resolve the problem that I observed.
> ---
> diff --git a/hw/pass-through.c b/hw/pass-through.c
> index b125d72..f28f6c7 100644
> --- a/hw/pass-through.c
> +++ b/hw/pass-through.c
> @@ -1556,7 +1556,7 @@ static void pt_pci_write_config(PCIDevice *d,
uint32_t address, uint32_t val,
>      val >>= ((address & 3) << 3);
>  
>  out:
> -    if (!reg->no_wb) {
> +    if (!(reg && reg->no_wb)) {  /* unknown regs are passed
through */
>          ret = pci_write_block(pci_dev, address, (uint8_t *)&val, len);
>  
>          if (!ret)
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Qing He writes ("Re: [Xen-devel] Re: Possible regression in
"passthough: add no_wb option for pci conf
write""):
> Can you try if the following patch fixes the problem?
Simon Horman writes ("Re: [Xen-devel] Re: Possible regression in
"passthough: add no_wb option for pci conf
write""):
> Thanks, that seems to resolve the problem that I observed.
Thanks to both of you and I''ve applied this patch.  (It didn''t
have a
Signed-Off-By from Qing He but it''s only one line so that''s
OK.)

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, 2009-12-07 at 20:45 +0800, Ian Jackson wrote:
> Thanks to both of you and I''ve applied this patch.  (It
didn''t have a
> Signed-Off-By from Qing He but it''s only one line so
that''s OK.)
That''s fine. Thank you.

Qing
> 
> Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel