Re: [Xen-devel] Network dies and kernel errors

Konrad Rzeszutek Wilk
Fri, 29 Jul 2011 17:31:29 -0000

> > Did you try that? Did that make any difference?
>
> Not tested I did install one.
>
> I think I found a way to keep it running.
> On the new igb driver I built from new intel source added module parameter
> IntMode=1.
>
> This puts it in msi mode. It was in msi-x mode.
> It''s never died with that setting.
> It''s up now over a day.
> No real experience with msi-x. I think it''s the first time I have
seen a
> driver use msi-x interrupts.
> Maybe that gives you more ideas?

That was my thought - the MSI-X aren''t somehow being ACKed properly.
But I don''t
know if the issue with Dom0 or Xen.
>
>
> >
> > > Any ideas?
> >
> > There is a Xen parameter called ''noirqbalance'' . Try
that. Also see if you
> > can limit the CPUs in the dom0 using these two arguments on Xen
> > hypervisor:
> >
> Should I turn off the irqbalence daemon also?

Sure.

> Just in case you wonder it does with out it.
>
> > dom0_vcpus=2 dom0_vcpus_pin=1
> >
> >
> > It would be interesting to narrow down _when_ you trigger this
failure. B/c
> > we can pull Xen to see what the MSI''s are ''xl
debug-keys M'' _before_ and
> > _after_ your failure to see if something is amiss.
> >
> > Mainly to figure out if the vectors are moving around the CPUs (or
not)
> >
> > (XEN)  MSI    29 vec=21 lowest  edge   assert  log lowest
dest=00000001
> > mask=0/0/-1
> >
> > and also ''xl debug-keys i'' to see if the domain has
ACK-ed the interrupt:
> > (XEN)    IRQ:  29 affinity:00000000,00000000,00000000,00000001 vec:21
> > type=PCI-MSI         status=00000010 in-flight=0
domain-list=0:275(----),
> >
> > (the last ''----'' might have something else in in
them - if so that is a
> > sign that dom0 hasn''t picked up the event/vector).
>
> Much of my frustration is that I have not found a way to get it to fail
other
> than waiting a long time :-(

Ah that sucks. Well, just make a nice shell script that will run those
continously
(and also ''xl dmesg'') and pipe the log to a file.