freebsd stable - Jul 2015 - Interesting Boot failure on HEAD with a large number of IGB nics

All
 I am wondering if anyone has run into this issue before , and if there is
a fix.
I have a Scalable Informatics siRouter with 24 Intel I350 igb nics and 8
intel 82599ES ixgbe/ix nic . The SiRouter is a Supermicro X9DRX+-F with a
bunch of intel nics.

On Head I run into a boot panic out of the box while plumbing the usb
controller . The box crashes with

ehci0: <Intel Patsburg USB 2.0 controller> mem 0xdd923000-0xdd9233ff irq
16
at device 26.0 on pci0
panic: Couldn't find an APIC vector for IRQ 16

However it has only probed 8 of the 24 igb when is crashes. Here is the
complete boot up with some additional info.

http://pastebin.com/MfbEwwg4

Now the interesting thing.

 DragonFly 4.2x boots fine  see the dmesg here
http://dmesgd.nycbug.org/index.cgi?action=dmesgd&do=view&id=2778

OpenBSD boots and finds the nics as well see dmesg here
http://dmesgd.nycbug.org/index.cgi?action=dmesgd&do=view&id=2777

NetBSD Pukes while probing something out of the box and I cant get the
serial console to work.

Linux 2.6 CentOS 6.6 works

OmniOS / Illumos-gate and Illumos-joyent work out of the box



So my goal here is to make the box boot FreeBSD . It had been used prior
running OmniOS and CentOS and well now I want to make it work. :)

Any ideas here ?

mark saad | nonesuch at longcount.org

On Wed, Jul 22, 2015 at 12:11:09PM -0400, Mark Saad
wrote:
> All
>  I am wondering if anyone has run into this issue before , and if there is
> a fix.
> I have a Scalable Informatics siRouter with 24 Intel I350 igb nics and 8
> intel 82599ES ixgbe/ix nic . The SiRouter is a Supermicro X9DRX+-F with a
> bunch of intel nics.
> 
> On Head I run into a boot panic out of the box while plumbing the usb
> controller . The box crashes with
> 
> ehci0: <Intel Patsburg USB 2.0 controller> mem 0xdd923000-0xdd9233ff
irq 16
> at device 26.0 on pci0
> panic: Couldn't find an APIC vector for IRQ 16
> 
> However it has only probed 8 of the 24 igb when is crashes. Here is the
> complete boot up with some additional info.
> 
> http://pastebin.com/MfbEwwg4
> 
> Now the interesting thing.
> 
>  DragonFly 4.2x boots fine  see the dmesg here
> http://dmesgd.nycbug.org/index.cgi?action=dmesgd&do=view&id=2778
> 
> OpenBSD boots and finds the nics as well see dmesg here
> http://dmesgd.nycbug.org/index.cgi?action=dmesgd&do=view&id=2777
> 
> NetBSD Pukes while probing something out of the box and I cant get the
> serial console to work.
> 
> Linux 2.6 CentOS 6.6 works
> 
> OmniOS / Illumos-gate and Illumos-joyent work out of the box
> 
> 
> 
> So my goal here is to make the box boot FreeBSD . It had been used prior
> running OmniOS and CentOS and well now I want to make it work. :)
> 
> Any ideas here ?

I'm no expert, but you may want to try setting

hw.igb.num_queues=1

and maybe

hw.ixgbe.num_queues=1

in the boot loader and trying that.

Regards,

Gary

P.S. freebsd-net@ may be a better list than either -stable or -hackers.