On 09/27/2016 08:57, Borja Marcos wrote:>
>> On 27 Sep 2016, at 15:48, Jan Henrik Sylvester <me at janh.de>
wrote:
>>
>> On 09/27/2016 12:16, Borja Marcos wrote:
>>> I have noticed that the GENERIC kernel in 11-STABLE includes the
>>> PCI_HP option, and the hotplug bits seem to be present in the
>>> kernel, but I don?t see any userland support for it.
>>>
>>> Is it somewhat complete and in that case am I missing something?
>>
>> I do not know kind of userland support you mean. I just tried:
>>
>> Plugging in my USB 3.0 ExpressCard while 11.0 is running, the
>> controller was detected and I was able to use USB devices with it.
>> Great.
>
> Thanks :)
>
> I was hoping (and I assume it?s the ultimate goal of the project) to
> be able to hot plug PCIe devices such as NVMe drives.
>
> On Solaris you can replace them provided you power them off
> previously (there?s a command for that, ?hotplug?).
>
> On FreeBSD I?ve tried using devctl but powering off, disabling a
> device and enabling it again has led to a panic.
>
> Interestingly, I disabled nvme0 using devctl and "nvmecontrol
> devlist" didn?t find any nvme controllers despite having 10
> controllers and 10 drives. However, the ZFS pool of 10 NVMe drives
> was working happily. Degraded of course, with one NVMe missing.
To my knowledge, all the necessary PCIe-layer code is present. However,
that's just one layer: Many drivers will likely need changes in order
to cope with surprise removal of their devices.
For that reason, HotPlug needs a lot of testing on a variety of
platforms. The FreeBSD developer base is much smaller than its user
base, of course, so the variety of our testing is rather limited. You
can help immensely by giving us detailed bug reports, either on a
mailing list or in Bugzilla. For a panic, the panic messages and stack
trace of the current thread will be very helpful. Complete crashinfo(8)
output would be great.
The most relevant userland tool is devctl, followed closely by devinfo
and pciconf.
In the case of Jan's USB 3.0 ExpressCard, it's possible that one or all
of the USB controller drivers (xhci, ehci, uhci) didn't cope with the
surprise removal of the controller. Before removing the card, try
detaching the driver(s) with "devctl detach xhciN". There might be
more
than one device. Use "pciconf -lc" to find the HotPlug-capable pcib
devices (bridges). Use devinfo to find which one is your ExpressCard
slot and find all the devices attached to it. Then use devctl to detach
the devices. There could be a tree of devices; in that case, you can
usually start at the level immediately under pcibN; you don't need to
detach every device from the bottom up. Once all the devices are
detached, you should be able to remove the card safely.
Eric