On Mon, Jun 25, 2007 at 03:40:20PM +0200, Lukas Hejtmanek
wrote:> I''m trying to get working infiniband device in Xen. I have Xen v3
with 2.6.18
> kernel. I export PCI device from Dom0 into DomU. I''ve enabled
permisive device
> in Dom0.
>
> Do you have any advices what should I try? The device running in Dom0 works
as
> expected. The problem is in DomU with exported PCI device.
>
> It seems that the iomapped adrea returns all ones instead of reasonable
values.
I have found that this code is problematic in DomU. In Dom0, it works ok.
After the reset, the device respondes so it continues on the good label, but it
seems that PCI config is not restored as it should be.
hca_header = kmalloc(256, GFP_KERNEL);
if (!hca_header) {
err = -ENOMEM;
mthca_err(mdev, "Couldn''t allocate memory to save HCA
"
"PCI header, aborting.\n");
goto out;
}
for (i = 0; i < 64; ++i) {
if (i == 22 || i == 23)
continue;
if (pci_read_config_dword(mdev->pdev, i * 4, hca_header + i)) {
err = -ENODEV;
mthca_err(mdev, "Couldn''t save HCA "
"PCI header, aborting.\n");
goto out;
}
}
/* actually hit reset */
{
void __iomem *reset = ioremap(pci_resource_start(mdev->pdev, 0) +
MTHCA_RESET_OFFSET, 4);
if (!reset) {
err = -ENOMEM;
mthca_err(mdev, "Couldn''t map HCA reset register,
"
"aborting.\n");
goto out;
}
writel(MTHCA_RESET_VALUE, reset);
iounmap(reset);
}
/* Docs say to wait one second before accessing device */
msleep(1000);
/* Now wait for PCI device to start responding again */
{
u32 v;
int c = 0;
for (c = 0; c < 100; ++c) {
if (pci_read_config_dword(bridge ? bridge : mdev->pdev, 0, &v))
{
err = -ENODEV;
mthca_err(mdev, "Couldn''t access HCA after
reset, "
"aborting.\n");
goto out;
}
if (v != 0xffffffff)
goto good;
msleep(100);
}
err = -ENODEV;
mthca_err(mdev, "PCI device did not come back after reset, "
"aborting.\n");
}
good:
devctl = hca_header[(hca_pcie_cap + PCI_EXP_DEVCTL) / 4];
if (pci_write_config_word(mdev->pdev, hca_pcie_cap + PCI_EXP_DEVCTL, devctl))
{
err = -ENODEV;
mthca_err(mdev, "Couldn''t restore HCA PCI Express "
"Device Control register, aborting.\n");
goto out;
}
linkctl = hca_header[(hca_pcie_cap + PCI_EXP_LNKCTL) / 4];
if (pci_write_config_word(mdev->pdev, hca_pcie_cap + PCI_EXP_LNKCTL,
linkctl)) {
err = -ENODEV;
mthca_err(mdev, "Couldn''t restore HCA PCI Express "
"Link control register,
aborting.\n");
goto out;
}
for (i = 0; i < 16; ++i) {
if (i * 4 == PCI_COMMAND)
continue;
if (pci_write_config_dword(mdev->pdev, i * 4, hca_header[i])) {
err = -ENODEV;
mthca_err(mdev, "Couldn''t restore HCA reg %x,
"
"aborting.\n", i);
goto out;
}
}
if (pci_write_config_dword(mdev->pdev, PCI_COMMAND, hca_header[PCI_COMMAND /
4])) {
err = -ENODEV;
mthca_err(mdev, "Couldn''t restore HCA COMMAND, "
"aborting.\n");
goto out;
}
--
Lukáš Hejtmánek
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel