Xen users - Sep 2012 - issue using SRIOV "Unable to start - perhaps the PF driver is not up yet", while PF driver is actually up

Hi,

I am trying to use SRIOV, usnig xen 4.0 on debian.

On my dom0, xm pci-list-assignable-devices show that
0000:0f:10.0
0000:0f:10.2
0000:0f:10.4
0000:0f:10.6
0000:0f:11.2
0000:0f:11.4
0000:0f:11.6

these virtual interfaces correspond to eth2,

i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0

on my dom0 machine, i can ping other machine using eth2, (implying PF on
eth2 is active)

on my domU machine, when i attach the virtual function, i get the following
messages

[ 2282.688356] ixgbevf 0000:00:00.0: Xen PCI mapped GSI0 to IRQ28
[ 2282.688470] ixgbevf 0000:00:00.0: setting latency timer to 64
[ 2282.690187] ixgbevf 0000:00:00.0: PF still in reset state, assigning new
address

while PF on eth2 is  there and active, since i can ping other machine.

Now, when I try to bring up the VF interface on domU,
 i get the following error

 2476.295582] Unable to start - perhaps the PF Driver isn''t up yet
SIOCSIFFLAGS: Network is down
[ 2476.296917] Unable to start - perhaps the PF Driver isn''t up yet
SIOCSIFFLAGS: Network is down

and on dmesg on domU,
[ 2476.295582] Unable to start - perhaps the PF Driver isn''t up yet
[ 2476.296917] Unable to start - perhaps the PF Driver isn''t up yet


Any thoughts on what could be wrong? I have been struggling with this for
quite some time
and would really appreciate your thoughts on it.

Thanks,
Ashok


_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

Hi,

I am trying to use SRIOV, usnig xen 4.0 on debian.

On my dom0, xm pci-list-assignable-devices show that
0000:0f:10.0
0000:0f:10.2
0000:0f:10.4
0000:0f:10.6
0000:0f:11.2
0000:0f:11.4
0000:0f:11.6

these virtual interfaces correspond to eth2,

i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0

on my dom0 machine, i can ping other machine using eth2, (implying PF on
eth2 is active)

on my domU machine, when i attach the virtual function, i get the following
messages

[ 2282.688356] ixgbevf 0000:00:00.0: Xen PCI mapped GSI0 to IRQ28
[ 2282.688470] ixgbevf 0000:00:00.0: setting latency timer to 64
[ 2282.690187] ixgbevf 0000:00:00.0: PF still in reset state, assigning new
address

while PF on eth2 is  there and active, since i can ping other machine.

Now, when I try to bring up the VF interface on domU,
 i get the following error

 2476.295582] Unable to start - perhaps the PF Driver isn''t up yet
SIOCSIFFLAGS: Network is down
[ 2476.296917] Unable to start - perhaps the PF Driver isn''t up yet
SIOCSIFFLAGS: Network is down

and on dmesg on domU,
[ 2476.295582] Unable to start - perhaps the PF Driver isn''t up yet
[ 2476.296917] Unable to start - perhaps the PF Driver isn''t up yet


Any thoughts on what could be wrong? I have been struggling with this for
quite some time
and would really appreciate your thoughts on it.

Thanks,
Ashok


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On 10/09/12 11:24, Ashok Anand wrote:
> Hi, 
>
> I am trying to use SRIOV, usnig xen 4.0 on debian. 
>
> On my dom0, xm pci-list-assignable-devices show that
> 0000:0f:10.0
> 0000:0f:10.2
> 0000:0f:10.4
> 0000:0f:10.6
> 0000:0f:11.2
> 0000:0f:11.4
> 0000:0f:11.6
>
> these virtual interfaces correspond to eth2,
Are you sure?  0f:10.0 is probably the physical function which you must
keep in dom0.

~Andrew
>
> i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0
>
> on my dom0 machine, i can ping other machine using eth2, (implying PF
> on eth2 is active)
>
> on my domU machine, when i attach the virtual function, i get the
> following messages
>
> [ 2282.688356 <tel:%5B%202282.688356>] ixgbevf 0000:00:00.0: Xen PCI
> mapped GSI0 to IRQ28
> [ 2282.688470 <tel:%5B%202282.688470>] ixgbevf 0000:00:00.0: setting
> latency timer to 64
> [ 2282.690187 <tel:%5B%202282.690187>] ixgbevf 0000:00:00.0: PF still
> in reset state, assigning new address
>
> while PF on eth2 is  there and active, since i can ping other machine.
>
> Now, when I try to bring up the VF interface on domU, 
>  i get the following error
>
>  2476.295582] Unable to start - perhaps the PF Driver isn''t up yet
> SIOCSIFFLAGS: Network is down
> [ 2476.296917] Unable to start - perhaps the PF Driver isn''t up
yet
> SIOCSIFFLAGS: Network is down
>
> and on dmesg on domU, 
> [ 2476.295582] Unable to start - perhaps the PF Driver isn''t up
yet
> [ 2476.296917] Unable to start - perhaps the PF Driver isn''t up
yet
>
>
> Any thoughts on what could be wrong? I have been struggling with this
> for quite some time
> and would really appreciate your thoughts on it.
>
> Thanks, 
> Ashok
>
>
>
>
>
>
-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On Mon, Sep 10, 2012 at 03:54:34PM +0530, Ashok Anand
wrote:
>    Hi,
>    I am trying to use SRIOV, usnig xen 4.0 on debian.
>    On my dom0, xm pci-list-assignable-devices show that
>    0000:0f:10.0
>    0000:0f:10.2
>    0000:0f:10.4
>    0000:0f:10.6
>    0000:0f:11.2
>    0000:0f:11.4
>    0000:0f:11.6
>    these virtual interfaces correspond to eth2,
>    i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0
>    on my dom0 machine, i can ping other machine using eth2, (implying PF on
>    eth2 is active)
>    on my domU machine, when i attach the virtual function, i get the
>    following messages
>    [1][ 2282.688356] ixgbevf 0000:00:00.0: Xen PCI mapped GSI0 to IRQ28
>    [2][ 2282.688470] ixgbevf 0000:00:00.0: setting latency timer to 64
>    [3][ 2282.690187] ixgbevf 0000:00:00.0: PF still in reset state,
assigning
>    new address
>    while PF on eth2 is  there and active, since i can ping other machine.
>    Now, when I try to bring up the VF interface on domU,
>     i get the following error
>     2476.295582] Unable to start - perhaps the PF Driver isn''t up
yet
>    SIOCSIFFLAGS: Network is down
>    [ 2476.296917] Unable to start - perhaps the PF Driver isn''t up
yet
>    SIOCSIFFLAGS: Network is down
>    and on dmesg on domU,
>    [ 2476.295582] Unable to start - perhaps the PF Driver isn''t up
yet
>    [ 2476.296917] Unable to start - perhaps the PF Driver isn''t up
yet
>    Any thoughts on what could be wrong? I have been struggling with this
for
>    quite some time
>    and would really appreciate your thoughts on it.
>
Did you do "ifconfig ethX up" in dom0? 

-- Pasi

Are you sure?  0f:10.0 is probably the physical function which you
must
> keep in dom0.
>
> ~Andrew
>
I find from xm pci-list-assignable-devices  that 0f:10.0 is available - it
is also hidden from dom0 (using pciback.hide in kernel arguments)

if that is not the case, which physical function i can assign to domU?
i tried with 0f:11.6, and others too, but i got similar messages.

>
>
>  i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu 0f:10.0
>
>  on my dom0 machine, i can ping other machine using eth2, (implying PF on
> eth2 is active)
>
>  on my domU machine, when i attach the virtual function, i get the
> following messages
>
>  [ 2282.688356] ixgbevf 0000:00:00.0: Xen PCI mapped GSI0 to IRQ28
> [ 2282.688470] ixgbevf 0000:00:00.0: setting latency timer to 64
> [ 2282.690187] ixgbevf 0000:00:00.0: PF still in reset state, assigning
> new address
>
>  while PF on eth2 is  there and active, since i can ping other machine.
>
>  Now, when I try to bring up the VF interface on domU,
>  i get the following error
>
>   2476.295582] Unable to start - perhaps the PF Driver isn''t up
yet
> SIOCSIFFLAGS: Network is down
> [ 2476.296917] Unable to start - perhaps the PF Driver isn''t up
yet
>  SIOCSIFFLAGS: Network is down
>
>  and on dmesg on domU,
>  [ 2476.295582] Unable to start - perhaps the PF Driver isn''t up
yet
> [ 2476.296917] Unable to start - perhaps the PF Driver isn''t up
yet
>
>
>  Any thoughts on what could be wrong? I have been struggling with this
> for quite some time
> and would really appreciate your thoughts on it.
>
>  Thanks,
>  Ashok
>
>
>
>
>
>
>
> --
> Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
> T: +44 (0)1223 225 900, http://www.citrix.com
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On 10/09/12 11:34, Ashok Anand wrote:
>
>
>     Are you sure?  0f:10.0 is probably the physical function which you
>     must keep in dom0.
>
>     ~Andrew
>
>
> I find from xm pci-list-assignable-devices  that 0f:10.0 is available
> - it is also hidden from dom0 (using pciback.hide in kernel arguments)
>
> if that is not the case, which physical function i can assign to domU?
> i tried with 0f:11.6, and others too, but i got similar messages.
What does lspci -vv for the physical function say?

As for the physical function: it is unsafe to pass physical functions to
a non-trusted guest, as the physical function has complete control over
the all the virtual functions, even if they are passed through to other
guests.  For that reason, the physical function should remain in dom0
(or a device driver domain if you are going for disaggregation).
>
>
>
>>
>>     i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu
>>     0f:10.0
>>
>>     on my dom0 machine, i can ping other machine using eth2,
>>     (implying PF on eth2 is active)
>>
>>     on my domU machine, when i attach the virtual function, i get the
>>     following messages
>>
>>     [ 2282.688356 <tel:%5B%202282.688356>] ixgbevf 0000:00:00.0:
Xen
>>     PCI mapped GSI0 to IRQ28
>>     [ 2282.688470 <tel:%5B%202282.688470>] ixgbevf 0000:00:00.0:
>>     setting latency timer to 64
>>     [ 2282.690187 <tel:%5B%202282.690187>] ixgbevf 0000:00:00.0:
PF
>>     still in reset state, assigning new address
>>
>>     while PF on eth2 is  there and active, since i can ping other
>>     machine.
>>
>>     Now, when I try to bring up the VF interface on domU, 
>>      i get the following error
>>
>>      2476.295582] Unable to start - perhaps the PF Driver
isn''t up yet
>>     SIOCSIFFLAGS: Network is down
>>     [ 2476.296917] Unable to start - perhaps the PF Driver
isn''t up yet
>>     SIOCSIFFLAGS: Network is down
>>
>>     and on dmesg on domU, 
>>     [ 2476.295582] Unable to start - perhaps the PF Driver
isn''t up yet
>>     [ 2476.296917] Unable to start - perhaps the PF Driver
isn''t up yet
>>
>>
>>     Any thoughts on what could be wrong? I have been struggling with
>>     this for quite some time
>>     and would really appreciate your thoughts on it.
>>
>>     Thanks, 
>>     Ashok
>>
>>
>>
>>
>>
>>
>
>     -- 
>     Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>     T: +44 (0)1223 225 900 <tel:%2B44%20%280%291223%20225%20900>,
http://www.citrix.com
>
>
>     _______________________________________________
>     Xen-devel mailing list
>     Xen-devel@lists.xen.org <mailto:Xen-devel@lists.xen.org>
>     http://lists.xen.org/xen-devel
>
>
-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

I am passing virtual functions. please see below ,based on the message from
lspci.
> What does lspci -vv for the physical function say?
>
> As for the physical function: it is unsafe to pass physical functions to a
> non-trusted guest, as the physical function has complete control over the
> all the virtual functions, even if they are passed through to other
> guests.  For that reason, the physical function should remain in dom0 (or a
> device driver domain if you are going for disaggregation).
>
here is the message lspci -vv | grep Eth gives me,

0f:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
Network Connection (rev 01)
Subsystem: Intel Corporation Ethernet Server Adapter X520-2
0f:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
Network Connection (rev 01)
Subsystem: Intel Corporation Ethernet Server Adapter X520-2
0f:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:10.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:10.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:10.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:11.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:11.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:11.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)
0f:11.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller
Virtual Function (rev 01)

Here, 0f:00.0 is the physical function, while 0f:10.0 and others are
virtual functions -- so I am attaching virtual functions.



>
>
>
>
>>
>>
>>  i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu
0f:10.0
>>
>>  on my dom0 machine, i can ping other machine using eth2, (implying PF
>> on eth2 is active)
>>
>>  on my domU machine, when i attach the virtual function, i get the
>> following messages
>>
>>  [ 2282.688356 <%5B%202282.688356>] ixgbevf 0000:00:00.0: Xen PCI
mapped
>> GSI0 to IRQ28
>> [ 2282.688470 <%5B%202282.688470>] ixgbevf 0000:00:00.0: setting
latency
>> timer to 64
>> [ 2282.690187 <%5B%202282.690187>] ixgbevf 0000:00:00.0: PF still
in
>> reset state, assigning new address
>>
>>  while PF on eth2 is  there and active, since i can ping other machine.
>>
>>  Now, when I try to bring up the VF interface on domU,
>>  i get the following error
>>
>>   2476.295582] Unable to start - perhaps the PF Driver isn''t
up yet
>> SIOCSIFFLAGS: Network is down
>> [ 2476.296917] Unable to start - perhaps the PF Driver isn''t
up yet
>>  SIOCSIFFLAGS: Network is down
>>
>>  and on dmesg on domU,
>>  [ 2476.295582] Unable to start - perhaps the PF Driver isn''t
up yet
>> [ 2476.296917] Unable to start - perhaps the PF Driver isn''t
up yet
>>
>>
>>  Any thoughts on what could be wrong? I have been struggling with this
>> for quite some time
>> and would really appreciate your thoughts on it.
>>
>>  Thanks,
>>  Ashok
>>
>>
>>
>>
>>
>>
>>
>>  --
>> Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>> T: +44 (0)1223 225 900, http://www.citrix.com
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> http://lists.xen.org/xen-devel
>>
>>
>
> --
> Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
> T: +44 (0)1223 225 900, http://www.citrix.com
>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Any more thoughts on it? I will really appreciate it.

regards,
Ashok

On Mon, Sep 10, 2012 at 4:16 PM, Ashok Anand <ashok.anand@gmail.com>
wrote:
>
> I am passing virtual functions. please see below ,based on the message
> from lspci.
>
>> What does lspci -vv for the physical function say?
>>
>> As for the physical function: it is unsafe to pass physical functions
to
>> a non-trusted guest, as the physical function has complete control over
the
>> all the virtual functions, even if they are passed through to other
>> guests.  For that reason, the physical function should remain in dom0
(or a
>> device driver domain if you are going for disaggregation).
>>
>
> here is the message lspci -vv | grep Eth gives me,
>
> 0f:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
> Network Connection (rev 01)
>  Subsystem: Intel Corporation Ethernet Server Adapter X520-2
> 0f:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
> Network Connection (rev 01)
>  Subsystem: Intel Corporation Ethernet Server Adapter X520-2
> 0f:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:10.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:10.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:10.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:11.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:11.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:11.4 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
> 0f:11.6 Ethernet controller: Intel Corporation 82599 Ethernet Controller
> Virtual Function (rev 01)
>
> Here, 0f:00.0 is the physical function, while 0f:10.0 and others are
> virtual functions -- so I am attaching virtual functions.
>
>
>
>
>>
>>
>>
>>
>>>
>>>
>>>  i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach ubuntu
0f:10.0
>>>
>>>  on my dom0 machine, i can ping other machine using eth2, (implying
PF
>>> on eth2 is active)
>>>
>>>  on my domU machine, when i attach the virtual function, i get the
>>> following messages
>>>
>>>  [ 2282.688356 <%5B%202282.688356>] ixgbevf 0000:00:00.0: Xen
PCI
>>> mapped GSI0 to IRQ28
>>> [ 2282.688470 <%5B%202282.688470>] ixgbevf 0000:00:00.0:
setting
>>> latency timer to 64
>>> [ 2282.690187 <%5B%202282.690187>] ixgbevf 0000:00:00.0: PF
still in
>>> reset state, assigning new address
>>>
>>>  while PF on eth2 is  there and active, since i can ping other
machine.
>>>
>>>  Now, when I try to bring up the VF interface on domU,
>>>  i get the following error
>>>
>>>   2476.295582] Unable to start - perhaps the PF Driver
isn''t up yet
>>> SIOCSIFFLAGS: Network is down
>>> [ 2476.296917] Unable to start - perhaps the PF Driver
isn''t up yet
>>>  SIOCSIFFLAGS: Network is down
>>>
>>>  and on dmesg on domU,
>>>  [ 2476.295582] Unable to start - perhaps the PF Driver
isn''t up yet
>>> [ 2476.296917] Unable to start - perhaps the PF Driver
isn''t up yet
>>>
>>>
>>>  Any thoughts on what could be wrong? I have been struggling with
this
>>> for quite some time
>>> and would really appreciate your thoughts on it.
>>>
>>>  Thanks,
>>>  Ashok
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>  --
>>> Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>>> T: +44 (0)1223 225 900, http://www.citrix.com
>>>
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xen.org
>>> http://lists.xen.org/xen-devel
>>>
>>>
>>
>> --
>> Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>> T: +44 (0)1223 225 900, http://www.citrix.com
>>
>>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On 10/09/12 19:32, Ashok Anand wrote:
> Any more thoughts on it? I will really appreciate it.
Sorry - I am debugging a memory corruption issue, so not paying too
close attention to emails.

Have you ensured that the pif is up in dom0 before trying to use a vif?

~Andrew
>
> regards,
> Ashok
>
> On Mon, Sep 10, 2012 at 4:16 PM, Ashok Anand <ashok.anand@gmail.com
> <mailto:ashok.anand@gmail.com>> wrote:
>
>
>     I am passing virtual functions. please see below ,based on the
>     message from lspci. 
>
>         What does lspci -vv for the physical function say?
>
>         As for the physical function: it is unsafe to pass physical
>         functions to a non-trusted guest, as the physical function has
>         complete control over the all the virtual functions, even if
>         they are passed through to other guests.  For that reason, the
>         physical function should remain in dom0 (or a device driver
>         domain if you are going for disaggregation).
>
>
>     here is the message lspci -vv | grep Eth gives me, 
>
>     0f:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
>     SFI/SFP+ Network Connection (rev 01)
>     Subsystem: Intel Corporation Ethernet Server Adapter X520-2
>     0f:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
>     SFI/SFP+ Network Connection (rev 01)
>     Subsystem: Intel Corporation Ethernet Server Adapter X520-2
>     0f:10.0 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:10.2 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:10.4 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:10.6 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:11.0 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:11.2 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:11.4 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>     0f:11.6 Ethernet controller: Intel Corporation 82599 Ethernet
>     Controller Virtual Function (rev 01)
>
>     Here, 0f:00.0 is the physical function, while 0f:10.0 and others
>     are virtual functions -- so I am attaching virtual functions. 
>
>
>      
>
>
>
>>
>>
>>
>>>
>>>             i attach 0f:10.0 to a domU ubuntu machine, xm
pci-attach
>>>             ubuntu 0f:10.0
>>>
>>>             on my dom0 machine, i can ping other machine using
eth2,
>>>             (implying PF on eth2 is active)
>>>
>>>             on my domU machine, when i attach the virtual function,
>>>             i get the following messages
>>>
>>>             [ 2282.688356 <tel:%5B%202282.688356>] ixgbevf
>>>             0000:00:00.0: Xen PCI mapped GSI0 to IRQ28
>>>             [ 2282.688470 <tel:%5B%202282.688470>] ixgbevf
>>>             0000:00:00.0: setting latency timer to 64
>>>             [ 2282.690187 <tel:%5B%202282.690187>] ixgbevf
>>>             0000:00:00.0: PF still in reset state, assigning new
address
>>>
>>>             while PF on eth2 is  there and active, since i can ping
>>>             other machine.
>>>
>>>             Now, when I try to bring up the VF interface on domU, 
>>>              i get the following error
>>>
>>>              2476.295582] Unable to start - perhaps the PF Driver
>>>             isn''t up yet
>>>             SIOCSIFFLAGS: Network is down
>>>             [ 2476.296917] Unable to start - perhaps the PF Driver
>>>             isn''t up yet
>>>             SIOCSIFFLAGS: Network is down
>>>
>>>             and on dmesg on domU, 
>>>             [ 2476.295582] Unable to start - perhaps the PF Driver
>>>             isn''t up yet
>>>             [ 2476.296917] Unable to start - perhaps the PF Driver
>>>             isn''t up yet
>>>
>>>
>>>             Any thoughts on what could be wrong? I have been
>>>             struggling with this for quite some time
>>>             and would really appreciate your thoughts on it.
>>>
>>>             Thanks, 
>>>             Ashok
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>             -- 
>>             Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>>             T: +44 (0)1223 225 900
<tel:%2B44%20%280%291223%20225%20900>, http://www.citrix.com
>>
>>
>>             _______________________________________________
>>             Xen-devel mailing list
>>             Xen-devel@lists.xen.org
<mailto:Xen-devel@lists.xen.org>
>>             http://lists.xen.org/xen-devel
>>
>>
>
>         -- 
>         Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>         T: +44 (0)1223 225 900 <tel:%2B44%20%280%291223%20225%20900>,
http://www.citrix.com
>
>
>
-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On Tue, Sep 11, 2012 at 10:32:25AM +0100, Andrew Cooper
wrote:
>    On 10/09/12 19:32, Ashok Anand wrote:
> 
>      Any more thoughts on it? I will really appreciate it.
> 
>    Sorry - I am debugging a memory corruption issue, so not paying too
close
>    attention to emails.
> 
>    Have you ensured that the pif is up in dom0 before trying to use a vif?
> 
Yes, "ifconfig ethX up" for the PF is required in dom0 before the VF
works in the VM.

-- Pasi
>    ~Andrew
> 
>      regards,
>      Ashok
> 
>      On Mon, Sep 10, 2012 at 4:16 PM, Ashok Anand
<[1]ashok.anand@gmail.com>
>      wrote:
> 
>        I am passing virtual functions. please see below ,based on the
message
>        from lspci.
> 
>          What does lspci -vv for the physical function say?
> 
>          As for the physical function: it is unsafe to pass physical
>          functions to a non-trusted guest, as the physical function has
>          complete control over the all the virtual functions, even if they
>          are passed through to other guests.  For that reason, the physical
>          function should remain in dom0 (or a device driver domain if you
are
>          going for disaggregation).
> 
>        here is the message lspci -vv | grep Eth gives me,
>        0f:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
>        SFI/SFP+ Network Connection (rev 01)
>        Subsystem: Intel Corporation Ethernet Server Adapter X520-2
>        0f:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
>        SFI/SFP+ Network Connection (rev 01)
>        Subsystem: Intel Corporation Ethernet Server Adapter X520-2
>        0f:10.0 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:10.2 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:10.4 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:10.6 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:11.0 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:11.2 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:11.4 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        0f:11.6 Ethernet controller: Intel Corporation 82599 Ethernet
>        Controller Virtual Function (rev 01)
>        Here, 0f:00.0 is the physical function, while 0f:10.0 and others are
>        virtual functions -- so I am attaching virtual functions.
> 
> 
>                i attach 0f:10.0 to a domU ubuntu machine, xm pci-attach
>                ubuntu 0f:10.0
>                on my dom0 machine, i can ping other machine using eth2,
>                (implying PF on eth2 is active)
>                on my domU machine, when i attach the virtual function, i
get
>                the following messages
>                [2][ 2282.688356] ixgbevf 0000:00:00.0: Xen PCI mapped GSI0
to
>                IRQ28
>                [3][ 2282.688470] ixgbevf 0000:00:00.0: setting latency
timer
>                to 64
>                [4][ 2282.690187] ixgbevf 0000:00:00.0: PF still in reset
>                state, assigning new address
>                while PF on eth2 is  there and active, since i can ping
other
>                machine.
>                Now, when I try to bring up the VF interface on domU,
>                 i get the following error
>                 2476.295582] Unable to start - perhaps the PF Driver
isn''t up
>                yet
>                SIOCSIFFLAGS: Network is down
>                [ 2476.296917] Unable to start - perhaps the PF Driver
isn''t
>                up yet
>                SIOCSIFFLAGS: Network is down
>                and on dmesg on domU,
>                [ 2476.295582] Unable to start - perhaps the PF Driver
isn''t
>                up yet
>                [ 2476.296917] Unable to start - perhaps the PF Driver
isn''t
>                up yet
>                Any thoughts on what could be wrong? I have been struggling
>                with this for quite some time
>                and would really appreciate your thoughts on it.
>                Thanks,
>                Ashok
> 
>  --
>  Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>  T: [5]+44 (0)1223 225 900, [6]http://www.citrix.com
> 
>              _______________________________________________
>              Xen-devel mailing list
>              [7]Xen-devel@lists.xen.org
>              [8]http://lists.xen.org/xen-devel
> 
>  --
>  Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>  T: [9]+44 (0)1223 225 900, [10]http://www.citrix.com
> 
>  --
>  Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
>  T: +44 (0)1223 225 900, [11]http://www.citrix.com
> 
> References
> 
>    Visible links
>    1. mailto:ashok.anand@gmail.com
>    2. file:///tmp/tel:%5B%202282.688356
>    3. file:///tmp/tel:%5B%202282.688470
>    4. file:///tmp/tel:%5B%202282.690187
>    5. file:///tmp/tel:%2B44%20%280%291223%20225%20900
>    6. http://www.citrix.com/
>    7. mailto:Xen-devel@lists.xen.org
>    8. http://lists.xen.org/xen-devel
>    9. file:///tmp/tel:%2B44%20%280%291223%20225%20900
>   10. http://www.citrix.com/
>   11. http://www.citrix.com/
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel