Pkg xen devel - Aug 2023 - Bug#1042842: network interface names wrong in domU (>10 interfaces)

Package: xen-utils-4.17
Version: 4.17.1+2-gb773c48e36-1
Severity: important

Dear Maintainers,

On one of our domUs we discovered that the network interface names were 
wrongly assigned since recreating the domU after an upgrade to bookworm.

If over 10 network interfaces are configured the mapping (dom0) vifX.10 
<-> eth10 (domU) does not apply anymore.
Instead the interfaces on dom0 are sorted primarily by the leftmost 
digit. so for 11 interfaces we will end up with:
vifX.0 <> eth0
vifX.1 <> eth1
vifX.10 <> eth2
vifX.2 <> eth3
vifX.3 <> eth4
....

This was observed with linux-kernel versions 5.10.179-3 and 6.1.38-2 
(all combinations of domU and dom0) and xen 4.17.1+2-gb773c48e36-1.
You can find a config snippet and "xl network-list" + "ip a"
command
output demonstrating the issue below.
Booting the host with Xen 4.14.5+94-ge49571868d-1 restored the expected 
behaviour.

Looking for relevant changes i found commit fce6999 [0] which changes 
the way libxl__device_list works, but i'm not sure that's the cause of 
this issue.

Thanks for your help,
Valentin

[0] http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=fce6999

Sample vif configuration (ascending MACs):
vif         = [
                 'mac=00:16:3e:fd:83:2f,bridge=lanbr',
                 'mac=00:16:3e:fd:83:30,bridge=lanbr',
                 'mac=00:16:3e:fd:83:31,bridge=lanbr',
                 'mac=00:16:3e:fd:83:32,bridge=lanbr',
                 'mac=00:16:3e:fd:83:33,bridge=lanbr',
                 'mac=00:16:3e:fd:83:34,bridge=lanbr',
                 'mac=00:16:3e:fd:83:35,bridge=lanbr',
                 'mac=00:16:3e:fd:83:36,bridge=lanbr',
                 'mac=00:16:3e:fd:83:37,bridge=lanbr',
                 'mac=00:16:3e:fd:83:38,bridge=lanbr',
                 'mac=00:16:3e:fd:83:39,bridge=lanbr',
               ]

dom0# xl network-list 3
Idx BE Mac Addr.         handle state evt-ch   tx-/rx-ring-ref BE-path 

0   0  00:16:3e:fd:83:2f     0     4     -1    -1/-1 
/local/domain/0/backend/vif/3/0
1   0  00:16:3e:fd:83:30     1     4     -1    -1/-1 
/local/domain/0/backend/vif/3/1
10  0  00:16:3e:fd:83:39    10     4     -1    -1/-1 
/local/domain/0/backend/vif/3/10
2   0  00:16:3e:fd:83:31     2     4     -1    -1/-1 
/local/domain/0/backend/vif/3/2
3   0  00:16:3e:fd:83:32     3     4     -1    -1/-1 
/local/domain/0/backend/vif/3/3
4   0  00:16:3e:fd:83:33     4     4     -1    -1/-1 
/local/domain/0/backend/vif/3/4
5   0  00:16:3e:fd:83:34     5     4     -1    -1/-1 
/local/domain/0/backend/vif/3/5
6   0  00:16:3e:fd:83:35     6     4     -1    -1/-1 
/local/domain/0/backend/vif/3/6
7   0  00:16:3e:fd:83:36     7     4     -1    -1/-1 
/local/domain/0/backend/vif/3/7
8   0  00:16:3e:fd:83:37     8     4     -1    -1/-1 
/local/domain/0/backend/vif/3/8
9   0  00:16:3e:fd:83:38     9     4     -1    -1/-1 
/local/domain/0/backend/vif/3/9

domU# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
group default qlen 1000
     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
     inet 127.0.0.1/8 scope host lo
        valid_lft forever preferred_lft forever
     inet6 ::1/128 scope host
        valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP 
group default qlen 1000
     link/ether 00:16:3e:fd:83:2f brd ff:ff:ff:ff:ff:ff
     inet X.X.X.X/16 brd X.X.X.X scope global eth0
        valid_lft forever preferred_lft forever
     inet6 fe80::216:3eff:fefd:832f/64 scope link
        valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group 
default qlen 1000
     link/ether 00:16:3e:fd:83:30 brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group 
default qlen 1000
     link/ether 00:16:3e:fd:83:39 brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group 
default qlen 1000
     link/ether 00:16:3e:fd:83:31 brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group 
default qlen 1000
     link/ether 00:16:3e:fd:83:32 brd ff:ff:ff:ff:ff:ff
7: eth5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group 
default qlen 1000
     link/ether 00:16:3e:fd:83:33 brd ff:ff:ff:ff:ff:ff
8: eth6: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group 
default qlen 1000
     link/ether 00:16:3e:fd:83:34 brd ff:ff:ff:ff:ff:ff
9: eth7: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group 
default qlen 1000
     link/ether 00:16:3e:fd:83:35 brd ff:ff:ff:ff:ff:ff
10: eth8: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group 
default qlen 1000
     link/ether 00:16:3e:fd:83:36 brd ff:ff:ff:ff:ff:ff
11: eth9: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group 
default qlen 1000
     link/ether 00:16:3e:fd:83:37 brd ff:ff:ff:ff:ff:ff
12: eth10: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group 
default qlen 1000
     link/ether 00:16:3e:fd:83:38 brd ff:ff:ff:ff:ff:ff

Hello,

the bug has been mentionned on #xen-devel, will keep you posted.

Meanwhile, you may try to force the domU vif names with a letter, like :

vif         = [
                 'mac=00:16:3e:fd:83:2f,bridge=lanbr,vifname=domu-a',
                 'mac=00:16:3e:fd:83:30,bridge=lanbr,vifname=domu-b',
                 'mac=00:16:3e:fd:83:31,bridge=lanbr,vifname=domu-c',
                 ...
               ]

Note it's just a workaround, and I've not tested it.
I only guess letters would be sorted correctly.
If you test this, can you report back please ?


--
zithro / Cyril

Hi,
> the bug has been mentionned on #xen-devel, will keep you posted.
Thanks. I wasn't able to find such a report, could you link the archive 
or post the threads subject so i can find it?
> Meanwhile, you may try to force the domU vif names with a letter
The sorting with letters doesn't work out as renaming the interface is a 
secondary step.

...
[53408.899507] vif vif-5-0 sort-a: renamed from vif5.0
...

Our current workaround is to edit the interface names in the domUs 
config to match the wrong sorting. And be extra careful that the domUs 
MACs match the ones we expect on that network.

Valentin

On 02 Aug 2023 10:22, Valentin Kleibel wrote:
> Hi,
> 
>> the bug has been mentionned on #xen-devel, will keep you posted.
> 
> Thanks. I wasn't able to find such a report, could you link the archive
> or post the threads subject so i can find it?
#xen-devel is the IRC Xen channel. I just pinged them, I'll wait.
Depending on their answers, I'll post on the xen-devel mailing list.
>> Meanwhile, you may try to force the domU vif names with a letter
> 
> The sorting with letters doesn't work out as renaming the interface is
a
> secondary step.
> ...
> [53408.899507] vif vif-5-0 sort-a: renamed from vif5.0
Yeah I just tried with vifnames, no more luck, sorry.
Running "xenstore-ls /local/domain/DOMU_ID/device/vif" shows that
vif10+
are sorted before vif10- (ie. vif1, vif10, vif11, vif2, ...).
> Our current workaround is to edit the interface names in the domUs 
> config to match the wrong sorting. And be extra careful that the domUs 
> MACs match the ones we expect on that network.
Via udev (MAC matching) or /etc/network/interfaces ?
I ask because it may help others, while this gets resolved.


--
zithro / Cyril

> #xen-devel is the IRC Xen channel. I just pinged them, I'll wait.
> Depending on their answers, I'll post on the xen-devel mailing list.
thanks for the clarification, looking forward to an answer.
>> Our current workaround is to edit the interface names in the domUs 
>> config to match the wrong sorting. And be extra careful that the domUs 
>> MACs match the ones we expect on that network.
> 
> Via udev (MAC matching) or /etc/network/interfaces ?
> I ask because it may help others, while this gets resolved.
We just edited /etc/network/interfaces, as it only affects a few of our 
domUs.
i think udev rules matching the MAC would be a better solution. I just 
didn't take the time to write them and went for the quick and dirty 
solution.

Valentin

On 02 Aug 2023 18:09, Valentin Kleibel wrote:
>> #xen-devel is the IRC Xen channel. I just pinged them, I'll wait.
>> Depending on their answers, I'll post on the xen-devel mailing
list.
> 
> thanks for the clarification, looking forward to an answer.
I posted on xen-devel, you can follow from :
https://lists.xenproject.org/archives/html/xen-devel/2023-08/msg00244.html
(Unfortunately, the formatting is weird via html, split the IRC part on 
"- ").

Note that, at first sight, I was told this seems "not-a-Xen" bug (read
the IRC excerpts).
>>> Our current workaround is to edit the interface names in the domUs 
>>> config to match the wrong sorting. And be extra careful that the 
>>> domUs MACs match the ones we expect on that network.
>>
>> Via udev (MAC matching) or /etc/network/interfaces ?
>> I ask because it may help others, while this gets resolved.
> 
> We just edited /etc/network/interfaces, as it only affects a few of our 
> domUs.
> i think udev rules matching the MAC would be a better solution. I just 
> didn't take the time to write them and went for the quick and dirty 
> solution.
Till it works, "whatever the bottle, till we have the poison" ;)

This link may be useful: https://wiki.debian.org/NetworkInterfaceNames

> I posted on xen-devel, you can follow from :
> https://lists.xenproject.org/archives/html/xen-devel/2023-08/msg00244.html
> (Unfortunately, the formatting is weird via html, split the IRC part on 
> "- ").
Thank you for posting upstream.
> Note that, at first sight, I was told this seems "not-a-Xen" bug
(read
> the IRC excerpts).
All documentation i found found on the Xen wiki suggests that interfaces 
are connected vifX.Y <-> ethY. [0] [1]
The only other way i know of for identifying the interfaces are MAC 
Addresses which can be randomly assigned if you don't configure them.

I also checked which net_ids udev knows about and the only things that 
pop up are:
ID_NET_NAMING_SCHEME=v247
ID_NET_NAME_MAC=enx00163efd832b
ID_OUI_FROM_DATABASE=Xensource, Inc.

Either i am missing the way you're supposed to do this, or there is a 
bug somewhere in the toolchain.
Unfortunately i'm not able to pinpoint the source of the issue, any help 
would be appreciated.

Valentin

[0] 
https://wiki.xenproject.org/wiki/Xen_Networking#Paravirtualised_Network_Devices
[1] 
https://wiki.xenproject.org/wiki/Network_Throughput_and_Performance_Guide#Technical_Overview

On 08 Aug 2023 12:08, Valentin Kleibel wrote:
>> I posted on xen-devel, you can follow from :
>>
https://lists.xenproject.org/archives/html/xen-devel/2023-08/msg00244.html
>> (Unfortunately, the formatting is weird via html, split the IRC part 
>> on "- ").
> 
> Thank you for posting upstream.
No prob, although if that very answer does not answer your question, I 
guess you'd be better off replying on xen-devel ML (reply to my post or 
at least reference it).
> All documentation i found found on the Xen wiki suggests that interfaces 
> are connected vifX.Y <-> ethY. [0] [1]
> The only other way i know of for identifying the interfaces are MAC 
> Addresses which can be randomly assigned if you don't configure them.
On [0], you can read "In both cases the device naming is subject to the 
usual guest or backend domain facilities for renaming network devices".
It says "naming/renaming", but you can assume "detecting".
> I also checked which net_ids udev knows about and the only things that 
> pop up are:
> ID_NET_NAMING_SCHEME=v247
> ID_NET_NAME_MAC=enx00163efd832b
> ID_OUI_FROM_DATABASE=Xensource, Inc.
Is it from dom0 or domU ?
Are you using "net.ifnames=0" on the domU kernel command line ?
"v247" looks like systemd "predictive naming scheme" (eth
-> enX).
 From bookworm on, domUs vifs get named enXN (enX0, enX1, ...).
Read on :
https://www.debian.org/releases/stable/i386/release-notes/ch-information.en.html#xen-network
> Either i am missing the way you're supposed to do this, or there is a 
> bug somewhere in the toolchain.
> Unfortunately i'm not able to pinpoint the source of the issue, any
help
> would be appreciated.
I made some tests with a domU using many interfaces, like :

[...]
vif = [ 'bridge=xbr-tst,mac=00:16:3e:de:bd:00,type=vif,vifname=domu-a' ,
         'bridge=xbr-tst,mac=00:16:3e:de:bd:01,type=vif,vifname=domu-b'
,
         'bridge=xbr-tst,mac=00:16:3e:de:bd:02,type=vif,vifname=domu-c'
,
         'bridge=xbr-tst,mac=00:16:3e:de:bd:03,type=vif,vifname=domu-d'
,
         'bridge=xbr-tst,mac=00:16:3e:de:bd:04,type=vif,vifname=domu-e'
,
         'bridge=xbr-tst,mac=00:16:3e:de:bd:05,type=vif,vifname=domu-f'
,
         'bridge=xbr-tst,mac=00:16:3e:de:bd:06,type=vif,vifname=domu-g'
,
         'bridge=xbr-tst,mac=00:16:3e:de:bd:07,type=vif,vifname=domu-h'
,
         'bridge=xbr-tst,mac=00:16:3e:de:bd:08,type=vif,vifname=domu-i'
,
         'bridge=xbr-tst,mac=00:16:3e:de:bd:09,type=vif,vifname=domu-j'
,
         'bridge=xbr-tst,mac=00:16:3e:de:bd:10,type=vif,vifname=domu-k'
,
         ]
[...]

-----
This is dom0's corresponding dmesg:

[...]
xbr-tst: port 3(domu-b) entered blocking state
xbr-tst: port 3(domu-b) entered disabled state
device domu-b entered promiscuous mode
xbr-tst: port 4(domu-i) entered blocking state
xbr-tst: port 4(domu-i) entered disabled state
device domu-i entered promiscuous mode
[...]

Here you can see :
port 3 <-> domu-b
port 4 <-> domu-i

We learn here that dom0 did not detect vifs serially.

-----
In the domU, "ip link" shows :

[...]
eth0
     link/ether 00:16:3e:de:bd:00
     altname enX0
eth1
     link/ether 00:16:3e:de:bd:01
     altname enX1
eth2
     link/ether 00:16:3e:de:bd:10
     altname enX10
eth3
     link/ether 00:16:3e:de:bd:02
     altname enX2
[...]

See how ethN interfaces get messed up, like in your setup, but 
predictable names would work, as you can see in "altname enXN" :
eth1 (:01) -> enX1
eth2 (:10) -> enX10
eth3 (:02) -> enX2

So, my answer does not tell you if something changed in Xen itself, only 
in Debian.
But I guess it relates to what Xen devs told us : vifs detection order 
cannot be relied upon, that's why "predictable names" were
invented.
The vif detection part is related to the domains kernels, not Xen itself 
(at least that's what I understood).

Using eth0 nowadays is a bit like using /dev/sda for hard drives, it's 
considered legacy as it may create problems in some setups, like yours 
(ie. for disks, it's recommended to use UUIDs or /dev/disk/by-*).

I hope this answers your question.

> On [0], you can read "In both cases the device naming is subject to
the
> usual guest or backend domain facilities for renaming network
devices".
> It says "naming/renaming", but you can assume
"detecting".
> 
>> I also checked which net_ids udev knows about and the only things that 
>> pop up are:
>> ID_NET_NAMING_SCHEME=v247
>> ID_NET_NAME_MAC=enx00163efd832b
>> ID_OUI_FROM_DATABASE=Xensource, Inc.
> 
> Is it from dom0 or domU ?
> Are you using "net.ifnames=0" on the domU kernel command line ?
> "v247" looks like systemd "predictive naming scheme"
(eth -> enX).
>  From bookworm on, domUs vifs get named enXN (enX0, enX1, ...).
> Read on :
>
https://www.debian.org/releases/stable/i386/release-notes/ch-information.en.html#xen-network
This is from the domU, running bullseye with a bookworm dom0.
> See how ethN interfaces get messed up, like in your setup, but 
> predictable names would work, as you can see in "altname enXN" :
> eth1 (:01) -> enX1
> eth2 (:10) -> enX10
> eth3 (:02) -> enX2
I could not get our bullseye domU to show the "predictable names" even
though i tried installing the bullseye-backports kernel 6.1.
After you wrote this i installed udev 252.5 from backports and it now 
uses the correct enXn interface names, even with kernel 5.10.
> So, my answer does not tell you if something changed in Xen itself, only 
> in Debian.
> But I guess it relates to what Xen devs told us : vifs detection order 
> cannot be relied upon, that's why "predictable names" were
invented.
> The vif detection part is related to the domains kernels, not Xen itself 
> (at least that's what I understood).
> 
> Using eth0 nowadays is a bit like using /dev/sda for hard drives, it's 
> considered legacy as it may create problems in some setups, like yours 
> (ie. for disks, it's recommended to use UUIDs or /dev/disk/by-*).
> 
> I hope this answers your question.
Thank you, yes it does.

In our case the dom0 was updated to bookworm while the domU is still 
running bullseye.
-> updated Xen so the vif detection order changed (which we relied on)
-> the predictable network names for Xen don't work with bullseye

So my new resolution for bullseye domUs on a bookworm dom0 is to install 
udev from backports and change the domUs network config to use the new 
enXn naming scheme instead of ethn.

Hi,

On 8/8/23 15:22, Valentin Kleibel wrote:
>> On [0], you can read "In both cases the device naming is subject
to the
>> usual guest or backend domain facilities for renaming network
devices".
>> It says "naming/renaming", but you can assume
"detecting".
>>
>>> I also checked which net_ids udev knows about and the only things
that
>>> pop up are:
>>> ID_NET_NAMING_SCHEME=v247
>>> ID_NET_NAME_MAC=enx00163efd832b
>>> ID_OUI_FROM_DATABASE=Xensource, Inc.
What I do is stuff like this:

-$ cat /etc/udev/rules.d/70-persistent-vifname.rules

SUBSYSTEM=="net", DRIVERS=="?*",
ATTRS{nodename}=="device/vif/0",
NAME="vlan2"
SUBSYSTEM=="net", DRIVERS=="?*",
ATTRS{nodename}=="device/vif/1",
NAME="vlan3"
SUBSYSTEM=="net", DRIVERS=="?*",
ATTRS{nodename}=="device/vif/2",
NAME="vlan4"
SUBSYSTEM=="net", DRIVERS=="?*",
ATTRS{nodename}=="device/vif/3",
NAME="vlan6"
SUBSYSTEM=="net", DRIVERS=="?*",
ATTRS{nodename}=="device/vif/4",
NAME="vlan9"
SUBSYSTEM=="net", DRIVERS=="?*",
ATTRS{nodename}=="device/vif/5",
NAME="vlan10"
SUBSYSTEM=="net", DRIVERS=="?*",
ATTRS{nodename}=="device/vif/6",
NAME="vlan11"
SUBSYSTEM=="net", DRIVERS=="?*",
ATTRS{nodename}=="device/vif/7",
NAME="vlan12"
SUBSYSTEM=="net", DRIVERS=="?*",
ATTRS{nodename}=="device/vif/8",
NAME="vlan13"
SUBSYSTEM=="net", DRIVERS=="?*",
ATTRS{nodename}=="device/vif/9",
NAME="vlan14"
SUBSYSTEM=="net", DRIVERS=="?*",
ATTRS{nodename}=="device/vif/10",
NAME="vlan15"
SUBSYSTEM=="net", DRIVERS=="?*",
ATTRS{nodename}=="device/vif/11",
NAME="vlan16"

The vif/X always matches the order in which you define the interfaces
inside the guest config file.

After starting to build router VMs (well before the whole interface
naming madness was a thing), it took only the first time when we wanted
to throw away a vlan, to realize that all the ethX numbers would shift 1
up, and from then on, I've always been using this so set my own style
predictable names (whenever there's more than one, otherwise it's just
eth0).
>> Is it from dom0 or domU ?
>> Are you using "net.ifnames=0" on the domU kernel command line
?
>> "v247" looks like systemd "predictive naming
scheme" (eth -> enX).
>>  From bookworm on, domUs vifs get named enXN (enX0, enX1, ...).
>> Read on :
>>
https://www.debian.org/releases/stable/i386/release-notes/ch-information.en.html#xen-network
> 
> This is from the domU, running bullseye with a bookworm dom0.
> 
>> See how ethN interfaces get messed up, like in your setup, but 
>> predictable names would work, as you can see in "altname
enXN" :
>> eth1 (:01) -> enX1
>> eth2 (:10) -> enX10
>> eth3 (:02) -> enX2
But yeah, so, even while not depending on whatever order it gets
initialized, and still having it function correctly, this is still just
pretty annoying... If I'm doing stuff around here, and just quickly want
to look up things (e.g. messing around with vlan15 settings), and
quickly type ip a instead of having to spend more time typing ip a show
dev vlan15 jadijadi, I still every time get this short "WTF huh,
argh",
raises arms, does table flip, grmbl grbml feeling for a split second.

2: vlan2: <BROADCAST,MULT[...]
    [...]
    altname enX0
3: vlan3: <BROADCAST,MULT[...]
    [...]
    altname enX1
4: vlan15: <BROADCAST,MULT[...]
    [...]
    altname enX10
5: vlan16: <BROADCAST,MULT[...]
    [...]
    altname enX11
6: vlan4: <BROADCAST,MULT[...]
    [...]
    altname enX2
7: vlan6: <BROADCAST,MULT[...]
    [...]
    altname enX3
8: vlan9: <BROADCAST,MULT[...]
    [...]
    altname enX4
9: vlan10: <BROADCAST,MULT[...]
    [...]
    altname enX5
10: vlan11: <BROADCAST,MULT[...]
    [...]
    altname enX6
11: vlan12: <BROADCAST,MULT[...]
    [...]
    altname enX7
12: vlan13: <BROADCAST,MULT[...]
    [...]
    altname enX8
13: vlan14: <BROADCAST,MULT[...]
    [...]
    altname enX9

First, there's the split second start of "wait, what, I just added an
IP
address, where did the whole interface go?!" and then the "Oh, right,
of
course, it's the silly thing again.", and then it costs an extra second
looking at the screen like a confused Jeffrey Lebowski to find the right
line.
> I could not get our bullseye domU to show the "predictable names"
even
> though i tried installing the bullseye-backports kernel 6.1.
> After you wrote this i installed udev 252.5 from backports and it now 
> uses the correct enXn interface names, even with kernel 5.10.
> 
>> So, my answer does not tell you if something changed in Xen itself,
only
>> in Debian.
>> But I guess it relates to what Xen devs told us : vifs detection order 
>> cannot be relied upon, that's why "predictable names"
were invented.
>> The vif detection part is related to the domains kernels, not Xen
itself
>> (at least that's what I understood).
>>
>> Using eth0 nowadays is a bit like using /dev/sda for hard drives,
it's
>> considered legacy as it may create problems in some setups, like yours 
>> (ie. for disks, it's recommended to use UUIDs or /dev/disk/by-*).
>>
>> I hope this answers your question.
> 
> Thank you, yes it does.
> 
> In our case the dom0 was updated to bookworm while the domU is still 
> running bullseye.
> -> updated Xen so the vif detection order changed (which we relied on)
I didn't read the other mailthread on the xen list fully yet. But, I
think it's shouldn't be very hard to find the code changes and see if
it's deterministic and can just be fixed. Simply just to decrease the
totally unnecessary amount of silliness.
> -> the predictable network names for Xen don't work with bullseye
> 
> So my new resolution for bullseye domUs on a bookworm dom0 is to install 
> udev from backports and change the domUs network config to use the new 
> enXn naming scheme instead of ethn.
Or the "device/vif/X" way...

So, anyway, did someone already did some test "just because we can" to
see how much network interfaces you can get added for fun, and if the
pattern keeps looking the same, also with enX4 enX40 .. enX49 enX5 etc?
:D enX1 enX10 enX100 .. enX109 enX11 enX110 argh o_O

Have fun,
Hans

On 08 Aug 2023 16:59, Hans van Kranenburg wrote:
> I didn't read the other mailthread on the xen list fully yet.
You gave me the idea to post the IRC digest, so the report here is more 
complete, and people not tracking the xen-devel ML can read it nicely.
For those who do, the mail is dated "02 Aug 2023 18:19", and titled 
"Network interfaces naming changes in domUs with >10 vifs (Debian bug 
1042842)".
The ML post got no answer yet.

[--- IRC ---]
- AFAIK, there is no sorting in Xenstored. And you should not expect 
that even if libxl sorted properly it will be seen in the same order on 
the other end.
- is the ethN number in domU related to vif number in xenstore, or to 
device detection order?
- there's no order to eth names at all. they're allocated 
first-come-first-serve, so it entirely depends on how parallel the 
probing of nic drivers are. even if netfront is serialised around 
xenstore accesses, it probably allocates in the order that XS_DIRECTORY 
comes back with
- from simple tests, it looks like VIFs are created in Xenstore in the 
order of the config file, but if you "xenstore-ls /[...]/vif", you can
see vifs are ordered like vif1,vif10,vif11,vif2,etc
- the order is different between Xen 4.14 and 4.17 (ie. the "expected"
order works on 4.14, not 4.17)
- But really, Debian should have never relied on how the nodes are 
ordered. This is not something we guarantee in the Xenstored API
- the last big batch of XSA content for the xenstoreds did some major 
rearranging of oxenstored. We dropped a NIH second garbage collector, 
and a NIH weakref system IIRC. I could entirely believe that the 
apparent sort order changed as a result
- generally, I think Linux world established quite some time ago that 
ethN names are not stable
- It's definitely a complicated issue.  Perhaps best to post to 
xen-devel so we can have a discussion. I expect the answer is not-a-Xen 
bug, but I don't think we have a clear understanding of the problem yet
[--- /IRC ---]

I'll report back when having tested the 111 vifs domU ... if my system 
agrees o_O
As it requires a script to populate the cfg, one could also enhance it 
to try how dynamically adding/removing vifs is handled.

(BTW, before this report I thought Xen had a hard limit of 8 vifs per 
domU. Or was that only on FreeBSD domUs ? Can't remember).

Hello,

I'm currently in the process of cleaning up and i noticed this bug i 
reported is still open.

Based on what we know now i think the bug can be closed.

To summarize:
* the ordering of network interfaces (and therefore the ethn names) was 
never meant to be stable and changed with Xen 4.17
* domUs starting with bookworm use enXn by default and their order 
matches the config
* bullseye domUs on a bookworm dom0 with more than 10 network interfaces 
need a workaround:
   * use udev from bullseye-backports and enXn naming scheme
   * use custom udev rules to get fixed interface names

Thanks for your help with this issue,
Valentin

Your message dated Thu, 21 Mar 2024 18:02:31 +0100
with message-id <322cbbdb-94a4-4f3d-88af-1d80f9774aca at rabbit.lu>
and subject line Re: Bug#1042842: network interface names wrong in domU (>10
interfaces)
has caused the Debian Bug report #1042842,
regarding network interface names wrong in domU (>10 interfaces)
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner at bugs.debian.org
immediately.)


-- 
1042842: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1042842
Debian Bug Tracking System
Contact owner at bugs.debian.org with problems
-------------- next part --------------
An embedded message was scrubbed...
From: Valentin Kleibel <valentin at vrvis.at>
Subject: network interface names wrong in domU (>10 interfaces)
Date: Tue, 1 Aug 2023 20:36:20 +0200
Size: 6727
URL:
<http://alioth-lists.debian.net/pipermail/pkg-xen-devel/attachments/20240321/fd440033/attachment.eml>
-------------- next part --------------
An embedded message was scrubbed...
From: zithro <slack at rabbit.lu>
Subject: Re: Bug#1042842: network interface names wrong in domU (>10
interfaces)
Date: Thu, 21 Mar 2024 18:02:31 +0100
Size: 5460
URL:
<http://alioth-lists.debian.net/pipermail/pkg-xen-devel/attachments/20240321/fd440033/attachment-0001.eml>