Hi All, I have a machine with three network cards, and the third card(named e1000g2) is in use now. I want to use e1000g2 as default NIC when I create Guest. How can I change the default NIC? I can override the default value of config/default-nic property of svc:/system/xvm/xend. But xend still use e1000g0 when creating Guest. BTW, I can explicitly set "bridge=e1000g2" when creating Guest and it works. Does any body know how to solve the problem? Best Regards Jiajun
David Edmondson
2007-Dec-17 09:46 UTC
Re: How to configure default NIC when creating Guest
Xu, Jiajun wrote:> I have a machine with three network cards, and the third card(named > e1000g2) is in use now. I want to use e1000g2 as default NIC when I > create Guest. How can I change the default NIC? I can override the > default value of config/default-nic property of svc:/system/xvm/xend. > But xend still use e1000g0 when creating Guest.Either you didn''t set it properly or there is a bug. Can you tell us the steps you took to set it? dme.
David Edmondson <> scribbled on :> Xu, Jiajun wrote: >> I have a machine with three network cards, and the third card(named >> e1000g2) is in use now. I want to use e1000g2 as default NIC when I >> create Guest. How can I change the default NIC? I can override the >> default value of config/default-nic property of svc:/system/xvm/xend. >> But xend still use e1000g0 when creating Guest. > > Either you didn''t set it properly or there is a bug. > > Can you tell us the steps you took to set it? > > dme.My steps following: [vt-dp3@/]svccfg -s xvm/xend setprop config/default-nic = astring: \"e1000g2\" [vt-dp3@/]svcadm refresh xvm/xend [vt-dp3@/]svcadm restart xvm/xend [vt-dp3@/]svccfg -s xvm/xend listprop | grep default config/default-nic astring e1000g2 But when I create a Guest and check with /usr/lib/vna, xend still use e1000g0 as default. [vt-dp3@/]/usr/lib/vna 900 e1000g0 0:16:3e:14:5d:27 [vt-dp3@/]dladm show-link e1000g0 type: non-vlan mtu: 1500 device: e1000g0 e1000g1 type: non-vlan mtu: 1500 device: e1000g1 e1000g2 type: non-vlan mtu: 1500 device: e1000g2 vnic900 type: non-vlan mtu: 1500 device: vnic900 [vt-dp3@/]uname -a SunOS vt-dp3 5.11 snv_77 i86pc i386 i86xpv Best Regards Jiajun
David Edmondson
2007-Dec-18 06:13 UTC
Re: How to configure default NIC when creating Guest
Xu, Jiajun wrote:> David Edmondson <> scribbled on : > >> Xu, Jiajun wrote: >>> I have a machine with three network cards, and the third card(named >>> e1000g2) is in use now. I want to use e1000g2 as default NIC when I >>> create Guest. How can I change the default NIC? I can override the >>> default value of config/default-nic property of svc:/system/xvm/xend. >>> But xend still use e1000g0 when creating Guest. >> Either you didn''t set it properly or there is a bug. >> >> Can you tell us the steps you took to set it? >> >> dme. > > My steps following:That looked fine. Did you reboot after changing the value? It occurs to me that the scripts cache the value of default-nic (or whatever they guess) in xenstore, which is perhaps the cause of the problem. dme.
David Edmondson <> scribbled on :> Xu, Jiajun wrote: >> David Edmondson <> scribbled on : >> >>> Xu, Jiajun wrote: >>>> I have a machine with three network cards, and the third card(named >>>> e1000g2) is in use now. I want to use e1000g2 as default NIC when I >>>> create Guest. How can I change the default NIC? I can override the >>>> default value of config/default-nic property of >>>> svc:/system/xvm/xend. But xend still use e1000g0 when creating >>>> Guest. >>> Either you didn''t set it properly or there is a bug. >>> >>> Can you tell us the steps you took to set it? >>> >>> dme. >> >> My steps following: > > That looked fine. Did you reboot after changing the value? It occurs > to me that the scripts cache the value of default-nic (or whatever > they guess) in xenstore, which is perhaps the cause of the problem. > > dme.After reboot xend still use e1000g0. :( Best Regards Jiajun
David Edmondson
2007-Dec-18 14:55 UTC
Re: How to configure default NIC when creating Guest
Xu, Jiajun wrote:> After reboot xend still use e1000g0. :(Please provide the output of "svcprop xvm/xend" and "/usr/lib/xen/bin/xenstore-read device-misc/vif/default-nic". dme.
Xu, Jiajun wrote>> David Edmondson <> scribbled on : > > > Xu, Jiajun wrote: > >> David Edmondson <> scribbled on : > >> > >>> Xu, Jiajun wrote: > >>>> I have a machine with three network cards, and the third card(named > >>>> e1000g2) is in use now. I want to use e1000g2 as default NIC when I > >>>> create Guest. How can I change the default NIC? I can override the > >>>> default value of config/default-nic property of > >>>> svc:/system/xvm/xend. But xend still use e1000g0 when creating > >>>> Guest. > >>> Either you didn''t set it properly or there is a bug. > >>> > >>> Can you tell us the steps you took to set it? > >>> > >> My steps following: > > > > That looked fine. Did you reboot after changing the value? It occurs > > to me that the scripts cache the value of default-nic (or whatever > > they guess) in xenstore, which is perhaps the cause of the problem. > > > > After reboot xend still use e1000g0. :(I think the default nic is cached in the "xenstore" registry Check with "/usr/lib/xen/bin/xenstore-read device-misc/vif/default-nic". You probably have to remove that property from xenstore, so that a new default nic is determined from svcprops or dladm (and written to xenstore). "/usr/lib/xen/bin/xenstore-rm device-misc/vif/default-nic" should remove the old default nic key/value. This message posted from opensolaris.org
David Edmondson
2007-Dec-18 15:16 UTC
Re: How to configure default NIC when creating Guest
Jürgen Keil wrote:> I think the default nic is cached in the "xenstore" registryIt is.> Check with "/usr/lib/xen/bin/xenstore-read device-misc/vif/default-nic". > > You probably have to remove that property from xenstore, > so that a new default nic is determined from svcprops or > dladm (and written to xenstore). > > "/usr/lib/xen/bin/xenstore-rm device-misc/vif/default-nic" > should remove the old default nic key/value.When I added this code the store was re-built from scratch every boot, so the entry didn''t persist across reboots. Did that change? dme.
David Edmondson <> scribbled on :> Xu, Jiajun wrote: >> After reboot xend still use e1000g0. :( > > Please provide the output of "svcprop xvm/xend" and > "/usr/lib/xen/bin/xenstore-read device-misc/vif/default-nic". > > dme.[vt-dp3@/]/usr/lib/xen/bin/xenstore-read device-misc/vif/default-nic e1000g0 [vt-dp3@/]svcprop xvm/xend general/enabled boolean true general/entity_stability astring Unstable general/single_instance boolean true config/dom0-cpus integer 0 config/dom0-min-mem integer 196 config/enable-dump boolean true config/stability astring Unstable config/xend-relocation-server boolean true config/xend-unix-server boolean true config/vncpasswd astring \\ config/xend-relocation-address astring localhost config/xend-relocation-hosts-allow astring \^flax$\ \^localhost$ config/default-nic astring e1000g2 xenstored/entities fmri svc:/system/xvm/store xenstored/grouping astring require_all xenstored/restart_on astring restart xenstored/type astring service start/exec astring /lib/svc/method/xend\ %m start/timeout_seconds count 0 start/type astring method stop/exec astring :kill stop/timeout_seconds count 60 stop/type astring method tm_common_name/C ustring Hypervisor\ Control\ Daemon tm_man_xend/manpath astring /usr/share/man tm_man_xend/section astring 1M tm_man_xend/title astring xend restarter/logfile astring /var/svc/log/system-xvm-xend:default.log restarter/contract count 47 restarter/start_pid count 313 restarter/start_method_timestamp time 1197982037.850521000 restarter/start_method_waitstatus integer 0 restarter/auxiliary_state astring none restarter/next_state astring none restarter/state astring online restarter/state_timestamp time 1197982037.853434000 Best Regards Jiajun
> I think the default nic is cached in the "xenstore" registry > > Check with "/usr/lib/xen/bin/xenstore-read > device-misc/vif/default-nic". > > You probably have to remove that property from xenstore, > so that a new default nic is determined from svcprops or > dladm (and written to xenstore). > > "/usr/lib/xen/bin/xenstore-rm device-misc/vif/default-nic" > should remove the old default nic key/value.Thanks for your info. But the command does not work on my system. [vt-dp3@/]/usr/lib/xen/bin/xenstore-read device-misc/vif/default-nic e1000g0 [vt-dp3@/]/usr/lib/xen/bin/xenstore-rm device-misc/vif/default-nic /usr/lib/xen/bin/xenstore-rm: could not remove path device-misc/vif/default-nic Best Regards Jiajun
> > I think the default nic is cached in the "xenstore" registry > > > Check with "/usr/lib/xen/bin/xenstore-read device-misc/vif/default-nic". > > > You probably have to remove that property from xenstore, > > so that a new default nic is determined from svcprops or > > dladm (and written to xenstore). > > > "/usr/lib/xen/bin/xenstore-rm device-misc/vif/default-nic" > > should remove the old default nic key/value. > > Thanks for your info. But the command does not work > on my system. > [vt-dp3@/]/usr/lib/xen/bin/xenstore-read device-misc/vif/default-nic > e1000g0 > [vt-dp3@/]/usr/lib/xen/bin/xenstore-rm device-misc/vif/default-nic > /usr/lib/xen/bin/xenstore-rm: could not remove path device-misc/vif/default-nicThat xenstore-rm failure might explain why the wrong interface is persistent across reboots. The snv77 smf(5) startup method for xend (/lib/svc/method/xend) tries to run exactly the same command to force selecting a new default nic when the first domain starts: /lib/svc/method/xend: 27 default_nic_path=device-misc/vif/default-nic .... 40 # 41 # Ensure that booting the first domain will determine 42 # the default NIC correctly. 43 # 44 /usr/lib/xen/bin/xenstore-rm $default_nic_path 2>/dev/null 45 46 /usr/lib/xend start During a reboot, did you notice any error messages on the console or in /var/adm/messages about a corrupted xenstore database? This message posted from opensolaris.org
> That xenstore-rm failure might explain why the wrong interface > is persistent across reboots. The snv77 smf(5) startup method for > xend (/lib/svc/method/xend) tries to run exactly the same command > to force selecting a new default nic when the first domain starts: > > /lib/svc/method/xend: > > 27 default_nic_path=device-misc/vif/default-nic .... > 40 # > 41 # Ensure that booting the first domain will determine > 42 # the default NIC correctly. > 43 # > 44 /usr/lib/xen/bin/xenstore-rm $default_nic_path > 2>/dev/null 45 > 46 /usr/lib/xend start > > > During a reboot, did you notice any error messages on the > console or in /var/adm/messages about a corrupted xenstore > database?There are some lines in /var/adm/messages which indicates xenstore corrupted: ################## Dec 20 09:38:58 vt-dp3 unix: [ID 881207 kern.notice] NOTICE: IRQ19 is shared Dec 20 09:38:58 vt-dp3 e1000g: [ID 766679 kern.info] Intel(R) PRO/1000 Network Connection, Driver Ver. 5.2.3 Dec 20 09:38:58 vt-dp3 xenstored: [ID 702911 daemon.error] Checking store ... Dec 20 09:38:58 vt-dp3 mac: [ID 469746 kern.info] NOTICE: e1000g2 registered Dec 20 09:38:58 vt-dp3 xpv_psm: [ID 803547 kern.info] xen_psm: pciex8086,10b9 (e1000g) instance 2 vector 0x10 ioapic 0x8 intin 0x10 is bound to cpu 6 Dec 20 09:38:58 vt-dp3 e1000g: [ID 766679 kern.info] Intel(R) PRO/1000 Network Connection, Driver Ver. 5.2.3 Dec 20 09:38:58 vt-dp3 xenstored: [ID 702911 daemon.error] Checking store complete. Dec 20 09:38:58 vt-dp3 xenstored: [ID 702911 daemon.error] Checking store ... Dec 20 09:38:58 vt-dp3 xenstored: [ID 702911 daemon.error] Checking store complete. Dec 20 09:38:58 vt-dp3 xpvd: [ID 395608 kern.info] evtchn@0, evtchn0 Dec 20 09:38:58 vt-dp3 genunix: [ID 936769 kern.info] evtchn0 is /xpvd/evtchn@0 Dec 20 09:38:58 vt-dp3 xenstored: [ID 702911 daemon.error] TDB error on read: Corrupt database ################## And I tried to remove /var/lib/xenstored/tdb and reboot the machine. After that, I even can not read device-misc/vif/default-nic. [vt-dp3@/]/usr/lib/xen/bin/xenstore-read device-misc/vif/default-nic /usr/lib/xen/bin/xenstore-read: couldn''t read path device-misc/vif/default-nic But after I create a Guest, I can read it and the default value is e1000g2 now. [vt-dp3@/]/usr/lib/xen/bin/xenstore-read device-misc/vif/default-nic e1000g2 Best Regards Jiajun
Hi,
For what it''s worth:
The new version 4.0 of NetBSD now supports Xen. But unfortunately there
are only non-pae kernels on the i386 installation cdrom for NetBSD at
http://www.netbsd.org/mirrors/
I did not find a pae Xen kernel for NetBSD yet
regards
Bernd
--
Bernd Schemmer, Frankfurt am Main, Germany
http://home.arcor.de/bnsmb/index.html
M s temprano que tarde el mundo cambiar .
Fidel Castro
Bernd Schemmer wrote:> Hi, > > For what it''s worth: > > The new version 4.0 of NetBSD now supports Xen. But unfortunately there > are only non-pae kernels on the i386 installation cdrom for NetBSD at > http://www.netbsd.org/mirrors/ > > I did not find a pae Xen kernel for NetBSD yet > > regards > > Bernd > > >NetBSD has supported Xen for a while, although 64bit support came in late (it''s not in 4.x). The reason that you can''t find a PAE kernel is because the NetBSD low-level VM code (pmap, we''d call it HAT) does not support PAE. So your best bet to run a NetBSD PV domU is probably to grab a NetBSD-current 64bit Xen snapshot. - Frank
Works for me, I don''t have a 32-bit machine with more than 4GB ram, actually all my machines are 64-bit here that need to be. But I do want to know how stable it is, or how robust, I know NetBSD was the first non-Linux host for Xen, but I don''t have experience with it. Problems I see personally are bad SMP scaling, I have experience with NetBSD on x86, x86_64, SPARC32, and SPARC64, and the scheduler is pretty horrific. There''s also no ZFS on NetBSD. Too bad FreeBSD''s Xen isn''t functional, yet it has functional ZFS and there''s experimental dtrace.. that is my preferred platform, even over Solaris, if only Xen was decent enough. James On Dec 19, 2007, at 11:52 PM, Bernd Schemmer wrote:> Hi, > > For what it''s worth: > > The new version 4.0 of NetBSD now supports Xen. But unfortunately > there > are only non-pae kernels on the i386 installation cdrom for NetBSD at > http://www.netbsd.org/mirrors/ > > I did not find a pae Xen kernel for NetBSD yet > > regards > > Bernd > > > -- > Bernd Schemmer, Frankfurt am Main, Germany > http://home.arcor.de/bnsmb/index.html > > M s temprano que tarde el mundo cambiar . > Fidel Castro > > _______________________________________________ > xen-discuss mailing list > xen-discuss@opensolaris.org
> Dec 20 09:38:58 vt-dp3 xenstored: [ID 702911 daemon.error] TDB error on read: Corrupt databaseAha! (I''ve had a couple of xenstory registry corruptions in the past, too)> And I tried to remove /var/lib/xenstored/tdb and > reboot the machine. After that, I even can not read > device-misc/vif/default-nic. > [vt-dp3@/]/usr/lib/xen/bin/xenstore-read device-misc/vif/default-nic > /usr/lib/xen/bin/xenstore-read: couldn''t read path device-misc/vif/default-nicOk, that''s the expected behaviour. A fresh / empty xenstore registry was created on the next start of the xenstore daemon.> But after I create a Guest, I can read it and the > default value is e1000g2 now. > [vt-dp3@/]/usr/lib/xen/bin/xenstore-read > device-misc/vif/default-nic > e1000g2Yep, the default nic was determined when starting the first guest domain, and the default nic was stored to the xenstore registry. This message posted from opensolaris.org
>> Dec 20 09:38:58 vt-dp3 xenstored: [ID 702911 daemon.error] > TDB error on read: Corrupt database > > Aha! > > (I''ve had a couple of xenstory registry corruptions in the past, too)Thanks for your explanation. :) So, do you think it''s a issue? And can you suggest what Nevada build should solve the problem? Best Regards Jiajun