In trying to get dom0 running diskless, i''ve come across the problem where for a short time there is no network connectivity and so iscsi is unreachable. What is the accepted workaround? I can think of the following: . do it in initrd. I''d rather not put this sort of setup info in the initrd though. . patch brctl to do it in one hit. Looks easy but is it safe? Someone mentioned that the whole exe may not be loaded at that point and so could fail to load the rest if it needs to. . patch the kernel to support an ''add if to bridge + migrate address'' ioctl. This is probably best and looks pretty easy via an extra parameter to the existing addif ioctl. What is everyone else doing? James ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> In trying to get dom0 running diskless, i''ve come across the problem > where for a short time there is no network connectivity and so iscsi is > unreachable. What is the accepted workaround? I can think of the > following: > > . do it in initrd. I''d rather not put this sort of setup info in the > initrd though.You could allow your initrd to take configuration parameters from the kernel command line (they''re available as environment variables in the linuxrc).> . patch brctl to do it in one hit. Looks easy but is it safe? Someone > mentioned that the whole exe may not be loaded at that point and so > could fail to load the rest if it needs to.If you''re hacking brctl you could mlock the pages.> . patch the kernel to support an ''add if to bridge + migrate address'' > ioctl. This is probably best and looks pretty easy via an extra > parameter to the existing addif ioctl.Rather than doing this, I''d be inclined to hack the bridge code such that packets to a local IP are received rather than bridged. The current situation is at best inconsistent, where if a bridged interface has an IP address the host can still send packets with that IP on the interface, but it just can''t receive. I''m not convinced that''s the intended behaviour... Ian ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> > In trying to get dom0 running diskless, i''ve come across the problem > > where for a short time there is no network connectivity and so iscsiis> > unreachable. What is the accepted workaround? I can think of the > > following: > > > > . do it in initrd. I''d rather not put this sort of setup info in the > > initrd though. > > You could allow your initrd to take configuration parameters from > the kernel command line (they''re available as environment > variables in the linuxrc).I just had another idea, what about hacking the bridge code to take kernel parameters to autoconfig a bridge? eg ''br0=eth0''. In this way you have br0 at boot time and can use it from the word go. Kernel boottime dhcp would work etc. The only gotcha would be that you''d have to have the Ethernet driver you require compiled into the kernel, but that would be the case anyway if you were using kernel level dhcp.> > . patch brctl to do it in one hit. Looks easy but is it safe?Someone> > mentioned that the whole exe may not be loaded at that point and so > > could fail to load the rest if it needs to. > > If you''re hacking brctl you could mlock the pages.I thought there would be a way.> > . patch the kernel to support an ''add if to bridge + migrateaddress''> > ioctl. This is probably best and looks pretty easy via an extra > > parameter to the existing addif ioctl. > > Rather than doing this, I''d be inclined to hack the bridge code > such that packets to a local IP are received rather than > bridged. The current situation is at best inconsistent, where if > a bridged interface has an IP address the host can still send > packets with that IP on the interface, but it just can''t receive. > I''m not convinced that''s the intended behaviour...Hmmmm... that sounds like too much of a hack to me. I can understand the current behaviour... the kernel would ''route'' all packets received straight to the bridge, bypassing the original interface. James ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Kip Macy
2004-Oct-04 01:38 UTC
[Xen-devel] problems with falling off the network on enabling bridging
I''m writing in the hopes that someone might have some insight into some networking problems I''ve been encountering. I''m working with 2 dual Opteron 242 boxes from penguin which are supposed to be equivalent except for the memory configured the one falling off the net (shemp) has 2GB and the one working fine (curly) has 8GB. Because of issues I''ve seen with bridging on machines with multiple interfaces I have a separate brsetup script and set network-script to /bin/true in /etc/xen/xend-config.sxp. /etc/brsetup is nearly identical for curly and shemp - curly: #!/bin/csh -f /usr/sbin/brctl addbr xen-br0 /sbin/ifconfig xen-br0 172.16.9.211 netmask 255.255.255.0 up /usr/sbin/brctl addif xen-br0 eth0 /sbin/ip r d 172.16.9.0/24 dev eth0 /sbin/ip r a 172.16.9.0/24 dev xen-br0 /sbin/ip r d default via 172.16.9.1 dev eth0 /sbin/ip r a default via 172.16.9.1 dev xen-br0 shemp: #!/bin/csh -f /usr/sbin/brctl addbr xen-br0 /sbin/ifconfig xen-br0 172.16.9.212 netmask 255.255.255.0 up /usr/sbin/brctl addif xen-br0 eth0 /sbin/ip r d 172.16.9.0/24 dev eth0 /sbin/ip r a 172.16.9.0/24 dev xen-br0 /sbin/ip r d default via 172.16.9.1 dev eth0 /sbin/ip r a default via 172.16.9.1 dev xen-br0 About 2 minutes after running /etc/brsetup on shemp, shemp will stop responding to pings. They''re both running the same kernel from changeset 1.1354: kmacy@shemp cat /proc/version Linux version 2.6.8.1-xen0 (kmacy@siml3.eng.netapp.com) (gcc version 3.2 20020903 (Red Hat Linux 8.0 3.2-7)) #3 Sun Oct 3 16:38:26 PDT 2004 kmacy@curly cat /proc/version Linux version 2.6.8.1-xen0 (kmacy@siml3.eng.netapp.com) (gcc version 3.2 20020903 (Red Hat Linux 8.0 3.2-7)) #3 Sun Oct 3 16:38:26 PDT 2004 The difference between the dmesg is small: kmacy@curly diff ~/dmesg.curly ~/dmesg.shemp 14c14 < Xen reported: 1603.677 MHz processor. ---> Xen reported: 1603.650 MHz processor.52c52 < eth0: Tigon3 [partno(BCM95702A20) rev 1002 PHY(5703)] (PCI:66MHz:32-bit) 10/100/1000BaseT Ethernet 00:50:45:01:03:3a ---> eth0: Tigon3 [partno(BCM95702A20) rev 1002 PHY(5703)](PCI:66MHz:32-bit) 10/100/1000BaseT Ethernet 00:50:45:01:03:64 55c55 < eth1: Tigon3 [partno(BCM95702A20) rev 1002 PHY(5703)] (PCI:66MHz:32-bit) 10/100/1000BaseT Ethernet 00:50:45:01:03:3b ---> eth1: Tigon3 [partno(BCM95702A20) rev 1002 PHY(5703)](PCI:66MHz:32-bit) 10/100/1000BaseT Ethernet 00:50:45:01:03:65 82,83c82,83 < pIII_sse : 2502.000 MB/sec < raid5: using function: pIII_sse (2502.000 MB/sec) ---> pIII_sse : 2539.200 MB/sec > raid5: using function: pIII_sse (2539.200 MB/sec)Shemp initial console: raid5: automatically using best checksumming function: pIII_sse pIII_sse : 2480.000 MB/sec raid5: using function: pIII_sse (2480.000 MB/sec) Booting ''Xen 1.1354 / XenoLinux 2.6.8.1'' kernel /xen-1.1354.gz dom0_mem=262144 com1=9600,8n1 pdb=com1H console=com1 [Multiboot-elf, <0x100000:0x4dd50:0x346d0>, shtab=0x183078, entry=0x100000] module /vmlinuz-1.1354 root=/dev/hda3 ro [Multiboot-module @ 0x184000, 0x3fb134 bytes] __ __ ____ ___ _ _ \ \/ /___ _ __ |___ \ / _ \ | |__ ___| |_ __ _ \ // _ \ ''_ \ __) || | | |__| ''_ \ / _ \ __/ _` | / \ __/ | | | / __/ | |_| |__| |_) | __/ || (_| | /_/\_\___|_| |_| |_____(_)___/ |_.__/ \___|\__\__,_| http://www.cl.cam.ac.uk/netos/xen University of Cambridge Computer Laboratory Xen version 2.0-beta (kmacy@eng.netapp.com) (gcc version 3.2 20020903 (Red Hat Linux 8.0 3.2-7)) Fri Oct 1 16:32:56 PDT 2004 (XEN) Initialised 2047MB memory (524144 pages) on a 2047MB machine (XEN) Xen heap size is 10664KB (XEN) CPU0: Before vendor init, caps: 078bfbff e1d3fbff 00000000, vendor = 2 (XEN) CPU caps: 078bfbff e1d3fbff 00000000 00000000 (XEN) found SMP MP-table at 000f69a0 (XEN) Memory Reservation 0xf69a0, 4096 bytes (XEN) Memory Reservation 0x9bd70, 4096 bytes (XEN) ACPI: RSDP (v002 PTLTD ) @ 0x000f6920 (XEN) ACPI: XSDT (v001 PTLTD XSDT 0x06040000 LTP 0x00000000) @ 0x7ff734d8 (XEN) ACPI: FADT (v003 AMD HAMMER 0x06040000 PTEC 0x000f4240) @ 0x7ff75e46 (XEN) ACPI: MADT (v001 PTLTD APIC 0x06040000 LTP 0x00000000) @ 0x7ff75f3a (XEN) ACPI: SPCR (v001 PTLTD $UCRTBL$ 0x06040000 PTL 0x00000001) @ 0x7ff75fb0 (XEN) ACPI: DSDT (v001 AMD-K8 AMDACPI 0x06040000 MSFT 0x0100000e) @ 0x00000000 (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) (XEN) Processor #0 Unknown CPU [15:5] APIC version 16 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) (XEN) Processor #1 Unknown CPU [15:5] APIC version 16 (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) Using ACPI for processor (LAPIC) configuration information (XEN) Intel MultiProcessor Specification v1.4 (XEN) Virtual Wire compatibility mode. (XEN) OEM ID: AMD Product ID: HAMMER APIC at: 0xFEE00000 (XEN) I/O APIC #2 Version 17 at 0xFEC00000. (XEN) I/O APIC #3 Version 17 at 0xFC000000. (XEN) I/O APIC #4 Version 17 at 0xFC001000. (XEN) Enabling APIC mode: Flat. Using 3 I/O APICs (XEN) Processors: 2 curly: kernel /xen-1.1354.gz dom0_mem=262144 com1=9600,8n1 pdb=com1H console=com1 [Multiboot-elf, <0x100000:0x4dd50:0x346d0>, shtab=0x183078, entry=0x100000] module /vmlinuz-1.1354 root=/dev/hda3 ro [Multiboot-module @ 0x184000, 0x3fb134 bytes] __ __ ____ ___ _ _ \ \/ /___ _ __ |___ \ / _ \ | |__ ___| |_ __ _ \ // _ \ ''_ \ __) || | | |__| ''_ \ / _ \ __/ _` | / \ __/ | | | / __/ | |_| |__| |_) | __/ || (_| | /_/\_\___|_| |_| |_____(_)___/ |_.__/ \___|\__\__,_| http://www.cl.cam.ac.uk/netos/xen University of Cambridge Computer Laboratory Xen version 2.0-beta (kmacy@eng.netapp.com) (gcc version 3.2 20020903 (Red Hat Linux 8.0 3.2-7)) Fri Oct 1 16:32:56 PD T 2004 (XEN) Initialised 4031MB memory (1032048 pages) on a 4031MB machine (XEN) Xen heap size is 10600KB (XEN) CPU0: Before vendor init, caps: 078bfbff e1d3fbff 00000000, vendor = 2 (XEN) CPU caps: 078bfbff e1d3fbff 00000000 00000000 (XEN) found SMP MP-table at 000f69a0 (XEN) Memory Reservation 0xf69a0, 4096 bytes (XEN) Memory Reservation 0x9bd70, 4096 bytes (XEN) ACPI: RSDP (v002 PTLTD ) @ 0x000f6920 (XEN) ACPI: XSDT (v001 PTLTD XSDT 0x06040000 LTP 0x00000000) @ 0xfbf734d8 (XEN) ACPI: FADT (v003 AMD HAMMER 0x06040000 PTEC 0x000f4240) @ 0xfbf75e46 (XEN) ACPI: MADT (v001 PTLTD APIC 0x06040000 LTP 0x00000000) @ 0xfbf75f3a (XEN) ACPI: SPCR (v001 PTLTD $UCRTBL$ 0x06040000 PTL 0x00000001) @ 0xfbf75fb0 (XEN) ACPI: DSDT (v001 AMD-K8 AMDACPI 0x06040000 MSFT 0x0100000e) @ 0x00000000 (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) (XEN) Processor #0 Unknown CPU [15:5] APIC version 16 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) (XEN) Processor #1 Unknown CPU [15:5] APIC version 16 (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) Using ACPI for processor (LAPIC) configuration information (XEN) Intel MultiProcessor Specification v1.4 (XEN) Virtual Wire compatibility mode. (XEN) OEM ID: AMD Product ID: HAMMER APIC at: 0xFEE00000 (XEN) I/O APIC #2 Version 17 at 0xFEC00000. (XEN) I/O APIC #3 Version 17 at 0xFC000000. (XEN) I/O APIC #4 Version 17 at 0xFC001000. (XEN) Enabling APIC mode: Flat. Using 3 I/O APICs (XEN) Processors: 2 Thanks for any help. -Kip ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> I just had another idea, what about hacking the bridge code to take > kernel parameters to autoconfig a bridge? eg ''br0=eth0''. In this way you > have br0 at boot time and can use it from the word go. Kernel boottime > dhcp would work etc. The only gotcha would be that you''d have to have > the Ethernet driver you require compiled into the kernel, but that would > be the case anyway if you were using kernel level dhcp.If you go down this route, it''s probably best to have something like: "bridge=eth0 ip=:::::br0:dhcp" where bridge is a comma separated list of interfaces to be added to the bridge, perhaps with '':'' to separate multiple bridges. It all gets a bit messy if you need to set any of the other bridge parameters (stp,forward delay, hello, etc.). I''m not sure we''d ever get this in to the mainstream Linux tree, though it might be worth talking to the bridge folks. Personally, I''d prefer a solution for atomically creating and configuring the bridge that we could use after boot time too. Ian ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel