Marc G. Fournier
2008-Oct-29 00:14 UTC
Problem with Bridging ... and bge devices under FreeBSD 7.x?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm trying to run a QEMU VM on top of a FreeBSD 7.x server ... I've tried the exact same setup on my desktop, using 192.168.1.x and an fxp device, and it all works perfectly, but as soon as I do this on another machine on a public IP, I'm not getting any routing, I can't even ping it from the same machine ... My first thought was that there was an issue with IP aliases already on the bge device, but tried doing the following: ifconfig bridge0 destroy ifconfig tap0 destroy ifconfig fxp0 -alias 192.168.1.101 ifconfig fxp0 alias 192.168.1.101 netmask 255.255.255.255 ifconfig bridge0 create ifconfig tap0 create ifconfig bridge0 addm fxp0 addm tap0 up on my desktop here and then starting up the qemu image, and all worked as expected, so having an alias on the interface, before or after, doesn't make a difference ... at least with the fxp device ... Using VNC to connect to the VM, I can look at the interface, and it says it is connected ... and the IP/Gateway are all set right for the network I'm on, netmask is set to 255.255.255.0, same as on the 'private network' ... Please note that when I say "it works" on my private network / desktop, I'm using it to connect to my work computer, across the Internet, via Windows RDP, and it works flawlessly ... Looking at /var/log/messages, you can see the bridge being setup: Oct 27 18:53:21 io kernel: bridge0: Ethernet address: ce:44:c7:1b:47:40 as well as the tap device: Oct 27 18:53:25 io kernel: tap0: Ethernet address: 00:bd:96:ae:67:00 Oct 27 18:53:41 io kernel: tap0: promiscuous mode enabled and the ethernet going promiscuous: Oct 26 20:53:56 ganymede kernel: fxp0: promiscuous mode enabled So, all I have left is that everything is being setup okay, but there is something I'm missing here ... something with bridge<->bge, maybe? I've even tries to compare the output of 'ifconfig -a' as far as the bridge0 and tap0 devices are concerned, and other then the mac address, they look identical also ... So, pointers to what I may be missing here? a sysctl value that I need to set for this interface? Thanks ... - -- Marc G. Fournier Hub.Org Hosting Solutions S.A. (http://www.hub.org) Email . scrappy@hub.org MSN . scrappy@hub.org Yahoo . yscrappy Skype: hub.org ICQ . 7615664 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkHpscACgkQ4QvfyHIvDvPnFgCgk+6Pg+QeYO0BD9KMIkyZK2g7 JWgAn3VHq+F1OzD9M8VuYLEZDQLfFsNU =+3J/ -----END PGP SIGNATURE-----
Michael Proto
2008-Oct-29 02:35 UTC
Problem with Bridging ... and bge devices under FreeBSD 7.x?
On Tue, Oct 28, 2008 at 7:56 PM, Marc G. Fournier <freebsd@hub.org> wrote:> -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > I'm trying to run a QEMU VM on top of a FreeBSD 7.x server ... I've tried > the > exact same setup on my desktop, using 192.168.1.x and an fxp device, and it > all > works perfectly, but as soon as I do this on another machine on a public > IP, > I'm not getting any routing, I can't even ping it from the same machine ... > > My first thought was that there was an issue with IP aliases already on > the > bge device, but tried doing the following: > > ifconfig bridge0 destroy > ifconfig tap0 destroy > ifconfig fxp0 -alias 192.168.1.101 > ifconfig fxp0 alias 192.168.1.101 netmask 255.255.255.255 > ifconfig bridge0 create > ifconfig tap0 create > ifconfig bridge0 addm fxp0 addm tap0 up > > on my desktop here and then starting up the qemu image, and all worked as > expected, so having an alias on the interface, before or after, doesn't > make a > difference ... at least with the fxp device ... > > Using VNC to connect to the VM, I can look at the interface, and it says it > is > connected ... and the IP/Gateway are all set right for the network I'm on, > netmask is set to 255.255.255.0, same as on the 'private network' ... > > Please note that when I say "it works" on my private network / desktop, I'm > using it to connect to my work computer, across the Internet, via Windows > RDP, > and it works flawlessly ... > > Looking at /var/log/messages, you can see the bridge being setup: > > > Oct 27 18:53:21 io kernel: bridge0: Ethernet address: ce:44:c7:1b:47:40 > > as well as the tap device: > > Oct 27 18:53:25 io kernel: tap0: Ethernet address: 00:bd:96:ae:67:00 > Oct 27 18:53:41 io kernel: tap0: promiscuous mode enabled > > and the ethernet going promiscuous: > > Oct 26 20:53:56 ganymede kernel: fxp0: promiscuous mode enabled > > So, all I have left is that everything is being setup okay, but there is > something I'm missing here ... something with bridge<->bge, maybe? I've > even > tries to compare the output of 'ifconfig -a' as far as the bridge0 and tap0 > devices are concerned, and other then the mac address, they look identical > also > ... > > So, pointers to what I may be missing here? a sysctl value that I need to > set > for this interface? > >I'm having a little trouble understanding the setup you have. In your test case, is the IP of your VM 192.168.1.101? If so, then I don't think you want that IP aliased on the physical interface of your bridge. The VM NIC will answer for packets destined on your local segment, which the bridge would forward to the physical interface. If you assign the VM's IP to that physical interface, then your host would think that traffic is destined for itself and not pass it to the bridge. If I'm misunderstanding and the 192.168.1.101 alias (or whatever the equiv in your production setup) isn't being used by your VM then I would start looking at the ARP traffic crossing both the tap0, lo0, and physical interfaces. What does an 'ifconfig -a' look like on both systems? netstat -rn? Any packet filtering? -Proto
Marc G. Fournier
2008-Oct-29 03:35 UTC
Problem with Bridging ... and bge devices under FreeBSD 7.x?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - --On Tuesday, October 28, 2008 22:08:18 -0400 Michael Proto <mike@jellydonut.org> wrote:> > > > On Tue, Oct 28, 2008 at 7:56 PM, Marc G. Fournier <freebsd@hub.org> wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > I'm trying to run a QEMU VM on top of a FreeBSD 7.x server ... I've tried the > exact same setup on my desktop, using 192.168.1.x and an fxp device, and it > all > works perfectly, but as soon as I do this on another machine on a public IP, > I'm not getting any routing, I can't even ping it from the same machine ... > > My first thought was that there was an issue with IP aliases already on the > bge device, but tried doing the following: > > ifconfig bridge0 destroy > ifconfig tap0 destroy > ifconfig fxp0 -alias 192.168.1.101 > ifconfig fxp0 alias 192.168.1.101 netmask 255.255.255.255 > ifconfig bridge0 create > ifconfig tap0 create > ifconfig bridge0 addm fxp0 addm tap0 up > > on my desktop here and then starting up the qemu image, and all worked as > expected, so having an alias on the interface, before or after, doesn't make a > difference ... at least with the fxp device ... > > Using VNC to connect to the VM, I can look at the interface, and it says it is > connected ... and the IP/Gateway are all set right for the network I'm on, > netmask is set to 255.255.255.0, same as on the 'private network' ... > > Please note that when I say "it works" on my private network / desktop, I'm > using it to connect to my work computer, across the Internet, via Windows RDP, > and it works flawlessly ... > > Looking at /var/log/messages, you can see the bridge being setup: > > > Oct 27 18:53:21 io kernel: bridge0: Ethernet address: ce:44:c7:1b:47:40 > > as well as the tap device: > > Oct 27 18:53:25 io kernel: tap0: Ethernet address: 00:bd:96:ae:67:00 > Oct 27 18:53:41 io kernel: tap0: promiscuous mode enabled > > and the ethernet going promiscuous: > > Oct 26 20:53:56 ganymede kernel: fxp0: promiscuous mode enabled > > So, all I have left is that everything is being setup okay, but there is > something I'm missing here ... something with bridge<->bge, maybe? I've even > tries to compare the output of 'ifconfig -a' as far as the bridge0 and tap0 > devices are concerned, and other then the mac address, they look identical > also > ... > > So, pointers to what I may be missing here? a sysctl value that I need to set > for this interface? > > > > > I'm having a little trouble understanding the setup you have. In your test > case, is the IP of your VM 192.168.1.101? If so, then I don't think you want > that IP aliased on the physical interface of your bridge. The VM NIC will > answer for packets destined on your local segment, which the bridge would > forward to the physical interface. If you assign the VM's IP to that physical > interface, then your host would think that traffic is destined for itself and > not pass it to the bridge. > > If I'm misunderstanding and the 192.168.1.101 alias (or whatever the equiv in > your production setup) isn't being used by your VM then I would start looking > at the ARP traffic crossing both the tap0, lo0, and physical interfaces. > > What does an 'ifconfig -a' look like on both systems? netstat -rn? Any packet > filtering?I always fear I'm going to send more info then I should, and generate chaos and confusion :) On my test box, the VM is set to 192.168.1.100 ... the alias I added to fxp0 was to simulate what I have on the "public server", where there is a bge0 device with n aliases attached to it ... in no case is the IP assigned to the VM actually aliased onto any interface on the network itself Now, to try and answer your other questions ... netstat -nr on the 192 server shows the IP to be at:> netstat -nr | grep 168.1.100192.168.1.100 52:54:00:12:34:56 UHLW 1 1 fxp0 1128 which is very odd, as that MAC address is not found via ifconfig -a:> ifconfig -a | grep 52 >while arp -a also shows the 52:54 MAC, although MACs for the ifconfig -a are, in fact:> ifconfig -a | grep etherether 00:02:b3:ee:da:3e ether 5e:d1:e6:8b:55:50 ether 00:bd:25:18:6d:00 On the server, I'm getting nothing in arp or netstat for the IP in question: io# arp -a | grep 204.213 io# netstat -nr | grep 204.213 io# I've even tried doing a ping *from* the VM (logged in with VNC) to see if it will broadcast itself out, and nothing ... I'm starting QEMU on both servers with the same options as well: qemu -m 512M -net nic -net tap winxp.img just to confirm that I'm not doing anything different for attaching to the network ... So, right now, all I can see as being "different" is bge vs fxp interfaces ... both machines are running 7.x ... - -- Marc G. Fournier Hub.Org Hosting Solutions S.A. (http://www.hub.org) Email . scrappy@hub.org MSN . scrappy@hub.org Yahoo . yscrappy Skype: hub.org ICQ . 7615664 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkH2gcACgkQ4QvfyHIvDvNHUgCgtQORpycxkREQuiogWWOwydWG WfUAoOlRghz5Iy7XYWwwpOI5JgMjmBfi =3Q5f -----END PGP SIGNATURE-----