[crossbow-discuss] Networking problem between virtual hosts on a single machine...

Rand S. Huntzinger
Thu, 15 Jul 2010 14:56:59 -0000

I have a observed a networking problem occurring largely between virtual hosts
running on the same machine. I have observed the problem with both VirtualBox
and xVM guests and it appears to affect multiple protocols (at least ssh and
NFS). It appears that an application network connection gets clogged with
transfers of data as small as 2K or so. For the purposes of testing,
I''m using the command "ssh HOST cat /etc/passwd ... | wc"
(where ... are additional copies of /etc/passwd).  The /etc/passwd file is a
little over 1K in size. I consider a test successful if a cat over 6000 bytes
succeeds.  Typically, the test fails with two copies of /etc/passwd (about 2400
bytes).  I am not an expert in networking so I''m describing the problem
at the application level.

First, let me describe my environment.  I am using three machines for testing
all of which are Dell 1950''s with bnx ethernets. All are running
OpenSolaris dev build 134. To make things simple, all virtual machines on these
machines used for testing are running OpenSolaris dev build 134 as well. There
are other virtual machines on the machines as well.  The machines are as
follows:

H1 - 2 zones, 4 VirtualBox guests (2 tested). The zones and VirtualBox guests
all have their own Crossbow VNIC.  All VNIC''s share one bnx network
interface.

H2 - 5 xVM domU''s (2 tested).  It is my understanding the xVM uses
Crossbow VNIC''s on their domU''s. Again all of these share a
single bnx NIC.

H3 - configured like H2 but we''re only using the dom0 on this host as
an external host during this testing.

Virtual hosts: Z1 and Z2 are ipkg zones on H1. V1 and V2 are VirtualBox guests
on H1. X1 and X1 are xVM domU''s on H2.

What fallows is a matrix which shows the results of my tests:

H1 <-> Z1	OK
H1 <-> Z2	OK
Z1 <-> Z2	OK
H1 <-> V1	OK
H1 <-> V2	OK
V1 <-> V2	Fail
H1 <-> H3	OK
V1  -> H3	OK
VI <-  H3	Fail
V2  -> H3	OK
V2 <-  H3	Fail
H2 <-> X1	OK
H2 <-> X2	OK
X1 <-> X2	Fail
H3 <-> X1	OK
H3 <-> X2	OK

It appears that communicaton between zones on one machine is OK and
communication between separate physical hosts is OK except for ssh''s
into VirtualBox hosts. The problem occurs between VirtualBox guests on one host
and xVM domU''s on a single host.

Symptoms:
"ssh HOST cat /etc/passwd | wc" will uniformly succeed and print size
info.
"ssh HOST cat /etc/passwd /etc/passwd | wc" hangs on transfers maked
"Fail."

I have also ovserved NFS mount and trnasfer problems between Xen
domU''s.

---

The networking between xVM and VirtualBox virtual hosts using Crossbow
VNIC''s for networking appears to be problematic.  I''m not even
sure I''d be able to identify this problem from a bug report if I saw
it. Does anyone recognize this problem?  Is there a fix or a workaround?