Hi, I wonder if there is some way to control the network traffic of new domains(other than Dom0) in Xen. I mean although we can allocate an external IP address for each new domain, and the new domains can access the WAN as they wish, there is only one physical network card. So if each new domain has a lot of network traffic, the interrupts will keep the whole system very busy. I know we can control the CPU time, memory, and disk for new domains. So can we do something similar to that kind of scheduling for network? Or can someone tell me which part of the source code relates to this? I know little about network. So if my understanding is wrong, please point it out. Thanks. Xiaofang
> I wonder if there is some way to control the network traffic of new domains(other than Dom0) in Xen. > I mean although we can allocate an external IP address for each new domain, and the new domains can > access the WAN as they wish, there is only one physical network card. > So if each new domain has a lot of network traffic, the interrupts will keep the whole system very busy. > > I know we can control the CPU time, memory, and disk for new domains. > So can we do something similar to that kind of scheduling for network? > Or can someone tell me which part of the source code relates to this?See "xc_dom_control.py vif_setsched". You can use it to set rate limit for a domain''s transmit bandwidth in bytes per microsecond. As regards packets received, Xen is careful to do "early discard" on incoming packets for domains that are unable to keep up with packet arrivals, so only minimal resources are wasted. Under some deployment scenarios I guess it might be useful to be able to rate limit incoming traffic as well as out going. We currently don''t do this, but it might be possible to configure Linux tc to do this under the NGIO model in the unstable tree. Ian ------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we''ll give you the exam FREE. http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Hi, Can someone tell me what is the functionality of "backend" in XenoLinux? Take the network for example, it seems that all the virtual network interfaces corresponding to guest OS''s are created in Xen. And on the receiving side, interrupt VIRQ_NET is raised from Xen upward to XenoLinux. However, it seems that this kind of interrupts are not processed by "backend". So what does "backend" do for networking? For example, functions like ''netif_ctrlif_rx()'', ''net_rx_action()'', ''netif_be_start_xmit()'', etc. Thanks! Xiaofang
Hi, Can someone tell me what is the functionality of "backend" in XenoLinux? It seems that virtual network interfaces corresponding to guest OS''s are created in Xen, including ''net_get_target_vif()'', etc. And the virtual interrupt VIRQ_NET raised by Xen is handled by other parts than "backend" in XenoLinux. So what is the rough flow of network in the system? Take the receiving side for example, how does Xen cooperate with the "backend"? Thanks for your help! Xiaofang ------------------------------------------------------- This SF.Net email sponsored by Black Hat Briefings & Training. Attend Black Hat Briefings & Training, Las Vegas July 24-29 - digital self defense, top technical experts, no vendor pitches, unmatched networking opportunities. Visit www.blackhat.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Are you using Xen 1.2 or unstable? There have been big changes to networking in the unstable tree. In my previous e-mail I was describing the current state of the unstable tree. The old IO world: (as used by Xen 1.2 and as described in the SOSP paper) * Device drivers for physical network devices are in Xen. * All domains (including 0) use a "virtual" network driver, which talks to Xen. Xen then routes packets between domains or out onto the physical network. * VIRQ_NET is the IRQ associated with a virtual network device. Xen uses it to signal a domain when it has been sent data by another domain or from the physical network. * There''s no "backend", just Xen. The new IO world: (as used by the Unstable tree and the forthcoming 2.0 release) * Xen does not contain device drivers anymore. * Instead, the device drivers for physical devices run in domains. Usually, all the drivers for your hardware run in dom0. * A domain that''s controlling physical devices can export virtual devices to other domains. The driver which exports a device to other domains is called the "backend". * The driver which imports a virtual device from the backend is called the "frontend". This driver runs in unprivileged domains to give them their virtual devices. It''s a lot like the old (1.2) virtual network interface driver except that it talks to another domain instead of to Xen. * VIRQ_NET does not exist anymore. Instead, the backend driver (in the domain that controls the hardware) can send an interrupt to the frontend driver (in the domain using the virtual devices) using the event channel mechanism I described in my last e-mail. Does that make sense? Mark ------------------------------------------------------- This SF.Net email sponsored by Black Hat Briefings & Training. Attend Black Hat Briefings & Training, Las Vegas July 24-29 - digital self defense, top technical experts, no vendor pitches, unmatched networking opportunities. Visit www.blackhat.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Sorry for the spam. This message is because of our mail server problem. Xiaofang ----- Original Message ----- From: Xiaofang Chen To: xen-devel@lists.sourceforge.net Sent: Monday, June 28, 2004 9:50 PM Subject: [Xen-devel] Network questions Hi, Can someone tell me what is the functionality of "backend" in XenoLinux? Take the network for example, it seems that all the virtual network interfaces corresponding to guest OS''s are created in Xen. And on the receiving side, interrupt VIRQ_NET is raised from Xen upward to XenoLinux. However, it seems that this kind of interrupts are not processed by "backend". So what does "backend" do for networking? For example, functions like ''netif_ctrlif_rx()'', ''net_rx_action()'', ''netif_be_start_xmit()'', etc. Thanks! Xiaofang
Thanks a lot for the explainations! The idea seems clear to me now. I guess the code I was reading was a "middle" version between the ealy unstable and latest unstable. I got such a version by using "bk pull" based on the early unstable version, so that the code included both VIRQ_NET and "backend". For the latest unstable version, I am still not clear on how the "backend" controls the physical network device. For example, if I use a 3Com 3c59x.c driver, when packets come, after ''vortex_rx()'' is called, ''/net/core/dev.c: netif_rx()'' will be called. Along this way up, it seems that no functions in "backend" will be called. Also in /arch/xen/drivers/netif/backend/control.c: netif_ctrlif_init(), after the cmsg(CMSG_NETIF_BE_DRIVER_STATUS_CHANGED + NETIF_DRIVER_STATUS_UP) is put into ctrl_if tx_ring, seems no corresponding handlers are called because of id=0xFF. Confused here. Am I missing something? Thanks. Xiaofang> Are you using Xen 1.2 or unstable? There have been big changes tonetworking> in the unstable tree. In my previous e-mail I was describing the current > state of the unstable tree. > > The old IO world: (as used by Xen 1.2 and as described in the SOSP paper) > * Device drivers for physical network devices are in Xen. > * All domains (including 0) use a "virtual" network driver, which talks to > Xen. Xen then routes packets between domains or out onto the physicalnetwork.> * VIRQ_NET is the IRQ associated with a virtual network device. Xen usesit> to signal a domain when it has been sent data by another domain or fromthe> physical network. > * There''s no "backend", just Xen. > > The new IO world: (as used by the Unstable tree and the forthcoming 2.0 > release) > * Xen does not contain device drivers anymore. > * Instead, the device drivers for physical devices run in domains.Usually,> all the drivers for your hardware run in dom0. > * A domain that''s controlling physical devices can export virtual devicesto> other domains. The driver which exports a device to other domains iscalled> the "backend". > * The driver which imports a virtual device from the backend is called the > "frontend". This driver runs in unprivileged domains to give them their > virtual devices. It''s a lot like the old (1.2) virtual network interface > driver except that it talks to another domain instead of to Xen. > * VIRQ_NET does not exist anymore. Instead, the backend driver (in thedomain> that controls the hardware) can send an interrupt to the frontend driver(in> the domain using the virtual devices) using the event channel mechanism I > described in my last e-mail. > > Does that make sense? > > Mark > >------------------------------------------------------- This SF.Net email sponsored by Black Hat Briefings & Training. Attend Black Hat Briefings & Training, Las Vegas July 24-29 - digital self defense, top technical experts, no vendor pitches, unmatched networking opportunities. Visit www.blackhat.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> I guess the code I was reading was a "middle" version between the ealy > unstable and latest unstable. > I got such a version by using "bk pull" based on the early unstable version, > so that the code included both VIRQ_NET and "backend". > > For the latest unstable version, I am still not clear on how the "backend" > controls the physical network device. > For example, if I use a 3Com 3c59x.c driver, when packets come, after > ''vortex_rx()'' is called, ''/net/core/dev.c: netif_rx()'' will be called. > Along this way up, it seems that no functions in "backend" will be called.There is no net/core/dev.c in current Xen. I think your src repo is in a broken state. Reclone the repository. The backend netif driver appears in domain 0 Linux as just another network interface (vifX.Y). Packets tx''ed by the frontend appear as received packets here and vice versa. As to how packets get to the physical network, that''s down to how domain 0 is configured. You can either create an L2 bridge, or enable forwarding and configure IP routing appropriately. In both cases you can use iptables to provide firewalling functionality. Ian ------------------------------------------------------- This SF.Net email sponsored by Black Hat Briefings & Training. Attend Black Hat Briefings & Training, Las Vegas July 24-29 - digital self defense, top technical experts, no vendor pitches, unmatched networking opportunities. Visit www.blackhat.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
The until a few weeks ago, the unstable tree had a compile-time choice between "old" and "new" IO world. Recently, the old IO world has been removed, so if you have pulled recently, the VIRQ_NET and the drivers in Xen, etc should disappear. The backend driver creates an virtual "ethernet interface" for each unprivileged domain whose frontend connects to it. The kernel that is running the backend driver thinks that it''s connected to all the other domains by ethernet interfaces as well as to the physical network by the physical ethernet hardware. Getting packets between virtual interfaces of domains and the physical network is then just a task of bridging or routing, which is dealt with by the same code that would transfer packets between multiple ethernet interfaces in any Linux system. Currently, we use bridging by default. See the code in linux/net/bridge/ for more details about how the Linux bridging code works. In your example, when the packet is received on the Vortex card, it eventually gets passed to the bridging code, which identifies that it should be passed out on one of the virtual interfaces connected to another domain. It then passes the packet to the backend driver, which transfers it to the frontend driver of the destination domain and notifies that domain of the arrival. Since this stuff happens in the bridging code, you wouldn''t be able to find an obvious call path. As for the control messages, you should remember that the control interface is only used for a domain to talk to xend. The backend driver uses the message you described to tell Xend that it is UP and ready to accept connections from other domains. When other domains want to connect their network interfaces, they use their control interface to tell Xend. Xend co-ordinates the process of connecting, then once the connection is established, the two domains can talk directly using shared memory. You can find relevant code in tools/python/xen/xend/server/netif.py (I think) but you''ll need to read other bits of Xend to understand how this all fits together. It would be nice to have some documentation for this stuff (and we probably need some diagrams) in the interface manual... HTH, Mark ------------------------------------------------------- This SF.Net email sponsored by Black Hat Briefings & Training. Attend Black Hat Briefings & Training, Las Vegas July 24-29 - digital self defense, top technical experts, no vendor pitches, unmatched networking opportunities. Visit www.blackhat.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel