Hi, I work with Reiner and was formerly the manager of the L4 SawMill project. SawMill was a very different project where we were looking to manage the complexity of OS code and improve security using multiservers -- i.e., isolation of individual parts of the operating system from each other running on the hypervisor. Few modifications to the hypervisor (in this case, L4) were required for this, but many mods to the OS code. The new project aims to add security checks on hypervisor resources in the hypervisor. Thus, changes to the hypervisor are necessary. The best analogue for what is being proposed is the addition of the Linux Security Modules (LSM) interface to Linux 2.6 -- this enables mandatory access control to be enforced on the use of Linux resources by independent security modules which permits flexible security choices. Here is a brief list of the lessons that I think that we may be able to apply to Xen work based on both the SawMill and LSM experiences. - minimize (manual) changes required to guest OSs (at this time SawMill required many changes) - enable use of many drivers (i.e., enable driver development via driver model -- Xen aims to provide this) - isolation boundaries cost more than you might think, but computers are a lot faster now (10X+) - we need mandatory access control interface like LSM to have flexible control of resources (rather than dump resources to a control partition) - authorize access at bind time rather than at use time (no critical path impact) I am sure that there are others. Regards, Trent. ------------------------------------------------------------ Trent Jaeger IBM T.J. Watson Research Center 19 Skyline Drive, Hawthorne, NY 10532 (914) 784-7225, FAX (914) 784-7225 ----- Forwarded by Reiner Sailer/Watson/IBM on 01/18/2005 07:34 PM ----- Jacob Gorm Hansen <jacobg@diku.dk> 01/18/2005 07:28 PM To Reiner Sailer/Watson/IBM@IBMUS cc xen-devel@lists.sourceforge.net Subject Re: [Xen-devel] Xen on /. again Reiner Sailer wrote:> It is not that fast. sHype is not in any Xen source > yet. Currently it is running on our research hypervisor. > We want to discuss with the Xen community while we > are porting it to Xen so we can address comments and > adapt it where necessary.hi, is sHype related to the old L4 Sawmill project in any way? Are there any lessons learned from that project that would be relevant to the work on Xen? Jacob
Trent Jaeger wrote:> - isolation boundaries cost more than you might think, but computers are > a lot faster now (10X+) > - we need mandatory access control interface like LSM to have flexible > control of resources (rather than dump resources to a control partition) > - authorize access at bind time rather than at use time (no critical > path impact)Is there any reason to allow communication between VMs, other than driver access? It seems to me you can just use network communication (potentially signed or encrypted) when you need to communicate, and that anything else would violate the fundamental ''virtual machine'' abstraction? Also, I suppose you will wish to prevent covert channels between domains, e.g. domains communicating using various timing attacks (I move the disk head to the other end of the disk if I wish to tell you something), or by allocating/freeing certains parts of memory. How much will you need to dumb down the VMs view of what is going on in the machine to achieve this (not expose real time information, not expose real page tables), and how much of a VMM will there be left when you are done? best regards, Jacob ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> Is there any reason to allow communication between VMs, other than > driver access? It seems to me you can just use network communication > (potentially signed or encrypted) when you need to communicate, and that > anything else would violate the fundamental ''virtual machine'' abstraction?You might want to have two domU''s shared memory to implement a "direct point-to-point" ethernet link, to avoid the overheads of going through dom0. For most people this might not matter but it''s nice to have the option for maximal performance.> Also, I suppose you will wish to prevent covert channels between > domains, e.g. domains communicating using various timing attacks (I move > the disk head to the other end of the disk if I wish to tell you > something), or by allocating/freeing certains parts of memory. > > How much will you need to dumb down the VMs view of what is going on in > the machine to achieve this (not expose real time information, not > expose real page tables), and how much of a VMM will there be left when > you are done?Well domains are not aware of each other''s memory usage, so I wouldn''t have thought that allocation / exposing real page tables would matter. (Except dom0 can of course see everything if it wants). Timing related attacks are somewhat trickier to eliminate covert channels in, although some randomisation can limit the bandwidth. Cheers, Mark ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Mark Williamson wrote:> Well domains are not aware of each other''s memory usage, so I wouldn''t have > thought that allocation / exposing real page tables would matter. (Except > dom0 can of course see everything if it wants).Microkernel people like to make the argument that you could create a low-bandwidth covert channel by systematically allocating and freeing a set of pages, and because domains see real page frame numbers they can learn the state of the other guy by looking at what pages they get in return from an alloc call. I suppose you could randomize the page-allocator, but then you will have to leave a certain amount of pages unused at all times, to have enough random pages to choose from. (For myself, I would much rather have the real page tables and find a way to live with the covert channels, but I am not building military-grade systems). Jacob ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> Microkernel people like to make the argument that you could create a > low-bandwidth covert channel by systematically allocating and freeing a > set of pages, and because domains see real page frame numbers they can > learn the state of the other guy by looking at what pages they get in > return from an alloc call. I suppose you could randomize the > page-allocator, but then you will have to leave a certain amount of > pages unused at all times, to have enough random pages to choose from.Oh, I suppose you could do this using the balloon driver to relinquish / reclaim physical memory - I hadn''t thought of that. You could just restrict domain''s ability to do these operations and retain the ability to see the real page tables. Ballooning probably isn''t that important... The mechanisms for the network driver donating pages to the backend would be the other place where restrictions might be needed in this respect.> (For myself, I would much rather have the real page tables and find a > way to live with the covert channels, but I am not building > military-grade systems).Yes, it''s worth being able to limit these things as much as possible whilst still retaining functionality - ideally I guess there''d be a sliding scale of covert channel bandwidth vs. performance / functionality. Cheers, Mark ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> Is there any reason to allow communication between VMs, other thandriver access? In general, I agree with Mark''s reply to this. We may want to implement a faster mechanism for communication between 2 VMs where security requirements allow. However, this implies that we have a precise understanding of how the mechanisms impact security requirements.> Also, I suppose you will wish to prevent covert channels betweenPreventing covert channels is not a focus of our initial work. Our focus initially is on providing basic mandatory access control mechanisms where MAC policies may aim for confinement of errors (e.g., type enforcement policy), protect VM integrity (e.g., via Biba or Clark-Wilson), and/or prevent leakage (e.g., MLS). We have done work on policies of all these types for Linux. For MLS, I expect that we will work to identify covert storage channels (these are generally straightforward) and identify possible solutions, so that these can be closed. However, some mechanisms may be useful in some cases, but not meet security requirements in others. I am more familiar with the I/O descriptor ring mechanism than how the real page tables are exposed, so I''ll use this as an example. The I/O descriptor rings provide high performance communication, but cannot be used to implement a low secrecy to high secrecy communication. Of course, the low secrecy VM can see how the high secrecy component removes data from the ring, so this is a covert (storage) channel. Thus, another mechanism will have to be used. This could be setup above Xen (d0?), but the choice must meet the VMs security requirements. Timing channels are much more difficult to find/solve and much more expensive to solve, so I don''t expect much work on them for a while. Some general techniques are known (e.g., fuzzy time), but these are expensive, so it is not clear when/if there will be a demand.> How much will you need to dumb down the VMs view of what is going on inToo early to say, but hopefully we only need to dumb down VMs where requirements dictate. Regards, Trent. ------------------------------------------------------------ Trent Jaeger IBM T.J. Watson Research Center 19 Skyline Drive, Hawthorne, NY 10532 (914) 784-7225, FAX (914) 784-7225 Jacob Gorm Hansen <jacobg@diku.dk> 01/20/2005 05:11 PM To: Trent Jaeger/Watson/IBM@IBMUS cc: Xen-devel@lists.sourceforge.net Subject: Re: Fw: [Xen-devel] Xen on /. again Trent Jaeger wrote:> - isolation boundaries cost more than you might think, but computers are > a lot faster now (10X+) > - we need mandatory access control interface like LSM to have flexible > control of resources (rather than dump resources to a control partition) > - authorize access at bind time rather than at use time (no critical > path impact)Is there any reason to allow communication between VMs, other than driver access? It seems to me you can just use network communication (potentially signed or encrypted) when you need to communicate, and that anything else would violate the fundamental ''virtual machine'' abstraction? Also, I suppose you will wish to prevent covert channels between domains, e.g. domains communicating using various timing attacks (I move the disk head to the other end of the disk if I wish to tell you something), or by allocating/freeing certains parts of memory. How much will you need to dumb down the VMs view of what is going on in the machine to achieve this (not expose real time information, not expose real page tables), and how much of a VMM will there be left when you are done? best regards, Jacob
Mark Williamson wrote:>>Also, I suppose you will wish to prevent covert channels between >>domains, e.g. domains communicating using various timing attacks (I move >>the disk head to the other end of the disk if I wish to tell you >>something), or by allocating/freeing certains parts of memory. >> >>How much will you need to dumb down the VMs view of what is going on in >>the machine to achieve this (not expose real time information, not >>expose real page tables), and how much of a VMM will there be left when >>you are done? > > Well domains are not aware of each other''s memory usage, so I wouldn''t have > thought that allocation / exposing real page tables would matter. (Except > dom0 can of course see everything if it wants).Information about other domains'' memory usage is leaked via the hardware->physical mapping.> Timing related attacks are somewhat trickier to eliminate covert channels in, > although some randomisation can limit the bandwidth.Eliminating covert channels is completely infeasible. I don''t see any value in aiming for this. It''s not a useful security property in most circumstances. -- David Hopwood <david.nospam.hopwood@blueyonder.co.uk> ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> Information about other domains'' memory usage is leaked via the > hardware->physical mapping.OK, I was forgetting about the domain memory reservation hypercalls. It''s probably reasonable just to throw away ballooning functionality where this might be a problem. The main problem (as I see it) is going to be the network interface, whose performance depends on page-flipping. You can eliminate the security problem without hiding machine address if you copy incoming packets but that''s going to hurt performance :-(> > Timing related attacks are somewhat trickier to eliminate covert channels > > in, although some randomisation can limit the bandwidth. > > Eliminating covert channels is completely infeasible. I don''t see any > value in aiming for this. It''s not a useful security property in most > circumstances.I agree it''s not useful in the majority of circumstances. If it''s required it can be implemented at a later date but the returns for the amount of time invested are likely to be smaller. Cheers, Mark ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
xen-devel-admin@lists.sourceforge.net wrote on 01/20/2005 07:48:58 PM:> Mark Williamson wrote: > >>Also, I suppose you will wish to prevent covert channels between > >>domains, e.g. domains communicating using various timing attacks (Imove> >>the disk head to the other end of the disk if I wish to tell you > >>something), or by allocating/freeing certains parts of memory. > >> > >>How much will you need to dumb down the VMs view of what is going onin> >>the machine to achieve this (not expose real time information, not > >>expose real page tables), and how much of a VMM will there be leftwhen> >>you are done? > > > > Well domains are not aware of each other''s memory usage, so I wouldn''thave> > thought that allocation / exposing real page tables would matter.(Except> > dom0 can of course see everything if it wants). > > Information about other domains'' memory usage is leaked via the > hardware->physical mapping.There might be a problem of object-reuse and isolation. If memory isn''t cleared before it is given to another partition, then this is an error. If disk space isn''t cleared before it is reused by another partition, then this seems serious, too. Question: a) Does the balloon driver clean pages before releasing it to other partitions? (I guess not) b) Does the page-swap thas happens when a packet is received clear the pages? (I guess not) c) If a) or b) use un-cleaned pages: did anybody try to sniff data (passwords, keys) from another non-cooperating partitions out of re-used pages?> > Timing related attacks are somewhat trickier to eliminate covert > channels in, > > although some randomisation can limit the bandwidth. > > Eliminating covert channels is completely infeasible. I don''t see any > value in aiming for this. It''s not a useful security property in most > circumstances. > -- > David Hopwood <david.nospam.hopwood@blueyonder.co.uk>To eliminate ALL covert channels might not be in our interest, but to ignore all of them seems unnatural as well. Certainly, covert channels are a way to spend many discussions. Reiner
> Question: > a) Does the balloon driver clean pages before releasing it to > other partitions? (I guess not)Yes it does.> b) Does the page-swap thas happens when a packet is received > clear the pages? (I guess not)There is a pool of pages that are used for network IO. They''re not normally scrubbed when they get flipped, on the grounds that network packets could be sniffed on the wire anyhow. Pages are scrubbed before entering the pool. I guess we could have a config option that also zeroed the payload area of a packet when the skb is freed, for those that want belt and braces. [NB: one area we need to investigate is whether the kernel ever does decrypt in-place in the skb data area, and if so, throw the page out of the pool.] Ian ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Mark Williamson wrote:>>Information about other domains'' memory usage is leaked via the >>hardware->physical mapping. > > OK, I was forgetting about the domain memory reservation hypercalls. It''s > probably reasonable just to throw away ballooning functionality where this > might be a problem. > > The main problem (as I see it) is going to be the network interface, whose > performance depends on page-flipping. You can eliminate the security problem > without hiding machine address if you copy incoming packets but that''s going > to hurt performance :-( > >>>Timing related attacks are somewhat trickier to eliminate covert channels >>>in, although some randomisation can limit the bandwidth. >> >>Eliminating covert channels is completely infeasible. I don''t see any >>value in aiming for this. It''s not a useful security property in most >>circumstances. > > I agree it''s not useful in the majority of circumstances. If it''s required it > can be implemented at a later date but the returns for the amount of time > invested are likely to be smaller.It almost certainly can''t be implemented at a later date. Even attempting to do so (without really succeeding) would require significant incompatible changes to the hypervisor interface. The idea of limiting covert channels should have been abandoned when it became clear that it isn''t feasible without severely constraining the efficiency and functionality of an operating system. Unfortunately it is too interesting a problem, so a lot of effort has been essentially wasted in research into this area, without ever coming up with any way to limit the bandwidth to a useful extent. Attackers only need a very small bandwidth to transmit many of the things that are most useful from their point of view (cryptographic keys, passwords, compressed answers from a program that can look at any amount of data), so claims about limiting the bandwidth really just give a false sense of security. -- David Hopwood <david.nospam.hopwood@blueyonder.co.uk> ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> It almost certainly can''t be implemented at a later date. Even attempting > to do so (without really succeeding) would require significant incompatible > changes to the hypervisor interface.What changes are required depend on what channels you''re trying to eliminate. You could limit the aforementioned covert channels in the network interface, block device head scheduling and also CPU scheduling without changing the hypervisor interface at all. Whether this is worth the effort is another matter, however, as you rightly point out ;-)> Attackers only need a very small > bandwidth to transmit many of the things that are most useful from their > point of view (cryptographic keys, passwords, compressed answers from a > program that can look at any amount of data), so claims about limiting the > bandwidth really just give a false sense of security.Yes. You just have to hope that organisational security measures compensate for the covert channels that remain. Cheers, Mark ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Thu, Jan 20, 2005 at 10:41:06PM +0000, Mark Williamson wrote:> You might want to have two domU''s shared memory to implement a "direct > point-to-point" ethernet link, to avoid the overheads of going through dom0. > For most people this might not matter but it''s nice to have the option for > maximal performance.This should be possible by using the xen front & backend network drivers, right? The only thing right now is that i can''t see how to get the backend interface to be created anywhere other than dom0. I can set up a domain as a network interface backend ok, and i can create a domain with a vif pointing at that domain, but no new vif appears in the network backend dom. NB1: I''m on a slightly old build right now (2004/12/12 17:03:10 1.1655) so i don''t know if it''s still the case, but when providing a backend domain for a vif only a number is accepted. It''d be much nicer if a name could be used. NB2: Again, i don''t know if it''s still the case, but the network backend driver can''t be built without physical device access enabled, but there isn''t a dependency for that in the kernel config system. J -- Jody Belka knew (at) pimb (dot) org ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
>Mark Williamson wrote: >>>Also, I suppose you will wish to prevent covert channels between >>>domains, e.g. domains communicating using various timing attacks (I move >>>the disk head to the other end of the disk if I wish to tell you >>>something), or by allocating/freeing certains parts of memory. >>> >>>How much will you need to dumb down the VMs view of what is going on in >>>the machine to achieve this (not expose real time information, not >>>expose real page tables), and how much of a VMM will there be left when >>>you are done? >> >> Well domains are not aware of each other''s memory usage, so I wouldn''t have >> thought that allocation / exposing real page tables would matter. (Except >> dom0 can of course see everything if it wants). > >Information about other domains'' memory usage is leaked via the >hardware->physical mapping.Unprivileged domains cannot see each others hardware->physical mappings. If we use full shadow mode (either with or without VT-x), domains cannot even see their own hardware->physical mappings. Explicit page reuse (via either network page-flipping or balloon drivers) can be eliminated, albeit at a cost to performance. Networking can still be provided by using either multiple NICs or hardware virtualization in the NIC (somewhat like the arseNIC work we did a few years back). Allowing networking of course has other well-known risks (see below).>> Timing related attacks are somewhat trickier to eliminate covert channels in >> although some randomisation can limit the bandwidth. > >Eliminating covert channels is completely infeasible. I don''t see any >value in aiming for this. It''s not a useful security property in most >circumstances.Well in the current version of Xen, there are plenty of non-covert channels (like network connections and raw shared memory event channels) which probably make more sense to look at first :-) I think received wisdom is that it''s certainly very difficult (and very expensive in terms of time and, usually, performance) to eliminate all covert channels. It may also be impossible depending on requirements (e.g. if one allows networked connections to arbitrary third parties). The full virtualization support in VT may be useful though. cheers, S. ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> >>Mark Williamson wrote: >>>>Also, I suppose you will wish to prevent covert channels between >>>>domains, e.g. domains communicating using various timing attacks (I move >>>>the disk head to the other end of the disk if I wish to tell you >>>>something), or by allocating/freeing certains parts of memory. >>>> >>>>How much will you need to dumb down the VMs view of what is going on in >>>>the machine to achieve this (not expose real time information, not >>>>expose real page tables), and how much of a VMM will there be left when >>>>you are done? >>> >>> Well domains are not aware of each other''s memory usage, so I wouldn''t have>>> thought that allocation / exposing real page tables would matter. (Except >>> dom0 can of course see everything if it wants). >> >>Information about other domains'' memory usage is leaked via the >>hardware->physical mapping. > >Unprivileged domains cannot see each others hardware->physical mappings.Oops - I read this originally as ''virtual->physical''. The MPT does indeed currently allow domains to view each others hardware->physical mappings. Replacing the single MPT with per-domain viewable sparse mapping structures is possible though, albeit at a performance cost. Full virt with VT-x also eliminates this issue. cheers, S. ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> > You might want to have two domU''s shared memory to implement a "direct > > point-to-point" ethernet link, to avoid the overheads of going through > > dom0. For most people this might not matter but it''s nice to have the > > option for maximal performance. > > This should be possible by using the xen front & backend network drivers, > right? The only thing right now is that i can''t see how to get the backend > interface to be created anywhere other than dom0.Currently the backend driver will only start if it''s in dom0 or in a network backend domain. The main obstacles to doing this are: 1) the backend domain currently needs to be privileged in order to map the frontends memory, so this can''t work between an arbitrary pair of domUs (this is beginning to change as grant tables are integrated). 2) the tools don''t support this kind of configuration yet> NB2: Again, i don''t know if it''s still the case, but the network backend > driver can''t be built without physical device access enabled, but there > isn''t a dependency for that in the kernel config system.Since the current assumption is that the backend controls a physical interface, the backend is built iff the domain has physical device support. Might need to change in future... Cheers, Mark ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Replied to this earlier but I think my MUA might have gobbled it :-(> This should be possible by using the xen front & backend network drivers, > right?Yes but it needs a bit of extra code. 1) AFAIK, the backend currently needs to be privileged to map the frontend''s memory - this in beginning to change as the grant tables stuff is merged 2) the tools don''t support this kind of config yet> The only thing right now is that i can''t see how to get the backend > interface to be created anywhere other than dom0. I can set up a domain as > a network interface backend ok, and i can create a domain with a vif > pointing at that domain, but no new vif appears in the network backend dom.Are you specifying what backend the frontend should be connected to in the frontend''s config file? Are you specifying that the backend is a backend in its config file? This path through Xend hasn''t been tested thoroughly, since most people aren''t using this sort of configuration yet.> NB2: Again, i don''t know if it''s still the case, but the network backend > driver can''t be built without physical device access enabled, but there > isn''t a dependency for that in the kernel config system.Currently backends are built iff physical device access is enabled, since the assumption is that they will run in a privileged domain and virtualise a real device. Cheers, Mark ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Fri, Jan 21, 2005 at 11:22:20AM +0000, Mark A. Williamson wrote:> > The only thing right now is that i can''t see how to get the backend > > interface to be created anywhere other than dom0. I can set up a domain as > > a network interface backend ok, and i can create a domain with a vif > > pointing at that domain, but no new vif appears in the network backend dom. > > Are you specifying what backend the frontend should be connected to in the > frontend''s config file?Yes> Are you specifying that the backend is a backend in its config file?Yes> > NB2: Again, i don''t know if it''s still the case, but the network backend > > driver can''t be built without physical device access enabled, but there > > isn''t a dependency for that in the kernel config system. > > Currently backends are built iff physical device access is enabled, since the > assumption is that they will run in a privileged domain and virtualise a real > device.What i was meaning, was that the network backend can be chosen in the config without physical device access enabled, but the kernel won''t then build. J -- Jody Belka knew (at) pimb (dot) org ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> > Are you specifying what backend the frontend should be connected to in > > the frontend''s config file? > Yes > > Are you specifying that the backend is a backend in its config file? > YesJust checking ;-) This stuff hasn''t been tested much, so if you''ve set all the appropriate variables it could be a Xend bug. Has anybody had driver domains working with the current Xend? When?> What i was meaning, was that the network backend can be chosen in the > config without physical device access enabled, but the kernel won''t then > build.Oh I see, sorry. I guess it doesn''t work because the relevant macros for mapping other domains memory are not defined. I think a subsequent patch has fixed this - thanks for pointing it out. Cheers, Mark ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Sat, Jan 22, 2005 at 05:20:35PM +0000, Mark Williamson wrote:> > > Are you specifying what backend the frontend should be connected to in > > > the frontend''s config file? > > Yes > > > Are you specifying that the backend is a backend in its config file? > > Yes > > Just checking ;-) This stuff hasn''t been tested much, so if you''ve set all > the appropriate variables it could be a Xend bug.That''s fine. Yeah, i marked the backend domain as a netif domain, started it up, inserted the domain ID into the config for the frontend domain (hassle, but necessary as the current code only takes an ID, and not a name), and started up the frontend domain. The additional interface showed up in the frontend domain, but nothing appeared in the backend. J -- Jody Belka knew (at) pimb (dot) org ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Mark Williamson
2005-Jan-23 17:52 UTC
Re: Multiple netif device channels (was Fw: [Xen-devel] Xen on /. again)
> That''s fine. Yeah, i marked the backend domain as a netif domain, started > it up, inserted the domain ID into the config for the frontend domain > (hassle, but necessary as the current code only takes an ID, and not a > name), and started up the frontend domain. The additional interface showed > up in the frontend domain, but nothing appeared in the backend.Remind me: is the frontend having netifs attached to two different backends? What happens if you try to attach both netifs to the driver domain (not dom0)? Hmmm, that''s strange. I''m surprised the additional interface showed up in one but not the other but I''m not familiar with the support for multiple netifs. Mike: any idea what might cause this behaviour? Is this known not to work? Do you have any experience in the Xend guts? You could try turning on tracing at the top of messages.py and observe exactly what control interface messages are being sent. This is likely to be quite tedious but may allow you to infer what information about netifs is flowing between the domains. You may also want to instrument the probing code in netfront.c and netback.c. Sorry not to be more helpful, Mark ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Jody Belka
2005-Jan-25 00:32 UTC
Re: Multiple netif device channels (was Fw: [Xen-devel] Xen on /. again)
On Sun, Jan 23, 2005 at 05:52:23PM +0000, Mark Williamson wrote:> > That''s fine. Yeah, i marked the backend domain as a netif domain, started > > it up, inserted the domain ID into the config for the frontend domain > > (hassle, but necessary as the current code only takes an ID, and not a > > name), and started up the frontend domain. The additional interface showed > > up in the frontend domain, but nothing appeared in the backend. > > Remind me: is the frontend having netifs attached to two different backends? > What happens if you try to attach both netifs to the driver domain (not > dom0)?Attempted layout is as follows: Dom0 - single physical network adaptor netif backend DomU1 - no physical network adaptor netif backend netif frontend -> Dom0 DomU2 - no physical network adaptor netif frontend -> Dom0 netif frontend -> DomU1 Interfaces for all the frontends appear inside the relevant domains, but the only the 2 dom0 backend vifs appear; nothing shows up inside domU1. I''ll try connecting DomU2 only to DomU1 wrt networking when i next get a chance.> Do you have any experience in the Xend guts? You could try turning on tracing > at the top of messages.py and observe exactly what control interface messages > are being sent. This is likely to be quite tedious but may allow you to > infer what information about netifs is flowing between the domains. You may > also want to instrument the probing code in netfront.c and netback.c.A little bit. i''ll try and see if i can see what''s happening with it. At a minimum it''d give me a trace i can post that might help someone else figure out what''s going on.> Sorry not to be more helpful,np, J -- Jody Belka knew (at) pimb (dot) org ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Mark Williamson
2005-Jan-25 13:52 UTC
Re: Multiple netif device channels (was Fw: [Xen-devel] Xen on /. again)
> Attempted layout is as follows: > > Dom0 - single physical network adaptor > netif backendFine.> DomU1 - no physical network adaptor > netif backend > netif frontend -> Dom0Ah, this would be a problem in any case. Backend drivers need to run in privileged domains. This basically means that they have access to a PCI device. You could try the workaround Andy suggested a few days ago (grant access to a PCI device but don''t compile the driver into the kernel so it leaves it alone).> DomU2 - no physical network adaptor > netif frontend -> Dom0 > netif frontend -> DomU1Fine.> A little bit. i''ll try and see if i can see what''s happening with it. At a > minimum it''d give me a trace i can post that might help someone else figure > out what''s going on.Yup. But try getting privileges to work properly in DomU1 first. HTH, Mark> > Sorry not to be more helpful, > > np, > > > J------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Sun, Jan 23, 2005 at 05:52:23PM +0000, Mark Williamson wrote:> Hmmm, that''s strange. I''m surprised the additional interface showed up in one > but not the other but I''m not familiar with the support for multiple netifs.Ok, an update on the matter. The reason no interface appeared in the domU netif backend was because xend wasn''t actually configuring the domain as a backend. The reason being that xend was setting the backend configuration flags /after/ constructing the domain image [1], so nothing was happening. With that changed, a vif appears in the netif backend when the other domU is started up. However, no actual network connection can be made between the domains; ping just responds back with a destination host unreachable error. Although that is better than before i upgraded to the latest -testing (which i thought i should do before proceeding further); before the upgrade any traffic from the netif domain to the vif caused a panic in said domain. Seperately, although still connected to networking, the netfront driver has code in it that prevents it from loading in a backend domain as well as the initial domain. That should probably be reduced down to just checking for the initial domain. And lastly, on a completely seperate xen matter, i seem to recall some people mentioning recently about not being able to add/remove vbd''s in running domains. I had a look at the code, and i''d guess that (like me lately) these people are running 2.6 in the domU''s. The code to update vbd devices is currently commented out in the 2.6 sparse tree, as it hasn''t been ported to the 2.6 blkfront driver yet (the code in place is a pure copy from the 2.4 driver). J [1] XendDomainInfo.py / XendDomainInfo.construct() - method starts @ line 427 The configure_backends() line needs to be before the construct_image() one. The configure_restart() line doesn''t really matter, as that''s purely a xend thing. -- Jody Belka knew (at) pimb (dot) org ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> Ok, an update on the matter. The reason no interface appeared in the domU > netif backend was because xend wasn''t actually configuring the domain as a > backend. The reason being that xend was setting the backend configuration > flags /after/ constructing the domain image [1], so nothing was happening.Ah. Do you have a patch for this we could apply?> With that changed, a vif appears in the netif backend when the other domU > is started up. However, no actual network connection can be made between > the domains; ping just responds back with a destination host unreachable > error.Have you tried the hack Andy suggested (giving the backend domain access to a PCI device it has no drivers for) so that the backend domain will be privileged? If you don''t elevate its privilege level some how it will definitely not be able to function as a backend.> Seperately, although still connected to networking, the netfront driver has > code in it that prevents it from loading in a backend domain as well as the > initial domain. That should probably be reduced down to just checking for > the initial domain.Yes, this can probably be safely removed now. Does removing the backend check work for you?> And lastly, on a completely seperate xen matter, i seem to recall some > people mentioning recently about not being able to add/remove vbd''s in > running domains. I had a look at the code, and i''d guess that (like me > lately) these people are running 2.6 in the domU''s. The code to update vbd > devices is currently commented out in the 2.6 sparse tree, as it hasn''t > been ported to the 2.6 blkfront driver yet (the code in place is a pure > copy from the 2.4 driver).Right, it''s probably a reasonably simple update if somebody takes the time to trawl through and figure out what needs updating. Thanks for working with us on this. Cheers, Mark ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Wed, Jan 26, 2005 at 08:36:34PM +0000, Mark Williamson wrote:> > Ok, an update on the matter. The reason no interface appeared in the domU > > netif backend was because xend wasn''t actually configuring the domain as a > > backend. The reason being that xend was setting the backend configuration > > flags /after/ constructing the domain image [1], so nothing was happening. > > Ah. Do you have a patch for this we could apply?Yep, patch attached. Also includes a change to make specifying the backend domain more user friendly.> > With that changed, a vif appears in the netif backend when the other domU > > is started up. However, no actual network connection can be made between > > the domains; ping just responds back with a destination host unreachable > > error. > > Have you tried the hack Andy suggested (giving the backend domain access to a > PCI device it has no drivers for) so that the backend domain will be > privileged? If you don''t elevate its privilege level some how it will > definitely not be able to function as a backend.Yay. Tried that (pointed the backend domain to the unused onboard audio controller :) and it worked perfectly. One thing though; all backend interfaces have the same mac address, FE:FF:FF:FF:FF:FF. I see that Keir committed this change, along with a note about problems with bridging with the previous code. If i modify the code such that i can specify an explicit backend vif mac address to xend, and then that gets passed into the backend (with fe:ff:ff:ff:ff:ff being sent if no explicit one is given), what would be the best way to go about avoiding the problems mentioned when doing so? In case you''re wondering why i need this change, it''s because i need some way to distingush the vif''s in the backend domain, so that i can configure them easily with the linux hotplug system (since, obviously, xend is neither running in the domain, nor can the dom0 instance configure the interface)> > Seperately, although still connected to networking, the netfront driver has > > code in it that prevents it from loading in a backend domain as well as the > > initial domain. That should probably be reduced down to just checking for > > the initial domain. > > Yes, this can probably be safely removed now. Does removing the backend check > work for you?Seems to work fine. Tried it out after i finally got the inter-domU networking working. No problems appeared with front only, back only or front-and-back configurations. Patch attached for this as well.> Thanks for working with us on this.No probs. Had a good time working through the code to figure out what was going on. J -- Jody Belka knew (at) pimb (dot) org
Realised i forgot proper comments and the sign-offs. Patch: tools.patch Comment: Moves backend configuration to before construction of domain image, so that the backend drivers actually get activated in driver domains. Also allows domain names as well as IDs when specifying backend domains for vbd and disk configuration items. Signed-off-by: knew-xen@pimb.org Patch: netfront.patch Comment: Remove the check preventing the netfront driver from starting up in network backend domains. Signed-off-by: knew-xen@pimb.org J -- Jody Belka knew (at) pimb (dot) org ------------------------------------------------------- This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting Tool for open source databases. Create drag-&-drop reports. Save time by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc. Download a FREE copy at http://www.intelliview.com/go/osdn_nl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel