Currently we assign devices with VT-d in Xend, this raises two issues: 1) assign devices regardless of they are hidden by pciback or not. If the device is not hidden, it results in the device doesn''t work in Dom0; 2) device is assigned one by one, if assign multiple devices, some devices may have been assigned when problem happens, it results in assigned devices don''t work in Dom0. I think Xend is not a good place to assign devices. This patch adds a parameter to xc_assign_device(), let it just do check in Xend whether the devices can be assigned or not, and move real device assignment to qemu. Signed-off-by: Weidong Han <weidong.han@intel.com> _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Muli Ben-Yehuda
2007-Dec-10 08:01 UTC
Re: [Xen-devel] [VTD][PATCH] Change xc_assign_device()
On Mon, Dec 10, 2007 at 02:21:55PM +0800, Han, Weidong wrote:> Currently we assign devices with VT-d in Xend, this raises two issues: > 1) assign devices regardless of they are hidden by pciback or not. If > the device is not hidden, it results in the device doesn''t work in Dom0; > 2) device is assigned one by one, if assign multiple devices, some > devices may have been assigned when problem happens, it results in > assigned devices don''t work in Dom0. I think Xend is not a good place to > assign devices. This patch adds a parameter to xc_assign_device(), let > it just do check in Xend whether the devices can be assigned or not, and > move real device assignment to qemu.Why is qemu a better place to assign the devices? How does moving it to qemu solve the two issues mentioned above? Cheers, Muli _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Xend is too early to do real assignment, do some checking is better. These two issues will be found by the checks in Xend ( 1) issue will be found by pciback). After passing these checks in Xend, it''s suitable to do real device assignment and memory and ioport mappings in qemu. Randy (Weidong) Muli Ben-Yehuda wrote:> On Mon, Dec 10, 2007 at 02:21:55PM +0800, Han, Weidong wrote: > >> Currently we assign devices with VT-d in Xend, this raises two >> issues: 1) assign devices regardless of they are hidden by pciback >> or not. If the device is not hidden, it results in the device >> doesn''t work in Dom0; 2) device is assigned one by one, if assign >> multiple devices, some devices may have been assigned when problem >> happens, it results in assigned devices don''t work in Dom0. I think >> Xend is not a good place to assign devices. This patch adds a >> parameter to xc_assign_device(), let it just do check in Xend >> whether the devices can be assigned or not, and move real device >> assignment to qemu. > > Why is qemu a better place to assign the devices? How does moving it > to qemu solve the two issues mentioned above? > > Cheers, > Muli_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Muli Ben-Yehuda
2007-Dec-10 08:23 UTC
Re: [Xen-devel] [VTD][PATCH] Change xc_assign_device()
On Mon, Dec 10, 2007 at 04:13:45PM +0800, Han, Weidong wrote:> Xend is too early to do real assignment, do some checking is better. > These two issues will be found by the checks in Xend ( 1) issue will > be found by pciback). After passing these checks in Xend, it''s > suitable to do real device assignment and memory and ioport mappings > in qemu.Considering Xend already does the checks, wouldn''t it be better to just move the assignments to after the checks are done, but still keep them in Xend? Cheers, Muli _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Sounds to me like qemu-dm is too late to do those other checks or you need better error handling in qemu-dm. Really you have a choice: do all tests and assignment in qemu-dm, or do all tests and assignment in xend. Doing half-and-half is stupid. If the pass-through can still fail even after the check you leave in xend, why check at all in xend? Probably you need to fail the domain create, or at least shutdown the domain, when a device that should be passed through cannot be passed through. That will naturally cause the device assignment to be torn down (if it was set set up), as the domain is destroyed. Whether this is all actioned from xend or from qemu-dm doesn''t much matter, but the split responsibility isn''t clean. Probably xend is better just because the error handling is easier and earlier there. -- Keir On 10/12/07 08:13, "Han, Weidong" <weidong.han@intel.com> wrote:> Xend is too early to do real assignment, do some checking is better. > These two issues will be found by the checks in Xend ( 1) issue will be > found by pciback). After passing these checks in Xend, it''s suitable to > do real device assignment and memory and ioport mappings in qemu. > > Randy (Weidong) > > Muli Ben-Yehuda wrote: >> On Mon, Dec 10, 2007 at 02:21:55PM +0800, Han, Weidong wrote: >> >>> Currently we assign devices with VT-d in Xend, this raises two >>> issues: 1) assign devices regardless of they are hidden by pciback >>> or not. If the device is not hidden, it results in the device >>> doesn''t work in Dom0; 2) device is assigned one by one, if assign >>> multiple devices, some devices may have been assigned when problem >>> happens, it results in assigned devices don''t work in Dom0. I think >>> Xend is not a good place to assign devices. This patch adds a >>> parameter to xc_assign_device(), let it just do check in Xend >>> whether the devices can be assigned or not, and move real device >>> assignment to qemu. >> >> Why is qemu a better place to assign the devices? How does moving it >> to qemu solve the two issues mentioned above? >> >> Cheers, >> Muli > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Muli Ben-Yehuda wrote:> On Mon, Dec 10, 2007 at 04:13:45PM +0800, Han, Weidong wrote: > >> Xend is too early to do real assignment, do some checking is better. >> These two issues will be found by the checks in Xend ( 1) issue will >> be found by pciback). After passing these checks in Xend, it''s >> suitable to do real device assignment and memory and ioport mappings >> in qemu. > > Considering Xend already does the checks, wouldn''t it be better to > just move the assignments to after the checks are done, but still keep > them in Xend? >The checks added for VT-d in Xend doesn''t check whether it''s hidden or not, which is checked by pciback (xend/server/pciif.py). Randy (Weidong) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
I agree the early do those checks the better. But without mapping memory and I/O port in qemu, the assigned device can''t be really used. That''s to say the assignment is not completed in a way. In addition, if we want to support hotplug devices, it''s not suitable to do assignment in Xend. Randy (Weidong) Keir Fraser wrote:> Sounds to me like qemu-dm is too late to do those other checks or you > need better error handling in qemu-dm. Really you have a choice: do > all tests and assignment in qemu-dm, or do all tests and assignment > in xend. Doing half-and-half is stupid. If the pass-through can still > fail even after the check you leave in xend, why check at all in xend? > > Probably you need to fail the domain create, or at least shutdown the > domain, when a device that should be passed through cannot be passed > through. That will naturally cause the device assignment to be torn > down (if it was set set up), as the domain is destroyed. Whether this > is all actioned from xend or from qemu-dm doesn''t much matter, but > the split responsibility isn''t clean. Probably xend is better just > because the error handling is easier and earlier there. > > -- Keir > > On 10/12/07 08:13, "Han, Weidong" <weidong.han@intel.com> wrote: > >> Xend is too early to do real assignment, do some checking is better. >> These two issues will be found by the checks in Xend ( 1) issue will >> be found by pciback). After passing these checks in Xend, it''s >> suitable to do real device assignment and memory and ioport mappings >> in qemu. >> >> Randy (Weidong) >> >> Muli Ben-Yehuda wrote: >>> On Mon, Dec 10, 2007 at 02:21:55PM +0800, Han, Weidong wrote: >>> >>>> Currently we assign devices with VT-d in Xend, this raises two >>>> issues: 1) assign devices regardless of they are hidden by pciback >>>> or not. If the device is not hidden, it results in the device >>>> doesn''t work in Dom0; 2) device is assigned one by one, if assign >>>> multiple devices, some devices may have been assigned when problem >>>> happens, it results in assigned devices don''t work in Dom0. I think >>>> Xend is not a good place to assign devices. This patch adds a >>>> parameter to xc_assign_device(), let it just do check in Xend >>>> whether the devices can be assigned or not, and move real device >>>> assignment to qemu. >>> >>> Why is qemu a better place to assign the devices? How does moving it >>> to qemu solve the two issues mentioned above? >>> >>> Cheers, >>> Muli >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Then the whole lot should be done in qemu-dm and it sounds like the error handling needs to be improved. -- Keir On 10/12/07 08:45, "Han, Weidong" <weidong.han@intel.com> wrote:> I agree the early do those checks the better. But without mapping memory > and I/O port in qemu, the assigned device can''t be really used. That''s > to say the assignment is not completed in a way. In addition, if we want > to support hotplug devices, it''s not suitable to do assignment in Xend. > > Randy (Weidong) > > Keir Fraser wrote: >> Sounds to me like qemu-dm is too late to do those other checks or you >> need better error handling in qemu-dm. Really you have a choice: do >> all tests and assignment in qemu-dm, or do all tests and assignment >> in xend. Doing half-and-half is stupid. If the pass-through can still >> fail even after the check you leave in xend, why check at all in xend? >> >> Probably you need to fail the domain create, or at least shutdown the >> domain, when a device that should be passed through cannot be passed >> through. That will naturally cause the device assignment to be torn >> down (if it was set set up), as the domain is destroyed. Whether this >> is all actioned from xend or from qemu-dm doesn''t much matter, but >> the split responsibility isn''t clean. Probably xend is better just >> because the error handling is easier and earlier there. >> >> -- Keir >> >> On 10/12/07 08:13, "Han, Weidong" <weidong.han@intel.com> wrote: >> >>> Xend is too early to do real assignment, do some checking is better. >>> These two issues will be found by the checks in Xend ( 1) issue will >>> be found by pciback). After passing these checks in Xend, it''s >>> suitable to do real device assignment and memory and ioport mappings >>> in qemu. >>> >>> Randy (Weidong) >>> >>> Muli Ben-Yehuda wrote: >>>> On Mon, Dec 10, 2007 at 02:21:55PM +0800, Han, Weidong wrote: >>>> >>>>> Currently we assign devices with VT-d in Xend, this raises two >>>>> issues: 1) assign devices regardless of they are hidden by pciback >>>>> or not. If the device is not hidden, it results in the device >>>>> doesn''t work in Dom0; 2) device is assigned one by one, if assign >>>>> multiple devices, some devices may have been assigned when problem >>>>> happens, it results in assigned devices don''t work in Dom0. I think >>>>> Xend is not a good place to assign devices. This patch adds a >>>>> parameter to xc_assign_device(), let it just do check in Xend >>>>> whether the devices can be assigned or not, and move real device >>>>> assignment to qemu. >>>> >>>> Why is qemu a better place to assign the devices? How does moving it >>>> to qemu solve the two issues mentioned above? >>>> >>>> Cheers, >>>> Muli >>> >>> _______________________________________________ >>> Xen-devel mailing list >>> Xen-devel@lists.xensource.com >>> http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
The checks in Xend just check high level errors, such as VT-d disabled, device is not exist, or not hidden. When find these errors, quit domain creation immediately. The real actions are taken in qemu. What error handling do you mean? Randy (Weidong) Keir Fraser wrote:> Then the whole lot should be done in qemu-dm and it sounds like the > error handling needs to be improved. > > -- Keir > > On 10/12/07 08:45, "Han, Weidong" <weidong.han@intel.com> wrote: > >> I agree the early do those checks the better. But without mapping >> memory and I/O port in qemu, the assigned device can''t be really >> used. That''s to say the assignment is not completed in a way. In >> addition, if we want to support hotplug devices, it''s not suitable >> to do assignment in Xend. >> >> Randy (Weidong) >> >> Keir Fraser wrote: >>> Sounds to me like qemu-dm is too late to do those other checks or >>> you need better error handling in qemu-dm. Really you have a >>> choice: do all tests and assignment in qemu-dm, or do all tests and >>> assignment in xend. Doing half-and-half is stupid. If the >>> pass-through can still fail even after the check you leave in xend, >>> why check at all in xend? >>> >>> Probably you need to fail the domain create, or at least shutdown >>> the domain, when a device that should be passed through cannot be >>> passed through. That will naturally cause the device assignment to >>> be torn down (if it was set set up), as the domain is destroyed. >>> Whether this is all actioned from xend or from qemu-dm doesn''t much >>> matter, but the split responsibility isn''t clean. Probably xend is >>> better just because the error handling is easier and earlier there. >>> >>> -- Keir >>> >>> On 10/12/07 08:13, "Han, Weidong" <weidong.han@intel.com> wrote: >>> >>>> Xend is too early to do real assignment, do some checking is >>>> better. These two issues will be found by the checks in Xend ( 1) >>>> issue will be found by pciback). After passing these checks in >>>> Xend, it''s suitable to do real device assignment and memory and >>>> ioport mappings in qemu. >>>> >>>> Randy (Weidong) >>>> >>>> Muli Ben-Yehuda wrote: >>>>> On Mon, Dec 10, 2007 at 02:21:55PM +0800, Han, Weidong wrote: >>>>> >>>>>> Currently we assign devices with VT-d in Xend, this raises two >>>>>> issues: 1) assign devices regardless of they are hidden by >>>>>> pciback or not. If the device is not hidden, it results in the >>>>>> device doesn''t work in Dom0; 2) device is assigned one by one, >>>>>> if assign multiple devices, some devices may have been assigned >>>>>> when problem happens, it results in assigned devices don''t work >>>>>> in Dom0. I think Xend is not a good place to assign devices. >>>>>> This patch adds a parameter to xc_assign_device(), let it just >>>>>> do check in Xend whether the devices can be assigned or not, and >>>>>> move real device assignment to qemu. >>>>> >>>>> Why is qemu a better place to assign the devices? How does moving >>>>> it to qemu solve the two issues mentioned above? >>>>> >>>>> Cheers, >>>>> Muli >>>> >>>> _______________________________________________ >>>> Xen-devel mailing list >>>> Xen-devel@lists.xensource.com >>>> http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Mon, Dec 10, 2007 at 09:00:49AM +0000, Keir Fraser wrote:> Then the whole lot should be done in qemu-dm and it sounds like the error > handling needs to be improved.Can we figure out a better way for communication between qemu-dm and Xend? After creation, qemu-dm is detached from Xend without any tools to notify Xend. Maybe, Xend can poll the xenstore for qemu-dm device assignment success?> > -- Keir > > On 10/12/07 08:45, "Han, Weidong" <weidong.han@intel.com> wrote: > > > I agree the early do those checks the better. But without mapping memory > > and I/O port in qemu, the assigned device can''t be really used. That''s > > to say the assignment is not completed in a way. In addition, if we want > > to support hotplug devices, it''s not suitable to do assignment in Xend. > > > > Randy (Weidong) > > > > Keir Fraser wrote: > >> Sounds to me like qemu-dm is too late to do those other checks or you > >> need better error handling in qemu-dm. Really you have a choice: do > >> all tests and assignment in qemu-dm, or do all tests and assignment > >> in xend. Doing half-and-half is stupid. If the pass-through can still > >> fail even after the check you leave in xend, why check at all in xend? > >> > >> Probably you need to fail the domain create, or at least shutdown the > >> domain, when a device that should be passed through cannot be passed > >> through. That will naturally cause the device assignment to be torn > >> down (if it was set set up), as the domain is destroyed. Whether this > >> is all actioned from xend or from qemu-dm doesn''t much matter, but > >> the split responsibility isn''t clean. Probably xend is better just > >> because the error handling is easier and earlier there. > >> > >> -- Keir > >> > >> On 10/12/07 08:13, "Han, Weidong" <weidong.han@intel.com> wrote: > >> > >>> Xend is too early to do real assignment, do some checking is better. > >>> These two issues will be found by the checks in Xend ( 1) issue will > >>> be found by pciback). After passing these checks in Xend, it''s > >>> suitable to do real device assignment and memory and ioport mappings > >>> in qemu. > >>> > >>> Randy (Weidong) > >>> > >>> Muli Ben-Yehuda wrote: > >>>> On Mon, Dec 10, 2007 at 02:21:55PM +0800, Han, Weidong wrote: > >>>> > >>>>> Currently we assign devices with VT-d in Xend, this raises two > >>>>> issues: 1) assign devices regardless of they are hidden by pciback > >>>>> or not. If the device is not hidden, it results in the device > >>>>> doesn''t work in Dom0; 2) device is assigned one by one, if assign > >>>>> multiple devices, some devices may have been assigned when problem > >>>>> happens, it results in assigned devices don''t work in Dom0. I think > >>>>> Xend is not a good place to assign devices. This patch adds a > >>>>> parameter to xc_assign_device(), let it just do check in Xend > >>>>> whether the devices can be assigned or not, and move real device > >>>>> assignment to qemu. > >>>> > >>>> Why is qemu a better place to assign the devices? How does moving it > >>>> to qemu solve the two issues mentioned above? > >>>> > >>>> Cheers, > >>>> Muli > >>> > >>> _______________________________________________ > >>> Xen-devel mailing list > >>> Xen-devel@lists.xensource.com > >>> http://lists.xensource.com/xen-devel > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >-- best rgds, edwin _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
The reason for moving device assignment to qemu-dm is that the assignment can fail even after the assignment has happened. I''m happy to move the assignment to qemu-dm, but then the ''check'' in xend gets stripped out. -- Keir On 10/12/07 09:01, "Han, Weidong" <weidong.han@intel.com> wrote:> The checks in Xend just check high level errors, such as VT-d disabled, > device is not exist, or not hidden. When find these errors, quit domain > creation immediately. The real actions are taken in qemu. What error > handling do you mean? > > Randy (Weidong) > > Keir Fraser wrote: >> Then the whole lot should be done in qemu-dm and it sounds like the >> error handling needs to be improved. >> >> -- Keir >> >> On 10/12/07 08:45, "Han, Weidong" <weidong.han@intel.com> wrote: >> >>> I agree the early do those checks the better. But without mapping >>> memory and I/O port in qemu, the assigned device can''t be really >>> used. That''s to say the assignment is not completed in a way. In >>> addition, if we want to support hotplug devices, it''s not suitable >>> to do assignment in Xend. >>> >>> Randy (Weidong) >>> >>> Keir Fraser wrote: >>>> Sounds to me like qemu-dm is too late to do those other checks or >>>> you need better error handling in qemu-dm. Really you have a >>>> choice: do all tests and assignment in qemu-dm, or do all tests and >>>> assignment in xend. Doing half-and-half is stupid. If the >>>> pass-through can still fail even after the check you leave in xend, >>>> why check at all in xend? >>>> >>>> Probably you need to fail the domain create, or at least shutdown >>>> the domain, when a device that should be passed through cannot be >>>> passed through. That will naturally cause the device assignment to >>>> be torn down (if it was set set up), as the domain is destroyed. >>>> Whether this is all actioned from xend or from qemu-dm doesn''t much >>>> matter, but the split responsibility isn''t clean. Probably xend is >>>> better just because the error handling is easier and earlier there. >>>> >>>> -- Keir >>>> >>>> On 10/12/07 08:13, "Han, Weidong" <weidong.han@intel.com> wrote: >>>> >>>>> Xend is too early to do real assignment, do some checking is >>>>> better. These two issues will be found by the checks in Xend ( 1) >>>>> issue will be found by pciback). After passing these checks in >>>>> Xend, it''s suitable to do real device assignment and memory and >>>>> ioport mappings in qemu. >>>>> >>>>> Randy (Weidong) >>>>> >>>>> Muli Ben-Yehuda wrote: >>>>>> On Mon, Dec 10, 2007 at 02:21:55PM +0800, Han, Weidong wrote: >>>>>> >>>>>>> Currently we assign devices with VT-d in Xend, this raises two >>>>>>> issues: 1) assign devices regardless of they are hidden by >>>>>>> pciback or not. If the device is not hidden, it results in the >>>>>>> device doesn''t work in Dom0; 2) device is assigned one by one, >>>>>>> if assign multiple devices, some devices may have been assigned >>>>>>> when problem happens, it results in assigned devices don''t work >>>>>>> in Dom0. I think Xend is not a good place to assign devices. >>>>>>> This patch adds a parameter to xc_assign_device(), let it just >>>>>>> do check in Xend whether the devices can be assigned or not, and >>>>>>> move real device assignment to qemu. >>>>>> >>>>>> Why is qemu a better place to assign the devices? How does moving >>>>>> it to qemu solve the two issues mentioned above? >>>>>> >>>>>> Cheers, >>>>>> Muli >>>>> >>>>> _______________________________________________ >>>>> Xen-devel mailing list >>>>> Xen-devel@lists.xensource.com >>>>> http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 10/12/07 09:52, "Zhai, Edwin" <edwin.zhai@intel.com> wrote:> On Mon, Dec 10, 2007 at 09:00:49AM +0000, Keir Fraser wrote: >> Then the whole lot should be done in qemu-dm and it sounds like the error >> handling needs to be improved. > > Can we figure out a better way for communication between qemu-dm and Xend? > After creation, qemu-dm is detached from Xend without any tools to notify > Xend. > > Maybe, Xend can poll the xenstore for qemu-dm device assignment success?Writing an error node in xenstore would be good. Apart from that, qemu-dm shutting down the domain should provide notification to Xen that something''s gone wrong. The main problem is that in a command-line environment there''s not really a way to notify the user or administrator. Then again, that''s what GUIs are for (or some other higher-level wrapper over xm and xend -- there are plenty). -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
The assignment is in qemu-dm originally, we moved it to xend for user friendly, because it can prompt user on screen. If strip it out, it is not easy for end user to find failure reason. Randy (Weidong) Keir Fraser wrote:> The reason for moving device assignment to qemu-dm is that the > assignment can fail even after the assignment has happened. I''m happy > to move the assignment to qemu-dm, but then the ''check'' in xend gets > stripped out. > > -- Keir > > On 10/12/07 09:01, "Han, Weidong" <weidong.han@intel.com> wrote: > >> The checks in Xend just check high level errors, such as VT-d >> disabled, device is not exist, or not hidden. When find these >> errors, quit domain creation immediately. The real actions are taken >> in qemu. What error handling do you mean? >> >> Randy (Weidong) >> >> Keir Fraser wrote: >>> Then the whole lot should be done in qemu-dm and it sounds like the >>> error handling needs to be improved. >>> >>> -- Keir >>> >>> On 10/12/07 08:45, "Han, Weidong" <weidong.han@intel.com> wrote: >>> >>>> I agree the early do those checks the better. But without mapping >>>> memory and I/O port in qemu, the assigned device can''t be really >>>> used. That''s to say the assignment is not completed in a way. In >>>> addition, if we want to support hotplug devices, it''s not suitable >>>> to do assignment in Xend. >>>> >>>> Randy (Weidong) >>>> >>>> Keir Fraser wrote: >>>>> Sounds to me like qemu-dm is too late to do those other checks or >>>>> you need better error handling in qemu-dm. Really you have a >>>>> choice: do all tests and assignment in qemu-dm, or do all tests >>>>> and assignment in xend. Doing half-and-half is stupid. If the >>>>> pass-through can still fail even after the check you leave in >>>>> xend, why check at all in xend? >>>>> >>>>> Probably you need to fail the domain create, or at least shutdown >>>>> the domain, when a device that should be passed through cannot be >>>>> passed through. That will naturally cause the device assignment to >>>>> be torn down (if it was set set up), as the domain is destroyed. >>>>> Whether this is all actioned from xend or from qemu-dm doesn''t >>>>> much matter, but the split responsibility isn''t clean. Probably >>>>> xend is better just because the error handling is easier and >>>>> earlier there. >>>>> >>>>> -- Keir >>>>> >>>>> On 10/12/07 08:13, "Han, Weidong" <weidong.han@intel.com> wrote: >>>>> >>>>>> Xend is too early to do real assignment, do some checking is >>>>>> better. These two issues will be found by the checks in Xend ( 1) >>>>>> issue will be found by pciback). After passing these checks in >>>>>> Xend, it''s suitable to do real device assignment and memory and >>>>>> ioport mappings in qemu. >>>>>> >>>>>> Randy (Weidong) >>>>>> >>>>>> Muli Ben-Yehuda wrote: >>>>>>> On Mon, Dec 10, 2007 at 02:21:55PM +0800, Han, Weidong wrote: >>>>>>> >>>>>>>> Currently we assign devices with VT-d in Xend, this raises two >>>>>>>> issues: 1) assign devices regardless of they are hidden by >>>>>>>> pciback or not. If the device is not hidden, it results in the >>>>>>>> device doesn''t work in Dom0; 2) device is assigned one by one, >>>>>>>> if assign multiple devices, some devices may have been assigned >>>>>>>> when problem happens, it results in assigned devices don''t work >>>>>>>> in Dom0. I think Xend is not a good place to assign devices. >>>>>>>> This patch adds a parameter to xc_assign_device(), let it just >>>>>>>> do check in Xend whether the devices can be assigned or not, >>>>>>>> and move real device assignment to qemu. >>>>>>> >>>>>>> Why is qemu a better place to assign the devices? How does >>>>>>> moving it to qemu solve the two issues mentioned above? >>>>>>> >>>>>>> Cheers, >>>>>>> Muli >>>>>> >>>>>> _______________________________________________ >>>>>> Xen-devel mailing list >>>>>> Xen-devel@lists.xensource.com >>>>>> http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
What if the device has not been hidden by pciback (a more likely failure mode, I should think)? -- Keir On 10/12/07 11:41, "Han, Weidong" <weidong.han@intel.com> wrote:> The assignment is in qemu-dm originally, we moved it to xend for user > friendly, because it can prompt user on screen. If strip it out, it is > not easy for end user to find failure reason. > > Randy (Weidong) > > Keir Fraser wrote: >> The reason for moving device assignment to qemu-dm is that the >> assignment can fail even after the assignment has happened. I''m happy >> to move the assignment to qemu-dm, but then the ''check'' in xend gets >> stripped out. >> >> -- Keir >> >> On 10/12/07 09:01, "Han, Weidong" <weidong.han@intel.com> wrote: >> >>> The checks in Xend just check high level errors, such as VT-d >>> disabled, device is not exist, or not hidden. When find these >>> errors, quit domain creation immediately. The real actions are taken >>> in qemu. What error handling do you mean? >>> >>> Randy (Weidong) >>> >>> Keir Fraser wrote: >>>> Then the whole lot should be done in qemu-dm and it sounds like the >>>> error handling needs to be improved. >>>> >>>> -- Keir >>>> >>>> On 10/12/07 08:45, "Han, Weidong" <weidong.han@intel.com> wrote: >>>> >>>>> I agree the early do those checks the better. But without mapping >>>>> memory and I/O port in qemu, the assigned device can''t be really >>>>> used. That''s to say the assignment is not completed in a way. In >>>>> addition, if we want to support hotplug devices, it''s not suitable >>>>> to do assignment in Xend. >>>>> >>>>> Randy (Weidong) >>>>> >>>>> Keir Fraser wrote: >>>>>> Sounds to me like qemu-dm is too late to do those other checks or >>>>>> you need better error handling in qemu-dm. Really you have a >>>>>> choice: do all tests and assignment in qemu-dm, or do all tests >>>>>> and assignment in xend. Doing half-and-half is stupid. If the >>>>>> pass-through can still fail even after the check you leave in >>>>>> xend, why check at all in xend? >>>>>> >>>>>> Probably you need to fail the domain create, or at least shutdown >>>>>> the domain, when a device that should be passed through cannot be >>>>>> passed through. That will naturally cause the device assignment to >>>>>> be torn down (if it was set set up), as the domain is destroyed. >>>>>> Whether this is all actioned from xend or from qemu-dm doesn''t >>>>>> much matter, but the split responsibility isn''t clean. Probably >>>>>> xend is better just because the error handling is easier and >>>>>> earlier there. >>>>>> >>>>>> -- Keir >>>>>> >>>>>> On 10/12/07 08:13, "Han, Weidong" <weidong.han@intel.com> wrote: >>>>>> >>>>>>> Xend is too early to do real assignment, do some checking is >>>>>>> better. These two issues will be found by the checks in Xend ( 1) >>>>>>> issue will be found by pciback). After passing these checks in >>>>>>> Xend, it''s suitable to do real device assignment and memory and >>>>>>> ioport mappings in qemu. >>>>>>> >>>>>>> Randy (Weidong) >>>>>>> >>>>>>> Muli Ben-Yehuda wrote: >>>>>>>> On Mon, Dec 10, 2007 at 02:21:55PM +0800, Han, Weidong wrote: >>>>>>>> >>>>>>>>> Currently we assign devices with VT-d in Xend, this raises two >>>>>>>>> issues: 1) assign devices regardless of they are hidden by >>>>>>>>> pciback or not. If the device is not hidden, it results in the >>>>>>>>> device doesn''t work in Dom0; 2) device is assigned one by one, >>>>>>>>> if assign multiple devices, some devices may have been assigned >>>>>>>>> when problem happens, it results in assigned devices don''t work >>>>>>>>> in Dom0. I think Xend is not a good place to assign devices. >>>>>>>>> This patch adds a parameter to xc_assign_device(), let it just >>>>>>>>> do check in Xend whether the devices can be assigned or not, >>>>>>>>> and move real device assignment to qemu. >>>>>>>> >>>>>>>> Why is qemu a better place to assign the devices? How does >>>>>>>> moving it to qemu solve the two issues mentioned above? >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Muli >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Xen-devel mailing list >>>>>>> Xen-devel@lists.xensource.com >>>>>>> http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
If the device has not been hidden by pciback, pciback will quit domain creation and prompt on screen. But current check we added in xenddomaininfo.py is executed before pciback check. Randy (Weidong) Keir Fraser wrote:> What if the device has not been hidden by pciback (a more likely > failure mode, I should think)? > > -- Keir > > On 10/12/07 11:41, "Han, Weidong" <weidong.han@intel.com> wrote: > >> The assignment is in qemu-dm originally, we moved it to xend for user >> friendly, because it can prompt user on screen. If strip it out, it >> is not easy for end user to find failure reason. >> >> Randy (Weidong) >> >> Keir Fraser wrote: >>> The reason for moving device assignment to qemu-dm is that the >>> assignment can fail even after the assignment has happened. I''m >>> happy to move the assignment to qemu-dm, but then the ''check'' in >>> xend gets stripped out. >>> >>> -- Keir >>> >>> On 10/12/07 09:01, "Han, Weidong" <weidong.han@intel.com> wrote: >>> >>>> The checks in Xend just check high level errors, such as VT-d >>>> disabled, device is not exist, or not hidden. When find these >>>> errors, quit domain creation immediately. The real actions are >>>> taken in qemu. What error handling do you mean? >>>> >>>> Randy (Weidong) >>>> >>>> Keir Fraser wrote: >>>>> Then the whole lot should be done in qemu-dm and it sounds like >>>>> the error handling needs to be improved. >>>>> >>>>> -- Keir >>>>> >>>>> On 10/12/07 08:45, "Han, Weidong" <weidong.han@intel.com> wrote: >>>>> >>>>>> I agree the early do those checks the better. But without mapping >>>>>> memory and I/O port in qemu, the assigned device can''t be really >>>>>> used. That''s to say the assignment is not completed in a way. In >>>>>> addition, if we want to support hotplug devices, it''s not >>>>>> suitable to do assignment in Xend. >>>>>> >>>>>> Randy (Weidong) >>>>>> >>>>>> Keir Fraser wrote: >>>>>>> Sounds to me like qemu-dm is too late to do those other checks >>>>>>> or you need better error handling in qemu-dm. Really you have a >>>>>>> choice: do all tests and assignment in qemu-dm, or do all tests >>>>>>> and assignment in xend. Doing half-and-half is stupid. If the >>>>>>> pass-through can still fail even after the check you leave in >>>>>>> xend, why check at all in xend? >>>>>>> >>>>>>> Probably you need to fail the domain create, or at least >>>>>>> shutdown the domain, when a device that should be passed >>>>>>> through cannot be passed through. That will naturally cause the >>>>>>> device assignment to be torn down (if it was set set up), as >>>>>>> the domain is destroyed. Whether this is all actioned from xend >>>>>>> or from qemu-dm doesn''t much matter, but the split >>>>>>> responsibility isn''t clean. Probably xend is better just >>>>>>> because the error handling is easier and earlier there. >>>>>>> >>>>>>> -- Keir >>>>>>> >>>>>>> On 10/12/07 08:13, "Han, Weidong" <weidong.han@intel.com> wrote: >>>>>>> >>>>>>>> Xend is too early to do real assignment, do some checking is >>>>>>>> better. These two issues will be found by the checks in Xend ( >>>>>>>> 1) issue will be found by pciback). After passing these checks >>>>>>>> in Xend, it''s suitable to do real device assignment and memory >>>>>>>> and ioport mappings in qemu. >>>>>>>> >>>>>>>> Randy (Weidong) >>>>>>>> >>>>>>>> Muli Ben-Yehuda wrote: >>>>>>>>> On Mon, Dec 10, 2007 at 02:21:55PM +0800, Han, Weidong wrote: >>>>>>>>> >>>>>>>>>> Currently we assign devices with VT-d in Xend, this raises >>>>>>>>>> two issues: 1) assign devices regardless of they are hidden >>>>>>>>>> by pciback or not. If the device is not hidden, it results >>>>>>>>>> in the device doesn''t work in Dom0; 2) device is assigned >>>>>>>>>> one by one, if assign multiple devices, some devices may >>>>>>>>>> have been assigned when problem happens, it results in >>>>>>>>>> assigned devices don''t work in Dom0. I think Xend is not a >>>>>>>>>> good place to assign devices. This patch adds a parameter to >>>>>>>>>> xc_assign_device(), let it just do check in Xend whether the >>>>>>>>>> devices can be assigned or not, and move real device >>>>>>>>>> assignment to qemu. >>>>>>>>> >>>>>>>>> Why is qemu a better place to assign the devices? How does >>>>>>>>> moving it to qemu solve the two issues mentioned above? >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> Muli >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Xen-devel mailing list >>>>>>>> Xen-devel@lists.xensource.com >>>>>>>> http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
I think we''re going round in circles now, but anyway: If you do all your checks in xend, then the setup in qemu-dm shouldn''t fail. If the setup in qemu-dm *does* fail then the domain should be shut down. If the domain is shut down, the assignment will be torn down. So, why not leave the assignment in xend? -- Keir On 10/12/07 11:59, "Han, Weidong" <weidong.han@intel.com> wrote:> If the device has not been hidden by pciback, pciback will quit domain > creation and prompt on screen. But current check we added in > xenddomaininfo.py is executed before pciback check. > > Randy (Weidong) > > Keir Fraser wrote: >> What if the device has not been hidden by pciback (a more likely >> failure mode, I should think)? >> >> -- Keir >> >> On 10/12/07 11:41, "Han, Weidong" <weidong.han@intel.com> wrote: >> >>> The assignment is in qemu-dm originally, we moved it to xend for user >>> friendly, because it can prompt user on screen. If strip it out, it >>> is not easy for end user to find failure reason. >>> >>> Randy (Weidong) >>> >>> Keir Fraser wrote: >>>> The reason for moving device assignment to qemu-dm is that the >>>> assignment can fail even after the assignment has happened. I''m >>>> happy to move the assignment to qemu-dm, but then the ''check'' in >>>> xend gets stripped out. >>>> >>>> -- Keir >>>> >>>> On 10/12/07 09:01, "Han, Weidong" <weidong.han@intel.com> wrote: >>>> >>>>> The checks in Xend just check high level errors, such as VT-d >>>>> disabled, device is not exist, or not hidden. When find these >>>>> errors, quit domain creation immediately. The real actions are >>>>> taken in qemu. What error handling do you mean? >>>>> >>>>> Randy (Weidong) >>>>> >>>>> Keir Fraser wrote: >>>>>> Then the whole lot should be done in qemu-dm and it sounds like >>>>>> the error handling needs to be improved. >>>>>> >>>>>> -- Keir >>>>>> >>>>>> On 10/12/07 08:45, "Han, Weidong" <weidong.han@intel.com> wrote: >>>>>> >>>>>>> I agree the early do those checks the better. But without mapping >>>>>>> memory and I/O port in qemu, the assigned device can''t be really >>>>>>> used. That''s to say the assignment is not completed in a way. In >>>>>>> addition, if we want to support hotplug devices, it''s not >>>>>>> suitable to do assignment in Xend. >>>>>>> >>>>>>> Randy (Weidong) >>>>>>> >>>>>>> Keir Fraser wrote: >>>>>>>> Sounds to me like qemu-dm is too late to do those other checks >>>>>>>> or you need better error handling in qemu-dm. Really you have a >>>>>>>> choice: do all tests and assignment in qemu-dm, or do all tests >>>>>>>> and assignment in xend. Doing half-and-half is stupid. If the >>>>>>>> pass-through can still fail even after the check you leave in >>>>>>>> xend, why check at all in xend? >>>>>>>> >>>>>>>> Probably you need to fail the domain create, or at least >>>>>>>> shutdown the domain, when a device that should be passed >>>>>>>> through cannot be passed through. That will naturally cause the >>>>>>>> device assignment to be torn down (if it was set set up), as >>>>>>>> the domain is destroyed. Whether this is all actioned from xend >>>>>>>> or from qemu-dm doesn''t much matter, but the split >>>>>>>> responsibility isn''t clean. Probably xend is better just >>>>>>>> because the error handling is easier and earlier there. >>>>>>>> >>>>>>>> -- Keir >>>>>>>> >>>>>>>> On 10/12/07 08:13, "Han, Weidong" <weidong.han@intel.com> wrote: >>>>>>>> >>>>>>>>> Xend is too early to do real assignment, do some checking is >>>>>>>>> better. These two issues will be found by the checks in Xend ( >>>>>>>>> 1) issue will be found by pciback). After passing these checks >>>>>>>>> in Xend, it''s suitable to do real device assignment and memory >>>>>>>>> and ioport mappings in qemu. >>>>>>>>> >>>>>>>>> Randy (Weidong) >>>>>>>>> >>>>>>>>> Muli Ben-Yehuda wrote: >>>>>>>>>> On Mon, Dec 10, 2007 at 02:21:55PM +0800, Han, Weidong wrote: >>>>>>>>>> >>>>>>>>>>> Currently we assign devices with VT-d in Xend, this raises >>>>>>>>>>> two issues: 1) assign devices regardless of they are hidden >>>>>>>>>>> by pciback or not. If the device is not hidden, it results >>>>>>>>>>> in the device doesn''t work in Dom0; 2) device is assigned >>>>>>>>>>> one by one, if assign multiple devices, some devices may >>>>>>>>>>> have been assigned when problem happens, it results in >>>>>>>>>>> assigned devices don''t work in Dom0. I think Xend is not a >>>>>>>>>>> good place to assign devices. This patch adds a parameter to >>>>>>>>>>> xc_assign_device(), let it just do check in Xend whether the >>>>>>>>>>> devices can be assigned or not, and move real device >>>>>>>>>>> assignment to qemu. >>>>>>>>>> >>>>>>>>>> Why is qemu a better place to assign the devices? How does >>>>>>>>>> moving it to qemu solve the two issues mentioned above? >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Muli >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Xen-devel mailing list >>>>>>>>> Xen-devel@lists.xensource.com >>>>>>>>> http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
If don''t consider hotplug case, do assignment in xend is no problem, but must guarantee that devices are hidden before assignment. Randy (Weidong) Keir Fraser wrote:> I think we''re going round in circles now, but anyway: > > If you do all your checks in xend, then the setup in qemu-dm > shouldn''t fail. > > If the setup in qemu-dm *does* fail then the domain should be shut > down. > > If the domain is shut down, the assignment will be torn down. > > So, why not leave the assignment in xend? > > -- Keir > > On 10/12/07 11:59, "Han, Weidong" <weidong.han@intel.com> wrote: > >> If the device has not been hidden by pciback, pciback will quit >> domain creation and prompt on screen. But current check we added in >> xenddomaininfo.py is executed before pciback check. >> >> Randy (Weidong) >> >> Keir Fraser wrote: >>> What if the device has not been hidden by pciback (a more likely >>> failure mode, I should think)? >>> >>> -- Keir >>> >>> On 10/12/07 11:41, "Han, Weidong" <weidong.han@intel.com> wrote: >>> >>>> The assignment is in qemu-dm originally, we moved it to xend for >>>> user friendly, because it can prompt user on screen. If strip it >>>> out, it is not easy for end user to find failure reason. >>>> >>>> Randy (Weidong) >>>> >>>> Keir Fraser wrote: >>>>> The reason for moving device assignment to qemu-dm is that the >>>>> assignment can fail even after the assignment has happened. I''m >>>>> happy to move the assignment to qemu-dm, but then the ''check'' in >>>>> xend gets stripped out. >>>>> >>>>> -- Keir >>>>> >>>>> On 10/12/07 09:01, "Han, Weidong" <weidong.han@intel.com> wrote: >>>>> >>>>>> The checks in Xend just check high level errors, such as VT-d >>>>>> disabled, device is not exist, or not hidden. When find these >>>>>> errors, quit domain creation immediately. The real actions are >>>>>> taken in qemu. What error handling do you mean? >>>>>> >>>>>> Randy (Weidong) >>>>>> >>>>>> Keir Fraser wrote: >>>>>>> Then the whole lot should be done in qemu-dm and it sounds like >>>>>>> the error handling needs to be improved. >>>>>>> >>>>>>> -- Keir >>>>>>> >>>>>>> On 10/12/07 08:45, "Han, Weidong" <weidong.han@intel.com> wrote: >>>>>>> >>>>>>>> I agree the early do those checks the better. But without >>>>>>>> mapping memory and I/O port in qemu, the assigned device can''t >>>>>>>> be really used. That''s to say the assignment is not completed >>>>>>>> in a way. In addition, if we want to support hotplug devices, >>>>>>>> it''s not suitable to do assignment in Xend. >>>>>>>> >>>>>>>> Randy (Weidong) >>>>>>>> >>>>>>>> Keir Fraser wrote: >>>>>>>>> Sounds to me like qemu-dm is too late to do those other checks >>>>>>>>> or you need better error handling in qemu-dm. Really you have >>>>>>>>> a choice: do all tests and assignment in qemu-dm, or do all >>>>>>>>> tests and assignment in xend. Doing half-and-half is stupid. >>>>>>>>> If the pass-through can still fail even after the check you >>>>>>>>> leave in xend, why check at all in xend? >>>>>>>>> >>>>>>>>> Probably you need to fail the domain create, or at least >>>>>>>>> shutdown the domain, when a device that should be passed >>>>>>>>> through cannot be passed through. That will naturally cause >>>>>>>>> the device assignment to be torn down (if it was set set up), >>>>>>>>> as the domain is destroyed. Whether this is all actioned from >>>>>>>>> xend or from qemu-dm doesn''t much matter, but the split >>>>>>>>> responsibility isn''t clean. Probably xend is better just >>>>>>>>> because the error handling is easier and earlier there. >>>>>>>>> >>>>>>>>> -- Keir >>>>>>>>> >>>>>>>>> On 10/12/07 08:13, "Han, Weidong" <weidong.han@intel.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Xend is too early to do real assignment, do some checking is >>>>>>>>>> better. These two issues will be found by the checks in Xend >>>>>>>>>> ( 1) issue will be found by pciback). After passing these >>>>>>>>>> checks in Xend, it''s suitable to do real device assignment >>>>>>>>>> and memory and ioport mappings in qemu. >>>>>>>>>> >>>>>>>>>> Randy (Weidong) >>>>>>>>>> >>>>>>>>>> Muli Ben-Yehuda wrote: >>>>>>>>>>> On Mon, Dec 10, 2007 at 02:21:55PM +0800, Han, Weidong >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Currently we assign devices with VT-d in Xend, this raises >>>>>>>>>>>> two issues: 1) assign devices regardless of they are hidden >>>>>>>>>>>> by pciback or not. If the device is not hidden, it results >>>>>>>>>>>> in the device doesn''t work in Dom0; 2) device is assigned >>>>>>>>>>>> one by one, if assign multiple devices, some devices may >>>>>>>>>>>> have been assigned when problem happens, it results in >>>>>>>>>>>> assigned devices don''t work in Dom0. I think Xend is not a >>>>>>>>>>>> good place to assign devices. This patch adds a parameter >>>>>>>>>>>> to xc_assign_device(), let it just do check in Xend >>>>>>>>>>>> whether the devices can be assigned or not, and move real >>>>>>>>>>>> device assignment to qemu. >>>>>>>>>>> >>>>>>>>>>> Why is qemu a better place to assign the devices? How does >>>>>>>>>>> moving it to qemu solve the two issues mentioned above? >>>>>>>>>>> >>>>>>>>>>> Cheers, >>>>>>>>>>> Muli >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Xen-devel mailing list >>>>>>>>>> Xen-devel@lists.xensource.com >>>>>>>>>> http://lists.xensource.com/xen-devel > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
If assignment is successful, but others (eg. interrupt binding) fail, we also need to shut down the domain, and then tear down the assignment. So, qemu may be better, because all actions are put together there. Randy (Weidong) Keir Fraser wrote:> I think we''re going round in circles now, but anyway: > > If you do all your checks in xend, then the setup in qemu-dm > shouldn''t fail. > > If the setup in qemu-dm *does* fail then the domain should be shut > down. > > If the domain is shut down, the assignment will be torn down. > > So, why not leave the assignment in xend? > > -- Keir > > On 10/12/07 11:59, "Han, Weidong" <weidong.han@intel.com> wrote: > >> If the device has not been hidden by pciback, pciback will quit >> domain creation and prompt on screen. But current check we added in >> xenddomaininfo.py is executed before pciback check. >> >> Randy (Weidong) >> >> Keir Fraser wrote: >>> What if the device has not been hidden by pciback (a more likely >>> failure mode, I should think)? >>> >>> -- Keir >>> >>> On 10/12/07 11:41, "Han, Weidong" <weidong.han@intel.com> wrote: >>> >>>> The assignment is in qemu-dm originally, we moved it to xend for >>>> user friendly, because it can prompt user on screen. If strip it >>>> out, it is not easy for end user to find failure reason. >>>> >>>> Randy (Weidong) >>>> >>>> Keir Fraser wrote: >>>>> The reason for moving device assignment to qemu-dm is that the >>>>> assignment can fail even after the assignment has happened. I''m >>>>> happy to move the assignment to qemu-dm, but then the ''check'' in >>>>> xend gets stripped out. >>>>> >>>>> -- Keir >>>>> >>>>> On 10/12/07 09:01, "Han, Weidong" <weidong.han@intel.com> wrote: >>>>> >>>>>> The checks in Xend just check high level errors, such as VT-d >>>>>> disabled, device is not exist, or not hidden. When find these >>>>>> errors, quit domain creation immediately. The real actions are >>>>>> taken in qemu. What error handling do you mean? >>>>>> >>>>>> Randy (Weidong) >>>>>> >>>>>> Keir Fraser wrote: >>>>>>> Then the whole lot should be done in qemu-dm and it sounds like >>>>>>> the error handling needs to be improved. >>>>>>> >>>>>>> -- Keir >>>>>>> >>>>>>> On 10/12/07 08:45, "Han, Weidong" <weidong.han@intel.com> wrote: >>>>>>> >>>>>>>> I agree the early do those checks the better. But without >>>>>>>> mapping memory and I/O port in qemu, the assigned device can''t >>>>>>>> be really used. That''s to say the assignment is not completed >>>>>>>> in a way. In addition, if we want to support hotplug devices, >>>>>>>> it''s not suitable to do assignment in Xend. >>>>>>>> >>>>>>>> Randy (Weidong) >>>>>>>> >>>>>>>> Keir Fraser wrote: >>>>>>>>> Sounds to me like qemu-dm is too late to do those other checks >>>>>>>>> or you need better error handling in qemu-dm. Really you have >>>>>>>>> a choice: do all tests and assignment in qemu-dm, or do all >>>>>>>>> tests and assignment in xend. Doing half-and-half is stupid. >>>>>>>>> If the pass-through can still fail even after the check you >>>>>>>>> leave in xend, why check at all in xend? >>>>>>>>> >>>>>>>>> Probably you need to fail the domain create, or at least >>>>>>>>> shutdown the domain, when a device that should be passed >>>>>>>>> through cannot be passed through. That will naturally cause >>>>>>>>> the device assignment to be torn down (if it was set set up), >>>>>>>>> as the domain is destroyed. Whether this is all actioned from >>>>>>>>> xend or from qemu-dm doesn''t much matter, but the split >>>>>>>>> responsibility isn''t clean. Probably xend is better just >>>>>>>>> because the error handling is easier and earlier there. >>>>>>>>> >>>>>>>>> -- Keir >>>>>>>>> >>>>>>>>> On 10/12/07 08:13, "Han, Weidong" <weidong.han@intel.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Xend is too early to do real assignment, do some checking is >>>>>>>>>> better. These two issues will be found by the checks in Xend >>>>>>>>>> ( 1) issue will be found by pciback). After passing these >>>>>>>>>> checks in Xend, it''s suitable to do real device assignment >>>>>>>>>> and memory and ioport mappings in qemu. >>>>>>>>>> >>>>>>>>>> Randy (Weidong) >>>>>>>>>> >>>>>>>>>> Muli Ben-Yehuda wrote: >>>>>>>>>>> On Mon, Dec 10, 2007 at 02:21:55PM +0800, Han, Weidong >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Currently we assign devices with VT-d in Xend, this raises >>>>>>>>>>>> two issues: 1) assign devices regardless of they are hidden >>>>>>>>>>>> by pciback or not. If the device is not hidden, it results >>>>>>>>>>>> in the device doesn''t work in Dom0; 2) device is assigned >>>>>>>>>>>> one by one, if assign multiple devices, some devices may >>>>>>>>>>>> have been assigned when problem happens, it results in >>>>>>>>>>>> assigned devices don''t work in Dom0. I think Xend is not a >>>>>>>>>>>> good place to assign devices. This patch adds a parameter >>>>>>>>>>>> to xc_assign_device(), let it just do check in Xend >>>>>>>>>>>> whether the devices can be assigned or not, and move real >>>>>>>>>>>> device assignment to qemu. >>>>>>>>>>> >>>>>>>>>>> Why is qemu a better place to assign the devices? How does >>>>>>>>>>> moving it to qemu solve the two issues mentioned above? >>>>>>>>>>> >>>>>>>>>>> Cheers, >>>>>>>>>>> Muli >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Xen-devel mailing list >>>>>>>>>> Xen-devel@lists.xensource.com >>>>>>>>>> http://lists.xensource.com/xen-devel > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Oh, I think I see. So pciback gets told about the pass-through setup, and is allowed to set up a dummy device at that time. And you want to delay the assignment until after that? Anyway, the patch as is could be improved. I''d add another domctl command rather than add a hacky boolean field to the structure. The new command can share the existing assign_device structure. -- Keir On 10/12/07 12:34, "Han, Weidong" <weidong.han@intel.com> wrote:> If don''t consider hotplug case, do assignment in xend is no problem, but > must guarantee that devices are hidden before assignment. > > Randy (Weidong) > > Keir Fraser wrote: >> I think we''re going round in circles now, but anyway: >> >> If you do all your checks in xend, then the setup in qemu-dm >> shouldn''t fail. >> >> If the setup in qemu-dm *does* fail then the domain should be shut >> down. >> >> If the domain is shut down, the assignment will be torn down. >> >> So, why not leave the assignment in xend? >> >> -- Keir >> >> On 10/12/07 11:59, "Han, Weidong" <weidong.han@intel.com> wrote: >> >>> If the device has not been hidden by pciback, pciback will quit >>> domain creation and prompt on screen. But current check we added in >>> xenddomaininfo.py is executed before pciback check. >>> >>> Randy (Weidong) >>> >>> Keir Fraser wrote: >>>> What if the device has not been hidden by pciback (a more likely >>>> failure mode, I should think)? >>>> >>>> -- Keir >>>> >>>> On 10/12/07 11:41, "Han, Weidong" <weidong.han@intel.com> wrote: >>>> >>>>> The assignment is in qemu-dm originally, we moved it to xend for >>>>> user friendly, because it can prompt user on screen. If strip it >>>>> out, it is not easy for end user to find failure reason. >>>>> >>>>> Randy (Weidong) >>>>> >>>>> Keir Fraser wrote: >>>>>> The reason for moving device assignment to qemu-dm is that the >>>>>> assignment can fail even after the assignment has happened. I''m >>>>>> happy to move the assignment to qemu-dm, but then the ''check'' in >>>>>> xend gets stripped out. >>>>>> >>>>>> -- Keir >>>>>> >>>>>> On 10/12/07 09:01, "Han, Weidong" <weidong.han@intel.com> wrote: >>>>>> >>>>>>> The checks in Xend just check high level errors, such as VT-d >>>>>>> disabled, device is not exist, or not hidden. When find these >>>>>>> errors, quit domain creation immediately. The real actions are >>>>>>> taken in qemu. What error handling do you mean? >>>>>>> >>>>>>> Randy (Weidong) >>>>>>> >>>>>>> Keir Fraser wrote: >>>>>>>> Then the whole lot should be done in qemu-dm and it sounds like >>>>>>>> the error handling needs to be improved. >>>>>>>> >>>>>>>> -- Keir >>>>>>>> >>>>>>>> On 10/12/07 08:45, "Han, Weidong" <weidong.han@intel.com> wrote: >>>>>>>> >>>>>>>>> I agree the early do those checks the better. But without >>>>>>>>> mapping memory and I/O port in qemu, the assigned device can''t >>>>>>>>> be really used. That''s to say the assignment is not completed >>>>>>>>> in a way. In addition, if we want to support hotplug devices, >>>>>>>>> it''s not suitable to do assignment in Xend. >>>>>>>>> >>>>>>>>> Randy (Weidong) >>>>>>>>> >>>>>>>>> Keir Fraser wrote: >>>>>>>>>> Sounds to me like qemu-dm is too late to do those other checks >>>>>>>>>> or you need better error handling in qemu-dm. Really you have >>>>>>>>>> a choice: do all tests and assignment in qemu-dm, or do all >>>>>>>>>> tests and assignment in xend. Doing half-and-half is stupid. >>>>>>>>>> If the pass-through can still fail even after the check you >>>>>>>>>> leave in xend, why check at all in xend? >>>>>>>>>> >>>>>>>>>> Probably you need to fail the domain create, or at least >>>>>>>>>> shutdown the domain, when a device that should be passed >>>>>>>>>> through cannot be passed through. That will naturally cause >>>>>>>>>> the device assignment to be torn down (if it was set set up), >>>>>>>>>> as the domain is destroyed. Whether this is all actioned from >>>>>>>>>> xend or from qemu-dm doesn''t much matter, but the split >>>>>>>>>> responsibility isn''t clean. Probably xend is better just >>>>>>>>>> because the error handling is easier and earlier there. >>>>>>>>>> >>>>>>>>>> -- Keir >>>>>>>>>> >>>>>>>>>> On 10/12/07 08:13, "Han, Weidong" <weidong.han@intel.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Xend is too early to do real assignment, do some checking is >>>>>>>>>>> better. These two issues will be found by the checks in Xend >>>>>>>>>>> ( 1) issue will be found by pciback). After passing these >>>>>>>>>>> checks in Xend, it''s suitable to do real device assignment >>>>>>>>>>> and memory and ioport mappings in qemu. >>>>>>>>>>> >>>>>>>>>>> Randy (Weidong) >>>>>>>>>>> >>>>>>>>>>> Muli Ben-Yehuda wrote: >>>>>>>>>>>> On Mon, Dec 10, 2007 at 02:21:55PM +0800, Han, Weidong >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Currently we assign devices with VT-d in Xend, this raises >>>>>>>>>>>> two issues: 1) assign devices regardless of they are hidden >>>>>>>>>>>> by pciback or not. If the device is not hidden, it results >>>>>>>>>>>> in the device doesn''t work in Dom0; 2) device is assigned >>>>>>>>>>>> one by one, if assign multiple devices, some devices may >>>>>>>>>>>> have been assigned when problem happens, it results in >>>>>>>>>>>> assigned devices don''t work in Dom0. I think Xend is not a >>>>>>>>>>>> good place to assign devices. This patch adds a parameter >>>>>>>>>>>> to xc_assign_device(), let it just do check in Xend >>>>>>>>>>>> whether the devices can be assigned or not, and move real >>>>>>>>>>>> device assignment to qemu. >>>>>>>>>>>> >>>>>>>>>>>> Why is qemu a better place to assign the devices? How does >>>>>>>>>>>> moving it to qemu solve the two issues mentioned above? >>>>>>>>>>>> >>>>>>>>>>>> Cheers, >>>>>>>>>>>> Muli >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Xen-devel mailing list >>>>>>>>>>> Xen-devel@lists.xensource.com >>>>>>>>>>> http://lists.xensource.com/xen-devel >> >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser wrote:> Oh, I think I see. So pciback gets told about the pass-through setup, > and is allowed to set up a dummy device at that time. And you want to > delay the assignment until after that? >Yes.> Anyway, the patch as is could be improved. I''d add another domctl > command rather than add a hacky boolean field to the structure. The > new command can share the existing assign_device structure. >The patch is changed per your suggestion, it adds a new domctl command insteads of the boolean field. Randy (Weidong) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel