I ran into the following problem: I setup a shared ring page and an event channel between two domains (dom0 and one domU), DomU increases req_prod by one and notify the dom0 via the event channel, dom0 then check req_prod pointer and process the request in the shared ring. But I noticed that sometimes dom0 get the irq triggered by the notification event, but the req_prod remains the same. This situation only happened occasionally. Why? Because the req_prod is saved in the shared, mapped page, I suspect that there is a delay of updates between two domains. Is that true? If so, how to handle this problem? Your advice is highly appreciated! -x ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Are you running this on a single processor core? If not, it sounds to me like a cache coherency issue - you''ll need a write memory barrier after writing to the req_prod. If you''re using the ring macros from io/ring.h this should be already taken care of for you in the push macros. c. On Fri, 18 Feb 2005 16:44:30 -0500 (EST), Xin Zhao <zhaoxin@eecs.umich.edu> wrote:> I ran into the following problem: > > I setup a shared ring page and an event channel between two domains (dom0 > and one domU), DomU increases req_prod by one and notify the dom0 via the > event channel, dom0 then check req_prod pointer and process the request in > the shared ring. But I noticed that sometimes dom0 get the irq triggered > by the notification event, but the req_prod remains the same. This > situation only happened occasionally. Why? Because the req_prod is saved > in the shared, mapped page, I suspect that there is a delay of updates > between two domains. Is that true? If so, how to handle this problem? > > Your advice is highly appreciated! > > -x > > ------------------------------------------------------- > SF email is sponsored by - The IT Product Guide > Read honest & candid reviews on hundreds of IT Products from real users. > Discover which products truly live up to the hype. Start reading now. > http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xen-devel >------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Thanks for your advice. I tried rmb() and wmb() before I sent last email, they does not help. Actually I am running a single processor core. :( So it should not have any delay. But I still ran into the problem. Another interesting thing is that my working flow should be as follows: domU issue a request -> generate an irq at dom0 -> domU waits for reply-> dom0 gets the request, processes it and reply -> domU wakes up by irq caused by notification event sent by dom0, gets the result -> DomU sends another request That means, if domU does not get the irq from dom0, it will not issue next request. Because of the problem of req_prod updates, dom0 failed to get the request and thus will not process it, not mention to reply. So dom0 should not be able to receive next request or next irq. But the fact is that dom0 can still get irq even it failed to get last request. It appears that dom0 issues multiple irqs for one event. How could that happed? -x On Fri, 18 Feb 2005, Christopher Clark wrote:> Are you running this on a single processor core? > > If not, it sounds to me like a cache coherency issue - you''ll need a > write memory barrier after writing to the req_prod. If you''re using > the ring macros from io/ring.h this should be already taken care of > for you in the push macros. > > c. > > > On Fri, 18 Feb 2005 16:44:30 -0500 (EST), Xin Zhao > <zhaoxin@eecs.umich.edu> wrote: > > I ran into the following problem: > > > > I setup a shared ring page and an event channel between two domains (dom0 > > and one domU), DomU increases req_prod by one and notify the dom0 via the > > event channel, dom0 then check req_prod pointer and process the request in > > the shared ring. But I noticed that sometimes dom0 get the irq triggered > > by the notification event, but the req_prod remains the same. This > > situation only happened occasionally. Why? Because the req_prod is saved > > in the shared, mapped page, I suspect that there is a delay of updates > > between two domains. Is that true? If so, how to handle this problem? > > > > Your advice is highly appreciated! > > > > -x > > > > ------------------------------------------------------- > > SF email is sponsored by - The IT Product Guide > > Read honest & candid reviews on hundreds of IT Products from real users. > > Discover which products truly live up to the hype. Start reading now. > > http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/xen-devel > > >------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
This sounds like the symptoms you''d experience with multiple functions using the same event channel port. How are you obtaining the ports you''re using to communicate? Are you using alloc_unbound ? c. On Fri, 2005-02-18 at 22:41, Xin Zhao wrote:> Thanks for your advice. I tried rmb() and wmb() before I sent last email, > they does not help. > > Actually I am running a single processor core. :( So it should not have > any delay. But I still ran into the problem. Another interesting thing is > that my working flow should be as follows: > > domU issue a request -> generate an irq at dom0 -> domU waits for reply-> > dom0 gets the request, processes it and reply -> domU wakes up by irq > caused by notification event sent by dom0, gets the result -> DomU sends > another request > > That means, if domU does not get the irq from dom0, it will not issue next > request. Because of the problem of req_prod updates, dom0 failed to get > the request and thus will not process it, not mention to reply. So dom0 > should not be able to receive next request or next irq. > > But the fact is that dom0 can still get irq even it failed to get last > request. It appears that dom0 issues multiple irqs for one event. How > could that happed? > > -x > > > On Fri, 18 Feb 2005, Christopher Clark wrote: > > > Are you running this on a single processor core? > > > > If not, it sounds to me like a cache coherency issue - you''ll need a > > write memory barrier after writing to the req_prod. If you''re using > > the ring macros from io/ring.h this should be already taken care of > > for you in the push macros. > > > > c. > > > > > > On Fri, 18 Feb 2005 16:44:30 -0500 (EST), Xin Zhao > > <zhaoxin@eecs.umich.edu> wrote: > > > I ran into the following problem: > > > > > > I setup a shared ring page and an event channel between two domains (dom0 > > > and one domU), DomU increases req_prod by one and notify the dom0 via the > > > event channel, dom0 then check req_prod pointer and process the request in > > > the shared ring. But I noticed that sometimes dom0 get the irq triggered > > > by the notification event, but the req_prod remains the same. This > > > situation only happened occasionally. Why? Because the req_prod is saved > > > in the shared, mapped page, I suspect that there is a delay of updates > > > between two domains. Is that true? If so, how to handle this problem? > > > > > > Your advice is highly appreciated! > > > > > > -x > > > > > > ------------------------------------------------------- > > > SF email is sponsored by - The IT Product Guide > > > Read honest & candid reviews on hundreds of IT Products from real users. > > > Discover which products truly live up to the hype. Start reading now. > > > http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click > > > _______________________________________________ > > > Xen-devel mailing list > > > Xen-devel@lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/xen-devel > > > > >------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> I would like to do the same thing, right now I have shared mem between > domU and dom0, but no good syncronization. Is there an example somewhere > of how to do this in the new xcs world? It should be easier now that > this code has been separated out, right?If you''re sharing memory directly between the two then you don''t need to involve xcs (other than in the setup phase) since that just does control messages. You might find the ring macros in xen/include/public/ring.h useful. They make it easy to declare ring datatypes (with appropriately sized rings) and maintain producer / consumer indexes, with the appropriate memory barriers. Examples of use are in linux-2.6.10-xen-sparse/drivers/xen/{blk,usb} {back,front}. (the net drivers use a slightly different ring format instead). Cheers, Mark ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Xin Zhao wrote:> I ran into the following problem: > > I setup a shared ring page and an event channel between two domains (dom0 > and one domU), DomU increases req_prod by one and notify the dom0 via the > event channel, dom0 then check req_prod pointer and process the request in > the shared ring. But I noticed that sometimes dom0 get the irq triggered > by the notification event, but the req_prod remains the same. This > situation only happened occasionally.hi, I would like to do the same thing, right now I have shared mem between domU and dom0, but no good syncronization. Is there an example somewhere of how to do this in the new xcs world? It should be easier now that this code has been separated out, right? thanks, Jacob ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> Hmm will this be any different if the guy I wish to share this with in > dom0 is a userspace process? Can I use /dev/evtchn for that, or do I > have to cook my own device node for this?I think nowadays there can be multiple listeners on /dev/evtchn (as long as, obviously, they''re all listening to different things), so yes that should work fine. Of course, I''m not sure anyone''s tried this... Cheers, Mark ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Mark Williamson wrote:> If you''re sharing memory directly between the two then you don''t need to > involve xcs (other than in the setup phase) since that just does control > messages. > > You might find the ring macros in xen/include/public/ring.h useful. They make > it easy to declare ring datatypes (with appropriately sized rings) and > maintain producer / consumer indexes, with the appropriate memory barriers. > > Examples of use are in linux-2.6.10-xen-sparse/drivers/xen/{blk,usb} > {back,front}. (the net drivers use a slightly different ring format > instead).Hmm will this be any different if the guy I wish to share this with in dom0 is a userspace process? Can I use /dev/evtchn for that, or do I have to cook my own device node for this? thanks, Jacob ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Mark Williamson wrote:>I think nowadays there can be multiple listeners on /dev/evtchn (as long as, >obviously, they''re all listening to different things), so yes that should >work fine. Of course, I''m not sure anyone''s tried this... > >I have. It worked fine for me.>Cheers, >Mark > > >------------------------------------------------------- >SF email is sponsored by - The IT Product Guide >Read honest & candid reviews on hundreds of IT Products from real users. >Discover which products truly live up to the hype. Start reading now. >http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click >_______________________________________________ >Xen-devel mailing list >Xen-devel@lists.sourceforge.net >https://lists.sourceforge.net/lists/listinfo/xen-devel > > >-- Anthony Liguori anthony@codemonkey.ws ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Anthony Liguori wrote:> Mark Williamson wrote: > >> I think nowadays there can be multiple listeners on /dev/evtchn (as >> long as, obviously, they''re all listening to different things), so yes >> that should work fine. Of course, I''m not sure anyone''s tried this... >> >> > I have. It worked fine for me.Just to be lazy: do you have any code you could post? Do I just open the device and read whatever is in there, or how does it work? Jacob ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Jacob Gorm Hansen wrote:> Just to be lazy: do you have any code you could post? Do I just open the > device and read whatever is in there, or how does it work?Never mind, doesn''t look to hard ;-) Jacob ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel