Hello, With xen 3.3.0 version, I understand that by increasing blkif_reqs you possibly speed up writes by being able to write whole of tracks. And, shared ring size is 64 between blkfront & blkback to fit in a page of size PAGE_SIZE. Both blkfront & blkback accomodate shared ring within a page size. Wouldn''t the shared ring be likely a bottleneck in very large number os IOs, have anybody ran into such issue? Is there a reason to _always_ accomodate ring buffer in a page? How can I increase the shared ring size between blkfront & blkback? Do you have any suggestions/recommendations? thanks, ~Sam _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hello, Samvel Yuri, le Tue 18 Nov 2008 15:29:48 -0800, a écrit :> Wouldn''t the shared ring be likely a bottleneck in very large number > os IOs, have anybody ran into such issue?Compared to disk speeds, it is almost a bottleneck nowadays. In mini-os, I could achieve almost native performance, but that was by using the whole buffer page. When disk will get faster, one page won''t be enough. Samuel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
hi Samuel- Would you please explain what do you mean by "using the whole buffer page"? Parallax (project of University of British Columbia) modified XEN by allocating an additional shared ring page in the blkback driver. Probably one of options? thanks, ~Sam ________________________________ From: Samuel Thibault <samuel.thibault@ens-lyon.org> To: Samvel Yuri <samvelox@yahoo.com> Cc: xen-devel <xen-devel@lists.xensource.com> Sent: Tuesday, November 18, 2008 3:39:43 PM Subject: Re: [Xen-devel] blkif shared ring Hello, Samvel Yuri, le Tue 18 Nov 2008 15:29:48 -0800, a écrit :> Wouldn''t the shared ring be likely a bottleneck in very large number > os IOs, have anybody ran into such issue?Compared to disk speeds, it is almost a bottleneck nowadays. In mini-os, I could achieve almost native performance, but that was by using the whole buffer page. When disk will get faster, one page won''t be enough. Samuel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Samvel Yuri, le Tue 18 Nov 2008 15:49:26 -0800, a écrit :> Would you please explain what do you mean by "using the whole buffer page"?Filling the whole buffer with as many block requests as possible at a time. Samuel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, Nov 18, 2008 at 3:49 PM, Samvel Yuri <samvelox@yahoo.com> wrote:> > hi Samuel- > > Would you please explain what do you mean by "using the whole buffer page"? > > Parallax (project of University of British Columbia) modified XEN by > allocating an > additional shared ring page in the blkback driver. Probably one of options? >Hi, Regarding Parallax, we added support for multi-page ring to increase the throughput of a guest block device when using an iscsi filer as a backend. The single page ring is enough to saturate a standard SATA drive but we got a noticeable performance increase (from 65 to a 105MBps I think) when we doubled the ring size with the iscsi backend. I remember seeing multi-page ring patches for mini-os and fs-back from Samuel. Is there a plan to generalize these patches to blkfront/back/tap? We had patches for blkfront, blkback and blktap. The size of the ring was a compile time option for blkfront. The patches are pretty old (3.1 I think) but could probably be refreshed without too much effort. geoffrey _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Geoffrey Lefebvre, le Tue 18 Nov 2008 17:16:21 -0800, a écrit :> The single page ring is enough to saturate a standard SATA > drive but we got a noticeable performance increase (from 65 to a > 105MBps I think) when we doubled the ring size with the iscsi backend.That''s probably what I would have noticed with hardware better than what I had at the time (which was limited to ~65 indeed). Samuel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
hello Geoffrey- Ahh, so Parallax project just do multi-page ring (and not entirely another set of shared ring). I am going to try 4 * PAGE_SIZE and see whether the box stays alive. Were the patches self contained in blkfront & blkback? If possible a pointer to blkfront & blkback will be useful. thanks for you reply, Sameer thanks, ~Sam ________________________________ From: Geoffrey Lefebvre <geoffrey@cs.ubc.ca> To: Samvel Yuri <samvelox@yahoo.com> Cc: Samuel Thibault <samuel.thibault@ens-lyon.org>; xen-devel <xen-devel@lists.xensource.com>; Dutch Meyer <dmeyer@cs.ubc.ca>; Andrew Warfield <andy@cs.ubc.ca> Sent: Tuesday, November 18, 2008 5:16:21 PM Subject: Re: [Xen-devel] blkif shared ring On Tue, Nov 18, 2008 at 3:49 PM, Samvel Yuri <samvelox@yahoo.com> wrote:> > hi Samuel- > > Would you please explain what do you mean by "using the whole buffer page"? > > Parallax (project of University of British Columbia) modified XEN by > allocating an > additional shared ring page in the blkback driver. Probably one of options? >Hi, Regarding Parallax, we added support for multi-page ring to increase the throughput of a guest block device when using an iscsi filer as a backend. The single page ring is enough to saturate a standard SATA drive but we got a noticeable performance increase (from 65 to a 105MBps I think) when we doubled the ring size with the iscsi backend. I remember seeing multi-page ring patches for mini-os and fs-back from Samuel. Is there a plan to generalize these patches to blkfront/back/tap? We had patches for blkfront, blkback and blktap. The size of the ring was a compile time option for blkfront. The patches are pretty old (3.1 I think) but could probably be refreshed without too much effort. geoffrey _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, Nov 18, 2008 at 6:02 PM, Samvel Yuri <samvelox@yahoo.com> wrote:> > hello Geoffrey- > > Ahh, so Parallax project just do multi-page ring (and not entirely another > set of shared ring).Yes that''s correct. We added a page to the existing ring.> am going to try 4 * PAGE_SIZE and see whether the box stays alive. Were theIn our case going to a 4 pages ring didn''t buy any additional performance. With 2 pages we were pretty close to saturating the gigabit link to the filer.> patches self > contained in blkfront & blkback? If possible a pointer to blkfront & blkback > will be useful. >The patches for blkfront and back were pretty self contained. Blktap was a bit more spread out because it changed user space code as well. For blkfront/back, the most significant changes were to the ring setup protocol to pass more than one grant reference. I might have time to refresh the patches and post them to the list next week if that is of interest to anyone. cheers, geoffrey _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Thanks for your reply, Geoffrey. If its not too much work then the blkfront & blkback patches even with old XEN version will be useful. thanks, Sam ________________________________ From: Geoffrey Lefebvre <geoffrey@cs.ubc.ca> To: Samvel Yuri <samvelox@yahoo.com> Cc: Dutch Meyer <dmeyer@cs.ubc.ca>; Samuel Thibault <samuel.thibault@ens-lyon.org>; xen-devel <xen-devel@lists.xensource.com>; Andrew Warfield <andy@cs.ubc.ca> Sent: Wednesday, November 19, 2008 8:57:11 PM Subject: Re: [Xen-devel] blkif shared ring On Tue, Nov 18, 2008 at 6:02 PM, Samvel Yuri <samvelox@yahoo.com> wrote:> > hello Geoffrey- > > Ahh, so Parallax project just do multi-page ring (and not entirely another > set of shared ring).Yes that''s correct. We added a page to the existing ring.> am going to try 4 * PAGE_SIZE and see whether the box stays alive. Were theIn our case going to a 4 pages ring didn''t buy any additional performance. With 2 pages we were pretty close to saturating the gigabit link to the filer.> patches self > contained in blkfront & blkback? If possible a pointer to blkfront & blkback > will be useful. >The patches for blkfront and back were pretty self contained. Blktap was a bit more spread out because it changed user space code as well. For blkfront/back, the most significant changes were to the ring setup protocol to pass more than one grant reference. I might have time to refresh the patches and post them to the list next week if that is of interest to anyone. cheers, geoffrey _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi Geoff, et al -- Did these blkfront/blkback multi-page patches ever make it upstream? Thanks, Dan> -----Original Message----- > From: Geoffrey Lefebvre [mailto:geoffrey@cs.ubc.ca] > Sent: Wednesday, November 19, 2008 9:57 PM > To: Samvel Yuri > Cc: Dutch Meyer; Samuel Thibault; xen-devel; Andrew Warfield > Subject: Re: [Xen-devel] blkif shared ring > > > On Tue, Nov 18, 2008 at 6:02 PM, Samvel Yuri > <samvelox@yahoo.com> wrote: > > > > hello Geoffrey- > > > > Ahh, so Parallax project just do multi-page ring (and not > entirely another > > set of shared ring). > > Yes that''s correct. We added a page to the existing ring. > > > am going to try 4 * PAGE_SIZE and see whether the box stays > alive. Were the > > In our case going to a 4 pages ring didn''t buy any additional > performance. With 2 pages we were pretty close to saturating the > gigabit link to the filer. > > > patches self > > contained in blkfront & blkback? If possible a pointer to > blkfront & blkback > > will be useful. > > > > The patches for blkfront and back were pretty self contained. Blktap > was a bit more spread out because it changed user space code as well. > For blkfront/back, the most significant changes were to the ring setup > protocol to pass more than one grant reference. I might have time to > refresh the patches and post them to the list next week if that is of > interest to anyone. > > cheers, > > geoffrey > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
They did not. -- Keir On 11/08/2009 00:20, "Dan Magenheimer" <dan.magenheimer@oracle.com> wrote:> Hi Geoff, et al -- > > Did these blkfront/blkback multi-page patches > ever make it upstream? > > Thanks, > Dan > >> -----Original Message----- >> From: Geoffrey Lefebvre [mailto:geoffrey@cs.ubc.ca] >> Sent: Wednesday, November 19, 2008 9:57 PM >> To: Samvel Yuri >> Cc: Dutch Meyer; Samuel Thibault; xen-devel; Andrew Warfield >> Subject: Re: [Xen-devel] blkif shared ring >> >> >> On Tue, Nov 18, 2008 at 6:02 PM, Samvel Yuri >> <samvelox@yahoo.com> wrote: >>> >>> hello Geoffrey- >>> >>> Ahh, so Parallax project just do multi-page ring (and not >> entirely another >>> set of shared ring). >> >> Yes that''s correct. We added a page to the existing ring. >> >>> am going to try 4 * PAGE_SIZE and see whether the box stays >> alive. Were the >> >> In our case going to a 4 pages ring didn''t buy any additional >> performance. With 2 pages we were pretty close to saturating the >> gigabit link to the filer. >> >>> patches self >>> contained in blkfront & blkback? If possible a pointer to >> blkfront & blkback >>> will be useful. >>> >> >> The patches for blkfront and back were pretty self contained. Blktap >> was a bit more spread out because it changed user space code as well. >> For blkfront/back, the most significant changes were to the ring setup >> protocol to pass more than one grant reference. I might have time to >> refresh the patches and post them to the list next week if that is of >> interest to anyone. >> >> cheers, >> >> geoffrey >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
I have a blkback patch that could be upstreamed. I''ll put together a blkfront patch and post them together. Paul Keir Fraser wrote:> They did not. > > -- Keir > > On 11/08/2009 00:20, "Dan Magenheimer" <dan.magenheimer@oracle.com> wrote: > >> Hi Geoff, et al -- >> >> Did these blkfront/blkback multi-page patches >> ever make it upstream? >> >> Thanks, >> Dan >> >>> -----Original Message----- >>> From: Geoffrey Lefebvre [mailto:geoffrey@cs.ubc.ca] >>> Sent: Wednesday, November 19, 2008 9:57 PM >>> To: Samvel Yuri >>> Cc: Dutch Meyer; Samuel Thibault; xen-devel; Andrew Warfield >>> Subject: Re: [Xen-devel] blkif shared ring >>> >>> >>> On Tue, Nov 18, 2008 at 6:02 PM, Samvel Yuri >>> <samvelox@yahoo.com> wrote: >>>> hello Geoffrey- >>>> >>>> Ahh, so Parallax project just do multi-page ring (and not >>> entirely another >>>> set of shared ring). >>> Yes that''s correct. We added a page to the existing ring. >>> >>>> am going to try 4 * PAGE_SIZE and see whether the box stays >>> alive. Were the >>> >>> In our case going to a 4 pages ring didn''t buy any additional >>> performance. With 2 pages we were pretty close to saturating the >>> gigabit link to the filer. >>> >>>> patches self >>>> contained in blkfront & blkback? If possible a pointer to >>> blkfront & blkback >>>> will be useful. >>>> >>> The patches for blkfront and back were pretty self contained. Blktap >>> was a bit more spread out because it changed user space code as well. >>> For blkfront/back, the most significant changes were to the ring setup >>> protocol to pass more than one grant reference. I might have time to >>> refresh the patches and post them to the list next week if that is of >>> interest to anyone. >>> >>> cheers, >>> >>> geoffrey >>> >>> _______________________________________________ >>> Xen-devel mailing list >>> Xen-devel@lists.xensource.com >>> http://lists.xensource.com/xen-devel >>> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel-- ==============================Paul Durrant, Software Engineer Citrix Systems (R&D) Ltd. First Floor, Building 101 Cambridge Science Park Milton Road Cambridge CB4 0FY United Kingdom ============================== _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel