Hi, I am trying to educate myself on prefetching/readahead algorithm for Lustre''s read. For a starter I only have two simple questions. 1 - Does Lustre detect linear or random I/O pattern or it always triggers readahead? 2 - If readahead is triggered, how many pages are read in addition to what is necessary? Thanks, Arifa.
On 2010-07-28, at 10:43, Arifa Nisar wrote:> I am trying to educate myself on prefetching/readahead algorithm for > Lustre''s read. For a starter I only have two simple questions. > > 1 - Does Lustre detect linear or random I/O pattern or it always triggers > readahead?The current algorithm detects sequential and strided read patterns.> 2 - If readahead is triggered, how many pages are read in addition to what > is necessary?It is variable based on how long the same pattern was repeating and the number of pages per request. I would be happy to have you look at the strided readahead detection code more closely, since I suspect it would not correctly detect stride patterns with odd byte size and offset vs. stride patterns with full-page size and offset. We disussed a while ago to rewrite the readahead code to be layered better on the client, but haven''t had time to work on it yet. Cheers, Andreas -- Andreas Dilger Lustre Technical Lead Oracle Corporation Canada Inc.
Thanks for the response. If an access pattern triggers page aligned strided access then how many pages will be readahead? Are there some environment variables which may give insight about readahead behavior? Arifa. -----Original Message----- From: Andreas Dilger [mailto:andreas.dilger at oracle.com] Sent: Wednesday, July 28, 2010 5:43 PM To: Arifa Nisar Cc: lustre-discuss at lists.lustre.org Subject: Re: [Lustre-discuss] Read ahead / prefetching On 2010-07-28, at 10:43, Arifa Nisar wrote:> I am trying to educate myself on prefetching/readahead algorithm for > Lustre''s read. For a starter I only have two simple questions. > > 1 - Does Lustre detect linear or random I/O pattern or it always triggers > readahead?The current algorithm detects sequential and strided read patterns.> 2 - If readahead is triggered, how many pages are read in addition to what > is necessary?It is variable based on how long the same pattern was repeating and the number of pages per request. I would be happy to have you look at the strided readahead detection code more closely, since I suspect it would not correctly detect stride patterns with odd byte size and offset vs. stride patterns with full-page size and offset. We disussed a while ago to rewrite the readahead code to be layered better on the client, but haven''t had time to work on it yet. Cheers, Andreas -- Andreas Dilger Lustre Technical Lead Oracle Corporation Canada Inc.
On 2010-07-29, at 14:02, Arifa Nisar wrote:> Thanks for the response. If an access pattern triggers page aligned strided > access then how many pages will be readahead? Are there some environment > variables which may give insight about readahead behavior?Because Lustre is tuned for RPC-sized IO, it will always readahead at least a full RPC at a time (by default 1MB), unless the application is reading larger chunks than this, then it reads ahead in units of the IO size aligned to RPC-sized boundaries.> -----Original Message----- > From: Andreas Dilger [mailto:andreas.dilger at oracle.com] > Sent: Wednesday, July 28, 2010 5:43 PM > To: Arifa Nisar > Cc: lustre-discuss at lists.lustre.org > Subject: Re: [Lustre-discuss] Read ahead / prefetching > > On 2010-07-28, at 10:43, Arifa Nisar wrote: >> I am trying to educate myself on prefetching/readahead algorithm for >> Lustre''s read. For a starter I only have two simple questions. >> >> 1 - Does Lustre detect linear or random I/O pattern or it always triggers >> readahead? > > The current algorithm detects sequential and strided read patterns. > >> 2 - If readahead is triggered, how many pages are read in addition to what >> is necessary? > > It is variable based on how long the same pattern was repeating and the > number of pages per request. > > I would be happy to have you look at the strided readahead detection code > more closely, since I suspect it would not correctly detect stride patterns > with odd byte size and offset vs. stride patterns with full-page size and > offset. > > We disussed a while ago to rewrite the readahead code to be layered better > on the client, but haven''t had time to work on it yet. > > > Cheers, Andreas > -- > Andreas Dilger > Lustre Technical Lead > Oracle Corporation Canada Inc. >Cheers, Andreas -- Andreas Dilger Lustre Technical Lead Oracle Corporation Canada Inc.
Consider a case where stripe count = 4, stripe size = 1MB and all I/O accesses are aligned with the stripe boundary. If a client reads every 4th full stripe (accessing same OST always) then one stripe will be readahead from that OST in every call? Thanks, Arifa. -----Original Message----- From: Andreas Dilger [mailto:andreas.dilger at oracle.com] Sent: Thursday, July 29, 2010 11:41 PM To: Arifa Nisar Cc: lustre-discuss at lists.lustre.org Subject: Re: [Lustre-discuss] Read ahead / prefetching On 2010-07-29, at 14:02, Arifa Nisar wrote:> Thanks for the response. If an access pattern triggers page alignedstrided> access then how many pages will be readahead? Are there some environment > variables which may give insight about readahead behavior?Because Lustre is tuned for RPC-sized IO, it will always readahead at least a full RPC at a time (by default 1MB), unless the application is reading larger chunks than this, then it reads ahead in units of the IO size aligned to RPC-sized boundaries.> -----Original Message----- > From: Andreas Dilger [mailto:andreas.dilger at oracle.com] > Sent: Wednesday, July 28, 2010 5:43 PM > To: Arifa Nisar > Cc: lustre-discuss at lists.lustre.org > Subject: Re: [Lustre-discuss] Read ahead / prefetching > > On 2010-07-28, at 10:43, Arifa Nisar wrote: >> I am trying to educate myself on prefetching/readahead algorithm for >> Lustre''s read. For a starter I only have two simple questions. >> >> 1 - Does Lustre detect linear or random I/O pattern or it always triggers >> readahead? > > The current algorithm detects sequential and strided read patterns. > >> 2 - If readahead is triggered, how many pages are read in addition towhat>> is necessary? > > It is variable based on how long the same pattern was repeating and the > number of pages per request. > > I would be happy to have you look at the strided readahead detection code > more closely, since I suspect it would not correctly detect stridepatterns> with odd byte size and offset vs. stride patterns with full-page size and > offset. > > We disussed a while ago to rewrite the readahead code to be layered better > on the client, but haven''t had time to work on it yet. > > > Cheers, Andreas > -- > Andreas Dilger > Lustre Technical Lead > Oracle Corporation Canada Inc. >Cheers, Andreas -- Andreas Dilger Lustre Technical Lead Oracle Corporation Canada Inc.
If the size of the read calls is 1MB then after 2 reads at 4MB strides the 3rd read would trigger readahead initially for the next stride (1MB+1MB). If the same stride pattern continues the readahead size will grow an additional 1MB each time up to the limit. Cheers, Andreas On 2010-07-29, at 22:53, Arifa Nisar <a-nisar at u.northwestern.edu> wrote:> Consider a case where stripe count = 4, stripe size = 1MB and all I/O > accesses are aligned with the stripe boundary. If a client reads every 4th > full stripe (accessing same OST always) then one stripe will be readahead > from that OST in every call? > > Thanks, > Arifa. > > -----Original Message----- > From: Andreas Dilger [mailto:andreas.dilger at oracle.com] > Sent: Thursday, July 29, 2010 11:41 PM > To: Arifa Nisar > Cc: lustre-discuss at lists.lustre.org > Subject: Re: [Lustre-discuss] Read ahead / prefetching > > On 2010-07-29, at 14:02, Arifa Nisar wrote: >> Thanks for the response. If an access pattern triggers page aligned > strided >> access then how many pages will be readahead? Are there some environment >> variables which may give insight about readahead behavior? > > Because Lustre is tuned for RPC-sized IO, it will always readahead at least > a full RPC at a time (by default 1MB), unless the application is reading > larger chunks than this, then it reads ahead in units of the IO size aligned > to RPC-sized boundaries. > >> -----Original Message----- >> From: Andreas Dilger [mailto:andreas.dilger at oracle.com] >> Sent: Wednesday, July 28, 2010 5:43 PM >> To: Arifa Nisar >> Cc: lustre-discuss at lists.lustre.org >> Subject: Re: [Lustre-discuss] Read ahead / prefetching >> >> On 2010-07-28, at 10:43, Arifa Nisar wrote: >>> I am trying to educate myself on prefetching/readahead algorithm for >>> Lustre''s read. For a starter I only have two simple questions. >>> >>> 1 - Does Lustre detect linear or random I/O pattern or it always triggers >>> readahead? >> >> The current algorithm detects sequential and strided read patterns. >> >>> 2 - If readahead is triggered, how many pages are read in addition to > what >>> is necessary? >> >> It is variable based on how long the same pattern was repeating and the >> number of pages per request. >> >> I would be happy to have you look at the strided readahead detection code >> more closely, since I suspect it would not correctly detect stride > patterns >> with odd byte size and offset vs. stride patterns with full-page size and >> offset. >> >> We disussed a while ago to rewrite the readahead code to be layered better >> on the client, but haven''t had time to work on it yet. >> >> >> Cheers, Andreas >> -- >> Andreas Dilger >> Lustre Technical Lead >> Oracle Corporation Canada Inc. >> > > > Cheers, Andreas > -- > Andreas Dilger > Lustre Technical Lead > Oracle Corporation Canada Inc. >
Hi Andrea, How can I know for sure that my Lustre''s deployment has this feature or not? Is it available in version 1.6.x or its only for 1.8.x? Thanks, Arifa. -----Original Message----- From: Andreas Dilger [mailto:andreas.dilger at oracle.com] Sent: Friday, July 30, 2010 1:32 AM To: Arifa Nisar Cc: lustre-discuss at lists.lustre.org Subject: Re: [Lustre-discuss] Read ahead / prefetching If the size of the read calls is 1MB then after 2 reads at 4MB strides the 3rd read would trigger readahead initially for the next stride (1MB+1MB). If the same stride pattern continues the readahead size will grow an additional 1MB each time up to the limit. Cheers, Andreas On 2010-07-29, at 22:53, Arifa Nisar <a-nisar at u.northwestern.edu> wrote:> Consider a case where stripe count = 4, stripe size = 1MB and all I/O > accesses are aligned with the stripe boundary. If a client reads every 4th > full stripe (accessing same OST always) then one stripe will be readahead > from that OST in every call? > > Thanks, > Arifa. > > -----Original Message----- > From: Andreas Dilger [mailto:andreas.dilger at oracle.com] > Sent: Thursday, July 29, 2010 11:41 PM > To: Arifa Nisar > Cc: lustre-discuss at lists.lustre.org > Subject: Re: [Lustre-discuss] Read ahead / prefetching > > On 2010-07-29, at 14:02, Arifa Nisar wrote: >> Thanks for the response. If an access pattern triggers page aligned > strided >> access then how many pages will be readahead? Are there some environment >> variables which may give insight about readahead behavior? > > Because Lustre is tuned for RPC-sized IO, it will always readahead atleast> a full RPC at a time (by default 1MB), unless the application is reading > larger chunks than this, then it reads ahead in units of the IO sizealigned> to RPC-sized boundaries. > >> -----Original Message----- >> From: Andreas Dilger [mailto:andreas.dilger at oracle.com] >> Sent: Wednesday, July 28, 2010 5:43 PM >> To: Arifa Nisar >> Cc: lustre-discuss at lists.lustre.org >> Subject: Re: [Lustre-discuss] Read ahead / prefetching >> >> On 2010-07-28, at 10:43, Arifa Nisar wrote: >>> I am trying to educate myself on prefetching/readahead algorithm for >>> Lustre''s read. For a starter I only have two simple questions. >>> >>> 1 - Does Lustre detect linear or random I/O pattern or it alwaystriggers>>> readahead? >> >> The current algorithm detects sequential and strided read patterns. >> >>> 2 - If readahead is triggered, how many pages are read in addition to > what >>> is necessary? >> >> It is variable based on how long the same pattern was repeating and the >> number of pages per request. >> >> I would be happy to have you look at the strided readahead detection code >> more closely, since I suspect it would not correctly detect stride > patterns >> with odd byte size and offset vs. stride patterns with full-page size and >> offset. >> >> We disussed a while ago to rewrite the readahead code to be layeredbetter>> on the client, but haven''t had time to work on it yet. >> >> >> Cheers, Andreas >> -- >> Andreas Dilger >> Lustre Technical Lead >> Oracle Corporation Canada Inc. >> > > > Cheers, Andreas > -- > Andreas Dilger > Lustre Technical Lead > Oracle Corporation Canada Inc. >