Heya, just a short blurb of what I understand from grokking dmu_zfetch.c Basically the current code issues a prefetch (ie. create a new prefecth stream) whenever a block (level 0, DB_RF_NOPREFETCH is not set) is is read in dbuf_read. Since ZFS is multi-threaded, we allow up to 8 such prefetch streams to be running concurrently. dmu_zfetch calls dmu_zfetch_find, which tries to categorize the access pattern based on the current prefetch streams into 1 of 4 patterns: 1. Forward sequential access 2. Backward sequential access 3. Forward sequential strided access (does this mean we skip blocks, but forwards?) 4. Backward sequential strided access (ditto...) The new prefetch stream being created is then optimized for 1 of the 4 cases. By default, we prefetch blocks in a forward manner: if the access pattern doesn''t match 1 of the 4 cases, or doesn''t always work well with a forward access pattern, then we suffer a potential performance loss, in terms of wastage of bandwidth and arc_cache space. Is this accurate? And what other performance pitfalls/corner cases are there currently? Thanks! :) -- Regards, Jeremy
Jeremy Teo wrote:> Heya, > > just a short blurb of what I understand from grokking dmu_zfetch.c > > Basically the current code issues a prefetch (ie. create a new > prefecth stream) whenever a block (level 0, DB_RF_NOPREFETCH is not > set) is is read in dbuf_read. > > Since ZFS is multi-threaded, we allow up to 8 such prefetch streams to > be running concurrently. > > dmu_zfetch calls dmu_zfetch_find, which tries to categorize the access > pattern based on the current prefetch streams into 1 of 4 patterns: > > 1. Forward sequential access > 2. Backward sequential access > 3. Forward sequential strided access (does this mean we skip blocks, > but forwards?) > 4. Backward sequential strided access (ditto...) > > The new prefetch stream being created is then optimized for 1 of the 4 > cases. > > By default, we prefetch blocks in a forward manner: if the access > pattern doesn''t match 1 of the 4 cases, or doesn''t always work well > with a forward access pattern, then we suffer a potential performance > loss, in terms of wastage of bandwidth and arc_cache space. > > Is this accurate? And what other performance pitfalls/corner cases are > there currently? >Basically correct... although I am not sure I understand what you mean by: "we prefetch blocks in a forward manner". We will prefetch blocks in a "backward manner" if we detect a reverse stride. But regardless of that, if the application does not actually need the blocks we are prefetching, then yes, we will waste IO bandwidth and cache space. -Mark
Group, Their seems to be three large single allocation sections within the zfs code, local buf_init(), that perform KMEM_NOSLEEPs. Based on failure they drop to 1/2 of value and retry until succeed. Would''t this type of logic force a low memory condition if they were set too high? IMO, this would allow these three allocs to consume basicly all of kmem alloc''able memory and generate no warning that NOSLEEP memory allocations had failed. The single kmem_sleeps could then consume the rest with large mem allocs and forced reallocations. Thus, IMO, since they originate at a single location, return based on failure, generate a BAD CONFIGURATION message, and retry with a medium amount of memory consumption, and if that failed, retry with a low ZFS memory consumption mode. On the other had if one wishes to repeat the failed allocations without medium or low memory set configs, dropping to a conservative 1/8th each time could also prevent low memory conditions.. Mitchell Erblich ------------------