Btrfs devel - May 2013 - exploit preload daemon to defragment most needed files

Hey list,

Although this is about the preload daemon, my intended audience for this 
matter is the btrfs community. So I''m posting this here.

I''ve created a small script here[1] to read the preload daemon state
file
and use this to run the btrfs defragmenter/compressor on these files. The 
idea was to have some proof-of-concept to maybe integrate defragmentation 
capabilities directly into preload.

While I''d like to read some feedback from the community, maybe collect
some
more ideas, I already discovered one drawback which is a show-stopper: btrfs 
won''t defragment files currently in use (probably memory-mapped files
only
but these are a lot while a desktop env is running). I didn''t look into
it
any closer yet. And I didn''t run benchmarks. But my first impressions
are
that preloading is a bit faster and less io-intensive.

Regards,
Kai

[1]: https://gist.github.com/kakra/5520200

On Sun, May 05, 2013 at 11:42:24AM +0200, Kai Krakow
wrote:
> Hey list,
> 
> Although this is about the preload daemon, my intended audience for this 
> matter is the btrfs community. So I''m posting this here.
> 
> I''ve created a small script here[1] to read the preload daemon
state file
> and use this to run the btrfs defragmenter/compressor on these files. The 
> idea was to have some proof-of-concept to maybe integrate defragmentation 
> capabilities directly into preload.
  In similar spirit, systemd''s readhead defrag files on btrfs.
Here''s the commit:
http://cgit.freedesktop.org/systemd/systemd/commit/?id=746f89063666be8c4d538271c646585ac4fc4353


-- 
Tomasz Torcz               "Never underestimate the bandwidth of a station
xmpp: zdzichubg@chrome.pl    wagon filled with backup tapes." -- Jim Gray

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Tomasz Torcz <tomek@pipebreaker.pl> schrieb:
>> Although this is about the preload daemon, my intended audience for
this
>> matter is the btrfs community. So I''m posting this here.
>> 
>> I''ve created a small script here[1] to read the preload daemon
state file
>> and use this to run the btrfs defragmenter/compressor on these files.
The
>> idea was to have some proof-of-concept to maybe integrate
defragmentation
>> capabilities directly into preload.
> 
>   In similar spirit, systemd''s readhead defrag files on btrfs.
Here''s the
>   commit:
Yes! :-) Actually that is where I had my idea from. But preload takes on 
(boot finished, preload user''s enviroment) where readahead stops
(readahead
files needed by daemons, until about 10s after last daemon loaded) working 
with my files.

I think it is a great idea doing this kind of stuff in preloaders and 
"readaheaders".

If I understood it right, systemd readahead does not only just defrag the 
files, it relocates them into the order needed during boot. Actually, that 
seems pretty effective, as my system (no SSD) finishes booting all daemons 
within less than 10s (on mraid1/draid0 3 device btrfs, plain old harddisks, 
no SSD). Just lightdm/kdm then take ages to load. Still need to figure out 
why but that''s another story.

Regards,
Kai

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html