Hi there, Providing compression and/or encryption at the file system layer has proven to be contentious and polemic, at least in the linux kernel world. However, I would like to probe you guys (specially Chis Mason) about your thoughts on the issue. I have seen quite a few benchs were compression provides better performance, since it reduces the amount of IO. It seems to be a cpu/io trade-off. And given that cpu power is more easily accessible than fast IO.. it is starting to look a good idea that should be more deeply explored. Is compression for data on BTRFS a feature that is: - completely out of the table - it will not happen at the BTRFS layer - considered, but no efforts on developing this feature are planed. - considered, and design decisions are contemplating it.. but no code yet.. not this soon. (not for 1.0) Kind regards! -- Miguel Sousa Filipe -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Miguel Sousa Filipe wrote:> Hi there, > > Providing compression and/or encryption at the file system layer has > proven to be contentious and polemic, at least in the linux kernel > world. > > However, I would like to probe you guys (specially Chis Mason) about > your thoughts on the issue. > I have seen quite a few benchs were compression provides better > performance, since it reduces the amount of IO. > It seems to be a cpu/io trade-off. And given that cpu power is more > easily accessible than fast IO.. it is starting to look a good idea > that should be more deeply explored. > > Is compression for data on BTRFS a feature that is: > - completely out of the table - it will not happen at the BTRFS layer > - considered, but no efforts on developing this feature are planed. > - considered, and design decisions are contemplating it.. but no code > yet.. not this soon. (not for 1.0) > > > Kind regards! > >One big issue with compression is that it is very workload dependent. For example, if you are storing lots of video, sound or image files, the format itself it normally already compressed so you end up doing nothing but burning CPU cycles ;-) Also, for very small files, compression works best when you bunch them up (say compress the contents of a subdirectory). Also note that some storage devices can do compression for you beneath the file system or do other fancy features like block level data de-duplication. ric -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, 2008-06-11 at 12:13 +0100, Miguel Sousa Filipe wrote:> Hi there, > > Providing compression and/or encryption at the file system layer has > proven to be contentious and polemic, at least in the linux kernel > world. > > However, I would like to probe you guys (specially Chis Mason) about > your thoughts on the issue. > I have seen quite a few benchs were compression provides better > performance, since it reduces the amount of IO. > It seems to be a cpu/io trade-off. And given that cpu power is more > easily accessible than fast IO.. it is starting to look a good idea > that should be more deeply explored. > > Is compression for data on BTRFS a feature that is: > - completely out of the table - it will not happen at the BTRFS layer > - considered, but no efforts on developing this feature are planed. > - considered, and design decisions are contemplating it.. but no code > yet.. not this soon. (not for 1.0)Compression is definitely on the table, although it probably won''t be in in until after 1.0 unless someone volunteers. There are lots of different ways to code compression, and some are much harder than others. If you restrict the compression support to only files small enough to have their data packed into the btree, things get much much easier. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html