Btrfs devel - Aug 2013 - Deduplication

Hello,

some questions regarding btrfs deduplication.

- What is the state of it? Is it "safe" to use?
https://btrfs.wiki.kernel.org/index.php/Deduplication does not yield
much information.

- https://pypi.python.org/pypi/bedup says: "bedup looks for new and
changed files, making sure that multiple copies of identical files
share space on disk. It integrates deeply with btrfs so that scans are
incremental and low-impact."

Is is file-based deduplcation only or is there also a block-based
mode? Does not seem so according to the docs.

If so, what is the advantage to numerous other tools that replace
duplicates by hard links? Performance?

Thanks,

Florian
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Hi,

On Thu, Aug 22, 2013 at 09:28:09PM +0200, Florian Lindner
wrote:
> Hello,
> 
> some questions regarding btrfs deduplication.
> 
> - What is the state of it? Is it "safe" to use?
> https://btrfs.wiki.kernel.org/index.php/Deduplication does not yield
> much information.
For inband dedup, it''s experimental because it introduces some format
changes.
> 
> - https://pypi.python.org/pypi/bedup says: "bedup looks for new and
> changed files, making sure that multiple copies of identical files
> share space on disk. It integrates deeply with btrfs so that scans are
> incremental and low-impact."
> 
> Is is file-based deduplcation only or is there also a block-based
> mode? Does not seem so according to the docs.
Both inband dedup and offband dedup patchset are floating on the list, and
inband is block-based, but I''m not sure about offband dedup.

-liubo
> 
> If so, what is the advantage to numerous other tools that replace
> duplicates by hard links? Performance?
> 
> Thanks,
> 
> Florian
> --
> To unsubscribe from this list: send the line "unsubscribe
linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On 23/08/2013 09:42, Liu Bo wrote:
> Hi,
>
> On Thu, Aug 22, 2013 at 09:28:09PM +0200, Florian Lindner wrote:
>> Hello,
>>
>> some questions regarding btrfs deduplication.
>>
>> - What is the state of it? Is it "safe" to use?
>> https://btrfs.wiki.kernel.org/index.php/Deduplication does not yield
>> much information.
>
> For inband dedup, it''s experimental because it introduces some
format changes.
>
In our tests we are trying btrfs for our internal storage backup backend 
and we like a lot compression! Now would like to test inband dedup.
Does those patchs (are v6 the last?) can be used on ubuntu lts? Do have 
a simple howt-to to follow?

Thanks,
Michele

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Fri, Aug 23, 2013 at 12:04:10PM +0200, mpe wrote:
> On 23/08/2013 09:42, Liu Bo wrote:
> >Hi,
> >
> >On Thu, Aug 22, 2013 at 09:28:09PM +0200, Florian Lindner wrote:
> >>Hello,
> >>
> >>some questions regarding btrfs deduplication.
> >>
> >>- What is the state of it? Is it "safe" to use?
> >>https://btrfs.wiki.kernel.org/index.php/Deduplication does not
yield
> >>much information.
> >
> >For inband dedup, it''s experimental because it introduces some
format changes.
> >
> 
> In our tests we are trying btrfs for our internal storage backup
> backend and we like a lot compression! Now would like to test inband
> dedup.
dedup can cope with compression, too.
> Does those patchs (are v6 the last?) can be used on ubuntu lts? Do
> have a simple howt-to to follow?
I didn''t try ubuntu lts yet, but the v6 patch set can be well applied
on
3.11-rc4.

To enable dedup, you need to
1) mount DEV /PATH_OF_MNT -o dedup (or -o dedup_bs=xxxK)
2) btrfs dedup register /PATH_OF_MNT
3) write your data now :)


Feel free to tell me if any questions, thanks.

-liubo
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html