I''ve been trying to test the snappy compression patches, but
I''m
getting corruptions when trying to use snappy as built on my system.
I''m checking out the Linux 3.2.6 kernel, merging that with the latest
''for-linus'' branch on Chris Mason''s kernel.org repo,
and then
integrating the snappy and lz4 patches from David Sterba''s git
repository (dev/compression-squad branch).
I''ve tried a simple merge of the dev/compression-squad branch (which
merged without complaint by git), and I also tried a second build of
the kernel integrating the dev/compression-squad branch patches with
format-patch (git format-patch -k -m -U5 --stdout <range> | git am -3
-k).
I''ve also tried sourcing my snappy patches from Chris Mason''s
snappy
branch on kernel.org with the same result.
I get the same results either way.
My system is a Core 2 Duo x86_64 Sabayon based system (Gentoo is our
parent distro). My target btrfs/snappy partition is a 16 GB partition
I use for testing on a 500GB Western Digital Hard Drive.
When I copy directory containing a kernel sources git repository to a
freshly formated partition mounted with snappy, I get a corrupted
copy. If I mount with lzo or lz4 compression, I don''t see any
corruptions from the copy.
I''m not showing any errors in dmesg.
My procedure to replicate is as follows:
mkfs.btrfs -m single /dev/sdb6
mount -o compress-force=snappy,autodefrag /dev/sdb6 /mnt/benchmark/
cp -a /var/tmp/portage/subdir1 /mnt/benchmark/
diff -Naur /var/tmp/portage/subdir1 /mnt/benchmark/subdir1 | less
I''ve spot checked the corrupted files with filefrag, and the
one''s
I''ve checked are *NOT* in-lined.
The following is a brief, representative excerpt of the diff (the full
diff is very long):
diff -Naur
/var/tmp/portage/subdir1/linux-btrfs-backport/drivers/net/bnx2x/bnx2x_reg.h
/mnt/benchmark/subdir1/linux-btrfs-backport/drivers/net/bnx2x/bnx2x_reg.h
--- /var/tmp/portage/subdir1/linux-btrfs-backport/drivers/net/bnx2x/bnx2x_reg.h
2011-01-12 16:31:39.000000000 -0600
+++ /mnt/benchmark/subdir1/linux-btrfs-backport/drivers/net/bnx2x/bnx2x_reg.h
2011-01-12 16:31:39.000000000 -0600
@@ -4051,76 +4051,73 @@
#define UCM_REG_TM_INIT_CRD 0xe021c
/* [RW 28] The CM header for Timers expiration command. */
#define UCM_REG_TM_UCM_HDR 0xe009c
-/* [RW 1] Timers - CM Interface enable. If 0 - the valid input is
+/* [RW Timimers - CM Interface enable. If 0 - the valid input is
disregarded; acknowledge output is deasserted; all other signals are
treated as usual; if 1 - normal activity. */
-#define UCM_REG_TM_UCM_IFEN 0xe001c
-/* [RW 3] The weight of the Timers input in the WRR mechanism. 0 stands for
- weight 8 (the most prioritised); 1 stands for weight 1(least
- prioritised); 2 stands for weight 2; tc. */
+#define UCM_REG_TM_UCM_IFEN 0xe00
+/* * [RW 3] The weight of the mers input i in the WRR mechanism. 0 stands for
+ ightht 8 (the most prioritised); 1 stands foreighght 1(least
+ ioritised); ; 2 stands for weight 2; tc. */
#define UCM_REG_TM_WEIGHT 0xe00d4
-/* [RW 1] Input tsem Interface enable. If 0 - the valid input is
+/* [RW TiInput tsem Interface enable. If 0 - the valid input is
disregarded; acknowledge output is deasserted; all other signals are
treated as usual; if 1 - normal activity. */
-#define UCM_REG_TSEM_IFEN 0xe0024
-/* [RC 1] Set when the message length mismatch (relative to last indication)
- at the tsem interface is detected. */
+#define UCM_REG_TSEM_IFEN 0xe00
+/* * [RC 1] Set when the message length mismatch (relative to last indication)
+ thehe tsem terfrface is detected. */
#define UCM_REG_TSEM_LENGTH_MIS 0xe015c
Here''s a list of the top commits in my git repository:
54ca0b5 btrfs: lz4: tune speed/compression ratio
7ead5fb btrfs: lz4: add wrapper functions and enable it
3a611a7 btrfs: lz4: add wrapper for context size estimation
ae2de6b btrfs: add LZ4 compression method
56e0aa6 btrfs: prepare incompat flags for more compression methods
540f832 Btrfs: fix decompressing of snappy-compressed inline extents
8f8b19f8 Btrfs: fix incompat flags setting
369cc38 SNAPPY: Add dual GPL/BSD license to snappy module
b479967 Add snappy interface to crypto API
e68c69a BTRFS: Add snappy support v2
428ac51 Add the snappy-c compressor to lib v2
4d0defa Merge branch ''for-linus'' of
git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs into
linux-3.2.6-btrfs-lz4-v6.3
c2db2e2 Linux 3.2.6
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Mitch Harder posted on Wed, 15 Feb 2012 16:46:00 -0600 as excerpted:> I''ve been trying to test the snappy compression patches, but I''m getting > corruptions when trying to use snappy as built on my system. > > I''m checking out the Linux 3.2.6 kernel, merging that with the latest > ''for-linus'' branch on Chris Mason''s kernel.org repo, and then > integrating the snappy and lz4 patches from David Sterba''s git > repository (dev/compression-squad branch).Given that we''re past the 3.3 kernel merge window, wouldn''t Chris''s "for- linus" branch be based on the patches that went into 3.3, now? IOW, you''re basing on 3.2.x but AFAIK you should be basing on 3.3, now, so you''re missing the 3.3 patches which CM''s for-linus tree branch should be assuming, at this point. Unless you know otherwise of course. I run the mainline tree here, and don''t actually run anything btrfs yet, as once I started investigating I realized that it''s missing features (like multi-mirror not just dual- mirror) that I need, and isn''t yet as stable as I had hoped, either. So I don''t know that much about the specifics of CM''s tree, but in the general case, a for-linus branch would be based on 3.3 now, since it''s well past the merge window. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Feb 15, 2012 at 5:03 PM, Duncan <1i5t5.duncan@cox.net> wrote:> Mitch Harder posted on Wed, 15 Feb 2012 16:46:00 -0600 as excerpted: > >> I''ve been trying to test the snappy compression patches, but I''m getting >> corruptions when trying to use snappy as built on my system. >> >> I''m checking out the Linux 3.2.6 kernel, merging that with the latest >> ''for-linus'' branch on Chris Mason''s kernel.org repo, and then >> integrating the snappy and lz4 patches from David Sterba''s git >> repository (dev/compression-squad branch). > > Given that we''re past the 3.3 kernel merge window, wouldn''t Chris''s "for- > linus" branch be based on the patches that went into 3.3, now? IOW, > you''re basing on 3.2.x but AFAIK you should be basing on 3.3, now, so > you''re missing the 3.3 patches which CM''s for-linus tree branch should be > assuming, at this point. >Chris''s ''for-linus'' branch is currently based on 3.2 (but not 3.2.6). -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Mitch Harder wrote:> I''ve been trying to test the snappy compression patches, but I''m > getting corruptions when trying to use snappy as built on my system. > > I''m checking out the Linux 3.2.6 kernel, merging that with the latest > ''for-linus'' branch on Chris Mason''s kernel.org repo, and then > integrating the snappy and lz4 patches from David Sterba''s git > repository (dev/compression-squad branch). > > I''ve tried a simple merge of the dev/compression-squad branch (which > merged without complaint by git), and I also tried a second build of > the kernel integrating the dev/compression-squad branch patches with > format-patch (git format-patch -k -m -U5 --stdout <range> | git am -3 > -k). > > I''ve also tried sourcing my snappy patches from Chris Mason''s snappy > branch on kernel.org with the same result. >A month ago I saved emails as patches and applied them on linux-btrfs'' for-linus branch, and I got the same problem. Then I used the snappy branch (plus my fix), and it was fine. Don''t know why.> I get the same results either way. > > My system is a Core 2 Duo x86_64 Sabayon based system (Gentoo is our > parent distro). My target btrfs/snappy partition is a 16 GB partition > I use for testing on a 500GB Western Digital Hard Drive. > > When I copy directory containing a kernel sources git repository to a > freshly formated partition mounted with snappy, I get a corrupted > copy. If I mount with lzo or lz4 compression, I don''t see any > corruptions from the copy. > > I''m not showing any errors in dmesg. >You''re not seeing any errors, right? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Feb 15, 2012 at 8:06 PM, Li Zefan <lizf@cn.fujitsu.com> wrote:> Mitch Harder wrote: >> When I copy directory containing a kernel sources git repository to a >> freshly formated partition mounted with snappy, I get a corrupted >> copy. If I mount with lzo or lz4 compression, I don''t see any >> corruptions from the copy. >> >> I''m not showing any errors in dmesg. >> > > You''re not seeing any errors, right?That is correct. I''m not seeing any errors thrown out directly from the copy command, nor am I seeing anything else in the dmesg log. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Feb 16, 2012 at 10:06:13AM +0800, Li Zefan wrote:> Mitch Harder wrote: > > I''ve been trying to test the snappy compression patches, but I''m > > getting corruptions when trying to use snappy as built on my system. > > > > I''ve also tried sourcing my snappy patches from Chris Mason''s snappy > > branch on kernel.org with the same result. > > > > A month ago I saved emails as patches and applied them on linux-btrfs'' > for-linus branch, and I got the same problem. Then I used the snappy > branch (plus my fix), and it was fine. Don''t know why.The decompression corruptions Mitch reported look same as what you sent. And I don''t think they are caused by misapplied patches, I''ve checked both ''mbox via mutt'' -> git or pull from Chris'' branch, + your patches to fix the inlined extents, no other changes. I did this test: * mkfs * mount with compress-force=snappy * copy linux-3.2 * drop caches (or umount/mount) * check md5sums from original Result: md5sum: WARNING: 32986 computed checksums did NOT match there are 37617 files in linux-3.2/ syslog says just: [ 728.744179] Btrfs loaded [ 728.746332] device fsid 194bbb84-1f87-4a37-8091-957d557633f1 devid 1 transid 4 /dev/sda9 [ 728.747693] btrfs: enabling auto defrag [ 728.747698] btrfs: enabling inode map caching [ 728.747702] btrfs: use snappy compression [ 728.747704] btrfs: disk space caching is enabled> > My system is a Core 2 Duo x86_64 Sabayon based system (Gentoo is our > > parent distro). My target btrfs/snappy partition is a 16 GB partition > > I use for testing on a 500GB Western Digital Hard Drive.I use the same testbox and disk as always, no related hw problems in syslog. david -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Just to add another data point, I built Chris Mason''s ''snappy'' branch from git.kernel.org as-is (a 3.2.0 kernel plus the snappy patches) with no additional patches. This 3.2.0 snappy kernel also exhibits the problem copying the kernel sources to a snappy-compressed partition. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Feb 15, 2012 at 04:46:00PM -0600, Mitch Harder wrote:> I''ve been trying to test the snappy compression patches, but I''m > getting corruptions when trying to use snappy as built on my system.I went through the original C++ implementation (svn release r52) and compared it to the C implementation. I did not see any problems with it, it''s direct transformation from C++ -> C, nothing like off-by-one somewhere or typos in condition expressions. No clue where the corruption comes from. david -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html