search for: deduping

Displaying 20 results from an estimated 265 matches for "deduping".

Did you mean: deducing
2012 Jul 30
10
encfs on top of zfs
Dear ZFS-Users, I want to switch to ZFS, but still want to encrypt my data. Native Encryption for ZFS was added in "ZFS Pool Version Number 30<http://en.wikipedia.org/wiki/ZFS#Release_history>", but I''m using ZFS on FreeBSD with Version 28. My question is how would encfs (fuse encryption) affect zfs specific features like data Integrity and deduplication? Regards
2009 Dec 15
7
ZFS Dedupe reporting incorrect savings
Hi, Created a zpool with 64k recordsize and enabled dedupe on it. zpool create -O recordsize=64k TestPool device1 zfs set dedup=on TestPool I copied files onto this pool over nfs from a windows client. Here is the output of zpool list Prompt:~# zpool list NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT TestPool 696G 19.1G 677G 2% 1.13x ONLINE - When I ran a
2011 Jan 28
8
ZFS Dedup question
I created a zfs pool with dedup with the following settings: zpool create data c8t1d0 zfs create data/shared zfs set dedup=on data/shared The thing I was wondering about was it seems like ZFS only dedup at the file level and not the block. When I make multiple copies of a file to the store I see an increase in the deup ratio, but when I copy similar files the ratio stays at 1.00x. -- This
2010 Aug 18
10
Networker & Dedup @ ZFS
Hi, We are considering using a ZFS based storage as a staging disk for Networker. We''re aiming at providing enough storage to be able to keep 3 months worth of backups on disk, before it''s moved to tape. To provide storage for 3 months of backups, we want to utilize the dedup functionality in ZFS. I''ve searched around for these topics and found no success stories,
2010 May 15
7
Unable to Destroy One Particular Snapshot
Howdy All, I''ve a bit of a strange problem here. I have a filesystem with one snapshot that simply refuses to be destroyed. The snapshots just prior to it and just after it were destroyed without problem. While running the zfs destroy command on this particular snapshot, the server becomes more-or-less hung. It''s pingable but will not open a new shell (local or via ssh) however
2013 Jun 26
6
[PROGS PATCH] Import btrfs-extent-same
...gv) +{ + int ret, src_fd, i, numfiles; + char *srcf, *destf; + struct btrfs_ioctl_same_args *same; + struct btrfs_ioctl_same_extent_info *info; + unsigned long long bytes = 0ULL; + + if (argc < 6 || (argc % 2)) { + usage(argv[0]); + return 1; + } + + numfiles = (argc / 2) - 2; + + printf("Deduping %d total files\n", numfiles + 1); + + same = calloc(1, + sizeof(struct btrfs_ioctl_same_args) + + sizeof(struct btrfs_ioctl_same_extent_info) * numfiles); + if (!same) + return -ENOMEM; + + srcf = argv[2]; + same->length = atoll(argv[1]); + same->logical_offset = atoll(arg...
2011 Aug 10
9
zfs destory snapshot takes an hours.
Hi, I am facing issue with zfs destroy, this takes almost 3 Hours to delete the snapshot of size 150G. Could you please help me to resolve this issue, why zfs destroy takes this much time. While taking snapshot, it''s done within few seconds. I have tried with removing with old snapshot but still problem is same. =========================== I am using : Release : OpenSolaris
2011 Jan 05
52
Offline Deduplication for Btrfs
Here are patches to do offline deduplication for Btrfs. It works well for the cases it''s expected to, I''m looking for feedback on the ioctl interface and such, I''m well aware there are missing features for the userspace app (like being able to set a different blocksize). If this interface is acceptable I will flesh out the userspace app a little more, but I believe the
2011 Apr 28
4
Finding where dedup''d files are
Is there an easy way to find out what datasets have dedup''d data in them. Even better would be to discover which files in a particular dataset are dedup''d. I ran # zdb -DDDD which gave output like: index 1055c9f21af63 refcnt 2 single DVA[0]=<0:1e274ec3000:2ac00:STD:1> [L0 deduplicated block] sha256 uncompressed LE contiguous unique unencrypted 1-copy size=20000L/20000P
2010 Jun 18
1
Question : Sun Storage 7000 dedup ratio per share
Dear All : Under Sun Storage 7000 system, can we see per share ratio after enable dedup function ? We would like deep to see each share dedup ratio. On Web GUI, only show dedup ratio entire storage pool. Thanks a lot, -- Rex -- This message posted from opensolaris.org
2013 Apr 01
5
[RFC] Online dedup for Btrfs
...tions). This creates a btree indexed by the hash and the bytenr. Then we lookup the hash and just link the extent in if it matches the hash. You can use -o paranoid-dedup if you are paranoid about hash collisions and this will force it to do the memcmp() dance to make sure that the extent we are deduping really matches the extent. So performance wise obviously the compat mode sucks. It''s about 50% slower on disk and about 20% slower on my Fusion card. We get pretty good space savings, about 10% in my horrible test (just copy a git tree onto the fs), but IMHO not worth the performance hi...
2010 Sep 25
4
dedup testing?
Hi all Has anyone done any testing with dedup with OI? On opensolaris there is a nifty "feature" that allows the system to hang for hours or days if attempting to delete a dataset on a deduped pool. This is said to be fixed, but I haven''t seen that myself, so I''m just wondering... I''ll get a 10TB test box released for testing OI in a few weeks, but before
2009 Dec 30
3
what happens to the deduptable (DDT) when you set dedup=off ???
I tried the deduplication feature but the performance of my fileserver dived from writing 50MB/s via CIFS to 4MB/s. what happens to the deduped blocks when you set dedup=off? are they written back to disk? is the deduptable deleted or is it still there? thanks -- This message posted from opensolaris.org
2010 Jan 27
13
zfs destroy hangs machine if snapshot exists- workaround found
Hi, I was suffering for weeks from the following problem: a zfs dataset contained an automatic snapshot (monthly) that used 2.8 TB of data. The dataset was deprecated, so I chose to destroy it after I had deleted some files; eventually it was completely blank besides the snapshot that still locked 2.8 TB on the pool. ''zfs destroy -r pool/dataset'' hung the machine within seconds
2009 Dec 08
5
Deduplication - deleting the original
In reading this blog post: http://blogs.sun.com/bobn/entry/taking_zfs_deduplication_for_a a question came to mind..... To understand the context of the question, consider the opening paragraph from the above post; Here is my test case: I have 2 directories of photos, totaling about 90MB > each. And here''s the trick - they are almost complete duplicates of each > other. I downloaded
2011 Feb 12
1
existing performance data for on-disk dedup?
Hello. I am looking to see if performance data exists for on-disk dedup. I am currently in the process of setting up some tests based on input from Roch, but before I get started, thought I''d ask here. Thanks for the help, Janice
2013 Aug 22
3
Deduplication
Hello, some questions regarding btrfs deduplication. - What is the state of it? Is it "safe" to use? https://btrfs.wiki.kernel.org/index.php/Deduplication does not yield much information. - https://pypi.python.org/pypi/bedup says: "bedup looks for new and changed files, making sure that multiple copies of identical files share space on disk. It integrates deeply with btrfs so
2006 Dec 19
2
Dedupping Has_many through, :unique=>true
Hi, In the Agile book, it is told that by putting a :unique => true will dedup the row with ActiveRecord. But it''s not working out for me. Do I need edge rails for this? I simply want to dedup any join model associations, for instance: category_id | inventory_id 384 1 first entry 384 2 this would be ok. 384 1 this would
2009 Jun 16
2
dedup in dovecot?
...system correctly, usually an MTA is calling dovecot on every single message, meaning the message itself won't necessarily be a duplicate in terms of headers (BCCs and so on). Would it be possible to dedup the content in terms of using separate files for the header and content and then deduping this with hard or symbolic links? Sorry if this is an old question, but I couldn't find anything about it. roy -- Roy Sigurd Karlsbakk (+47) 97542685 roy at karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et element...
2010 Mar 02
2
dedup source code
Hello ZFS experts: I would like to study ZFS de-duplication feature. Can someone please let me know which directory/files I should be looking at? Thanks in advance. -- This message posted from opensolaris.org