search for: deduplicate

Hello, I'm trying to create automated backup recovery using "doveadm import" and "doveadm deduplicate". During testing I noticed that deduplicate only deletes some duplicates and has to be called multiple times to find them all. Here's what I've been trying (in shell commands): First, expunge inbox (the end result is the same even if you delete only some messages): # doveadm expun...

On 13.2.2015 16:59, Kevin Laurie wrote: > Dear Jiri, > > I tried the following to try to get the inbox deduplicated. My inbox is > quite large and urgently need to remove the duplicated messages. > Is there an easy way to do this? > Sorry for being so presistant but I need help. > > The command I tried:- > > doveadm deduplicate -u user at domain.net mailbox inbox > I'd try this (i...

...> >> Hello, >> I just migrated my emails from gmail using getmail. >> In the process I got some emails that have been doubled or tripled >> How do I run the doveadm command to delete copies of same emails? >> >> I tried running the following:- >> doveadm deduplicate -u user at domain.net inbox >> >> but I get error:- >> doveadm(root): Fatal: Unknown argument INBOX >> >> Could someone share some way to automatically remove duplicated messages? >> >> Thanks >> Kevin >> >> > See documentation - man do...

...On Sat, Dec 16, 2017 at 4:08 AM George Rimar <grimar at accesssoftek.com<mailto:grimar at accesssoftek.com>> wrote: >Wasn't our (lld/ELF's) position on debug info size that we should focus on providing a great split-dwarf workflow and not try go too far out of our way to deduplicate >or otherwise reduce debug info size inside LLD? I recall there being some patches that made linking of large debug binaries like 1.5GB+ clang faster, but we decided to >reject those changes because split-dwarf was the "right" solution. > >Rafael, Rui? > >(I even recall...

Hi list, is there any working solution for deduplication of data for centos? We are trying to find a solution for our backup server which runs a bash script invoking xdelta(3). But having this functionality in fs is much more friendly... We have looked into lessfs, sdfs and ddar. Are these filesystems ready to use (on centos)? ddar is sthg different, I know. Thx Rainer

I have deduplication active in my first mdbox: type mailbox, but how do I find out how well the deduplication works? Is there a way of finding out how much disk space I saved (if I saved some :) )? -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax:

>Wasn't our (lld/ELF's) position on debug info size that we should focus on providing a great split-dwarf workflow and not try go too far out of our way to deduplicate >or otherwise reduce debug info size inside LLD? I recall there being some patches that made linking of large debug binaries like 1.5GB+ clang faster, but we decided to >reject those changes because split-dwarf was the "right" solution. > >Rafael, Rui? > >(I even recall...

...potential number of duplicated blocks at the given block size. For example: $ guestfish -N fs:vfat -m /dev/sda1 tgz-in /tmp/nbdkit-1.21.10.tar.gz / $ ./dedup.pl test1.img 4096 disk size = 1073741824 blocks of zeros: 259699 / 262144 99.1% blocks that could not be deduplicated: 2336 / 262144 0.9% number of duplicates: 22 count: 1 0.0% number of duplicates: 11 count: 2 0.0% number of duplicates: 9 count: 1 0.0% number of duplicates: 8 count: 1 0.0% number of duplicates: 6 count: 1 0.0% number of duplicates: 3 count: 2 0.0%...

Hello, I just migrated my emails from gmail using getmail. In the process I got some emails that have been doubled or tripled How do I run the doveadm command to delete copies of same emails? I tried running the following:- doveadm deduplicate -u user at domain.net inbox but I get error:- doveadm(root): Fatal: Unknown argument INBOX Could someone share some way to automatically remove duplicated messages? Thanks Kevin

Have you considered to introduce a "deduplicate mode" for rsync, replacing duplicate files in the destination directory by hard links? Of course there might be a lot of problems together with this feature, but on creating backups it could help to save a lot of disk space. --link-dest might be interesting in this context, too. Regards Ha...

Hi: I am very interested in the ZFS deduplication, where can I get the the source code? Or would anybody send me an actual source code achive which contains the ZFS deduplication? I am looking forward your kind help. Thanks very much. Best Rgards -- Yujuan Tan, PhD Candidate Wuhan National Laboratory for Optoelectronics, Wuhan, China Key Lab of Information Storage, Ministry of Education,

Hello, some questions regarding btrfs deduplication. - What is the state of it? Is it "safe" to use? https://btrfs.wiki.kernel.org/index.php/Deduplication does not yield much information. - https://pypi.python.org/pypi/bedup says: "bedup looks for new and changed files, making sure that multiple copies of identical files share space on disk. It integrates deeply with btrfs so

Dear Jiri, I tried the following to try to get the inbox deduplicated. My inbox is quite large and urgently need to remove the duplicated messages. Is there an easy way to do this? Sorry for being so presistant but I need help. The command I tried:- doveadm deduplicate -u user at domain.net mailbox inbox On Fri, Feb 13, 2015 at 11:40 PM, Kevin Laurie <superin...

Hi all, I'm running dovecot 2.3.7.1, with mdbox and single instance storage. I use offline deduplication like this: mail_attachment_fs = sis-queue /mailboxes/%d/_attachments_/queue:posix Each week I run doveadm sis deduplicate to deduplicate the mail attachments. Everything was fine until I mistakenly run doveadm sis deduplicate twice, simultaneously. I ended up having some error messages like this: Error: unlink(/mailboxes/domain.com/_attachments_/queue/067518ec61f872420813727326dde0933c7449e4-11ae16131c3f445d7e030000...

...set straight. * I understand that I can convert an ext4 filesystem to btrfs. Will such conversion work with an almost full ext4 filesystem? How much overhead will be needed to perform the conversion? I can (temporarily) remove some files that already are on this backup. * Is it possible to deduplicate the files that are already in it? As mentioned before, there are likely to be many, and some of them are on the order of 1 to 2GBs. * Doing a defragmentation with the filesystem mounted with compression will recompress the files (if they are deemed compressible by the filesystem). Is that...

Hi All: We have been looking at implementing deduplication on a backup server. >From what I have been able to find the available documentation is pretty thin. I ended up trying to install LessFS on this CentOS 5.7 box but we have now encountered problems with fuse version. Has anyone out there been able to get LessFS running on CentOS 5.7 and can provide some pointers? If not LessFS can you

Will the material ever be posted. Looks there is some huge bugs with zfs deduplication that the organizers do not want to post it also there is no indication on sun website if there will be a deduplication feature. I think its best they concentrate on improving zfs performance and speed with compression enabled. -------------- next part -------------- An HTML attachment was scrubbed... URL:

...gt; > On Sat, Dec 16, 2017 at 4:08 AM George Rimar <grimar at accesssoftek.com> > wrote: > >> >Wasn't our (lld/ELF's) position on debug info size that we should focus >> on providing a great split-dwarf workflow and not try go too far out of our >> way to deduplicate >or otherwise reduce debug info size inside LLD? I >> recall there being some patches that made linking of large debug binaries >> like 1.5GB+ clang faster, but we decided to >reject those changes because >> split-dwarf was the "right" solution. >> > >&...

Hi, I'm currently running GlusterFS over XFS, and it works quite well. I'm wondering if it's possible to add data deduplication into the mix by: glusterfs --> lessfs --> xfs or glusterfs --> opendedupe --> xfs Has anybody tried doing this? We're running VM images on gluster, and I figure we could get a bit of space saving bu deduplicating the data. Gerald

>Not quite sure what you mean by "on linker side" - but I guess you mean using linker features like comdats etc, rather than DWARF parsing/reassembly/etc. I mean that it probably not a good idea for external library. I feel it is much more convinent to do such proccessing in a linker. Linker do and knows much more about things like sections that are ICFed, eliminated, about COMDATs