search for: 51gb

Forgive me if this has been addressed here before. We have a remote office that we need to backup to our NAS. We have a site to site certificate VPN. The remote site has over 51gb that needs to be backed up to our NAS over that VPN. I have tried this command: rsync --verbose --progress --stats --compress --recursive --times --perms --links --delete /Share/* / smb://192.168.123.6/Backup/EdensLandCorp and it just sits there and appears to do nothing. Does rsync make a tarball...

...off at about > 100,000 records an hour. This is consistent with the roughly 24 hour > rebuild time I was experiencing with Xapian 1.2.21 (chert). However, > after 75 hours of build time, the index is about 75% complete and > records are processing at a rate of 10,000/hr. The index is 51GB is > size, 30GB is position.glass. One of the big differences between chert and glass is that glass stores positional data in a different order such that phrase searches are much more I/O efficient. The downside is that this means extra work at index time, and more data to batch up in memory...

Hi Olly, Thanks for the detailed response. I hadn’t realized there was a new xapian haystack backend. I’m going to try that but I have some upgrades to do first. Django 1.8, etc. Thanks, Ryan > On Feb 28, 2017, at 3:40 PM, Olly Betts <olly at survex.com> wrote: > > On Mon, Feb 27, 2017 at 10:29:46AM -0800, Ryan Cross wrote: >> I am trying to rebuild an index of 2+

...t; 100,000 records an hour. This is consistent with the roughly 24 hour > > rebuild time I was experiencing with Xapian 1.2.21 (chert). However, > > after 75 hours of build time, the index is about 75% complete and > > records are processing at a rate of 10,000/hr. The index is 51GB is > > size, 30GB is position.glass. > > One of the big differences between chert and glass is that glass stores > positional data in a different order such that phrase searches are much > more I/O efficient. The downside is that this means extra work at index > time, and m...

...performance starts off at about 100,000 records an hour. This is consistent with the roughly 24 hour rebuild time I was experiencing with Xapian 1.2.21 (chert). However, after 75 hours of build time, the index is about 75% complete and records are processing at a rate of 10,000/hr. The index is 51GB is size, 30GB is position.glass. Here is a one minute strace summary % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 63.97 1.272902 13 100240 pread 33.71 0.670733 14 48175...