Hi, While trying to setup xapian replication (initially for backup purposes), I'm encountering some errors. Our "fresh" index starts replication, and ends up with an index size that matches the replication master (4.5GB), but then throws : "Getting update for fresh from fresh xapian-replicate: NetworkError: Unable to fully synchronise: Database changing too fast" I tried running xapian-replicate in both one-shot and continous modes, but that didn't change anything. For our larger but not that frequently modified index I instead get this error : "Getting update for index from index xapian-replicate: NetworkError: Insane message length specified!" The replica set never gets close to the master index size in this case (Master is 63GB, largest I've seen the replica get to was around 30GB). What can be the cause of the insane message length? This is on Debian Wheezy and the versions we're running are: ii libsearch-xapian-perl 1.2.10.0-1 ii libxapian22 1.2.12-1 ii xapian-tools 1.2.12-1 Any clues? I did try to google the messages extensively, and tried searching Gmane archives, but I might have missed out on relevant threads. If so I am sorry. Best Regards -- Denis Braekhus
On 11/14/2012 04:25 PM, Denis Br?khus wrote:> Hi, > > While trying to setup xapian replication (initially for backup > purposes), I'm encountering some errors. > > Our "fresh" index starts replication, and ends up with an index size > that matches the replication master (4.5GB), but then throws : > > "Getting update for fresh from fresh > xapian-replicate: NetworkError: Unable to fully synchronise: Database > changing too fast"Sorry about this, some more googling points me toward not having setup our indexer with changesets.> "Getting update for index from index > xapian-replicate: NetworkError: Insane message length specified!"This however I am still stumped by.. Regards -- Denis Br?khus Driftssjef ABC Startsiden AS
The reason why Master has 63GB and Replica has only 30GB is because deleted documents from Master are not replicated to Replica. In Master you have all documents deleted and not deleted, only not deleted, documents are sent to Replica which contains only active documents. Kevin Duraj http://myhealthcare.com On Nov 14, 2012, at 7:25 AM, Denis Br?khus <denis at startsiden.no> wrote:> Hi, > > While trying to setup xapian replication (initially for backup purposes), I'm encountering some errors. > > Our "fresh" index starts replication, and ends up with an index size that matches the replication master (4.5GB), but then throws : > > "Getting update for fresh from fresh > xapian-replicate: NetworkError: Unable to fully synchronise: Database changing too fast" > > I tried running xapian-replicate in both one-shot and continous modes, but that didn't change anything. > > > For our larger but not that frequently modified index I instead get this error : > > "Getting update for index from index > xapian-replicate: NetworkError: Insane message length specified!" > > The replica set never gets close to the master index size in this case (Master is 63GB, largest I've seen the replica get to was around 30GB). > > > What can be the cause of the insane message length? > > This is on Debian Wheezy and the versions we're running are: > ii libsearch-xapian-perl 1.2.10.0-1 > ii libxapian22 1.2.12-1 > ii xapian-tools 1.2.12-1 > > Any clues? I did try to google the messages extensively, and tried searching Gmane archives, but I might have missed out on relevant threads. If so I am sorry. > > > Best Regards > -- > Denis Braekhus > > _______________________________________________ > Xapian-discuss mailing list > Xapian-discuss at lists.xapian.org > http://lists.xapian.org/mailman/listinfo/xapian-discuss
On Fri, Nov 16, 2012 at 12:19:10PM -0800, Kevin Duraj wrote:> The reason why Master has 63GB and Replica has only 30GB is because > deleted documents from Master are not replicated to Replica. > > In Master you have all documents deleted and not deleted, only not > deleted, documents are sent to Replica which contains only active > documents.This is incorrect. Replication works at the block level. Cheers, Olly
Olly, Then our Xapian implementation is incorrect and we need to correct it. Only not deleted document should be replicated. Cheers, Kevin Duraj http://myhealthcare.com/ On Fri, Nov 16, 2012 at 1:02 PM, Olly Betts <olly at survex.com> wrote:> On Fri, Nov 16, 2012 at 12:19:10PM -0800, Kevin Duraj wrote: > > The reason why Master has 63GB and Replica has only 30GB is because > > deleted documents from Master are not replicated to Replica. > > > > In Master you have all documents deleted and not deleted, only not > > deleted, documents are sent to Replica which contains only active > > documents. > > This is incorrect. Replication works at the block level. > > Cheers, > Olly >