Hello, I've configured a 6 boxs cluster with DHT and replicate. Each machine creates locally lucene's indexes and then they are copied to the gulester storage. Now I've got this: -????????? ? ? ? ? ? _0.fdt -????????? ? ? ? ? ? _0.fdx -????????? ? ? ? ? ? _0.fnm -????????? ? ? ? ? ? _0.frq -????????? ? ? ? ? ? _0.nrm -????????? ? ? ? ? ? _0.prx -????????? ? ? ? ? ? _0.tii -????????? ? ? ? ? ? _0.tis -????????? ? ? ? ? ? _0.tvd -????????? ? ? ? ? ? _0.tvf -????????? ? ? ? ? ? _0.tvx -rw-r--r-- 1 toor adm 4 2010-01-31 23:15 _1.fdt -rw-r--r-- 1 toor adm 4 2010-01-31 23:15 _1.fdx -rw-r--r-- 1 toor adm 526 2010-01-31 23:15 _1.fnm -rw-r--r-- 1 toor adm 0 2010-01-31 23:15 _1.frq -rw-r--r-- 1 toor adm 4 2010-01-31 23:15 _1.nrm -rw-r--r-- 1 toor adm 0 2010-01-31 23:15 _1.prx -rw-r--r-- 1 toor adm 24 2010-01-31 23:15 _1.tii -rw-r--r-- 1 toor adm 24 2010-01-31 23:15 _1.tis -rw-r--r-- 1 toor adm 4 2010-01-31 23:15 _1.tvd -rw-r--r-- 1 toor adm 4 2010-01-31 23:15 _1.tvf -rw-r--r-- 1 toor adm 4 2010-01-31 23:15 _1.tvx -????????? ? ? ? ? ? segments_2 -rw-r--r-- 1 toor adm 58 2010-01-31 23:15 segments_4 -rw-r--r-- 1 toor adm 20 2010-01-31 23:15 segments.gen -rw-r--r-- 1 toor adm 20 2010-01-31 23:15 segments.gen I can't delete this files. -- Roberto Franchini http://www.celi.it http://www.blogmeter.it http://www.memesphere.it Tel +39.011.562.71.15 jabber:ro.franchini at gmail.com skype:ro.franchini
On Sun, Jan 31, 2010 at 11:39 PM, Roberto Franchini <franchini at celi.it> wrote:> Hello, > I've configured a 6 boxs cluster with DHT and replicate. > Each machine creates locally lucene's indexes and then they are copied > to the gulester ?storage. > Now I've got this: > -????????? ? ? ? ?? ? ? ? ? ? ? ? ? ? ? ?? _0.fdt > -????????? ? ? ? ?? ? ? ? ? ? ? ? ? ? ? ?? _0.fdx > -????????? ? ? ? ?? ? ? ? ? ? ? ? ? ? ? ?? _0.fnm > -????????? ? ? ? ?? ? ? ? ? ? ? ? ? ? ? ?? _0.frq > -????????? ? ? ? ?? ? ? ? ? ? ? ? ? ? ? ?? _0.nrm > -????????? ? ? ? ?? ? ? ? ? ? ? ? ? ? ? ?? _0.prx > -????????? ? ? ? ?? ? ? ? ? ? ? ? ? ? ? ?? _0.tii > -????????? ? ? ? ?? ? ? ? ? ? ? ? ? ? ? ?? _0.tis > -????????? ? ? ? ?? ? ? ? ? ? ? ? ? ? ? ?? _0.tvd > -????????? ? ? ? ?? ? ? ? ? ? ? ? ? ? ? ?? _0.tvf > -????????? ? ? ? ?? ? ? ? ? ? ? ? ? ? ? ?? _0.tvx > -rw-r--r-- 1 toor adm ? 4 2010-01-31 23:15 _1.fdt > -rw-r--r-- 1 toor adm ? 4 2010-01-31 23:15 _1.fdx > -rw-r--r-- 1 toor adm 526 2010-01-31 23:15 _1.fnm > -rw-r--r-- 1 toor adm ? 0 2010-01-31 23:15 _1.frq > -rw-r--r-- 1 toor adm ? 4 2010-01-31 23:15 _1.nrm > -rw-r--r-- 1 toor adm ? 0 2010-01-31 23:15 _1.prx > -rw-r--r-- 1 toor adm ?24 2010-01-31 23:15 _1.tii > -rw-r--r-- 1 toor adm ?24 2010-01-31 23:15 _1.tis > -rw-r--r-- 1 toor adm ? 4 2010-01-31 23:15 _1.tvd > -rw-r--r-- 1 toor adm ? 4 2010-01-31 23:15 _1.tvf > -rw-r--r-- 1 toor adm ? 4 2010-01-31 23:15 _1.tvx > -????????? ? ? ? ?? ? ? ? ? ? ? ? ? ? ? ?? segments_2 > -rw-r--r-- 1 toor adm ?58 2010-01-31 23:15 segments_4 > -rw-r--r-- 1 toor adm ?20 2010-01-31 23:15 segments.gen > -rw-r--r-- 1 toor adm ?20 2010-01-31 23:15 segments.gen > > I can't delete this files. rm -rf doesn't works: it hangs up. >Sorry, I sent, but the mail was'nt finished. The situation above is on the storage, but in the /data/export local to one of the box i'vo got this: ll /data/export/dd4-indexer/working/lete/index/it/201001/20100130/leteExtract/ total 88 -rw-r--r-- 1 toor adm 4 2010-01-30 23:14 _0.fdx -rw-r--r-- 1 toor adm 0 2010-01-30 23:14 _0.frq -rw-r--r-- 1 toor adm 4 2010-01-30 23:14 _0.nrm -rw-r--r-- 1 toor adm 0 2010-01-30 23:14 _0.prx -rw-r--r-- 1 toor adm 24 2010-01-30 23:14 _0.tis -rw-r--r-- 1 toor adm 4 2010-01-30 23:14 _0.tvd -rw-r--r-- 1 toor adm 4 2010-01-30 23:14 _0.tvx -rw-r--r-- 1 toor adm 526 2010-01-31 23:15 _1.fnm -rw-r--r-- 1 toor adm 0 2010-01-31 23:15 _1.prx -rw-r--r-- 1 toor adm 4 2010-01-31 23:15 _1.tvd -rw-r--r-- 1 toor adm 4 2010-01-31 23:15 _1.tvf -rw-r--r-- 1 toor adm 4 2010-01-31 23:15 _1.tvx -rw-r--r-- 1 toor adm 58 2010-01-31 23:15 segments_4 -rw-r--r-- 1 toor adm 20 2010-01-30 23:14 segments.gen That is the previous version of the index! Before to copy new indexes to the storage, the batch process delete the previous copy (via Java), then it copies the new files. So: - new index is created locally - when finished we delete the old copy on the storage - then we copy the new index - but the new in inconsistent - on some machine i found the old index: deletion isn't propagated ?!?!? Please, help. I don't, and can't (the disk is full), go back to NFS Regards, R. -- Roberto Franchini http://www.celi.it http://www.blogmeter.it http://www.memesphere.it Tel +39.011.562.71.15 jabber:ro.franchini at gmail.com skype:ro.franchini
Hi Roberto, what version of glusterfs are you using? Can you send us the client and server log files? regards, On Mon, Feb 1, 2010 at 2:49 AM, Roberto Franchini <franchini at celi.it> wrote:> On Sun, Jan 31, 2010 at 11:39 PM, Roberto Franchini <franchini at celi.it> > wrote: > > Hello, > > I've configured a 6 boxs cluster with DHT and replicate. > > Each machine creates locally lucene's indexes and then they are copied > > to the gulester storage. > > Now I've got this: > > -????????? ? ? ? ? ? _0.fdt > > -????????? ? ? ? ? ? _0.fdx > > -????????? ? ? ? ? ? _0.fnm > > -????????? ? ? ? ? ? _0.frq > > -????????? ? ? ? ? ? _0.nrm > > -????????? ? ? ? ? ? _0.prx > > -????????? ? ? ? ? ? _0.tii > > -????????? ? ? ? ? ? _0.tis > > -????????? ? ? ? ? ? _0.tvd > > -????????? ? ? ? ? ? _0.tvf > > -????????? ? ? ? ? ? _0.tvx > > -rw-r--r-- 1 toor adm 4 2010-01-31 23:15 _1.fdt > > -rw-r--r-- 1 toor adm 4 2010-01-31 23:15 _1.fdx > > -rw-r--r-- 1 toor adm 526 2010-01-31 23:15 _1.fnm > > -rw-r--r-- 1 toor adm 0 2010-01-31 23:15 _1.frq > > -rw-r--r-- 1 toor adm 4 2010-01-31 23:15 _1.nrm > > -rw-r--r-- 1 toor adm 0 2010-01-31 23:15 _1.prx > > -rw-r--r-- 1 toor adm 24 2010-01-31 23:15 _1.tii > > -rw-r--r-- 1 toor adm 24 2010-01-31 23:15 _1.tis > > -rw-r--r-- 1 toor adm 4 2010-01-31 23:15 _1.tvd > > -rw-r--r-- 1 toor adm 4 2010-01-31 23:15 _1.tvf > > -rw-r--r-- 1 toor adm 4 2010-01-31 23:15 _1.tvx > > -????????? ? ? ? ? ? segments_2 > > -rw-r--r-- 1 toor adm 58 2010-01-31 23:15 segments_4 > > -rw-r--r-- 1 toor adm 20 2010-01-31 23:15 segments.gen > > -rw-r--r-- 1 toor adm 20 2010-01-31 23:15 segments.gen > > > > I can't delete this files. rm -rf doesn't works: it hangs up. > > > > Sorry, I sent, but the mail was'nt finished. > > The situation above is on the storage, but in the /data/export local > to one of the box i'vo got this: > > ll > /data/export/dd4-indexer/working/lete/index/it/201001/20100130/leteExtract/ > total 88 > -rw-r--r-- 1 toor adm 4 2010-01-30 23:14 _0.fdx > -rw-r--r-- 1 toor adm 0 2010-01-30 23:14 _0.frq > -rw-r--r-- 1 toor adm 4 2010-01-30 23:14 _0.nrm > -rw-r--r-- 1 toor adm 0 2010-01-30 23:14 _0.prx > -rw-r--r-- 1 toor adm 24 2010-01-30 23:14 _0.tis > -rw-r--r-- 1 toor adm 4 2010-01-30 23:14 _0.tvd > -rw-r--r-- 1 toor adm 4 2010-01-30 23:14 _0.tvx > -rw-r--r-- 1 toor adm 526 2010-01-31 23:15 _1.fnm > -rw-r--r-- 1 toor adm 0 2010-01-31 23:15 _1.prx > -rw-r--r-- 1 toor adm 4 2010-01-31 23:15 _1.tvd > -rw-r--r-- 1 toor adm 4 2010-01-31 23:15 _1.tvf > -rw-r--r-- 1 toor adm 4 2010-01-31 23:15 _1.tvx > -rw-r--r-- 1 toor adm 58 2010-01-31 23:15 segments_4 > -rw-r--r-- 1 toor adm 20 2010-01-30 23:14 segments.gen > > > That is the previous version of the index! > Before to copy new indexes to the storage, the batch process delete > the previous copy (via Java), then it copies the new files. > So: > - new index is created locally > - when finished we delete the old copy on the storage > - then we copy the new index > - but the new in inconsistent > - on some machine i found the old index: deletion isn't propagated ?!?!? > > Please, help. I don't, and can't (the disk is full), go back to NFS > Regards, > R. > -- > Roberto Franchini > http://www.celi.it > http://www.blogmeter.it > http://www.memesphere.it > Tel +39.011.562.71.15 > jabber:ro.franchini at gmail.com <jabber%3Aro.franchini at gmail.com>skype:ro.franchini > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >-- Raghavendra G
On Sun, Jan 31, 2010 at 11:49 PM, Roberto Franchini <franchini at celi.it> wrote:> On Sun, Jan 31, 2010 at 11:39 PM, Roberto Franchini <franchini at celi.it> wrote: >> Hello, >> I've configured a 6 boxs cluster with DHT and replicate. >> Each machine creates locally lucene's indexes and then they are copied >> to the gulester ?storage.[cut] One of the boexes was misconfigured, now I fixed it. A lot of files in the storage are broken, and a lot of the were writed when the configuration was right, e.g. before adding the last 2 nodes last friday. I changed the way I do the copy in my Java batches and at the moment I'm trying to rebuild the broken contents. Broken files can't be deleted, so I'm moving them, e.g. I rename the dir. This leave the broken files in the storage, but at the moment I don't know how to delete them. Thanks, regards, Roberto -- Roberto Franchini http://www.celi.it http://www.blogmeter.it http://www.memesphere.it Tel +39.011.562.71.15 jabber:ro.franchini at gmail.com skype:ro.franchini