I have a DB built by omindex of about 224,000 web pages/documents. I perform a simple search of a single keyword. It takes on the average 15-30 seconds to display the results. The search time reported in the output is typically 0.1 - 0.25 seconds. At first I put the discrepency down to all 224,000 files being in the same directory and a lot of time being wasted perhaps dealing with the file system. I then split up the files so that no more than 1000 files were in a single directory. This seems to have had little or no effect. I am the only one currently using that system so it doesn't appear to be any contention problem. Anyone have any ideas on what the problem might be and how I might speed the results display up? It is running on a dual 3.0ghtz 32 bit intel processor with 4gb of ram and 1.2tb of fast raid storage under a Linux 2.6 kernel. Everything else seems to fly on this box. - Michael
Michael A. Lewis wrote:> I have a DB built by omindex of about 224,000 web pages/documents. I perform a simple search of a single keyword. It takes on the average 15-30 seconds to display the results. The search time reported in the output is typically 0.1 - 0.25 seconds. At first I put the discrepency down to all 224,000 files being in the same directory and a lot of time being wasted perhaps dealing with the file system. I then split up the files so that no more than 1000 files were in a single directory. This seems to have had little or no effect. I am the only one currently using that system so it doesn't appear to be any contention problem. Anyone have any ideas on what the problem might be and how I might speed the results display up? It is running on a dual 3.0ghtz 32 bit intel processor with 4gb of ram and 1.2tb of fast raid storage under a Linux 2.6 kernel. Everything else seems to fly on this box.Are you using the "topterms" facility? Turning that off can give a large speed boost.