Displaying 1 result from an estimated 1 matches for "page_scan".
Did you mean:
page_scans
2006 Aug 17
1
Worker pool worker for page scanners: lost connection to mysql
...ant to scan many URLs simultaneously with a new scanner worker for
each URL, but I keep getting Mysql::Error: Lost connection to MySQL.
Yes, allow_concurrency=false.
Each scanner thread needs access to my ActiveRecord models to store
discovered feeds and whatnot. There is a DB table called page_scans
that acts as a big queue holding URLs waiting to be scanned.
It would be nice to run a worker that spawns up to 10 scanner workers
at a time to chip away at the queue, but first I need to figure out
what kinds of problems kill the mysql connection in a worker.
Any thoughts are much appreci...