thr3ads.net - Rails - Error with fcgi and apache (web scraping, image manipulating) [Jan 2005]

If this information is useful, please help other people find it:
Share via:

Berndt Jung

2005-Jan-19 01:49 UTC

Error with fcgi and apache (web scraping, image manipulating)

Hi,

I''m having an issue with fcgi an apache.  The app works, but it
crashes before the results display.  All the files get created and are
correct, but it seems that fcgi or apache get impatient.  It seems to
work when I only grab one page at a time, but runs into problems with
3+ pages.

Here is the error:

[Tue Jan 18 17:44:58 2005] [error] [client 127.0.0.1] FastCGI: comm
with (dynamic) server
"/Library/WebServer/Documents/magick_montage/public/dispatch.fcgi"
aborted: (first read) idle timeout (30 sec)
[Tue Jan 18 17:44:58 2005] [error] [client 127.0.0.1] FastCGI:
incomplete headers (0 bytes) received from server
"/Library/WebServer/Documents/magick_montage/public/dispatch.fcgi"

And here is the code:

def montage
    scrape("toes", 0)
    render ''picture/test''
  end
  
  def scrape(keyword, start)
    tiles = Array.new
    re = /img\ssrc=\/images\S+/
    (0..4).each do |st|
      scrape = Array.new
     
open("http://images.google.com/images?q=#{keyword}&hl=en&lr=&safe=off&start=#{(20
* st).to_s}&sa=N") do |f|
        scrape = f.read.scan(re)
      end
      scrape.each do |s|
        s.sub!(/img\ssrc=\//, '''')
        is = open(''http://images.google.com/'' + s)
        os = Tempfile.new(''montage'')
        os.write(is.read)
        is.close
        os.close
        image = Image.read(os.path).first
        tiles << store(image)
      end
    end
    test_arr(tiles)
  end
  
  def store(image)
    if (image.columns > image.rows)
      (image.crop!(CenterGravity, image.rows, image.rows)).resize!(80,80)
    else
      (image.crop!(CenterGravity, image.columns, image.columns)).resize!(80,80)
    end
    @picture = Picture.new
    @picture.picture = image.to_blob
    @picture.kind = ''tile''
    @picture.attributes = get_profile(image)
    if @picture.save
      @picture.id
    else
      error
    end
  end
  
  def test_arr(tiles)
    j = 0
    tiles.each do |i|
      img = Image.from_blob(Picture.find(i).picture).first
     
img.write("/Library/WebServer/Documents/magick_montage/public/images/test#{j}.jpg")
      j += 1
    end
  end

Michael Koziarski

2005-Jan-19 03:09 UTC

head link

Re: Error with fcgi and apache (web scraping, image manipulating)

On Tue, 18 Jan 2005 17:49:44 -0800, Berndt Jung
<berndtj-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
wrote:> Hi,
> 
> I''m having an issue with fcgi an apache.  The app works, but it
> crashes before the results display.  All the files get created and are
> correct, but it seems that fcgi or apache get impatient.  It seems to
> work when I only grab one page at a time, but runs into problems with
> 3+ pages.
> 
> Here is the error:
> 
> [Tue Jan 18 17:44:58 2005] [error] [client 127.0.0.1] FastCGI: comm
> with (dynamic) server
>
"/Library/WebServer/Documents/magick_montage/public/dispatch.fcgi"
> aborted: (first read) idle timeout (30 sec)
Are you perhaps having the problem David mentioned at:

http://weblog.rubyonrails.com/archives/2005/01/03/watch-for-huge-requests-on-default-fcgi/

> [Tue Jan 18 17:44:58 2005] [error] [client 127.0.0.1] FastCGI:
> incomplete headers (0 bytes) received from server
>
"/Library/WebServer/Documents/magick_montage/public/dispatch.fcgi"
> 
> And here is the code:
> 
> def montage
>     scrape("toes", 0)
>     render ''picture/test''
>   end
> 
>   def scrape(keyword, start)
>     tiles = Array.new
>     re = /img\ssrc=\/images\S+/
>     (0..4).each do |st|
>       scrape = Array.new
>      
open("http://images.google.com/images?q=#{keyword}&hl=en&lr=&safe=off&start=#{(20
> * st).to_s}&sa=N") do |f|
>         scrape = f.read.scan(re)
>       end
>       scrape.each do |s|
>         s.sub!(/img\ssrc=\//, '''')
>         is = open(''http://images.google.com/'' + s)
>         os = Tempfile.new(''montage'')
>         os.write(is.read)
>         is.close
>         os.close
>         image = Image.read(os.path).first
>         tiles << store(image)
>       end
>     end
>     test_arr(tiles)
>   end
> 
>   def store(image)
>     if (image.columns > image.rows)
>       (image.crop!(CenterGravity, image.rows, image.rows)).resize!(80,80)
>     else
>       (image.crop!(CenterGravity, image.columns,
image.columns)).resize!(80,80)
>     end
>     @picture = Picture.new
>     @picture.picture = image.to_blob
>     @picture.kind = ''tile''
>     @picture.attributes = get_profile(image)
>     if @picture.save
>       @picture.id
>     else
>       error
>     end
>   end
> 
>   def test_arr(tiles)
>     j = 0
>     tiles.each do |i|
>       img = Image.from_blob(Picture.find(i).picture).first
>      
img.write("/Library/WebServer/Documents/magick_montage/public/images/test#{j}.jpg")
>       j += 1
>     end
>   end
> _______________________________________________
> Rails mailing list
> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
> 

-- 
Cheers

Koz

Norman Timmler

2005-Jan-19 11:12 UTC

head link

Re: Error with fcgi and apache (web scraping, image manipulating)

maybe apache has a probelm with logfile permissions. we ran into this
problem yesterday. give it a try.

ciao,
norman

Am Dienstag, den 18.01.2005, 17:49 -0800 schrieb Berndt
Jung:> Hi,
> 
> I''m having an issue with fcgi an apache.  The app works, but it
> crashes before the results display.  All the files get created and are
> correct, but it seems that fcgi or apache get impatient.  It seems to
> work when I only grab one page at a time, but runs into problems with
> 3+ pages.
> 
> Here is the error:
> 
> [Tue Jan 18 17:44:58 2005] [error] [client 127.0.0.1] FastCGI: comm
> with (dynamic) server
>
"/Library/WebServer/Documents/magick_montage/public/dispatch.fcgi"
> aborted: (first read) idle timeout (30 sec)
> [Tue Jan 18 17:44:58 2005] [error] [client 127.0.0.1] FastCGI:
> incomplete headers (0 bytes) received from server
>
"/Library/WebServer/Documents/magick_montage/public/dispatch.fcgi"
> 
> And here is the code:
> 
> def montage
>     scrape("toes", 0)
>     render ''picture/test''
>   end
>   
>   def scrape(keyword, start)
>     tiles = Array.new
>     re = /img\ssrc=\/images\S+/
>     (0..4).each do |st|
>       scrape = Array.new
>      
open("http://images.google.com/images?q=#{keyword}&hl=en&lr=&safe=off&start=#{(20
> * st).to_s}&sa=N") do |f|
>         scrape = f.read.scan(re)
>       end
>       scrape.each do |s|
>         s.sub!(/img\ssrc=\//, '''')
>         is = open(''http://images.google.com/'' + s)
>         os = Tempfile.new(''montage'')
>         os.write(is.read)
>         is.close
>         os.close
>         image = Image.read(os.path).first
>         tiles << store(image)
>       end
>     end
>     test_arr(tiles)
>   end
>   
>   def store(image)
>     if (image.columns > image.rows)
>       (image.crop!(CenterGravity, image.rows, image.rows)).resize!(80,80)
>     else
>       (image.crop!(CenterGravity, image.columns,
image.columns)).resize!(80,80)
>     end
>     @picture = Picture.new
>     @picture.picture = image.to_blob
>     @picture.kind = ''tile''
>     @picture.attributes = get_profile(image)
>     if @picture.save
>       @picture.id
>     else
>       error
>     end
>   end
>   
>   def test_arr(tiles)
>     j = 0
>     tiles.each do |i|
>       img = Image.from_blob(Picture.find(i).picture).first
>      
img.write("/Library/WebServer/Documents/magick_montage/public/images/test#{j}.jpg")
>       j += 1
>     end
>   end
> _______________________________________________
> Rails mailing list
> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>

Rails - Jan 2005 - Error with fcgi and apache (web scraping, image manipulating)

Error with fcgi and apache (web scraping, image manipulating)

Re: Error with fcgi and apache (web scraping, image manipulating)

Re: Error with fcgi and apache (web scraping, image manipulating)