I have a very strange situation that I''ve been tracking for some months now. This may or may not be a rails issue. The symptoms don''t feel like it, so I guess I''m appealing to my rails colleagues to lend me their thoughts. Throughout the day, HomeStars.com will get a number of invalid requests(< 50) such as : /companies/1199238-action-assembly/forums - is not valid /companies/1199238-action-assembly - is valid /forums - is valid /forums is linked to in the menu on each page (so is /sessions/new which also exhibits the same behaviour) All I can surmise is that something is hitting the forums relative url thinking it''s relative, but it''s an absolute reference. <a href="/forums">Forum</a> So this feels like either a bug in a scraper, or a page pre-loader (plugin that visits links on a page to pre-load them). I see enough of them, with differing user agents, and vastly different IPs, and not enough requests to not feel like a scraper. So I''m left with thinking it''s a browser plugin - pre-loading. any thoughts or pointers are appreciated. thank you. Jodi --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---
A short update on this - never saw this behaviour before - but will see more now Googlebot just hit this url - /companies/1199238-action-assembly/forum All I can imagine is that the relative url must exist - but for the life of me I can''t see it. Jodi ------ I have a very strange situation that I''ve been tracking for some months now. This may or may not be a rails issue. The symptoms don''t feel like it, so I guess I''m appealing to my rails colleagues to lend me their thoughts. Throughout the day, HomeStars.com will get a number of invalid requests(< 50) such as : /companies/1199238-action-assembly/forums - is not valid /companies/1199238-action-assembly - is valid /forums - is valid /forums is linked to in the menu on each page (so is /sessions/new which also exhibits the same behaviour) All I can surmise is that something is hitting the forums relative url thinking it''s relative, but it''s an absolute reference. <a href="/forums">Forum</a> So this feels like either a bug in a scraper, or a page pre-loader (plugin that visits links on a page to pre-load them). I see enough of them, with differing user agents, and vastly different IPs, and not enough requests to not feel like a scraper. So I''m left with thinking it''s a browser plugin - pre-loading. any thoughts or pointers are appreciated. thank you. Jodi --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en -~----------~----~----~----~------~----~------~--~---