I am close to launching a web site and so I have directed Google (via
Webmaster Tools) to index parts of the site via a sitemap (a rather
interesting approach to testing I might add).
Anyway, there are places where I require a login and generally if it
fails to find a logged in user, it does...
request.env["HTTP_REFERER"]
redirect_to :back
return false
Now a GoogleBot doesn''t submit a reply to the
request.env["HTTP_REFERRER"] so it dutifully sends me an e-mail via my
exception handler.
How do people handle these things?
Should I be doing something like...
my_url = request.env["HTTP_REFERRER"] || ":action =>
''index''"
redirect_to my_url
return false
I am sure that at least at this point, I don''t want to not send error
mails on 404''s just as a sanity check.
Craig
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
--
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.
Maybe I got you wrong, but wouldn''t a rel="nofollow" in your login-links be a solution for this situation? -- Posted via http://www.ruby-forum.com/. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
On Fri, 2010-02-12 at 15:40 +0100, Tom Ha wrote:> Maybe I got you wrong, but wouldn''t a rel="nofollow" in your login-links > be a solution for this situation?---- I think that for this usage, it''s not appropriate. In essence, I am providing a sitemap for all of the valid controllers/actions but clearly there are a few that require a login or else the user is ''redirect_to :back'' with a flash[:notice] explanation. But this doesn''t work for GoogleBot since it doesn''t supply an answer to request.env["HTTP_REFERRER"] and thus is redirected to nowhere which generates an error for the Bot and sends me an e-mail... neither of which are particularly useful. I suspect that even if I were to remove those specific controller/actions that require a valid login from the sitemap, that Bots will eventually find them anyway so I think the better long term thought is to handle them in the methods now anyway. I figured that there are people who have run into this before on the list and might have some suggestions. Craig -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
On Fri, Feb 12, 2010 at 6:34 AM, Craig White <craigwhite-BQ75lA0ptkhBDgjK7y7TUQ@public.gmane.org> wrote:> I am close to launching a web site and so I have directed Google (via > Webmaster Tools) to index parts of the site via a sitemap (a rather > interesting approach to testing I might add).Um, well; there are other link-checkers available, btw, but...> Anyway, there are places where I require a login and generally if it > fails to find a logged in user, it does... > request.env["HTTP_REFERER"] > redirect_to :back > return falseThat seems like a really bad idea to me: referer isn''t a required header, period. Some paranoid clients won''t send them at all, and if for instance your legitimate user types in the URL, or invokes it from a bookmark, there won''t be a referer. Most apps redirect to a login page if login is required. :-) FWIW, -- Hassan Schroeder ------------------------ hassan.schroeder-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org twitter: @hassan -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Craig White wrote:> I am close to launching a web site and so I have directed Google (via > Webmaster Tools) to index parts of the site via a sitemap (a rather > interesting approach to testing I might add).[...] And perhaps not a good one. Do you really want your not-yet-launched pages in Google''s index before launch? Best, -- Marnen Laibow-Koser http://www.marnen.org marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org -- Posted via http://www.ruby-forum.com/. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
On Fri, 2010-02-12 at 17:54 +0100, Marnen Laibow-Koser wrote:> Craig White wrote: > > I am close to launching a web site and so I have directed Google (via > > Webmaster Tools) to index parts of the site via a sitemap (a rather > > interesting approach to testing I might add). > [...] > > And perhaps not a good one. Do you really want your not-yet-launched > pages in Google''s index before launch?---- yes because from the ''visitors'' perspective, it is mostly finished and it takes time to get the ''bots'' to index and hopefully list the site in searches. I am mostly concerned with handling ''routing'' issues caused by things like ''redirect_to :back'' which doesn''t work for ''bots'' because they can''t (or won''t) provide any useful information to ''request.env["HTTP_REFERRER"] Craig -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
On Feb 12, 2010, at 11:54 AM, Marnen Laibow-Koser wrote:> Craig White wrote: >> I am close to launching a web site and so I have directed Google (via >> Webmaster Tools) to index parts of the site via a sitemap (a rather >> interesting approach to testing I might add). > [...] > > And perhaps not a good one. Do you really want your not-yet-launched > pages in Google''s index before launch? > > Best, > -- > Marnen Laibow-Koser > http://www.marnen.org > marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.orgOf course! You want to give Google (and other bots) time to find and index your site so that it''s primed for launch. The timing is the only tricky part. I had to redesign a "deep" page to act more like an entry page because so many visitors got there first by coming from Google search results. -Rob Rob Biedenharn http://agileconsultingllc.com Rob-xa9cJyRlE0mWcWVYNo9pwxS2lgjeYSpx@public.gmane.org -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.