Hi, There doesn''t seem to be any reference for taking HTML snapshots from within a Rails server. I wonder how one could implement Google''s crawlable AJAX spec (http://code.google.com/web/ajaxcrawling/docs/learn-more.html)on a Rails application? To summarize: I have a Rails application with a Javascript front-end with lots of AJAX. I need Google to index the AJAX content, hence need to implement the above spec. Now, I can send an AJAX request to Rails for a link that the crawler asks; I need Rails server to respond with HTML snapshot. Can this be handled on a single Rails running on nginx? Or do we need to send the link to a HTMLUnit headless browser to take a snapshot? Has anyone done this for a Rails app? -- Posted via http://www.ruby-forum.com/. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
On Monday, May 16, 2011 7:08:11 PM UTC-6, Ruby-Forum.com User wrote:> > Hi, > There doesn''t seem to be any reference for taking HTML snapshots from > within a Rails server. I wonder how one could implement Google''s > crawlable AJAX spec > (http://code.google.com/web/ajaxcrawling/docs/learn-more.html)on a Rails > application?As always, there are several ways...> To summarize: I have a Rails application with a Javascript front-end > with lots of AJAX. I need Google to index the AJAX content, hence need > to implement the above spec. Now, I can send an AJAX request to Rails > for a link that the crawler asks; I need Rails server to respond with > HTML snapshot. Can this be handled on a single Rails running on nginx? > Or do we need to send the link to a HTMLUnit headless browser to take a > snapshot? >Does your "Javascript front-end with lots of AJAX" _create_ or _render_ lots of new HTML content? Or, is your AJAX the kind mostly manipulates the DOM by getting new HTML document fragments via XHR requests? As the Google docs on the subject itself mentions, if it is the former case then you may want to consider a server-side "browser" like HTMLUnit. Otherwise, you might want to focus more on your actual rails code. Even within the framework of rails conventions, there is so much latitude in how sites implement AJAX applications that there are lots of possible answers. For example, I''ve got a rails app (its an older rails 2 app) that has fair amount of AJAX. I first developed it statically and used "progressive enhancement" techniques to add AJAX functionality. The result is that in many cases I have controller actions that when executed may "return" (render) either a full HTML document or a document fragment, depending on whether the request is an XHR. If I were updating this site (quick and dirty) to support this Google spec., I''d simply make it so that said actions return a full HTML document when an AJAX request has the special _escaped_fragment_ parameter. However, I can conceive of several different techniques (and have used different ones to various degrees) that would require a different approach. This might be an area that would be good for some kind of rails (and/or rack) gem built around a specific set of AJAX conventions and design patterns, that integrates or is solely written to implement this Google spec. If, indeed, such a beast doesn''t already exist. Such a solution would still only work for those who want to, are willing to, or already do adhere to the chosen conventions. But, then, rails users do the same for web app. dev. in general. Anyone else care to let their mind wander too? -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Kendall Gifford wrote in post #999301:> On Monday, May 16, 2011 7:08:11 PM UTC-6, Ruby-Forum.com User wrote:> Does your "Javascript front-end with lots of AJAX" _create_ or _render_ > lots > of new HTML content? Or, is your AJAX the kind mostly manipulates the > DOM by > getting new HTML document fragments via XHR requests?It''s the former, a Javascript-minified web application, it manages the entire front-end.> > As the Google docs on the subject itself mentions, if it is the former > case > then you may want to consider a server-side "browser" like HTMLUnit. > Otherwise, you might want to focus more on your actual rails code. Even > within the framework of rails conventions, there is so much latitude in > how > sites implement AJAX applications that there are lots of possible > answers.It''s the former one, hence the need for HTMLunit. I came across this (http://tinyurl.com/6yxrch7) implementing HTMLUnit on GWT. I''m not an expert in GWT, hence deffering it for now until I can find a better solution.> > For example, I''ve got a rails app (its an older rails 2 app) that has > fair > amount of AJAX. I first developed it statically and used "progressive > enhancement" techniques to add AJAX functionality. The result is that in > many cases I have controller actions that when executed may "return" > (render) either a full HTML document or a document fragment, depending > on > whether the request is an XHR. If I were updating this site (quick and > dirty) to support this Google spec., I''d simply make it so that said > actions > return a full HTML document when an AJAX request has the special > _escaped_fragment_ parameter.Mine is a Rails2 app too, I return raw data to the client where it gets put in my custom templates. I reckon you mean RJS by "return a full HTML document", though it can get very complicated if I were to build the styles in a few Rails views.> > However, I can conceive of several different techniques (and have used > different ones to various degrees) that would require a different > approach. > > This might be an area that would be good for some kind of rails (and/or > rack) gem built around a specific set of AJAX conventions and design > patterns, that integrates or is solely written to implement this Google > spec. If, indeed, such a beast doesn''t already exist. Such a solution > would > still only work for those who want to, are willing to, or already do > adhere > to the chosen conventions. But, then, rails users do the same for web > app. > dev. in general. > > Anyone else care to let their mind wander too?I''ve come across Crowljax, which seems to be mostly for testing. I agree, it''d be great if there was a gem that ushered _escaped_fragment requests to a GAE app and returned back the HTML snapshot. It wouldn''t require high traffic volume, running only for crawling requests. Without such a solution, folks like me will go ahead and build their own GAE app I imagine... unless there is another way! -- Posted via http://www.ruby-forum.com/. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.