> The log command will be executed wherever it is in the template. > By "current system user", what do you mean? There'll need to be > write permission for the CGI process, which is probably the same > identity as the web server process.I basically changed the ownership of the /var/log/omega directory to the current user using: sudo chown `whoami` /var/log/omega But I forgot to grant write permission to that directory. After granting the required permission it works fine. By current system user, I meant the `whoami`.> Why does it matter? You can dedupe in post-processing, which is > probably easier than detecting a genuinely new search.You are right; deduplicating in post-processing seems better option. Thanks, Vivek -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xapian.org/pipermail/xapian-devel/attachments/20170609/dec091b4/attachment.html>
Hi James, I've completed the first step of logging the click data i.e. logging separate entries for each executed search from the query template. More on that progress here [1]. Now for the next step, i.e. logging the query IDs and click URL on each click event, as planned, I created a new template which has another log command [2]. To redirect to the clicklog template, we could add an onClick event inside the <a> tag in line 109 in the query template [3] which calls an external javascript function something like this one [4] to redirect to the clicklog template. But, I can't see how we would pass $query and the clicked doc $id to the template using that function. We want to be able to do that in order to log them using the clicklog template. How do you suggest I go about doing this? Thanks, Vivek [1] https://trac.xapian.org/wiki/GSoC2017/LetorClickstream/Journal#CodingWeek2:June10June16 [2] https://gist.github.com/ivmarkp/374fc8781d3789b0522713a79416edc8 [3] https://github.com/xapian/xapian/blob/master/xapian-applications/omega/templates/query#L109 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xapian.org/pipermail/xapian-devel/attachments/20170614/cdba8777/attachment.html>
Sorry, forgot to add the last reference. [4] https://gist.github.com/ivmarkp/faf2f17aee7ac6f12d6cd518bd6e3f3b -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xapian.org/pipermail/xapian-devel/attachments/20170614/c36a0a04/attachment.html>
On 14 Jun 2017, at 00:17, Vivek Pal <vivekpal.dtu at gmail.com> wrote:> I've completed the first step of logging the click data i.e. logging > separate entries for each executed search from the query template. > More on that progress here [1].Great! When we come to merge this, we'll probably want to have a "letor-enabled" template (or maybe "clickstream enabled"), so people can choose between that and the simpler default query template.> Now for the next step, i.e. logging the query IDs and click URL on each > click event, as planned, I created a new template which has another log > command [2].Easier than passing in the entire query string would be to pass in the pre-calculated qid.> To redirect to the clicklog template, we could add an onClick > event inside the <a> tag in line 109 in the query template [3] which calls > an external javascript function something like this one [4] to redirect to > the clicklog template.I'm generally opposed to using Javascript when you can do this instead: * in the query template, at the place you've indicated, change the href to point back to omega.cgi, specifying the second template and passing through as CGI parameters the qid and docid, and also the URL (to avoid having to look it up from the database) * the second query template can then cause the omega CGI to redirect to the final URL; you can look a CGI parameter up using $cgi{PARAMNAME}, and set an HTTP header using $httpheader{}. The only thing you cannot do is to change the HTTP status code, so you'll need a new omegascript command for that. (You could probably have $httpredirect{URL}, avoiding having to have a separate $httpheader{}.) J -- James Aylett, occasional troublemaker & project governance xapian.org
> The only thing you cannot do is to change the HTTP status code, so you'll > need a new omegascript command for that. (You could probably have > $httpredirect{URL}, avoiding having to have a separate $httpheader{}.)There's ShellExecute function which can potentially be used for the implementation of $httpredirect command but it'll only work on Windows. Although, I'm not aware of anything reliable similar to ShellExecute for UNIX based system. I was wondering if it would be better to use a meta tag for redirection like so? <meta http-equiv="refresh" content="0.1;url=$cgi{URL}"> I have updated the clicklog template [1] with what I have locally now. I was also wondering if there's a faster way of checking that the new log command works along with other changes than actually installing the Omega (system wide) from the branch I'm corrently working on? I tried doing that but messed up already running omega installation and had to spent almost whole yesterday just to get it back in working condition. Also, it would be great if you could just take a quick look at the git diff output [2] and let me know if anything needs work. I'm currently working on writing a preprocessing script in python and will open a PR soon after that's done. clicklog template might be missing in the output but can be found at [1]. [1] https://gist.github.com/ivmarkp/374fc8781d3789b0522713a79416edc8 [2] https://gist.github.com/ivmarkp/46f71c445e2fcb81fa6e1a5cb2273186 Thanks, Vivek -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xapian.org/pipermail/xapian-devel/attachments/20170617/5b6b9cd5/attachment.html>
[Please keep discussions on the mailing list.]> On 17 Jun 2017, at 12:53, Vivek Pal <vivekpal.dtu at gmail.com> wrote: > > > The only thing you cannot do is to change the HTTP status code, so you'll > > need a new omegascript command for that. (You could probably have > > $httpredirect{URL}, avoiding having to have a separate $httpheader{}.) > > There's ShellExecute function which can potentially be used for the > implementation of $httpredirect command but it'll only work on Windows. > Although, I'm not aware of anything reliable similar to ShellExecute for > UNIX based system.No, you just need to control the first line of the HTTP response. Nothing to do with executing anything; at the moment we're using the default CGI behaviour, which is to return HTTP 200. Thinking about this, the way that CGI works is you set a 'Status' header which changes this behaviour. So we don't actually need a new command, just something like: $httpheader{Status,302 Found} $httpheader{Location,$cgi{URL}} (303 is technically the right choice rather than 302, but I can't remember offhand if there remain problems with major browsers in getting the required behaviour right.) See http://www.oreilly.com/openbook/cgi/ch03_07.html and the parse_omegascript() function (in query.cc).> I was wondering if it would be better to use a meta tag for redirection > like so? <meta http-equiv="refresh" content="0.1;url=$cgi{URL}">You can do that _as well_, but not _instead of_ the above.> I was also wondering if there's a faster way of checking that the new > log command works along with other changes than actually installing > the Omega (system wide) from the branch I'm corrently working on? I > tried doing that but messed up already running omega installation and > had to spent almost whole yesterday just to get it back in working > condition.You can run omega.cgi from the command line without installing it and going via a web server. This is what the omegatests do, for instance. Once you've got it all actually working, you'll still need to test it in a web browser, of course, but it should reduce length of the development cycle.> Also, it would be great if you could just take a quick look at the git > diff output [2] and let me know if anything needs work.Don't do it like this; you can open a PR and note in the comment that it isn't ready for merge, and then we get much better tools for discussing the source code. I suggest you aim to get a PR merged with the new $qid{} command (but without using it in the default query template, since a. it should probably be a variant template, and b. you need the second template complete for it to be useful). It should be possible to review and merge $qid{} fairly quickly. J -- James Aylett, occasional troublemaker & project governance xapian.org
> [Please keep discussions on the mailing list.]I accidentally clicked on "Reply" instead of "Reply to all" but later I did send a separate mail to the mailing list.> Thinking about this, the way that CGI works is you set a 'Status' > header which changes this behaviour. So we don't actually need a > new command, just something like: > > $httpheader{Status,302 Found} > $httpheader{Location,$cgi{URL}}Ok, got it. I was initially under the impression that we needed a new Omegascript command to enable opening a webpage given a url.> See http://www.oreilly.com/openbook/cgi/ch03_07.html and the > parse_omegascript() function (in query.cc).Thanks, I found that link very helpful. Great book on CGI programming.> You can run omega.cgi from the command line without installing it and > going via a web server. This is what the omegatests do, for instance.Yeah, I was thinking about doing the same. Thanks, will try that.> Don't do it like this; you can open a PR and note in the comment that > it isn't ready for merge, and then we get much better tools for > discussing the source code.Okay, will open a PR with adding the new $qid{} command first. Although, I was going to wait until the preprocessing script was complete and then open a single PR for the whole logging functionality. Thanks, Vivek -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xapian.org/pipermail/xapian-devel/attachments/20170618/8f7126b0/attachment.html>