Xapian devel - Mar 2014 - [GSoC 2014] About "Clustering of Search Results"

Hello. I am Liu Chi(??) from Peking University,  China. I am planning to
join GSoC. I am interested in Xapian and looking forward to find something
interesting in GSoC 2014 Project Ideas List.

The topic of "Clustering of Search Results" looks interesting and I
think
it suits me. I have been involved in a project that aims to clustering
tweets based on the text similarity and user profile. I noticed that
"Clustering of Search Results" have mentioned disappointing
performance.I
am puzzled that is this project just concerned improve the performance of
the old code or also trying to find new approaches? Besides clustering
speed, how to evaluate clustering effect?

Look forward to exchanging ideas with you.

Thanks
-- 
Chi Liu (??)
+86-15210624786
Undergraduate Student
Team of Search Engine and Web Mining
School of Electronic Engineering  and Computer Science
Peking University, Beijing, 100871, P.R.China
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.xapian.org/pipermail/xapian-devel/attachments/20140310/e77649a6/attachment-0002.html>

On Mon, Mar 10, 2014 at 08:50:14PM +0800, Chi Liu wrote:
> The topic of "Clustering of Search Results" looks interesting and
I think
> it suits me. I have been involved in a project that aims to clustering
> tweets based on the text similarity and user profile. I noticed that
> "Clustering of Search Results" have mentioned disappointing
performance.I
> am puzzled that is this project just concerned improve the performance of
> the old code or also trying to find new approaches?
Most applications of Xapian are interactive, so to actually be
practically useful clustering needs to complete in a reasonable amount
of time (a fraction of a second ideally).  I think that needs to be a key
aim of the project.

But if that aim is addressed, exactly what else the project consists of
is largely up to you.

If by "find new approaches" you mean different approach to that used
by
the existing clustering branch, then sure.  If you're talking about
doing original research, I'd be a little cautious about that, as
clustering is a relatively mature field, and I'm a bit dubious a student
could development and implement a new approach in the GSoC timescale.
> Besides clustering speed, how to evaluate clustering effect?
That's a good question - I'm not sure how clustering effectiveness is
typically measured.  But if we're implementing known approaches,
a formal evaluation of effectiveness is probably less necessary.

Cheers,
    Olly

2014-03-11 8:47 GMT+08:00 Olly Betts <olly at survex.com> wrote:
> Most applications of Xapian are interactive, so to actually be
> practically useful clustering needs to complete in a reasonable amount
> of time (a fraction of a second ideally).  I think that needs to be a key
> aim of the project.
> If by "find new approaches" you mean different approach to that
used by
> the existing clustering branch, then sure.  If you're talking about
> doing original research, I'd be a little cautious about that, as
> clustering is a relatively mature field, and I'm a bit dubious a
student
> could development and implement a new approach in the GSoC timescale.
> But if that aim is addressed, exactly what else the project consists of
> is largely up to you.


Thank you for your patient explanation about the project. My understanding
about
the project "Clustering of Search Results" is that we mainly focus on
processing
speed of the existing code.

By "find new approaches" I mean trying other known clustering
algorithms.
What I am
concerned is whether the low efficiency is caused by improper algorithm. I
am reading
the existing clustering branch code and have not completely finished yet. I
might be
able to talk more about existing code in my application of GSoC. But now, I
really
can not comment before fully understanding exiting code.



> That's a good question - I'm not sure how clustering effectiveness
is
> typically measured.  But if we're implementing known approaches,
> a formal evaluation of effectiveness is probably less necessary.


My idea about measure clustering effectiveness is that when we trying other
known
clustering algorithms, we can use the old clustering result as a baseline.
If the difference
of clustering results is acceptable and new clustering algorithm has high
efficiency,
we may find a better approach. I will give more details about this in
my application of GSoC.



Thanks
Liu Chi


2014-03-11 8:47 GMT+08:00 Olly Betts <olly at survex.com>:
> On Mon, Mar 10, 2014 at 08:50:14PM +0800, Chi Liu wrote:
> > The topic of "Clustering of Search Results" looks
interesting and I think
> > it suits me. I have been involved in a project that aims to clustering
> > tweets based on the text similarity and user profile. I noticed that
> > "Clustering of Search Results" have mentioned disappointing
performance.I
> > am puzzled that is this project just concerned improve the performance
of
> > the old code or also trying to find new approaches?
>
> Most applications of Xapian are interactive, so to actually be
> practically useful clustering needs to complete in a reasonable amount
> of time (a fraction of a second ideally).  I think that needs to be a key
> aim of the project.
>
> But if that aim is addressed, exactly what else the project consists of
> is largely up to you.
>
> If by "find new approaches" you mean different approach to that
used by
> the existing clustering branch, then sure.  If you're talking about
> doing original research, I'd be a little cautious about that, as
> clustering is a relatively mature field, and I'm a bit dubious a
student
> could development and implement a new approach in the GSoC timescale.
>
> > Besides clustering speed, how to evaluate clustering effect?
>
> That's a good question - I'm not sure how clustering effectiveness
is
> typically measured.  But if we're implementing known approaches,
> a formal evaluation of effectiveness is probably less necessary.
>
> Cheers,
>     Olly
>


-- 
Chi Liu
+86-15210624786
Undergraduate Student
Team of Search Engine and Web Mining
School of Electronic Engineering  and Computer Science
Peking University, Beijing, 100871, P.R.China
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.xapian.org/pipermail/xapian-devel/attachments/20140311/9bab424f/attachment-0002.html>