dovecot - Jun 2019 - What does Solr index do and how to handle its high avaliablity?

Hi, guys,

   Can you give me an example of solr usage in dovecot?  As far as I know,
you can search email easily by MUA like outlook, so which role does solr
play?

  And based on https://dovecot.org/pipermail/dovecot/2019-April/115575.html

I'm going to use an VIP to host 2 mail servers. Currently, it works in fail
over and fail back test except solr index, so how to resolve this?

Is it possible to put dovecot index on shared storage like NFS?

How to if it's a yes.  It seems it's under /var/solr/data/dovecot/data/

How to rebuild if I don't put index data on shared storage in case of fail
over?

Thanks,
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20190614/97cffa2c/attachment.html>

Just read docs, solr in dovecot seems as index of emails to accelarate the
speed of mail searching.

On Fri, Jun 14, 2019 at 4:11 PM luckydog xf <luckydogxf at gmail.com>
wrote:
> Hi, guys,
>
>    Can you give me an example of solr usage in dovecot?  As far as I know,
> you can search email easily by MUA like outlook, so which role does solr
> play?
>
>   And based on
> https://dovecot.org/pipermail/dovecot/2019-April/115575.html
>
> I'm going to use an VIP to host 2 mail servers. Currently, it works in
> fail over and fail back test except solr index, so how to resolve this?
>
> Is it possible to put dovecot index on shared storage like NFS?
>
> How to if it's a yes.  It seems it's under
/var/solr/data/dovecot/data/
>
> How to rebuild if I don't put index data on shared storage in case of
fail
> over?
>
> Thanks,
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20190614/72840753/attachment.html>

On 6/14/2019 2:11 AM, luckydog xf via dovecot wrote:
>  ? And based on
https://dovecot.org/pipermail/dovecot/2019-April/115575.html
> I'm going to use an VIP to host 2 mail servers. Currently, it works in 
> fail over and fail back test except solr index, so how to resolve this?
Solr has high availability options.  Probably the easiest is SolrCloud, 
because the legacy master-slave replication option basically has the 
master as a single point of failure.  Switching a slave to master is a 
very manual job.  SolrCloud is a true cluster -- no masters and no slaves.

Normally I would suggest a load balancer in front of Solr, but with 
SolrCloud and this usage, you could have a VIP switch between two hosts. 
  You will need at least three servers for a redundant SolrCloud 
install, because ZooKeeper absolutely requires three servers for 
redundancy.  SolrCloud is Solr+ZooKeeper.  Only two of the servers would 
need to run Solr, though.  The third server would only need to run 
ZooKeeper, and normally the system requirements for that are very low.
> Is it possible to put dovecot index on shared storage like NFS?
Solr does not do well on network file systems.  They usually don't offer 
the file locking capability that Lucene wants.  Solr is written using 
the Lucene API.

And you can't have two running Solr servers using the same index 
directory even if you disable file locking so that NFS works.  Each 
server needs its own copy of the index.
> How to rebuild if I don't put index data on shared storage in case of 
> fail over?
Building and rebuilding is accomplished by dovecot.  Here's a wiki page 
for rebuilding Solr indexes.  Admittedly it falls into the "not all that 
helpful" category:

https://wiki.apache.org/solr/HowToReindex

Thanks,
Shawn

Thanks a lot.

On Sat, Jun 15, 2019 at 11:33 AM Shawn Heisey via dovecot <
dovecot at dovecot.org> wrote:
> On 6/14/2019 2:11 AM, luckydog xf via dovecot wrote:
> >    And based on
> https://dovecot.org/pipermail/dovecot/2019-April/115575.html
> > I'm going to use an VIP to host 2 mail servers. Currently, it
works in
> > fail over and fail back test except solr index, so how to resolve
this?
>
> Solr has high availability options.  Probably the easiest is SolrCloud,
> because the legacy master-slave replication option basically has the
> master as a single point of failure.  Switching a slave to master is a
> very manual job.  SolrCloud is a true cluster -- no masters and no slaves.
>
> Normally I would suggest a load balancer in front of Solr, but with
> SolrCloud and this usage, you could have a VIP switch between two hosts.
>   You will need at least three servers for a redundant SolrCloud
> install, because ZooKeeper absolutely requires three servers for
> redundancy.  SolrCloud is Solr+ZooKeeper.  Only two of the servers would
> need to run Solr, though.  The third server would only need to run
> ZooKeeper, and normally the system requirements for that are very low.
>
> > Is it possible to put dovecot index on shared storage like NFS?
>
> Solr does not do well on network file systems.  They usually don't
offer
> the file locking capability that Lucene wants.  Solr is written using
> the Lucene API.
>
> And you can't have two running Solr servers using the same index
> directory even if you disable file locking so that NFS works.  Each
> server needs its own copy of the index.
>
> > How to rebuild if I don't put index data on shared storage in case
of
> > fail over?
>
> Building and rebuilding is accomplished by dovecot.  Here's a wiki page
> for rebuilding Solr indexes.  Admittedly it falls into the "not all
that
> helpful" category:
>
> https://wiki.apache.org/solr/HowToReindex
>
> Thanks,
> Shawn
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20190617/f90604df/attachment.html>