dovecot - Jan 2020 - solr fts and removing accounts

i use solr fts indexing. It worls very well but it have one database per 
system, not per user.

Lets suppose i delete one or more e-mail users in system.

How to remove them in solr database to reclaim space?

On 1/19/2020 3:31 AM, Wojciech Puchar wrote:
> i use solr fts indexing. It worls very well but it have one database per 
> system, not per user.
> 
> Lets suppose i delete one or more e-mail users in system.
> 
> How to remove them in solr database to reclaim space?
I cannot say whether there is anything that you can do from the Dovecot 
side, but I can explain how to delete them from the Solr side if you 
like.  The end result would likely be the same either way.

I'm not sure that doing so is worthwhile.  Deleting data from a Solr 
index just marks it as deleted.  Until the index segments are merged, 
space will not be reclaimed.  And it is entirely possible that the 
segment in question might never get merged during normal operation.

It is possible to force the merging ... the operation is called 
"optimize" by Solr.  But it is a heavyweight operation, not one that 
should be done frequently, and in general the Solr project doesn't 
recommend doing it at all.

Deleting one user's data, even if that user has a large amount of email 
and the delete is followed by an optimize, is not likely to make much of 
a change in the size of the inverted index, just due to how it works. 
Looking over the schema for fts_solr, the large fields are not stored, 
so there will not be all that much stored data.  Stored data in the 
index is compressed as of Solr 4.1, which makes it even smaller.

Thanks,
Shawn

Cit?t Shawn Heisey <elyograg at elyograg.org>:
> On 1/19/2020 3:31 AM, Wojciech Puchar wrote:
>> i use solr fts indexing. It worls very well but it have one  
>> database per system, not per user.
>>
>> Lets suppose i delete one or more e-mail users in system.
>>
>> How to remove them in solr database to reclaim space?
>
> I cannot say whether there is anything that you can do from the  
> Dovecot side, but I can explain how to delete them from the Solr  
> side if you like.  The end result would likely be the same either way.
>
> I'm not sure that doing so is worthwhile.  Deleting data from a Solr  
> index just marks it as deleted.  Until the index segments are  
> merged, space will not be reclaimed.  And it is entirely possible  
> that the segment in question might never get merged during normal  
> operation.
>
> It is possible to force the merging ... the operation is called  
> "optimize" by Solr.  But it is a heavyweight operation, not one
that
> should be done frequently, and in general the Solr project doesn't  
> recommend doing it at all.
>
> Deleting one user's data, even if that user has a large amount of  
> email and the delete is followed by an optimize, is not likely to  
> make much of a change in the size of the inverted index, just due to  
> how it works. Looking over the schema for fts_solr, the large fields  
> are not stored, so there will not be all that much stored data.   
> Stored data in the index is compressed as of Solr 4.1, which makes  
> it even smaller.
>
> Thanks,
> Shawn

We are deleting all user data after it's removed, it can easily done  
with this HTTP GET command to solr:

/solr-url/update?stream.body=<delete><query>user:name at
domain.tld</query></delete>&commit=false