thr3ads.net - dovecot - fts_solr: Error: fts_solr: received invalid uid '0' [Dec 2019]

If this information is useful, please help other people find it:
Share via:

Larry Rosenman

2019-Oct-10 02:05 UTC

fts_solr: Error: fts_solr: received invalid uid '0'

Is there a way to get this as a patchset against 2.3.8?


On Tue, Oct 8, 2019 at 12:03 PM Stephan Bosch via dovecot <
dovecot at dovecot.org> wrote:
>
>
> On 08/10/2019 02:52, Stephan Bosch via dovecot wrote:
> >
> >
> > On 07/10/2019 15:13, Fabian Kuran via dovecot wrote:
> >> Hi,
> >>
> >>> Am 23.09.2019 um 09:36 schrieb Filip Hanes <filip.hanes at
ringier.sk
> >>> <mailto:filip.hanes at ringier.sk>>:
> >>>
> >>>
> >>> ?t 19. 9. 2019 o 15:30 Fabian via dovecot <dovecot at
dovecot.org
> >>> <mailto:dovecot at dovecot.org>> nap?sal(a):
> >>>
> >>>     Are there any other hints or tips regarding this ?invalid
uid
> >>>     ?0?"-message?
> >>>
> >>> from fts-solr source:
> >>>     case SOLR_XML_CONTENT_STATE_UID:
> >>>         if (uint32_parse(str, len, &ctx->uid) < 0 ||
ctx->uid == 0) {
> >>>
>             i_error("fts_solr: received invalid uid
'%s'", t_strndup(str, len));
>
> >>>
> >>> It means that plugin tries to parse '0' when in state
of parsing uid
> >>> in xml response from solr or there is a bug in xml response
parser.
> >>> I would enable rawlog_dir in config and look there for unusual
> >>> response or extra tags near usual uid in xml response. For
example:
> >>> |plugin { fts = solr fts_solr = url=...
> >>> rawlog_dir=/var/log/fts-solr/ }|
> >>>
> >>
> >> Thanks for this tip! The rawlog_dir option is not yet supported in
> >> our version 2.3.4. What we could provide now would be the
> >> XML-Response from Solr. The corresponding search query sometimes
> >> causes this error. But Solr's answer seems to always remain
the same
> >> - in my opinion this already indicates a bug of Dovecot?
> >>
> >> The XML response can be downloaded here:
> >> https://owncloud.eideo.de/index.php/s/8oyJaf5HBK6zdbr
> >
> > Based on the XML response above, I investigated this problem
> > thoroughly and determined that this is a pretty severe bug in the Solr
> > XML response parsing code. This occurs only when the response is
> > rather large and the boundary between two read chunks falls in the
> > middle of a numeric value (that happens to end in '0').
> >
> > Tracking internally as DOP-1470.
> >
> > I think this may also explain some of the weird reports about invalid
> > UIDs that we've seen in recent history on this mailing list.
>
> Preliminary fix is here:
> https://github.com/stephanbosch/dovecot-core/commits/fix-solr-xml-parser
>
> It's big due to some code restructuring for making a proper unit test.
I
> am not sure when this will be in a release.
>
> Regards,
>
> Stephan.
>
>
-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 214-642-9640 (c)     E-Mail: larryrtx at gmail.com
US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20191009/ac4856d9/attachment-0001.html>

Stephan Bosch

2019-Oct-13 07:24 UTC

head link

fts_solr: Error: fts_solr: received invalid uid '0'

On 10/10/2019 04:05, Larry Rosenman via dovecot wrote:> Is there a way to get this as a patchset against 2.3.8?
>
https://github.com/stephanbosch/dovecot-core/tree/fix-solr-xml-parser-2.3.8

This hasn't seen any QA yet, so handle with care.

Regards,

Stephan.
>
> On Tue, Oct 8, 2019 at 12:03 PM Stephan Bosch via dovecot 
> <dovecot at dovecot.org <mailto:dovecot at dovecot.org>> wrote:
>
>
>
>     On 08/10/2019 02:52, Stephan Bosch via dovecot wrote:
>     >
>     >
>     > On 07/10/2019 15:13, Fabian Kuran via dovecot wrote:
>     >> Hi,
>     >>
>     >>> Am 23.09.2019 um 09:36 schrieb Filip Hanes
>     <filip.hanes at ringier.sk <mailto:filip.hanes at ringier.sk>
>     >>> <mailto:filip.hanes at ringier.sk
<mailto:filip.hanes at ringier.sk>>>:
>     >>>
>     >>>
>     >>> ?t 19. 9. 2019 o?15:30 Fabian via dovecot <dovecot at
dovecot.org
>     <mailto:dovecot at dovecot.org>
>     >>> <mailto:dovecot at dovecot.org <mailto:dovecot at
dovecot.org>>>
>     nap?sal(a):
>     >>>
>     >>> ??? Are there any other hints or tips regarding this
?invalid uid
>     >>> ??? ?0?"-message?
>     >>>
>     >>> from fts-solr source:
>     >>> ????case?SOLR_XML_CONTENT_STATE_UID:
>     >>>
>    
????????if?(uint32_parse(str,?len,?&ctx->uid)?<?0?||?ctx->uid?==?0)?{
>     >>>
>    
????????????i_error("fts_solr:?received?invalid?uid?'%s'",?t_strndup(str,?len));
>
>     >>>
>     >>> It means that plugin tries to parse '0' when in
state of
>     parsing uid
>     >>> in xml response from solr or there is a bug in xml
response
>     parser.
>     >>> I would enable rawlog_dir in config and look there for
unusual
>     >>> response or extra tags near usual uid in xml response. For
>     example:
>     >>> |plugin { fts = solr fts_solr = url=...
>     >>> rawlog_dir=/var/log/fts-solr/ }|
>     >>>
>     >>
>     >> Thanks for this tip!?The rawlog_dir option is not yet
supported in
>     >> our version 2.3.4. What we could provide now would be the
>     >> XML-Response from Solr. The corresponding search query
sometimes
>     >> causes this error. But Solr's answer seems to always
remain the
>     same
>     >> - in my opinion this already indicates a bug of Dovecot?
>     >>
>     >> The XML response can be downloaded here:
>     >> https://owncloud.eideo.de/index.php/s/8oyJaf5HBK6zdbr
>     >
>     > Based on the XML response above, I investigated this problem
>     > thoroughly and determined that this is a pretty severe bug in
>     the Solr
>     > XML response parsing code. This occurs only when the response is
>     > rather large and the boundary between two read chunks falls in the
>     > middle of a numeric value (that happens to end in '0').
>     >
>     > Tracking internally as DOP-1470.
>     >
>     > I think this may also explain some of the weird reports about
>     invalid
>     > UIDs that we've seen in recent history on this mailing list.
>
>     Preliminary fix is here:
>    
https://github.com/stephanbosch/dovecot-core/commits/fix-solr-xml-parser
>
>     It's big due to some code restructuring for making a proper unit
>     test. I
>     am not sure when this will be in a release.
>
>     Regards,
>
>     Stephan.
>
>
>
> -- 
> Larry Rosenman http://www.lerctr.org/~ler
> Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com 
> <mailto:larryrtx at gmail.com>
> US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106

Larry Rosenman

2019-Dec-10 18:57 UTC

head link

fts_solr: Error: fts_solr: received invalid uid '0'

Any news on this getting into a release?

On Sun, Oct 13, 2019 at 2:24 AM Stephan Bosch <stephan at rename-it.nl>
wrote:
>
>
> On 10/10/2019 04:05, Larry Rosenman via dovecot wrote:
> > Is there a way to get this as a patchset against 2.3.8?
> >
>
> https://github.com/stephanbosch/dovecot-core/tree/fix-solr-xml-parser-2.3.8
>
> This hasn't seen any QA yet, so handle with care.
>
> Regards,
>
> Stephan.
>
> >
> > On Tue, Oct 8, 2019 at 12:03 PM Stephan Bosch via dovecot
> > <dovecot at dovecot.org <mailto:dovecot at dovecot.org>>
wrote:
> >
> >
> >
> >     On 08/10/2019 02:52, Stephan Bosch via dovecot wrote:
> >     >
> >     >
> >     > On 07/10/2019 15:13, Fabian Kuran via dovecot wrote:
> >     >> Hi,
> >     >>
> >     >>> Am 23.09.2019 um 09:36 schrieb Filip Hanes
> >     <filip.hanes at ringier.sk <mailto:filip.hanes at
ringier.sk>
> >     >>> <mailto:filip.hanes at ringier.sk
<mailto:filip.hanes at ringier.sk
> >>>:
> >     >>>
> >     >>>
> >     >>> ?t 19. 9. 2019 o 15:30 Fabian via dovecot <dovecot
at dovecot.org
> >     <mailto:dovecot at dovecot.org>
> >     >>> <mailto:dovecot at dovecot.org <mailto:dovecot
at dovecot.org>>>
> >     nap?sal(a):
> >     >>>
> >     >>>     Are there any other hints or tips regarding this
?invalid uid
> >     >>>     ?0?"-message?
> >     >>>
> >     >>> from fts-solr source:
> >     >>>     case SOLR_XML_CONTENT_STATE_UID:
> >     >>>
> >             if (uint32_parse(str, len, &ctx->uid) < 0 ||
ctx->uid == 0) {
> >     >>>
> >
>              i_error("fts_solr: received invalid uid
'%s'", t_strndup(str, len));
> >
> >     >>>
> >     >>> It means that plugin tries to parse '0' when
in state of
> >     parsing uid
> >     >>> in xml response from solr or there is a bug in xml
response
> >     parser.
> >     >>> I would enable rawlog_dir in config and look there
for unusual
> >     >>> response or extra tags near usual uid in xml
response. For
> >     example:
> >     >>> |plugin { fts = solr fts_solr = url=...
> >     >>> rawlog_dir=/var/log/fts-solr/ }|
> >     >>>
> >     >>
> >     >> Thanks for this tip! The rawlog_dir option is not yet
supported in
> >     >> our version 2.3.4. What we could provide now would be the
> >     >> XML-Response from Solr. The corresponding search query
sometimes
> >     >> causes this error. But Solr's answer seems to always
remain the
> >     same
> >     >> - in my opinion this already indicates a bug of Dovecot?
> >     >>
> >     >> The XML response can be downloaded here:
> >     >> https://owncloud.eideo.de/index.php/s/8oyJaf5HBK6zdbr
> >     >
> >     > Based on the XML response above, I investigated this problem
> >     > thoroughly and determined that this is a pretty severe bug in
> >     the Solr
> >     > XML response parsing code. This occurs only when the response
is
> >     > rather large and the boundary between two read chunks falls
in the
> >     > middle of a numeric value (that happens to end in
'0').
> >     >
> >     > Tracking internally as DOP-1470.
> >     >
> >     > I think this may also explain some of the weird reports about
> >     invalid
> >     > UIDs that we've seen in recent history on this mailing
list.
> >
> >     Preliminary fix is here:
> >
> https://github.com/stephanbosch/dovecot-core/commits/fix-solr-xml-parser
> >
> >     It's big due to some code restructuring for making a proper
unit
> >     test. I
> >     am not sure when this will be in a release.
> >
> >     Regards,
> >
> >     Stephan.
> >
> >
> >
> > --
> > Larry Rosenman http://www.lerctr.org/~ler
> > Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com
> > <mailto:larryrtx at gmail.com>
> > US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
>
>
-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 214-642-9640 (c)     E-Mail: larryrtx at gmail.com
US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20191210/d40a0d04/attachment.html>

Reasonably Related Threads

Search for more possibly parallel threads

dovecot - Dec 2019 - fts_solr: Error: fts_solr: received invalid uid '0'

fts_solr: Error: fts_solr: received invalid uid '0'

fts_solr: Error: fts_solr: received invalid uid '0'

fts_solr: Error: fts_solr: received invalid uid '0'

Reasonably Related Threads