Hello, I am developing a hosting platform on Debian Bullseye, with Dovecot amongst other tools. I am trying to use the xapian full test search plugin, but I can see the attachments are skipped: This is what I have in the logs when running the indexing in verbose mode: ----------------------------------------------------------------------- doveadm(camille): Info: FTS Xapian: fts_backend_xapian_check_access doveadm(camille): Info: FTS Xapian: Memory stats : Used = 56 MB, Free 66 MB doveadm(camille): Info: FTS Xapian: fts_backend_xapian_index_hdr doveadm(camille): Info: FTS Xapian: fts_backend_xapian_query doveadm(camille): Info: FTS Xapian: Query= uid:"44" doveadm(camille): Info: FTS Xapian: Ngram(S) -> 63 items (total 0 KB) doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_unset_build_key doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_set_build_key doveadm(camille): Info: FTS Xapian: New part (Header=Message- Id,Type=(null),Disposition=(null)) doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_build_more doveadm(camille): Info: FTS Xapian: fts_backend_xapian_check_access doveadm(camille): Info: FTS Xapian: Memory stats : Used = 56 MB, Free 66 MB doveadm(camille): Info: FTS Xapian: fts_backend_xapian_index_hdr doveadm(camille): Info: FTS Xapian: fts_backend_xapian_query doveadm(camille): Info: FTS Xapian: Query= uid:"44" doveadm(camille): Info: FTS Xapian: Ngram(XMID) -> 4 items (total 0 KB) doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_unset_build_key doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_set_build_key doveadm(camille): Info: FTS Xapian: New part (Header=X- Mailer,Type=(null),Disposition=(null)) doveadm(camille): Info: FTS Xapian: Unknown header (indexing) 'xmailer' doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_set_build_key doveadm(camille): Info: FTS Xapian: New part (Header=MIME- Version,Type=(null),Disposition=(null)) doveadm(camille): Info: FTS Xapian: Unknown header (indexing) 'mimeversion' doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_set_build_key doveadm(camille): Info: FTS Xapian: New part (Header=Content- Type,Type=(null),Disposition=(null)) doveadm(camille): Info: FTS Xapian: Unknown header (indexing) 'contenttype' doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_set_build_key doveadm(camille): Info: FTS Xapian: New part (Header=Authentication- Results,Type=(null),Disposition=(null)) doveadm(camille): Info: FTS Xapian: Unknown header (indexing) 'authenticationresults' doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_set_build_key doveadm(camille): Info: FTS Xapian: New part (Header=X-AV- Checked,Type=(null),Disposition=(null)) doveadm(camille): Info: FTS Xapian: Unknown header (indexing) 'xavchecked' doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_set_build_key doveadm(camille): Info: FTS Xapian: New part (Header=Content- Type,Type=(null),Disposition=(null)) doveadm(camille): Info: FTS Xapian: Unknown header (indexing) 'contenttype' doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_set_build_key doveadm(camille): Info: FTS Xapian: New part (Header=(null),Type=text/plain,Disposition=(null)) doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_build_more doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_unset_build_key doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_set_build_key doveadm(camille): Info: FTS Xapian: New part (Header=Content- Type,Type=(null),Disposition=(null)) doveadm(camille): Info: FTS Xapian: Unknown header (indexing) 'contenttype' doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_set_build_key doveadm(camille): Info: FTS Xapian: New part (Header=Content- Description,Type=(null),Disposition=(null)) doveadm(camille): Info: FTS Xapian: Unknown header (indexing) 'contentdescription' doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_set_build_key doveadm(camille): Info: FTS Xapian: New part (Header=Content- Disposition,Type=(null),Disposition=(null)) doveadm(camille): Info: FTS Xapian: Unknown header (indexing) 'contentdisposition' doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_set_build_key doveadm(camille): Info: FTS Xapian: New part (Header=Content-Transfer- Encoding,Type=(null),Disposition=(null)) doveadm(camille): Info: FTS Xapian: Unknown header (indexing) 'contenttransferencoding' doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_set_build_key doveadm(camille): Info: FTS Xapian: New part (Header=(null),Type=text/csv,Disposition=attachment; filename="file.csv") doveadm(camille): Info: FTS Xapian: Skipping part of type 'text/csv' and disposition 'attachment; filename="file.csv"' doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_set_mailbox doveadm(camille): Info: FTS Xapian: Unset box 'INBOX' (c0d4e304584e5460dae30000075d7e67) doveadm(camille): Info: FTS Xapian: fts_backend_xapian_oldbox doveadm(camille): Info: FTS Xapian: Done indexing 'INBOX' (c0d4e304584e5460dae30000075d7e67) (13 msgs in 261 ms, rate: 49.8) doveadm(camille): Info: FTS Xapian: fts_backend_xapian_release (unset_box) doveadm(camille): Info: FTS Xapian: Committed 'unset_box' in 17 ms doveadm(camille): Info: FTS Xapian: Box is empty doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_deinit (/home/users/camille/mails/indexes/xapian-indexes) doveadm(camille): Info: FTS Xapian: fts_backend_xapian_release (update_deinit) doveadm(camille): Info: FTS Xapian: Committed 'update_deinit' in 0 ms doveadm(camille): Info: FTS Xapian: Deinit /home/users/camille/mails/indexes/xapian-indexes) ----------------------------------------------------------------------- Especially thos line: doveadm(camille): Info: FTS Xapian: Skipping part of type 'text/csv' and disposition 'attachment; filename="file.csv"' I have seen this as a bug on fts-xapian: https://github.com/grosjo/fts-xapian/issues/68 Which then references a PR on Dovecot core itself: https://github.com/dovecot/core/pull/155 Any idea on how to use xapian with non-text attachments ? Thanks a lot for your help! Andr?
Hello, The version packaged on Bullseye is slightly out of date, I have filled a bug report: https://bugs.debian.org/985654 Thanks to the maintainers for their hard work! Andr? On Sun, 2021-03-21 at 10:51 +0000, Andr? Rodier wrote:> Hello, > > I am developing a hosting platform on Debian Bullseye, with Dovecot > amongst other tools. > > I am trying to use the xapian full test search plugin, but I can see > the attachments are skipped: > > This is what I have in the logs when running the indexing in verbose > mode: > > ------------------------------------------------------------------- > ---- > doveadm(camille): Info: FTS Xapian: fts_backend_xapian_check_access > doveadm(camille): Info: FTS Xapian: Memory stats : Used = 56 MB, Free > > 66 MB > doveadm(camille): Info: FTS Xapian: fts_backend_xapian_index_hdr > doveadm(camille): Info: FTS Xapian: fts_backend_xapian_query > doveadm(camille): Info: FTS Xapian: Query= uid:"44" > doveadm(camille): Info: FTS Xapian: Ngram(S) -> 63 items (total 0 KB) > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_unset_build_key > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=Message- > Id,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_build_more > doveadm(camille): Info: FTS Xapian: fts_backend_xapian_check_access > doveadm(camille): Info: FTS Xapian: Memory stats : Used = 56 MB, Free > > 66 MB > doveadm(camille): Info: FTS Xapian: fts_backend_xapian_index_hdr > doveadm(camille): Info: FTS Xapian: fts_backend_xapian_query > doveadm(camille): Info: FTS Xapian: Query= uid:"44" > doveadm(camille): Info: FTS Xapian: Ngram(XMID) -> 4 items (total 0 > KB) > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_unset_build_key > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=X- > Mailer,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: Unknown header (indexing) > 'xmailer' > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=MIME- > Version,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: Unknown header (indexing) > 'mimeversion' > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=Content- > Type,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: Unknown header (indexing) > 'contenttype' > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=Authentication- > Results,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: Unknown header (indexing) > 'authenticationresults' > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=X-AV- > Checked,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: Unknown header (indexing) > 'xavchecked' > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=Content- > Type,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: Unknown header (indexing) > 'contenttype' > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part > (Header=(null),Type=text/plain,Disposition=(null)) > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_build_more > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_unset_build_key > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=Content- > Type,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: Unknown header (indexing) > 'contenttype' > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=Content- > Description,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: Unknown header (indexing) > 'contentdescription' > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=Content- > Disposition,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: Unknown header (indexing) > 'contentdisposition' > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=Content- > Transfer- > Encoding,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: Unknown header (indexing) > 'contenttransferencoding' > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part > (Header=(null),Type=text/csv,Disposition=attachment; > filename="file.csv") > doveadm(camille): Info: FTS Xapian: Skipping part of type 'text/csv' > and disposition 'attachment; filename="file.csv"' > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_mailbox > doveadm(camille): Info: FTS Xapian: Unset box 'INBOX' > (c0d4e304584e5460dae30000075d7e67) > doveadm(camille): Info: FTS Xapian: fts_backend_xapian_oldbox > doveadm(camille): Info: FTS Xapian: Done indexing 'INBOX' > (c0d4e304584e5460dae30000075d7e67) (13 msgs in 261 ms, rate: 49.8) > doveadm(camille): Info: FTS Xapian: fts_backend_xapian_release > (unset_box) > doveadm(camille): Info: FTS Xapian: Committed 'unset_box' in 17 ms > doveadm(camille): Info: FTS Xapian: Box is empty > doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_deinit > (/home/users/camille/mails/indexes/xapian-indexes) > doveadm(camille): Info: FTS Xapian: fts_backend_xapian_release > (update_deinit) > doveadm(camille): Info: FTS Xapian: Committed 'update_deinit' in 0 ms > doveadm(camille): Info: FTS Xapian: Deinit > /home/users/camille/mails/indexes/xapian-indexes) > ------------------------------------------------------------------- > ---- > > Especially thos line: > > doveadm(camille): Info: FTS Xapian: Skipping part of type 'text/csv' > and disposition 'attachment; filename="file.csv"' > > I have seen this as a bug on fts-xapian: > https://github.com/grosjo/fts-xapian/issues/68 > > Which then references a PR on Dovecot core itself: > > https://github.com/dovecot/core/pull/155 > > Any idea on how to use xapian with non-text attachments ? > > Thanks a lot for your help! > > Andr? >
It is now very out of date. @Jello : Kindly update please On 2021-03-21 12:58, Andr? Rodier wrote:> Hello, > > The version packaged on Bullseye is slightly out of date, I have filled > a bug report: > > https://bugs.debian.org/985654 > > Thanks to the maintainers for their hard work! > > Andr? > > On Sun, 2021-03-21 at 10:51 +0000, Andr? Rodier wrote: > >> Hello, >> >> I am developing a hosting platform on Debian Bullseye, with Dovecot >> amongst other tools. >> >> I am trying to use the xapian full test search plugin, but I can see >> the attachments are skipped: >> >> This is what I have in the logs when running the indexing in verbose >> mode: >> >> ------------------------------------------------------------------- >> ---- >> doveadm(camille): Info: FTS Xapian: fts_backend_xapian_check_access >> doveadm(camille): Info: FTS Xapian: Memory stats : Used = 56 MB, Free >> >> 66 MB >> doveadm(camille): Info: FTS Xapian: fts_backend_xapian_index_hdr >> doveadm(camille): Info: FTS Xapian: fts_backend_xapian_query >> doveadm(camille): Info: FTS Xapian: Query= uid:"44" >> doveadm(camille): Info: FTS Xapian: Ngram(S) -> 63 items (total 0 KB) >> doveadm(camille): Info: FTS Xapian: >> fts_backend_xapian_update_unset_build_key >> doveadm(camille): Info: FTS Xapian: >> fts_backend_xapian_update_set_build_key >> doveadm(camille): Info: FTS Xapian: New part (Header=Message- >> Id,Type=(null),Disposition=(null)) >> doveadm(camille): Info: FTS Xapian: >> fts_backend_xapian_update_build_more >> doveadm(camille): Info: FTS Xapian: fts_backend_xapian_check_access >> doveadm(camille): Info: FTS Xapian: Memory stats : Used = 56 MB, Free >> >> 66 MB >> doveadm(camille): Info: FTS Xapian: fts_backend_xapian_index_hdr >> doveadm(camille): Info: FTS Xapian: fts_backend_xapian_query >> doveadm(camille): Info: FTS Xapian: Query= uid:"44" >> doveadm(camille): Info: FTS Xapian: Ngram(XMID) -> 4 items (total 0 >> KB) >> doveadm(camille): Info: FTS Xapian: >> fts_backend_xapian_update_unset_build_key >> doveadm(camille): Info: FTS Xapian: >> fts_backend_xapian_update_set_build_key >> doveadm(camille): Info: FTS Xapian: New part (Header=X- >> Mailer,Type=(null),Disposition=(null)) >> doveadm(camille): Info: FTS Xapian: Unknown header (indexing) >> 'xmailer' >> doveadm(camille): Info: FTS Xapian: >> fts_backend_xapian_update_set_build_key >> doveadm(camille): Info: FTS Xapian: New part (Header=MIME- >> Version,Type=(null),Disposition=(null)) >> doveadm(camille): Info: FTS Xapian: Unknown header (indexing) >> 'mimeversion' >> doveadm(camille): Info: FTS Xapian: >> fts_backend_xapian_update_set_build_key >> doveadm(camille): Info: FTS Xapian: New part (Header=Content- >> Type,Type=(null),Disposition=(null)) >> doveadm(camille): Info: FTS Xapian: Unknown header (indexing) >> 'contenttype' >> doveadm(camille): Info: FTS Xapian: >> fts_backend_xapian_update_set_build_key >> doveadm(camille): Info: FTS Xapian: New part (Header=Authentication- >> Results,Type=(null),Disposition=(null)) >> doveadm(camille): Info: FTS Xapian: Unknown header (indexing) >> 'authenticationresults' >> doveadm(camille): Info: FTS Xapian: >> fts_backend_xapian_update_set_build_key >> doveadm(camille): Info: FTS Xapian: New part (Header=X-AV- >> Checked,Type=(null),Disposition=(null)) >> doveadm(camille): Info: FTS Xapian: Unknown header (indexing) >> 'xavchecked' >> doveadm(camille): Info: FTS Xapian: >> fts_backend_xapian_update_set_build_key >> doveadm(camille): Info: FTS Xapian: New part (Header=Content- >> Type,Type=(null),Disposition=(null)) >> doveadm(camille): Info: FTS Xapian: Unknown header (indexing) >> 'contenttype' >> doveadm(camille): Info: FTS Xapian: >> fts_backend_xapian_update_set_build_key >> doveadm(camille): Info: FTS Xapian: New part >> (Header=(null),Type=text/plain,Disposition=(null)) >> doveadm(camille): Info: FTS Xapian: >> fts_backend_xapian_update_build_more >> doveadm(camille): Info: FTS Xapian: >> fts_backend_xapian_update_unset_build_key >> doveadm(camille): Info: FTS Xapian: >> fts_backend_xapian_update_set_build_key >> doveadm(camille): Info: FTS Xapian: New part (Header=Content- >> Type,Type=(null),Disposition=(null)) >> doveadm(camille): Info: FTS Xapian: Unknown header (indexing) >> 'contenttype' >> doveadm(camille): Info: FTS Xapian: >> fts_backend_xapian_update_set_build_key >> doveadm(camille): Info: FTS Xapian: New part (Header=Content- >> Description,Type=(null),Disposition=(null)) >> doveadm(camille): Info: FTS Xapian: Unknown header (indexing) >> 'contentdescription' >> doveadm(camille): Info: FTS Xapian: >> fts_backend_xapian_update_set_build_key >> doveadm(camille): Info: FTS Xapian: New part (Header=Content- >> Disposition,Type=(null),Disposition=(null)) >> doveadm(camille): Info: FTS Xapian: Unknown header (indexing) >> 'contentdisposition' >> doveadm(camille): Info: FTS Xapian: >> fts_backend_xapian_update_set_build_key >> doveadm(camille): Info: FTS Xapian: New part (Header=Content- >> Transfer- >> Encoding,Type=(null),Disposition=(null)) >> doveadm(camille): Info: FTS Xapian: Unknown header (indexing) >> 'contenttransferencoding' >> doveadm(camille): Info: FTS Xapian: >> fts_backend_xapian_update_set_build_key >> doveadm(camille): Info: FTS Xapian: New part >> (Header=(null),Type=text/csv,Disposition=attachment; >> filename="file.csv") >> doveadm(camille): Info: FTS Xapian: Skipping part of type 'text/csv' >> and disposition 'attachment; filename="file.csv"' >> doveadm(camille): Info: FTS Xapian: >> fts_backend_xapian_update_set_mailbox >> doveadm(camille): Info: FTS Xapian: Unset box 'INBOX' >> (c0d4e304584e5460dae30000075d7e67) >> doveadm(camille): Info: FTS Xapian: fts_backend_xapian_oldbox >> doveadm(camille): Info: FTS Xapian: Done indexing 'INBOX' >> (c0d4e304584e5460dae30000075d7e67) (13 msgs in 261 ms, rate: 49.8) >> doveadm(camille): Info: FTS Xapian: fts_backend_xapian_release >> (unset_box) >> doveadm(camille): Info: FTS Xapian: Committed 'unset_box' in 17 ms >> doveadm(camille): Info: FTS Xapian: Box is empty >> doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_deinit >> (/home/users/camille/mails/indexes/xapian-indexes) >> doveadm(camille): Info: FTS Xapian: fts_backend_xapian_release >> (update_deinit) >> doveadm(camille): Info: FTS Xapian: Committed 'update_deinit' in 0 ms >> doveadm(camille): Info: FTS Xapian: Deinit >> /home/users/camille/mails/indexes/xapian-indexes) >> ------------------------------------------------------------------- >> ---- >> >> Especially thos line: >> >> doveadm(camille): Info: FTS Xapian: Skipping part of type 'text/csv' >> and disposition 'attachment; filename="file.csv"' >> >> I have seen this as a bug on fts-xapian: >> https://github.com/grosjo/fts-xapian/issues/68 >> >> Which then references a PR on Dovecot core itself: >> >> https://github.com/dovecot/core/pull/155 >> >> Any idea on how to use xapian with non-text attachments ? >> >> Thanks a lot for your help! >> >> Andr?-------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20210703/5d38c828/attachment-0001.html>