Thanks Aki, I'll check it out :) Am 03.08.20 um 11:40 schrieb Aki Tuomi:>> On 03/08/2020 12:31 Ralf Becker <rb at egroupware.org> wrote: >> >> >> We're currently using the OX push driver, which is straight forward >> (simple web hook) and allows to store (multiple) push tokens of our >> webmailer direct in mailbox metadata. >> >> Only drawback is that it only supports new arriving mails in the INBOX, >> even mails moved via Sieve to other folders are NOT reported. >> >> Therefore we updated now to Dovecot 2.3(.10.1) to also get mails moved >> by user or Sieve scripts, deleted mails or flag changes. >> >> As far as I read the example Lua scripts and (a little) the Dovecot C >> code, the nice indirection of using mailbox metadata to a) enable push >> and b) store push tokens (modify reported user attribute with them) does >> NOT exist in the Lua driver by default. >> >> So my questions is: how can I access mailbox metadata from within the >> Lua script, to make eg. the example script behave like the OX driver >> with user_from_metadata set? >> >> I'm happy to update the Lua examples, if that's any help ... >> >> Ralf >> >> -- >> Ralf Becker >> EGroupware GmbH [www.egroupware.org] >> Handelsregister HRB Kaiserslautern 3587 >> Gesch?ftsf?hrer Birgit und Ralf Becker >> Leibnizstr. 17, 67663 Kaiserslautern, Germany >> Telefon +49 631 31657-0 > Actually it does exist: > > https://doc.dovecot.org/admin_manual/lua/#mail_user.metadata_get > > or > > https://doc.dovecot.org/admin_manual/lua/#object-mailbox > > mailbox:metadata_get() > > You get both of these objects from the push notification data, you just have to keep them in the context state. (See the example scripts) > > function dovecot_lua_notify_begin_txn(user) > local meta = user:metadata_get("/private/key") > return {messages={}, ep=user:plugin_getenv("push_lua_url"), username=user.username, meta=meta} > end > > Aki-- Ralf Becker EGroupware GmbH [www.egroupware.org] Handelsregister HRB Kaiserslautern 3587 Gesch?ftsf?hrer Birgit und Ralf Becker Leibnizstr. 17, 67663 Kaiserslautern, Germany Telefon +49 631 31657-0 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: <https://dovecot.org/pipermail/dovecot/attachments/20200803/1cff792c/attachment-0001.sig>
Making progress :)
I'll document some obtracles I found, to make it easier for the next one
implementing push with Dovecot and Lua.
First I tried with my usual Alpine based container, but Alpine seems not
to build the Lua stuff for Dovecot :(
So I moved to an Ubuntu 18.04 based container and the official Dovecot
CE repo:
FROM ubuntu:18.04
RUN apt-get update && \
apt-get install -y apt-transport-https gpg curl && \
curl https://repo.dovecot.org/DOVECOT-REPO-GPG | gpg --import && \
gpg --export ED409DA1 > /etc/apt/trusted.gpg.d/dovecot.gpg && \ echo
"deb https://repo.dovecot.org/ce-2.3-latest/ubuntu/bionic bionic main"
> /etc/apt/sources.list.d/dovecot.list && \
apt-get update && \
bash -c "apt-get install -y
dovecot-{core,imapd,sqlite,managesieved,sieve,pop3d,lmtpd,submissiond,lua}
lua-socket"
CMD [
"/usr/sbin/dovecot","-F","-c","/etc/dovecot/dovecot.conf"
]
I had to install lua-socket, which is used by the example script and not
required by dovecot-lua, which is ok, you just need to know.
Using Aki's code snippet as user=<metadata> lead to an other error:
Aug 03 14:54:15 Error: doveadm: lua: /usr/share/lua/5.2/socket/url.lua:31: bad
argument #1 to 'gsub' (string expected, got nil)
Aug 03 14:54:15 Error: lmtp(<username> 38): lmtp-server: conn
10.9.94.14:42092 [1]: rcpt <username>: lua:
/usr/share/lua/5.2/socket/url.lua:31: bad argument #1 to 'gsub' (string
expected, got nil)
I'm now skipping the notification, if no metadata is set, like the OX
driver does:
function dovecot_lua_notify_event_message_new(ctx, event)
-- check if there is a push token registered if (ctx.meta == nil or ctx.meta
== '') then return end
Currently looking into the following questions:
- can I get the rfc 5423 type of event somehow (obviously I can set it
on the event myself depending of the function called)
- looking at the example code, it looks like it can be called for
multiple messages, when does that happen (LMTP send more then one)
- why is the mailbox status put into an other structure and send with a
different notifiction
- does anyone have a code snippet to send a JSON encoded message
(probably easy to figure out looking at Lua docu)
Ralf
Am 03.08.20 um 11:56 schrieb Ralf Becker:> Thanks Aki, I'll check it out :)
>
>
> Am 03.08.20 um 11:40 schrieb Aki Tuomi:
>>> On 03/08/2020 12:31 Ralf Becker <rb at egroupware.org> wrote:
>>>
>>>
>>> We're currently using the OX push driver, which is straight
forward
>>> (simple web hook) and allows to store (multiple) push tokens of our
>>> webmailer direct in mailbox metadata.
>>>
>>> Only drawback is that it only supports new arriving mails in the
INBOX,
>>> even mails moved via Sieve to other folders are NOT reported.
>>>
>>> Therefore we updated now to Dovecot 2.3(.10.1) to also get mails
moved
>>> by user or Sieve scripts, deleted mails or flag changes.
>>>
>>> As far as I read the example Lua scripts and (a little) the Dovecot
C
>>> code, the nice indirection of using mailbox metadata to a) enable
push
>>> and b) store push tokens (modify reported user attribute with them)
does
>>> NOT exist in the Lua driver by default.
>>>
>>> So my questions is: how can I access mailbox metadata from within
the
>>> Lua script, to make eg. the example script behave like the OX
driver
>>> with user_from_metadata set?
>>>
>>> I'm happy to update the Lua examples, if that's any help
...
>>>
>>> Ralf
>>>
>>> --
>>> Ralf Becker
>>> EGroupware GmbH [www.egroupware.org]
>>> Handelsregister HRB Kaiserslautern 3587
>>> Gesch?ftsf?hrer Birgit und Ralf Becker
>>> Leibnizstr. 17, 67663 Kaiserslautern, Germany
>>> Telefon +49 631 31657-0
>> Actually it does exist:
>>
>> https://doc.dovecot.org/admin_manual/lua/#mail_user.metadata_get
>>
>> or
>>
>> https://doc.dovecot.org/admin_manual/lua/#object-mailbox
>>
>> mailbox:metadata_get()
>>
>> You get both of these objects from the push notification data, you just
have to keep them in the context state. (See the example scripts)
>>
>> function dovecot_lua_notify_begin_txn(user)
>> local meta = user:metadata_get("/private/key")
>> return {messages={},
ep=user:plugin_getenv("push_lua_url"), username=user.username,
meta=meta}
>> end
>>
>> Aki
--
Ralf Becker
EGroupware GmbH [www.egroupware.org]
Handelsregister HRB Kaiserslautern 3587
Gesch?ftsf?hrer Birgit und Ralf Becker
Leibnizstr. 17, 67663 Kaiserslautern, Germany
Telefon +49 631 31657-0
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20200803/06d89cb4/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20200803/06d89cb4/attachment-0001.sig>
Some answers to my questions, a first version of my script and more questions ;) Am 03.08.20 um 18:15 schrieb Ralf Becker:> Currently looking into the following questions: > > - can I get the rfc 5423 type of event somehow (obviously I can set it > on the event myself depending of the function called)event.name> - looking at the example code, it looks like it can be called for > multiple messages, when does that happen (LMTP send more then one)still no idea, maybe Ake? I noticed that some events have the same uid-validity, are the from a single transaction, eg. I delete my Trash?> - why is the mailbox status put into an other structure and send with > a different notifiction > - does anyone have a code snippet to send a JSON encoded message > (probably easy to figure out looking at Lua docu)these two I managed to solve im my current version of the script, which also support now all message event types: -- To use -- -- plugin { -- push_notification_driver lua:file=/etc/dovecot/dovecot-push.lua -- push_lua_url http://push.notification.server/handler -- } -- -- server is sent a PUT message with JSON body like push_notification_driver ox:url=<push_lua_url> user_from_metadata -- local http = require "socket.http" local ltn12 = require "ltn12" -- luarocks install json-lua local json = require "JSON" function table_get(t, k, d) return t[k] or d end function script_init() return 0 end function dovecot_lua_notify_begin_txn(user) local meta = user:metadata_get("/private/vendor/vendor.dovecot/http-notify") return {user=user, event=dovecot.event(), ep=user:plugin_getenv("push_lua_url"), messages={}, meta=meta} end function dovecot_lua_notify_event_message_new(ctx, event) -- check if there is a push token registered if (ctx.meta == nil or ctx.meta == '') then return end -- get mailbox status local mbox = ctx.user:mailbox(event.mailbox) mbox:sync() local status = mbox:status(dovecot.storage.STATUS_RECENT, dovecot.storage.STATUS_UNSEEN, dovecot.storage.STATUS_MESSAGES) mbox:free() table.insert(ctx.messages, { user = ctx.meta, ["imap-uidvalidity"] = event.uid_validity, ["imap-uid"] = event.uid, folder = event.mailbox, event = event.name, from = event.from, subject = event.subject, snippet = event.snippet, unseen = status.unseen }) end function dovecot_lua_notify_event_message_append(ctx, event) dovecot_lua_notify_event_message_new(ctx, event) end function dovecot_lua_notify_event_message_read(ctx, event) -- check if there is a push token registered if (ctx.meta == nil or ctx.meta == '') then return end -- get mailbox status local mbox = ctx.user:mailbox(event.mailbox) mbox:sync() local status = mbox:status(dovecot.storage.STATUS_RECENT, dovecot.storage.STATUS_UNSEEN, dovecot.storage.STATUS_MESSAGES) mbox:free() table.insert(ctx.messages, { user = ctx.meta, ["imap-uidvalidity"] = event.uid_validity, ["imap-uid"] = event.uid, folder = event.mailbox, event = event.name, unseen = status.unseen }) end function dovecot_lua_notify_event_message_trash(ctx, event) dovecot_lua_notify_event_message_read(ctx, event) end function dovecot_lua_notify_event_message_expunge(ctx, event) dovecot_lua_notify_event_message_read(ctx, event) end function dovecot_lua_notify_event_flags_set(ctx, event) -- check if there is a push token registered if (ctx.meta == nil or ctx.meta == '') then return end table.insert(ctx.messages, { user = ctx.meta, ["imap-uidvalidity"] = event.uid_validity, ["imap-uid"] = event.uid, folder = event.mailbox, event = event.name, flags = event.flags, ["keywords-set"] = event.keywords_set }) end function dovecot_lua_notify_event_flags_clear(ctx, event) -- check if there is a push token registered if (ctx.meta == nil or ctx.meta == '') then return end table.insert(ctx.messages, { user = ctx.meta, ["imap-uidvalidity"] = event.uid_validity, ["imap-uid"] = event.uid, folder = event.mailbox, event = event.name, flags = event.flags, ["keywords-clear"] = event.keywords_clear, ["keywords-old"] = event.keywords_old }) end function dovecot_lua_notify_end_txn(ctx) -- report all states for i,msg in ipairs(ctx.messages) do local e = dovecot.event(ctx.event) e:set_name("lua_notify_mail_finished") reqbody = json:encode(msg) e:log_debug(ctx.ep .. " - sending " .. reqbody) res, code = http.request({ method = "PUT", url = ctx.ep, source = ltn12.source.string(reqbody), headers={ ["content-type"] = "application/json; charset=utf-8", ["content-length"] = tostring(#reqbody) } }) e:add_int("result_code", code) e:log_info("Mail notify status " .. tostring(code)) end end This leads to a couple more questions ;) - is there a way (eg. return value) to stop event processing already in dovecot_lua_notify_begin_txn - sometimes multiple events are generated, eg. when I read an email: {"event":"FlagsClear","flags":[],"folder":"INBOX/eGroupWare/calconnect","imap-uid":2275,"imap-uidvalidity":1499767470,"keywords-old":[],"user":"user=5::42;***"} {"event":"FlagsSet","flags":["\\Seen"],"folder":"INBOX/eGroupWare/calconnect","imap-uid":2275,"imap-uidvalidity":1499767470,"user":"user=5::42;***"} {"event":"MessageRead","folder":"INBOX/eGroupWare/calconnect","imap-uid":2275,"imap-uidvalidity":1499767470,"unseen":0,"user":"user=5::42;***"} Ralf -- Ralf Becker EGroupware GmbH [www.egroupware.org] Handelsregister HRB Kaiserslautern 3587 Gesch?ftsf?hrer Birgit und Ralf Becker Leibnizstr. 17, 67663 Kaiserslautern, Germany Telefon +49 631 31657-0 -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20200803/581220a1/attachment-0001.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: <https://dovecot.org/pipermail/dovecot/attachments/20200803/581220a1/attachment-0001.sig>