Serveria Support
2022-Aug-30 12:33 UTC
Dovecot mail-crypt webmail can't read encrypted messages
Update: I managed to remove the master user query so users are not getting marked as master_user on login. However, that doesn't seem to affect anything. I'm still unable to read encoded emails in webmail. No new errors are showing up in the log. I have even created a brand new user and all new keys via doveadm but still no go. Any suggestions? On 2022-08-29 16:30, Serveria Support wrote:> Upon closer review, it seems you're probably right: both users are in > fact marked master_user. How is that possible? I haven't marked new > user as a master_user. Are users marked master_user by default? What's > even more interesting, /etc/dovecot/dovecot-master-users doesn't > contain this user's data. Is it possible to unset this master_user > flag somehow? I browsed through the db in mySQL but wasn't able to > locate any master_users as well. Sorry for being such a noob... :) > > On 2022-08-29 11:33, Aki Tuomi wrote: >> Hard to say. >> >> If you are logging is master_user, there will be different password >> than normal user. Usually. With your setup, you can only access user's >> mail if you are using the exact same password that the user was using. >> >> Your logs seem to indicate that you are logging as master_user, so you >> are probably unable to access mails. >> >> Aki >> >> >>> On 29/08/2022 10:51 EEST Serveria Support <support at serveria.com> >>> wrote: >>> >>> >>> Emm, sorry for the confusion, there are two users authenticating - >>> master user "postmaster" and the second user called "test". I have >>> just >>> obfuscated users by replacing usernames with myuser. So no, this >>> shouldn't be the issue. >>> >>> Any other suggestions? >>> >>> On 2022-08-29 10:30, Aki Tuomi wrote: >>> >> On 29/08/2022 09:26 EEST Serveria Support <support at serveria.com> >>> >> wrote: >>> >> >>> >> >>> >> It's a testing install my main goal is to make it work. I will play >>> >> around with password encryption before going live. >>> >> >>> >> I have enabled all possible debugging yet I can's see the value you >>> >> mentioned in the log file. Could you please point me? >>> >> >>> >> Aug 29 01:46:30 mx dovecot: auth-worker(648543): Debug: conn >>> >> unix:auth-worker (pid=648542,uid=110): auth-worker<1>: >>> >> sql(myuser at mydomain.xyz,xx.xx.xx.xx,<Y+4ayVrnJ9VV/kpz>): query: SELECT >>> >> mailbox.password, mailbox.allow_nets FROM mailbox,domain WHERE >>> >> mailbox.username='myuser at mydomain.xyz' AND mailbox.`enableimaptls`=1 >>> >> AND >>> >> mailbox.active=1 AND mailbox.domain=domain.domain AND >>> >> domain.backupmx=0 >>> >> AND domain.active=1 >>> > >>> > it's not set here. >>> > >>> > >>> >> Aug 29 01:46:30 mx dovecot: auth-worker(648543): Debug: conn >>> >> unix:auth-worker (pid=648542,uid=110): auth-worker<2>: >>> >> sql(myuser at mydomain.xyz,xx.xx.xx.xx,<Y+4ayVrnJ9VV/kpz>): SELECT >>> >> LOWER('myuser at mydomain.xyz') AS master_user, >>> >> LOWER(CONCAT(mailbox.storagebasedirectory, '/', mailbox.storagenode, >>> >> '/', mailbox.maildir)) AS home, CONCAT(mailbox.mailboxformat, ':~/', >>> >> mailbox.mailboxfolder) AS mail, CONCAT('*:bytes=', >>> >> mailbox.quota*1048576) AS quota_rule FROM mailbox,domain WHERE >>> >> mailbox.username='myuser at mydomain.xyz' AND mailbox.`enableimaptls`=1 >>> >> AND >>> >> mailbox.active=1 AND mailbox.domain=domain.domain AND >>> >> domain.backupmx=0 >>> >> AND domain.active=1 >>> > >>> > it's not set here either. >>> > >>> > So. You are doing master user login, and are wondering why user's >>> > password is not available? >>> > >>> > Master user logins are not really compatible with using user's >>> > password as encryption key. >>> > >>> > Aki >>> > >>> >> >>> >> On 2022-08-29 07:56, Aki Tuomi wrote: >>> >> >> On 28/08/2022 09:20 EEST Serveria Support <support at serveria.com> >>> >> >> wrote: >>> >> >> >>> >> >> >>> >> >> I'm trying to setup Dovecot with mail-crypt plugin with per-user >>> >> >> encryption. >>> >> >> >>> >> >> I have configured mail-crypt plugin as per official guide here: >>> >> >> https://doc.dovecot.org/configuration_manual/mail_crypt_plugin/ >>> >> >> >>> >> >> After that I created a user and an encrypted key by running this >>> >> >> command: doveadm -o \plugin/mail_crypt_private_password=12345 mailbox >>> >> >> cryptokey generate -u mail at example.org -URf (replacing dummy data ofc) >>> >> >> >>> >> >> I can log in to webmail (and Dovecot) just fine, emails are getting >>> >> >> sent >>> >> >> and delivered. I have also checked the storage and the messages seem >>> >> >> to >>> >> >> be stored encrypted. >>> >> >> >>> >> >> However, I can't read the emails in webmail (just headers can be seen) >>> >> >> and in Dovecot logs I can see the following error: >>> >> >> >>> >> >> failed: Private key not available: Cannot decrypt key ### Cannot >>> >> >> decrypt >>> >> >> key ### <8632: Password not available (FETCH RFC822.HEADER) >>> >> >> >>> >> >> There seems to be an issue with mySQL query. The query I'm using >>> >> >> (Select >>> >> >> username as "user", password,"%w" as >>> >> >> userdb_mail_crypt_private_password >>> >> >> from mailbox;) seems to work just fine, when run from mysql prompt it >>> >> >> outputs the usernames and passwords, but the error is still there >>> >> >> (Cannot decrypt key ### Password not available). >>> >> >> >>> >> >> Any ideas? What am I missing >>> >> > >>> >> > Hi! >>> >> > >>> >> > First of all, it's super-unsafe to use user's password like that as >>> >> > private password, at least run it through SHA256. This prevents >>> >> > dovecot from doing expansions on it by accident. >>> >> > >>> >> > Secondly, enable mail_debug=yes and auth_debug=yes, run it again, and >>> >> > make sure the correct value gets added as >>> >> > 'plugin/mail_crypt_private_password' when using with webmail. >>> >> > >>> >> > Aki
You need to return the private password in your passdb query, like SELECT .... '%w' AS userdb_mail_crypt_private_password ... not in your userdb query, as %w will not be available there. Aki> On 30/08/2022 15:33 EEST Serveria Support <support at serveria.com> wrote: > > > Update: I managed to remove the master user query so users are not > getting marked as master_user on login. However, that doesn't seem to > affect anything. I'm still unable to read encoded emails in webmail. No > new errors are showing up in the log. I have even created a brand new > user and all new keys via doveadm but still no go. Any suggestions? > > On 2022-08-29 16:30, Serveria Support wrote: > > Upon closer review, it seems you're probably right: both users are in > > fact marked master_user. How is that possible? I haven't marked new > > user as a master_user. Are users marked master_user by default? What's > > even more interesting, /etc/dovecot/dovecot-master-users doesn't > > contain this user's data. Is it possible to unset this master_user > > flag somehow? I browsed through the db in mySQL but wasn't able to > > locate any master_users as well. Sorry for being such a noob... :) > > > > On 2022-08-29 11:33, Aki Tuomi wrote: > >> Hard to say. > >> > >> If you are logging is master_user, there will be different password > >> than normal user. Usually. With your setup, you can only access user's > >> mail if you are using the exact same password that the user was using. > >> > >> Your logs seem to indicate that you are logging as master_user, so you > >> are probably unable to access mails. > >> > >> Aki > >> > >> > >>> On 29/08/2022 10:51 EEST Serveria Support <support at serveria.com> > >>> wrote: > >>> > >>> > >>> Emm, sorry for the confusion, there are two users authenticating - > >>> master user "postmaster" and the second user called "test". I have > >>> just > >>> obfuscated users by replacing usernames with myuser. So no, this > >>> shouldn't be the issue. > >>> > >>> Any other suggestions? > >>> > >>> On 2022-08-29 10:30, Aki Tuomi wrote: > >>> >> On 29/08/2022 09:26 EEST Serveria Support <support at serveria.com> > >>> >> wrote: > >>> >> > >>> >> > >>> >> It's a testing install my main goal is to make it work. I will play > >>> >> around with password encryption before going live. > >>> >> > >>> >> I have enabled all possible debugging yet I can's see the value you > >>> >> mentioned in the log file. Could you please point me? > >>> >> > >>> >> Aug 29 01:46:30 mx dovecot: auth-worker(648543): Debug: conn > >>> >> unix:auth-worker (pid=648542,uid=110): auth-worker<1>: > >>> >> sql(myuser at mydomain.xyz,xx.xx.xx.xx,<Y+4ayVrnJ9VV/kpz>): query: SELECT > >>> >> mailbox.password, mailbox.allow_nets FROM mailbox,domain WHERE > >>> >> mailbox.username='myuser at mydomain.xyz' AND mailbox.`enableimaptls`=1 > >>> >> AND > >>> >> mailbox.active=1 AND mailbox.domain=domain.domain AND > >>> >> domain.backupmx=0 > >>> >> AND domain.active=1 > >>> > > >>> > it's not set here. > >>> > > >>> > > >>> >> Aug 29 01:46:30 mx dovecot: auth-worker(648543): Debug: conn > >>> >> unix:auth-worker (pid=648542,uid=110): auth-worker<2>: > >>> >> sql(myuser at mydomain.xyz,xx.xx.xx.xx,<Y+4ayVrnJ9VV/kpz>): SELECT > >>> >> LOWER('myuser at mydomain.xyz') AS master_user, > >>> >> LOWER(CONCAT(mailbox.storagebasedirectory, '/', mailbox.storagenode, > >>> >> '/', mailbox.maildir)) AS home, CONCAT(mailbox.mailboxformat, ':~/', > >>> >> mailbox.mailboxfolder) AS mail, CONCAT('*:bytes=', > >>> >> mailbox.quota*1048576) AS quota_rule FROM mailbox,domain WHERE > >>> >> mailbox.username='myuser at mydomain.xyz' AND mailbox.`enableimaptls`=1 > >>> >> AND > >>> >> mailbox.active=1 AND mailbox.domain=domain.domain AND > >>> >> domain.backupmx=0 > >>> >> AND domain.active=1 > >>> > > >>> > it's not set here either. > >>> > > >>> > So. You are doing master user login, and are wondering why user's > >>> > password is not available? > >>> > > >>> > Master user logins are not really compatible with using user's > >>> > password as encryption key. > >>> > > >>> > Aki > >>> > > >>> >> > >>> >> On 2022-08-29 07:56, Aki Tuomi wrote: > >>> >> >> On 28/08/2022 09:20 EEST Serveria Support <support at serveria.com> > >>> >> >> wrote: > >>> >> >> > >>> >> >> > >>> >> >> I'm trying to setup Dovecot with mail-crypt plugin with per-user > >>> >> >> encryption. > >>> >> >> > >>> >> >> I have configured mail-crypt plugin as per official guide here: > >>> >> >> https://doc.dovecot.org/configuration_manual/mail_crypt_plugin/ > >>> >> >> > >>> >> >> After that I created a user and an encrypted key by running this > >>> >> >> command: doveadm -o \plugin/mail_crypt_private_password=12345 mailbox > >>> >> >> cryptokey generate -u mail at example.org -URf (replacing dummy data ofc) > >>> >> >> > >>> >> >> I can log in to webmail (and Dovecot) just fine, emails are getting > >>> >> >> sent > >>> >> >> and delivered. I have also checked the storage and the messages seem > >>> >> >> to > >>> >> >> be stored encrypted. > >>> >> >> > >>> >> >> However, I can't read the emails in webmail (just headers can be seen) > >>> >> >> and in Dovecot logs I can see the following error: > >>> >> >> > >>> >> >> failed: Private key not available: Cannot decrypt key ### Cannot > >>> >> >> decrypt > >>> >> >> key ### <8632: Password not available (FETCH RFC822.HEADER) > >>> >> >> > >>> >> >> There seems to be an issue with mySQL query. The query I'm using > >>> >> >> (Select > >>> >> >> username as "user", password,"%w" as > >>> >> >> userdb_mail_crypt_private_password > >>> >> >> from mailbox;) seems to work just fine, when run from mysql prompt it > >>> >> >> outputs the usernames and passwords, but the error is still there > >>> >> >> (Cannot decrypt key ### Password not available). > >>> >> >> > >>> >> >> Any ideas? What am I missing > >>> >> > > >>> >> > Hi! > >>> >> > > >>> >> > First of all, it's super-unsafe to use user's password like that as > >>> >> > private password, at least run it through SHA256. This prevents > >>> >> > dovecot from doing expansions on it by accident. > >>> >> > > >>> >> > Secondly, enable mail_debug=yes and auth_debug=yes, run it again, and > >>> >> > make sure the correct value gets added as > >>> >> > 'plugin/mail_crypt_private_password' when using with webmail. > >>> >> > > >>> >> > Aki
Serveria Support
2022-Sep-02 14:22 UTC
Dovecot mail-crypt webmail can't read encrypted messages
Still banging my head against the wall...
Upon running this query: SELECT username as user, password, '%w' AS
userdb_mail_crypt_private_password FROM mailbox;
I'm getting the following output:
+----------------------------+-----------------------------------------------------------------------------------------------------------+------------------------------------+
| user | password
|
userdb_mail_crypt_private_password |
+----------------------------+-----------------------------------------------------------------------------------------------------------+------------------------------------+
| user1 at mydomain.xyz |
{SSHA512}1Z3spq1ySn4dmF2vgv5DRNW5KedMJH/Yspi+33PUTGe/9T6k8Uad62pAnKMUXOhGJliXSkPK24dHWs7UfHtMJFUpJdOX8kV9
| %w |
| user2 at mydomain.xyz |
{SSHA512}xE+M7bD8nlbdfJl314iYmLgDjG0sud3oakQuxSYkcJYrLSoqgv9W7MD4ghtflZ32VgV6mnhcYIJrRK77tVCoDOG1dkA=
| %w |
| user3 at mydomain.xyz |
{SSHA512}LgmL2XS1b+cb/g/SVjWaic8fMn4h8pwud1Ark7rkpfcSOuIcib4eeKVO2yKW3BivulHuj3DA7s5591VLYiURek6SCJhKiqHo
| %w |
| user4 at mydomain.xyz |
{SSHA512}2e/VxDvjHbJZmBUMKYNlFM6Of7nbc5bDtm14Nz1IiFXQuq0VMgwOPCA8p9RTMav+KLX/tlCbKfkcbYzaNI/dHF2Nhiinyvnv
| %w |
| user5 at mydomain.xyz |
{SSHA512}/WDzitklDJ6yZH1eeRHljQvN1/MEB+bdXftSx7Sw2VBSyH21WlxjRx5dp3fiibhTNrHeIL5qWiADaBifJyfmKhn0Zi0TSwvp
| %w |
+----------------------------+-----------------------------------------------------------------------------------------------------------+------------------------------------+
5 rows in set (0.000 sec)
As soon as I'm adding WHERE username='%u'; at the end, query returns
0
rows.
This is the exact contents of my file:
password_query = SELECT \
username as user, password, \
'%w' AS userdb_mail_crypt_private_password \
FROM mailbox WHERE username="%u";
Is it ok to run it like this? Or should I construct a separate query
"user_query=" ? It was like this in your tutorial...
On 2022-08-30 16:13, Aki Tuomi wrote:> You need to return the private password in your passdb query, like
>
> SELECT .... '%w' AS userdb_mail_crypt_private_password ...
>
> not in your userdb query, as %w will not be available there.
>
> Aki
>
>> On 30/08/2022 15:33 EEST Serveria Support <support at
serveria.com>
>> wrote:
>>
>>
>> Update: I managed to remove the master user query so users are not
>> getting marked as master_user on login. However, that doesn't seem
to
>> affect anything. I'm still unable to read encoded emails in
webmail.
>> No
>> new errors are showing up in the log. I have even created a brand new
>> user and all new keys via doveadm but still no go. Any suggestions?
>>
>> On 2022-08-29 16:30, Serveria Support wrote:
>> > Upon closer review, it seems you're probably right: both users
are in
>> > fact marked master_user. How is that possible? I haven't
marked new
>> > user as a master_user. Are users marked master_user by default?
What's
>> > even more interesting, /etc/dovecot/dovecot-master-users
doesn't
>> > contain this user's data. Is it possible to unset this
master_user
>> > flag somehow? I browsed through the db in mySQL but wasn't
able to
>> > locate any master_users as well. Sorry for being such a noob... :)
>> >
>> > On 2022-08-29 11:33, Aki Tuomi wrote:
>> >> Hard to say.
>> >>
>> >> If you are logging is master_user, there will be different
password
>> >> than normal user. Usually. With your setup, you can only
access user's
>> >> mail if you are using the exact same password that the user
was using.
>> >>
>> >> Your logs seem to indicate that you are logging as
master_user, so you
>> >> are probably unable to access mails.
>> >>
>> >> Aki
>> >>
>> >>
>> >>> On 29/08/2022 10:51 EEST Serveria Support <support at
serveria.com>
>> >>> wrote:
>> >>>
>> >>>
>> >>> Emm, sorry for the confusion, there are two users
authenticating -
>> >>> master user "postmaster" and the second user
called "test". I have
>> >>> just
>> >>> obfuscated users by replacing usernames with myuser. So
no, this
>> >>> shouldn't be the issue.
>> >>>
>> >>> Any other suggestions?
>> >>>
>> >>> On 2022-08-29 10:30, Aki Tuomi wrote:
>> >>> >> On 29/08/2022 09:26 EEST Serveria Support
<support at serveria.com>
>> >>> >> wrote:
>> >>> >>
>> >>> >>
>> >>> >> It's a testing install my main goal is to
make it work. I will play
>> >>> >> around with password encryption before going
live.
>> >>> >>
>> >>> >> I have enabled all possible debugging yet I
can's see the value you
>> >>> >> mentioned in the log file. Could you please point
me?
>> >>> >>
>> >>> >> Aug 29 01:46:30 mx dovecot: auth-worker(648543):
Debug: conn
>> >>> >> unix:auth-worker (pid=648542,uid=110):
auth-worker<1>:
>> >>> >> sql(myuser at
mydomain.xyz,xx.xx.xx.xx,<Y+4ayVrnJ9VV/kpz>): query: SELECT
>> >>> >> mailbox.password, mailbox.allow_nets FROM
mailbox,domain WHERE
>> >>> >> mailbox.username='myuser at mydomain.xyz'
AND mailbox.`enableimaptls`=1
>> >>> >> AND
>> >>> >> mailbox.active=1 AND mailbox.domain=domain.domain
AND
>> >>> >> domain.backupmx=0
>> >>> >> AND domain.active=1
>> >>> >
>> >>> > it's not set here.
>> >>> >
>> >>> >
>> >>> >> Aug 29 01:46:30 mx dovecot: auth-worker(648543):
Debug: conn
>> >>> >> unix:auth-worker (pid=648542,uid=110):
auth-worker<2>:
>> >>> >> sql(myuser at
mydomain.xyz,xx.xx.xx.xx,<Y+4ayVrnJ9VV/kpz>): SELECT
>> >>> >> LOWER('myuser at mydomain.xyz') AS
master_user,
>> >>> >> LOWER(CONCAT(mailbox.storagebasedirectory,
'/', mailbox.storagenode,
>> >>> >> '/', mailbox.maildir)) AS home,
CONCAT(mailbox.mailboxformat, ':~/',
>> >>> >> mailbox.mailboxfolder) AS mail,
CONCAT('*:bytes=',
>> >>> >> mailbox.quota*1048576) AS quota_rule FROM
mailbox,domain WHERE
>> >>> >> mailbox.username='myuser at mydomain.xyz'
AND mailbox.`enableimaptls`=1
>> >>> >> AND
>> >>> >> mailbox.active=1 AND mailbox.domain=domain.domain
AND
>> >>> >> domain.backupmx=0
>> >>> >> AND domain.active=1
>> >>> >
>> >>> > it's not set here either.
>> >>> >
>> >>> > So. You are doing master user login, and are
wondering why user's
>> >>> > password is not available?
>> >>> >
>> >>> > Master user logins are not really compatible with
using user's
>> >>> > password as encryption key.
>> >>> >
>> >>> > Aki
>> >>> >
>> >>> >>
>> >>> >> On 2022-08-29 07:56, Aki Tuomi wrote:
>> >>> >> >> On 28/08/2022 09:20 EEST Serveria
Support <support at serveria.com>
>> >>> >> >> wrote:
>> >>> >> >>
>> >>> >> >>
>> >>> >> >> I'm trying to setup Dovecot with
mail-crypt plugin with per-user
>> >>> >> >> encryption.
>> >>> >> >>
>> >>> >> >> I have configured mail-crypt plugin as
per official guide here:
>> >>> >> >>
https://doc.dovecot.org/configuration_manual/mail_crypt_plugin/
>> >>> >> >>
>> >>> >> >> After that I created a user and an
encrypted key by running this
>> >>> >> >> command: doveadm -o
\plugin/mail_crypt_private_password=12345 mailbox
>> >>> >> >> cryptokey generate -u mail at
example.org -URf (replacing dummy data ofc)
>> >>> >> >>
>> >>> >> >> I can log in to webmail (and Dovecot)
just fine, emails are getting
>> >>> >> >> sent
>> >>> >> >> and delivered. I have also checked the
storage and the messages seem
>> >>> >> >> to
>> >>> >> >> be stored encrypted.
>> >>> >> >>
>> >>> >> >> However, I can't read the emails in
webmail (just headers can be seen)
>> >>> >> >> and in Dovecot logs I can see the
following error:
>> >>> >> >>
>> >>> >> >> failed: Private key not available:
Cannot decrypt key ### Cannot
>> >>> >> >> decrypt
>> >>> >> >> key ### <8632: Password not available
(FETCH RFC822.HEADER)
>> >>> >> >>
>> >>> >> >> There seems to be an issue with mySQL
query. The query I'm using
>> >>> >> >> (Select
>> >>> >> >> username as "user",
password,"%w" as
>> >>> >> >> userdb_mail_crypt_private_password
>> >>> >> >> from mailbox;) seems to work just fine,
when run from mysql prompt it
>> >>> >> >> outputs the usernames and passwords, but
the error is still there
>> >>> >> >> (Cannot decrypt key ### Password not
available).
>> >>> >> >>
>> >>> >> >> Any ideas? What am I missing
>> >>> >> >
>> >>> >> > Hi!
>> >>> >> >
>> >>> >> > First of all, it's super-unsafe to use
user's password like that as
>> >>> >> > private password, at least run it through
SHA256. This prevents
>> >>> >> > dovecot from doing expansions on it by
accident.
>> >>> >> >
>> >>> >> > Secondly, enable mail_debug=yes and
auth_debug=yes, run it again, and
>> >>> >> > make sure the correct value gets added as
>> >>> >> > 'plugin/mail_crypt_private_password'
when using with webmail.
>> >>> >> >
>> >>> >> > Aki