dovecot - Aug 2016 - RFC 3501 violation in FETCH BODY responses

Hi there,

Quoting RFC 3501 sec. 7.4.2 ?FETCH Response? (data item BODYSTRUCTURE):

    ?A body type of type MESSAGE and subtype RFC822 contains,
     immediately after the basic fields, the envelope structure, body
     structure, and size in text lines of the encapsulated message.?

According the ABNF (RFC 3501 sec. 9) the envelope structure is that of
the ENVELOPE FETCH data item, and the env-{from,sender,reply-to,to,cc,
bcc} fields are non-space-separated address lists:

    body-type-msg = media-message SP body-fields SP envelope SP body SP
body-fld-lines
    envelope      = "(" env-from SP ? SP env-to SP ? ")"
    env-from      = "(" 1*address ")" / nil
    env-to        = "(" 1*address ")" / nil

While this is indeed the case for ?FETCH ? (ENVELOPE)?, for ?FETCH ?
(BODY)? dovecot 2.2.25 adds a space between addresses of an address list
of the envelope structure of an encapsulated MESSAGE/RFC822 message.

See the attached patch to ?src/lib-imap/test-imap-bodystructure.c?,
which currently (2.2.25) fails as follows

    test-imap-bodystructure.c:122: Assert failed: strcmp(str_c(str),
testmsg_body) == 0
    test-imap-bodystructure.c:129: Assert failed: strcmp(str_c(str),
testmsg_bodystructure) == 0
    imap bodystructure parser ............................................ :
FAILED

because the ?env-to? field of the envelope structure of the encapsulated
MESSAGE/RFC822 message is printed as

    ((NIL NIL "sub-to1" "domain.org") (NIL NIL
"sub-to2" "domain.org"))

while it should be

    ((NIL NIL "sub-to1" "domain.org")(NIL NIL
"sub-to2" "domain.org"))

After a quick look at the source, this seems to be due to
src/lib-imap/imap-bodystructure.c:imap_write_list, which always
separates list items with spaces.  In the case of an envelope, only the
top-level list should be space-separated.  Indeed, not adding a space
between items of type IMAP_ARG_LIST in the recursive call makes the test
pass again.

Cheers,
-- 
Guilhem.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test-imap-bodystructure.diff
Type: text/x-diff
Size: 3361 bytes
Desc: not available
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20160822/9f581aef/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: not available
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20160822/9f581aef/attachment-0001.sig>

Op 8/22/2016 om 2:29 PM schreef Guilhem Moulin:
> Hi there,
>
> Quoting RFC 3501 sec. 7.4.2 ?FETCH Response? (data item BODYSTRUCTURE):
>
>     ?A body type of type MESSAGE and subtype RFC822 contains,
>      immediately after the basic fields, the envelope structure, body
>      structure, and size in text lines of the encapsulated message.?
>
> According the ABNF (RFC 3501 sec. 9) the envelope structure is that of
> the ENVELOPE FETCH data item, and the env-{from,sender,reply-to,to,cc,
> bcc} fields are non-space-separated address lists:
>
>     body-type-msg = media-message SP body-fields SP envelope SP body SP
body-fld-lines
>     envelope      = "(" env-from SP ? SP env-to SP ?
")"
>     env-from      = "(" 1*address ")" / nil
>     env-to        = "(" 1*address ")" / nil
>
> While this is indeed the case for ?FETCH ? (ENVELOPE)?, for ?FETCH ?
> (BODY)? dovecot 2.2.25 adds a space between addresses of an address list
> of the envelope structure of an encapsulated MESSAGE/RFC822 message.
>
> See the attached patch to ?src/lib-imap/test-imap-bodystructure.c?,
> which currently (2.2.25) fails as follows
>
>     test-imap-bodystructure.c:122: Assert failed: strcmp(str_c(str),
testmsg_body) == 0
>     test-imap-bodystructure.c:129: Assert failed: strcmp(str_c(str),
testmsg_bodystructure) == 0
>     imap bodystructure parser ............................................
: FAILED
>
> because the ?env-to? field of the envelope structure of the encapsulated
> MESSAGE/RFC822 message is printed as
>
>     ((NIL NIL "sub-to1" "domain.org") (NIL NIL
"sub-to2" "domain.org"))
>
> while it should be
>
>     ((NIL NIL "sub-to1" "domain.org")(NIL NIL
"sub-to2" "domain.org"))
>
> After a quick look at the source, this seems to be due to
> src/lib-imap/imap-bodystructure.c:imap_write_list, which always
> separates list items with spaces.  In the case of an envelope, only the
> top-level list should be space-separated.  Indeed, not adding a space
> between items of type IMAP_ARG_LIST in the recursive call makes the test
> pass again.
Fixed:
https://github.com/dovecot/core/commit/f549b400d50935754cbeb6ceabd922ab777b4d77

Regards,

Stephan.