dovecot - Dec 2014 - stacking istreams and ostreams

Hello,

I'm developing an encryption plugin for dovecot and ran into a problem
with the stacking of i/o-streams.

The encryption i/o-streams are working fine on any kind of mail the test
suite is passing through them. But as soon as the zlib plugin is enabled
the logs show an cache error:

failed: Cached message size larger than expected (214 > 206, box=INBOX,
UID=1)

I've already double-checked the return values of ostream's sendv and
istream's read function. They seem correct (and equal).

If the order of the streams are changed (by changing the number in the
lib-filename libxx_scrambler.so); meaning that the encryption is done
before the compression (which isn't efficient) both streams are working
correct without any errors.

Is there some way the zlib plugin changes the cached message size? Is
there some behaviour of the zlib plugin that I'm missing? Any help would
be very welcome.

Best regards,
Philipp

-- 
simia.tech GbR
http://simiatech.com

Well, I've found the bug. I've got confused with the stream-stacking
function pointers. The encryption istream was stacked on top of the
parent and the ostream below the parent. That caused this very confusing
bug.

Best,
Philipp

Am 11.12.2014 um 12:16 schrieb Philipp Br?ll:
> Hello,
> 
> I'm developing an encryption plugin for dovecot and ran into a problem
> with the stacking of i/o-streams.
> 
> The encryption i/o-streams are working fine on any kind of mail the test
> suite is passing through them. But as soon as the zlib plugin is enabled
> the logs show an cache error:
> 
> failed: Cached message size larger than expected (214 > 206, box=INBOX,
> UID=1)
> 
> I've already double-checked the return values of ostream's sendv
and
> istream's read function. They seem correct (and equal).
> 
> If the order of the streams are changed (by changing the number in the
> lib-filename libxx_scrambler.so); meaning that the encryption is done
> before the compression (which isn't efficient) both streams are working
> correct without any errors.
> 
> Is there some way the zlib plugin changes the cached message size? Is
> there some behaviour of the zlib plugin that I'm missing? Any help
would
> be very welcome.
> 
> Best regards,
> Philipp
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20141212/0f0b0c52/attachment.sig>

Here is my ugly solution for this:

static int plugin_mail_save_begin(
  struct mail_save_context *context,
  struct istream *input
) {
  ...

  if (mbox->super.save_begin(context, input) < 0)
    return -1;

  output = scrambler_ostream_create(
    context->data.output->real_stream->parent,
    suser->public_key);			

  o_stream_unref(&context->data.output->real_stream->parent);

  context->data.output->real_stream->parent = output;

  return 0;
}

The solution is ugly, because it's only working if there is another
ostream (in my case the zlib ostream). It would be better to add the
scrambler ostream to the other side of the ostream chain, instead of
messing with the ostream's parent.

I've tried to re-order the plugin itself - which brings the ostreams in
order - but than the istream order is messed up.

What would be the right solution here? Maybe someone with deeper
insights into dovecot's architecture can help.

Kind regards,
Philipp

Am 12.12.2014 um 15:25 schrieb Philipp Br?ll:
> Well, I've found the bug. I've got confused with the
stream-stacking
> function pointers. The encryption istream was stacked on top of the
> parent and the ostream below the parent. That caused this very confusing
> bug.
> 
> Best,
> Philipp
> 
> Am 11.12.2014 um 12:16 schrieb Philipp Br?ll:
>> Hello,
>>
>> I'm developing an encryption plugin for dovecot and ran into a
problem
>> with the stacking of i/o-streams.
>>
>> The encryption i/o-streams are working fine on any kind of mail the
test
>> suite is passing through them. But as soon as the zlib plugin is
enabled
>> the logs show an cache error:
>>
>> failed: Cached message size larger than expected (214 > 206,
box=INBOX,
>> UID=1)
>>
>> I've already double-checked the return values of ostream's
sendv and
>> istream's read function. They seem correct (and equal).
>>
>> If the order of the streams are changed (by changing the number in the
>> lib-filename libxx_scrambler.so); meaning that the encryption is done
>> before the compression (which isn't efficient) both streams are
working
>> correct without any errors.
>>
>> Is there some way the zlib plugin changes the cached message size? Is
>> there some behaviour of the zlib plugin that I'm missing? Any help
would
>> be very welcome.
>>
>> Best regards,
>> Philipp
>>
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20141212/d0fd5fe8/attachment.sig>