dovecot - Apr 2015 - pop3 retr responses too large lines - Fails with current python's poplib

El vie, 10-04-2015 a las 10:21 +0900, Timo Sirainen
escribi?:
> On 09 Apr 2015, at 06:52, Guillermo M. Narvaja <guillermo.narvaja at
fierro.com.ar> wrote:
> > 
> > Since December 2014, clients using the Python poplib library (getmail
in
> > my case) started to limit the line length of RETR and other command
> > responses sent from the server to 2048 bytes:
> > 
> > See https://bugs.python.org/issue16041 (Included at least in
> > Python-2.7.9)
> > 
> > Dovecot doesn't have line split in the POP3 module, so Python
fails to
> > retrieve messages with long lines.
> > 
> > RFC 1939 says:
> > "
> >   Responses in the POP3 consist of a status indicator and a keyword
> >   possibly followed by additional information.  All responses are
> >   terminated by a CRLF pair.  Responses may be up to 512 characters
> >   long, including the terminating CRLF.
> > "
> > 
> > I think dovecot-pop3 should split it's output in lines of at least
512
> > characters. Or at least it should have a
"pop3_max_line_length" setting
> > (like "imap_max_line_length").
> 
> Um. This is talking about POP3 responses themselves - not about the
> actual email message body. Dovecot's POP3 reponses are short always I
> think. The message body is sent exactly as it was originally saved.
> Changing this could break things like PGP/SMIME signing and so on.
So there is no way to break the lines without breaking the message?

Please note that if you are right, I should post a bug on the Python
project.

You can see in this "traceforward" that Python's poplib will throw
an
error_proto exception if RETR returns lines longer than 2048 bytes:

https://hg.python.org/cpython/file/0db36098b908/Lib/poplib.py#l227
https://hg.python.org/cpython/file/0db36098b908/Lib/poplib.py#l165
https://hg.python.org/cpython/file/0db36098b908/Lib/poplib.py#l142
https://hg.python.org/cpython/file/0db36098b908/Lib/poplib.py#l370

        def _getline(self):
            line = ""
            renewline = re.compile(r'.*?\n')
            match = renewline.match(self.buffer)
            while not match:
                self._fillBuffer()
>>>              if len(self.buffer) > _MAXLINE:  # _MAXLINE=2048
>>>                  raise error_proto('line too long')
                match = renewline.match(self.buffer)
            line = match.group(0)
	    ...



-- 
Guillermo M. Narvaja
Lambda Sistemas S.R.L.
www.fierro-soft.com.ar - radiocut.fm
Tel: (5411) 3220-1520 (rotativas) y 4857-6662
Cel: (5411) 15-6783-4435
Email: guillermo.narvaja at fierro-soft.com.ar
MSN: guillermo_narvaja at hotmail.com
Skype: guillermonarvaja
Lavalleja 519 1er Piso - Ciudad de Buenos Aires - Argentina

On 10 Apr 2015, at 17:51, Guillermo M. Narvaja <guillermo.narvaja at
fierro.com.ar> wrote:
> 
> El vie, 10-04-2015 a las 10:21 +0900, Timo Sirainen escribi?:
>> On 09 Apr 2015, at 06:52, Guillermo M. Narvaja <guillermo.narvaja at
fierro.com.ar> wrote:
>>> 
>>> Since December 2014, clients using the Python poplib library
(getmail in
>>> my case) started to limit the line length of RETR and other command
>>> responses sent from the server to 2048 bytes:
>>> 
>>> See https://bugs.python.org/issue16041 (Included at least in
>>> Python-2.7.9)
>>> 
>>> Dovecot doesn't have line split in the POP3 module, so Python
fails to
>>> retrieve messages with long lines.
>>> 
>>> RFC 1939 says:
>>> "
>>>  Responses in the POP3 consist of a status indicator and a keyword
>>>  possibly followed by additional information.  All responses are
>>>  terminated by a CRLF pair.  Responses may be up to 512 characters
>>>  long, including the terminating CRLF.
>>> "
>>> 
>>> I think dovecot-pop3 should split it's output in lines of at
least 512
>>> characters. Or at least it should have a
"pop3_max_line_length" setting
>>> (like "imap_max_line_length").
>> 
>> Um. This is talking about POP3 responses themselves - not about the
>> actual email message body. Dovecot's POP3 reponses are short always
I
>> think. The message body is sent exactly as it was originally saved.
>> Changing this could break things like PGP/SMIME signing and so on.
> 
> So there is no way to break the lines without breaking the message?
Right.
> Please note that if you are right, I should post a bug on the Python
> project.
Yes. They are crazy if they think their behavior would work with any POP3
server. I think they have simply misunderstood the RFC.

El vie, 10-04-2015 a las 17:59 +0300, Timo Sirainen
escribi?:
> On 10 Apr 2015, at 17:51, Guillermo M. Narvaja <guillermo.narvaja at
fierro.com.ar> wrote:
> > 
> > El vie, 10-04-2015 a las 10:21 +0900, Timo Sirainen escribi?:
> >> On 09 Apr 2015, at 06:52, Guillermo M. Narvaja
<guillermo.narvaja at fierro.com.ar> wrote:
> >>> 
> >>> Since December 2014, clients using the Python poplib library
(getmail in
> >>> my case) started to limit the line length of RETR and other
command
> >>> responses sent from the server to 2048 bytes:
> >>> 
> >>> See https://bugs.python.org/issue16041 (Included at least in
> >>> Python-2.7.9)
> >>> 
> >>> Dovecot doesn't have line split in the POP3 module, so
Python fails to
> >>> retrieve messages with long lines.
> >>> 
> >>> RFC 1939 says:
> >>> "
> >>>  Responses in the POP3 consist of a status indicator and a
keyword
> >>>  possibly followed by additional information.  All responses
are
> >>>  terminated by a CRLF pair.  Responses may be up to 512
characters
> >>>  long, including the terminating CRLF.
> >>> "
> >>> 
> >>> I think dovecot-pop3 should split it's output in lines of
at least 512
> >>> characters. Or at least it should have a
"pop3_max_line_length" setting
> >>> (like "imap_max_line_length").
> >> 
> >> Um. This is talking about POP3 responses themselves - not about
the
> >> actual email message body. Dovecot's POP3 reponses are short
always I
> >> think. The message body is sent exactly as it was originally
saved.
> >> Changing this could break things like PGP/SMIME signing and so on.
> > 
> > So there is no way to break the lines without breaking the message?
> 
> Right.
> 
> > Please note that if you are right, I should post a bug on the Python
> > project.
> 
> Yes. They are crazy if they think their behavior would work with any POP3
server. I think they have simply misunderstood the RFC.
> 
Ok, I posted an issue on the Python's issue tracker:

http://bugs.python.org/issue23906

I expect this can be fixed in either of the sides.


-- 
Guillermo M. Narvaja
Lambda Sistemas S.R.L.
www.fierro-soft.com.ar - radiocut.fm
Tel: (5411) 3220-1520 (rotativas) y 4857-6662
Cel: (5411) 15-6783-4435
Email: guillermo.narvaja at fierro-soft.com.ar
MSN: guillermo_narvaja at hotmail.com
Skype: guillermonarvaja
Lavalleja 519 1er Piso - Ciudad de Buenos Aires - Argentina