dovecot - Apr 2015 - pop3 retr responses too large lines - Fails with current python's poplib

Since December 2014, clients using the Python poplib library (getmail in
my case) started to limit the line length of RETR and other command
responses sent from the server to 2048 bytes:

See https://bugs.python.org/issue16041 (Included at least in
Python-2.7.9)

Dovecot doesn't have line split in the POP3 module, so Python fails to
retrieve messages with long lines.

RFC 1939 says:
"
   Responses in the POP3 consist of a status indicator and a keyword
   possibly followed by additional information.  All responses are
   terminated by a CRLF pair.  Responses may be up to 512 characters
   long, including the terminating CRLF.
"

I think dovecot-pop3 should split it's output in lines of at least 512
characters. Or at least it should have a "pop3_max_line_length"
setting
(like "imap_max_line_length").

Server side:
# dovecot --version
2.2.13

Client side:
# python --version
Python 2.7.9
# getmail --version
getmail 4.47.0


Temporary workaround (on client-side):

--- poplib.py.orig      2015-04-08 18:46:48.280879211 -0300
+++ poplib.py   2015-04-08 18:26:01.288556028 -0300
@@ -36,7 +36,7 @@
 # reading arbitrary length lines. RFC 1939 limits POP3 line length to
 # 512 characters, including CRLF. We have selected 2048 just to be on
 # the safe side.
-_MAXLINE = 2048
+_MAXLINE = 2048*100
 
 
 class POP3:


Thanks in advance.


-- 
Guillermo M. Narvaja
Lambda Sistemas S.R.L.
www.fierro-soft.com.ar - radiocut.fm
Tel: (5411) 3220-1520 (rotativas) y 4857-6662
Cel: (5411) 15-6783-4435
Email: guillermo.narvaja at fierro-soft.com.ar
MSN: guillermo_narvaja at hotmail.com
Skype: guillermonarvaja
Lavalleja 519 1er Piso - Ciudad de Buenos Aires - Argentina

On 09 Apr 2015, at 06:52, Guillermo M. Narvaja <guillermo.narvaja at
fierro.com.ar> wrote:
> 
> Since December 2014, clients using the Python poplib library (getmail in
> my case) started to limit the line length of RETR and other command
> responses sent from the server to 2048 bytes:
> 
> See https://bugs.python.org/issue16041 (Included at least in
> Python-2.7.9)
> 
> Dovecot doesn't have line split in the POP3 module, so Python fails to
> retrieve messages with long lines.
> 
> RFC 1939 says:
> "
>   Responses in the POP3 consist of a status indicator and a keyword
>   possibly followed by additional information.  All responses are
>   terminated by a CRLF pair.  Responses may be up to 512 characters
>   long, including the terminating CRLF.
> "
> 
> I think dovecot-pop3 should split it's output in lines of at least 512
> characters. Or at least it should have a "pop3_max_line_length"
setting
> (like "imap_max_line_length").
Um. This is talking about POP3 responses themselves - not about the actual email
message body. Dovecot's POP3 reponses are short always I think. The message
body is sent exactly as it was originally saved. Changing this could break
things like PGP/SMIME signing and so on.

El vie, 10-04-2015 a las 10:21 +0900, Timo Sirainen
escribi?:
> On 09 Apr 2015, at 06:52, Guillermo M. Narvaja <guillermo.narvaja at
fierro.com.ar> wrote:
> > 
> > Since December 2014, clients using the Python poplib library (getmail
in
> > my case) started to limit the line length of RETR and other command
> > responses sent from the server to 2048 bytes:
> > 
> > See https://bugs.python.org/issue16041 (Included at least in
> > Python-2.7.9)
> > 
> > Dovecot doesn't have line split in the POP3 module, so Python
fails to
> > retrieve messages with long lines.
> > 
> > RFC 1939 says:
> > "
> >   Responses in the POP3 consist of a status indicator and a keyword
> >   possibly followed by additional information.  All responses are
> >   terminated by a CRLF pair.  Responses may be up to 512 characters
> >   long, including the terminating CRLF.
> > "
> > 
> > I think dovecot-pop3 should split it's output in lines of at least
512
> > characters. Or at least it should have a
"pop3_max_line_length" setting
> > (like "imap_max_line_length").
> 
> Um. This is talking about POP3 responses themselves - not about the
> actual email message body. Dovecot's POP3 reponses are short always I
> think. The message body is sent exactly as it was originally saved.
> Changing this could break things like PGP/SMIME signing and so on.
So there is no way to break the lines without breaking the message?

Please note that if you are right, I should post a bug on the Python
project.

You can see in this "traceforward" that Python's poplib will throw
an
error_proto exception if RETR returns lines longer than 2048 bytes:

https://hg.python.org/cpython/file/0db36098b908/Lib/poplib.py#l227
https://hg.python.org/cpython/file/0db36098b908/Lib/poplib.py#l165
https://hg.python.org/cpython/file/0db36098b908/Lib/poplib.py#l142
https://hg.python.org/cpython/file/0db36098b908/Lib/poplib.py#l370

        def _getline(self):
            line = ""
            renewline = re.compile(r'.*?\n')
            match = renewline.match(self.buffer)
            while not match:
                self._fillBuffer()
>>>              if len(self.buffer) > _MAXLINE:  # _MAXLINE=2048
>>>                  raise error_proto('line too long')
                match = renewline.match(self.buffer)
            line = match.group(0)
	    ...



-- 
Guillermo M. Narvaja
Lambda Sistemas S.R.L.
www.fierro-soft.com.ar - radiocut.fm
Tel: (5411) 3220-1520 (rotativas) y 4857-6662
Cel: (5411) 15-6783-4435
Email: guillermo.narvaja at fierro-soft.com.ar
MSN: guillermo_narvaja at hotmail.com
Skype: guillermonarvaja
Lavalleja 519 1er Piso - Ciudad de Buenos Aires - Argentina