Tmail talk - Sep 2009 - 6 tests fail using OS X 10.5's apple ruby

This doesn''t look anything like the contributing page:

   http://tmail.rubyforge.org/contributing/index.html

I get lots of extra output about making and destroying directories and  
6 failures.  Is my ruby version too old?

$ ruby -v
ruby 1.8.6 (2008-08-11 patchlevel 287) [universal-darwin9.0]
$ svnversion
261
$ rake test
(in /Users/drbrain/Work/svn/tmail/trunk)
Loaded suite test
Started
..F..........................................rm -rf tmp
mkdir tmp
rm -rf tmp
.rm -rf tmp
mkdir tmp
rm -rf tmp
.rm -rf tmp
mkdir tmp
rm -rf tmp
.rm -rf tmp
mkdir tmp
rm -rf tmp
.rm -rf tmp
mkdir tmp
rm -rf tmp
.rm -rf tmp
mkdir tmp
rm -rf tmp
.rm -rf tmp
mkdir tmp
rm -rf tmp
.rm -rf tmp
mkdir tmp
rm -rf tmp
.rm -rf tmp
mkdir tmp
rm -rf tmp
.rm -rf tmp
mkdir tmp
rm -rf tmp
.rm -rf tmp
mkdir tmp
rm -rf tmp
.rm -rf tmp
mkdir tmp
rm -rf tmp
.rm -rf tmp
mkdir tmp
rm -rf tmp
.rm -rf tmp
mkdir tmp
rm -rf tmp
.rm -rf tmp
mkdir tmp
rm -rf tmp
...rm -rf _mh
mkdir _mh
rm -rf _mh
.rm -rf _mh
mkdir _mh
rm -rf _mh
.rm -rf _mh
mkdir _mh
rm -rf _mh
.rm -rf _mh
mkdir _mh
rm -rf _mh
.rm -rf _mh
mkdir _mh
rm -rf _mh
.rm -rf _mh
mkdir _mh
rm -rf _mh
.rm -rf _mh
mkdir _mh
rm -rf _mh
.rm -rf _mh
mkdir _mh
rm -rf _mh
.rm -rf _mh
mkdir _mh
rm -rf _mh
.rm -rf _mh
mkdir _mh
rm -rf _mh
.rm -rf _mh
mkdir _mh
rm -rf _mh
.rm -rf _mh
mkdir _mh
rm -rf _mh
.rm -rf _mh
mkdir _mh
rm -rf _mh
.rm -rf tmp
mkdir tmp
rm -rf tmp
......rm -rf tmp
mkdir tmp
rm -rf tmp
.....................................E 
.E 
........F 
...E 
...F 
........................................................................
Finished in 0.472688 seconds.

   1) Failure:
test_ATTRS(ContentDispositionHeaderTester)
     [./test/test_header.rb:839:in `_test_raw_iso2022jp''
      ./test/test_header.rb:771:in `test_ATTRS'']:
<"attachment"> expected but was
<nil>.

   2) Error:
test_at_char_in_address(TestAddress):
TMail::SyntaxError: parse error on token "<"
     lib/tmail/parser.y:379:in `on_error''
     ./test/../lib/tmail/port.rb:339:in `_racc_yyparse_c''
     lib/tmail/parser.y:375:in `scan''
     lib/tmail/parser.y:375:in `parse_in''
     /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ 
ruby/1.8/racc/parser.rb:152:in `_racc_yyparse_c''
     /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ 
ruby/1.8/racc/parser.rb:152:in `__send__''
     /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ 
ruby/1.8/racc/parser.rb:152:in `yyparse''
     lib/tmail/parser.y:366:in `parse''
     lib/tmail/parser.y:344:in `parse''
     ./test/../lib/tmail/address.rb:84:in `parse''
     ./test/test_address.rb:90:in `validate_case__address''
     ./test/test_address.rb:1137:in `test_at_char_in_address''

   3) Error:
test_dot_in_address(TestAddress):
TMail::SyntaxError: parse error on token "<"
     lib/tmail/parser.y:379:in `on_error''
     lib/tmail/parser.y:379:in `_racc_yyparse_c''
     lib/tmail/parser.y:375:in `scan''
     lib/tmail/parser.y:375:in `parse_in''
     /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ 
ruby/1.8/racc/parser.rb:152:in `_racc_yyparse_c''
     /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ 
ruby/1.8/racc/parser.rb:152:in `__send__''
     /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ 
ruby/1.8/racc/parser.rb:152:in `yyparse''
     lib/tmail/parser.y:366:in `parse''
     lib/tmail/parser.y:344:in `parse''
     ./test/../lib/tmail/address.rb:84:in `parse''
     ./test/test_address.rb:90:in `validate_case__address''
     ./test/test_address.rb:1093:in `test_dot_in_address''

   4) Failure:
test_parse__mailtools(TestAddress)
     [./test/test_address.rb:91:in `validate_case__address''
      ./test/test_address.rb:691:in `test_parse__mailtools'']:
"(foo at bar.com (foobar), ned at foo.com (nedfoo) )  
<kevin at goess.org>" (phrase)
.
<"(foo at bar.com (foobar), ned at foo.com (nedfoo) )"> expected
but was
<nil>.

   5) Error:
test_parse__rawjp(TestAddress):
TMail::SyntaxError: parse error on token error
     lib/tmail/parser.y:379:in `on_error''
     lib/tmail/parser.y:379:in `_racc_yyparse_c''
     lib/tmail/parser.y:375:in `scan''
     lib/tmail/parser.y:375:in `parse_in''
     /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ 
ruby/1.8/racc/parser.rb:152:in `_racc_yyparse_c''
     /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ 
ruby/1.8/racc/parser.rb:152:in `__send__''
     /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ 
ruby/1.8/racc/parser.rb:152:in `yyparse''
     lib/tmail/parser.y:366:in `parse''
     lib/tmail/parser.y:344:in `parse''
     ./test/../lib/tmail/address.rb:84:in `parse''
     ./test/test_address.rb:90:in `validate_case__address''
     ./test/test_address.rb:243:in `_test_parse__euc''
     ./test/test_address.rb:233:in `test_parse__rawjp''

   6) Failure:
test_trailing_dot_in_name(TestAddress) [./test/test_address.rb:1211]:
<"Sandy M. <noreply at rubyforge.org>"> expected but was
<nil>.

210 tests, 4004 assertions, 3 failures, 3 errors

On Sep 30, 2009, at 12:41, Eric Hodel wrote:
>  6) Failure:
> test_trailing_dot_in_name(TestAddress) [./test/test_address.rb:1211]:
> <"Sandy M. <noreply at rubyforge.org>"> expected but
was
> <nil>.
I started looking into this last test first and determined that there  
are two scanners, one written in ruby and one in C.  (Maybe one of  
them is out of date and I''m using the out-of-date one?)

Unfortunately it looks like the Rakefile is broken and `rake test`  
doesn''t build extensions before starting.  From setup.rb:

   task :test do exec_test end

rake -T indicates that `rake setup` will build extensions, but it  
doesn''t:

$ rake -T setup
(in /Users/drbrain/Work/svn/tmail/trunk)
rake setup  # Compiles ruby extentions
$ rake setup
(in /Users/drbrain/Work/svn/tmail/trunk)
make: *** No targets specified and no makefile found.  Stop.
rake aborted!
system("make") failed

(See full trace by running task with --trace)
$

I think, setup.rb should be replaced with Hoe + rake-compiler which  
are state-of-the-art.

Not only will it remove ~70KB of code:

$ wc work/script/* setup.rb
       26      44     374 work/script/make
       39      95     851 work/script/rdoc
     1616    3582   36900 work/script/setup
       30      74     635 work/script/test
     1482    3751   37073 setup.rb
     3193    7546   75833 total

(Nokogiri''s Rakefile is about 6KB and does what all the above code
does)

But I can also give you a toolset that will actually work like your  
contribution page says it should.

On Thu, Oct 1, 2009 at 12:11 PM, Eric Hodel <drbrain at segment7.net>
wrote:

eric, sorry for the late reply.  The state of the TMail repo has gone
into a bit of disrepair as I have been focusing on my new mail gem to
hopefully replace out tmail soonish. But I still want to keep tmail in
maintain, just not new features as it is getting unwieldy
> On Sep 30, 2009, at 12:41, Eric Hodel wrote:
> I started looking into this last test first and determined that there are
> two scanners, one written in ruby and one in C. ?(Maybe one of them is out
> of date and I''m using the out-of-date one?)
We were thinking about ditching the C scanner in TMail.. lot of work
for not a lot of gain.
> Unfortunately it looks like the Rakefile is broken and `rake test`
doesn''t
> build extensions before starting. ?From setup.rb:
Yes... the tool set on TMail is in a bit of a shambles.  Someone tried
to put a whole new toolset in, and it has never really worked 100%,
but they didn''t finish, so... guess that''s where it is at.
> I think, setup.rb should be replaced with Hoe + rake-compiler which are
> state-of-the-art.
> But I can also give you a toolset that will actually work like your
> contribution page says it should.
Awesome!  Go for it.  If you can fork the github repo
(http://github.com/mikel/tmail/) and patch to your hearts content,
then we''ll release a new gem and then backupload it to rubyforge.
It''s been a long while waiting.

Regards

Mikel

-- 
http://lindsaar.net/
Rails, RSpec and Life blog....

Hi Mikel,

Sorry to hear that maintenance of TMail is on second place now.  
However, as a developer of email software I am using TMail, but I''m  
not quite happy with it. Due to the legacy code it is difficult to  
maintain and extend. RubyMail is a bit cleaner, but it has a low of  
flaws. For specific purposes, such as parsing received headers, I  
have written my own parser code.

If you plan to develop a completely new gem, think twice about what  
you are getting into. It is a daunting task to fully understand the  
RFCs, and also be able to parse the mails that are not fully  
compliant with RFCs. Did you also think about re-using a proven  
existing C/C++ library using FFI, such as VMime or GMime? The  
developer of the Nokogiri HTML/XML parser gem choose to use libxml2  
under the hood, and his gem is a big hit!

Maarten O.

On 2-okt-2009, at 1:45, Mikel Lindsaar wrote:
> On Thu, Oct 1, 2009 at 12:11 PM, Eric Hodel <drbrain at segment7.net>
> wrote:
>
> eric, sorry for the late reply.  The state of the TMail repo has gone
> into a bit of disrepair as I have been focusing on my new mail gem to
> hopefully replace out tmail soonish. But I still want to keep tmail in
> maintain, just not new features as it is getting unwieldy
>
>> On Sep 30, 2009, at 12:41, Eric Hodel wrote:
>> I started looking into this last test first and determined that  
>> there are
>> two scanners, one written in ruby and one in C.  (Maybe one of  
>> them is out
>> of date and I''m using the out-of-date one?)
>
> We were thinking about ditching the C scanner in TMail.. lot of work
> for not a lot of gain.
>
>> Unfortunately it looks like the Rakefile is broken and `rake test`  
>> doesn''t
>> build extensions before starting.  From setup.rb:
>
> Yes... the tool set on TMail is in a bit of a shambles.  Someone tried
> to put a whole new toolset in, and it has never really worked 100%,
> but they didn''t finish, so... guess that''s where it is
at.
>
>> I think, setup.rb should be replaced with Hoe + rake-compiler  
>> which are
>> state-of-the-art.
>> But I can also give you a toolset that will actually work like your
>> contribution page says it should.
>
> Awesome!  Go for it.  If you can fork the github repo
> (http://github.com/mikel/tmail/) and patch to your hearts content,
> then we''ll release a new gem and then backupload it to rubyforge.
> It''s been a long while waiting.
>
> Regards
>
> Mikel
>
> -- 
> http://lindsaar.net/
> Rails, RSpec and Life blog....
> _______________________________________________
> Tmail-talk mailing list
> Tmail-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/tmail-talk

On Fri, Oct 2, 2009 at 5:47 PM, Maarten Oelering
<maarten at solovirtuoso.com> wrote:
> Sorry to hear that maintenance of TMail is on second place now. However, as
> a developer of email software I am using TMail, but I''m not quite
happy with
> it. Due to the legacy code it is difficult to maintain and extend. RubyMail
> is a bit cleaner, but it has a low of flaws. For specific purposes, such as
> parsing received headers, I have written my own parser code.
Dear Maarten,

Understood on your concerns, the new mail library has been designed
with the viewpoint of handling at least everything TMail can and more.

I have designed it from the ground up with the RFCs.  It is very
resilient to error messages, in fact, handles every email in the
current TMail test suite without crashing.

You can take a look at it and run the specs for it by grabbing
http://github.com/mikel/mail

In fact... I would really like it if you could run some of your more
problematic emails against it and tell me what works and what breaks.

Mail works for all RFC 2822 and almost everything in 822 (there are
three "out there" obsolete test cases that I don''t handle
yet).  Mail
works in 1.8.6, 1.8.7 and 1.9 equally well and runs multiruby -S rake
spec on all three versions without an error

The only thing I have remaining is auto encode / decode of quoted pair
and base64 header fields.  Otherwise, it is pretty much ready for a
gem release.

Plus, it is nice to use, mail can do this right now:

Mail.defaults do
  smtp ''127.0.0.1''
end

Mail.deliver do
  to ''nicolas at test.lindsaar.net.au''
  from ''Mikel Lindsaar <mikel at test.lindsaar.net.au>''
  subject ''First multipart email sent with Mail''
  text_part do
    body ''Here is the attachment you wanted''
  end
  html_part do
    content_type ''text/html; charset=UTF-8''
    body ''<h1>Funky Title</h1><p>Here it
is</p>''
  end
  add_file :filename => ''/path/to/myfile.pdf''
end

That gets you a multipart alternate email, html and text with a pdf
file attached with the correct mime types specified.

Oh... and it sends it as well.

Anyway, please check it out.

if you would like to help maintain TMail, please let me know, would
appreciate the help.

Regards

Mikel

Hi Mikel,

Thanks for sharing your new development. I did not realize you were  
that far. It all sounds very cool.

I noticed that you are also using treetop. I have been testing with  
treetop to parse received headers (RFC 2821 section 4.4).

Regards,

Maarten

On 2-okt-2009, at 10:41, Mikel Lindsaar wrote:
> I have designed it from the ground up with the RFCs.  It is very
> resilient to error messages, in fact, handles every email in the
> current TMail test suite without crashing.
>
> You can take a look at it and run the specs for it by grabbing
> http://github.com/mikel/mail
>
> In fact... I would really like it if you could run some of your more
> problematic emails against it and tell me what works and what breaks.
>
> Mail works for all RFC 2822 and almost everything in 822 (there are
> three "out there" obsolete test cases that I don''t
handle yet).  Mail
> works in 1.8.6, 1.8.7 and 1.9 equally well and runs multiruby -S rake
> spec on all three versions without an error
>
> The only thing I have remaining is auto encode / decode of quoted pair
> and base64 header fields.  Otherwise, it is pretty much ready for a
> gem release.
>
> Plus, it is nice to use, mail can do this right now:
>
> Mail.defaults do
>   smtp ''127.0.0.1''
> end
>
> Mail.deliver do
>   to ''nicolas at test.lindsaar.net.au''
>   from ''Mikel Lindsaar <mikel at
test.lindsaar.net.au>''
>   subject ''First multipart email sent with Mail''
>   text_part do
>     body ''Here is the attachment you wanted''
>   end
>   html_part do
>     content_type ''text/html; charset=UTF-8''
>     body ''<h1>Funky Title</h1><p>Here it
is</p>''
>   end
>   add_file :filename => ''/path/to/myfile.pdf''
> end
>
> That gets you a multipart alternate email, html and text with a pdf
> file attached with the correct mime types specified.
>
> Oh... and it sends it as well.
>
> Anyway, please check it out.
>
> if you would like to help maintain TMail, please let me know, would
> appreciate the help.
>
> Regards
>
> Mikel
> _______________________________________________
> Tmail-talk mailing list
> Tmail-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/tmail-talk

On Fri, Oct 2, 2009 at 7:45 PM, Maarten Oelering
<maarten at solovirtuoso.com> wrote:
> Thanks for sharing your new development. I did not realize you were that
> far. It all sounds very cool.
Yeah... been doing a little work on it on the side :)
> I noticed that you are also using treetop. I have been testing with treetop
> to parse received headers (RFC 2821 section 4.4).
Cool.  I have basically a complete rfc2822 parse tree done AFAICT

Maybe you can help out.

Mikel

On Oct 2, 2009, at 01:41, Mikel Lindsaar wrote:
> On Fri, Oct 2, 2009 at 5:47 PM, Maarten Oelering
> <maarten at solovirtuoso.com> wrote:
>> Sorry to hear that maintenance of TMail is on second place now.  
>> However, as
>> a developer of email software I am using TMail, but I''m not
quite
>> happy with
>> it. Due to the legacy code it is difficult to maintain and extend.  
>> RubyMail
>> is a bit cleaner, but it has a low of flaws. For specific purposes,  
>> such as
>> parsing received headers, I have written my own parser code.
>
> Dear Maarten,
>
> Understood on your concerns, the new mail library has been designed
> with the viewpoint of handling at least everything TMail can and more.
>
> I have designed it from the ground up with the RFCs.  It is very
> resilient to error messages, in fact, handles every email in the
> current TMail test suite without crashing.
>
> You can take a look at it and run the specs for it by grabbing
> http://github.com/mikel/mail
>
> In fact... I would really like it if you could run some of your more
> problematic emails against it and tell me what works and what breaks.
>
> Mail works for all RFC 2822 and almost everything in 822 (there are
> three "out there" obsolete test cases that I don''t
handle yet).  Mail
> works in 1.8.6, 1.8.7 and 1.9 equally well and runs multiruby -S rake
> spec on all three versions without an error
>
> The only thing I have remaining is auto encode / decode of quoted pair
> and base64 header fields.  Otherwise, it is pretty much ready for a
> gem release.
>
> Plus, it is nice to use, mail can do this right now:
>
> Mail.defaults do
>  smtp ''127.0.0.1''
> end
>
> Mail.deliver do
>  to ''nicolas at test.lindsaar.net.au''
>  from ''Mikel Lindsaar <mikel at
test.lindsaar.net.au>''
>  subject ''First multipart email sent with Mail''
>  text_part do
>    body ''Here is the attachment you wanted''
>  end
>  html_part do
>    content_type ''text/html; charset=UTF-8''
>    body ''<h1>Funky Title</h1><p>Here it
is</p>''
>  end
>  add_file :filename => ''/path/to/myfile.pdf''
> end
>
> That gets you a multipart alternate email, html and text with a pdf
> file attached with the correct mime types specified.
>
> Oh... and it sends it as well.
This is gorgeous.
> Anyway, please check it out.
>
> if you would like to help maintain TMail, please let me know, would
> appreciate the help.
If you''ve got this pile of total awesome waiting for release,
I''d
rather jump on it.

I''ve been thinking about writing a replacement for ActionMailer since  
it seems largely unmaintaned.  Basing it on top of your library would  
make it only require the templating portion of ActionMailer.  (And  
maybe support for ar_sendmail.)

> If you''ve got this pile of total awesome waiting for release,
I''d rather
> jump on it.
>
> I''ve been thinking about writing a replacement for ActionMailer
since it
> seems largely unmaintaned. ?Basing it on top of your library would make it
> only require the templating portion of ActionMailer. ?(And maybe support
for
> ar_sendmail.)
Yeah... I want to swap out tmail out of actionmailer and replace it
with Mail.  it would just abstract SO much out of actionmailer...
actionmailer basically tries to "rubyize" tmail if you like.

Anyway, probably over the next couple of weeks I''ll be wrapping up
what I need to do for a 0.1 release of mail.  There are some parts of
it that need some cleanup and wrap up...

The last feature waiting to be handled, as I said, is multilingual
header field handling.  It''s not a lot of work, maybe a couple to
three hours of programming... but time time time... I''ll see what I
can wrap up tonight.

Mikel