sup talk - Sep 2007 - problems with rubymail on an initial sup-add/sync

Hi,

I''ve just installed 0.1 of sup with the current stable version of ruby
via gems.

i''ve just tried adding some of my mbox''s and this seems to
happen...


Scanning mbox:///u1/trhpc/jtang/Mail/system.rancid...
Scanned 0, added 0, updated 0 messages from
mbox:///u1/trhpc/jtang/Mail/system.rancid.
Scanning sup://drafts...
Scanned 0, added 0, updated 0 messages from sup://drafts.
Scanning mbox:///u1/trhpc/jtang/Mail/system.RequestTracker...
## 2300 (61.25%) read; 0:01:00 elapsed; 0:00:37 remaining
[Wed Sep 19 13:32:59 +0100 2007] problem getting messages from
mbox:///u1/trhpc/jtang/Mail/system.RequestTracker: For some bizarre reason,
RubyMail was unable to parse this message.
[Wed Sep 19 13:32:59 +0100 2007] problem getting messages from
mbox:///u1/trhpc/jtang/Mail/system.RequestTracker: For some bizarre reason,
RubyMail was unable to parse this message.

is there anyway of turning on more debugging to see whats causing the
problem...?

i tried using -v on the commandline as well but it doesnt seem to say
anything...


Thanks,
Jimmy

-- 
Jimmy Tang
Trinity Centre for High Performance Computing,
Lloyd Building, Trinity College Dublin, Dublin 2, Ireland.
http://www.tchpc.tcd.ie/ | http://www.tchpc.tcd.ie/~jtang

Excerpts from Jimmy Tang''s message of Wed Sep 19 05:36:16 -0700
2007:
> [Wed Sep 19 13:32:59 +0100 2007] problem getting messages from
mbox:///u1/trhpc/jtang/Mail/system.RequestTracker: For some bizarre reason,
RubyMail was unable to parse this message.
Interesting. I''ve never seen that triggered, to date. Congratulations!
Basically this means I asked RubyMail to parse the message, and RubyMail
gave up. 
> is there anyway of turning on more debugging to see whats causing the
> problem...?
Unfortunately no. The best I can offer is the following patch against
0.1, which will print out the message ids. You can then grep the mbox
for those and see if there''s anything weird about the messages.
I''m
guessing the MIME is screwed up in such a way that RubyMail chokes.

I''m interested in figuring out why this happens.

--- cut here ---
--- lib/sup/message.rb  (revision 503)
+++ lib/sup/message.rb  (working copy)
@@ -320,7 +320,7 @@
         $1
       end
 
-    m.body && body = m.decode or raise MessageFormatError, "For
some bizarre reason, RubyMail was unable to parse this message."
+    m.body && body = m.decode or raise MessageFormatError, "For
some bizarre reason, RubyMail was unable to parse this message
(#{m.header[''message-id'']})."
 
     if charset
       begin
--- cut here ---

-- 
William <wmorgan-sup at masanjin.net>

Hi again

replying to my own mail, i eventually figured out why the previous problem was
failing (corrupt mbox''s)
but now, this is happening after starting up sup for about 5mins as it scans my
mboxes

23:32:23 jtang at vual ~ $ sup
/u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/buffer.rb:32:
warning: already initialized constant KEY_ENTER
/u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/buffer.rb:33:
warning: already initialized constant KEY_CANCEL
[Wed Sep 19 23:32:25 +0100 2007] locking /u1/trhpc/jtang/.sup/lock...
[Wed Sep 19 23:32:25 +0100 2007] loading index...
[Wed Sep 19 23:32:25 +0100 2007] loaded index of 65133 messages
[Wed Sep 19 23:32:25 +0100 2007] starting curses
[Wed Sep 19 23:41:20 +0100 2007] stopped cursing
[Wed Sep 19 23:41:20 +0100 2007] oh crap, an exception
[Wed Sep 19 23:41:20 +0100 2007] unlocking /u1/trhpc/jtang/.sup/lock...
----------------------------------------------------------------
I''m very sorry, but it seems that an error occurred in Sup.
Please accept my sincere apologies. If you don''t mind, please
send the backtrace below and a brief report of the circumstances
to sup-talk at rubyforge dot orgs so that I might address this
problem. Thank you!

Sincerely,
William
----------------------------------------------------------------

The problem was: ''just added message <200610312309.38117.rj200@  but
couldn''t find it in a search'' (error type RuntimeError)
A backtrace follows:
/u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/index.rb:187:in
`sync_message'': just added message <200610312309.38117.rj200@  but
couldn''t find it in a search (RuntimeError)
        from
/u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/util.rb:401:in
`send''
        from
/u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/util.rb:401:in
`method_missing''
        from
/u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/poll.rb:122:in
`add_messages_from''
        from
/u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/source.rb:93:in
`each''
        from
/u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/util.rb:437:in
`send''
        from
/u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/util.rb:437:in
`__pass''
        from
/u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/util.rb:426:in
`method_missing''
        from
/u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/poll.rb:101:in
`add_messages_from''
         ... 26 levels...
        from
/u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/lib/sup/modes/thread-index-mode.rb:394:in
`load_threads''
        from
/u1/trhpc/jtang/local/ruby/lib/ruby/gems/1.8/gems/sup-0.1/bin/sup:173
        from /u1/trhpc/jtang/local/ruby/bin/sup:16:in `load''
        from /u1/trhpc/jtang/local/ruby/bin/sup:16


Thanks,
Jimmy

Excerpts from Jimmy Tang''s message of Wed Sep 19 13:36:16 +0100
2007:
> Hi,
> 
> I''ve just installed 0.1 of sup with the current stable version of
ruby via gems.
> 
> i''ve just tried adding some of my mbox''s and this seems
to happen...
> 
> 
> Scanning mbox:///u1/trhpc/jtang/Mail/system.rancid...
> Scanned 0, added 0, updated 0 messages from
> mbox:///u1/trhpc/jtang/Mail/system.rancid.
> Scanning sup://drafts...
> Scanned 0, added 0, updated 0 messages from sup://drafts.
> Scanning mbox:///u1/trhpc/jtang/Mail/system.RequestTracker...
> ## 2300 (61.25%) read; 0:01:00 elapsed; 0:00:37 remaining
> [Wed Sep 19 13:32:59 +0100 2007] problem getting messages from
> mbox:///u1/trhpc/jtang/Mail/system.RequestTracker: For some bizarre reason,
> RubyMail was unable to parse this message.
> [Wed Sep 19 13:32:59 +0100 2007] problem getting messages from
> mbox:///u1/trhpc/jtang/Mail/system.RequestTracker: For some bizarre reason,
> RubyMail was unable to parse this message.
> 
> is there anyway of turning on more debugging to see whats causing the
> problem...?
> 
> i tried using -v on the commandline as well but it doesnt seem to say
> anything...
> 
> 
> Thanks,
> Jimmy
> 
-- 
Jimmy Tang
Trinity Centre for High Performance Computing,
Lloyd Building, Trinity College Dublin, Dublin 2, Ireland.
http://www.tchpc.tcd.ie/ | http://www.tchpc.tcd.ie/~jtang

Hi William,

Excerpts from William Morgan''s message of Wed Sep 19 15:41:12 +0100
2007:
> Excerpts from Jimmy Tang''s message of Wed Sep 19 05:36:16 -0700
2007:
> > [Wed Sep 19 13:32:59 +0100 2007] problem getting messages from
mbox:///u1/trhpc/jtang/Mail/system.RequestTracker: For some bizarre reason,
RubyMail was unable to parse this message.
> 
> Interesting. I''ve never seen that triggered, to date.
Congratulations!
> Basically this means I asked RubyMail to parse the message, and RubyMail
> gave up. 
> 
not too sure whats wrong with that mail box, i eventually found the
offending mails and *cough* just deleted them *cough* they are just old
archives that I have that i wanted indexed.
> > is there anyway of turning on more debugging to see whats causing the
> > problem...?
> 
> Unfortunately no. The best I can offer is the following patch against
> 0.1, which will print out the message ids. You can then grep the mbox
> for those and see if there''s anything weird about the messages.
I''m
> guessing the MIME is screwed up in such a way that RubyMail chokes.
> 
> I''m interested in figuring out why this happens.
Thanks for the small patch, if i come across it again, i will save the
email and send it on to you to figure out whats broken. In hindsight I
should have kept some samples before deleting things.

on a slightly off topic on sup, i''ve eventually got sup working
relatively well with my archives of mail. and i quite like it :), I have
one question about the overall list of the inbox-box mode. Is there any
way of say doing an "all mails" mode? or is there a magic query that
will
let me list *all* of my mails much like how gmail has the "all mails"
list where it shows "all mails except spam and trash" ?


Thanks,
Jimmy.

-- 
Jimmy Tang
Trinity Centre for High Performance Computing,
Lloyd Building, Trinity College Dublin, Dublin 2, Ireland.
http://www.tchpc.tcd.ie/ | http://www.tchpc.tcd.ie/~jtang

Excerpts from Jimmy Tang''s message of Thu Sep 20 07:31:09 -0700
2007:
> not too sure whats wrong with that mail box, i eventually found the
> offending mails and *cough* just deleted them *cough* they are just
> old archives that I have that i wanted indexed.
That last batch of errors looks like more corrupted email, but I could
be wrong. If it works now, it works.
> Is there any way of say doing an "all mails" mode? or is there a
magic
> query that will let me list *all* of my mails much like how gmail has
> the "all mails" list where it shows "all mails except spam
and trash"
> ?
Yes, search for "*".

-- 
William <wmorgan-sup at masanjin.net>