sup talk - Aug 2009 - [PATCH] add Message#load_from_index! shortcut

Message#parse_header is slow, taking up to 2/3 of the time spent loading
threads in thread-index-mode. This patch adds a shortcut method that the index
can use to efficiently initialize a message.
---
 lib/sup/message.rb      |   25 +++++++++++++++++++++++++
 lib/sup/xapian_index.rb |   30 +++++++++++-------------------
 2 files changed, 36 insertions(+), 19 deletions(-)

diff --git a/lib/sup/message.rb b/lib/sup/message.rb
index 965c10e..56e66de 100644
--- a/lib/sup/message.rb
+++ b/lib/sup/message.rb
@@ -127,6 +127,31 @@ class Message
     @list_unsubscribe = header["list-unsubscribe"]
   end
 
+  ## Expected index entry format:
+  ## :message_id, :subject => String
+  ## :date => Time
+  ## :refs, :replytos => Array of String
+  ## :from => Person
+  ## :to, :cc, :bcc => Array of Person
+  def load_from_index! entry
+    @id = entry[:message_id]
+    @from = entry[:from]
+    @date = entry[:date]
+    @subj = entry[:subject]
+    @to = entry[:to]
+    @cc = entry[:cc]
+    @bcc = entry[:bcc]
+    @refs = (@refs + entry[:refs]).uniq
+    @replytos = entry[:replytos]
+
+    @replyto = nil
+    @list_address = nil
+    @recipient_email = nil
+    @source_marked_read = false
+    @list_subscribe = nil
+    @list_unsubscribe = nil
+  end
+
   def add_ref ref
     @refs << ref
     @dirty = true
diff --git a/lib/sup/xapian_index.rb b/lib/sup/xapian_index.rb
index 85f6ef0..c260728 100644
--- a/lib/sup/xapian_index.rb
+++ b/lib/sup/xapian_index.rb
@@ -71,25 +71,17 @@ class XapianIndex < BaseIndex
     source = SourceManager[entry[:source_id]]
     raise "invalid source #{entry[:source_id]}" unless source
 
-    mk_addrs = lambda { |l| l.map { |e,n| "#{n} <#{e}>" } *
'', '' }
-    mk_refs = lambda { |l| l.map { |r| "<#{r}>" } * ''
'' }
-    fake_header = {
-      ''message-id'' => entry[:message_id],
-      ''date'' => Time.at(entry[:date]),
-      ''subject'' => entry[:subject],
-      ''from'' => mk_addrs[[entry[:from]]],
-      ''to'' => mk_addrs[entry[:to]],
-      ''cc'' => mk_addrs[entry[:cc]],
-      ''bcc'' => mk_addrs[entry[:bcc]],
-      ''reply-tos'' => mk_refs[entry[:replytos]],
-      ''references'' => mk_refs[entry[:refs]],
-     }
-
-      m = Message.new :source => source, :source_info =>
entry[:source_info],
-                  :labels => entry[:labels],
-                  :snippet => entry[:snippet]
-      m.parse_header fake_header
-      m
+    m = Message.new :source => source, :source_info =>
entry[:source_info],
+                    :labels => entry[:labels], :snippet =>
entry[:snippet]
+
+    mk_person = lambda { |x| Person.new(*x.reverse!) }
+    entry[:from] = mk_person[entry[:from]]
+    entry[:to].map!(&mk_person)
+    entry[:cc].map!(&mk_person)
+    entry[:bcc].map!(&mk_person)
+
+    m.load_from_index! entry
+    m
   end
 
   def add_message m; sync_message m end
-- 
1.6.4

Another 10% query performance boost.
---
 lib/sup/xapian_index.rb |   10 +++++++++-
 1 files changed, 9 insertions(+), 1 deletions(-)

diff --git a/lib/sup/xapian_index.rb b/lib/sup/xapian_index.rb
index c260728..3a23951 100644
--- a/lib/sup/xapian_index.rb
+++ b/lib/sup/xapian_index.rb
@@ -74,7 +74,7 @@ class XapianIndex < BaseIndex
     m = Message.new :source => source, :source_info =>
entry[:source_info],
                     :labels => entry[:labels], :snippet =>
entry[:snippet]
 
-    mk_person = lambda { |x| Person.new(*x.reverse!) }
+    mk_person = lambda { |x| QuickPerson.new(*x) }
     entry[:from] = mk_person[entry[:from]]
     entry[:to].map!(&mk_person)
     entry[:cc].map!(&mk_person)
@@ -84,6 +84,14 @@ class XapianIndex < BaseIndex
     m
   end
 
+  class QuickPerson < Person
+    def initialize email, name
+      raise ArgumentError, "email can''t be nil" unless email
+      @email = email
+      @name = name
+    end
+  end
+
   def add_message m; sync_message m end
   def update_message m; sync_message m end
   def update_message_state m; sync_message m end
-- 
1.6.4

Reformatted excerpts from Rich Lane''s message of
2009-08-22:
> Message#parse_header is slow, taking up to 2/3 of the time spent
> loading threads in thread-index-mode. This patch adds a shortcut
> method that the index can use to efficiently initialize a message.
Yes, great catch. I''ve applied this to xapian-updates branch and
remerged into next.

BTW can you base your patches off master instead of next? It will save
me a couple keystrokes.
-- 
William <wmorgan-sup at masanjin.net>

Reformatted excerpts from Rich Lane''s message of
2009-08-22:
> -    mk_person = lambda { |x| Person.new(*x.reverse!) }
> +    mk_person = lambda { |x| QuickPerson.new(*x) }
What about lambda { |x| Person.new x[1], x[0] }. Surely that must be
even faster?
-- 
William <wmorgan-sup at masanjin.net>

Excerpts from William Morgan''s message of Mon Aug 24 19:24:19 -0400
2009:
> Reformatted excerpts from Rich Lane''s message of 2009-08-22:
> > -    mk_person = lambda { |x| Person.new(*x.reverse!) }
> > +    mk_person = lambda { |x| QuickPerson.new(*x) }
> 
> What about lambda { |x| Person.new x[1], x[0] }. Surely that must be
> even faster?
The slow part is the processing in Person#initialize, which QuickPerson
overrides. You might also be able to avoid that by moving the
initialize() code into Person.from_address.

Reformatted excerpts from Rich Lane''s message of
2009-08-25:
> The slow part is the processing in Person#initialize, which
> QuickPerson overrides. You might also be able to avoid that by moving
> the initialize() code into Person.from_address.
Yeah, I think that''s a better approach. There''s no need for
the
constructor to do all that work. I''ll work on this when I get a chance,
or feel free to beat me to it.
-- 
William <wmorgan-sup at masanjin.net>