sup talk - Jan 2010 - Questions about offlineimap+msmtp+maildir+gmail syncing

Hi,

This is my first real email with sup! I configured sup to work with my
Gmail(hosted apps) account as follows:

1. Offlineimap fetches email and saves to Maildir (I am behind a very
   slow connection, I need to mirror email)
2. The appropriate Maildir is added as a source in sup. Specifically,
   the Gmail.[All Mail] Folder, which has "cur".
3. I configured msmtp to be used for sending email instead of sendmail.

So far I can read all my emails, and the experience is great. I have a
few minor questions:

1. When I send email via sup, I see the same email as an unread message
   in my Gmail web frontend in the Inbox folder rather than the sent
   folder.
2. Any changes in sup aren''t reflected in my web frontend. I am
guessing
   this is a problem with my offlineimap configuration. Deleting or
   archiving an email doesn''t change anything in the web client.
3. I am unable to work with the drafts already existing in Gmail and the
   web interface is unaware of the drafts I have here. A suggestion on
   how to fix this would be awesome!.
4. Any other suggestions on how to make the experience with offlineimap
   would be awesome! I initially used offlineimap''s imap server
   mirroring feature, but I found putting the emails in Maildir format
   made it a lot easier.

I''ve pasted some of my config files here:
1. My config.yaml - http://antileet.pastebin.com/m3209dc32
2. My sources.yaml - http://antileet.pastebin.com/m6fc935ee
3. My .msmtprc - http://antileet.pastebin.com/m564eafc3
4. My .offlineimaprc - http://antileet.pastebin.com/m3ac306b6

Please help me get sup working! This is exactly what I''ve been looking
for. I promise to write up a detailed tutorial for the Wiki if I can get
it to work :)

Thanks,
Anirudh

--
Senior undergraduate student, Indian Institute of Technology, Kharagpur
http://anirudhsanjeev.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL:
<http://rubyforge.org/pipermail/sup-talk/attachments/20100106/5e259e73/attachment.bin>

Excerpts from Anirudh Sanjeev''s message of Wed Jan 06 06:38:20 -0500
2010:
> 1. Offlineimap fetches email and saves to Maildir (I am behind a very
>    slow connection, I need to mirror email)
> 2. The appropriate Maildir is added as a source in sup. Specifically,
>    the Gmail.[All Mail] Folder, which has "cur".
> 3. I configured msmtp to be used for sending email instead of sendmail.
My setup is similar.
> 1. When I send email via sup, I see the same email as an unread message
>    in my Gmail web frontend in the Inbox folder rather than the sent
>    folder.
It looks like in your sup config.yaml you''re only using one of your two
accounts to send mail from.  You''re probably seeing messages sent from 
anirudh at anirudhsanjeev.org in your andy at ninjagod.com gmail interface?

I would add the ninjagod account to your sup config.yaml and take a look at 
the wiki [MultipleAccountsAndReply][1] and the reply-from/before-edit 
[Hooks][2].

[1]: http://sup.rubyforge.org/wiki/wiki.pl?MultipleAccountsAndReply
[2]: http://sup.rubyforge.org/wiki/wiki.pl?Hooks
> 2. Any changes in sup aren''t reflected in my web frontend. I am
guessing
>    this is a problem with my offlineimap configuration. Deleting or
>    archiving an email doesn''t change anything in the web client.
This is because sup is primarily concerned with its own index, and does not 
at present manipulate its sources by default.  Though there is
sup-sync-back which can propagate deleted messages or marked-as-spam back 
to sources--it only supports mbox at present.

The fact that you''re not seeing changes is not your fault--sup just
doesn''t
do it.
> 3. I am unable to work with the drafts already existing in Gmail and the
>    web interface is unaware of the drafts I have here. A suggestion on
>    how to fix this would be awesome!.
You could add your gmail drafts as another source to get them into sup, 
though sup won''t actually treat those messages as drafts and you
won''t see
sup drafts in gmail.
> 4. Any other suggestions on how to make the experience with offlineimap
>    would be awesome! I initially used offlineimap''s imap server
>    mirroring feature, but I found putting the emails in Maildir format
>    made it a lot easier.
If sup actually manipulated its sources, you wouldn''t have these
issues,
but it doesn''t.  So aside from the missing account in your sup
config.yaml,
I''d say your setup is fine and is working as it should.

Sup doesn''t play well with other interfaces manipulating its 
sources--you''ll be doing the sup-sync dance a lot.  I just stopped
using
other mail clients with my sources and I''ve been happy ever since.

Good luck,
-eric

Hi!

Excerpts from Eric Sherman''s message of Wed Jan 06 18:26:20 +0530
2010:
> I would add the ninjagod account to your sup config.yaml and take a look at
> the wiki [MultipleAccountsAndReply][1] and the reply-from/before-edit 
> [Hooks][2].
That helped, and my outgoing email seems to be sorted out. Thank you.
> The fact that you''re not seeing changes is not your fault--sup
just doesn''t
> do it.
Oh, that''s too bad. Are there any plans to support syncing with IMAP or
Maildir anytime soon? Either way, looks like I''ll have to quit the Web
interface and Evolution cold turkey and stick with sup!

Since sup supports sync with mbox, is there any utility that takes IMAP
and packs it into mbox format, or maybe converts Maildir messages into
mbox?

Thanks!
Anirudh
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL:
<http://rubyforge.org/pipermail/sup-talk/attachments/20100107/0a14be52/attachment.bin>

Reformatted excerpts from Anirudh Sanjeev''s message of
2010-01-06:
> Since sup supports sync with mbox, is there any utility that takes IMAP
> and packs it into mbox format, or maybe converts Maildir messages into
> mbox?
Fetchmail for the first, and I think there''s probably and md2mbox or
something out there for the second.
-- 
William <wmorgan-sup at masanjin.net>

for the first, I''ve used fetchmail to do that:

: 576...; cat fetchmailrc



poll hostname.domain.com proto IMAP

  user "jmason" pass "[your imap password]" is
"jmason"

  folder "name_of_folder"

  ssl

  fetchall

  keep

  no rewrite

  expunge 100

  mda "/usr/bin/procmail -f %F /home/jmason/Mail/procmailrc";



: 577...; cat procmailrc

MAILDIR=$HOME
DEFAULT=$MAILDIR/mbox



:0

$DEFAULT





and it''ll append it all to ~/mbox.

--j.

On Thu, Jan 14, 2010 at 14:07, William Morgan <wmorgan-sup at
masanjin.net> wrote:
> Reformatted excerpts from Anirudh Sanjeev''s message of 2010-01-06:
>> Since sup supports sync with mbox, is there any utility that takes IMAP
>> and packs it into mbox format, or maybe converts Maildir messages into
>> mbox?
>
> Fetchmail for the first, and I think there''s probably and md2mbox
or
> something out there for the second.
> --
> William <wmorgan-sup at masanjin.net>
> _______________________________________________
> sup-talk mailing list
> sup-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/sup-talk
>
>


-- 
--j.

hmm.  apologies for the additional line breaks; a cut and paste bug
with Synergy.  here''s fixed versions:

: 576...; cat fetchmailrc

poll hostname.domain.com proto IMAP
 ?user "jmason" pass "[your imap password]" is
"jmason"
 ?folder "name_of_folder"
 ?ssl
 ?fetchall
 ?keep
 ?no rewrite
 ?expunge 100
 ?mda "/usr/bin/procmail -f %F /home/jmason/Mail/procmailrc";

: 577...; cat procmailrc
MAILDIR=$HOME
DEFAULT=$MAILDIR/mbox

:0
$DEFAULT


--j.

On Thu, Jan 14, 2010 at 14:30, Justin Mason <jm at jmason.org>
wrote:
> for the first, I''ve used fetchmail to do that:
>
> : 576...; cat fetchmailrc
>
>
>
> poll hostname.domain.com proto IMAP
>
> ?user "jmason" pass "[your imap password]" is
"jmason"
>
> ?folder "name_of_folder"
>
> ?ssl
>
> ?fetchall
>
> ?keep
>
> ?no rewrite
>
> ?expunge 100
>
> ?mda "/usr/bin/procmail -f %F /home/jmason/Mail/procmailrc";
>
>
>
> : 577...; cat procmailrc
>
> MAILDIR=$HOME
> DEFAULT=$MAILDIR/mbox
>
>
>
> :0
>
> $DEFAULT
>
>
>
>
>
> and it''ll append it all to ~/mbox.
>
> --j.
>
> On Thu, Jan 14, 2010 at 14:07, William Morgan <wmorgan-sup at
masanjin.net> wrote:
>> Reformatted excerpts from Anirudh Sanjeev''s message of
2010-01-06:
>>> Since sup supports sync with mbox, is there any utility that takes
IMAP
>>> and packs it into mbox format, or maybe converts Maildir messages
into
>>> mbox?
>>
>> Fetchmail for the first, and I think there''s probably and
md2mbox or
>> something out there for the second.
>> --
>> William <wmorgan-sup at masanjin.net>
>> _______________________________________________
>> sup-talk mailing list
>> sup-talk at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/sup-talk
>>
>>
>
>
>
> --
> --j.
>


-- 
--j.

On Wed, Jan 6, 2010 at 2:09 PM, Anirudh Sanjeev <anirudh at
anirudhsanjeev.org
> wrote:
> Hi!
>
> Excerpts from Eric Sherman''s message of Wed Jan 06 18:26:20 +0530
2010:
> > I would add the ninjagod account to your sup config.yaml and take a
look
> at
> > the wiki [MultipleAccountsAndReply][1] and the reply-from/before-edit
> > [Hooks][2].
> That helped, and my outgoing email seems to be sorted out. Thank you.
>
> > The fact that you''re not seeing changes is not your
fault--sup just
> doesn''t
> > do it.
> Oh, that''s too bad. Are there any plans to support syncing with
IMAP or
> Maildir anytime soon? Either way, looks like I''ll have to quit the
Web
> interface and Evolution cold turkey and stick with sup!
>
> Since sup supports sync with mbox, is there any utility that takes IMAP
> and packs it into mbox format, or maybe converts Maildir messages into
> mbox?
>
>
I''m currently working on getting sup to support syncing back to
maildirs.
I''m using a setup a lot like yours, with offlineimap syncing with an
imap
server.

I''ve currently got a local branch that makes sup use maildir ids as the
internal sup id.  I''ve almost got sup to the point that it will sync
back
changes to a maildir.  I have a few more bugs to work out, but I intend to
post a patch relatively soon.



-- 
Scott Henson
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/sup-talk/attachments/20100116/2c467538/attachment.html>

Scott Henson, 2010-01-16 19:21:
> I''m currently working on getting sup to support syncing back to
maildirs.
Wonderful!  I love you!  :-*
> I''ve currently got a local branch that makes sup use maildir ids
as the
> internal sup id.
Why would you need to do that?  Afaik storing the maildir id to index
should be enough so that you can map sup index back to maildir.  Am I
overlooking somethin here?

-- 
Tero Tilus ## 050 3635 235 ## http://tero.tilus.net/

Reformatted excerpts from Tero Tilus''s message of
2010-01-16:
> Why would you need to do that?  Afaik storing the maildir id to index
> should be enough so that you can map sup index back to maildir.  Am I
> overlooking somethin here?
Yeah, there''s a Maildir#filename_for_id that you should be able to use.
-- 
William <wmorgan-sup at masanjin.net>

Hi,

I wrote a small document describing how I got sup to work with Gmail,
since there isn''t any "definitive" guide on it.

I really want some review and feedback on the text:
http://sup.rubyforge.org/wiki/wiki.pl?GmailOfflineImapSmtp

Thank you,
Anirudh

--
http://anirudhsanjeev.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL:
<http://rubyforge.org/pipermail/sup-talk/attachments/20100117/9205e03f/attachment-0001.bin>

On Sun, Jan 17, 2010 at 9:54 AM, William Morgan <wmorgan-sup at
masanjin.net>wrote:
> Reformatted excerpts from Tero Tilus''s message of 2010-01-16:
> > Why would you need to do that?  Afaik storing the maildir id to index
> > should be enough so that you can map sup index back to maildir.  Am I
> > overlooking somethin here?
>
> Yeah, there''s a Maildir#filename_for_id that you should be able to
use.
>
>
Using the maildir id as the sup id allows sup to play a little fast and
loose with the access of mail in the maildir.  I''ve got a 15% speed
increase
(measured by sup-sync --all).  This also allows sup to play nice with other
mail clients accessing the maildir.

I''m down to the last few issues with my new maildir sync back code, I
just
can''t get sup to show me the deleted and spam messages.  I''m
working on the
next branch, and it seems like it might be having problems with ruby 1.8
that comes with F12 and I seem to be unable to get ruby 1.9 installed.  Once
I get things working I''ll publish the branch to github.


-- 
Scott Henson
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/sup-talk/attachments/20100117/9564fde2/attachment.html>

Reformatted excerpts from Scott Henson''s message of
2010-01-17:
> I''m down to the last few issues with my new maildir sync back
code, I
> just can''t get sup to show me the deleted and spam messages.
If you''re going through index.each_message or each_id, you need to
specify :load_spam => true and load_deleted => true.
-- 
William <wmorgan-sup at masanjin.net>

I''m using Redwood::PollManager.each_message_from source like
sup-sync-back
uses.  Looking at the definition of that function, it will load all messages
from the source without discrimination.  I''ve put my repo up on github
(
http://github.com/shenson/sup) with my code in the release branch.  It
includes my code to change the source id to the maildir id.  The code
I''m
actually running is in bin/sup-clean.

As far as I can tell, I''m using the same basic structure as
sup-sync-back,
so I should be getting back all the messages.  If anyone wants to clue me in
on what I''m doing wrong, I''d be very grateful.  Thanks.


On Sun, Jan 17, 2010 at 3:42 PM, William Morgan <wmorgan-sup at
masanjin.net>wrote:
> Reformatted excerpts from Scott Henson''s message of 2010-01-17:
> > I''m down to the last few issues with my new maildir sync back
code, I
> > just can''t get sup to show me the deleted and spam messages.
>
> If you''re going through index.each_message or each_id, you need to
> specify :load_spam => true and load_deleted => true.
> --
> William <wmorgan-sup at masanjin.net>
> _______________________________________________
> sup-talk mailing list
> sup-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/sup-talk
>


-- 
Scott Henson
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/sup-talk/attachments/20100118/db44fd8b/attachment-0001.html>