We''re using Vpim to parse iCal and vCard files, and so far it''s been great. However, when we were parsing some iCal files we noticed that all the endlines in the description section were disappearing. I start peering through the source code and find (in the Field#to_text method that is being called): # The value as text. Text can have escaped newlines, commas, and escape # characters, this method will strip them, if present. Is there some reason that this is done? I''d much prefer to keep my commas and newlines... Thanks, Emmett
Quoting emmett.shear at gmail.com, on Sun, Apr 02, 2006 at 06:47:12PM -0400:> We''re using Vpim to parse iCal and vCard files, and so far it''s been > great. However, when we were parsing some iCal files we noticed that > all the endlines in the description section were disappearing. I start > peering through the source code and find (in the Field#to_text method > that is being called): > > # The value as text. Text can have escaped newlines, commas, and escape > # characters, this method will strip them, if present. > > Is there some reason that this is done? I''d much prefer to keep my > commas and newlines...The comment isn''t well written, its stripping the ESCAPING, which should get you back the newlines. See the definition of Vpim::decode_text. What version are you using, and can I have some sample input? Maybe there is a bug. But since it works correctly, to my knowledge: % ruby -I lib -r vpim/field -e ''puts Vpim::DirectoryInfo::Field.decode("F:a\\, b\\, c\\ndnext line\\nnext line\\n").to_text'' a, b, c dnext line next line I''m wondering if you might not have something like this: ---- NOTE:some text some more text another bunch of text ---- Because that note doesn''t have any newlines in it. A 3 line note would look like: ---- NOTE:line 1\nline 2\nline 3 ---- And would have the identical value if wrapped, where wrapping can occur anywhere: ---- NOTE:lin e 1\nl in e 2\nl ine 3 ---- The above still only has two newlines in it. I didn''t have an encode_text method until recently, so maybe this has caused you trouble. You can add a note to a vCard using Vcard#make and Vcard::Maker#add_note, see the online docs for the last release. Unfortunately, I''m about half-way through adding similar APIs for Vevent. Maybe I''ll post what I''ve done this week, unstable though it may be, it will do description, summary, and the other text fields. Anyhow, see Vpim::encode_text if you want to put text containing newlines, and commas, and other special characters, into a field value. Cheers, Sam
I''m parsing the iCal feed from webcal://upcoming.org/calendar/metro/45 (the webcal:// protocol is exactly like http, so in ruby you can get the contents using Net::HTTP.get if you replace "webcal" with "http"). The description fields of the events look like "text text\n\r \n\r text text" in their raw form, and then "text text text text" in their decoded form. Do I need to re-escape the endlines before processing? Thanks, Emmett On 4/2/06, Sam Roberts <sroberts at uniserve.com> wrote:> Quoting emmett.shear at gmail.com, on Sun, Apr 02, 2006 at 06:47:12PM -0400: > > We''re using Vpim to parse iCal and vCard files, and so far it''s been > > great. However, when we were parsing some iCal files we noticed that > > all the endlines in the description section were disappearing. I start > > peering through the source code and find (in the Field#to_text method > > that is being called): > > > > # The value as text. Text can have escaped newlines, commas, and escape > > # characters, this method will strip them, if present. > > > > Is there some reason that this is done? I''d much prefer to keep my > > commas and newlines... > > The comment isn''t well written, its stripping the ESCAPING, which > should get you back the newlines. See the definition of > Vpim::decode_text. > > What version are you using, and can I have some sample input? Maybe > there is a bug. > > But since it works correctly, to my knowledge: > > % ruby -I lib -r vpim/field -e ''puts Vpim::DirectoryInfo::Field.decode("F:a\\, b\\, c\\ndnext line\\nnext line\\n").to_text'' > a, b, c > dnext line > next line > > > I''m wondering if you might not have something like this: > > ---- > NOTE:some text > some more text > another bunch of text > ---- > > Because that note doesn''t have any newlines in it. A 3 line note would > look like: > > ---- > NOTE:line 1\nline 2\nline 3 > ---- > > And would have the identical value if wrapped, where wrapping can > occur anywhere: > > ---- > NOTE:lin > e 1\nl > in > e 2\nl > ine > 3 > ---- > > The above still only has two newlines in it. > > I didn''t have an encode_text method until recently, so maybe this has > caused you trouble. > > You can add a note to a vCard using Vcard#make and > Vcard::Maker#add_note, see the online docs for the last release. > > Unfortunately, I''m about half-way through adding similar APIs for > Vevent. Maybe I''ll post what I''ve done this week, unstable though it may > be, it will do description, summary, and the other text fields. Anyhow, > see Vpim::encode_text if you want to put text containing newlines, and > commas, and other special characters, into a field value. > > Cheers, > Sam > >
Quoting emmett.shear at gmail.com, on Sun, Apr 02, 2006 at 11:46:17PM -0400:> I''m parsing the iCal feed from webcal://upcoming.org/calendar/metro/45 > (the webcal:// protocol is exactly like http, so in ruby you can get > the contents using Net::HTTP.get if you replace "webcal" with "http"). > The description fields of the events look like "text text\n\r \n\r > text text" in their raw form,Not in the file I downloaded, they don''t. I grepped the entire file, and didn''t find a single occurence of a backslash followed by a n. What it is is this: BEGIN:VEVENT DTSTART;TZID=US-Pacific:20060402 DTEND;TZID=US-Pacific:20060403 SUMMARY:Daniel Bell + John Tejada + the Sunset Party comes to boca DESCRIPTION:Sunday\, April 2nd ^M NOON - 2am^M ^M it'\;s time to get...^M OUT OF^M [KONTROL]^M ^M ^M with 2 legends playing back to back!^M <b>DANIEL BELL</b> (7th City\, Accelerate\, Tresor)^M ^M +^M <b>JOHN TEJADA</b> ^M (Palette\, Poker Flat\, Playhouse)^M The description text appears to be html, which may not be illegal, but sure isn''t standard. It will look pretty bad in most calendar viewers, I''d think, especially since its one long line with random whitespace. It is pretty much unreadable in iCal. They are making some small attempts at escaping (you can see "'\;", the ";'' was escaped), but they are putting newlines in as raw carriage returns, instead of the two character sequence 0x5C 0x62 (backslash, n). Weird, I''d expect them to either not try to escape at all, or to get it right, but they are half-way.> and then "text text text text" in > their decoded form. Do I need to re-escape the endlines before > processing?Not sure what you mean by rescape, or by processing. What are you trying to do? If you tell me that, I can probably give some suggestions on how to deal with this. Off the top of my head, the only place a raw CR shows up, I think, is where they want a newline in a TEXT property value, so you could preprocess the calendar, doing basically calstr = calstr.gsub("\r", "\\n") Its late at night, so thats all for now. Cheers, Sam
Sam Roberts
2006-Apr-04 05:35 UTC
[Vpim-talk] passing a converter object into Icalendar.decode
Quoting sroberts at uniserve.com, on Sun, Apr 02, 2006 at 11:57:29PM -0700:> Quoting emmett.shear at gmail.com, on Sun, Apr 02, 2006 at 11:46:17PM -0400: > > I''m parsing the iCal feed from webcal://upcoming.org/calendar/metro/45 > > (the webcal:// protocol is exactly like http, so in ruby you can get> calstr = calstr.gsub("\r", "\\n")Ok, the above approach runs a good chance of being killingly memory intensive for large calendars, like the one you are looking at. Here''s a better approach: The object passed to Icalendar.decode needs to support #each, where each line is yielded one by one. Happens to work well for String and IO, but you can provide any kind of object you want, like this: require ''vpim/icalendar'' require ''open-uri'' class Convert def initialize(io) @io = io end def each @io.each do |line| line.gsub!("\r", "\\n") yield line end end end io = open("http://upcoming.org/calendar/metro/45") cvt = Convert.new(io) cal = Vpim::Icalendar.decode(cvt).first cal.components do |c| puts "-------------------------------------------------" puts c.description end Output has the newlines where you most likely want them. Some days I really, really like duck-typing. Cheers, Sam Btw, here''s the first few dozen lines of output of the above, with a line feed where I guess they were intended: ------------------------------------------------- http://www.mobilemonday.com/ ------------------------------------------------- GWEN CHAN, Interim Superintendent, San Francisco Unified School District DENNIS CONAGHAN, Executive Director, San Francisco Center for Economic Development AMIT GHOSH, Chief of Comprehensive Planning, San Francisco Planning Department JOHN KILLACKY, Arts and Culture Program Officer, San Francisco Foundation SCOTT SHAFER, Host, "The California Report," KQED ?? Moderator FREE FOR MEMBERS (More) WHERE HAVE ALL THE YOUNG PEOPLE GONE? WHY SAN FRANCISCO IS LOSING ITS YOUNG WORKERS AND FAMILIES According to a recent survey from the Mayor's Office, nearly half the families in San Francisco with preschool children stated their intent to move out in the next three years, citing the lack of affordable housing and the state of the public schools as primary motivators. The findings confirm what many have known for a long time - with the highest median home prices of any major U.S. city and the smallest share of children under age 18, San Francisco has a hard time attracting and retaining young workers and families. What is the city doing to improve affordable housing, job growth and the quality of public schools? Our panel will discuss what is needed to allow young residents and families to stay in San Francisco for the long term. 6:00 p.m., Check-in | 6:30 p.m., Program | 7:30 p.m., Wine and cheese reception Free for Members, $15 for Non-members http://www.commonwealthclub.org/featured.html#sfaffordability ------------------------------------------------- http://www.highlandteam.com/news/dev_series.html Alumni Career Services at the Stanford Graduate School of Business, together with Highland Team, a marketing strategy consulting firm, and Kathryn Ullrich Associates, Inc., experts in executive recruiting, present Getting to The Top: Getting Unstuck. Are you at a place in your career where you're asking questions such as: **How do I redefine myself and move in to a new career? **What can I do about a difficult boss? **How do I add more fun or creativity to my job? **What can I do to get beyond where I am today? In the upcoming program in the career development series, Getting to the Top, executive coaches share tools and tips for getting unstuck in your career. Featured executive coaches: **Susan Bernstein, Work from Within **Deb Colden, Executive Coach and Consultant