R devel - Sep 2017 - what do you think about write.table(... qmethod = "excel")?

Last week one of our clients reported trouble with a csv file I
generated with write.table.  He said that columns with quotes for
character variables were rejected by their data importer, which was
revised to match the way Microsoft Excel uses quotation marks in
character variables.  I explained to them that quoted character
variables are virtuous and wise, of course, but they say Microsoft
Excel CSV export no longer quotes characters unless they include
commas in the values.

They showed me a CSV file from Excel that looked like this

x1,x2,x3,x4 5 6
fred,barney,betty,x
bambam,"fred,wilma",pebbles,y

Note how the quotes only happen on row 2 column 2. I was surprised it
did that, but now I have some pressure to write a csv maker that has
that structure.  Its weird, even when there are spaces in values there
are no quotation marks.

Has anybody done this and verified that it matches CSV from MS Excel?
If I succeed will you consider a patch?

pj
-- 
Paul E. Johnson   http://pj.freefaculty.org
Director, Center for Research Methods and Data Analysis http://crmda.ku.edu

To write to me directly, please address me at pauljohn at ku.edu.

On 19 September 2017 at 12:04, Paul Johnson wrote:
| They showed me a CSV file from Excel that looked like this
| 
| x1,x2,x3,x4 5 6
| fred,barney,betty,x
| bambam,"fred,wilma",pebbles,y
| 
| Note how the quotes only happen on row 2 column 2. I was surprised it
| did that, but now I have some pressure to write a csv maker that has
| that structure.  Its weird, even when there are spaces in values there
| are no quotation marks.
| 
| Has anybody done this and verified that it matches CSV from MS Excel?
| If I succeed will you consider a patch?

R> data.table::fread("/tmp/paul.csv")
       x1         x2      x3 x4 5 6
1:   fred     barney   betty      x
2: bambam fred,wilma pebbles      y
R> data.table::fread("/tmp/paul.csv")
       x1         x2      x3 x4
1:   fred     barney   betty  x
2: bambam fred,wilma pebbles  y
R> 

The only difference is that between calls one and two, I removed the stray
    "5 6"
from the first line.

Dirk

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

On Tue, Sep 19, 2017 at 1:04 PM, Paul Johnson <pauljohn32 at gmail.com>
wrote:
> Last week one of our clients reported trouble with a csv file I
> generated with write.table.  He said that columns with quotes for
> character variables were rejected by their data importer, which was
> revised to match the way Microsoft Excel uses quotation marks in
> character variables.  I explained to them that quoted character
> variables are virtuous and wise, of course, but they say Microsoft
> Excel CSV export no longer quotes characters unless they include
> commas in the values.
>
> They showed me a CSV file from Excel that looked like this
>
> x1,x2,x3,x4 5 6
> fred,barney,betty,x
> bambam,"fred,wilma",pebbles,y
>
> Note how the quotes only happen on row 2 column 2. I was surprised it
> did that, but now I have some pressure to write a csv maker that has
> that structure.
I think you should resist that pressure. It really makes no sense to
write a .csv parser that _only_ supports .csv files created by Excel.
If you're going to use Excel as a model, a more sensible approach
would be to write a csv parser that supports all the formats that
Excel itself supports; Excel of course has no problem importing

"x1","x2","x3","x4"
"fred","barney","betty","x"
"bambam","fred,wilma","pebbles","y"

So, seriously, tell them to just fix their csv parser. Since they seem
hung up on Excel, it may help to point out that it does in fact import
csv produced by write.csv without complaint.

Best,
Ista

 Its weird, even when there are spaces in values there
> are no quotation marks.
>
> Has anybody done this and verified that it matches CSV from MS Excel?
> If I succeed will you consider a patch?
>
> pj
> --
> Paul E. Johnson   http://pj.freefaculty.org
> Director, Center for Research Methods and Data Analysis http://crmda.ku.edu
>
> To write to me directly, please address me at pauljohn at ku.edu.
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

On 19/09/2017 4:10 PM, Ista Zahn wrote:
> On Tue, Sep 19, 2017 at 1:04 PM, Paul Johnson <pauljohn32 at
gmail.com> wrote:
>> Last week one of our clients reported trouble with a csv file I
>> generated with write.table.  He said that columns with quotes for
>> character variables were rejected by their data importer, which was
>> revised to match the way Microsoft Excel uses quotation marks in
>> character variables.  I explained to them that quoted character
>> variables are virtuous and wise, of course, but they say Microsoft
>> Excel CSV export no longer quotes characters unless they include
>> commas in the values.
>>
>> They showed me a CSV file from Excel that looked like this
>>
>> x1,x2,x3,x4 5 6
>> fred,barney,betty,x
>> bambam,"fred,wilma",pebbles,y
>>
>> Note how the quotes only happen on row 2 column 2. I was surprised it
>> did that, but now I have some pressure to write a csv maker that has
>> that structure.
> 
> I think you should resist that pressure.
That depends on whether this is a paying client or not.

 > It really makes no sense to
> write a .csv parser that _only_ supports .csv files created by Excel.
That's true, but if that's what they want to do, and they're willing
to
pay to be able to write files that imitate Excel, then why not do what 
they ask?

On the other hand, if they aren't willing to pay for the work, then you 
should lecture them on how silly their request is.

In any case, base R functions should not include nonsense, so this is 
not something that should go into R.

Duncan Murdoch
> If you're going to use Excel as a model, a more sensible approach
> would be to write a csv parser that supports all the formats that
> Excel itself supports; Excel of course has no problem importing
> 
> "x1","x2","x3","x4"
> "fred","barney","betty","x"
> "bambam","fred,wilma","pebbles","y"
> 
> So, seriously, tell them to just fix their csv parser. Since they seem
> hung up on Excel, it may help to point out that it does in fact import
> csv produced by write.csv without complaint.
> 
> Best,
> Ista
> 
>   Its weird, even when there are spaces in values there
>> are no quotation marks.
>>
>> Has anybody done this and verified that it matches CSV from MS Excel?
>> If I succeed will you consider a patch?
>>
>> pj
>> --
>> Paul E. Johnson   http://pj.freefaculty.org
>> Director, Center for Research Methods and Data Analysis
http://crmda.ku.edu
>>
>> To write to me directly, please address me at pauljohn at ku.edu.
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>