John MacFarlane, the author of Pandoc, has been working on a project ( http://commonmark.org/) to define a standard reference for Markdown*. There are already two reference implementations, one in javascript, the other in C: https://github.com/jgm/cmark Regards, baptiste * There was some initial controversy with the original author of markdown, but in the long term it's probably one of the more reliable sources to follow. On 24 May 2015 at 12:00, Duncan Murdoch <murdoch.duncan at gmail.com> wrote:> On 23/05/2015 9:25 AM, G?bor Cs?rdi wrote: > > On Sat, May 23, 2015 at 8:14 AM, Duncan Murdoch > > <murdoch.duncan at gmail.com <mailto:murdoch.duncan at gmail.com>> wrote: > > [...] > > > > I think the harder problem is display. CRAN can run pandoc, but can > > users who install the package from source? I would expect some > obscure > > platforms (like Windows ;-) would not have it available. > > > > [...] > > > > I don't think pandoc is the best way to go with NEWS.md (and README.md, > > actually). I would be surprised if many package maintainer built their > > NEWS/README files with pandoc. They just look at them at GitHub (or > > another similar service). > > > > GitHub has API for building HTML from > > MarkDown: https://developer.github.com/v3/markdown/ > > It can build GitHub-flavored MarkDown, in which case you get links to > > GitHub issues, etc. or just plain MarkDown, like a GitHub README. > > > > If you don't want to rely on their service, then there are a multitude > > of lightweight MarkDown parsers available, > > e.g. https://github.com/markdown-it/markdown-it is a good one IMO. > > I wouldn't want R builds to depend on GitHub, so this sounds more > interesting. I took a look at that website, and it looks problematic to > me: the parser appears to be written in Javascript, and the install > instructions (using "npm" and "bower", whatever those are) depend on > some unstated prerequisites. In principle there's no reason not to > allow R builds to depend on these things, but adding a dependency like > that implies so much testing that I can't imagine anyone who could do it > would want to. > > It's likely that a suitable parser could be written in some combination > of C and R -- Markdown is not a complicated language. > > > Pandoc is great for vignettes, but you don't need its full power for > > READMEs and especially not for NEWS files. In fact most NEWS.md files > > look good as text. > > But we do need something, and it needs to be essentially universally > available, or small enough to include in the R sources. I think R > should eventually support Markdown as an acceptable language for > documentation (including NEWS.md, and also help files for functions), > but I think the effort required to do it now is too much. > > Duncan Murdoch > > > > > Gabor > > > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel >[[alternative HTML version deleted]]
While a parsed HTML version of the NEWS.md file would be nice, I would like something much simpler: being able to "see? this file in the Help pane in RStudio or being about to run something like show_news(?packagename?). Duncan mentioned issues with the news() function being able to process metadata represented in the Md file. What is the motivation of this structure?> On 24 May 2015, at 10:51 am, Baptiste Auguie <baptiste.auguie at gmail.com> wrote: > > John MacFarlane, the author of Pandoc, has been working on a project (http://commonmark.org/) to define a standard reference for Markdown*. There are already two reference implementations, one in javascript, the other in C: https://github.com/jgm/cmark > > Regards, > > baptiste > > * There was some initial controversy with the original author of markdown, but in the long term it's probably one of the more reliable sources to follow. > > On 24 May 2015 at 12:00, Duncan Murdoch <murdoch.duncan at gmail.com> wrote: > On 23/05/2015 9:25 AM, G?bor Cs?rdi wrote: > > On Sat, May 23, 2015 at 8:14 AM, Duncan Murdoch > > <murdoch.duncan at gmail.com <mailto:murdoch.duncan at gmail.com>> wrote: > > [...] > > > > I think the harder problem is display. CRAN can run pandoc, but can > > users who install the package from source? I would expect some obscure > > platforms (like Windows ;-) would not have it available. > > > > [...] > > > > I don't think pandoc is the best way to go with NEWS.md (and README.md, > > actually). I would be surprised if many package maintainer built their > > NEWS/README files with pandoc. They just look at them at GitHub (or > > another similar service). > > > > GitHub has API for building HTML from > > MarkDown: https://developer.github.com/v3/markdown/ > > It can build GitHub-flavored MarkDown, in which case you get links to > > GitHub issues, etc. or just plain MarkDown, like a GitHub README. > > > > If you don't want to rely on their service, then there are a multitude > > of lightweight MarkDown parsers available, > > e.g. https://github.com/markdown-it/markdown-it is a good one IMO. > > I wouldn't want R builds to depend on GitHub, so this sounds more > interesting. I took a look at that website, and it looks problematic to > me: the parser appears to be written in Javascript, and the install > instructions (using "npm" and "bower", whatever those are) depend on > some unstated prerequisites. In principle there's no reason not to > allow R builds to depend on these things, but adding a dependency like > that implies so much testing that I can't imagine anyone who could do it > would want to. > > It's likely that a suitable parser could be written in some combination > of C and R -- Markdown is not a complicated language. > > > Pandoc is great for vignettes, but you don't need its full power for > > READMEs and especially not for NEWS files. In fact most NEWS.md files > > look good as text. > > But we do need something, and it needs to be essentially universally > available, or small enough to include in the R sources. I think R > should eventually support Markdown as an acceptable language for > documentation (including NEWS.md, and also help files for functions), > but I think the effort required to do it now is too much. > > Duncan Murdoch > > > > > Gabor > > > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel >
On 23/05/2015 8:51 PM, Baptiste Auguie wrote:> John MacFarlane, the author of Pandoc, has been working on a project > (http://commonmark.org/) to define a standard reference for Markdown*. > There are already two reference implementations, one in javascript, the > other in C: https://github.com/jgm/cmarkIt sounds as though there are at least two possibilities for parsers that could be included in R: Sundown and commonmark. The "markdown" package does some of what R would need to make use of Sundown, but not all: we really do need an R object representation of the parse tree, for functions like news() to work with in the short term, and the help system in the longer term. To allow NEWS.md to be used, we'd also need someone to work out the conventions for what's allowed (probably very similar to the conventions for NEWS or NEWS.Rd), and write code to work with those files. Does anyone want to work on this? Duncan Murdoch> > Regards, > > baptiste > > * There was some initial controversy with the original author of > markdown, but in the long term it's probably one of the more reliable > sources to follow. > > On 24 May 2015 at 12:00, Duncan Murdoch <murdoch.duncan at gmail.com > <mailto:murdoch.duncan at gmail.com>> wrote: > > On 23/05/2015 9:25 AM, G?bor Cs?rdi wrote: > > On Sat, May 23, 2015 at 8:14 AM, Duncan Murdoch > > <murdoch.duncan at gmail.com <mailto:murdoch.duncan at gmail.com> > <mailto:murdoch.duncan at gmail.com <mailto:murdoch.duncan at gmail.com>>> > wrote: > > [...] > > > > I think the harder problem is display. CRAN can run pandoc, but can > > users who install the package from source? I would expect some obscure > > platforms (like Windows ;-) would not have it available. > > > > [...] > > > > I don't think pandoc is the best way to go with NEWS.md (and README.md, > > actually). I would be surprised if many package maintainer built their > > NEWS/README files with pandoc. They just look at them at GitHub (or > > another similar service). > > > > GitHub has API for building HTML from > > MarkDown: https://developer.github.com/v3/markdown/ > > It can build GitHub-flavored MarkDown, in which case you get links to > > GitHub issues, etc. or just plain MarkDown, like a GitHub README. > > > > If you don't want to rely on their service, then there are a multitude > > of lightweight MarkDown parsers available, > > e.g. https://github.com/markdown-it/markdown-it is a good one IMO. > > I wouldn't want R builds to depend on GitHub, so this sounds more > interesting. I took a look at that website, and it looks problematic to > me: the parser appears to be written in Javascript, and the install > instructions (using "npm" and "bower", whatever those are) depend on > some unstated prerequisites. In principle there's no reason not to > allow R builds to depend on these things, but adding a dependency like > that implies so much testing that I can't imagine anyone who could do it > would want to. > > It's likely that a suitable parser could be written in some combination > of C and R -- Markdown is not a complicated language. > > > Pandoc is great for vignettes, but you don't need its full power for > > READMEs and especially not for NEWS files. In fact most NEWS.md files > > look good as text. > > But we do need something, and it needs to be essentially universally > available, or small enough to include in the R sources. I think R > should eventually support Markdown as an acceptable language for > documentation (including NEWS.md, and also help files for functions), > but I think the effort required to do it now is too much. > > Duncan Murdoch > > > > > Gabor > > > > ______________________________________________ > R-devel at r-project.org <mailto:R-devel at r-project.org> mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > >
On 23/05/2015 9:15 PM, Imanuel Costigan wrote:> While a parsed HTML version of the NEWS.md file would be nice, I would like something much simpler: being able to "see? this file in the Help pane in RStudioThat isn't really any simpler. RStudio is just displaying HTML whenever it shows you anything in the Help pane. or being about to run something like show_news(?packagename?). Duncan mentioned issues with the news() function being able to process metadata represented in the Md file. What is the motivation of this structure? I don't understand your question. What issues did I mention? Or are you talking about Kurt's post, who first mentioned news()? And what structure are you talking about? Duncan Murdoch> > >> On 24 May 2015, at 10:51 am, Baptiste Auguie <baptiste.auguie at gmail.com> wrote: >> >> John MacFarlane, the author of Pandoc, has been working on a project (http://commonmark.org/) to define a standard reference for Markdown*. There are already two reference implementations, one in javascript, the other in C: https://github.com/jgm/cmark >> >> Regards, >> >> baptiste >> >> * There was some initial controversy with the original author of markdown, but in the long term it's probably one of the more reliable sources to follow. >> >> On 24 May 2015 at 12:00, Duncan Murdoch <murdoch.duncan at gmail.com> wrote: >> On 23/05/2015 9:25 AM, G?bor Cs?rdi wrote: >>> On Sat, May 23, 2015 at 8:14 AM, Duncan Murdoch >>> <murdoch.duncan at gmail.com <mailto:murdoch.duncan at gmail.com>> wrote: >>> [...] >>> >>> I think the harder problem is display. CRAN can run pandoc, but can >>> users who install the package from source? I would expect some obscure >>> platforms (like Windows ;-) would not have it available. >>> >>> [...] >>> >>> I don't think pandoc is the best way to go with NEWS.md (and README.md, >>> actually). I would be surprised if many package maintainer built their >>> NEWS/README files with pandoc. They just look at them at GitHub (or >>> another similar service). >>> >>> GitHub has API for building HTML from >>> MarkDown: https://developer.github.com/v3/markdown/ >>> It can build GitHub-flavored MarkDown, in which case you get links to >>> GitHub issues, etc. or just plain MarkDown, like a GitHub README. >>> >>> If you don't want to rely on their service, then there are a multitude >>> of lightweight MarkDown parsers available, >>> e.g. https://github.com/markdown-it/markdown-it is a good one IMO. >> >> I wouldn't want R builds to depend on GitHub, so this sounds more >> interesting. I took a look at that website, and it looks problematic to >> me: the parser appears to be written in Javascript, and the install >> instructions (using "npm" and "bower", whatever those are) depend on >> some unstated prerequisites. In principle there's no reason not to >> allow R builds to depend on these things, but adding a dependency like >> that implies so much testing that I can't imagine anyone who could do it >> would want to. >> >> It's likely that a suitable parser could be written in some combination >> of C and R -- Markdown is not a complicated language. >> >>> Pandoc is great for vignettes, but you don't need its full power for >>> READMEs and especially not for NEWS files. In fact most NEWS.md files >>> look good as text. >> >> But we do need something, and it needs to be essentially universally >> available, or small enough to include in the R sources. I think R >> should eventually support Markdown as an acceptable language for >> documentation (including NEWS.md, and also help files for functions), >> but I think the effort required to do it now is too much. >> >> Duncan Murdoch >> >>> >>> Gabor >>> >> >> ______________________________________________ >> R-devel at r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel >> >
I?d suggest going with CommonMark if this will be the basis of broader support for Markdown as it is well defined whereas GruberMarkdown is defined by broad conventions with specifics determined by parsers.> On 24 May 2015, at 12:01 pm, Duncan Murdoch <murdoch.duncan at gmail.com> wrote: > > On 23/05/2015 8:51 PM, Baptiste Auguie wrote: >> John MacFarlane, the author of Pandoc, has been working on a project >> (http://commonmark.org/) to define a standard reference for Markdown*. >> There are already two reference implementations, one in javascript, the >> other in C: https://github.com/jgm/cmark > > It sounds as though there are at least two possibilities for parsers > that could be included in R: Sundown and commonmark. The "markdown" > package does some of what R would need to make use of Sundown, but not > all: we really do need an R object representation of the parse tree, > for functions like news() to work with in the short term, and the help > system in the longer term. To allow NEWS.md to be used, we'd also need > someone to work out the conventions for what's allowed (probably very > similar to the conventions for NEWS or NEWS.Rd), and write code to work > with those files. > > Does anyone want to work on this? > > Duncan Murdoch >
On 24 May 2015 at 14:01, Duncan Murdoch <murdoch.duncan at gmail.com> wrote:> It sounds as though there are at least two possibilities for parsers > that could be included in R: Sundown and commonmark. >Sundown is in fact deprecated, https://github.com/vmg/sundown/commit/37728fb2d7137ff7c37d0a474cb827a8d6d846d8 in favor of commonmark which will likely become the standard tool for StackExchange, github, Reddit, among others. The markdown package has been somewhat superseded by rmarkdown, based on Pandoc. However commonmark would be easier to include (no dependency on Haskell), much faster, and more memory efficient. The good folks at Rstudio will probably have considered this option, and may even have started working on R bindings. Commonmark seems to use an intermediate Abstract Syntax Tree, which could be also useful for custom processing. Regards, baptiste [[alternative HTML version deleted]]
On Sun, May 24, 2015 at 4:01 AM, Duncan Murdoch <murdoch.duncan at gmail.com> wrote:> The "markdown" package does some of what R would need to make use of Sundown, but not all: we really do need an R object representation of the parse tree, for functions like news() to work with in the short term, and the help system in the longer term.John MacFarlane's reference implementation does provide the markdown parsing tree, although by default it is only exposed in xml form. A quick example: devtools::install_github("jeroenooms/commonmark") help(commonmark)>From the xml tree you can easily extract news items using xpath, or wecould hack the libcmark library a bit to get the parse tree as an R list.