Dear fellow Markdown enthusiasts, As you all might know, Markdown is a highly context sensitive language that the (often regex-based) syntax highlighting mechanisms in existing editors struggle to keep up with. I have taken John MacFarlane's excellent peg-markdown compiler and modified its parser to function as an interpreter for syntax highlighting purposes. The end result is a Markdown-specific syntax highlighter that should be able to match Markdown documents just as correctly as peg-markdown can, and that can easily be integrated into existing and new programs. You can find all the relevant information at the project web page: <http://hasseg.org/peg-markdown-highlight> best, - Ali Rantakari
Since MultiMarkdown is now based on peg-markdown, I'm quite excited about this. I only had time to glance at the source for a moment. You comment that the .leg file you use is modified slightly from MacFarlane's. How complex is the conversion? Is it possible to create a script to modify his file to become your file? The reasons I ask are: 1) If there was a script to make the conversion, it would be easy to update your code whenever he updates his version without bugging you 2) It could be possible for me to do the same thing with MMD so that the highlighter works with MMD-specific syntax features. This would be fantastic! It would be ridiculously easy to create a very simple app that did MD/MMD syntax highlighting and then processing to HTML or LaTeX or ODF(FODT) - at least on the Mac side. Thanks for announcing this!! F- On Jun 28, 2011, at 2:51 PM, Ali Rantakari wrote:> Dear fellow Markdown enthusiasts, > > As you all might know, Markdown is a highly context sensitive language that the (often regex-based) syntax highlighting mechanisms in existing editors struggle to keep up with. > > I have taken John MacFarlane's excellent peg-markdown compiler and modified its parser to function as an interpreter for syntax highlighting purposes. The end result is a Markdown-specific syntax highlighter that should be able to match Markdown documents just as correctly as peg-markdown can, and that can easily be integrated into existing and new programs. > > You can find all the relevant information at the project web page: > > <http://hasseg.org/peg-markdown-highlight> > > > best, > - Ali Rantakari > > > _______________________________________________ > Markdown-Discuss mailing list > Markdown-Discuss at six.pairlist.net > http://six.pairlist.net/mailman/listinfo/markdown-discuss-- Fletcher T. Penney fletcher at fletcherpenney.net
Related to this: For some time now there is a [markdown plugin][1] for the IntelliJ IDEA IDE and all derived ones (RubyMine, PhpStorm, etc). that does exactly what Ali is describing: syntax-highlighting based on the parser of an actual markdown processor, which in this case is [pegdown][2], that started off as a java port of John MacFarlane's peg-markdown. The way this works: pegdown does its markdown-to-HTML in two phases: 1. parsing of the markdown source into an Abstract Syntax Tree 2. serialization of the AST to HTML Since the AST generated in phase 1 retains all positional information about the underlying markdown source it's very easy to syntax-highlight the editor buffer based solely on this AST (which is exactly what the plugin is doing). So if anyone is looking for a markdown syntax-highlighting solution for the JVM pegdown is probably something to look at. For C-based applications the peg-markdown-highlight by Ali is certainly the better choice... Cheers, Mathias [1]: https://github.com/nicoulaj/idea-markdown [2]: http://www.pegdown.org --- mathias at parboiled.org http://www.parboiled.org On 28.06.2011, at 20:51, Ali Rantakari wrote:> Dear fellow Markdown enthusiasts, > > As you all might know, Markdown is a highly context sensitive language that the (often regex-based) syntax highlighting mechanisms in existing editors struggle to keep up with. > > I have taken John MacFarlane's excellent peg-markdown compiler and modified its parser to function as an interpreter for syntax highlighting purposes. The end result is a Markdown-specific syntax highlighter that should be able to match Markdown documents just as correctly as peg-markdown can, and that can easily be integrated into existing and new programs. > > You can find all the relevant information at the project web page: > > <http://hasseg.org/peg-markdown-highlight> > > > best, > - Ali Rantakari > > > _______________________________________________ > Markdown-Discuss mailing list > Markdown-Discuss at six.pairlist.net > http://six.pairlist.net/mailman/listinfo/markdown-discuss
> How complex is the conversion?The bulk of the grammar is the same but there are significant differences: - All of the grammar actions are completely different (both in terms of content and where they are applied, or not applied) - Angle brackets that delimit matched sections of interest within rules are applied differently because they determine the character offsets that will be accessible to the actions in the rule (as opposed to peg-markdown, where they delimit sections of "actual content" that the compiler wants to retain for translation purposes) - In order to get the correct start offsets for most matches I apply the `s` identifier to the first subrule (and in cases where a rule begins with a literal, I add the `LocMarker` subrule to the beginning so that I have something to apply the identifier to). This was the simplest way that I found for getting the correct start offset of a matched rule that contains repeating subrules. Section 5.1 in the report I wrote about the project discusses the necessary modifications on a more general level: <http://hasseg.org/peg-markdown-highlight/#paper>> Is it possible to create a script to modify his file to become your file?It would be a useful thing to have for the reasons you provide, but I doubt it would be feasible. The effort of writing + maintaining such a script and manually fixing all the corner cases where it breaks would probably be significantly higher than the effort of manually applying changes from peg-markdown.> This would be fantastic! It would be ridiculously easy to create a very simple app that did MD/MMD syntax highlighting and then processing to HTML or LaTeX or ODF(FODT) - at least on the Mac side.That has been the plan. :) Right now I am not aware of any differences between peg-markdown's and this highlighter's input matching behavior (although I'm sure some exist) so if the app uses peg-markdown for output generation, then the syntax highlighting and the output should match.