Markdown Discuss - Mar 2008 - on the philosophical aspects of a specification

a specification will _eventually_ be used, by someone,
to tell the user they are doing things "wrong", won't it?

and doesn't that turn markdown's genesis upside-down?
heck, next thing you know we'll be telling them to r.t.f.m.

i would prefer that implementers get more sophisticated
about teasing out the user's intent in "ambiguous" cases.

of course, i've always been suspicious of "specifications",
since many technoids think the job is finished once the
spec is "all done", before one line has been programmed;
give me _working_code_ any day of the week, thank you...

just a remark, intended to stir thought, _not_ a debate...
(please do reply if you must; i'll refrain from responding.)

-bowerbird



**************
Ideas to please picky eaters. Watch video on AOL Living.
      
(http://living.aol.com/video/how-to-please-your-picky-eater/rachel-campos-duffy/
2050827?NCID=aolcmp00300000002598)
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://six.pairlist.net/pipermail/markdown-discuss/attachments/20080303/5b0f4...>

* Bowerbird at aol.com <Bowerbird at aol.com> [2008-03-03
07:00]:
> i would prefer that implementers get more sophisticated
> about teasing out the user's intent in "ambiguous" cases.
If every implementor teases a different intent out the same
document, the user loses.

Is it possible for everyone to agree in all cases about how the
user?s intent should be teased out? Clearly it is conceivable
that enough effort could be made to write all agreements down.

And if you write down what intent should be teased out of
particular inputs, what have you created but a spec? And if the
effort has been made to agree on all possible cases, would this
spec not be unambiguous? And is it not yet obviously the case
that such a spec would not need to be inflexible about the syntax
it admits?

Why then does the fallacious argument that a spec would represent
a loss for the user continue?

Regards,
-- 
Aristotle Pagaltzis // <http://plasmasturm.org/>

Le 2008-03-03 ? 6:36, Aristotle Pagaltzis a ?crit :
> Why then does the fallacious argument that a spec would represent
> a loss for the user continue?
Some people have proposed before that some constructs may not be  
outlawed, that syntax rules should be changed, that Markdown is either  
too clever or not clever enough. People have been pushing a spec for a  
lot of reasons, and thus the consequences of writing a spec have  
become much more confusing.

Here's my take on it:

I don't think Markdown should be changed, we just need a spec to avoid  
each implementation from implementing "cleverness" in incompatible  
manners. Let's specify exactly how things *are* parsed now, minus the  
bugs.

(Note that by how things *are* parsed now, I'm not talking about  
specifying in english how Markdown.pl works; I'm talking about having  
the same result from a given input. Just making sure I'm not adding  
more unneeded confusion to that part of the discussion.)


Michel Fortin
michel.fortin at michelf.com
http://michelf.com/

>  I don't think Markdown should be changed, we just need a spec to avoid
>  each implementation from implementing "cleverness" in
incompatible
>  manners. Let's specify exactly how things *are* parsed now, minus the
>  bugs.
I agree.  Let's try to avoid any serious changes.  To the extent that
the implementations and "Markdown Syntax" agree, let's just go
with
that.  In cases where they don't, let's decide what makes most sense.
Needless to say, working out a spec means that at least some of the
implementations would need to be changed, and would no longer generate
the same output for the same input.  I just think that when resolving
the differences we should consider actual usage and the "spirit" of
the original syntax document, rather than the letter of "Markdown
Syntax".

In particular, I would consider going with the PHP Markdown Extra
solution over Markdown.pl in a few cases (at least for underscores in
the middle of the word, which is my pet peeve).

- yuri

-- 
http://sputnik.freewisdom.org/

Le 2008-03-03 ? 0:55, Bowerbird at aol.com a ?crit :
> a specification will _eventually_ be used, by someone,
> to tell the user they are doing things "wrong", won't it?
Well, that's not the specification I intend to do. All inputs are  
right in Markdown; there is no such thing as "invalid" or "non- 
conformant" Markdown, and that's how I intend to specify it.
> and doesn't that turn markdown's genesis upside-down?
> heck, next thing you know we'll be telling them to r.t.f.m.
The point of Markdown is that you can learn a good part of it just be  
looking at example and extrapolating from that. Reading a manual isn't  
a requirement, and it should stay that way. That doesn't mean there  
shouldn't be a manual nonetheless; when it doesn't do what you expect,  
you should have a reference to dig into, and some people like to read  
manuals.
> i would prefer that implementers get more sophisticated
> about teasing out the user's intent in "ambiguous" cases.
The problem is that in many cases, the author's intent is ambiguous  
even for a human. Then there are cases which can only the only way to  
understand what the author's intent is to actually read the text and  
use the formatting in a way that makes sense.

Markdown can't do anything about these two cases, except for one  
thing: be predictable by having syntax rules simple enough so that  
authors understand what happens and providing an obvious way out so  
that authors aren't stuck with nothing to do.
> of course, i've always been suspicious of "specifications",
> since many technoids think the job is finished once the
> spec is "all done", before one line has been programmed;
> give me _working_code_ any day of the week, thank you...
I intend to keep the specification evolving, and it obviously  
shouldn't be called stable until we have actual implementations of it  
proving that it works good enough. In fact, I plan to start with a  
specification very close, if not identical, to what PHP Markdown Extra  
does, except for the few things I'm considering bugs.
> just a remark, intended to stir thought, _not_ a debate...
> (please do reply if you must; i'll refrain from responding.)
Your concerns are valid ones, which seems to be shared by some other  
people too. I wanted to debunk them. Please don't refrain from writing  
the list if you have others.


Michel Fortin
michel.fortin at michelf.com
http://michelf.com/

well, i had said i wouldn't reply on this, but i think that
this post will still manage to be productive.   i hope so...

***

aristotle said:
>    If every implementor teases a different intent 
>    out the same document, the user loses.
well, yes.   but that's pretty obvious, isn't it?

there needs to be agreement among implementers.
i would think that goes without saying.

however, implementers can reach agreement easily,
by leaving users out in the cold, brushing them off
with a "you will need to follow the spec" which seems
-- if i understand markdown's cornerstone correctly --
to be outside gruber's comfort range for his creation...

nobody benefits from _true_ ambiguity.   kill that off...

but if you cannot be clever enough to allow "corner" cases
to be resolved _however_different_users_intended_them_
-- especially if those different intentions could be _easily_
understood by a naive person viewing them in plain text --
you need to go back to the drafting table and do more work.

in other words, the same input must create the same output.

but if people express different output using _similar_ input,
then you need to work to find a way to differentiate the input,
and not simply think that you can tell one of them to change...

so before considering two things as "the same", work _hard_
to see if there's some way -- any way -- to differentiate 'em...

i mean, that's _my_ opinion, but my opinion counts much less
than those of the implementers, and far less than gruber's...

the thing is, i like the _idea_ of markdown, which should not
be surprising, since i'd been working on my own non-markup
form of markup (for project gutenberg's e-texts in particular)
before markdown was invented.   since i couldn't "write a spec"
-- as my target documents are already widely propagated --
i had to _get_clever_ to tease out the different interpretations.
not only did it lead to code that's stronger _and_ more flexible, 
but i found the challenge to be stimulating _and_ productive...

>    Why then does the fallacious argument that 
>    a spec would represent a loss for the user continue?
aren't you loading the dice by labeling it as "fallacious"?

i'm not necessarily arguing that _any_ spec would do that.
but some might...   most especially by some implementers...
and according to markdown, users can't be wrong, can they?

so far, i've agreed with every aspect of michel's take on this...

-bowerbird



**************
It's Tax Time! Get tips, forms, and advice on AOL Money & 
Finance.
      (http://money.aol.com/tax?NCID=aolprf00030000000001)
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://six.pairlist.net/pipermail/markdown-discuss/attachments/20080303/ca70f...>

In article <d5d.1e15ea7e.34fdd932 at aol.com>,
 <markdown-discuss at six.pairlist.net> wrote:
>however, implementers can reach agreement easily,
>by leaving users out in the cold, brushing them off
>with a "you will need to follow the spec" which seems
>-- if i understand markdown's cornerstone correctly --
>to be outside gruber's comfort range for his creation...

    I've looked at this paragraph several times and I still have no idea
what you're talking about.

    If a user says "I want paragraphs to start with an explicit
paragraph symbol and all newlines to force a <br/>" , I *will* brush
them off with a "you will need to follow the spec" because that's
not
how Markdown works.    I can't imagine any other way to actually write
the language.


    -david parsons

david parsons wrote:
> In article <d5d.1e15ea7e.34fdd932 at aol.com>,
>  <markdown-discuss at six.pairlist.net> wrote:
>
>   
>> however, implementers can reach agreement easily,
>> by leaving users out in the cold, brushing them off
>> with a "you will need to follow the spec" which seems
>> -- if i understand markdown's cornerstone correctly --
>> to be outside gruber's comfort range for his creation...
>>     
>
>
>     I've looked at this paragraph several times and I still have no
idea
> what you're talking about.
>
>     If a user says "I want paragraphs to start with an explicit
> paragraph symbol and all newlines to force a <br/>" , I *will*
brush
> them off with a "you will need to follow the spec" because
that's not
> how Markdown works.    I can't imagine any other way to actually write
> the language.
>
>
>     -david parsons
> _______________________________________________
> Markdown-Discuss mailing list
> Markdown-Discuss at six.pairlist.net
> http://six.pairlist.net/mailman/listinfo/markdown-discuss
>
>
>   
I think what is trying to be said here is that in creating the spec you 
can't lose  the original focus of what Markdown is all about. Users 
(such as myself) don't really care that much about how the html is 
generated (breaks and explicit paragraphing are the domain of the 
parser). What we care about is that the original intent of our written 
source is maintained. It is very easy when creating a formal spec to 
lose track of the original intent and thus the usefulness of the tool. 
If I need to track exactly how many spaces I am allowed to use at the 
beginning of the line for certain implied formatting (like lists) then I 
am losing focus from the content I am writing, which is the exact 
opposite of what Markdown was created for.

If Markdown ends up diverging by creating too many rigid rules then 
users such as myself will just end up finding another tool. We want to 
type the content and let the tool create the form based on spacing and 
subtle signals in the content (such as *emphasis* etc). In the end it is 
the syntax that should be the defining spec because that is what the 
users understand and that is what determines the functionality of 
Markdown. Any formal grammar needs to derive from the syntax. My $.02 
CDN ($.02014US :)

In article <47CD5989.2020006 at idirect.ca>,
Seumas Mac Uilleachan  <markdown-discuss at six.pairlist.net>
wrote:
>david parsons wrote:
>> In article <d5d.1e15ea7e.34fdd932 at aol.com>,
>>  <markdown-discuss at six.pairlist.net> wrote:
>>> however, implementers can reach agreement easily,
>>> by leaving users out in the cold, brushing them off
>>> with a "you will need to follow the spec" which seems
>>> -- if i understand markdown's cornerstone correctly --
>>> to be outside gruber's comfort range for his creation...
>>     If a user says "I want paragraphs to start with an explicit
>> paragraph symbol and all newlines to force a <br/>" , I
*will* brush
>> them off with a "you will need to follow the spec" because
that's not
>> how Markdown works.    I can't imagine any other way to actually
write
>> the language.
>[...] What we care about is that the original intent of our written 
>source is maintained.

   I'm not surprised when

1986.  What a great season.

   generates a list item, because the existing spec tells me that

``[...]a _number-period-space_ sequence at the beginning of a
line[...]''

   will trigger an ordered list.




   But what's the intent of ***hello*, sailor**   ?

   Should it produce 
    1. <strong><em>hello</em>, sailor</strong>
    2. <strong>*hello*, sailor</strong>
    3. *<strong>hello*, sailor</strong>
    4. ***hello<em>, sailor<strong>
    5. ***hello*, sailor**
6. <em><strong>hello</strong></em><strong>,
sailor</strong>
7. <em><strong>hello</em>, sailor</strong> (which
makes baby XML cry) ?

   How about **Hello, sailor ?

Is it <strong>Hello, sailor, **Hello, sailor, or
<em></em>Hello, sailor?

   And how about _________cut here_________ ?


   Formal specifications are written to avoid surprises in the
   implementations;    As a user (and there's no way I'd have written an
   implementation if I wasn't a user) of the language I'd like to avoid
   surprises when I go between the markdown documents on my website,
   posts on my weblog, or posts on someone else's wiki and/or weblog.

On 4 Mar 2008 10:15:10 -0800, david parsons <orc at pell.portland.or.us>
wrote:
>
>    I'm not surprised when
>
>  1986.  What a great season.
>
>    generates a list item, because the existing spec tells me that
>
>    ``[...]a _number-period-space_ sequence at the beginning of a
line[...]''
>
>    will trigger an ordered list.
>
I don't know why I hadn't thought of this before, but the period
should be escapable and in fact, it is. That, I believe is the answer
to this problem of a line that begins with a number-period-space
sequence.

    1986\. What a great season.

becomes:

    <p>1986. What a great season.</p>

I just checked markdown.pl, php & python and it works correctly in all
three.
>
>
>    But what's the intent of ***hello*, sailor**   ?
>
>    Should it produce
>     1. <strong><em>hello</em>, sailor</strong>
>     2. <strong>*hello*, sailor</strong>
>     3. *<strong>hello*, sailor</strong>
>     4. ***hello<em>, sailor<strong>
>     5. ***hello*, sailor**
>     6.
<em><strong>hello</strong></em><strong>,
sailor</strong>
>     7. <em><strong>hello</em>, sailor</strong>
(which makes baby XML cry) ?
>
>    How about **Hello, sailor ?
>
>    Is it <strong>Hello, sailor, **Hello, sailor, or
<em></em>Hello, sailor?
>
>    And how about _________cut here_________ ?
>
>
>    Formal specifications are written to avoid surprises in the
>    implementations;    As a user (and there's no way I'd have
written an
>    implementation if I wasn't a user) of the language I'd like to
avoid
>    surprises when I go between the markdown documents on my website,
>    posts on my weblog, or posts on someone else's wiki and/or weblog.
>
>
> _______________________________________________
>  Markdown-Discuss mailing list
>  Markdown-Discuss at six.pairlist.net
>  http://six.pairlist.net/mailman/listinfo/markdown-discuss
>


-- 
----
Waylan Limberg
waylan at gmail.com

david parsons wrote:
> In article <47CD5989.2020006 at idirect.ca>,
> Seumas Mac Uilleachan  <markdown-discuss at six.pairlist.net> wrote:
>   
>> david parsons wrote:
>>     
>>> In article <d5d.1e15ea7e.34fdd932 at aol.com>,
>>>  <markdown-discuss at six.pairlist.net> wrote:
>>>       
>
>   
>>>> however, implementers can reach agreement easily,
>>>> by leaving users out in the cold, brushing them off
>>>> with a "you will need to follow the spec" which seems
>>>> -- if i understand markdown's cornerstone correctly --
>>>> to be outside gruber's comfort range for his creation...
>>>>         
>
>   
>>>     If a user says "I want paragraphs to start with an
explicit
>>> paragraph symbol and all newlines to force a <br/>" , I
*will* brush
>>> them off with a "you will need to follow the spec"
because that's not
>>> how Markdown works.    I can't imagine any other way to
actually write
>>> the language.
>>>       
>
>   
>> [...] What we care about is that the original intent of our written 
>> source is maintained.
>>     
>
>
>    I'm not surprised when
>
> 1986.  What a great season.
>
>    generates a list item, because the existing spec tells me that
>
>    ``[...]a _number-period-space_ sequence at the beginning of a
line[...]''
>
>    will trigger an ordered list.
>
>
>
>
>    But what's the intent of ***hello*, sailor**   ?
>   
The stated spec says text wrapped in one * is emphasis, two ** is strong.
>    Should it produce 
>     1. <strong><em>hello</em>, sailor</strong>
>     2. <strong>*hello*, sailor</strong>
>     3. *<strong>hello*, sailor</strong>
>     4. ***hello<em>, sailor<strong>
>     5. ***hello*, sailor**
>     6.
<em><strong>hello</strong></em><strong>,
sailor</strong>
>     7. <em><strong>hello</em>, sailor</strong>
(which makes baby XML cry) ?
>   
Item 4 only makes sense if the source is taken out of context (a 
"snippet"), and then how do we know? Items 2-5 all assume some *'s
are
literal which doesn't follow the spec if the text is wrapped. Items 1,6, 
and 7 all render the same in the browser and the choice of which is up 
to the implementation (following the syntax spec).
>    How about **Hello, sailor ?
>   
Is this a snippet? if not, then the text is not wrapped and the syntax 
does not apply.
>    Is it <strong>Hello, sailor, **Hello, sailor, or
<em></em>Hello, sailor?
>
>    And how about _________cut here_________ ?
>   
This is a problem. Anything more than 4 _ per side does not render but 
with 4 it does (in PHP) if you have ____cut here____ are you expecting 
it to be converted to <em><strong><em>cut
here</em></strong></em> ?
>
>    Formal specifications are written to avoid surprises in the
>    implementations;    As a user (and there's no way I'd have
written an
>    implementation if I wasn't a user) of the language I'd like to
avoid
>    surprises when I go between the markdown documents on my website,
>    posts on my weblog, or posts on someone else's wiki and/or weblog.
>   
I did not say that a formal spec was a bad idea. A formal **syntax** 
specification that is clear and unambiguous is where we need to start, 
without losing sight of the great usability that Markdown has right now. 
The biggest problem is that the syntax is ambiguous. The biggest 
strength is that the syntax is flexible (0,1,2 or three spaces at the 
start of a list).

And speaking of ambiguous

  * List
 * List
  * List
   * List
  *  List
 *   List
  *  List
   * List

converts to:

<ul>
<li>List

<ul>
<li>List</li>
</ul></li>
<li>List

<ul>
<li>List</li>

</ul></li>
<li>List

<ul>
<li>List</li>
</ul></li>
<li>List

<ul>
<li>List</li>
</ul></li>

</ul>

which shows up as:


    * List
          o List
    * List
          o List
    * List
          o List
    * List
          o List

What was the intent here? I would suggest more like

    * List
    * List
    * List
          o List
    * List
    * List
    * List 
          o List

Since only the 4th and 8th are indented 4 spaces.

Le 2008-03-04 ? 21:47, Seumas Mac Uilleachan a ?crit :
> david parsons wrote:
>
>>   And how about _________cut here_________ ?
>>
> This is a problem. Anything more than 4 _ per side does not render  
> but with 4 it does (in PHP) if you have ____cut here____ are you  
> expecting it to be converted to <em><strong><em>cut
here</em></
> strong></em> ?
Well, that's already much better than what you get from  
Markdown.pl. :-) But I agree it could still benefit from some  
improvement.
> [...]
>
> And speaking of ambiguous
>
> * List
> * List
> * List
>  * List
> *  List
> *   List
> *  List
>  * List
Yeah, the list implementation in Markdown.pl and PHP Markdown doesn't  
follow the at all the little of a spec we have now. I've been thinking  
about rewriting the list parser in PHP Markdown, but I'm wondering  
what to do to not suddenly change a myriad of documents which may  
depends on some part of this behaviour, such as:

     * item
       * subitem
       * subitem
     * item
       * subitem

(Here, no item is indented by four space, should this be a flat list?)

I know people have written lists like the above in their document.  
They did it because it produce what they expect in their Markdown  
implementation, because the thing is readable and make sense, and  
because didn't bother to read the spec.
> What was the intent here? I would suggest more like
>
>   * List
>   * List
>   * List
>         o List
>   * List
>   * List
>   * List          o List
>
> Since only the 4th and 8th are indented 4 spaces.
Eh, I don't see a four space indent anywhere in your example. But,  
assuming your output got screwed up while editing and that the last  
list element belongs on a separate line, I agree with you that it's  
probably the best possible output to represent the author's intent.


Michel Fortin
michel.fortin at michelf.com
http://michelf.com/

Michel Fortin wrote:
> Le 2008-03-04 ? 21:47, Seumas Mac Uilleachan a ?crit :
>
>> david parsons wrote:
>>
>>>   And how about _________cut here_________ ?
>>>
>> This is a problem. Anything more than 4 _ per side does not render 
>> but with 4 it does (in PHP) if you have ____cut here____ are you 
>> expecting it to be converted to <em><strong><em>cut 
>> here</em></strong></em> ?
>
> Well, that's already much better than what you get from Markdown.pl. 
> :-) But I agree it could still benefit from some improvement.
>
>> [...]
>>
>> And speaking of ambiguous
>>
>> * List
>> * List
>> * List
>>  * List
>> *  List
>> *   List
>> *  List
>>  * List
>
> Yeah, the list implementation in Markdown.pl and PHP Markdown doesn't 
> follow the at all the little of a spec we have now. I've been thinking 
> about rewriting the list parser in PHP Markdown, but I'm wondering 
> what to do to not suddenly change a myriad of documents which may 
> depends on some part of this behaviour, such as:
>
>     * item
>       * subitem
>       * subitem
>     * item
>       * subitem
>
> (Here, no item is indented by four space, should this be a flat list?)
We have to decide what the intent is - if indents are two spaces or more 
that indicates sublists, where one space indicates a mistake in typing? 
What if the mistake results in your sublist item becoming an upper list 
item (space one instead of two)? Lists in general need to be more 
precisely defined. I for one would like to see the ability for arbitrary 
starting points in numbered lists added. Then again, how many existing 
documents will that break?
>
> I know people have written lists like the above in their document. 
> They did it because it produce what they expect in their Markdown 
> implementation, because the thing is readable and make sense, and 
> because didn't bother to read the spec.
>
>> What was the intent here? I would suggest more like
>>
>>   * List
>>   * List
>>   * List
>>         o List
>>   * List
>>   * List
>>   * List          o List
>>
>> Since only the 4th and 8th are indented 4 spaces.
>
> Eh, I don't see a four space indent anywhere in your example. But, 
> assuming your output got screwed up while editing and that the last 
> list element belongs on a separate line, I agree with you that it's 
> probably the best possible output to represent the author's intent.
>
Um, looks like first space got chopped off all the lines. Indents were 
3,2,3,4,3,2,3,4. I was trying to show what happens if you aren't exactly 
precise on your spacing. I personally always start my lists with zero 
spaces when typing but when cutting and pasting lists I can end up with 
1,2,3, as much as six spaces in front (obviously with six I do need to 
revise but otherwise I just leave as is). If someone habitually uses two 
but at one point mistakenly types one or three instead - the intent was 
two but how should Markdown handle that? Currently (which is strange) 
the first line in my example has three spaces and is a first level list. 
The next line has only two but becomes a **second level**
list.
>
> Michel Fortin
> michel.fortin at michelf.com
> http://michelf.com/
>
>
> _______________________________________________
> Markdown-Discuss mailing list
> Markdown-Discuss at six.pairlist.net
> http://six.pairlist.net/mailman/listinfo/markdown-discuss
>
>

In article <47CEBB51.6010003 at idirect.ca>,
Seumas Mac Uilleachan  <markdown-discuss at six.pairlist.net>
wrote:
> Currently (which is strange) 
>the first line in my example has three spaces and is a first level list. 
>The next line has only two but becomes a **second level** list.
   I believe that the markdown you're using is being overly aggressive
when doing the prefix matching for "next item in list", and
it's only
   matching if the indent is exactly the indent of the current list item.
   And 2 != 3.    That's *really* bizarre.

   On the bright side, that sort of musical indentation is pretty close
   to unreadable, so the bizarre reformatting of the list is a good way
   to encourage cleanup of the source text.


   -david parsons

On Mar 4, 2008, at 11:09 PM, Michel Fortin wrote:
> Yeah, the list implementation in Markdown.pl and PHP Markdown  
> doesn't follow the at all the little of a spec we have now. I've  
> been thinking about rewriting the list parser in PHP Markdown, but  
> I'm wondering what to do to not suddenly change a myriad of  
> documents which may depends on some part of this behaviour, such as:
>
>    * item
>      * subitem
>      * subitem
>    * item
>      * subitem
>
> (Here, no item is indented by four space, should this be a flat list?)
>
> I know people have written lists like the above in their document.  
> They did it because it produce what they expect in their Markdown  
> implementation, because the thing is readable and make sense, and  
> because didn't bother to read the spec.
A list item's parent is the most recent list item whose bullet is  
indented less than its own.  If there's no such parent, then the item  
belongs to a root-level list.

http://six.pairlist.net/pipermail/markdown-discuss/2008-March/001076.html

Is there any case where this doesn't do the right thing?

> A list item's parent is the most recent list item whose bullet is
indented
> less than its own.  If there's no such parent, then the item belongs to
a
> root-level list.
>
> http://six.pairlist.net/pipermail/markdown-discuss/2008-March/001076.html
>
> Is there any case where this doesn't do the right thing?
I think this is a good proposal.  The only case I can think of where
it gives intuitively wrong results is this (which involves roman
numerals, not currently supported in core Markdown, though they are in
Pandoc):

   i. one
  ii. two
 iii. three
  iv. four
   v. five

But, on balance, I think this proposal is the best I've seen --
certainly better than expecting users to know the four-space indent
rule.

John

On Wed, Mar 5, 2008 at 10:53 AM, John Fraser <john at attacklab.net>
wrote:
>  A list item's parent is the most recent list item whose bullet is
>  indented less than its own.  If there's no such parent, then the item
>  belongs to a root-level list.
>
>  http://six.pairlist.net/pipermail/markdown-discuss/2008-March/001076.html
>
>  Is there any case where this doesn't do the right thing?
>
Well, there is the issue of code blocks nested inside list items.
Although, I suppose the parser could increment the list level for each
increment in indentation (up to 4 spaces so as not to eat indentation
on nested code blocks). Then document authors would need to be warned
that if they want nested code blocks in their lists, it is
*recommended* that they only indent their lists by 4 space increments
so as not to confuse the parser. However, if you are not nesting code
blocks, any amount of indentation up to four spaces is fine and each
increase in indentation (even of one space) will result in another
level of nesting.

My only other concern is when stepping back out of the nesting.
Suppose we have the following list:

* no spaces - level 1
    * 4 spaces - level 2
      * 6 spaces - level 3
  * 2 spaces - level 1.5 ???

Obviously, that would break. But what's the best way to handle that? I
do *not* think backtracking through the list and reorganizing the list
levels is a reasonable option. Perhaps, that last line should be root
of a *new* list. What do you think?



-- 
----
Waylan Limberg
waylan at gmail.com

>  My only other concern is when stepping back out of the nesting.
>  Suppose we have the following list:
>
>  * no spaces - level 1
>     * 4 spaces - level 2
>       * 6 spaces - level 3
>   * 2 spaces - level 1.5 ???
>
>  Obviously, that would break. But what's the best way to handle that? I
>  do *not* think backtracking through the list and reorganizing the list
>  levels is a reasonable option. Perhaps, that last line should be root
>  of a *new* list. What do you think?
>
With the rule just proposed, wouldn't the last line simply be level 2?
I think this rule has the bonus of being obvious. If it doesn't do
what someone expects, they can look at what they wrote and say "oh, as
long as I indent more than the previous level, it will make a
sub-list." (of course, that could just be obvious since we're talking
about it).

-V

On Wed, Mar 5, 2008 at 1:46 PM, Vinay Augustine <vinay at case.edu>
wrote:
> >  My only other concern is when stepping back out of the nesting.
>  >  Suppose we have the following list:
>  >
>  >  * no spaces - level 1
>  >     * 4 spaces - level 2
>  >       * 6 spaces - level 3
>  >   * 2 spaces - level 1.5 ???
>  >
>  >  Obviously, that would break. But what's the best way to handle
that? I
>  >  do *not* think backtracking through the list and reorganizing the
list
>  >  levels is a reasonable option. Perhaps, that last line should be
root
>  >  of a *new* list. What do you think?
>  >
>
>  With the rule just proposed, wouldn't the last line simply be level 2?
>  I think this rule has the bonus of being obvious. If it doesn't do
>  what someone expects, they can look at what they wrote and say "oh,
as
>  long as I indent more than the previous level, it will make a
>  sub-list." (of course, that could just be obvious since we're
talking
>  about it).
>
To me that is not at all obvious. If it were to work as you propose,
then the spec needs to specifically indicate that that is the expected
behavior.

Perhaps the reason this is *not* obvious to me is that I have a
relatively strong background coding python. In python code, whitespace
is significant. In fact, with few exceptions indentation is the only
way to indicate nesting. The above nesting would **break** at compile
time in python code. This is a "feature" and considered a "good
thing"
by most python users.

I also realize that one of many people's beefs with python is the
significance of whitespace. So I'm not going to suggest that markdown
adopt python's whitespace rules (although personally I'd love it).
However, I bring this up because I doubt I'm the only one that
wouldn't see what *should* happen (besides choking) in this instance.


-- 
----
Waylan Limberg
waylan at gmail.com

On Mar 5, 2008, at 2:40 PM, Waylan Limberg wrote:
> On Wed, Mar 5, 2008 at 1:46 PM, Vinay Augustine <vinay at case.edu>  
> wrote:
>>
>>> * no spaces - level 1
>>>    * 4 spaces - level 2
>>>      * 6 spaces - level 3
>>>  * 2 spaces - level 1.5 ???
>>
>> With the rule just proposed, wouldn't the last line simply be level
>> 2?
>> I think this rule has the bonus of being obvious. If it doesn't do
>> what someone expects, they can look at what they wrote and say
"oh,
>> as
>> long as I indent more than the previous level, it will make a
>> sub-list." (of course, that could just be obvious since we're
talking
>> about it).
>>
>
> To me that is not at all obvious. If it were to work as you propose,
> then the spec needs to specifically indicate that that is the expected
> behavior.
>
> Perhaps the reason this is *not* obvious to me is that I have a
> relatively strong background coding python. In python code, whitespace
> is significant. In fact, with few exceptions indentation is the only
> way to indicate nesting. The above nesting would **break** at compile
> time in python code. This is a "feature" and considered a
"good thing"
> by most python users.
Maybe down the road it'll be worth building something like a  Markdown- 
lint, so we can warn users when they do wacky stuff like this.  But it  
absolutely shouldn't be part of Markdown itself; we just need to do  
the best we can with the input we've got.  And personally, I think  
having that last item be level 2 is pretty reasonable.  That's how I'd  
read it.

Waylan Limberg wrote:
> {snip}
>
> My only other concern is when stepping back out of the nesting.
> Suppose we have the following list:
>
> * no spaces - level 1
>     * 4 spaces - level 2
>       * 6 spaces - level 3
>   * 2 spaces - level 1.5 ???
>
> Obviously, that would break. But what's the best way to handle that? I
> do *not* think backtracking through the list and reorganizing the list
> levels is a reasonable option. Perhaps, that last line should be root
> of a *new* list. What do you think?
>
>
>   
Looking at the list as a whole, could the intent have been

* no spaces - level 1
    * 4 spaces - level 3
      * 6 spaces - level 4
  * 2 spaces - level 2

not that this makes much sense to me offhand (unless a level 2 was deleted)

In article <1F9AE98D-CD1E-424D-8B4C-584055CBFC90 at attacklab.net>,
John Fraser  <markdown-discuss at six.pairlist.net>
wrote:
>
>A list item's parent is the most recent list item whose bullet is  
>indented less than its own.  If there's no such parent, then the item  
>belongs to a root-level list.
>
>http://six.pairlist.net/pipermail/markdown-discuss/2008-March/001076.html
>
>Is there any case where this doesn't do the right thing?
   When I write a really long list,

    *  sometimes, after a particularly long and
       detailed list item, I'll lose track of the
       exact indentation and 
     * add one too many spaces to the leading
       indent.

    so it would be bad if that broke nesting.


    -david parsons

On Mar 5, 2008, at 2:38 PM, david parsons wrote:
>   When I write a really long list,
>
>    *  sometimes, after a particularly long and
>       detailed list item, I'll lose track of the
>       exact indentation and
>     * add one too many spaces to the leading
>       indent.
>
>    so it would be bad if that broke nesting.
Good point.  But Markdown's indentation rules are a mess, and I don't  
think there's any way to fix them without breaking some examples like  
this.  Hell, I don't see how we can achieve a standard unless we're  
willing to break a few edges cases in the various implementations.

Again, tools might help here: I'd love it if we could build conversion  
utilities that run, say, Markdown.pl's algorithm over some input,  
produce an abstract syntax tree, and then output that in the new  
version of Markdown.  Pie in the sky, but worth considering.

In article <DA0F6FC9-95D9-47C6-A970-6ACC0A6FD91B at attacklab.net>,
John Fraser  <markdown-discuss at six.pairlist.net>
wrote:
>
>On Mar 5, 2008, at 2:38 PM, david parsons wrote:
>>   When I write a really long list,
>>
>>    *  sometimes, after a particularly long and
>>       detailed list item, I'll lose track of the
>>       exact indentation and
>>     * add one too many spaces to the leading
>>       indent.
>>
>>    so it would be bad if that broke nesting.
>
>Good point.  But Markdown's indentation rules are a mess,
    They are?   I find some of the blocking rules somewhat infuriating
    (that paragraphs are greedier at root level than they are inside
    a list or quote) but a 4-space indent seems quite natural to me.

    The attempts to reform list indentation by restricting the definition
    of a list item seem like they'd cause more of a mess than they would
    otherwise.    I use lists all over the place, and I don't really want
    to have to go back in and correct them all because a v2 spec corrected
    markdown to the point where previously correct markup became incorrect.
    (I get to deal with that all the time with gcc and their flexible 
    interpretation of the C programming language; I don't want to do it
    with things I do for fun, not money.)


    -david parsons

On Mar 5, 2008, at 2:38 PM, david parsons wrote:
>   When I write a really long list,
>
>    *  sometimes, after a particularly long and
>       detailed list item, I'll lose track of the
>       exact indentation and
>     * add one too many spaces to the leading
>       indent.
>
>    so it would be bad if that broke nesting.
It'd be nice to get near-miss list indentation right.

BUT . . . if I make this mistake and Markdown mis-nests, the mistake  
will be obvious when I look over the output.  What's more, it'll be  
obvious how to fix it.

One of the advantages of Markdown syntax is that when something weird  
happens, it's usually very easy to spot and to debug.  I'd rather have  
clear and intuitive syntax that produces predictable outputs than get  
all of the near-misses and edge cases right.  It's good to be  
forgiving of user goofs, but it's also good to provide good implicit  
feedback on how to clean them up.  Enforcing a rule that items at the  
same level of intended indentation should start with the same number  
of spaces seems like a good case for being rigid, because a user who  
messes it up (as I've often done) can easily spot the problem and  
recover gracefully.

James

Le 2008-03-04 ? 13:15, david parsons a ?crit :
>   But what's the intent of ***hello*, sailor**   ?
>
>   Should it produce
>    1. <strong><em>hello</em>, sailor</strong>
>    2. <strong>*hello*, sailor</strong>
>    3. *<strong>hello*, sailor</strong>
>    4. ***hello<em>, sailor<strong>
>    5. ***hello*, sailor**
>    6.
<em><strong>hello</strong></em><strong>,
sailor</strong>
>    7. <em><strong>hello</em>, sailor</strong>
(which makes baby XML
> cry) ?
I'd say 1:

     <strong><em>hello</em>, sailor</strong>

This is the saner well-formed markup which can reflect the intent.  
This is what PHP Markdown is designed to produce, and it's part of the  
test suite in MDTest. 6 is probably acceptable too.

A better question is what to do with this:

     *hello **dear* boy**
>   How about **Hello, sailor ?
>
>   Is it <strong>Hello, sailor, **Hello, sailor, or
<em></em>Hello,
> sailor?
Empty emphasis doesn't make sense, it shouldn't be allowed. The first  
interpretation is correct.


Michel Fortin
michel.fortin at michelf.com
http://michelf.com/

On Tue, Mar 4, 2008 at 10:08 PM, Michel Fortin
<michel.fortin at michelf.com> wrote:
> Le 2008-03-04 ? 13:15, david parsons a ?crit :
>
>
>  >   But what's the intent of ***hello*, sailor**   ?
>  >
>  >   Should it produce
>  >    1. <strong><em>hello</em>, sailor</strong>
>  >    2. <strong>*hello*, sailor</strong>
>  >    3. *<strong>hello*, sailor</strong>
>  >    4. ***hello<em>, sailor<strong>
>  >    5. ***hello*, sailor**
>  >    6.
<em><strong>hello</strong></em><strong>,
sailor</strong>
>  >    7. <em><strong>hello</em>, sailor</strong>
(which makes baby XML
>  > cry) ?
>
>  I'd say 1:
<snip>
>
>  A better question is what to do with this:
>
>      *hello **dear* boy**
For cases like these, I'd say that Markdown shouldn't do
anything.
>From the official Markdown page:
> The overriding design goal for Markdown's formatting syntax is to make
> it as readable as possible. The idea is that a Markdown-formatted
> document should be publishable as-is, as plain text, without looking
> like it's been marked up with tags or formatting instructions. While
> Markdown's syntax has been influenced by several existing text-to-HTML
> filters, the single biggest source of inspiration for Markdown's syntax
> is the format of plain text email.
So, the question is: Would you ever see **mismatched *asterisks***
intentionally written in plain text email?

I don't think so, because it doesn't make intuitive sense. And if I
can't make sense of the plain text, why should Markdown define one
interpretation as being correct?

Steve

Steve Hoelzer wrote:
> On Tue, Mar 4, 2008 at 10:08 PM, Michel Fortin
> <michel.fortin at michelf.com> wrote:
>   
>> Le 2008-03-04 ? 13:15, david parsons a ?crit :
>>
>>
>>  >   But what's the intent of ***hello*, sailor**   ?
>>  >
>>  >   Should it produce
>>  >    1. <strong><em>hello</em>,
sailor</strong>
>>  >    2. <strong>*hello*, sailor</strong>
>>  >    3. *<strong>hello*, sailor</strong>
>>  >    4. ***hello<em>, sailor<strong>
>>  >    5. ***hello*, sailor**
>>  >    6.
<em><strong>hello</strong></em><strong>,
sailor</strong>
>>  >    7. <em><strong>hello</em>,
sailor</strong> (which makes baby XML
>>  > cry) ?
>>
>>  I'd say 1:
>>     
>
> <snip>
>
>   
>>  A better question is what to do with this:
>>
>>      *hello **dear* boy**
>>     
>
> For cases like these, I'd say that Markdown shouldn't do anything.
> >From the official Markdown page:
>
>   
>> The overriding design goal for Markdown's formatting syntax is to
make
>> it as readable as possible. The idea is that a Markdown-formatted
>> document should be publishable as-is, as plain text, without looking
>> like it's been marked up with tags or formatting instructions.
While
>> Markdown's syntax has been influenced by several existing
text-to-HTML
>> filters, the single biggest source of inspiration for Markdown's
syntax
>> is the format of plain text email.
>>     
>
> So, the question is: Would you ever see **mismatched *asterisks***
> intentionally written in plain text email?
>
> I don't think so, because it doesn't make intuitive sense. And if I
> can't make sense of the plain text, why should Markdown define one
> interpretation as being correct?
>
> Steve
> _______________________________________________
> Markdown-Discuss mailing list
> Markdown-Discuss at six.pairlist.net
> http://six.pairlist.net/mailman/listinfo/markdown-discuss
>
>
>   
Perhaps if strong and emphasis is desired in such a situation the spec 
should state to use * and __ or _ and ** to better distinguish between 
them. So *** would simply be *** unless balanced with a *** on the other 
side of the wrapped text. The sample could thus be *hello __dear* boy__

In article <588192970803050528t7ff808f7kc2244d0f214770fb at
mail.gmail.com>,
Steve Hoelzer <markdown-discuss at six.pairlist.net>
wrote:
>On Tue, Mar 4, 2008 at 10:08 PM, Michel Fortin
><michel.fortin at michelf.com> wrote:
>> Le 2008-03-04 ? 13:15, david parsons a ?crit :
>>  >   But what's the intent of ***hello*, sailor**   ?
>So, the question is: Would you ever see **mismatched *asterisks***
>intentionally written in plain text email?
>I don't think so, because it doesn't make intuitive sense. And if I
>can't make sense of the plain text, why should Markdown define one
>interpretation as being correct?
    And thus the argument that we can't more accurately define the
    existing language because it will make things unintuitive is not
    really a feasable one  (which is sort of the point that I was
    muddling towards before you wrote it much more clearly.)



    -david parsons

* Michel Fortin <michel.fortin at michelf.com> [2008-03-05
05:10]:
> A better question is what to do with this:
>
>     *hello **dear* boy**
That?s a very good question. Here?s a counterquestion: what does
a human reader see in that text? Based on the visual apperance I
think I would make it translate to this:

    <em>hello <strong>dear</strong> boy</em>

Really.

Regards,
-- 
Aristotle Pagaltzis // <http://plasmasturm.org/>

Aristotle Pagaltzis wrote:
> * Michel Fortin <michel.fortin at michelf.com> [2008-03-05 05:10]:
>   
>> A better question is what to do with this:
>>
>>     *hello **dear* boy**
>>     
>
> That?s a very good question. Here?s a counterquestion: what does
> a human reader see in that text? Based on the visual apperance I
> think I would make it translate to this:
>
>     <em>hello <strong>dear</strong> boy</em>
>
> Really.
>
> Regards,
>   
See, now that's not at all what I inferred from this case. If we infer 
different meanings here then there are undoubtedly many cases we can't 
even think of that would be inferred differently. How is the spec 
supposed to handle all this? Admittedly we can just go back and make 
changes if the result doesn't match our expectations but which was the 
real error in that case? The result or the expectation?

BTW I inferred

<em>hello <strong>dear</em> boy</strong>

which maybe for strict XHTML should be

<em>hello <strong>dear</strong></em><strong>
boy</strong>

On Thu, Mar 6, 2008 at 9:39 AM, Seumas Mac Uilleachan <seumas at
idirect.ca> wrote:
> Aristotle Pagaltzis wrote:
>  > * Michel Fortin <michel.fortin at michelf.com> [2008-03-05
05:10]:
>  >
>  >> A better question is what to do with this:
>  >>
>  >>     *hello **dear* boy**
>  >>
>  >
>  > That's a very good question. Here's a counterquestion: what
does
>  > a human reader see in that text? Based on the visual apperance I
>  > think I would make it translate to this:
>  >
>  >     <em>hello <strong>dear</strong> boy</em>
Ah, so your assuming the parser should automatically close unclosed
tags much as a browser in quirks mode does. Sure, you and I understand
how that works, but should we expect authors who are unfamiliar with
html to get that? I doubt it. I also suspect that it's those same
authors that will most likely purposely write a document containing
text formatted like that. I agree with Seumas that such people would
expect:
>
>  <em>hello <strong>dear</em> boy</strong>
Yeah, we could give them output that displays as they expect and fix
it under the hood by doing:
>  <em>hello <strong>dear</strong></em><strong>
boy</strong>
>
But, the output **I** would expect is one of:

<em>hello </em><em>dear</em> boy**

<em>hello **dear</em> boy**

*hello <strong>dear* boy</strong>

Yeah, I think we should force authors to close any tags they open. If
they don't, then the text is assumed to be literal, not markup. Maybe
that's too restrictive for some peoples taste. But that's what I see
when I look at that text. In my mind I keep going back and forth
between the three and can never decide which the author intended.
Finally, I cringe as I realize they probably intended what Seumas
suggested.

If we want to throw valid markup to the wind, then sure, Seumans first
suggestion (and how markdown.pl currently works) is the answer.
Otherwise, any one of my suggestions could be the anwser. This tells
the author (who hopefully is previewing anyway) that they have an
error in their markup and need to make a change. With Aristotle's
suggested output, those unfamiliar with html will, IMO, not be able to
easily discern why the output doesn't match their expectations.
However, by leaving some of the markup literal, they have some clues
to work with.

To me, that is an important factor that seems to be ignored by some
here. Sometimes, IMO, the best thing to do is to pass the markup
through as literal text and give the author a clue that his formatting
is unclear!

-- 
----
Waylan Limberg
waylan at gmail.com

Waylan Limberg wrote:
> On Thu, Mar 6, 2008 at 9:39 AM, Seumas Mac Uilleachan <seumas at
idirect.ca> wrote:
>   
>> Aristotle Pagaltzis wrote:
>>  > * Michel Fortin <michel.fortin at michelf.com> [2008-03-05
05:10]:
>>  >
>>  >> A better question is what to do with this:
>>  >>
>>  >>     *hello **dear* boy**
>>  >>
>>  >
>>  > That's a very good question. Here's a counterquestion:
what does
>>  > a human reader see in that text? Based on the visual apperance I
>>  > think I would make it translate to this:
>>  >
>>  >     <em>hello <strong>dear</strong>
boy</em>
>>     
>
> Ah, so your assuming the parser should automatically close unclosed
> tags much as a browser in quirks mode does. Sure, you and I understand
> how that works, but should we expect authors who are unfamiliar with
> html to get that? I doubt it. I also suspect that it's those same
> authors that will most likely purposely write a document containing
> text formatted like that. I agree with Seumas that such people would
> expect:
>
>   
>>  <em>hello <strong>dear</em> boy</strong>
>>     
>
> Yeah, we could give them output that displays as they expect and fix
> it under the hood by doing:
>
>   
>>  <em>hello
<strong>dear</strong></em><strong> boy</strong>
>>
>>     
> But, the output **I** would expect is one of:
>
> <em>hello </em><em>dear</em> boy**
>
> <em>hello **dear</em> boy**
>
> *hello <strong>dear* boy</strong>
>
> Yeah, I think we should force authors to close any tags they open. If
> they don't, then the text is assumed to be literal, not markup. Maybe
> that's too restrictive for some peoples taste. But that's what I
see
> when I look at that text. In my mind I keep going back and forth
> between the three and can never decide which the author intended.
> Finally, I cringe as I realize they probably intended what Seumas
> suggested.
>
> If we want to throw valid markup to the wind, then sure, Seumans first
> suggestion (and how markdown.pl currently works) is the answer.
> Otherwise, any one of my suggestions could be the anwser. This tells
> the author (who hopefully is previewing anyway) that they have an
> error in their markup and need to make a change. With Aristotle's
> suggested output, those unfamiliar with html will, IMO, not be able to
> easily discern why the output doesn't match their expectations.
> However, by leaving some of the markup literal, they have some clues
> to work with.
>
> To me, that is an important factor that seems to be ignored by some
> here. Sometimes, IMO, the best thing to do is to pass the markup
> through as literal text and give the author a clue that his formatting
> is unclear!
>
>   
Many users (especially where markdown is being used as a plugin for a 
blogsite etc) will not even be aware of valid/invalid markup and closing 
tags properly or not. Turning *hello **dear* boy** into
<em>hello <strong>dear</strong> boy</em>
makes assumptions about the trailing asterisks and errors in formatting 
that are possibly different from the user's intent (like mixing up ** 
and * to close the tags), and implies an error-checking mechanism built 
into markdown to catch such cases.

Maybe what is needed is some kind of syntax checker to run the source 
through to point out to users where there are errors and/or confusing 
markup. This could be a separate function from markdown itself, like 
markdown-lint, or a separate output option of markdown. A separate 
function would keep the markdown parser smaller. A syntax checker would 
also help users identify what the problem is when they get unexpected 
results.

>  Maybe what is needed is some kind of syntax checker to run the source
>  through to point out to users where there are errors and/or confusing
>  markup. This could be a separate function from markdown itself, like
>  markdown-lint, or a separate output option of markdown. A separate
>  function would keep the markdown parser smaller. A syntax checker would
>  also help users identify what the problem is when they get unexpected
>  results.
Maruku, which uses a parser, does warn[^1] the user if something
strange is read (unclosed */**, not valid XML, unclosed ![...], etc.).
Some users wrote me that they really liked such feature because it
helped them to write proper documents.


[^1]: via stderr for command-line maruku; via a callback when maruku
is used as a library.

-- 
Andrea Censi
PhD student, Control & Dynamical Systems, Caltech
http://www.cds.caltech.edu/~andrea/
    "Life is too important to be taken seriously" (Oscar Wilde)

Hi Yuri, Weylan and Seumas,

* Yuri Takhteyev <qaramazov at gmail.com> [2008-03-07
08:50]:
> > >     *hello **dear* boy**
> >
> > That's a very good question. Here's a counterquestion: what
> > does a human reader see in that text?
> 
> When I try to look at this with my normal-person eye, what I
> see here is incorrect markup
Sorry, but if you see ?markup? (much less ?incorrect markup?)
you?re not looking at it with a normal-person eye. :-)
> So, the user will type in something like this and get
> "<em>hello **dear</em> boy**". Not much of a tradegy.
They will
> say, oh, silly me, must have screwed something up. (They did!)
> Then they'll go and fix it. I am all for flexibility, but not
> to the point of trying to divine the meaning of ambiguous or
> ill-formed markup.
Only a small minority will do that. Most people most of the time
don?t care enough about that particular piece of text to actually
fix any small nits in it, any more than they?ll care to fix all
of their small spelling and grammar mistakes. (Less, actually.)
That has certainly been my experience on wikis and weblogs that
use shorthand markups like Markdown.

Hence my bias in favour of trying to divine *some* meaning from
anything that looks like markup somehow, as long as there is any
chance at all that the result won?t be actively contrarian to the
user?s wishes.
> I think any rule would be ok, as long as it satisfies the
> following criteria:
> 
> 1. It's _simple_
I think the full rule needn?t be simple; as long as edge cases
don?t produce actively contrarian results, it?s OK not to mention
how they?re resolved in the interest of simplification.
> My reg-exp eye says: "strong" before "em" (longer
pattern
> first), starting from the right for each. I am pretty sure this
> rule satisfies 1, 2, and 3.
So the spec is going to make assumptions about the method of
implementation?
> Let's stop this non-sense and get back to defining a spec for
> the _normal_ markdown.
What?s the point? We already have one; John Gruber wrote it.
Interoperability problems crop up in the edge cases, not the
unproblematic stuff. That is what?s important to specify.



* Waylan Limberg <waylan at gmail.com> [2008-03-06
17:00]:
> Aristotle Pagaltzis wrote:
> > a human reader see in that text? Based on the visual
> > apperance I think I would make it translate to this:
> >
> >     <em>hello <strong>dear</strong> boy</em>
> 
> Ah, so your assuming the parser should automatically close
> unclosed tags much as a browser in quirks mode does.
No, a browser in quirks mode would not interpret such a
construction in the way I proposed at all.
> Sure, you and I understand how that works, but should we expect
> authors who are unfamiliar with html to get that?
I was precisely trying **not** to think about it in terms of
HTML. What I proposed was, as explicitly said, purely based on
visual apperance:

           ,-----,------ ?dear? is enclosed in emphasis markers
           v     v
    *hello **dear* boy**
    ^                  ^
    `------------------`----- the whole phase is also enclosed in
                              emphasis markers

And that?s why I prosed the translation in question.

It?s funny that everybody assumed I was thinking in terms of some
kind of error-correcting parser. I didn?t even consider how this
would be implemented while I was mulling over what to do with
that fragment.
> >  <em>hello <strong>dear</em> boy</strong>
> 
> Yeah, we could give them output that displays as they expect
> and fix it under the hood by doing:
> 
> >  <em>hello
<strong>dear</strong></em><strong> boy</strong>
> >
> But, the output **I** would expect is one of:
> 
> <em>hello </em><em>dear</em> boy**
> 
> <em>hello **dear</em> boy**
> 
> *hello <strong>dear* boy</strong>
I could live with that one:

    <em>hello **dear</em> boy**

That makes the least non-sense, of them all. But it would mean
that a lot of people will leave visible slop in their
Markdown-formatted text because they won?t care enough to fix it.

Remember that even with the output I proposed, it?s not hard for
someone to get *any* of these other interpretations, as long they
actually care enough to explicitly write their markup so as to
produce that.
> In my mind I keep going back and forth between the three and
> can never decide which the author intended. Finally, I cringe
> as I realize they probably intended what Seumas suggested.
Well, clearly, they meant to nest a few kinds of emphasis.

They probably wrote it that way either because they were thinking
in terms of markup and wanted overlapping regions, in which case
they can still do that with a bit more effort.

Or they wrote it hastily without caring a whole lot about the
output; in which case my proposal would at least keep markup slop
out of the output.
> To me, that is an important factor that seems to be ignored by
> some here. Sometimes, IMO, the best thing to do is to pass the
> markup through as literal text and give the author a clue that
> his formatting is unclear!
Again, that works if the author cares. If they don?t, it means
the output will be ugly.



* Seumas Mac Uilleachan <seumas at idirect.ca> [2008-03-06
19:50]:
> implies an error-checking mechanism built into markdown to
> catch such cases.
No it doesn?t. In fact I?m pretty sure I can implement the
translation I proposed purely as a regex substitution, at least
in Perl.
> Maybe what is needed is some kind of syntax checker to run the
> source through to point out to users where there are errors
> and/or confusing markup.
Wasn?t the original point that Markdown should have no concept of
an invalid document? I know Michel said that was his goal for
Markdown Extra, and it?s pretty much the argument which which
?Bowerbird? started this thread.

Regards,
-- 
Aristotle Pagaltzis // <http://plasmasturm.org/>

On Mar 7, 2008, at 5:17 AM, Aristotle Pagaltzis wrote:
> Hi Yuri, Weylan and Seumas,
>
> * Yuri Takhteyev <qaramazov at gmail.com> [2008-03-07 08:50]:
>>>>    *hello **dear* boy**
>>>
>>> That's a very good question. Here's a counterquestion: what
>>> does a human reader see in that text?
>>
>> When I try to look at this with my normal-person eye, what I
>> see here is incorrect markup
>
> Sorry, but if you see ?markup? (much less ?incorrect markup?)
> you?re not looking at it with a normal-person eye. :-)
>
>> So, the user will type in something like this and get
>> "<em>hello **dear</em> boy**". Not much of a
tradegy. They will
>> say, oh, silly me, must have screwed something up. (They did!)
>> Then they'll go and fix it. I am all for flexibility, but not
>> to the point of trying to divine the meaning of ambiguous or
>> ill-formed markup.
>
> Only a small minority will do that. Most people most of the time
> don?t care enough about that particular piece of text to actually
> fix any small nits in it, any more than they?ll care to fix all
> of their small spelling and grammar mistakes. (Less, actually.)
> That has certainly been my experience on wikis and weblogs that
> use shorthand markups like Markdown.
Given that, I would take advantage of the fact that Markdown source is  
highly readable.  If an input is too ambiguous, leave it unparsed.   
The source will be reasonably clear.  As the rules get more complex  
and try to make assumptions about what authors intended, there will be  
more cases in which the rules get it wrong and the output contains  
something that's both unintended and harder to puzzle out than the  
source would have been.

James

On Fri, Mar 7, 2008 at 5:17 AM, Aristotle Pagaltzis <pagaltzis at gmx.de>
wrote:
>
>  > To me, that is an important factor that seems to be ignored by
>  > some here. Sometimes, IMO, the best thing to do is to pass the
>  > markup through as literal text and give the author a clue that
>  > his formatting is unclear!
>
>  Again, that works if the author cares. If they don't, it means
>  the output will be ugly.
>
And if they don't care why should we? Seriously, if the author can't
be bothered to check his work and fix any errors, then I don't care
either. Sure, I may be annoyed when I read the document later, but now
I know how much effort s/he put into it. If it's obvious the author
doesn't care about it, then I probably won't be so easily persuaded by
the point s/he's trying to make. I'd say there's no harm done.


-- 
----
Waylan Limberg
waylan at gmail.com

>  >     *hello **dear* boy**
>
>  That's a very good question. Here's a counterquestion: what does
>  a human reader see in that text?
When I try to look at this with my normal-person eye, what I see here
is incorrect markup, which I then want to leave it as is and move on.
When I look at it with my formalistic left-parsing eye, I see
"<em>hello **dear</em> boy**".   When I look at it with my
reg-exps in
a loop eye, I see "*hello <strong>dear* boy</strong>". 
Either one of
those is ok with me.  Let's just pick one.  Everything else is from
the devil, I say.  Please, let's keep it simple.

So, the user will type in something like this and get "<em>hello
**dear</em> boy**".  Not much of a tradegy.  They will say, oh, silly
me, must have screwed something up.  (They did!)  Then they'll go and
fix it.  I am all for flexibility, but not to the point of trying to
divine the meaning of ambiguous or ill-formed markup.

I don't think it really matters what we output for cases like this.  I
think any rule would be ok, as long as it satisfies the following
criteria:

1. It's _simple_
2. It always produces valid XHTML (unless input has HTML tags)
3. It should produce appropriate HTML for "normal" markdown.

My reg-exp eye says:  "strong" before "em" (longer pattern
first),
starting from the right for each.  I am pretty sure this rule
satisfies 1, 2, and 3.

Let's stop this non-sense and get back to defining a spec for the
_normal_ markdown.

  - yuri

-- 
http://sputnik.freewisdom.org/

On Mar 7, 2008, at 2:45 AM, Yuri Takhteyev wrote:
>>>    *hello **dear* boy**
>>
>> That's a very good question. Here's a counterquestion: what
does
>> a human reader see in that text?
>
> When I try to look at this with my normal-person eye, what I see here
> is incorrect markup, which I then want to leave it as is and move on.
> When I look at it with my formalistic left-parsing eye, I see
> "<em>hello **dear</em> boy**".   When I look at it
with my reg-exps in
> a loop eye, I see "*hello <strong>dear*
boy</strong>".  Either one of
> those is ok with me.  Let's just pick one.  Everything else is from
> the devil, I say.  Please, let's keep it simple.
>
> So, the user will type in something like this and get "<em>hello
> **dear</em> boy**".  Not much of a tradegy.  They will say, oh,
silly
> me, must have screwed something up.  (They did!)  Then they'll go and
> fix it.  I am all for flexibility, but not to the point of trying to
> divine the meaning of ambiguous or ill-formed markup.
I strongly agree.  (Or is that emphatically agree?)

In this context, if the output is
	<em>hello **dear</em> boy**
or
	*hello <strong>dear* boy</strong>
and the user never notices the problem, the output is still readable.   
If the user does notice that something's wrong, it's easy to look at  
the output and realize roughly what happened.  ("Some the *s must not  
have matched up correctly, because they made it through into the  
output.")  Either of these solutions is a healthy response by the  
parser.

I would prefer "formalistic left-parsing" to "reg-exps in a
loop"
because (a) working from left to right is closer to the informal  
intuitive model that most users are likely to have as to how the text  
transformations work, and (b) if we actually specify a grammar, the  
more we stick to "formalistic left-parsing," the cleaner it will be.
> I don't think it really matters what we output for cases like this.  I
> think any rule would be ok, as long as it satisfies the following
> criteria:
>
> 1. It's _simple_
> 2. It always produces valid XHTML (unless input has HTML tags)
> 3. It should produce appropriate HTML for "normal" markdown.
>
I agree, though I might have said it ought to be **simple**.

James
> My reg-exp eye says:  "strong" before "em" (longer
pattern first),
> starting from the right for each.  I am pretty sure this rule
> satisfies 1, 2, and 3.
>
> Let's stop this non-sense and get back to defining a spec for the
> _normal_ markdown.
>
>  - yuri
>
> -- 
> http://sputnik.freewisdom.org/
> _______________________________________________
> Markdown-Discuss mailing list
> Markdown-Discuss at six.pairlist.net
> http://six.pairlist.net/mailman/listinfo/markdown-discuss

>  I think what is trying to be said here is that in creating the spec you
>  can't lose  the original focus of what Markdown is all about. Users
>  (such as myself) don't really care that much about how the html is
>  generated (breaks and explicit paragraphing are the domain of the
>  parser). What we care about is that the original intent of our written
>  source is maintained. It is very easy when creating a formal spec to
>  lose track of the original intent and thus the usefulness of the tool.
>  If I need to track exactly how many spaces I am allowed to use at the
>  beginning of the line for certain implied formatting (like lists) then I
>  am losing focus from the content I am writing, which is the exact
>  opposite of what Markdown was created for.
I don't understand this line of reasoning that already came up on the list.

As a user, don't you want that the same input correspond to the same
output using a different Markdown interpreter? For example, you
compose your blog post offline and preview it using Maruku, and then
you post it to your Movable Type blog that uses Markdown.pl. You
really want that the HTML generated is the same.

This is possible only if we agree on an unambiguous specification.
>  If Markdown ends up diverging by creating too many rigid rules then
>  users such as myself will just end up finding another tool.
Now, that a specification be unambiguous doesn't imply that the
"rules" are "rigid" from the point of view of the user. I
think this
is the misunderstanding for many people; I hope we can get past it.
> My $.02 CDN ($.02014US :)
Oh, my opinion counts more (0.02 EUR =  $0.03 ).

No, wait, now I get paid in dollars  :-(

-- 
Andrea Censi
PhD student, Control & Dynamical Systems, Caltech
http://www.cds.caltech.edu/~andrea/
    "Life is too important to be taken seriously" (Oscar Wilde)

* Bowerbird at aol.com <Bowerbird at aol.com> [2008-03-03
23:45]:
> > Is it possible for everyone to agree in all cases about how
> > the user?s intent should be teased out? Clearly it is
> > conceivable that enough effort could be made to write all
> > agreements down.
> > 
> > And if you write down what intent should be teased out of
> > particular inputs, what have you created but a spec? And if
> > the effort has been made to agree on all possible cases,
> > would this spec not be unambiguous? And is it not yet
> > obviously the case that such a spec would not need to be
> > inflexible about the syntax it admits?
> > 
> > Why then does the fallacious argument that a spec would
> > represent a loss for the user continue?
> 
> aren't you loading the dice by labeling it as "fallacious"?
No. The reasoning I outlined in questions [I?ve taken the liberty
to re-insert the quotation you elided] shows that the premises do
not support the argument. That means, like it or not, that the
argument is fallacious.

Sorry. I can?t change that.
> i'm not necessarily arguing that _any_ spec would do that.
> but some might...   most especially by some implementers...
Then those specs would not describe Markdown. As I understand,
the current discussion is about writing a specification for
Markdown, not for some other markup language.

If those ?some? implementors want to write a specification for
something other than Markdown, they are welcome to do so, but
they and their spec are irrelevant to the ongoing discussion.
> and according to markdown, users can't be wrong, can they?
Users can very well be wrong according to Markdown. As John has
said, if a human would have trouble figuring out what structure
some piece of text is supposed to signify, the computer can be
forgiven for likewise failing to infer a useful meaning.

Regards,
-- 
Aristotle Pagaltzis // <http://plasmasturm.org/>

aristotle said:
>    The reasoning I outlined in questions 
>    [I've taken the liberty to re-insert the quotation you elided] 
>    shows that the premises do not support the argument. 
your questions created a straw-man.

i agreed that the exact same input should create the same output.
it'd be ludicrous to say anything else.   that's why it's a
straw-man...

then i went on to the _real_ question, which is whether or not
_similar_ input interpretable by human beings in different ways
should be "regularized" by a spec for implementer convenience,
which i opined would seem contrary to markdown's philosophy...

>    That means, like it or not, that the argument is fallacious.
your straw-man was fallacious.   straw-men always are.

>   Users can very well be wrong according to Markdown. 
>    As John has said, if a human would have trouble figuring out 
>    what structure some piece of text is supposed to signify, 
>    the computer can be forgiven for likewise failing to infer 
>    a useful meaning.
and here we have another straw-man.   get over it.   please.

a situation where "a human would have trouble figuring out"
the interpretation is _not_ the situation that's troubling to me.
it's patently obvious that doesn't meet the spirit of markdown.

the situation that's troubling is where one set of input is clear,
while another set of similar-but-not-identical input leads to
an interpretation that is _different_ yet still _equally-clear_,
but you can't easily write routines to differentiate the two, so
you have the spec disallow one of 'em because that's an easy
"solution", even though it disenfranchises one of the users...

the question is "who is the boss?" -- the spec, or the users?

-bowerbird



**************
It's Tax Time! Get tips, forms, and advice on AOL Money & 
Finance.
      (http://money.aol.com/tax?NCID=aolprf00030000000001)
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://six.pairlist.net/pipermail/markdown-discuss/attachments/20080306/3fa7c...>

On Thu, Mar 6, 2008 at 12:22 PM,  <Bowerbird at aol.com>
wrote:
>
>  the situation that's troubling is where one set of input is clear,
>  while another set of similar-but-not-identical input leads to
>  an interpretation that is _different_ yet still _equally-clear_,
>  but you can't easily write routines to differentiate the two, so
>  you have the spec disallow one of 'em because that's an easy
>  "solution", even though it disenfranchises one of the users...
Srsly, talk of disenfranchisement seems heated rhetoric given that no
one will be prevented from writing a markdown document.

On a constructive tip:  What we're trying to do is design a
perspective, by specifying what markdown does now, from which
implementations of markdown can consistently interpret the same input.
 The first stage has always been to just write down some rules, etc.
about what markdown is doing now.  (Of course, the second stage is
"???" and the third is "profit".  If only we were underpants
gnomes.)

-- 
Joseph Lorenzo Hall
UC Berkeley School of Information
http://josephhall.org/

joseph said:
>    On a constructive tip:? What we're trying to do is 
>    design a perspective, by specifying what markdown does now, 
>    from which implementations of markdown can 
>    consistently interpret the same input.
and, if you will allow me to offer a constructive tip to _you_,
i'd suggest that instead you look closely at the cases where
_implementations_differ_, work to divine the logic of each,
and find a way that allows users to continue to have it all...

here's a rule of thumb for you:
if you break existing documents, you're doing it wrong.

and here's the rule of thumb on allowable exceptions:
if users thank you for the spec, in spite of the fact that
it breaks some of their existing documents, because
they think the new clarity is worth it, you did well enough.

***

aristotle said:
>   Wasn't the original point that Markdown 
>    should have no concept of an invalid document? 
>    I know Michel said that was his goal for Markdown Extra, 
>    and it's pretty much the argument which which
>    Bowerbird started this thread.
well, to get the argument straight, it's _not_ that
"there is no such thing as an invalid document"...
(garbage happens, and garbage-in-garbage-out.)

it's more along the lines of "if a human can see
the structure which was intended by the author,
markdown should display the structure correctly,
and _not_ tell users they didn't follow the spec."

-bowerbird

p.s.   and now, folks, i really will bow out of this thread...
if you haven't gotten what i'm saying by now, you won't.



**************
It's Tax Time! Get tips, forms, and advice on AOL Money & 
Finance.
      (http://money.aol.com/tax?NCID=aolprf00030000000001)
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://six.pairlist.net/pipermail/markdown-discuss/attachments/20080307/7d26a...>