markdown

CommonMark, plus what GitHub renders, for sysl. The module is sh.sysl.markdown.

Status: 0.4.0 — CommonMark and GitHub's Markdown complete, plus smart punctuation and definition lists. Every one of the CommonMark spec's 652 examples renders exactly, with the GitHub extensions off and on; every extension example of GitHub's spec renders exactly; hostile inputs are read in linear time; and differential checks against cmark and cmark-gfm agree on their whole corpora but for the cases listed below.

dependencies {
  markdown { git = "github.com/sysl-lang/markdown", version = "0.4.0" }
}

import sh.sysl.markdown.{parse, to_html}

print(to_html(parse("# Hello\n")))      // <h1>Hello</h1>

The API

parse(src: string) -> Doc                                   // CommonMark: parse_with(src, commonmark())
parse_with(src: string, opts: ParseOptions) -> Doc
parse_with_regions(src: string, opts: ParseOptions, regions: Buf[Region]) -> Result[Doc, RegionError]

to_html(d: Doc) -> string                                   // as the spec renders: to_html_with(d, commonmark_html())
to_html_with(d: Doc, opts: HtmlOptions) -> string
render_html(d: Doc, opts: HtmlOptions, out: *Writer)

plain_text(d: Doc, n: NodeId) -> string                     // the text under a node, for slugs, alt text, search

struct ParseOptions                                         // commonmark(): no extensions
    exts: Registry                                          // gfm(): tables, strikethrough, autolinks, task lists, math, mermaid

struct HtmlOptions                                          // commonmark_html(): true, "\n", None, false, Standard, Standard
    raw_html: bool                                          // gfm_html(): true, "\n", None, true, Standard, Standard
    softbreak: string
    highlight: Option[&Fn(string, string) -> Option[string]]   // (info, code) -> the block's markup
    tagfilter: bool                                         // GitHub's filter on raw HTML
    math: ClientMarkup                                      // Standard (KaTeX/MathJax auto-render) or GitHub
    mermaid: ClientMarkup                                   // Standard (mermaid.js) or GitHub

struct Region                                               // a run of program lines, for weave
    span: Span
    info: string
    literal: string

A parse and a rendering are told separately, so one parse can be rendered several ways; the link reference definitions live on the Doc.
raw_html: true passes raw HTML through as the spec renders it (the CommonMark preset); false writes  in place of each HTML block and each piece of inline HTML, as cmark's "safe" mode does. It governs raw HTML only -- a javascript: link is written either way.
highlight is asked about every code block with its whole info string and its text; what it answers replaces the whole <pre><code>…</code></pre>, and None falls back to that.
parse_with_regions reads a woven file: each region -- whole lines, in source order, not overlapping -- closes every open block and becomes a code block of the document whose span points into the real file. The rest is Markdown, and it is one document with one set of link references.
plain_text is the text a reader sees: entities decoded, code spans' content, link and image text but not their destinations, no raw HTML, a soft break as a space and a hard break as a newline, one newline between blocks. A node an extension made reads as its Extension.plain_text answers (Unseen, Words(text) or Block(text)), followed by what it holds: a table is a row a line with its cells separated by a tab, an alert its title and then its body, a formula its TeX, an emoji its character, a footnote reference its number, a mermaid diagram nothing.

This is checked line for line by tests_api.sysl's readme_example:

import sh.sysl.markdown.*

val d = parse("# Hello, *world*\n\nSee <b>this</b>.\n")

print(to_html(d))                                       // <h1>Hello, <em>world</em></h1> <p>See <b>this</b>.</p>
print(to_html_with(d, HtmlOptions(false, "\n", None, false, Standard, Standard)))  // ... <p>See <!-- raw HTML omitted -->this<!-- raw HTML omitted -->.</p>
print(plain_text(d, d.root()))                          // Hello, world / See this.

val code: &Fn(string, string) -> Option[string] = (info, text) ->
    if info == "sysl" then Some("<pre class=\"sysl\">" + escape_html(text) + "</pre>") else None

val opts = HtmlOptions(true, "\n", Some(code), false, Standard, Standard)

print(to_html_with(parse("```sysl\na < b\n```\n"), opts))   // <pre class="sysl">a &lt; b</pre>

GitHub Flavored Markdown

val d = parse_with("| a | b |\n|---|--:|\n| ~~c~~ | www.d.com |\n", gfm())

print(to_html_with(d, gfm_html()))

gfm() turns on the five extensions cmark-gfm has, then GitHub's emoji shortcodes, math, mermaid and alerts, and gfm_html() its tag filter. Each extension is an ordinary Extension, so a parse wanting some of them builds its own Registry: footnotes() (first, as cmark-gfm settles footnotes before the others look at the tree), tables(), strikethrough(), autolinks(), task_lists(), emoji(), math(), mermaid(), alerts(). Every rule of the first five is cmark-gfm's (0.29.0.gfm.13, which GitHub runs), down to the corners its spec does not write down:

Tables -- a delimiter row under a paragraph whose last line has as many cells; \| is a pipe in a cell, code spans included; a short row is filled out and a long one cut; any line that is not blank continues the table unless a block starts on it; a paragraph that met a delimiter row of the wrong width never becomes a table; and a table stops taking rows once it has filled out 524,288 cells, so a wide header over many one-cell rows stays linear. Cells carry align="…".
Strikethrough -- ~a~ or ~~a~~, both runs the same length; three or more ~ are text.
Task list items -- [ ], [x] or [X] after a list marker that begins the line, written as a disabled checkbox (Item(Some(checked)) in the tree).
Extended autolinks -- http://, https://, ftp://, www. and email addresses (with mailto: and xmpp: forms), without trailing punctuation or an unmatched ).
The tag filter -- HtmlOptions.tagfilter writes the < of title, textarea, style, xmp, iframe, noembed, noframes, script and plaintext as <.
Footnotes -- [^label] refers to a definition [^label]: text, which holds blocks: the rest of its line, then lines indented four columns, empty lines and lazy lines. Labels match as link labels do, the first of two definitions wins, footnotes are numbered by first reference (those inside definitions counting where they stand), an unreferenced definition is dropped and a reference to nothing is the text [^label]. The definitions are written at the end in <section class="footnotes" data-footnotes><ol>, each <li id="fn-label"> ending in ↩ back-references, the second and later references to a footnote taking ids fnref-label-2… The one place this departs from cmark-gfm is a reference with a line break inside it, whose label cmark-gfm measures by column and loses (tests_gfm_deviations.sysl).
Emoji -- :smile: is 😄, for every alias of GitHub's gemoji table; anything else between colons, GitHub's own pictures such as :octocat: included, stays text. Read wherever text is and never in code or a destination; \: before one keeps it text. emoji_for("smile") looks one up.
Alerts -- a block quote directly in the document whose first line is [!NOTE], [!TIP], [!IMPORTANT], [!WARNING] or [!CAUTION] (any case, nothing else on the line) and which holds something more. cmark-gfm does not read these -- GitHub adds them afterwards -- so the rules are what GitHub's renderer was observed to do (its /markdown API in gfm mode, 2026-10-08): a marker inside a list item or a nested quote is text, \[!NOTE] is still a marker and *[!NOTE]* is not, and a quote holding only the marker stays a quote. The HTML is GitHub's structure without the octicon <svg> it puts before the title's word, and with a newline between blocks as everywhere else:
```
<div class="markdown-alert markdown-alert-note">
<p class="markdown-alert-title">Note</p>
<p>body</p>
</div>
```

Heading ids are an extension of their own, heading_ids(), and not in gfm(): GitHub's spec shows headings without ids, and the preset renders what the spec shows.

val r = registry()

r.add(heading_ids())
print(to_html(parse_with("# Hello, World!\n# Hello, World!\n", ParseOptions(r))))
// <h1 id="hello-world">Hello, World!</h1>
// <h1 id="hello-world-1">Hello, World!</h1>

The slug is GitHub's (github_slug, github-slugger's: lowercased, punctuation and symbols dropped, spaces to -, Unicode letters kept), taken from the heading's text, a shortcode counted as written (## :rocket: Go is rocket-go, as on GitHub); a slug already used is numbered -1, -2 in document order. Each heading is then Heading(level, Some(id)) in the tree, which is what a table of contents reads. heading_ids_with(HeadingIdOptions(slug, prefix, anchors)) plugs in a slug of one's own, writes a prefix before every id (GitHub's pages use user-content-), and with anchors gives each heading GitHub's <a id="…" class="anchor" aria-hidden="true" href="#…"> link to itself, which then carries the id in the heading's place. A heading whose slug comes out empty (# alone, # !) has no id and no anchor, and takes no number from the headings after it.

Smart punctuation is an extension of its own too, smart_punctuation(), and not in gfm(): GitHub writes quotes as they are typed. Its rules are cmark's --smart, and tools/differential.sh --smart compares the two over the whole corpus.

val r = registry()

r.add(smart_punctuation())
print(to_html(parse_with("\"Don't,\" she said -- 'wait'...\n", ParseOptions(r))))
// <p>“Don’t,” she said – ‘wait’…</p>

A " or ' pairs with another as emphasis does, by the same flanking rules (an opener may not follow ] or )), and a pair makes no node, so a quotation may overlap emphasis as text does; a ' that pairs with nothing is an apostrophe, ’. A run of hyphens is em dashes where its length is a multiple of three, en dashes where it is even, and otherwise em dashes then one en dash or two; a single hyphen stays. ... is …. Code spans, code blocks, autolinks, raw HTML, entities and backslash escapes are never changed.

Definition lists are definition_lists(), in no preset either: PHP Markdown Extra's and Pandoc's syntax, a paragraph of terms (one to a line) followed by definitions, each opened by : and a space or tab.

val r = registry()

r.add(definition_lists())
print(to_html(parse_with("Apple\n: A round fruit.\n: A company.\n", ParseOptions(r))))
// <dl>
// <dt>Apple</dt>
// <dd>A round fruit.</dd>
// <dd>A company.</dd>
// </dl>

A definition is a container, as a list item is: a line indented to its text continues it, so it may hold several paragraphs, a list or code, and its paragraph takes lazy continuation lines. A : opens one at most three spaces in, and may interrupt a paragraph, whose lines become the terms; ::, a : with nothing after it, and a : not followed by a space or tab open nothing. A definition after a definition joins its list; one that follows no paragraph -- at the start of a container, after a heading -- is a paragraph again, its : kept as text. A blank line between any two parts of a list makes it loose, each definition's paragraphs in <p> (what the Scala juicer writes for its blank-line-separated items); otherwise a definition's paragraph is bare in the <dd>, as a tight list item's is. The tree is Ext nodes named definition_list, tagged DEFINITION_LIST_NODE, DEFINITION_TERM_NODE and DEFINITION_NODE.

Math and mermaid have no spec and no cmark-gfm extension; their rules are github.com's, read off its own renderer (gh api markdown, gfm mode) and written out at the top of gfm_math.sysl:

Math -- $…$ and $`…`$ inline, $$…$$ for display, and a fence whose info string begins math. An opening $ is not followed by a space and not preceded by a letter or digit; its closing $ is the next unescaped one, not preceded by a space and not followed by a letter or digit -- so $5 and $10 is text. A paragraph of nothing but $$…$$ is a display block and may run over several lines. The TeX is literal: no emphasis or escape is read inside it (\$ stays \$, TeX's dollar), and a code span, raw HTML or autolink is never cut in half by a $.
Mermaid -- a fence whose info string begins mermaid is a diagram.
Both take precedence over highlight: the fences become Ext nodes when the parse finishes, so the renderer never shows them to the hook. With the extensions off they are ordinary code blocks, and the hook is asked.
The markup is the rendering's -- HtmlOptions.math and .mermaid, each a ClientMarkup. Standard (both presets) writes <span class="math math-inline">$x$</span>, <div class="math math-display">\[x\]</div> (<span> for $$…$$ inside a line) and <pre class="mermaid">…</pre>: what KaTeX's or MathJax's auto-render and mermaid.js find with their defaults. GitHub writes github.com's <math-renderer class="js-inline-math">$x$</math-renderer> and js-display-math elements, and <pre lang="mermaid"><code>…</code></pre>.

Writing an extension

An extension is an impl Extension; every method but name has a default that declines, so it writes only the hooks it uses. The built-in extensions are written against exactly this.

trait Extension
    name(self) -> string
    block_start(self, line: Line) -> BlockStart = Declined
    interrupts_paragraph(self) -> bool = false
    block_continue(self, line: Line, kind: Kind, state: *u64) -> BlockContinue = Ends
    block_close(self, c: Closing, state: u64) -> Kind = c.kind
    triggers(self) -> string = ""
    inline_match(self, s: Subject) -> InlineMatch = NoMatch
    delimiter_node(self, ch: u8, opener: usize, closer: usize) -> Option[Kind] = None
    bracket(self, s: Subject, from: usize) -> Option[Kind] = None
    post_pass(self, d: Doc) = ()
    post_pass_phase(self) -> PostPhase = Structure              // Structure passes, then Annotate ones
    render(self, w: *HtmlWriter, d: Doc, n: NodeId, ev: Event) = ()
    plain_text(self, d: Doc, n: NodeId) -> Seen = Unseen         // what a reader sees, for plain_text

enum Seen                                                       // Unseen: nothing of its own; children still read
    Unseen
    Words(text: string)                                         // in the run of text
    Block(text: string)                                         // a block of its own; "" is a row's separation

enum BlockStart
    Declined
    Mark(word: u64)                                             // decline, and keep a word with the paragraph
    Leaf(kind: Kind, consumed: usize, state: u64)               // lines taken verbatim
    Container(kind: Kind, consumed: usize, state: u64)          // holds blocks
    Lines(kind: Kind, consumed: usize, state: u64, from_para: usize)   // lines taken as a paragraph's
    Retag(container: Kind, consumed: usize)                     // re-mark the block the line reached

enum InlineMatch
    NoMatch
    Inline(kind: Kind, consumed: usize)                         // a node; Text(lit) puts lit in the bytes' place
    Delimiters(count: usize, can_open: bool, can_close: bool)   // a run, paired into a node as emphasis is
    Paired(can_open: bool, can_close: bool, alone: string, opening: string, closing: string)
                                                                // one byte pairing as a quotation mark: no node

struct ExtNode                                                  // Kind.Ext(node: ExtNode)
    ext: string                                                 // the extension that made it, and renders it
    tag: u32
    data: string
    holds: bool                                                 // has children

A node of the extension's own is Ext(ExtNode(...)), so an extension outside this package adds kinds of node without touching Kind. The document keeps the registry it was read with, and the HTML renderer hands each Ext node to the render of the extension ext names -- on entering it, and on exiting it where it holds children -- with the HtmlWriter the page goes to: lit, tag (nothing inside an image's alt text), esc, href, cr and options().
A block hook sees a Line -- the text, where the containers' prefixes stop, the block the line has reached (within), the open paragraph's text (para) and the word the extension kept with it (para_word). Lines is how a block takes a paragraph's last lines as its own, as a table's header does; Retag is how a task list marks its list item.
A closing block is shown a Closing: its text, and child(parent, kind, from, to) and inlines(node, pieces), which give it children read from that text -- each mapped back to the source for its span, and pieces leaving out bytes the block's own syntax owns, as a cell's \|.
An inline hook sees a Subject -- the leaf's text, the trigger's offset, and whether a link's brackets are open -- and answers a node (a container node is given the bytes it read as its Text) or a run of delimiters, paired by the rule of three as emphasis is, then by delimiter_node.
bracket is asked about a ] that closed an active [ or ![ without making a link -- a Subject at the ] and where the bracketed text begins -- and Some(kind) replaces the brackets and everything between them with one node, as a footnote reference does.
block_continue may answer Indented(columns) where the block takes indentation by column, a tab counted to its stop, as a footnote definition takes its four.
post_pass sees the finished Doc and rewrites it with set_kind, append_child, insert_after, append_node and unlink, as the email autolinks, the footnotes and the alerts do.
Every Structure pass runs before every Annotate pass, each phase in registry order, so a pass that reads what another made does not depend on the order a program added them in. Structure is the default; heading_ids is Annotate, so ## Notes[^n] is notes1 whether heading_ids() was added before footnotes() or after. A pass that only reads the tree another pass has finished answers Annotate from post_pass_phase.

What it is

The syntax is CommonMark 0.31.2, checked against the spec's 652 examples, comparing HTML exactly with no normalisation: the spec's HTML is the reference renderer's own output.
Plus what GitHub renders, as extensions: tables, strikethrough, task lists, extended autolinks, footnotes, alerts (> [!NOTE]), heading ids slugged as GitHub slugs them, and emoji shortcodes.
Plus math and mermaid, as GitHub does them: $…$ , $$…$$ and ```math fences are written out as TeX for a client-side renderer; ```mermaid fences are handed on the way GitHub hands them to mermaid.js.
Raw HTML is passed through as the spec renders it. GitHub's tag filter is an option, off in the CommonMark preset.
Out unless asked for: Pandoc attributes, definition lists, smart punctuation.

How it is built

One arena of nodes, linked by index. Blocks and inlines alike live in one Buf[Node] on the Doc, each with its parent, first and last child, and siblings as indices, and a byte Span into the source. A rewrite over every node is a flat loop with set_kind; moving a node is a relink.
Walking and rendering do not recurse, so input nested a hundred thousand deep renders.
The HTML renderer writes what the reference renderers write, byte for byte: &, <, > and " escaped, a link destination percent-encoded as mdurl.encode does it (an existing %XX kept, ä as %C3%A4), and a code block's class from the info string's first word as cmark cuts it, at ASCII whitespace, with no second language- before one that has it.
Linear on hostile input. cmark's pathological inputs are read in time proportional to their size, and references expand to at most the document's size (or 100,000 bytes, whichever is more), as cmark caps them, so a long definition used many times cannot make the output explode.
Extensions are an open trait, Extension, asked at three points -- a block starter (with whether it may interrupt a paragraph, a continuation check and a close), an inline trigger byte, and a pass over the finished tree -- and asked to render the nodes it made. The built-in extensions are written against the same trait; see Writing an extension.
Link reference definitions live on the Doc, so one parse can be rendered several ways.

The spec

spec/spec.json is CommonMark 0.31.2's, unmodified, as published at https://spec.commonmark.org/0.31.2/spec.json:

sha256  d431b29d97b6f73e69d547109cf5081578fac931e72afe95639ebe766c1b2a20

tools/gen_spec_tests.py writes it out as sh/sysl/markdown/tests_spec_*.sysl, one file per spec section and one @test per example. A section the parser does not read yet is generated with its examples ignored; listing it in the script's ENABLED and rerunning makes them run. The generated files are committed.

spec/gfm-spec.txt is GitHub Flavored Markdown's spec, unmodified, as test/spec.txt of github/cmark-gfm at the tag 0.29.0.gfm.13 (commit 587a12bb54d95ac37241377e6ddc93ea0e45439b):

sha256  7d8e5814befec287ac116786d81ff14e0adc9b13295b4494649e995408fd871c

The same script writes its 24 extension examples -- 8 tables, 2 strikethrough, 2 task lists, 11 autolinks, 1 tag filter -- as tests_gfm_<extension>.sysl, read with gfm() and gfm_html() and compared exactly. The two task-list examples' HTML is written normalised in that spec (its runner normalises before comparing), so their tests assert the HTML cmark-gfm writes. Its CommonMark examples are 0.29's and are not taken; instead tests_gfm_commonmark.sysl reads all 652 of CommonMark 0.31.2's with every extension on. Eleven render differently, as cmark-gfm's own runner lists such examples -- six HTML blocks the tag filter rewrites and five autolink examples whose text an extended autolink now links -- and each asserts what cmark-gfm renders with the same extensions. Math and mermaid change none of the 652: no example's $ meets the delimiter rules, and none has a math or mermaid fence. Footnotes and alerts change none either.

spec/cmark-gfm-extensions.txt is cmark-gfm's own extension tests, unmodified, as test/extensions.txt at the same tag:

sha256  a2a45e98be9fca95f564f927265a0f63beea6cae5369d1cf4bde44caa51b2a3a

GitHub's spec has no footnote examples; this file's three are written out as tests_gfm_footnotes.sysl, compared exactly.

spec/entities.json is WHATWG's list of HTML5 named character references, unmodified, as published at https://html.spec.whatwg.org/entities.json:

sha256  d741d877ac77c4194c4ad526b5b4a19aef8dfe411ab840a466891cdbb9f362e6

tools/gen_entities.py writes the 2,125 names that end in ; -- the only ones CommonMark recognises -- into sh/sysl/markdown/entity_table.sysl as one sorted string, which a lookup bisects. One string literal adds about 0.01 s to a cold build, where a table of pairs would add seconds. The generated file is committed.

spec/gemoji.json is db/emoji.json of github/gemoji, unmodified, at commit fadaeaf1f1a9be82b321316a6c5502e43138b2f6; spec/github-slugger-regex.js and spec/github-slugger-fixtures.json are regex.js and test/fixtures.json of Flet/github-slugger 2.0.0, unmodified, at commit 3461c4350868329c8530904d170358bca1d31448:

sha256  b174ae2aeb321b52f64adb9ff412f966a7f338839d780784dd15dcad702c2dd6  gemoji.json
sha256  3bcbb3836a4e76f0f343c34d223ddd9eac83cc73a4e2c7a1118ef0829bc9129b  github-slugger-regex.js
sha256  26e59aec9fb3adee7a56445049ef2d537126b28581802903b529d2556778b248  github-slugger-fixtures.json

tools/gen_github_tables.py writes the 1,913 aliases into emoji_table.sysl in the entity table's layout (34,775 bytes packed), the 977,500 code points a slug drops into slug_table.sysl as 735 ranges -- reading the regex as JavaScript does, surrogate pairs and all, and checking the set against node where node is installed -- and the 78 fixtures into tests_slug_fixtures.sysl, run once through one slugger and once as headings of one document. The two tables add about 0.04 s to a cold build of a program using gfm(). The generated files are committed.

Testing

sysl test .

sh/sysl/markdown/tests_pathological.sysl carries every input of cmark's test/pathological_tests.py at the tag 0.31.2 (https://github.com/commonmark/cmark/blob/0.31.2/test/pathological_tests.py), each at cmark's size and a tenth of it, with the HTML checked exactly at both and the time checked against the input's growth: a ratio under 20 where linear reading gives about 10 and quadratic about 100. tests_gfm_scaling.sysl does the same for the extensions' own hostile inputs: tables of 50,000 columns and of 50,000 rows, a cell of escaped pipes, a paragraph of mismatched delimiter rows, runs of ~, and text an autolink almost starts in at every byte. tests_gfm_math_scaling.sysl adds math's: runs of 50,000 $, and openers that never close or whose closers each fail a different rule. tests_footnote_scaling.sysl does it for footnotes and alerts: 20,000 references to one footnote, 20,000 footnotes, 20,000 unreferenced definitions, 20,000 definitions nested one inside the next, undefined references and unclosed [^, 20,000 alerts and a 20,000-byte almost-marker.

tests_footnotes.sysl holds 36 footnote cases, each asserting the HTML cmark-gfm writes for it, and tests_alerts.sysl 34 alert cases asserting the HTML GitHub writes (less the icon).

tests_smart_punctuation.sysl holds the examples of the CommonMark spec's smart_punct.txt and the edges around them -- quotes at a paragraph's ends, beside punctuation and brackets, nested, inside emphasis and links, hyphen runs of two to thirteen -- each asserting the HTML cmark --smart writes.

tests_definition_lists.sysl holds definition lists as exact HTML: tight and loose, several terms and definitions, lazy and indented continuation, blocks inside a definition, lists inside a list item and a block quote, inline markup, the colons that open nothing, definitions that follow no term, and the extension off. The loose case is the page juicer's documentation renders with the Scala juicer.

tests_gfm_math.sysl holds math's and mermaid's rules as exact HTML. Neither has a spec or a cmark-gfm extension, so the oracle is github.com itself: the delimiter cases its header names were rendered by gh api markdown -f mode=gfm, and the GitHub markup is asserted against what it wrote.

tests_gfm_edges.sysl holds 187 corner cases for the extensions, each asserting the HTML cmark-gfm writes for it -- the inputs tools/differential.sh --gfm was run over while they were written.

The differential check against cmark

tools/differential.sh [--gfm | --smart] [file or directory ...]

renders a corpus through this package and through cmark --unsafe and names every file whose HTML differs, keeping a diff of each. With no arguments the corpus is the spec's examples, cmark's regression examples, the README of every repository under ~/dev/sysl-lang and the pages of ~/dev/sysl-lang/sysl-census-34/docs/content; it needs cmark on the PATH (brew install cmark) and python3. SYSL and CMARK name other binaries.

--gfm compares GitHub Flavored Markdown: gfm() less emoji, math, mermaid and alerts (which cmark-gfm does not read) and gfm_html() here against cmark-gfm --unsafe -e footnotes -e table -e strikethrough -e autolink -e tasklist -e tagfilter (brew install cmark-gfm; CMARK_GFM names another binary), with GitHub's spec examples and cmark-gfm's extension tests added to the corpus. cmark-gfm reads CommonMark 0.29, so a file on which it and cmark 0.31.2 disagree with every extension off is marked as such: there the difference is the CommonMark version's, and this package renders as cmark 0.31.2 does. Over the whole corpus -- 1,591 files -- every difference is one of those.

--smart parses with smart_punctuation() alone against cmark --unsafe --smart. Over the default corpus -- 789 files -- none differs.

spec/cmark-regression.txt is cmark 0.31.2's test/regression.txt, unmodified:

sha256  aaa16d0e50464dfd03628acb4eaa4db462efb3ae523d7419ba15787add74d3ed

Where cmark and this package disagree and the spec decides for this package, each is pinned in tests_cmark_deviations.sysl:

input	cmark 0.31.2	the spec
a comment whose text ends in `-`: `<!--<!---->`	not a comment	§6.6, an HTML comment is `<!--`, text not including `-->`, and `-->` (example 625)
a CDATA section whose text ends in `]`: `<![CDATA[x]]]>`	not raw HTML	§6.6, a CDATA section (example 629)
`<![CDATA[…]]>` or `<!X…>` after an unclosed `<!--`	text	§6.6: each kind of raw HTML is matched on its own
leading whitespace of a lazy line, or of one indented four or more	kept, showing in code spans, after a hard break, and stopping a definition	§4.8, examples 222 and 223: leading spaces or tabs are skipped
`[a]: /u` then `"t"x` on the next line	`[a]` keeps the title `t`	§4.7, example 210: the title is not part of the definition
`[x [a](b) [c] ](d)`	a link inside a link	§6.3, examples 518 and 519: links never contain links
`<script/>` alone on a line	an HTML block	§4.6, start condition 7 excludes open tags named `pre`, `script`, `style` and `textarea`
a tab before a line ending	removed	§6.8: spaces at the end of a line are removed

Where cmark-gfm and this package disagree with the extensions on, each is pinned in tests_gfm_deviations.sysl:

input	cmark-gfm 0.29.0.gfm.13	this package
a link reference definition in the lines above a table's header	left as paragraph text, and never defined	a definition (CommonMark §4.7)
`\|` in the lines above a table's header	its backslash removed, inside a code span too	CommonMark's escape, which a code span does not read
a `'` in a link's destination	written `'` (cmark does the same)	written `'`, as commonmark.js writes it

License

ISC

Name		Name	Last commit message	Last commit date
sh/sysl/markdown		sh/sysl/markdown
spec		spec
tools		tools
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
package.hocon		package.hocon
sysl.sum		sysl.sum

Navigation Menu

Repository navigation

About

Repository files navigation

markdown

The API

GitHub Flavored Markdown

Writing an extension

What it is

How it is built

The spec

Testing

The differential check against cmark

License

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

markdown

The API

GitHub Flavored Markdown

Writing an extension

What it is

How it is built

The spec

Testing

The differential check against cmark

License

About

Resources

Security policy

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages