markdown

README

goldmark-markdown

Goldmark-markdown is a goldmark renderer that renders to markdown. It can be used directly as an auto-formatter for markdown source, or extended via goldmark's powerful AST transformers to programmatically transform markdown files.

This module was created for my update-a-changelog GitHub Action, to allow it to intelligently merge new changelog entries from Pull Requests into CHANGELOG.md, as well as add new versions to CHANGELOG.md when the corresponding tag is pushed.

As a formatter

You can use goldmark-markdown to format existing markdown documents. It removes extraneous whitespace, and enforces consistent style for things like indentation, headings, and lists.

// Create goldmark converter with markdown renderer object
// Can pass functional Options as arguments. This example converts headings to ATX style.
renderer := markdown.NewRenderer(markdown.WithHeadingStyle(markdown.HeadingStyleATX))
md := goldmark.New(goldmark.WithRenderer(renderer))

// "Convert" markdown to formatted markdown
source := `
My Document Title
=================
`
buf := bytes.Buffer{}
err := md.Convert([]byte(source), &buf)
if err != nil {
  log.Fatal(err)
}
log.Print(buf.String()) // # My Document Title

Options

You can control the style of various markdown elements via functional options that are passed to the renderer.

Functional Option	Type	Description
WithIndentStyle	markdown.IndentStyle	Indent nested blocks with spaces or tabs.
WithHeadingStyle	markdown.HeadingStyle	Render markdown headings as ATX (#-based), Setext (underlined with === or ---), or variants thereof.
WithThematicBreakStyle	markdown.ThematicBreakStyle	Render thematic breaks with -, *, or _.
WithThematicBreakLength	markdown.ThematicBreakLength	Number of characters to use in a thematic break (minimum 3).

As a markdown transformer

Goldmark supports writing transformers that can inspect and modify the parsed markdown AST before it gets sent to the renderer for output. You can use transformers in conjunction with goldmark-markdown's renderer to make changes to markdown sources while preserving valid syntax.

For example, you can scan the AST for text that matches a pattern for an external resource, and transform that text into a link to the resource, similar to GitHub's custom autolinks feature. Start by adding a struct that holds a regexp pattern to scan text for and a URL replacement for the pattern:

// RegexpLinkTransformer is an AST Transformer that transforms markdown text that matches a regex
// pattern into a link.
type RegexpLinkTransformer struct {
	LinkPattern *regexp.Regexp
	ReplUrl     []byte
}

Next, implement a Transform function that walks the AST and calls a LinkifyText function on any Text nodes encountered:

// Transform implements goldmark.parser.ASTTransformer
func (t *RegexpLinkTransformer) Transform(node *ast.Document, reader text.Reader, pc parser.Context) {
	source := reader.Source()

	// Walk the AST in depth-first fashion and apply transformations
	err := ast.Walk(node, func(node ast.Node, entering bool) (ast.WalkStatus, error) {
		// Each node will be visited twice, once when it is first encountered (entering), and again
		// after all the node's children have been visited (if any). Skip the latter.
		if !entering {
			return ast.WalkContinue, nil
		}
		// Skip the children of existing links to prevent double-transformation.
		if node.Kind() == ast.KindLink || node.Kind() == ast.KindAutoLink {
			return ast.WalkSkipChildren, nil
		}
		// Linkify any Text nodes encountered
		if node.Kind() == ast.KindText {
			textNode := node.(*ast.Text)
			t.LinkifyText(textNode, source)
		}

		return ast.WalkContinue, nil
	})

	if err != nil {
		log.Fatal("Error encountered while transforming AST:", err)
	}
}

The function passed to ast.Walk will be called for every node visited. The function controls which node is visited next via its return value. ast.WalkContinue causes the walk to continue to the next child, sibling, or parent node, depending on whether such nodes exist. ast.WalkSkipChildren continues only to sibling or parent nodes. Both ast.WalkStop and returning a non-nil error cause the walk to end.

For this example, we are only interested in linkifying Text nodes that are not already part of a link, and continue past everything else.

In order to "linkify" the Text nodes, the transformer will replace the original Text node with nodes for before and after the link, as well as the node for the link itself:

// LinkifyText finds all LinkPattern matches in the given Text node and replaces them with Link
// nodes that point to ReplUrl.
func (t *RegexpLinkTransformer) LinkifyText(node *ast.Text, source []byte) {
	parent := node.Parent()
	tSegment := node.Segment
	match := t.LinkPattern.FindIndex(tSegment.Value(source))
	if match == nil {
		return
	}
	// Create a text.Segment for the link text.
	lSegment := text.NewSegment(tSegment.Start+match[0], tSegment.Start+match[1])

	// Insert node for any text before the link
	if lSegment.Start != tSegment.Start {
		bText := ast.NewTextSegment(tSegment.WithStop(lSegment.Start))
		parent.InsertBefore(parent, node, bText)
	}

	// Insert Link node
	link := ast.NewLink()
	link.AppendChild(link, ast.NewTextSegment(lSegment))
	link.Destination = t.LinkPattern.ReplaceAll(lSegment.Value(source), t.ReplUrl)
	parent.InsertBefore(parent, node, link)

	// Update original node to represent the text after the link (may be empty)
	node.Segment = tSegment.WithStart(lSegment.Stop)

	// Linkify remaining text if not empty
	if node.Segment.Len() > 0 {
		t.LinkifyText(node, source)
	}
}

To use this transformer, we'll need to instantiate one or more RegexpLinkTransformer structs, then prioritize them and add them to the parser configuration of the goldmark object. The transformation(s) will then be automatically applied to all markdown documents converted by the goldmark object.

transformer := RegexpLinkTransformer{
  LinkPattern: regexp.MustCompile(`TICKET-\d+`),
  ReplUrl:     []byte("https://example.com/TICKET?query=$0"),
}
// Goldmark supports multiple AST transformers and runs them sequentially in order of priority.
prioritizedTransformer := util.Prioritized(&transformer, 0)
// Setup goldmark with the markdown renderer and our transformer
gm := goldmark.New(
  goldmark.WithRenderer(markdown.NewRenderer()),
  goldmark.WithParserOptions(parser.WithASTTransformers(prioritizedTransformer)),
)

The complete example can be found in autolink_example_test.go, or in the go doc for this package.

Documentation

Overview

Package markdown is a goldmark renderer that outputs markdown.

Example

package main

import (
	"bytes"
	"fmt"
	"log"
	"regexp"

	markdown "github.com/teekennedy/goldmark-markdown"
	"github.com/yuin/goldmark"
	"github.com/yuin/goldmark/ast"
	"github.com/yuin/goldmark/parser"
	"github.com/yuin/goldmark/text"
	"github.com/yuin/goldmark/util"
)

// RegexpLinkTransformer is an AST Transformer that transforms markdown text that matches a regex
// pattern into a link.
type RegexpLinkTransformer struct {
	LinkPattern *regexp.Regexp
	ReplUrl     []byte
}

// LinkifyText finds all LinkPattern matches in the given Text node and replaces them with Link
// nodes that point to ReplUrl.
func (t *RegexpLinkTransformer) LinkifyText(node *ast.Text, source []byte) {
	parent := node.Parent()
	tSegment := node.Segment
	match := t.LinkPattern.FindIndex(tSegment.Value(source))
	if match == nil {
		return
	}
	// Create a text.Segment for the link text.
	lSegment := text.NewSegment(tSegment.Start+match[0], tSegment.Start+match[1])

	// Insert node for any text before the link
	if lSegment.Start != tSegment.Start {
		bText := ast.NewTextSegment(tSegment.WithStop(lSegment.Start))
		parent.InsertBefore(parent, node, bText)
	}

	// Insert Link node
	link := ast.NewLink()
	link.AppendChild(link, ast.NewTextSegment(lSegment))
	link.Destination = t.LinkPattern.ReplaceAll(lSegment.Value(source), t.ReplUrl)
	parent.InsertBefore(parent, node, link)

	// Update original node to represent the text after the link (may be empty)
	node.Segment = tSegment.WithStart(lSegment.Stop)

	// Linkify remaining text if not empty
	if node.Segment.Len() > 0 {
		t.LinkifyText(node, source)
	}
}

// Transform implements goldmark.parser.ASTTransformer
func (t *RegexpLinkTransformer) Transform(node *ast.Document, reader text.Reader, pc parser.Context) {
	source := reader.Source()

	// Walk the AST in depth-first fashion and apply transformations
	err := ast.Walk(node, func(node ast.Node, entering bool) (ast.WalkStatus, error) {
		// Each node will be visited twice, once when it is first encountered (entering), and again
		// after all the node's children have been visited (if any). Skip the latter.
		if !entering {
			return ast.WalkContinue, nil
		}
		// Skip the children of existing links to prevent double-transformation.
		if node.Kind() == ast.KindLink || node.Kind() == ast.KindAutoLink {
			return ast.WalkSkipChildren, nil
		}
		// Linkify any Text nodes encountered
		if node.Kind() == ast.KindText {
			textNode := node.(*ast.Text)
			t.LinkifyText(textNode, source)
		}

		return ast.WalkContinue, nil
	})

	if err != nil {
		log.Fatal("Error encountered while transforming AST:", err)
	}
}

var source = `
Standup notes:
- Previous day:
  - Gave feedback on TICKET-123.
  - Outlined presentation on syntax-aware markdown transformations.
  - Finished my part of TICKET-456 and assigned to Emily.
- Today:
  - Add integration tests for TICKET-789.
  - Create slides for presentation.
`

func main() {
	// Instantiate our transformer
	transformer := RegexpLinkTransformer{
		LinkPattern: regexp.MustCompile(`TICKET-\d+`),
		ReplUrl:     []byte("https://example.com/TICKET?query=$0"),
	}
	// Goldmark supports multiple AST transformers and runs them sequentially in order of priority.
	prioritizedTransformer := util.Prioritized(&transformer, 0)
	// Setup goldmark with the markdown renderer and our transformer
	gm := goldmark.New(
		goldmark.WithRenderer(markdown.NewRenderer()),
		goldmark.WithParserOptions(parser.WithASTTransformers(prioritizedTransformer)),
	)
	// Output buffer
	buf := bytes.Buffer{}

	// Convert parses the source, applies transformers, and renders output to the given io.Writer
	err := gm.Convert([]byte(source), &buf)
	if err != nil {
		log.Fatalf("Encountered Markdown conversion error: %v", err)
	}
	fmt.Print(buf.String())

}

Output:
Standup notes:
- Previous day:
  - Gave feedback on [TICKET-123](https://example.com/TICKET?query=TICKET-123).
  - Outlined presentation on syntax-aware markdown transformations.
  - Finished my part of [TICKET-456](https://example.com/TICKET?query=TICKET-456) and assigned to Emily.
- Today:
  - Add integration tests for [TICKET-789](https://example.com/TICKET?query=TICKET-789).
  - Create slides for presentation.

Index

• Constants
• func WithHeadingStyle(style HeadingStyle) interface{ ... }
• func WithIndentStyle(style IndentStyle) interface{ ... }
• func WithThematicBreakLength(style ThematicBreakLength) interface{ ... }
• func WithThematicBreakStyle(style ThematicBreakStyle) interface{ ... }
• type Config
  • func NewConfig(options ...Option) *Config
  • func (c *Config) SetOption(name renderer.OptionName, value interface{})
• type HeadingStyle
  • func (i HeadingStyle) IsSetext() bool
• type IndentStyle
  • func (i IndentStyle) Bytes() []byte
• type Option
• type Renderer
  • func NewRenderer(options ...Option) *Renderer
  • func (r *Renderer) AddOptions(opts ...renderer.Option)
  • func (r *Renderer) Render(w io.Writer, source []byte, n ast.Node) error
• type ThematicBreakLength
• type ThematicBreakStyle

Examples

• Package

Constants

const (
	// IndentStyleSpaces indents with 4 spaces. This is the default as well as the zero-value.
	IndentStyleSpaces = iota
	// IndentStyleTabs indents with tabs.
	IndentStyleTabs
)

const (
	// HeadingStyleATX is the #-based style. This is the default heading style and zero value.
	// Ex: ## Foo
	HeadingStyleATX = iota
	// HeadingStyleATXSurround adds closing #s after your header.
	// Ex: ## Foo ##
	HeadingStyleATXSurround
	// HeadingStyleSetext uses setext heading underlines ('===' or '---') for heading levels 1 and
	// 2, respectively. Other header levels continue to use ATX headings.
	// Ex: Foo Bar
	//     ---
	HeadingStyleSetext
	// HeadingStyleFullWidthSetext extends setext heading underlines to the full width of the
	// header text.
	// Ex: Foo Bar
	//     -------
	HeadingStyleFullWidthSetext
)

const (
	// ThematicBreakStyleDashed uses '-' character for thematic breaks. This is the default and
	// zero value.
	// Ex: ---
	ThematicBreakStyleDashed = iota
	// ThematicBreakStyleStarred uses '*' character for thematic breaks.
	// Ex: ***
	ThematicBreakStyleStarred
	// ThematicBreakStyleUnderlined uses '_' character for thematic breaks.
	// Ex: ___
	ThematicBreakStyleUnderlined
)

const (
	// ThematicBreakLengthMinimum is the minimum length of a thematic break. This is the default.
	// Any lengths less than this minimum are converted to the minimum.
	// Ex: ---
	ThematicBreakLengthMinimum = 3
)

Functions

func WithHeadingStyle

func WithHeadingStyle(style HeadingStyle) interface {
	renderer.Option
	Option
}

WithHeadingStyle is a functional option that sets the style of markdown headings.

func WithIndentStyle

func WithIndentStyle(style IndentStyle) interface {
	renderer.Option
	Option
}

WithIndentStyle is a functional option that sets the string used to indent markdown blocks.

func WithThematicBreakLength

func WithThematicBreakLength(style ThematicBreakLength) interface {
	renderer.Option
	Option
}

WithThematicBreakLength is a functional option that sets the length of thematic breaks.

func WithThematicBreakStyle

func WithThematicBreakStyle(style ThematicBreakStyle) interface {
	renderer.Option
	Option
}

WithThematicBreakStyle is a functional option that sets the character used for thematic breaks.

Types

type Config

type Config struct {
	IndentStyle         IndentStyle
	HeadingStyle        HeadingStyle
	ThematicBreakStyle  ThematicBreakStyle
	ThematicBreakLength ThematicBreakLength
}

Config struct holds configurations for the markdown based renderer.

func NewConfig

func NewConfig(options ...Option) *Config

NewConfig returns a new Config with defaults and the given options.

func (*Config) SetOption

func (c *Config) SetOption(name renderer.OptionName, value interface{})

SetOption implements renderer.SetOptioner.SetOption.

type HeadingStyle

type HeadingStyle int

HeadingStyle is an enum expressing how markdown headings should look.

func (HeadingStyle) IsSetext

func (i HeadingStyle) IsSetext() bool

IsSetext returns true if heading style is one of the Setext options

type IndentStyle

type IndentStyle int

IndentStyle is an enum expressing how markdown blocks should be indented.

func (IndentStyle) Bytes

func (i IndentStyle) Bytes() []byte

String returns the string representation of the indent style

type Option

type Option interface {
	renderer.Option
	// SetMarkDownOption sets this option on the markdown renderer config
	SetMarkdownOption(*Config)
}

Option is an interface that sets options for Markdown based renderers.

type Renderer

type Renderer struct {
	// contains filtered or unexported fields
}

Renderer is an implementation of renderer.Renderer that renders nodes as Markdown

func NewRenderer

func NewRenderer(options ...Option) *Renderer

NewRenderer returns a new markdown Renderer that is configured by default values.

func (*Renderer) AddOptions

func (r *Renderer) AddOptions(opts ...renderer.Option)

AddOptions implements renderer.Renderer.AddOptions

func (*Renderer) Render

func (r *Renderer) Render(w io.Writer, source []byte, n ast.Node) error

Render implements renderer.Renderer.Render

type ThematicBreakLength

type ThematicBreakLength int

ThematicBreakLength configures the character length of thematic breaks

type ThematicBreakStyle

type ThematicBreakStyle int

ThematicBreakStyle is an enum expressing the character used for thematic breaks.