parser

package
v2.0.0-beta.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 8, 2026 License: MIT Imports: 9 Imported by: 0

Documentation

Overview

Package parser contains stuff that are related to parsing a Markdown text.

Index

Constants

This section is empty.

Variables

View Source 
var CommonMark = &commonMark{}

CommonMark is a commonmark compliant extension. This extension adds default block parsers, inline parsers, and paragraph transformers to the parser.

Block parsers:

  • SetextHeadingParser, 100
  • ThematicBreakParser, 200
  • ListParser, 300
  • ListItemParser, 400
  • CodeBlockParser, 500
  • ATXHeadingParser, 600
  • FencedCodeBlockParser, 700
  • BlockquoteParser, 800
  • HTMLBlockParser, 900
  • ParagraphParser, 1000

Inline parsers:

  • CodeSpanParser, 100
  • LinkParser, 200
  • AutoLinkParser, 300
  • RawHTMLParser, 400
  • EmphasisParser, 500

Paragraph transformers:

  • LinkReferenceParagraphTransformer, 100
View Source 
var Nil = ast.NewText()

Nil is a special AST node that represents an empty node. If a parser returns Nil, the parser is considered as successful but does not add any node to the AST tree.

Functions

func IsLeftFlankingDelimiterRun 

func IsLeftFlankingDelimiterRun(before, after rune) bool

IsLeftFlankingDelimiterRun returns true if the position represents a left-flanking delimiter run as defined by the CommonMark spec. before is the character preceding the delimiter run and after is the character immediately following it.

func IsRightFlankingDelimiterRun 

func IsRightFlankingDelimiterRun(before, after rune) bool

IsRightFlankingDelimiterRun returns true if the position represents a right-flanking delimiter run as defined by the CommonMark spec. before is the character preceding the delimiter run and after is the character immediately following it.

func ParseAttributes 

func ParseAttributes(reader text.Reader) ([]gast.Attribute, bool)

ParseAttributes parses attributes into a slice of ast.Attribute. ParseAttributes returns a parsed attributes and true if could parse attributes, otherwise nil and false.

func ProcessDelimiters 

func ProcessDelimiters(bottom ast.Node, pc Context)

ProcessDelimiters processes the delimiter list in the context. Processing will be stop when reaching the bottom.

If you implement an inline parser that can have other inline nodes as children, you should call this function when nesting span has closed.

func WithAttribute 

func WithAttribute() interface {
	Option
	HeadingOption
}

WithAttribute is a functional option that enables custom attributes. It can be used as a parser Option and a HeadingOption.

func WithAutoHeadingID 

func WithAutoHeadingID() interface {
	Option
	HeadingOption
}

WithAutoHeadingID is a functional option that enables custom heading ids and auto generated heading ids. It can be used as a parser Option and a HeadingOption.

func WithIDGenerator 

func WithIDGenerator(gen IDGenerator) interface {
	Option
	ContextOption
	IDsOption
}

WithIDGenerator is a functional option that sets a custom IDGenerator for element id generation. It can be used as a parser Option, a parser ContextOption, and a parser IDsOption.

Types

type ASTTransformer 

type ASTTransformer interface {
	// Transform transforms the given AST tree.
	Transform(node *ast.Document, reader text.Reader, pc Context)
}

ASTTransformer transforms entire Markdown document AST tree.

type Block 

type Block struct {
	// Node is a BlockNode.
	Node ast.Node
	// Parser is a BlockParser.
	Parser BlockParser
}

A Block struct holds a node and correspond parser pair.

type BlockParser 

type BlockParser interface {
	// Trigger returns a list of characters that triggers Parse method of
	// this parser.
	// If Trigger returns a nil, Open will be called with any lines.
	Trigger() []byte

	// Open parses the current line and returns a result of parsing.
	//
	// Open must not parse beyond the current line.
	// If Open has been able to parse the current line, Open must advance a reader
	// position by consumed byte length.
	//
	// If Open has not been able to parse the current line, Open should returns
	// (nil, NoChildren). If Open has been able to parse the current line, Open
	// should returns a new Block node and returns HasChildren or NoChildren.
	Open(parent ast.Node, reader text.Reader, pc Context) (ast.Node, State)

	// Continue parses the current line and returns a result of parsing.
	//
	// Continue must not parse beyond the current line.
	// If Continue has been able to parse the current line, Continue must advance
	// a reader position by consumed byte length.
	//
	// If Continue has not been able to parse the current line, Continue should
	// returns Close. If Continue has been able to parse the current line,
	// Continue should returns (Continue | NoChildren) or
	// (Continue | HasChildren)
	Continue(node ast.Node, reader text.Reader, pc Context) State

	// Close will be called when the parser returns Close.
	Close(node ast.Node, reader text.Reader, pc Context)

	// CanInterruptParagraph returns true if the parser can interrupt paragraphs,
	// otherwise false.
	CanInterruptParagraph() bool

	// CanAcceptIndentedLine returns true if the parser can open new node when
	// the given line is being indented more than 3 spaces.
	CanAcceptIndentedLine() bool
}

A BlockParser interface parses a block level element like Paragraph, List, Blockquote etc.

func NewATXHeadingParser 

func NewATXHeadingParser(opts ...HeadingOption) BlockParser

NewATXHeadingParser return a new BlockParser that can parse ATX headings.

func NewBlockquoteParser 

func NewBlockquoteParser() BlockParser

NewBlockquoteParser returns a new BlockParser that parses blockquotes.

func NewCodeBlockParser 

func NewCodeBlockParser() BlockParser

NewCodeBlockParser returns a new BlockParser that parses code blocks.

func NewFencedCodeBlockParser 

func NewFencedCodeBlockParser() BlockParser

NewFencedCodeBlockParser returns a new BlockParser that parses fenced code blocks.

func NewHTMLBlockParser 

func NewHTMLBlockParser() BlockParser

NewHTMLBlockParser return a new BlockParser that can parse html blocks.

func NewListItemParser 

func NewListItemParser() BlockParser

NewListItemParser returns a new BlockParser that parses list items.

func NewListParser 

func NewListParser() BlockParser

NewListParser returns a new BlockParser that parses lists. This parser must take precedence over the ListItemParser.

func NewParagraphParser 

func NewParagraphParser() BlockParser

NewParagraphParser returns a new BlockParser that parses paragraphs.

func NewSetextHeadingParser 

func NewSetextHeadingParser(opts ...HeadingOption) BlockParser

NewSetextHeadingParser return a new BlockParser that can parse Setext headings.

func NewThematicBreakParser 

func NewThematicBreakParser() BlockParser

NewThematicBreakParser returns a new BlockParser that parses thematic breaks.

type CloseBlocker 

type CloseBlocker interface {
	// CloseBlock will be called when a block is closed.
	CloseBlock(parent ast.Node, block text.Reader, pc Context)
}

A CloseBlocker interface is a callback function that will be called when block is closed in the inline parsing.

type Config 

type Config struct {
	// IDGenerator is a custom IDGenerator for element id generation.
	IDGenerator IDGenerator

	// EscapedSpace indicates that a '\' escaped half-space(0x20) should not trigger parsers.
	// nil means the option was not set via NewParser/AddOptions.
	EscapedSpace bool

	// Attribute indicates that custom attributes are enabled.
	// nil means the option was not set via NewParser/AddOptions.
	Attribute bool
	// contains filtered or unexported fields
}

A Config struct is a data structure that holds configuration of the Parser.

type Context 

type Context interface {
	// String implements Stringer.
	String() string

	// Get returns a value associated with the given key.
	Get(ContextKey) any

	// ComputeIfAbsent computes a value if a value associated with the given key is absent and returns the value.
	ComputeIfAbsent(ContextKey, func() any) any

	// Set sets the given value to the context.
	Set(ContextKey, any)

	// AddLinkDefinition adds the given link definition to this context.
	AddLinkDefinition(LinkDefinition)

	// LinkDefinition returns (a link definition, true) if a link definition associated with
	// the given label exists, otherwise (nil, false).
	LinkDefinition(label string) (LinkDefinition, bool)

	// LinkDefinitions returns a list of link definitions.
	LinkDefinitions() []LinkDefinition

	// IDs returns a collection of the element ids.
	IDs() *IDs

	// BlockOffset returns a first non-space character position on current line.
	// This value is valid only for BlockParser.Open.
	// BlockOffset returns -1 if current line is blank.
	BlockOffset() int

	// SetBlockOffset sets a first non-space character position on current line.
	// This value is valid only for BlockParser.Open.
	SetBlockOffset(int)

	// BlockIndent returns an indent width on current line.
	// This value is valid only for BlockParser.Open.
	// BlockIndent returns -1 if current line is blank.
	BlockIndent() int

	// SetBlockIndent sets an indent width on current line.
	// This value is valid only for BlockParser.Open.
	SetBlockIndent(int)

	// FirstDelimiter returns a first delimiter of the current delimiter list.
	FirstDelimiter() *Delimiter

	// LastDelimiter returns a last delimiter of the current delimiter list.
	LastDelimiter() *Delimiter

	// PushDelimiter appends the given delimiter to the tail of the current
	// delimiter list.
	PushDelimiter(delimiter *Delimiter)

	// RemoveDelimiter removes the given delimiter from the current delimiter list.
	RemoveDelimiter(d *Delimiter)

	// ClearDelimiters clears the current delimiter list.
	ClearDelimiters(bottom ast.Node)

	// OpenedBlocks returns a list of nodes that are currently in parsing.
	OpenedBlocks() []Block

	// SetOpenedBlocks sets a list of nodes that are currently in parsing.
	SetOpenedBlocks([]Block)

	// LastOpenedBlock returns a last node that is currently in parsing.
	LastOpenedBlock() Block

	// IsInLinkLabel returns true if current position seems to be in link label.
	IsInLinkLabel() bool
}

A Context interface holds a information that are necessary to parse Markdown text.

func NewContext 

func NewContext(opts ...ContextOption) Context

NewContext returns a new Context. By default, a new IDs with the default IDGenerator is used. Use WithIDGenerator as a ContextOption to customize ID generation.

type ContextKey 

type ContextKey int

ContextKey is a key that is used to set arbitrary values to the context. 

var ContextKeyMax ContextKey

ContextKeyMax is a maximum value of the ContextKey.

func NewContextKey 

func NewContextKey() ContextKey

NewContextKey return a new ContextKey value.

type ContextOption 

type ContextOption interface {
	SetContextOption(*contextConfig)
}

A ContextOption is an interface for options that can be passed to NewContext.

type Delimiter 

type Delimiter struct {
	ast.BaseInline

	Segment text.Segment

	// CanOpen is set true if this delimiter can open a span for a new node.
	// See https://spec.commonmark.org/0.30/#can-open-emphasis for details.
	CanOpen bool

	// CanClose is set true if this delimiter can close a span for a new node.
	// See https://spec.commonmark.org/0.30/#can-open-emphasis for details.
	CanClose bool

	// Length is a remaining length of this delimiter.
	Length int

	// OriginalLength is a original length of this delimiter.
	OriginalLength int

	// Char is a character of this delimiter.
	Char byte

	// PreviousDelimiter is a previous sibling delimiter node of this delimiter.
	PreviousDelimiter *Delimiter

	// NextDelimiter is a next sibling delimiter node of this delimiter.
	NextDelimiter *Delimiter

	// Processor is a DelimiterProcessor associated with this delimiter.
	Processor DelimiterProcessor
}

A Delimiter struct represents a delimiter like '*' of the Markdown text.

func NewDelimiter 

func NewDelimiter(canOpen, canClose bool, length int, char byte, processor DelimiterProcessor) *Delimiter

NewDelimiter returns a new Delimiter node.

func ParseDelimiter 

func ParseDelimiter(block text.Reader, minimum int, processor DelimiterProcessor, pc Context) *Delimiter

ParseDelimiter scans a delimiter from block, and if found sets its segment, advances the reader, pushes it onto the delimiter stack, and returns it.

func (*Delimiter) CalcComsumption 

func (d *Delimiter) CalcComsumption(closer *Delimiter) int

CalcComsumption calculates how many characters should be used for opening a new span correspond to given closer.

func (*Delimiter) ConsumeCharacters 

func (d *Delimiter) ConsumeCharacters(n int)

ConsumeCharacters consumes delimiters.

func (*Delimiter) Dump 

func (d *Delimiter) Dump(source []byte) *ast.NodeDump

Dump implements Node.Dump.

func (*Delimiter) Inline 

func (d *Delimiter) Inline()

Inline implements Inline.Inline.

func (*Delimiter) Kind 

func (d *Delimiter) Kind() ast.NodeKind

Kind implements Node.Kind.

func (*Delimiter) Text 

func (d *Delimiter) Text(source []byte) []byte

Text implements Node.Text.

type DelimiterProcessor 

type DelimiterProcessor interface {
	// IsDelimiter returns true if given character is a delimiter, otherwise false.
	IsDelimiter(byte) bool

	// CanOpenCloser returns true if given opener can close given closer, otherwise false.
	CanOpenCloser(opener, closer *Delimiter) bool

	// OnMatch will be called when new matched delimiter found.
	// OnMatch should return a new Node correspond to the matched delimiter.
	OnMatch(consumes int) ast.Node
}

A DelimiterProcessor interface provides a set of functions about Delimiter nodes.

type Extension 

type Extension interface {
	// ParserOptions returns a list of parser options to be applied to the parser.
	ParserOptions(*Config) []Option
}

Extension is an interface that represents an extension for the parser.

func NewCommonMark 

func NewCommonMark(opts ...Option) Extension

NewCommonMark returns a new CommonMark extension.

type HeadingConfig 

type HeadingConfig struct {
	// contains filtered or unexported fields
}

A HeadingConfig struct is a data structure that holds configuration of the renderers related to headings.

type HeadingOption 

type HeadingOption interface {
	// contains filtered or unexported methods
}

A HeadingOption interface sets options for heading parsers.

type IDGenerator 

type IDGenerator interface {
	// Generate generates a base element id for the given value and node kind.
	Generate(value []byte, kind ast.NodeKind) []byte
}

An IDGenerator generates element IDs from a value and node kind. Implementations should return a base ID; uniqueness is handled by IDs.

type IDs 

type IDs struct {
	// contains filtered or unexported fields
}

IDs is a collection of element ids, tracking uniqueness across a parse.

func NewIDs 

func NewIDs(opts ...IDsOption) *IDs

NewIDs returns a new IDs. By default, the default IDGenerator is used. Use WithIDGenerator as an IDsOption to customize ID generation.

func (*IDs) Generate 

func (s *IDs) Generate(value []byte, kind ast.NodeKind) []byte

Generate generates a unique element id for the given value and node kind. If the base id from the generator is already used, a numeric suffix is appended.

func (*IDs) Put 

func (s *IDs) Put(value []byte)

Put marks the given element id as used.

type IDsOption 

type IDsOption interface {
	SetIDsOption(*idsConfig)
}

An IDsOption is an interface for options that can be passed to NewIDs.

type InlineParser 

type InlineParser interface {
	// Trigger returns a list of characters that triggers Parse method of
	// this parser.
	// Trigger characters must be a punctuation or a halfspace.
	// Halfspaces triggers this parser when character is any spaces characters or
	// a head of line
	Trigger() []byte

	// Parse parse the given block into an inline node.
	//
	// Parse can parse beyond the current line.
	// If Parse has been able to parse the current line, it must advance a reader
	// position by consumed byte length.
	Parse(parent ast.Node, block text.Reader, pc Context) ast.Node
}

An InlineParser interface parses an inline level element like CodeSpan, Link etc.

func NewAutoLinkParser 

func NewAutoLinkParser() InlineParser

NewAutoLinkParser returns a new InlineParser that parses autolinks surrounded by '<' and '>' .

func NewCodeSpanParser 

func NewCodeSpanParser() InlineParser

NewCodeSpanParser return a new InlineParser that parses inline codes surrounded by '`' .

func NewEmphasisParser 

func NewEmphasisParser() InlineParser

NewEmphasisParser return a new InlineParser that parses emphasises.

func NewLinkParser 

func NewLinkParser() InlineParser

NewLinkParser return a new InlineParser that parses links.

func NewRawHTMLParser 

func NewRawHTMLParser() InlineParser

NewRawHTMLParser return a new InlineParser that can parse inline htmls.

type LinkDefinition 

type LinkDefinition interface {
	// String implements Stringer.
	String() string

	// Label returns a label of the link definition.
	Label() []byte

	// Destination returns a destination(URL) of the link definition.
	Destination() []byte

	// Title returns a title of the link definition.
	Title() []byte
}

A LinkDefinition interface represents a link definition in Markdown text.

func NewLinkDefinition 

func NewLinkDefinition(label, destination, title []byte) LinkDefinition

NewLinkDefinition returns a new LinkDefinition.

type Option 

type Option interface {
	SetParserOption(*Config)
}

An Option interface is a functional option type for the Parser.

func WithASTTransformers 

func WithASTTransformers(ps ...util.PrioritizedValue[ASTTransformer]) Option

WithASTTransformers is a functional option that allow you to add ASTTransformers to the parser.

func WithBlockParsers 

func WithBlockParsers(bs ...util.PrioritizedValue[BlockParser]) Option

WithBlockParsers is a functional option that allow you to add BlockParsers to the parser.

func WithDefaultParsers 

func WithDefaultParsers(v bool) Option

WithDefaultParsers is a functional option that indicates whether default parsers should be used.

func WithEscapedSpace 

func WithEscapedSpace() Option

WithEscapedSpace is a functional option indicates that a '\' escaped half-space(0x20) should not trigger parsers.

func WithExtensions 

func WithExtensions(ext ...Extension) Option

WithExtensions is a functional option that allows you to add extensions to the parser.

func WithInlineParsers 

func WithInlineParsers(bs ...util.PrioritizedValue[InlineParser]) Option

WithInlineParsers is a functional option that allow you to add InlineParsers to the parser.

func WithParagraphTransformers 

func WithParagraphTransformers(ps ...util.PrioritizedValue[ParagraphTransformer]) Option

WithParagraphTransformers is a functional option that allow you to add ParagraphTransformers to the parser.

type ParagraphTransformer 

type ParagraphTransformer interface {
	// Transform transforms the given paragraph.
	Transform(node *ast.Paragraph, reader text.Reader, pc Context)
}

A ParagraphTransformer transforms parsed Paragraph nodes. For example, link references are searched in parsed Paragraphs. 

var LinkReferenceParagraphTransformer ParagraphTransformer = &linkReferenceParagraphTransformer{}

LinkReferenceParagraphTransformer is a ParagraphTransformer implementation that parses and extracts link reference from paragraphs.

type Parser 

type Parser interface {
	// Parse parses the given Markdown text into AST nodes.
	Parse(source []byte) ast.Node

	// ParseStringSource is a helper function that parses a string source into AST nodes using the given parser.
	//
	// This function converts the string source into a read-only byte slice without copying the data, and then
	// calls the Parse method of the provided parser.
	ParseStringSource(source string) ast.Node
}

A Parser interface parses Markdown text into AST nodes.

func New 

func New(options ...Option) Parser

New returns a new Parser with given options.

type State 

type State int

State represents parser's state. State is designed to use as a bit flag. 

const (
	// None is a default value of the [State].
	None State = 1 << iota

	// Continue indicates parser can continue parsing.
	Continue

	// Close indicates parser cannot parse anymore.
	Close

	// HasChildren indicates parser may have child blocks.
	HasChildren

	// NoChildren indicates parser does not have child blocks.
	NoChildren

	// RequireParagraph indicates parser requires that the last node
	// must be a paragraph and is not converted to other nodes by
	// ParagraphTransformers.
	RequireParagraph
)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.