v0.21.4 Latest Latest

This package is not in the latest version of its module.

Go to latest
Published: Jan 11, 2023 License: BSD-2-Clause Imports: 10 Imported by: 1


Message Styling Tests

This package was written in part to act as a reference implementation for XEP-0393: Message Styling. To this end, the tests are designed to be exportable to a language agnostic format. For more information, see the file export_test.go. To export the tests run the following in this directory:

go test -tags export

This will result in the creation of the file decoder_tests.json. This file will be a JSON array of test cases. Each case has the following properties:

  • "Name": a string description of the test
  • "Input": the message styling string to parse
  • "Tokens": an array of tokens that result from parsing the string

Each token has the following properties:

  • Mask: a numeric bitmask containing all of the styles applied to the token
  • Data: the subslice of the input string that was detected as a token
  • Info: any info string present at the start of pre-formatted blocks
  • Quote: the numeric quotation depth of the token if inside a block quote

The values for "Mask" can be found in the "constants" section of the package documentation.

Known Limitations

The bitmask contains several styles, such as BlockPreStart, that are not part of the specification and may not be used by all implementations. These are to mark the start and end of spans and blocks and may be ignored if your implementation does not differentiate them.

Long plain text spans may also be broken up at arbitrary intervals depending on the parser buffer length. For example, the string "one two" could be a single token "one two" or it could be broken up into the tokens "one " and "two" or "one" and " two" or even "on" and "e two" by the tests. These test cases should easily fit within any reasonable buffer, but if your implementation uses a smaller buffer size or breaks up long spans differently you may have to account for this when running these test cases.



Package styling implements XEP-0393: Message Styling, a simple styling language.

This package does not convert message styling documents into a format usable by any other rendering engine (ie. HTML or LaTeX), instead it tokenizes the input and provides you with a bitmask of styles that should be applied to each token.


Message Styling borrows several familiar formatting options from Markdown (though it is much simpler and not compatible with Markdown parsers).

Several inline formats exist:

A string wrapped in _emph_ should be displayed emphasized (normally in italics), and strings wrapped in *strong* will have strong emphasis (normally bold). The `pre` styling directive results in an inline pre-formatted string (monospace with no child formatting allowed), and the ~strike~ style should result in text with a line through the middle (strike through).

There are also a few block formats such as a block quotation:

> A quotation
>> Can be multi-level

And a preformatted text block which can have no child blocks, should be displayed in a monospace font, and which should not be re-wrapped:

```an optional tag may go here.
Lines of pre-formatted text go here.
Example (Html)
package main

import (


func main() {
	r := strings.NewReader(`The full title is
_Twelfth Night, or What You Will_
but *most* people shorten it.`)
	d := styling.NewDecoder(r)

	var out strings.Builder
	out.WriteString("<!doctype HTML>\n")
	for d.Next() {
		tok := d.Token()

		switch {
		case tok.Mask&styling.SpanEmphStart == styling.SpanEmphStart:
		case tok.Mask&styling.SpanStrongStart == styling.SpanStrongStart:
		case tok.Mask&styling.SpanEmphEnd == styling.SpanEmphEnd:
		case tok.Mask&styling.SpanStrongEnd == styling.SpanStrongEnd:
			// TODO: no other styles implemented to keep the example short.

	err := d.Err()
	if err != nil && err != io.EOF {
		out.WriteString(html.EscapeString(fmt.Sprintf("Error encountered while parsing tokens: %v", err)))


<!doctype HTML>
The full title is
<em><code>_</code>Twelfth Night, or What You Will<code>_</code></em>
but <strong><code>*</code>most<code>*</code></strong> people shorten it.




View Source
const (
	// BlockPre represents a preformatted text block.
	// It should be displayed in a monospace font with no change to line breaks.
	BlockPre Style = 1 << iota

	// BlockQuote represents a nestable quotation.
	// To get the level of the quotation see the Quote method.

	// SpanEmph is an inline span of text that should be displayed in italics.

	// SpanStrong is an inline span of text that should be displayed bold.

	// SpanStrike is an inline span of text that should be displayed with a
	// horizontal line through the middle (strike through).

	// SpanPre is an inline span of text that should be displayed in a monospace
	// font.

	// Styling directive markers.
	// It is often desirable to distinguish the characters that triggered styling
	// from surrounding text. These bits are set only on styling directives, the
	// characters or sequences of characters that result in the style changing.
	// The corresponding style bit will also be set whenever the start or end bits
	// are set. For example, in *strong* the first "*" will have
	// SpanStrong|SpanStrongStart set, the "strong" will only have SpanStrong set,
	// and the last "*" will have SpanStrong|SpanStrongEnd set.

	// Various useful masks
	// These bitmasks are provided as a convenience to make it easy to check what
	// general category of styles are applied.
	Block               = BlockPre | BlockQuote
	Span                = SpanEmph | SpanStrong | SpanStrike | SpanPre
	SpanEmphDirective   = SpanEmphStart | SpanEmphEnd
	SpanStrongDirective = SpanStrongStart | SpanStrongEnd
	SpanStrikeDirective = SpanStrikeStart | SpanStrikeEnd
	SpanPreDirective    = SpanPreStart | SpanPreEnd
	SpanDirective       = SpanEmphDirective | SpanStrongDirective | SpanStrikeDirective | SpanPreDirective
	SpanStartDirective  = SpanEmphStart | SpanStrongStart | SpanStrikeStart | SpanPreStart
	SpanEndDirective    = SpanEmphEnd | SpanStrongEnd | SpanStrikeEnd | SpanPreEnd
	BlockPreDirective   = BlockPreStart | BlockPreEnd
	BlockQuoteDirective = BlockQuoteStart | BlockQuoteEnd
	BlockDirective      = BlockPreDirective | BlockQuoteDirective
	BlockStartDirective = BlockPreStart | BlockQuoteStart
	BlockEndDirective   = BlockPreEnd | BlockQuoteEnd
	Directive           = SpanDirective | BlockDirective
	StartDirective      = SpanStartDirective | BlockStartDirective
	EndDirective        = SpanEndDirective | BlockEndDirective

The style bits.

View Source
const (
	// NS is the message styling namespace, exported as a convenience.
	NS = "urn:xmpp:styling:0"


View Source
var (
	Feature = info.Feature{Var: NS}

A list of service discovery features that are supported by this package.


func Disable added in v0.18.0

func Disable(r xml.TokenReader) xml.TokenReader

Disable is an xmlstream.Transformer that inserts a hint into any message read through r that disables styling for the body of the message.

func Scan

func Scan() bufio.SplitFunc

Scan returns a new stateful split function for a bufio.Scanner that splits on message styling tokens. This function is different from a Decoder in that it will not return block quote end tokens.

This is a low-level building block of this package. Most users will want to use a Decoder instead.


type Decoder

type Decoder struct {
	// contains filtered or unexported fields

A Decoder represents a styling lexer reading a particular input stream. The parser assumes that input is encoded in UTF-8.

func NewDecoder

func NewDecoder(r io.Reader) *Decoder

NewDecoder creates a new styling parser reading from r. If r does not implement io.ByteReader, NewDecoder will do its own buffering.

func (*Decoder) Err added in v0.20.0

func (d *Decoder) Err() error

Err returns the error, if any, that was encountered while reading the input stream. In case of EOF, Err returns io.EOF

func (*Decoder) Next added in v0.20.0

func (d *Decoder) Next() bool

Next prepares the next styling token in the input stream for reading with the Token method. It returns true on success, or false if end of input stream is reached or an error happened while preparing it. Err should be consulted to distinguish between the two cases. Every call to Token, even the first one, must be preceded by a call to Next.

func (*Decoder) Quote

func (d *Decoder) Quote() uint

Quote is the blockquote depth at the current position in the document.

func (*Decoder) SkipBlock

func (d *Decoder) SkipBlock() bool

SkipBlock pops tokens from the decoder until it reaches the end of the current block, positioning the token stream at the beginning of the next block. It returns true if it succeeds without errors, false otherwise Err should be consulted to check for errors. If SkipBlock is called at the beginning of the input stream before any tokens have been popped or any blocks have been entered, it will skip the entire input stream as if everything were contained in an imaginary "root" block.

func (*Decoder) SkipSpan

func (d *Decoder) SkipSpan() bool

SkipSpan pops tokens from the decoder until it reaches the end of the current span, positioning the token stream at the beginning of the next span or block. If SkipSpan is called while no span is entered it behaves like SkipBlock. It returns true if it succeeds without errors, false otherwise Err should be consulted to check for errors.

func (*Decoder) Style

func (d *Decoder) Style() Style

Style returns a bitmask representing the currently applied styles at the current position in the document.

func (*Decoder) Token

func (d *Decoder) Token() Token

Token returns the next styling token in the input stream. Returned tokens do not always correspond directly to the input stream. For example, at the end of a block quote an empty token is returned with the mask BlockQuote|BlockQuoteEnd, but there is no explicit block quote terminator character in the input stream so its data will be empty.

Slices of bytes in the returned token data refer to the parser's internal buffer and remain valid only until the next call to Token. To acquire a copy of the bytes call the token's Copy method.

type Style

type Style uint32

Style is a bitmask that represents a set of styles that can be applied to text.

func (Style) String added in v0.19.0

func (i Style) String() string

type Token

type Token struct {
	Mask Style
	Data []byte
	Info []byte

A Token represents a styling directive or unstyled span of text. If the token is a preformatted text block start token, Info is a slice of the bytes between the code fence (```) and the newline.

func (Token) Copy

func (t Token) Copy() Token

Copy creates a new copy of the token.

type Unstyled added in v0.18.0

type Unstyled struct {
	XMLName xml.Name `xml:"urn:xmpp:styling:0 unstyled"`
	Value   bool

Unstyled is a type that can be added to messages to add a hint that will disable styling. When unmarshaled or marshaled its value indicates whether the unstyled hint was or will be present in the message.

func (Unstyled) MarshalXML added in v0.18.0

func (u Unstyled) MarshalXML(e *xml.Encoder, _ xml.StartElement) error

MarshalXML implements xml.Marshaler.

func (Unstyled) TokenReader added in v0.18.0

func (u Unstyled) TokenReader() xml.TokenReader

TokenReader implements xmlstream.Marshaler.

func (*Unstyled) UnmarshalXML added in v0.18.0

func (u *Unstyled) UnmarshalXML(d *xml.Decoder, start xml.StartElement) error

UnmarshalXML implements xml.Unmarshaler.

func (Unstyled) WriteXML added in v0.18.0

func (u Unstyled) WriteXML(w xmlstream.TokenWriter) (int, error)

WriteXML implements xmlstream.WriterTo.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL