README
¶
Message Styling Tests
This package was written in part to act as a reference implementation for
XEP-0393: Message Styling.
To this end, the tests are designed to be exportable to a language agnostic
format.
For more information, see the file export_test.go.
To export the tests run the following in this directory:
go test -tags export
This will result in the creation of the file decoder_tests.json.
This file will be a JSON array of test cases.
Each case has the following properties:
- "Name": a string description of the test
- "Input": the message styling string to parse
- "Tokens": an array of tokens that result from parsing the string
Each token has the following properties:
- Mask: a numeric bitmask containing all of the styles applied to the token
- Data: the subslice of the input string that was detected as a token
- Info: any info string present at the start of pre-formatted blocks
- Quote: the numeric quotation depth of the token if inside a block quote
The values for "Mask" can be found in the "constants" section of the package documentation.
Known Limitations
The bitmask contains several styles, such as BlockPreStart, that are not part
of the specification and may not be used by all implementations.
These are to mark the start and end of spans and blocks and may be ignored if
your implementation does not differentiate them.
Long plain text spans may also be broken up at arbitrary intervals depending on the parser buffer length. For example, the string "one two" could be a single token "one two" or it could be broken up into the tokens "one " and "two" or "one" and " two" or even "on" and "e two" by the tests. These test cases should easily fit within any reasonable buffer, but if your implementation uses a smaller buffer size or breaks up long spans differently you may have to account for this when running these test cases.
Documentation
¶
Overview ¶
Package styling implements XEP-0393: Message Styling, a simple styling language.
This package does not convert message styling documents into a format usable by any other rendering engine (ie. HTML or LaTeX), instead it tokenizes the input and provides you with a bitmask of styles that should be applied to each token.
Format ¶
Message Styling borrows several familiar formatting options from Markdown (though it is much simpler and not compatible with Markdown parsers).
Several inline formats exist:
A string wrapped in _emph_ should be displayed emphasized (normally in italics), and strings wrapped in *strong* will have strong emphasis (normally bold). The `pre` styling directive results in an inline pre-formatted string (monospace with no child formatting allowed), and the ~strike~ style should result in text with a line through the middle (strike through).
There are also a few block formats such as a block quotation:
> A quotation >> Can be multi-level
And a preformatted text block which can have no child blocks, should be displayed in a monospace font, and which should not be re-wrapped:
```an optional tag may go here. Lines of pre-formatted text go here. ```
Example (Html) ¶
package main
import (
"fmt"
"html"
"io"
"strings"
"mellium.im/xmpp/styling"
)
func main() {
r := strings.NewReader(`The full title is
_Twelfth Night, or What You Will_
but *most* people shorten it.`)
d := styling.NewDecoder(r)
var out strings.Builder
out.WriteString("<!doctype HTML>\n")
for d.Next() {
tok := d.Token()
switch {
case tok.Mask&styling.SpanEmphStart == styling.SpanEmphStart:
out.WriteString("<em><code>")
out.Write(tok.Data)
out.WriteString("</code>")
case tok.Mask&styling.SpanStrongStart == styling.SpanStrongStart:
out.WriteString("<strong><code>")
out.Write(tok.Data)
out.WriteString("</code>")
case tok.Mask&styling.SpanEmphEnd == styling.SpanEmphEnd:
out.WriteString("<code>")
out.Write(tok.Data)
out.WriteString("</code></em>")
case tok.Mask&styling.SpanStrongEnd == styling.SpanStrongEnd:
out.WriteString("<code>")
out.Write(tok.Data)
out.WriteString("</code></strong>")
// TODO: no other styles implemented to keep the example short.
default:
out.WriteString(html.EscapeString(string(tok.Data)))
}
}
err := d.Err()
if err != nil && err != io.EOF {
out.WriteString("<mark>")
out.WriteString(html.EscapeString(fmt.Sprintf("Error encountered while parsing tokens: %v", err)))
out.WriteString("</mark>")
}
fmt.Println(out.String())
}
Output: <!doctype HTML> The full title is <em><code>_</code>Twelfth Night, or What You Will<code>_</code></em> but <strong><code>*</code>most<code>*</code></strong> people shorten it.
Index ¶
Examples ¶
Constants ¶
const ( // BlockPre represents a preformatted text block. // It should be displayed in a monospace font with no change to line breaks. BlockPre Style = 1 << iota // BlockQuote represents a nestable quotation. // To get the level of the quotation see the Quote method. BlockQuote // SpanEmph is an inline span of text that should be displayed in italics. SpanEmph // SpanStrong is an inline span of text that should be displayed bold. SpanStrong // SpanStrike is an inline span of text that should be displayed with a // horizontal line through the middle (strike through). SpanStrike // SpanPre is an inline span of text that should be displayed in a monospace // font. SpanPre // Styling directive markers. // It is often desirable to distinguish the characters that triggered styling // from surrounding text. These bits are set only on styling directives, the // characters or sequences of characters that result in the style changing. // The corresponding style bit will also be set whenever the start or end bits // are set. For example, in *strong* the first "*" will have // SpanStrong|SpanStrongStart set, the "strong" will only have SpanStrong set, // and the last "*" will have SpanStrong|SpanStrongEnd set. BlockPreStart BlockPreEnd BlockQuoteStart BlockQuoteEnd SpanEmphStart SpanEmphEnd SpanStrongStart SpanStrongEnd SpanStrikeStart SpanStrikeEnd SpanPreStart SpanPreEnd // Various useful masks // These bitmasks are provided as a convenience to make it easy to check what // general category of styles are applied. Block = BlockPre | BlockQuote Span = SpanEmph | SpanStrong | SpanStrike | SpanPre SpanEmphDirective = SpanEmphStart | SpanEmphEnd SpanStrongDirective = SpanStrongStart | SpanStrongEnd SpanStrikeDirective = SpanStrikeStart | SpanStrikeEnd SpanPreDirective = SpanPreStart | SpanPreEnd SpanDirective = SpanEmphDirective | SpanStrongDirective | SpanStrikeDirective | SpanPreDirective SpanStartDirective = SpanEmphStart | SpanStrongStart | SpanStrikeStart | SpanPreStart SpanEndDirective = SpanEmphEnd | SpanStrongEnd | SpanStrikeEnd | SpanPreEnd BlockPreDirective = BlockPreStart | BlockPreEnd BlockQuoteDirective = BlockQuoteStart | BlockQuoteEnd BlockDirective = BlockPreDirective | BlockQuoteDirective BlockStartDirective = BlockPreStart | BlockQuoteStart BlockEndDirective = BlockPreEnd | BlockQuoteEnd Directive = SpanDirective | BlockDirective StartDirective = SpanStartDirective | BlockStartDirective EndDirective = SpanEndDirective | BlockEndDirective )
The style bits.
const (
// NS is the message styling namespace, exported as a convenience.
NS = "urn:xmpp:styling:0"
)
Variables ¶
var (
Feature = info.Feature{Var: NS}
)
A list of service discovery features that are supported by this package.
Functions ¶
func Disable ¶ added in v0.18.0
func Disable(r xml.TokenReader) xml.TokenReader
Disable is an xmlstream.Transformer that inserts a hint into any message read through r that disables styling for the body of the message.
func Scan ¶
Scan returns a new stateful split function for a bufio.Scanner that splits on message styling tokens. This function is different from a Decoder in that it will not return block quote end tokens.
This is a low-level building block of this package. Most users will want to use a Decoder instead.
Types ¶
type Decoder ¶
type Decoder struct {
// contains filtered or unexported fields
}
A Decoder represents a styling lexer reading a particular input stream. The parser assumes that input is encoded in UTF-8.
func NewDecoder ¶
NewDecoder creates a new styling parser reading from r. If r does not implement io.ByteReader, NewDecoder will do its own buffering.
func (*Decoder) Err ¶ added in v0.20.0
Err returns the error, if any, that was encountered while reading the input stream. In case of EOF, Err returns io.EOF
func (*Decoder) Next ¶ added in v0.20.0
Next prepares the next styling token in the input stream for reading with the Token method. It returns true on success, or false if end of input stream is reached or an error happened while preparing it. Err should be consulted to distinguish between the two cases. Every call to Token, even the first one, must be preceded by a call to Next.
func (*Decoder) SkipBlock ¶
SkipBlock pops tokens from the decoder until it reaches the end of the current block, positioning the token stream at the beginning of the next block. It returns true if it succeeds without errors, false otherwise Err should be consulted to check for errors. If SkipBlock is called at the beginning of the input stream before any tokens have been popped or any blocks have been entered, it will skip the entire input stream as if everything were contained in an imaginary "root" block.
func (*Decoder) SkipSpan ¶
SkipSpan pops tokens from the decoder until it reaches the end of the current span, positioning the token stream at the beginning of the next span or block. If SkipSpan is called while no span is entered it behaves like SkipBlock. It returns true if it succeeds without errors, false otherwise Err should be consulted to check for errors.
func (*Decoder) Style ¶
Style returns a bitmask representing the currently applied styles at the current position in the document.
func (*Decoder) Token ¶
Token returns the next styling token in the input stream. Returned tokens do not always correspond directly to the input stream. For example, at the end of a block quote an empty token is returned with the mask BlockQuote|BlockQuoteEnd, but there is no explicit block quote terminator character in the input stream so its data will be empty.
Slices of bytes in the returned token data refer to the parser's internal buffer and remain valid only until the next call to Token. To acquire a copy of the bytes call the token's Copy method.
type Style ¶
type Style uint32
Style is a bitmask that represents a set of styles that can be applied to text.
type Token ¶
A Token represents a styling directive or unstyled span of text. If the token is a preformatted text block start token, Info is a slice of the bytes between the code fence (```) and the newline.
type Unstyled ¶ added in v0.18.0
Unstyled is a type that can be added to messages to add a hint that will disable styling. When unmarshaled or marshaled its value indicates whether the unstyled hint was or will be present in the message.
func (Unstyled) MarshalXML ¶ added in v0.18.0
MarshalXML implements xml.Marshaler.
func (Unstyled) TokenReader ¶ added in v0.18.0
func (u Unstyled) TokenReader() xml.TokenReader
TokenReader implements xmlstream.Marshaler.
func (*Unstyled) UnmarshalXML ¶ added in v0.18.0
UnmarshalXML implements xml.Unmarshaler.
Source Files