utf8

Documentation

Overview

Package utf8 implements functions and constants to support text encoded in UTF-8. This package calls a Unicode character a rune for brevity.

Index

• Constants
• func DecodeLastRune(p []byte) (rune, size int)
• func DecodeLastRuneInString(s string) (rune, size int)
• func DecodeRune(p []byte) (rune, size int)
• func DecodeRuneInString(s string) (rune, size int)
• func EncodeRune(p []byte, rune int) int
• func FullRune(p []byte) bool
• func FullRuneInString(s string) bool
• func RuneCount(p []byte) int
• func RuneCountInString(s string) (n int)
• func RuneLen(rune int) int
• func RuneStart(b byte) bool
• type String
  • func NewString(contents string) *String
  • func (s *String) At(i int) int
  • func (s *String) Init(contents string) *String
  • func (s *String) IsASCII() bool
  • func (s *String) RuneCount() int
  • func (s *String) Slice(i, j int) string
  • func (s *String) String() string

Constants

const (
	RuneError = unicode.ReplacementChar // the "error" Rune or "replacement character".
	RuneSelf  = 0x80                    // characters below Runeself are represented as themselves in a single byte.
	UTFMax    = 4                       // maximum number of bytes of a UTF-8 encoded Unicode character.
)

Numbers fundamental to the encoding.

Functions

func DecodeLastRune

func DecodeLastRune(p []byte) (rune, size int)

DecodeLastRune unpacks the last UTF-8 encoding in p and returns the rune and its width in bytes.

func DecodeLastRuneInString

func DecodeLastRuneInString(s string) (rune, size int)

DecodeLastRuneInString is like DecodeLastRune but its input is a string.

func DecodeRune

func DecodeRune(p []byte) (rune, size int)

DecodeRune unpacks the first UTF-8 encoding in p and returns the rune and its width in bytes.

func DecodeRuneInString

func DecodeRuneInString(s string) (rune, size int)

DecodeRuneInString is like DecodeRune but its input is a string.

func EncodeRune

func EncodeRune(p []byte, rune int) int

EncodeRune writes into p (which must be large enough) the UTF-8 encoding of the rune. It returns the number of bytes written.

func FullRune

func FullRune(p []byte) bool

FullRune reports whether the bytes in p begin with a full UTF-8 encoding of a rune. An invalid encoding is considered a full Rune since it will convert as a width-1 error rune.

func FullRuneInString

func FullRuneInString(s string) bool

FullRuneInString is like FullRune but its input is a string.

func RuneCount

func RuneCount(p []byte) int

RuneCount returns the number of runes in p. Erroneous and short encodings are treated as single runes of width 1 byte.

func RuneCountInString

func RuneCountInString(s string) (n int)

RuneCountInString is like RuneCount but its input is a string.

func RuneLen

func RuneLen(rune int) int

RuneLen returns the number of bytes required to encode the rune.

func RuneStart

func RuneStart(b byte) bool

RuneStart reports whether the byte could be the first byte of an encoded rune. Second and subsequent bytes always have the top two bits set to 10.

Types

type String

type String struct {
	// contains filtered or unexported fields
}

String wraps a regular string with a small structure that provides more efficient indexing by code point index, as opposed to byte index. Scanning incrementally forwards or backwards is O(1) per index operation (although not as fast a range clause going forwards). Random access is O(N) in the length of the string, but the overhead is less than always scanning from the beginning. If the string is ASCII, random access is O(1). Unlike the built-in string type, String has internal mutable state and is not thread-safe.

func NewString

func NewString(contents string) *String

NewString returns a new UTF-8 string with the provided contents.

func (*String) At

func (s *String) At(i int) int

At returns the rune with index i in the String. The sequence of runes is the same as iterating over the contents with a "for range" clause.

func (*String) Init

func (s *String) Init(contents string) *String

Init initializes an existing String to hold the provided contents. It returns a pointer to the initialized String.

func (*String) IsASCII

func (s *String) IsASCII() bool

IsASCII returns a boolean indicating whether the String contains only ASCII bytes.

func (*String) RuneCount

func (s *String) RuneCount() int

RuneCount returns the number of runes (Unicode code points) in the String.

func (*String) Slice

func (s *String) Slice(i, j int) string

Slice returns the string sliced at rune positions [i:j].

func (*String) String

func (s *String) String() string

String returns the contents of the String. This method also means the String is directly printable by fmt.Print.