The highest tagged major version is
Documentation
¶
Overview ¶
Package buffer contains buffer and wrapper types for byte slices. It is useful for writing lexers or other high-performance byte slice handling.
The `Reader` and `Writer` types implement the `io.Reader` and `io.Writer` respectively and provide a thinner and faster interface than `bytes.Buffer`. The `Lexer` type is useful for building lexers because it keeps track of the start and end position of a byte selection, and shifts the bytes whenever a valid token is found. The `StreamLexer` does the same, but keeps a buffer pool so that it reads a limited amount at a time, allowing to parse from streaming sources.
Index ¶
- Variables
- type Lexer
- func (z *Lexer) Bytes() []byte
- func (z *Lexer) Err() error
- func (z *Lexer) Lexeme() []byte
- func (z *Lexer) Move(n int)
- func (z *Lexer) Offset() int
- func (z *Lexer) Peek(pos int) byte
- func (z *Lexer) PeekErr(pos int) error
- func (z *Lexer) PeekRune(pos int) (rune, int)
- func (z *Lexer) Pos() int
- func (z *Lexer) Restore()
- func (z *Lexer) Rewind(pos int)
- func (z *Lexer) Shift() []byte
- func (z *Lexer) Skip()
- type Reader
- type StreamLexer
- func (z *StreamLexer) Err() error
- func (z *StreamLexer) Free(n int)
- func (z *StreamLexer) Lexeme() []byte
- func (z *StreamLexer) Move(n int)
- func (z *StreamLexer) Peek(pos int) byte
- func (z *StreamLexer) PeekRune(pos int) (rune, int)
- func (z *StreamLexer) Pos() int
- func (z *StreamLexer) Rewind(pos int)
- func (z *StreamLexer) Shift() []byte
- func (z *StreamLexer) ShiftLen() int
- func (z *StreamLexer) Skip()
- type Writer
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var MinBuf = defaultBufSize
MinBuf specifies the default initial length of internal buffers. Solely here to support old versions of parse.
Functions ¶
This section is empty.
Types ¶
type Lexer ¶
type Lexer struct {
// contains filtered or unexported fields
}
Lexer is a buffered reader that allows peeking forward and shifting, taking an io.Reader. It keeps data in-memory until Free, taking a byte length, is called to move beyond the data.
func NewLexer ¶
NewLexerBytes returns a new Lexer for a given io.Reader, and uses ioutil.ReadAll to read it into a byte slice. If the io.Reader implements Bytes, that is used instead. It will append a NULL at the end of the buffer.
func NewLexerBytes ¶
NewLexerBytes returns a new Lexer for a given byte slice, and appends NULL at the end. To avoid reallocation, make sure the capacity has room for one more byte.
func (*Lexer) Err ¶
Err returns the error returned from io.Reader or io.EOF when the end has been reached.
func (*Lexer) Peek ¶
Peek returns the ith byte relative to the end position. Peek returns 0 when an error has occurred, Err returns the error.
func (*Lexer) PeekErr ¶
PeekErr returns the error at position pos. When pos is zero, this is the same as calling Err().
func (*Lexer) PeekRune ¶
PeekRune returns the rune and rune length of the ith byte relative to the end position.
func (*Lexer) Restore ¶
func (z *Lexer) Restore()
Restore restores the replaced byte past the end of the buffer by NULL.
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Reader implements an io.Reader over a byte slice.
func NewReader ¶
NewReader returns a new Reader for a given byte slice.
Example ¶
r := NewReader([]byte("Lorem ipsum"))
w := &bytes.Buffer{}
io.Copy(w, r)
fmt.Println(w.String())
Output: Lorem ipsum
type StreamLexer ¶
type StreamLexer struct {
// contains filtered or unexported fields
}
StreamLexer is a buffered reader that allows peeking forward and shifting, taking an io.Reader. It keeps data in-memory until Free, taking a byte length, is called to move beyond the data.
func NewStreamLexer ¶
func NewStreamLexer(r io.Reader) *StreamLexer
NewStreamLexer returns a new StreamLexer for a given io.Reader with a 4kB estimated buffer size. If the io.Reader implements Bytes, that buffer is used instead.
func NewStreamLexerSize ¶
func NewStreamLexerSize(r io.Reader, size int) *StreamLexer
NewStreamLexerSize returns a new StreamLexer for a given io.Reader and estimated required buffer size. If the io.Reader implements Bytes, that buffer is used instead.
func (*StreamLexer) Err ¶
func (z *StreamLexer) Err() error
Err returns the error returned from io.Reader. It may still return valid bytes for a while though.
func (*StreamLexer) Free ¶
func (z *StreamLexer) Free(n int)
Free frees up bytes of length n from previously shifted tokens. Each call to Shift should at one point be followed by a call to Free with a length returned by ShiftLen.
func (*StreamLexer) Lexeme ¶
func (z *StreamLexer) Lexeme() []byte
Lexeme returns the bytes of the current selection.
func (*StreamLexer) Peek ¶
func (z *StreamLexer) Peek(pos int) byte
Peek returns the ith byte relative to the end position and possibly does an allocation. Peek returns zero when an error has occurred, Err returns the error. TODO: inline function
func (*StreamLexer) PeekRune ¶
func (z *StreamLexer) PeekRune(pos int) (rune, int)
PeekRune returns the rune and rune length of the ith byte relative to the end position.
func (*StreamLexer) Pos ¶
func (z *StreamLexer) Pos() int
Pos returns a mark to which can be rewinded.
func (*StreamLexer) Rewind ¶
func (z *StreamLexer) Rewind(pos int)
Rewind rewinds the position to the given position.
func (*StreamLexer) Shift ¶
func (z *StreamLexer) Shift() []byte
Shift returns the bytes of the current selection and collapses the position to the end of the selection. It also returns the number of bytes we moved since the last call to Shift. This can be used in calls to Free.
func (*StreamLexer) ShiftLen ¶
func (z *StreamLexer) ShiftLen() int
ShiftLen returns the number of bytes moved since the last call to ShiftLen. This can be used in calls to Free because it takes into account multiple Shifts or Skips.
func (*StreamLexer) Skip ¶
func (z *StreamLexer) Skip()
Skip collapses the position to the end of the selection.
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Writer implements an io.Writer over a byte slice.
func NewWriter ¶
NewWriter returns a new Writer for a given byte slice.
Example ¶
w := NewWriter(make([]byte, 0, 11)) // initial buffer length is 11
w.Write([]byte("Lorem ipsum"))
fmt.Println(string(w.Bytes()))
Output: Lorem ipsum
func (*Writer) Reset ¶
func (w *Writer) Reset()
Reset empties and reuses the current buffer. Subsequent writes will overwrite the buffer, so any reference to the underlying slice is invalidated after this call.
Example ¶
w := NewWriter(make([]byte, 0, 11)) // initial buffer length is 10
w.Write([]byte("garbage that will be overwritten")) // does reallocation
w.Reset()
w.Write([]byte("Lorem ipsum"))
fmt.Println(string(w.Bytes()))
Output: Lorem ipsum
Source Files