README
¶
bytebuf
UPDATE: This implementation is deprecated as the patch is merged to the Go mainline (https://golang.org/cl/133715)
Example of how CL133375 cmd/compile/internal/gc: handle array slice self-assign in esc.go can be utilized to mitigate Go escape analysis limitations.
It shows how to fix problem described in cmd/compile, bytes: bootstrap array causes bytes.Buffer to always be heap-allocated using brand new escape analysis pattern.
Implementation difference
The whole implementation difference can be described as:
type Buffer struct {
buf []byte // contents are the bytes buf[off : len(buf)]
off int // read at &buf[off], write at &buf[len(buf)]
- bootstrap [64]byte // memory to hold first slice; helps small buffers avoid allocation.
+ bootstrap *[64]byte // memory to hold first slice; helps small buffers avoid allocation.
lastRead readOp // last read operation, so that Unread* can work correctly.
}
With updated escape analysis, it's possible to actually take benefits of
bootstrap array, but only if it's not "inlined" into Buffer object.
So, we need a pointer to array instead of normal array.
This makes it impossible to use zero value though, hence New function.
Performance comparison before bytes.Buffer optimization in CL133715
These results provided only for historical reference.
Given this code:
buf.Write(s)
globalString = buf.String()
Where s is:
| Label | Data |
|---|---|
| empty | empty string |
| 5 | 5-byte string |
| 64 | 64-byte string |
| 128 | 128-byte string |
| 1024 | 1024-byte string |
name old time/op new time/op delta
String/empty-8 138ns ±13% 24ns ± 0% -82.94% (p=0.000 n=10+8)
String/5-8 186ns ±11% 60ns ± 1% -67.82% (p=0.000 n=10+10)
String/64-8 225ns ±10% 108ns ± 6% -52.26% (p=0.000 n=10+10)
String/1024-8 889ns ± 0% 740ns ± 1% -16.78% (p=0.000 n=9+10)
name old alloc/op new alloc/op delta
String/empty-8 112B ± 0% 0B -100.00% (p=0.000 n=10+10)
String/5-8 117B ± 0% 5B ± 0% -95.73% (p=0.000 n=10+10)
String/64-8 176B ± 0% 64B ± 0% -63.64% (p=0.000 n=10+10)
String/1024-8 2.16kB ± 0% 2.05kB ± 0% -5.19% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
String/empty-8 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10)
String/5-8 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.000 n=10+10)
String/64-8 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.000 n=10+10)
String/1024-8 3.00 ± 0% 2.00 ± 0% -33.33% (p=0.000 n=10+10)
Performance comparison after bytes.Buffer optimization in CL133715
After improvements to bytes.Buffer from the standard library and
updated tests that do bytes.NewBuffer(make([]byte, 0, 64)),
results are mostly in favor of bytes.Buffer.
Also note that bytes.Buffer makes it possible to have a custom size
stack-allocated slice as a bootstrap storage, so it's overall a
better choice.
name old time/op new time/op delta
String/empty-8 23.8ns ± 0% 25.7ns ± 0% +8.04% (p=0.000 n=8+8)
String/5-8 59.6ns ± 1% 53.4ns ± 1% -10.32% (p=0.000 n=10+10)
String/64-8 102ns ±12% 95ns ± 4% -6.94% (p=0.001 n=10+8)
String/1024-8 745ns ± 0% 768ns ± 1% +3.08% (p=0.000 n=9+8)
name old alloc/op new alloc/op delta
String/empty-8 0.00B 0.00B ~ (all equal)
String/5-8 5.00B ± 0% 5.00B ± 0% ~ (all equal)
String/64-8 64.0B ± 0% 64.0B ± 0% ~ (all equal)
String/1024-8 2.05kB ± 0% 2.18kB ± 0% +6.25% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
String/empty-8 0.00 0.00 ~ (all equal)
String/5-8 1.00 ± 0% 1.00 ± 0% ~ (all equal)
String/64-8 1.00 ± 0% 1.00 ± 0% ~ (all equal)
String/1024-8 2.00 ± 0% 2.00 ± 0% ~ (all equal)
Documentation
¶
Index ¶
- Constants
- Variables
- type Buffer
- func (b *Buffer) Bytes() []byte
- func (b *Buffer) Cap() int
- func (b *Buffer) Grow(n int)
- func (b *Buffer) Len() int
- func (b *Buffer) Next(n int) []byte
- func (b *Buffer) Read(p []byte) (n int, err error)
- func (b *Buffer) ReadByte() (byte, error)
- func (b *Buffer) ReadBytes(delim byte) (line []byte, err error)
- func (b *Buffer) ReadFrom(r io.Reader) (n int64, err error)
- func (b *Buffer) ReadRune() (r rune, size int, err error)
- func (b *Buffer) ReadString(delim byte) (line string, err error)
- func (b *Buffer) Reset()
- func (b *Buffer) String() string
- func (b *Buffer) Truncate(n int)
- func (b *Buffer) UnreadByte() error
- func (b *Buffer) UnreadRune() error
- func (b *Buffer) Write(p []byte) (n int, err error)
- func (b *Buffer) WriteByte(c byte) error
- func (b *Buffer) WriteRune(r rune) (n int, err error)
- func (b *Buffer) WriteString(s string) (n int, err error)
- func (b *Buffer) WriteTo(w io.Writer) (n int64, err error)
Constants ¶
const MinRead = 512
MinRead is the minimum slice size passed to a Read call by Buffer.ReadFrom. As long as the Buffer has at least MinRead bytes beyond what is required to hold the contents of r, ReadFrom will not grow the underlying buffer.
Variables ¶
var ErrTooLarge = errors.New("bytes.Buffer: too large")
ErrTooLarge is passed to panic if memory cannot be allocated to store data in a buffer.
Functions ¶
This section is empty.
Types ¶
type Buffer ¶
type Buffer struct {
// contains filtered or unexported fields
}
A Buffer is a variable-sized buffer of bytes with Read and Write methods. Unlike bytes.Buffer, zero value is not suitable for use.
func NewBuffer ¶
NewBuffer creates and initializes a new Buffer using buf as its initial contents. The new Buffer takes ownership of buf, and the caller should not use buf after this call. NewBuffer is intended to prepare a Buffer to read existing data. It can also be used to set the initial size of the internal buffer for writing. To do that, buf should have the desired capacity but a length of zero.
In most cases, new(Buffer) (or just declaring a Buffer variable) is sufficient to initialize a Buffer.
func NewBufferString ¶
NewBufferString creates and initializes a new Buffer using string s as its initial contents. It is intended to prepare a buffer to read an existing string.
In most cases, new(Buffer) (or just declaring a Buffer variable) is sufficient to initialize a Buffer.
func (*Buffer) Bytes ¶
Bytes returns a slice of length b.Len() holding the unread portion of the buffer. The slice is valid for use only until the next buffer modification (that is, only until the next call to a method like Read, Write, Reset, or Truncate). The slice aliases the buffer content at least until the next buffer modification, so immediate changes to the slice will affect the result of future reads.
func (*Buffer) Cap ¶
Cap returns the capacity of the buffer's underlying byte slice, that is, the total space allocated for the buffer's data.
func (*Buffer) Grow ¶
Grow grows the buffer's capacity, if necessary, to guarantee space for another n bytes. After Grow(n), at least n bytes can be written to the buffer without another allocation. If n is negative, Grow will panic. If the buffer can't grow it will panic with ErrTooLarge.
func (*Buffer) Len ¶
Len returns the number of bytes of the unread portion of the buffer; b.Len() == len(b.Bytes()).
func (*Buffer) Next ¶
Next returns a slice containing the next n bytes from the buffer, advancing the buffer as if the bytes had been returned by Read. If there are fewer than n bytes in the buffer, Next returns the entire buffer. The slice is only valid until the next call to a read or write method.
func (*Buffer) Read ¶
Read reads the next len(p) bytes from the buffer or until the buffer is drained. The return value n is the number of bytes read. If the buffer has no data to return, err is io.EOF (unless len(p) is zero); otherwise it is nil.
func (*Buffer) ReadByte ¶
ReadByte reads and returns the next byte from the buffer. If no byte is available, it returns error io.EOF.
func (*Buffer) ReadBytes ¶
ReadBytes reads until the first occurrence of delim in the input, returning a slice containing the data up to and including the delimiter. If ReadBytes encounters an error before finding a delimiter, it returns the data read before the error and the error itself (often io.EOF). ReadBytes returns err != nil if and only if the returned data does not end in delim.
func (*Buffer) ReadFrom ¶
ReadFrom reads data from r until EOF and appends it to the buffer, growing the buffer as needed. The return value n is the number of bytes read. Any error except io.EOF encountered during the read is also returned. If the buffer becomes too large, ReadFrom will panic with ErrTooLarge.
func (*Buffer) ReadRune ¶
ReadRune reads and returns the next UTF-8-encoded Unicode code point from the buffer. If no bytes are available, the error returned is io.EOF. If the bytes are an erroneous UTF-8 encoding, it consumes one byte and returns U+FFFD, 1.
func (*Buffer) ReadString ¶
ReadString reads until the first occurrence of delim in the input, returning a string containing the data up to and including the delimiter. If ReadString encounters an error before finding a delimiter, it returns the data read before the error and the error itself (often io.EOF). ReadString returns err != nil if and only if the returned data does not end in delim.
func (*Buffer) Reset ¶
func (b *Buffer) Reset()
Reset resets the buffer to be empty, but it retains the underlying storage for use by future writes. Reset is the same as Truncate(0).
func (*Buffer) String ¶
String returns the contents of the unread portion of the buffer as a string. If the Buffer is a nil pointer, it returns "<nil>".
To build strings more efficiently, see the strings.Builder type.
func (*Buffer) Truncate ¶
Truncate discards all but the first n unread bytes from the buffer but continues to use the same allocated storage. It panics if n is negative or greater than the length of the buffer.
func (*Buffer) UnreadByte ¶
UnreadByte unreads the last byte returned by the most recent successful read operation that read at least one byte. If a write has happened since the last read, if the last read returned an error, or if the read read zero bytes, UnreadByte returns an error.
func (*Buffer) UnreadRune ¶
UnreadRune unreads the last rune returned by ReadRune. If the most recent read or write operation on the buffer was not a successful ReadRune, UnreadRune returns an error. (In this regard it is stricter than UnreadByte, which will unread the last byte from any read operation.)
func (*Buffer) Write ¶
Write appends the contents of p to the buffer, growing the buffer as needed. The return value n is the length of p; err is always nil. If the buffer becomes too large, Write will panic with ErrTooLarge.
func (*Buffer) WriteByte ¶
WriteByte appends the byte c to the buffer, growing the buffer as needed. The returned error is always nil, but is included to match bufio.Writer's WriteByte. If the buffer becomes too large, WriteByte will panic with ErrTooLarge.
func (*Buffer) WriteRune ¶
WriteRune appends the UTF-8 encoding of Unicode code point r to the buffer, returning its length and an error, which is always nil but is included to match bufio.Writer's WriteRune. The buffer is grown as needed; if it becomes too large, WriteRune will panic with ErrTooLarge.
func (*Buffer) WriteString ¶
WriteString appends the contents of s to the buffer, growing the buffer as needed. The return value n is the length of s; err is always nil. If the buffer becomes too large, WriteString will panic with ErrTooLarge.
Source Files