bitio

package module
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 7, 2021 License: Apache-2.0 Imports: 2 Imported by: 54

README

bitio

Build Status GoDoc Go Report Card codecov

Package bitio provides an optimized bit-level Reader and Writer for Go.

You can use Reader.ReadBits() to read arbitrary number of bits from an io.Reader and return it as an uint64, and Writer.WriteBits() to write arbitrary number of bits of an uint64 value to an io.Writer.

Both Reader and Writer also provide optimized methods for reading / writing 1 bit of information in the form of a bool value: Reader.ReadBool() and Writer.WriteBool(). These make this package ideal for compression algorithms that use Huffman coding for example, where decision whether to step left or right in the Huffman tree is the most frequent operation.

Reader and Writer give a bit-level view of the underlying io.Reader and io.Writer, but they also provide a byte-level view (io.Reader and io.Writer) at the same time. This means you can also use the Reader.Read() and Writer.Write() methods to read and write slices of bytes. These will give you best performance if the underlying io.Reader and io.Writer are aligned to a byte boundary (else all the individual bytes are assembled from / spread to multiple bytes). You can ensure byte boundary alignment by calling the Align() method of Reader and Writer. As an extra, io.ByteReader and io.ByteWriter are also implemented.

Bit order

The more general highest-bits-first order is used. So for example if the input provides the bytes 0x8f and 0x55:

HEXA    8    f     5    5
BINARY  1100 1111  0101 0101
        aaaa bbbc  ccdd dddd

Then ReadBits will return the following values:

r := NewReader(bytes.NewBuffer([]byte{0x8f, 0x55}))
a, err := r.ReadBits(4) //   1100 = 0x08
b, err := r.ReadBits(3) //    111 = 0x07
c, err := r.ReadBits(3) //    101 = 0x05
d, err := r.ReadBits(6) // 010101 = 0x15

Writing the above values would result in the same sequence of bytes:

b := &bytes.Buffer{}
w := NewWriter(b)
err := w.WriteBits(0x08, 4)
err = w.WriteBits(0x07, 3)
err = w.WriteBits(0x05, 3)
err = w.WriteBits(0x15, 6)
err = w.Close()
// b will hold the bytes: 0x8f and 0x55
Error handling

All ReadXXX() and WriteXXX() methods return an error which you are expected to handle. For convenience, there are also matching TryReadXXX() and TryWriteXXX() methods which do not return an error. Instead they store the (first) error in the Reader.TryError / Writer.TryError field which you can inspect later. These TryXXX() methods are a no-op if a TryError has been encountered before, so it's safe to call multiple TryXXX() methods and defer the error checking.

For example:

r := NewReader(bytes.NewBuffer([]byte{0x8f, 0x55}))
a := r.TryReadBits(4) //   1100 = 0x08
b := r.TryReadBits(3) //    111 = 0x07
c := r.TryReadBits(3) //    101 = 0x05
d := r.TryReadBits(6) // 010101 = 0x15
if r.TryError != nil {
    // Handle error
}

This allows you to easily convert the result of individual ReadBits(), like this:

r := NewReader(bytes.NewBuffer([]byte{0x8f, 0x55}))
a := byte(r.TryReadBits(4))   //   1100 = 0x08
b := int32(r.TryReadBits(3))  //    111 = 0x07
c := int64(r.TryReadBits(3))  //    101 = 0x05
d := uint16(r.TryReadBits(6)) // 010101 = 0x15
if r.TryError != nil {
    // Handle error
}

And similarly:

b := &bytes.Buffer{}
w := NewWriter(b)
w.TryWriteBits(0x08, 4)
w.TryWriteBits(0x07, 3)
w.TryWriteBits(0x05, 3)
w.TryWriteBits(0x15, 6)
if w.TryError != nil {
    // Handle error
}
err = w.Close()
// b will hold the bytes: 0x8f and 0x55
Number of processed bits

For performance reasons, Reader and Writer do not keep track of the number of read or written bits. If you happen to need the total number of processed bits, you may use the CountReader and CountWriter types which have identical API to that of Reader and Writer, but they also maintain the number of processed bits which you can query using the BitsCount field.

Documentation

Overview

Package bitio provides an optimized bit-level Reader and Writer.

You can use Reader.ReadBits() to read arbitrary number of bits from an io.Reader and return it as an uint64, and Writer.WriteBits() to write arbitrary number of bits of an uint64 value to an io.Writer.

Both Reader and Writer also provide optimized methods for reading / writing 1 bit of information in the form of a bool value: Reader.ReadBool() and Writer.WriteBool(). These make this package ideal for compression algorithms that use Huffman coding for example, where decision whether to step left or right in the Huffman tree is the most frequent operation.

Reader and Writer give a bit-level view of the underlying io.Reader and io.Writer, but they also provide a byte-level view (io.Reader and io.Writer) at the same time. This means you can also use the Reader.Read() and Writer.Write() methods to read and write slices of bytes. These will give you best performance if the underlying io.Reader and io.Writer are aligned to a byte boundary (else all the individual bytes are assembled from / spread to multiple bytes). You can ensure byte boundary alignment by calling the Align() method of Reader and Writer. As an extra, io.ByteReader and io.ByteWriter are also implemented.

Bit order

The more general highest-bits-first order is used. So for example if the input provides the bytes 0x8f and 0x55: 

HEXA    8    f     5    5
BINARY  1100 1111  0101 0101
        aaaa bbbc  ccdd dddd

Then ReadBits will return the following values: 

r := NewReader(bytes.NewBuffer([]byte{0x8f, 0x55}))
a, err := r.ReadBits(4) //   1100 = 0x08
b, err := r.ReadBits(3) //    111 = 0x07
c, err := r.ReadBits(3) //    101 = 0x05
d, err := r.ReadBits(6) // 010101 = 0x15

Writing the above values would result in the same sequence of bytes: 

b := &bytes.Buffer{}
w := NewWriter(b)
err := w.WriteBits(0x08, 4)
err = w.WriteBits(0x07, 3)
err = w.WriteBits(0x05, 3)
err = w.WriteBits(0x15, 6)
err = w.Close()
// b will hold the bytes: 0x8f and 0x55

Error handling

All ReadXXX() and WriteXXX() methods return an error which you are expected to handle. For convenience, there are also matching TryReadXXX() and TryWriteXXX() methods which do not return an error. Instead they store the (first) error in the Reader.TryError / Writer.TryError field which you can inspect later. These TryXXX() methods are a no-op if a TryError has been encountered before, so it's safe to call multiple TryXXX() methods and defer the error checking.

For example: 

r := NewReader(bytes.NewBuffer([]byte{0x8f, 0x55}))
a := r.TryReadBits(4) //   1100 = 0x08
b := r.TryReadBits(3) //    111 = 0x07
c := r.TryReadBits(3) //    101 = 0x05
d := r.TryReadBits(6) // 010101 = 0x15
if r.TryError != nil {
    // Handle error
}

This allows you to easily convert the result of individual ReadBits(), like this: 

r := NewReader(bytes.NewBuffer([]byte{0x8f, 0x55}))
a := byte(r.TryReadBits(4))   //   1100 = 0x08
b := int32(r.TryReadBits(3))  //    111 = 0x07
c := int64(r.TryReadBits(3))  //    101 = 0x05
d := uint16(r.TryReadBits(6)) // 010101 = 0x15
if r.TryError != nil {
    // Handle error
}

And similarly: 

b := &bytes.Buffer{}
w := NewWriter(b)
w.TryWriteBits(0x08, 4)
w.TryWriteBits(0x07, 3)
w.TryWriteBits(0x05, 3)
w.TryWriteBits(0x15, 6)
if w.TryError != nil {
    // Handle error
}
err = w.Close()
// b will hold the bytes: 0x8f and 0x55

Number of processed bits

For performance reasons, Reader and Writer do not keep track of the number of read or written bits. If you happen to need the total number of processed bits, you may use the CountReader and CountWriter types which have identical API to that of Reader and Writer, but they also maintain the number of processed bits which you can query using the BitsCount field.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type CountReader added in v1.1.0 

type CountReader struct {
	*Reader
	BitsCount int64 // Total number of bits read
}

CountReader is an improved version of Reader that also keeps track of the number of processed bits. If you don't need the number of processed bits, use the faster Reader.

For convenience, it also implements io.Reader and io.ByteReader.

func NewCountReader added in v1.1.0 

func NewCountReader(in io.Reader) *CountReader

NewCountReader returns a new CountReader using the specified io.Reader as the input (source).

func (*CountReader) Align added in v1.1.0 

func (r *CountReader) Align() (skipped uint8)

Align aligns the bit stream to a byte boundary, so next read will read/use data from the next byte. Returns the number of unread / skipped bits.

func (*CountReader) Read added in v1.1.0 

func (r *CountReader) Read(p []byte) (n int, err error)

Read reads up to len(p) bytes (8 * len(p) bits) from the underlying reader, and counts the number of bits read.

Read implements io.Reader, and gives a byte-level view of the bit stream. This will give best performance if the underlying io.Reader is aligned to a byte boundary (else all the individual bytes are assembled from multiple bytes). Byte boundary can be ensured by calling Align().

func (*CountReader) ReadBits added in v1.1.0 

func (r *CountReader) ReadBits(n uint8) (u uint64, err error)

ReadBits reads n bits and returns them as the lowest n bits of u.

func (*CountReader) ReadBool added in v1.1.0 

func (r *CountReader) ReadBool() (b bool, err error)

ReadBool reads the next bit, and returns true if it is 1.

func (*CountReader) ReadByte added in v1.1.0 

func (r *CountReader) ReadByte() (b byte, err error)

ReadByte reads the next 8 bits and returns them as a byte.

ReadByte implements io.ByteReader.

func (*CountReader) TryRead added in v1.1.0 

func (r *CountReader) TryRead(p []byte) (n int)

TryRead tries to read up to len(p) bytes (8 * len(p) bits) from the underlying reader.

If there was a previous TryError, it does nothing. Else it calls Read(), returns the data it provides and stores the error in the TryError field.

func (*CountReader) TryReadBits added in v1.1.0 

func (r *CountReader) TryReadBits(n uint8) (u uint64)

TryReadBits tries to read n bits.

If there was a previous TryError, it does nothing. Else it calls ReadBits(), returns the data it provides and stores the error in the TryError field.

func (*CountReader) TryReadBool added in v1.1.0 

func (r *CountReader) TryReadBool() (b bool)

TryReadBool tries to read the next bit, and return true if it is 1.

If there was a previous TryError, it does nothing. Else it calls ReadBool(), returns the data it provides and stores the error in the TryError field.

func (*CountReader) TryReadByte added in v1.1.0 

func (r *CountReader) TryReadByte() (b byte)

TryReadByte tries to read the next 8 bits and return them as a byte.

If there was a previous TryError, it does nothing. Else it calls ReadByte(), returns the data it provides and stores the error in the TryError field.

type CountWriter added in v1.1.0 

type CountWriter struct {
	*Writer
	BitsCount int64 // Total number of bits written
}

CountWriter is an improved version of Writer that also keeps track of the number of processed bits. If you don't need the number of processed bits, use the faster Writer.

For convenience, it also implements io.WriterCloser and io.ByteWriter.

func NewCountWriter added in v1.1.0 

func NewCountWriter(out io.Writer) *CountWriter

NewCountWriter returns a new CountWriter using the specified io.Writer as the output.

Must be closed in order to flush cached data. If you can't or don't want to close it, flushing data can also be forced by calling Align().

func (*CountWriter) Align added in v1.1.0 

func (w *CountWriter) Align() (skipped uint8, err error)

Align aligns the bit stream to a byte boundary, so next write will start/go into a new byte. If there are cached bits, they are first written to the output. Returns the number of skipped (unset but still written) bits.

func (*CountWriter) Close added in v1.1.0 

func (w *CountWriter) Close() (err error)

Close closes the bit writer, writes out cached bits. It does not close the underlying io.Writer.

Close implements io.Closer.

func (*CountWriter) TryAlign added in v1.1.0 

func (w *CountWriter) TryAlign() (skipped uint8)

TryAlign tries to align the bit stream to a byte boundary.

If there was a previous TryError, it does nothing. Else it calls Align(), returns the data it provides and stores the error in the TryError field.

func (*CountWriter) TryWrite added in v1.1.0 

func (w *CountWriter) TryWrite(p []byte) (n int)

TryWrite tries to write len(p) bytes (8 * len(p) bits) to the underlying writer.

If there was a previous TryError, it does nothing. Else it calls Write(), returns the data it provides and stores the error in the TryError field.

func (*CountWriter) TryWriteBits added in v1.1.0 

func (w *CountWriter) TryWriteBits(r uint64, n uint8)

TryWriteBits tries to write out the n lowest bits of r.

If there was a previous TryError, it does nothing. Else it calls WriteBits(), and stores the error in the TryError field.

func (*CountWriter) TryWriteBitsUnsafe added in v1.1.0 

func (w *CountWriter) TryWriteBitsUnsafe(r uint64, n uint8)

TryWriteBitsUnsafe tries to write out the n lowest bits of r.

If there was a previous TryError, it does nothing. Else it calls WriteBitsUnsafe(), and stores the error in the TryError field.

func (*CountWriter) TryWriteBool added in v1.1.0 

func (w *CountWriter) TryWriteBool(b bool)

TryWriteBool tries to write one bit: 1 if param is true, 0 otherwise.

If there was a previous TryError, it does nothing. Else it calls WriteBool(), and stores the error in the TryError field.

func (*CountWriter) TryWriteByte added in v1.1.0 

func (w *CountWriter) TryWriteByte(b byte)

TryWriteByte tries to write 8 bits.

If there was a previous TryError, it does nothing. Else it calls WriteByte(), and stores the error in the TryError field.

func (*CountWriter) Write added in v1.1.0 

func (w *CountWriter) Write(p []byte) (n int, err error)

Write writes len(p) bytes (8 * len(p) bits) to the underlying writer.

Write implements io.Writer, and gives a byte-level interface to the bit stream. This will give best performance if the underlying io.Writer is aligned to a byte boundary (else all the individual bytes are spread to multiple bytes). Byte boundary can be ensured by calling Align().

func (*CountWriter) WriteBits added in v1.1.0 

func (w *CountWriter) WriteBits(r uint64, n uint8) (err error)

WriteBits writes out the n lowest bits of r. Bits of r in positions higher than n are ignored.

For example: 

err := w.WriteBits(0x1234, 8)

is equivalent to: 

err := w.WriteBits(0x34, 8)

func (*CountWriter) WriteBitsUnsafe added in v1.1.0 

func (w *CountWriter) WriteBitsUnsafe(r uint64, n uint8) (err error)

WriteBitsUnsafe writes out the n lowest bits of r.

r must not have bits set at n or higher positions (zero indexed). If r might not satisfy this, a mask must be explicitly applied before passing it to WriteBitsUnsafe(), or WriteBits() should be used instead.

WriteBitsUnsafe() offers slightly better performance than WriteBits() because the input r is not masked. Calling WriteBitsUnsafe() with an r that does not satisfy this is undefined behavior (might corrupt previously written bits).

E.g. if you want to write 8 bits: 

err := w.WriteBitsUnsafe(0x34, 8)        // This is OK,
                                         // 0x34 has no bits set higher than the 8th
err := w.WriteBitsUnsafe(0x1234&0xff, 8) // &0xff masks out bits higher than the 8th

Or: 

err := w.WriteBits(0x1234, 8)            // bits higher than the 8th are ignored here

func (*CountWriter) WriteBool added in v1.1.0 

func (w *CountWriter) WriteBool(b bool) (err error)

WriteBool writes one bit: 1 if param is true, 0 otherwise.

func (*CountWriter) WriteByte added in v1.1.0 

func (w *CountWriter) WriteByte(b byte) (err error)

WriteByte writes 8 bits.

WriteByte implements io.ByteWriter.

type Reader 

type Reader struct {

	// TryError holds the first error occurred in TryXXX() methods.
	TryError error
	// contains filtered or unexported fields
}

Reader is the bit reader implementation.

If you need the number of processed bits, use CountReader.

For convenience, it also implements io.Reader and io.ByteReader.

func NewReader 

func NewReader(in io.Reader) *Reader

NewReader returns a new Reader using the specified io.Reader as the input (source).

func (*Reader) Align 

func (r *Reader) Align() (skipped uint8)

Align aligns the bit stream to a byte boundary, so next read will read/use data from the next byte. Returns the number of unread / skipped bits.

func (*Reader) Read added in v0.9.0 

func (r *Reader) Read(p []byte) (n int, err error)

Read reads up to len(p) bytes (8 * len(p) bits) from the underlying reader.

Read implements io.Reader, and gives a byte-level view of the bit stream. This will give best performance if the underlying io.Reader is aligned to a byte boundary (else all the individual bytes are assembled from multiple bytes). Byte boundary can be ensured by calling Align().

func (*Reader) ReadBits 

func (r *Reader) ReadBits(n uint8) (u uint64, err error)

ReadBits reads n bits and returns them as the lowest n bits of u.

func (*Reader) ReadBool 

func (r *Reader) ReadBool() (b bool, err error)

ReadBool reads the next bit, and returns true if it is 1.

func (*Reader) ReadByte added in v0.9.0 

func (r *Reader) ReadByte() (b byte, err error)

ReadByte reads the next 8 bits and returns them as a byte.

ReadByte implements io.ByteReader.

func (*Reader) TryRead added in v0.9.0 

func (r *Reader) TryRead(p []byte) (n int)

TryRead tries to read up to len(p) bytes (8 * len(p) bits) from the underlying reader.

If there was a previous TryError, it does nothing. Else it calls Read(), returns the data it provides and stores the error in the TryError field.

func (*Reader) TryReadBits added in v0.9.0 

func (r *Reader) TryReadBits(n uint8) (u uint64)

TryReadBits tries to read n bits.

If there was a previous TryError, it does nothing. Else it calls ReadBits(), returns the data it provides and stores the error in the TryError field.

func (*Reader) TryReadBool added in v0.9.0 

func (r *Reader) TryReadBool() (b bool)

TryReadBool tries to read the next bit, and return true if it is 1.

If there was a previous TryError, it does nothing. Else it calls ReadBool(), returns the data it provides and stores the error in the TryError field.

func (*Reader) TryReadByte added in v0.9.0 

func (r *Reader) TryReadByte() (b byte)

TryReadByte tries to read the next 8 bits and return them as a byte.

If there was a previous TryError, it does nothing. Else it calls ReadByte(), returns the data it provides and stores the error in the TryError field.

type Writer 

type Writer struct {

	// TryError holds the first error occurred in TryXXX() methods.
	TryError error
	// contains filtered or unexported fields
}

Writer is the bit writer implementation.

If you need the number of processed bits, use CountWriter.

For convenience, it also implements io.WriterCloser and io.ByteWriter.

func NewWriter 

func NewWriter(out io.Writer) *Writer

NewWriter returns a new Writer using the specified io.Writer as the output.

Must be closed in order to flush cached data. If you can't or don't want to close it, flushing data can also be forced by calling Align().

func (*Writer) Align 

func (w *Writer) Align() (skipped uint8, err error)

Align aligns the bit stream to a byte boundary, so next write will start/go into a new byte. If there are cached bits, they are first written to the output. Returns the number of skipped (unset but still written) bits.

func (*Writer) Close added in v0.9.0 

func (w *Writer) Close() (err error)

Close closes the bit writer, writes out cached bits. It does not close the underlying io.Writer.

Close implements io.Closer.

func (*Writer) TryAlign added in v0.9.0 

func (w *Writer) TryAlign() (skipped uint8)

TryAlign tries to align the bit stream to a byte boundary.

If there was a previous TryError, it does nothing. Else it calls Align(), returns the data it provides and stores the error in the TryError field.

func (*Writer) TryWrite added in v0.9.0 

func (w *Writer) TryWrite(p []byte) (n int)

TryWrite tries to write len(p) bytes (8 * len(p) bits) to the underlying writer.

If there was a previous TryError, it does nothing. Else it calls Write(), returns the data it provides and stores the error in the TryError field.

func (*Writer) TryWriteBits added in v0.9.0 

func (w *Writer) TryWriteBits(r uint64, n uint8)

TryWriteBits tries to write out the n lowest bits of r.

If there was a previous TryError, it does nothing. Else it calls WriteBits(), and stores the error in the TryError field.

func (*Writer) TryWriteBitsUnsafe added in v1.0.0 

func (w *Writer) TryWriteBitsUnsafe(r uint64, n uint8)

TryWriteBitsUnsafe tries to write out the n lowest bits of r.

If there was a previous TryError, it does nothing. Else it calls WriteBitsUnsafe(), and stores the error in the TryError field.

func (*Writer) TryWriteBool added in v0.9.0 

func (w *Writer) TryWriteBool(b bool)

TryWriteBool tries to write one bit: 1 if param is true, 0 otherwise.

If there was a previous TryError, it does nothing. Else it calls WriteBool(), and stores the error in the TryError field.

func (*Writer) TryWriteByte added in v0.9.0 

func (w *Writer) TryWriteByte(b byte)

TryWriteByte tries to write 8 bits.

If there was a previous TryError, it does nothing. Else it calls WriteByte(), and stores the error in the TryError field.

func (*Writer) Write added in v0.9.0 

func (w *Writer) Write(p []byte) (n int, err error)

Write writes len(p) bytes (8 * len(p) bits) to the underlying writer.

Write implements io.Writer, and gives a byte-level interface to the bit stream. This will give best performance if the underlying io.Writer is aligned to a byte boundary (else all the individual bytes are spread to multiple bytes). Byte boundary can be ensured by calling Align().

func (*Writer) WriteBits 

func (w *Writer) WriteBits(r uint64, n uint8) (err error)

WriteBits writes out the n lowest bits of r. Bits of r in positions higher than n are ignored.

For example: 

err := w.WriteBits(0x1234, 8)

is equivalent to: 

err := w.WriteBits(0x34, 8)

func (*Writer) WriteBitsUnsafe added in v1.0.0 

func (w *Writer) WriteBitsUnsafe(r uint64, n uint8) (err error)

WriteBitsUnsafe writes out the n lowest bits of r.

r must not have bits set at n or higher positions (zero indexed). If r might not satisfy this, a mask must be explicitly applied before passing it to WriteBitsUnsafe(), or WriteBits() should be used instead.

WriteBitsUnsafe() offers slightly better performance than WriteBits() because the input r is not masked. Calling WriteBitsUnsafe() with an r that does not satisfy this is undefined behavior (might corrupt previously written bits).

E.g. if you want to write 8 bits: 

err := w.WriteBitsUnsafe(0x34, 8)        // This is OK,
                                         // 0x34 has no bits set higher than the 8th
err := w.WriteBitsUnsafe(0x1234&0xff, 8) // &0xff masks out bits higher than the 8th

Or: 

err := w.WriteBits(0x1234, 8)            // bits higher than the 8th are ignored here

func (*Writer) WriteBool 

func (w *Writer) WriteBool(b bool) (err error)

WriteBool writes one bit: 1 if param is true, 0 otherwise.

func (*Writer) WriteByte added in v0.9.0 

func (w *Writer) WriteByte(b byte) (err error)

WriteByte writes 8 bits.

WriteByte implements io.ByteWriter.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.