README

restruct Build Status codecov.io godoc.org Go Report Card

restruct is a library for reading and writing binary data in Go. Similar to lunixbochs struc and encoding/binary, this library reads data based on the layout of structures and, like struc, based on what is contained in struct tags.

To install Restruct, use the following command:

go get github.com/go-restruct/restruct

restruct aims to provide a clean, flexible, robust implementation of struct packing. In the future, through fast-path optimizations and code generation, it also aims to be quick, but it is currently very slow.

restruct currently requires Go 1.7+.

Status

  • As of writing, coverage is hovering around 95%, but more thorough testing is always useful and desirable.
  • Unpacking and packing are fully functional.
  • More optimizations are probably possible.

Example

package main

import (
	"encoding/binary"
	"io/ioutil"
	"os"

	"github.com/go-restruct/restruct"
)

type Record struct {
	Message string `struct:"[128]byte"`
}

type Container struct {
	Version   int `struct:"int32"`
	NumRecord int `struct:"int32,sizeof=Records"`
	Records   []Record
}

func main() {
	var c Container

	file, _ := os.Open("records")
	defer file.Close()
	data, _ := ioutil.ReadAll(file)

	restruct.Unpack(data, binary.LittleEndian, &c)
}

Documentation

Overview

    Package restruct implements packing and unpacking of raw binary formats.

    Structures can be created with struct tags annotating the on-disk or in-memory layout of the structure, using the "struct" struct tag, like so:

    struct {
    	Length int `struct:"int32,sizeof=Packets"`
    	Packets []struct{
    		Source    string    `struct:"[16]byte"`
    		Timestamp int       `struct:"int32,big"`
    		Data      [256]byte `struct:"skip=8"`
    	}
    }
    

    To unpack data in memory to this structure, simply use Unpack with a byte slice:

    msg := Message{}
    restruct.Unpack(data, binary.LittleEndian, &msg)
    

    Index

    Constants

    This section is empty.

    Variables

    View Source
    var ErrInvalidBits = errors.New("bits specified on non-bitwise type")

      ErrInvalidBits is returned when bits is used on an invalid type.

      View Source
      var ErrInvalidSize = errors.New("size specified on fixed size type")

        ErrInvalidSize is returned when sizefrom is used on an invalid type.

        View Source
        var ErrInvalidSizeFrom = errors.New("sizefrom specified on fixed size type")

          ErrInvalidSizeFrom is returned when sizefrom is used on an invalid type.

          View Source
          var ErrInvalidSizeOf = errors.New("sizeof specified on fixed size type")

            ErrInvalidSizeOf is returned when sizefrom is used on an invalid type.

            Functions

            func BitSize

            func BitSize(v interface{}) (size int, err error)

              BitSize returns the binary encoded size of the given value, in bits.

              func EnableExprBeta

              func EnableExprBeta()

                EnableExprBeta enables you to use restruct expr while it is still in beta. Use at your own risk. Functionality may change in unforeseen, incompatible ways at any time.

                func Pack

                func Pack(order binary.ByteOrder, v interface{}) (data []byte, err error)

                  Pack writes data from a datastructure into a byteslice.

                  Two types of values are directly supported here: Packers and structs. You can pass them by value or by pointer.

                  Each structure is serialized in the same way it would be deserialized with Unpack. See Unpack documentation for the struct tag format.

                  func RegisterArrayType

                  func RegisterArrayType(array interface{})

                    RegisterArrayType is deprecated; it is now a noop.

                    func SizeOf

                    func SizeOf(v interface{}) (size int, err error)

                      SizeOf returns the binary encoded size of the given value, in bytes.

                      func Unpack

                      func Unpack(data []byte, order binary.ByteOrder, v interface{}) (err error)

                        Unpack reads data from a byteslice into a value.

                        Two types of values are directly supported here: Unpackers and structs. You can pass them by value or by pointer, although it is an error if Restruct is unable to set a value because it is unaddressable.

                        For structs, each field will be read sequentially based on a straightforward interpretation of the type. For example, an int32 will be read as a 32-bit signed integer, taking 4 bytes of memory. Structures and arrays are laid out flat with no padding or metadata.

                        Unexported fields are ignored, except for fields named _ - those fields will be treated purely as padding. Padding will not be preserved through packing and unpacking.

                        The behavior of deserialization can be customized using struct tags. The following struct tag syntax is supported:

                        `struct:"[flags...]"`
                        

                        Flags are comma-separated keys. The following are available:

                        type              A bare type name, e.g. int32 or []string. For integer
                                          types, it is possible to specify the number of bits,
                                          allowing the definition of bitfields, by appending a
                                          colon followed by the number of bits. For example,
                                          uint32:20 would specify a field that is 20 bits long.
                        
                        sizeof=[Field]    Specifies that the field should be treated as a count of
                                          the number of elements in Field.
                        
                        sizefrom=[Field]  Specifies that the field should determine the number of
                                          elements in itself by reading the counter in Field.
                        
                        skip=[Count]      Skips Count bytes before the field. You can use this to
                                          e.g. emulate C structure alignment.
                        
                        big,msb           Specifies big endian byte order. When applied to
                                          structs, this will apply to all fields under the struct.
                        
                        little,lsb        Specifies little endian byte order. When applied to
                                          structs, this will apply to all fields under the struct.
                        
                        variantbool       Specifies that the boolean `true` value should be
                                          encoded as -1 instead of 1.
                        
                        invertedbool      Specifies that the `true` and `false` encodings for
                                          boolean should be swapped.
                        

                        Types

                        type BitSizer

                        type BitSizer interface {
                        	BitSize() int
                        }

                          BitSizer is an interface for types that need to specify their own size in bit-level granularity. It has the same effect as Sizer.

                          type FieldFlags

                          type FieldFlags uint64

                            FieldFlags is a type for flags that can be applied to fields individually.

                            const (
                            	// VariantBoolFlag causes the true value of a boolean to be ~0 instead of
                            	// just 1 (all bits are set.) This emulates the behavior of VARIANT_BOOL.
                            	VariantBoolFlag FieldFlags = 1 << iota
                            
                            	// InvertedBoolFlag causes the true and false states of a boolean to be
                            	// flipped in binary.
                            	InvertedBoolFlag
                            
                            	// RootFlag is set when the field points to the root struct.
                            	RootFlag
                            
                            	// ParentFlag is set when the field points to the parent struct.
                            	ParentFlag
                            
                            	// DefaultFlag is set when the field is designated as a switch case default.
                            	DefaultFlag
                            )

                            type Packer

                            type Packer interface {
                            	Sizer
                            	Pack(buf []byte, order binary.ByteOrder) ([]byte, error)
                            }

                              Packer is a type capable of packing a native value into a binary representation. The Pack function is expected to overwrite a number of bytes in buf then return a slice of the remaining buffer. Note that you must also implement SizeOf, and returning an incorrect SizeOf will cause the encoder to crash. The SizeOf should be equal to the number of bytes consumed from the buffer slice in Pack. You may use a pointer receiver even if the type is used by value.

                              type Sizer

                              type Sizer interface {
                              	SizeOf() int
                              }

                                Sizer is a type which has a defined size in binary. The SizeOf function returns how many bytes the type will consume in memory. This is used during encoding for allocation and therefore must equal the exact number of bytes the encoded form needs. You may use a pointer receiver even if the type is used by value.

                                type Unpacker

                                type Unpacker interface {
                                	Unpack(buf []byte, order binary.ByteOrder) ([]byte, error)
                                }

                                  Unpacker is a type capable of unpacking a binary representation of itself into a native representation. The Unpack function is expected to consume a number of bytes from the buffer, then return a slice of the remaining bytes in the buffer. You may use a pointer receiver even if the type is used by value.

                                  Directories

                                  Path Synopsis
                                  formats
                                  png
                                  Package png implements some of the PNG format using Restruct.
                                  Package png implements some of the PNG format using Restruct.