Documentation

Overview

Package proto converts data structures to and from the wire format of protocol buffers. It works in concert with the Go source code generated for .proto files by the protocol compiler.

A summary of the properties of the protocol buffer interface for a protocol buffer variable v:

  - Names are turned from camel_case to CamelCase for export.
  - There are no methods on v to set fields; just treat
  	them as structure fields.
  - There are getters that return a field's value if set,
	and return the field's default value if unset.
	The getters work even if the receiver is a nil message.
  - The zero value for a struct is its correct initialization state.
	All desired fields must be set before marshaling.
  - A Reset() method will restore a protobuf struct to its zero state.
  - Non-repeated fields are pointers to the values; nil means unset.
	That is, optional or required field int32 f becomes F *int32.
  - Repeated fields are slices.
  - Helper functions are available to aid the setting of fields.
	Helpers for getting values are superseded by the
	GetFoo methods and their use is deprecated.
		msg.Foo = proto.String("hello") // set field
  - Constants are defined to hold the default values of all fields that
	have them.  They have the form Default_StructName_FieldName.
	Because the getter methods handle defaulted values,
	direct use of these constants should be rare.
  - Enums are given type names and maps from names to values.
	Enum values are prefixed with the enum's type name. Enum types have
	a String method, and a Enum method to assist in message construction.
  - Nested groups and enums have type names prefixed with the name of
  	the surrounding message type.
  - Extensions are given descriptor names that start with E_,
	followed by an underscore-delimited list of the nested messages
	that contain it (if any) followed by the CamelCased name of the
	extension field itself.  HasExtension, ClearExtension, GetExtension
	and SetExtension are functions for manipulating extensions.
  - Marshal and Unmarshal are functions to encode and decode the wire format.

The simplest way to describe this is to see an example. Given file test.proto, containing

package example;

enum FOO { X = 17; };

message Test {
  required string label = 1;
  optional int32 type = 2 [default=77];
  repeated int64 reps = 3;
  optional group OptionalGroup = 4 {
    required string RequiredField = 5;
  }
}

The resulting file, test.pb.go, is:

package example

import "github.com/coreos/etcd/third_party/code.google.com/p/gogoprotobuf/proto"

type FOO int32
const (
	FOO_X FOO = 17
)
var FOO_name = map[int32]string{
	17: "X",
}
var FOO_value = map[string]int32{
	"X": 17,
}

func (x FOO) Enum() *FOO {
	p := new(FOO)
	*p = x
	return p
}
func (x FOO) String() string {
	return proto.EnumName(FOO_name, int32(x))
}

type Test struct {
	Label            *string             `protobuf:"bytes,1,req,name=label" json:"label,omitempty"`
	Type             *int32              `protobuf:"varint,2,opt,name=type,def=77" json:"type,omitempty"`
	Reps             []int64             `protobuf:"varint,3,rep,name=reps" json:"reps,omitempty"`
	Optionalgroup    *Test_OptionalGroup `protobuf:"group,4,opt,name=OptionalGroup" json:"optionalgroup,omitempty"`
	XXX_unrecognized []byte              `json:"-"`
}
func (this *Test) Reset()         { *this = Test{} }
func (this *Test) String() string { return proto.CompactTextString(this) }
const Default_Test_Type int32 = 77

func (this *Test) GetLabel() string {
	if this != nil && this.Label != nil {
		return *this.Label
	}
	return ""
}

func (this *Test) GetType() int32 {
	if this != nil && this.Type != nil {
		return *this.Type
	}
	return Default_Test_Type
}

func (this *Test) GetOptionalgroup() *Test_OptionalGroup {
	if this != nil {
		return this.Optionalgroup
	}
	return nil
}

type Test_OptionalGroup struct {
	RequiredField    *string `protobuf:"bytes,5,req" json:"RequiredField,omitempty"`
	XXX_unrecognized []byte  `json:"-"`
}
func (this *Test_OptionalGroup) Reset()         { *this = Test_OptionalGroup{} }
func (this *Test_OptionalGroup) String() string { return proto.CompactTextString(this) }

func (this *Test_OptionalGroup) GetRequiredField() string {
	if this != nil && this.RequiredField != nil {
		return *this.RequiredField
	}
	return ""
}

func init() {
	proto.RegisterEnum("example.FOO", FOO_name, FOO_value)
}

To create and play with a Test object:

package main

import (
	"log"

	"github.com/coreos/etcd/third_party/code.google.com/p/gogoprotobuf/proto"
	"./example.pb"
)

func main() {
	test := &example.Test{
		Label: proto.String("hello"),
		Type:  proto.Int32(17),
		Optionalgroup: &example.Test_OptionalGroup{
			RequiredField: proto.String("good bye"),
		},
	}
	data, err := proto.Marshal(test)
	if err != nil {
		log.Fatal("marshaling error: ", err)
	}
	newTest := new(example.Test)
	err = proto.Unmarshal(data, newTest)
	if err != nil {
		log.Fatal("unmarshaling error: ", err)
	}
	// Now test and newTest contain the same data.
	if test.GetLabel() != newTest.GetLabel() {
		log.Fatalf("data mismatch %q != %q", test.GetLabel(), newTest.GetLabel())
	}
	// etc.
}

Index

Constants

const (
	WireVarint     = 0
	WireFixed64    = 1
	WireBytes      = 2
	WireStartGroup = 3
	WireEndGroup   = 4
	WireFixed32    = 5
)

Constants that identify the encoding of a value on the wire.


Variables

var (
	// ErrRepeatedHasNil is the error returned if Marshal is called with
	// a struct with a repeated field containing a nil element.
	ErrRepeatedHasNil = errors.New("proto: repeated field has nil element")

	// ErrNil is the error returned if Marshal is called with nil.
	ErrNil = errors.New("proto: Marshal called with nil")
)

var ErrMissingExtension = errors.New("proto: missing extension")

ErrMissingExtension is the error returned by GetExtension if the named extension is not in the message.


var ErrNoMessageTypeId = errors.New("proto does not have a message type ID")

ErrNoMessageTypeId occurs when a protocol buffer does not have a message type ID. A message type ID is required for storing a protocol buffer in a message set.


var ErrWrongType = errors.New("proto: field/encoding mismatch: wrong type for field")

ErrWrongType occurs when the wire encoding for the field disagrees with that specified in the type being decoded. This is usually caused by attempting to convert an encoded protocol buffer into a struct of the wrong type.


Functions

func Bool

func Bool(v bool) *bool

Bool is a helper routine that allocates a new bool value to store v and returns a pointer to it.

func BytesToExtensionsMap

func BytesToExtensionsMap(buf []byte) (map[int32]Extension, error)

func ClearExtension

func ClearExtension(pb extendableProto, extension *ExtensionDesc)

ClearExtension removes the given extension from pb.

func CompactText

func CompactText(w io.Writer, pb Message) error

CompactText writes a given protocol buffer in compact text format (one line).

func CompactTextString

func CompactTextString(pb Message) string

CompactTextString is the same as CompactText, but returns the string directly.

func DecodeVarint

func DecodeVarint(buf []byte) (x uint64, n int)

DecodeVarint reads a varint-encoded integer from the slice. It returns the integer and the number of bytes consumed, or zero if there is not enough. This is the format for the int32, int64, uint32, uint64, bool, and enum protocol buffer types.

func EncodeExtensionMap

func EncodeExtensionMap(m map[int32]Extension, data []byte) (n int, err error)

func EncodeVarint

func EncodeVarint(x uint64) []byte

EncodeVarint returns the varint encoding of x. This is the format for the int32, int64, uint32, uint64, bool, and enum protocol buffer types. Not used by the package itself, but helpful to clients wishing to use the same encoding.

func EnumName

func EnumName(m map[int32]string, v int32) string

EnumName is a helper function to simplify printing protocol buffer enums by name. Given an enum map and a value, it returns a useful string.

func Equal

func Equal(a, b Message) bool

Equal returns true iff protocol buffers a and b are equal. The arguments must both be pointers to protocol buffer structs.

Equality is defined in this way:

- Two messages are equal iff they are the same type,
  corresponding fields are equal, unknown field sets
  are equal, and extensions sets are equal.
- Two set scalar fields are equal iff their values are equal.
  If the fields are of a floating-point type, remember that
  NaN != x for all x, including NaN.
- Two repeated fields are equal iff their lengths are the same,
  and their corresponding elements are equal (a "bytes" field,
  although represented by []byte, is not a repeated field)
- Two unset fields are equal.
- Two unknown field sets are equal if their current
  encoded state is equal. (TODO)
- Two extension sets are equal iff they have corresponding
  elements that are pairwise equal.
- Every other combination of things are not equal.

The return value is undefined if a and b are not protocol buffers.

func Float32

func Float32(v float32) *float32

Float32 is a helper routine that allocates a new float32 value to store v and returns a pointer to it.

func Float64

func Float64(v float64) *float64

Float64 is a helper routine that allocates a new float64 value to store v and returns a pointer to it.

func GetBoolExtension

func GetBoolExtension(pb extendableProto, extension *ExtensionDesc, ifnotset bool) bool

func GetExtension

func GetExtension(pb extendableProto, extension *ExtensionDesc) (interface{}, error)

GetExtension parses and returns the given extension of pb. If the extension is not present it returns ErrMissingExtension.

func GetExtensions

func GetExtensions(pb Message, es []*ExtensionDesc) (extensions []interface{}, err error)

GetExtensions returns a slice of the extensions present in pb that are also listed in es. The returned slice has the same length as es; missing extensions will appear as nil elements.

func GetRawExtension

func GetRawExtension(m map[int32]Extension, id int32) ([]byte, error)

func HasExtension

func HasExtension(pb extendableProto, extension *ExtensionDesc) bool

HasExtension returns whether the given extension is present in pb.

func Int

func Int(v int) *int32

Int is a helper routine that allocates a new int32 value to store v and returns a pointer to it, but unlike Int32 its argument value is an int.

func Int32

func Int32(v int32) *int32

Int32 is a helper routine that allocates a new int32 value to store v and returns a pointer to it.

func Int64

func Int64(v int64) *int64

Int64 is a helper routine that allocates a new int64 value to store v and returns a pointer to it.

func Marshal

func Marshal(pb Message) ([]byte, error)

Marshal takes the protocol buffer and encodes it into the wire format, returning the data.

func MarshalJSONEnum

func MarshalJSONEnum(m map[int32]string, value int32) ([]byte, error)

func MarshalMessageSet

func MarshalMessageSet(m map[int32]Extension) ([]byte, error)

MarshalMessageSet encodes the extension map represented by m in the message set wire format. It is called by generated Marshal methods on protocol buffer messages with the message_set_wire_format option.

func MarshalText

func MarshalText(w io.Writer, pb Message) error

MarshalText writes a given protocol buffer in text format. The only errors returned are from w.

func MarshalTextString

func MarshalTextString(pb Message) string

MarshalTextString is the same as MarshalText, but returns the string directly.

func Merge

func Merge(dst, src Message)

Merge merges src into dst. Required and optional fields that are set in src will be set to that value in dst. Elements of repeated fields will be appended. Merge panics if src and dst are not the same type, or if dst is nil.

func RegisterEnum

func RegisterEnum(typeName string, unusedNameMap map[int32]string, valueMap map[string]int32)

RegisterEnum is called from the generated code to install the enum descriptor maps into the global table to aid parsing text format protocol buffers.

func RegisterExtension

func RegisterExtension(desc *ExtensionDesc)

RegisterExtension is called from the generated code.

func RegisterMessageSetType

func RegisterMessageSetType(i messageTypeIder, name string)

RegisterMessageSetType is called from the generated code.

func RegisteredExtensions

func RegisteredExtensions(pb Message) map[int32]*ExtensionDesc

RegisteredExtensions returns a map of the registered extensions of a protocol buffer struct, indexed by the extension number. The argument pb should be a nil pointer to the struct type.

func SetDefaults

func SetDefaults(pb Message)

SetDefaults sets unset protocol buffer fields to their default values. It only modifies fields that are both unset and have defined defaults. It recursively sets default values in any non-nil sub-messages.

func SetExtension

func SetExtension(pb extendableProto, extension *ExtensionDesc, value interface{}) error

SetExtension sets the specified extension of pb to the specified value.

func SetRawExtension

func SetRawExtension(base extendableProto, id int32, b []byte)

SetRawExtension is for testing only.

func Size

func Size(pb Message) (n int)

Size returns the encoded size of a protocol buffer.

func SizeOfExtensionMap

func SizeOfExtensionMap(m map[int32]Extension) (n int)

func Skip

func Skip(data []byte) (n int, err error)

func String

func String(v string) *string

String is a helper routine that allocates a new string value to store v and returns a pointer to it.

func StringFromExtensionsBytes

func StringFromExtensionsBytes(ext []byte) string

func StringFromExtensionsMap

func StringFromExtensionsMap(m map[int32]Extension) string

func Uint32

func Uint32(v uint32) *uint32

Uint32 is a helper routine that allocates a new uint32 value to store v and returns a pointer to it.

func Uint64

func Uint64(v uint64) *uint64

Uint64 is a helper routine that allocates a new uint64 value to store v and returns a pointer to it.

func Unmarshal

func Unmarshal(buf []byte, pb Message) error

Unmarshal parses the protocol buffer representation in buf and places the decoded result in pb. If the struct underlying pb does not match the data in buf, the results can be unpredictable.

Unmarshal resets pb before starting to unmarshal, so any existing data in pb is always removed. Use UnmarshalMerge to preserve and append to existing data.

func UnmarshalJSONEnum

func UnmarshalJSONEnum(m map[string]int32, data []byte, enumName string) (int32, error)

UnmarshalJSONEnum is a helper function to simplify recovering enum int values from their JSON-encoded representation. Given a map from the enum's symbolic names to its int values, and a byte buffer containing the JSON-encoded value, it returns an int32 that can be cast to the enum type by the caller.

The function can deal with both JSON representations, numeric and symbolic.

func UnmarshalMerge

func UnmarshalMerge(buf []byte, pb Message) error

UnmarshalMerge parses the protocol buffer representation in buf and writes the decoded result to pb. If the struct underlying pb does not match the data in buf, the results can be unpredictable.

UnmarshalMerge merges into existing data in pb. Most code should use Unmarshal instead.

func UnmarshalMessageSet

func UnmarshalMessageSet(buf []byte, m map[int32]Extension) error

UnmarshalMessageSet decodes the extension map encoded in buf in the message set wire format. It is called by generated Unmarshal methods on protocol buffer messages with the message_set_wire_format option.

func UnmarshalText

func UnmarshalText(s string, pb Message) error

UnmarshalText reads a protocol buffer in Text format. UnmarshalText resets pb before starting to unmarshal, so any existing data in pb is always removed.

Types

type Buffer

type Buffer struct {
	// contains filtered or unexported fields
}

A Buffer is a buffer manager for marshaling and unmarshaling protocol buffers. It may be reused between invocations to reduce memory usage. It is not necessary to use a Buffer; the global functions Marshal and Unmarshal create a temporary Buffer and are fine for most applications.

func NewBuffer

func NewBuffer(e []byte) *Buffer

NewBuffer allocates a new Buffer and initializes its internal data to the contents of the argument slice.

func (*Buffer) Bytes

func (p *Buffer) Bytes() []byte

Bytes returns the contents of the Buffer.

func (*Buffer) DebugPrint

func (o *Buffer) DebugPrint(s string, b []byte)

DebugPrint dumps the encoded data in b in a debugging format with a header including the string s. Used in testing but made available for general debugging.

func (*Buffer) DecodeFixed32

func (p *Buffer) DecodeFixed32() (x uint64, err error)

DecodeFixed32 reads a 32-bit integer from the Buffer. This is the format for the fixed32, sfixed32, and float protocol buffer types.

func (*Buffer) DecodeFixed64

func (p *Buffer) DecodeFixed64() (x uint64, err error)

DecodeFixed64 reads a 64-bit integer from the Buffer. This is the format for the fixed64, sfixed64, and double protocol buffer types.

func (*Buffer) DecodeRawBytes

func (p *Buffer) DecodeRawBytes(alloc bool) (buf []byte, err error)

DecodeRawBytes reads a count-delimited byte buffer from the Buffer. This is the format used for the bytes protocol buffer type and for embedded messages.

func (*Buffer) DecodeStringBytes

func (p *Buffer) DecodeStringBytes() (s string, err error)

DecodeStringBytes reads an encoded string from the Buffer. This is the format used for the proto2 string type.

func (*Buffer) DecodeVarint

func (p *Buffer) DecodeVarint() (x uint64, err error)

DecodeVarint reads a varint-encoded integer from the Buffer. This is the format for the int32, int64, uint32, uint64, bool, and enum protocol buffer types.

func (*Buffer) DecodeZigzag32

func (p *Buffer) DecodeZigzag32() (x uint64, err error)

DecodeZigzag32 reads a zigzag-encoded 32-bit integer from the Buffer. This is the format used for the sint32 protocol buffer type.

func (*Buffer) DecodeZigzag64

func (p *Buffer) DecodeZigzag64() (x uint64, err error)

DecodeZigzag64 reads a zigzag-encoded 64-bit integer from the Buffer. This is the format used for the sint64 protocol buffer type.

func (*Buffer) EncodeFixed32

func (p *Buffer) EncodeFixed32(x uint64) error

EncodeFixed32 writes a 32-bit integer to the Buffer. This is the format for the fixed32, sfixed32, and float protocol buffer types.

func (*Buffer) EncodeFixed64

func (p *Buffer) EncodeFixed64(x uint64) error

EncodeFixed64 writes a 64-bit integer to the Buffer. This is the format for the fixed64, sfixed64, and double protocol buffer types.

func (*Buffer) EncodeRawBytes

func (p *Buffer) EncodeRawBytes(b []byte) error

EncodeRawBytes writes a count-delimited byte buffer to the Buffer. This is the format used for the bytes protocol buffer type and for embedded messages.

func (*Buffer) EncodeStringBytes

func (p *Buffer) EncodeStringBytes(s string) error

EncodeStringBytes writes an encoded string to the Buffer. This is the format used for the proto2 string type.

func (*Buffer) EncodeVarint

func (p *Buffer) EncodeVarint(x uint64) error

EncodeVarint writes a varint-encoded integer to the Buffer. This is the format for the int32, int64, uint32, uint64, bool, and enum protocol buffer types.

func (*Buffer) EncodeZigzag32

func (p *Buffer) EncodeZigzag32(x uint64) error

EncodeZigzag32 writes a zigzag-encoded 32-bit integer to the Buffer. This is the format used for the sint32 protocol buffer type.

func (*Buffer) EncodeZigzag64

func (p *Buffer) EncodeZigzag64(x uint64) error

EncodeZigzag64 writes a zigzag-encoded 64-bit integer to the Buffer. This is the format used for the sint64 protocol buffer type.

func (*Buffer) Marshal

func (p *Buffer) Marshal(pb Message) error

Marshal takes the protocol buffer and encodes it into the wire format, writing the result to the Buffer.

func (*Buffer) Reset

func (p *Buffer) Reset()

Reset resets the Buffer, ready for marshaling a new protocol buffer.

func (*Buffer) SetBuf

func (p *Buffer) SetBuf(s []byte)

SetBuf replaces the internal buffer with the slice, ready for unmarshaling the contents of the slice.

func (*Buffer) Unmarshal

func (p *Buffer) Unmarshal(pb Message) error

Unmarshal parses the protocol buffer representation in the Buffer and places the decoded result in pb. If the struct underlying pb does not match the data in the buffer, the results can be unpredictable.

type Extension

type Extension struct {
	// contains filtered or unexported fields
}

Extension represents an extension in a message.

func NewExtension

func NewExtension(e []byte) Extension

func (*Extension) Equal

func (this *Extension) Equal(that *Extension) bool

func (Extension) GoString

func (this Extension) GoString() string

type ExtensionDesc

type ExtensionDesc struct {
	ExtendedType  Message     // nil pointer to the type that is being extended
	ExtensionType interface{} // nil pointer to the extension type
	Field         int32       // field number
	Name          string      // fully-qualified name of extension, for text formatting
	Tag           string      // protobuf tag style
}

ExtensionDesc represents an extension specification. Used in generated code from the protocol compiler.

type ExtensionRange

type ExtensionRange struct {
	Start, End int32 // both inclusive
}

ExtensionRange represents a range of message extensions for a protocol buffer. Used in code generated by the protocol compiler.

type Marshaler

type Marshaler interface {
	Marshal() ([]byte, error)
}

Marshaler is the interface representing objects that can marshal themselves.

type Message

type Message interface {
	Reset()
	String() string
	ProtoMessage()
}

Message is implemented by generated protocol buffer messages.

func Clone

func Clone(pb Message) Message

Clone returns a deep copy of a protocol buffer.

type MessageSet

type MessageSet struct {
	Item             []*_MessageSet_Item `protobuf:"group,1,rep"`
	XXX_unrecognized []byte
}

func (*MessageSet) Has

func (ms *MessageSet) Has(pb Message) bool

func (*MessageSet) Marshal

func (ms *MessageSet) Marshal(pb Message) error

func (*MessageSet) ProtoMessage

func (*MessageSet) ProtoMessage()

func (*MessageSet) Reset

func (ms *MessageSet) Reset()

func (*MessageSet) String

func (ms *MessageSet) String() string

func (*MessageSet) Unmarshal

func (ms *MessageSet) Unmarshal(pb Message) error

type ParseError

type ParseError struct {
	Message string
	Line    int // 1-based line number
	Offset  int // 0-based byte offset from start of input
}

func (*ParseError) Error

func (p *ParseError) Error() string

type Properties

type Properties struct {
	Name       string // name of the field, for error messages
	OrigName   string // original name before protocol compiler (always set)
	Wire       string
	WireType   int
	Tag        int
	Required   bool
	Optional   bool
	Repeated   bool
	Packed     bool   // relevant for repeated primitives only
	Enum       string // set for enum types only
	Default    string // default value
	CustomType string
	// contains filtered or unexported fields
}

Properties represents the protocol-specific behavior of a single struct field.

func (*Properties) Init

func (p *Properties) Init(typ reflect.Type, name, tag string, f *reflect.StructField)

Init populates the properties from a protocol buffer struct tag.

func (*Properties) Parse

func (p *Properties) Parse(s string)

Parse populates p by parsing a string in the protobuf struct field tag style.

func (*Properties) String

func (p *Properties) String() string

String formats the properties in the protobuf struct field tag style.

type RequiredNotSetError

type RequiredNotSetError struct {
	// contains filtered or unexported fields
}

RequiredNotSetError is the error returned if Marshal is called with a protocol buffer struct whose required fields have not all been initialized. It is also the error returned if Unmarshal is called with an encoded protocol buffer that does not include all the required fields.

When printed, RequiredNotSetError reports the first unset required field in a message. If the field cannot be precisely determined, it is reported as "{Unknown}".

func (*RequiredNotSetError) Error

func (e *RequiredNotSetError) Error() string

type Sizer

type Sizer interface {
	Size() int
}

type Stats

type Stats struct {
	Emalloc uint64 // mallocs in encode
	Dmalloc uint64 // mallocs in decode
	Encode  uint64 // number of encodes
	Decode  uint64 // number of decodes
	Chit    uint64 // number of cache hits
	Cmiss   uint64 // number of cache misses
	Size    uint64 // number of sizes
}

Stats records allocation details about the protocol buffer encoders and decoders. Useful for tuning the library itself.

func GetStats

func GetStats() Stats

GetStats returns a copy of the global Stats structure.

type StructProperties

type StructProperties struct {
	Prop []*Properties // properties for each field
	// contains filtered or unexported fields
}

StructProperties represents properties for all the fields of a struct. decoderTags and decoderOrigNames should only be used by the decoder.

func GetProperties

func GetProperties(t reflect.Type) *StructProperties

GetProperties returns the list of properties for the type represented by t.

func (*StructProperties) Len

func (sp *StructProperties) Len() int

func (*StructProperties) Less

func (sp *StructProperties) Less(i, j int) bool

func (*StructProperties) Swap

func (sp *StructProperties) Swap(i, j int)

type Unmarshaler

type Unmarshaler interface {
	Unmarshal([]byte) error
}

Unmarshaler is the interface representing objects that can unmarshal themselves. The method should reset the receiver before decoding starts. The argument points to data that may be overwritten, so implementations should not keep references to the buffer.