Documentation ¶
Overview ¶
Package serialize provides methods for reading and writing concatenable, bytewise-sortable forms of the datatypes defined in the datastore package.
Index ¶
- Constants
- Variables
- func Increment(bstr []byte) ([]byte, bool)
- func Invert(bs []byte) []byte
- func Join(itms ...[]byte) []byte
- func ReadGeoPoint(buf Buffer) (gp ds.GeoPoint, err error)
- func ReadIndexColumn(buf Buffer) (c ds.IndexColumn, err error)
- func ReadIndexDefinition(buf Buffer) (i ds.IndexDefinition, err error)
- func ReadKey(buf Buffer, context KeyContext, appid, namespace string) (ret *ds.Key, err error)
- func ReadKeyTok(buf Buffer) (ret ds.KeyTok, err error)
- func ReadProperty(buf Buffer, context KeyContext, appid, namespace string) (p ds.Property, err error)
- func ReadPropertyMap(buf Buffer, context KeyContext, appid, namespace string) (pm ds.PropertyMap, err error)
- func ReadTime(buf Buffer) (time.Time, error)
- func ToBytes(i interface{}) []byte
- func ToBytesErr(i interface{}) ([]byte, error)
- func ToBytesWithContext(i interface{}) []byte
- func ToBytesWithContextErr(i interface{}) ([]byte, error)
- func WriteGeoPoint(buf Buffer, gp ds.GeoPoint) (err error)
- func WriteIndexColumn(buf Buffer, c ds.IndexColumn) (err error)
- func WriteIndexDefinition(buf Buffer, i ds.IndexDefinition) (err error)
- func WriteKey(buf Buffer, context KeyContext, k *ds.Key) (err error)
- func WriteKeyTok(buf Buffer, tok ds.KeyTok) (err error)
- func WriteProperty(buf Buffer, context KeyContext, p ds.Property) (err error)
- func WritePropertyMap(buf Buffer, context KeyContext, pm ds.PropertyMap) (err error)
- func WriteTime(buf Buffer, t time.Time) error
- type Buffer
- type InvertibleBuffer
- type KeyContext
- type SerializedPmap
- type SerializedPslice
Constants ¶
const MaxIndexColumns = 64
MaxIndexColumns is the maximum number of sort columns (e.g. sort orders) that ReadIndexDefinition is willing to deserialize. 64 was chosen as a likely-astronomical number.
const ReadKeyNumToksReasonableLimit = 50
ReadKeyNumToksReasonableLimit is the maximum number of Key tokens that ReadKey is willing to read for a single key.
const ReadPropertyMapReasonableLimit uint64 = 30000
ReadPropertyMapReasonableLimit sets a limit on the number of rows and number of properties per row which can be read by ReadPropertyMap. The total number of Property objects readable by this method is this number squared (e.g. Limit rows * Limit properties)
Variables ¶
var WritePropertyMapDeterministic = false
WritePropertyMapDeterministic allows tests to make WritePropertyMap deterministic.
Functions ¶
func Increment ¶
Increment attempts to increment a copy of bstr as if adding 1 to an integer.
If it overflows, the returned []byte will be all 0's, and the overflow bool will be true.
func ReadGeoPoint ¶
ReadGeoPoint reads a GeoPoint from the buffer.
func ReadIndexColumn ¶
func ReadIndexColumn(buf Buffer) (c ds.IndexColumn, err error)
ReadIndexColumn reads an IndexColumn from the buffer.
func ReadIndexDefinition ¶
func ReadIndexDefinition(buf Buffer) (i ds.IndexDefinition, err error)
ReadIndexDefinition reads an IndexDefinition from the buffer.
func ReadKey ¶
ReadKey deserializes a key from the buffer. The value of context must match the value of context that was passed to WriteKey when the key was encoded. If context == WithoutContext, then the appid and namespace parameters are used in the decoded Key. Otherwise they're ignored.
func ReadKeyTok ¶
ReadKeyTok reads a KeyTok from the buffer. You usually want ReadKey instead of this.
func ReadProperty ¶
func ReadProperty(buf Buffer, context KeyContext, appid, namespace string) (p ds.Property, err error)
ReadProperty reads a Property from the buffer. `context`, `appid`, and `namespace` behave the same way they do for ReadKey, but only have an effect if the decoded property has a Key value.
func ReadPropertyMap ¶
func ReadPropertyMap(buf Buffer, context KeyContext, appid, namespace string) (pm ds.PropertyMap, err error)
ReadPropertyMap reads a PropertyMap from the buffer. `context` and friends behave the same way that they do for ReadKey.
func ToBytes ¶
func ToBytes(i interface{}) []byte
ToBytes serializes i to a byte slice, if it's one of the type supported by this library. If an error is encountered (e.g. `i` is not a supported type), this method panics.
Key types will be serialized using the 'WithoutContext' option (e.g. their encoded forms will not contain AppID or Namespace).
func ToBytesErr ¶
ToBytesErr serializes i to a byte slice, if it's one of the type supported by this library, otherwise it returns an error.
Key types will be serialized using the 'WithoutContext' option (e.g. their encoded forms will not contain AppID or Namespace).
func ToBytesWithContext ¶
func ToBytesWithContext(i interface{}) []byte
ToBytesWithContext serializes i to a byte slice, if it's one of the type supported by this library. If an error is encountered (e.g. `i` is not a supported type), this method panics.
Key types will be serialized using the 'WithContext' option (e.g. their encoded forms will not contain AppID or Namespace).
func ToBytesWithContextErr ¶
ToBytesWithContextErr serializes i to a byte slice, if it's one of the type supported by this library, otherwise it returns an error.
Key types will be serialized using the 'WithContext' option (e.g. their encoded forms will contain AppID and Namespace).
func WriteGeoPoint ¶
WriteGeoPoint writes a GeoPoint to the buffer.
func WriteIndexColumn ¶
func WriteIndexColumn(buf Buffer, c ds.IndexColumn) (err error)
WriteIndexColumn writes an IndexColumn to the buffer.
func WriteIndexDefinition ¶
func WriteIndexDefinition(buf Buffer, i ds.IndexDefinition) (err error)
WriteIndexDefinition writes an IndexDefinition to the buffer
func WriteKey ¶
func WriteKey(buf Buffer, context KeyContext, k *ds.Key) (err error)
WriteKey encodes a key to the buffer. If context is WithContext, then this encoded value will include the appid and namespace of the key.
func WriteKeyTok ¶
WriteKeyTok writes a KeyTok to the buffer. You usually want WriteKey instead of this.
func WriteProperty ¶
func WriteProperty(buf Buffer, context KeyContext, p ds.Property) (err error)
WriteProperty writes a Property to the buffer. `context` behaves the same way that it does for WriteKey, but only has an effect if `p` contains a Key as its Value.
func WritePropertyMap ¶
func WritePropertyMap(buf Buffer, context KeyContext, pm ds.PropertyMap) (err error)
WritePropertyMap writes an entire PropertyMap to the buffer. `context` behaves the same way that it does for WriteKey. If WritePropertyMapDeterministic is true, then the rows will be sorted by property name before they're serialized to buf (mostly useful for testing, but also potentially useful if you need to make a hash of the property data).
Write skips metadata keys.
Types ¶
type Buffer ¶
type Buffer interface { String() string Bytes() []byte Len() int Grow(int) Read([]byte) (int, error) ReadByte() (byte, error) Write([]byte) (int, error) WriteByte(c byte) error WriteString(s string) (int, error) }
Buffer is the interface which corresponds to the subset of *bytes.Buffer that this package requires.
type InvertibleBuffer ¶
InvertibleBuffer is just like Buffer, except that it also has a stateful Invert() method, which will cause all reads and writes to/from it to be inverted (e.g. every byte XOR 0xFF).
Implementing queries requires manipulating the index entries (e.g. synthesizing them, parsing them, etc.). In particular, when you have a reverse-sorted field (e.g. high to low instead of low to high), it's achieved by having all the bits inverted.
All the serialization formats include delimiter information, which the parsers only know to parse non-inverted. If we don't have this buffer, we'd basically have to invert every byte in the []byte array when we're trying to decode a reverse-ordered field (including the bytes of all fields after the one we intend to parse) so that the parser can consume as many bytes as it needs (and it only knows the number of bytes it needs as it decodes them). This InvertibleBuffer lets that happen on the fly without having to flip the whole []byte.
If you know you need it, you'll know it's the right thing. If you're not sure then you definitely don't need it!
func Invertible ¶
func Invertible(b Buffer) InvertibleBuffer
Invertible returns an InvertibleBuffer based on the Buffer.
type KeyContext ¶
type KeyContext bool
KeyContext controls whether the various Write and Read serializtion routines should encode the context of Keys (read: the appid and namespace). Frequently the appid and namespace of keys are known in advance and so there's no reason to redundantly encode them.
const ( WithContext KeyContext = true WithoutContext = false )
With- and WithoutContext indicate if the serialization method should include context for Keys. See KeyContext for more information.
type SerializedPmap ¶
type SerializedPmap map[string]SerializedPslice
SerializedPmap maps from
prop name -> [<serialized DSProperty>, ...]
includes special values '__key__' and '__ancestor__' which contains all of the ancestor entries for this key.
func PropertyMapPartially ¶
func PropertyMapPartially(k *ds.Key, pm ds.PropertyMap) (ret SerializedPmap)
PropertyMapPartially turns a regular PropertyMap into a SerializedPmap. Essentially all the []Property's become SerializedPslice, using cmpbin and datastore/serialize's encodings.
type SerializedPslice ¶
type SerializedPslice [][]byte
SerializedPslice is all of the serialized DSProperty values in qASC order.
func PropertySlice ¶
func PropertySlice(vals ds.PropertySlice) SerializedPslice
PropertySlice serializes a single row of a DSProperty map.
func (SerializedPslice) Len ¶
func (s SerializedPslice) Len() int
func (SerializedPslice) Less ¶
func (s SerializedPslice) Less(i, j int) bool
func (SerializedPslice) Swap ¶
func (s SerializedPslice) Swap(i, j int)