schema

package module
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 18, 2024 License: Apache-2.0 Imports: 7 Imported by: 7

README

Logical State Schema Framework

The cosmossdk.io/schema base module is designed to provide a stable, zero-dependency base layer for specifying the logical representation of module state schemas and implementing state indexing. This is intended to be used primarily for indexing modules in external databases and providing a standard human-readable state representation for genesis import and export.

The schema defined in this library does not aim to be general purpose and cover all types of schemas, such as those used for defining transactions. For instance, this schema does not include many types of composite objects such as nested objects and arrays. Rather, the schema defined here aims to cover state schemas only which are implemented as key-value pairs and usually have direct mappings to relational database tables or objects in a document store.

Also, this schema does not cover physical state layout and byte-level encoding, but simply describes a common logical format.

HasModuleCodec Interface

Any module which supports logical decoding and/or encoding should implement the HasModuleCodec interface. This interface provides a way to get the codec for the module, which can be used to decode the module's state and/or apply logical updates.

State frameworks such as collections or orm should directly provide ModuleCodec implementations so that this functionality basically comes for free if a compatible framework is used. Modules that do not use one of these frameworks can choose to manually implement logical decoding and/or encoding.

Documentation

Index

Constants

View Source 
const (
	// IntegerFormat is a regex that describes the format integer number strings must match. It specifies
	// that integers may have at most 100 digits.
	IntegerFormat = `^-?[0-9]{1,100}$`

	// DecimalFormat is a regex that describes the format decimal number strings must match. It specifies
	// that decimals may have at most 50 digits before and after the decimal point and may have an optional
	// exponent of up to 2 digits. These restrictions ensure that the decimal can be accurately represented
	// by a wide variety of implementations.
	DecimalFormat = `^-?[0-9]{1,50}(\.[0-9]{1,50})?([eE][-+]?[0-9]{1,2})?$`
)
View Source 
const MAX_VALID_KIND = JSONKind

MAX_VALID_KIND is the maximum valid kind value.

View Source 
const NameFormat = `^[a-zA-Z_][a-zA-Z0-9_]{0,63}$`

NameFormat is the regular expression that a name must match. A name must start with a letter or underscore and can only contain letters, numbers, and underscores. A name must be at least one character long and can be at most 64 characters long.

Variables

This section is empty.

Functions

func ValidateName 

func ValidateName(name string) bool

ValidateName checks if the given name is a valid name conforming to NameFormat.

func ValidateObjectKey 

func ValidateObjectKey(keyFields []Field, value interface{}, typeSet TypeSet) error

ValidateObjectKey validates that the value conforms to the set of fields as a Key in an StateObjectUpdate. See StateObjectUpdate.Key for documentation on the requirements of such keys.

func ValidateObjectValue 

func ValidateObjectValue(valueFields []Field, value interface{}, typeSet TypeSet) error

ValidateObjectValue validates that the value conforms to the set of fields as a Value in an StateObjectUpdate. See StateObjectUpdate.Value for documentation on the requirements of such values.

Types

type EnumType added in v0.2.0 

type EnumType struct {
	// Name is the name of the enum type.
	// It must conform to the NameFormat regular expression.
	// Its name must be unique between all enum types and object types in the module.
	// The same enum, however, can be used in multiple object types and fields as long as the
	// definition is identical each time.
	Name string `json:"name,omitempty"`

	// Values is a list of distinct, non-empty values that are part of the enum type.
	// Each value must conform to the NameFormat regular expression.
	Values []EnumValueDefinition `json:"values"`

	// NumericKind is the numeric kind used to represent the enum values numerically.
	// If it is left empty, Int32Kind is used by default.
	// Valid values are Uint8Kind, Int8Kind, Uint16Kind, Int16Kind, and Int32Kind.
	NumericKind Kind `json:"numeric_kind,omitempty"`
}

EnumType represents the definition of an enum type.

func (EnumType) GetNumericKind added in v0.2.0 

func (e EnumType) GetNumericKind() Kind

GetNumericKind returns the numeric kind used to represent the enum values numerically. When EnumType.NumericKind is not set, the default value of Int32Kind is returned here.

func (EnumType) TypeName added in v0.2.0 

func (e EnumType) TypeName() string

TypeName implements the Type interface.

func (EnumType) Validate added in v0.2.0 

func (e EnumType) Validate(TypeSet) error

Validate validates the enum definition.

func (EnumType) ValidateValue added in v0.2.0 

func (e EnumType) ValidateValue(value string) error

ValidateValue validates that the value is a valid enum value.

type EnumValueDefinition added in v0.2.0 

type EnumValueDefinition struct {
	// Name is the name of the enum value.
	// It must conform to the NameFormat regular expression.
	// Its name must be unique between all values in the enum.
	Name string `json:"name"`

	// Value is the numeric value of the enum.
	// It must be unique between all values in the enum.
	Value int32 `json:"value"`
}

EnumValueDefinition represents a value in an enum type.

type Field 

type Field struct {
	// Name is the name of the field. It must conform to the NameFormat regular expression.
	Name string `json:"name"`

	// Kind is the basic type of the field.
	Kind Kind `json:"kind"`

	// Nullable indicates whether null values are accepted for the field. Key fields CANNOT be nullable.
	Nullable bool `json:"nullable,omitempty"`

	// ReferencedType is the referenced type name when Kind is EnumKind.
	ReferencedType string `json:"referenced_type,omitempty"`
}

Field represents a field in an object type.

func (Field) Validate 

func (c Field) Validate(typeSet TypeSet) error

Validate validates the field.

func (Field) ValidateValue 

func (c Field) ValidateValue(value interface{}, typeSet TypeSet) error

ValidateValue validates that the value conforms to the field's kind and nullability. Unlike Kind.ValidateValue, it also checks that the value conforms to the EnumType if the field is an EnumKind.

type HasModuleCodec 

type HasModuleCodec interface {
	// ModuleCodec returns a ModuleCodec for the module.
	ModuleCodec() (ModuleCodec, error)
}

HasModuleCodec is an interface that modules can implement to provide a ModuleCodec. Usually these modules would also implement appmodule.AppModule, but that is not included to keep this package free of any dependencies.

type KVDecoder 

type KVDecoder = func(KVPairUpdate) ([]StateObjectUpdate, error)

KVDecoder is a function that decodes a key-value pair into one or more StateObjectUpdate's. If the KV-pair doesn't represent object updates, the function should return nil as the first and no error. The error result should only be non-nil when the decoder expected to parse a valid update and was unable to. In the case of an error, the decoder may return a non-nil value for the first return value, which can indicate which parts of the update were decodable to aid debugging.

type KVPairUpdate 

type KVPairUpdate = struct {
	// Key is the key of the key-value pair.
	Key []byte

	// Value is the value of the key-value pair. It should be ignored when Remove is true.
	Value []byte

	// Remove is a flag that indicates that the key-value pair was deleted. If it is false,
	// then it is assumed that this has been a set operation.
	Remove bool
}

KVPairUpdate represents a key-value pair set or delete.

type Kind 

type Kind int

Kind represents the basic type of a field in an object. Each kind defines the following encodings: Go Encoding: the golang type which should be accepted by listeners and generated by decoders when providing entity updates. JSON Encoding: the JSON encoding which should be used when encoding the field to JSON. When there is some non-determinism in an encoding, kinds should specify what values they accept and also what is the canonical, deterministic encoding which should be preferably emitted by serializers. 

const (
	// InvalidKind indicates that an invalid type.
	InvalidKind Kind = iota

	// StringKind is a string type.
	// Go Encoding: UTF-8 string with no null characters.
	// JSON Encoding: string
	StringKind

	// BytesKind is a bytes type.
	// Go Encoding: []byte
	// JSON Encoding: base64 encoded string, canonical values should be encoded with standard encoding and padding.
	// Either standard or URL encoding with or without padding should be accepted.
	BytesKind

	// Int8Kind represents an 8-bit signed integer.
	// Go Encoding: int8
	// JSON Encoding: number
	Int8Kind

	// Uint8Kind represents an 8-bit unsigned integer.
	// Go Encoding: uint8
	// JSON Encoding: number
	Uint8Kind

	// Int16Kind represents a 16-bit signed integer.
	// Go Encoding: int16
	// JSON Encoding: number
	Int16Kind

	// Uint16Kind represents a 16-bit unsigned integer.
	// Go Encoding: uint16
	// JSON Encoding: number
	Uint16Kind

	// Int32Kind represents a 32-bit signed integer.
	// Go Encoding: int32
	// JSON Encoding: number
	Int32Kind

	// Uint32Kind represents a 32-bit unsigned integer.
	// Go Encoding: uint32
	// JSON Encoding: number
	Uint32Kind

	// Int64Kind represents a 64-bit signed integer.
	// Go Encoding: int64
	// JSON Encoding: base10 integer string which matches the IntegerFormat regex
	// The canonical encoding should include no leading zeros.
	Int64Kind

	// Uint64Kind represents a 64-bit unsigned integer.
	// Go Encoding: uint64
	// JSON Encoding: base10 integer string which matches the IntegerFormat regex
	// Canonically encoded values should include no leading zeros.
	Uint64Kind

	// IntegerKind represents an arbitrary precision integer number.
	// Support for expressing the maximum bit precision of values will be added in the future.
	// Go Encoding: string which matches the IntegerFormat regex (unstable, subject to change).
	// JSON Encoding: base10 integer string
	// Canonically encoded values should include no leading zeros.
	// Equality comparison with integers should be done using numerical equality rather
	// than string equality.
	IntegerKind

	// DecimalKind represents an arbitrary precision decimal or integer number.
	// Support for optionally limiting the precision may be added in the future.
	// Go Encoding: string which matches the DecimalFormat regex
	// JSON Encoding: base10 decimal string
	// Canonically encoded values should include no leading zeros or trailing zeros,
	// and exponential notation with a lowercase 'e' should be used for any numbers
	// with an absolute value less than or equal to 1e-6 or greater than or equal to 1e6.
	// Equality comparison with decimals should be done using numerical equality rather
	// than string equality.
	DecimalKind

	// BoolKind represents a boolean true or false value.
	// Go Encoding: bool
	// JSON Encoding: boolean
	BoolKind

	// TimeKind represents a nanosecond precision UNIX time value (with zero representing January 1, 1970 UTC).
	// Its valid range is +/- 2^63 (the range of a 64-bit signed integer).
	// Go Encoding: time.Time
	// JSON Encoding: Any value IS0 8601 time stamp should be accepted.
	// Canonical values should be encoded with UTC time zone Z, nanoseconds should
	// be encoded with no trailing zeros, and T time values should always be present
	// even at 00:00:00.
	TimeKind

	// DurationKind represents the elapsed time between two nanosecond precision time values.
	// Its valid range is +/- 2^63 (the range of a 64-bit signed integer).
	// Go Encoding: time.Duration
	// JSON Encoding: the number of seconds as a decimal string with no trailing zeros followed by
	// a lowercase 's' character to represent seconds.
	DurationKind

	// Float32Kind represents an IEEE-754 32-bit floating point number.
	// Go Encoding: float32
	// JSON Encoding: number
	Float32Kind

	// Float64Kind represents an IEEE-754 64-bit floating point number.
	// Go Encoding: float64
	// JSON Encoding: number
	Float64Kind

	// AddressKind represents an account address which is represented by a variable length array of bytes.
	// Addresses usually have a human-readable rendering, such as bech32, and tooling should provide
	// a way for apps to define a string encoder for friendly user-facing display.
	// Go Encoding: []byte
	// JSON Encoding: addresses should be encoded as strings using the human-readable address renderer
	// provided to the JSON encoder.
	AddressKind

	// EnumKind represents a value of an enum type.
	// Fields of this type are expected to set the EnumType field in the field definition to the enum
	// definition.
	// Go Encoding: string
	// JSON Encoding: string
	EnumKind

	// JSONKind represents arbitrary JSON data.
	// Go Encoding: json.RawMessage
	// JSON Encoding: any valid JSON value
	JSONKind
)

func KindForGoValue 

func KindForGoValue(value interface{}) Kind

KindForGoValue finds the simplest kind that can represent the given go value. It will not, however, return kinds such as IntegerKind, DecimalKind, AddressKind, or EnumKind which all can be represented as strings.

func (Kind) MarshalJSON added in v0.2.0 

func (t Kind) MarshalJSON() ([]byte, error)

MarshalJSON marshals the kind to a JSON string and returns an error if the kind is invalid.

func (Kind) String 

func (t Kind) String() string

String returns a string representation of the kind.

func (*Kind) UnmarshalJSON added in v0.2.0 

func (t *Kind) UnmarshalJSON(data []byte) error

UnmarshalJSON unmarshals the kind from a JSON string and returns an error if the kind is invalid.

func (Kind) ValidKeyKind added in v0.2.0 

func (t Kind) ValidKeyKind() bool

ValidKeyKind returns true if the kind is a valid key kind. All kinds except Float32Kind, Float64Kind, and JSONKind are valid key kinds because they do not define a strict form of equality.

func (Kind) Validate 

func (t Kind) Validate() error

Validate returns an errContains if the kind is invalid.

func (Kind) ValidateValue 

func (t Kind) ValidateValue(value interface{}) error

ValidateValue returns an errContains if the value does not conform to the expected go type and format. It is more thorough, but slower, than Kind.ValidateValueType and validates that Integer, Decimal and JSON values are formatted correctly. It cannot validate enum values because Kind's do not have enum schemas.

func (Kind) ValidateValueType 

func (t Kind) ValidateValueType(value interface{}) error

ValidateValueType returns an errContains if the value does not conform to the expected go type. Some fields may accept nil values, however, this method does not have any notion of nullability. This method only validates that the go type of the value is correct for the kind and does not validate string or json formats. Kind.ValidateValue does a more thorough validation of number and json string formatting.

type MapValueUpdates 

type MapValueUpdates map[string]interface{}

MapValueUpdates is a map-based implementation of ValueUpdates which always iterates over keys in sorted order.

func (MapValueUpdates) Iterate 

func (m MapValueUpdates) Iterate(fn func(col string, value interface{}) bool) error

Iterate implements the ValueUpdates interface.

type ModuleCodec 

type ModuleCodec struct {
	// Schema is the schema for the module. It is required.
	Schema ModuleSchema

	// KVDecoder is a function that decodes a key-value pair into an StateObjectUpdate.
	// If it is nil, the module doesn't support state decoding directly.
	KVDecoder KVDecoder
}

ModuleCodec is a struct that contains the schema and a KVDecoder for a module.

type ModuleSchema 

type ModuleSchema struct {
	// contains filtered or unexported fields
}

ModuleSchema represents the logical schema of a module for purposes of indexing and querying.

func CompileModuleSchema added in v0.2.0 

func CompileModuleSchema(types ...Type) (ModuleSchema, error)

CompileModuleSchema compiles the types into a ModuleSchema and validates it. Any module schema returned without an error is guaranteed to be valid.

func MustCompileModuleSchema added in v0.2.0 

func MustCompileModuleSchema(types ...Type) ModuleSchema

MustCompileModuleSchema constructs a new ModuleSchema and panics if it is invalid. This should only be used in test code or static initialization where it is safe to panic!

func (ModuleSchema) AllTypes added in v0.2.0 

func (s ModuleSchema) AllTypes(f func(Type) bool)

AllTypes calls the provided function for each type in the module schema and stops if the function returns false. The types are iterated over in sorted order by name. This function is compatible with go 1.23 iterators.

func (ModuleSchema) EnumTypes added in v0.2.0 

func (s ModuleSchema) EnumTypes(f func(EnumType) bool)

EnumTypes iterators over all the enum types in the schema in alphabetical order.

func (ModuleSchema) LookupEnumType added in v0.3.0 

func (s ModuleSchema) LookupEnumType(name string) (t EnumType, found bool)

LookupEnumType is a convenience method that looks up an EnumType by name.

func (ModuleSchema) LookupStateObjectType added in v0.3.0 

func (s ModuleSchema) LookupStateObjectType(name string) (t StateObjectType, found bool)

LookupObjectType is a convenience method that looks up an ObjectType by name.

func (ModuleSchema) LookupType added in v0.2.0 

func (s ModuleSchema) LookupType(name string) (Type, bool)

LookupType looks up a type by name in the module schema.

func (ModuleSchema) MarshalJSON added in v0.2.0 

func (s ModuleSchema) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface for ModuleSchema. It marshals the module schema into a JSON object with the object types and enum types under the keys "object_types" and "enum_types" respectively.

func (ModuleSchema) StateObjectTypes added in v0.3.0 

func (s ModuleSchema) StateObjectTypes(f func(StateObjectType) bool)

ObjectTypes iterators over all the object types in the schema in alphabetical order.

func (*ModuleSchema) UnmarshalJSON added in v0.2.0 

func (s *ModuleSchema) UnmarshalJSON(data []byte) error

UnmarshalJSON implements the json.Unmarshaler interface for ModuleSchema. See MarshalJSON for the JSON format.

func (ModuleSchema) Validate 

func (s ModuleSchema) Validate() error

Validate validates the module schema.

func (ModuleSchema) ValidateObjectUpdate 

func (s ModuleSchema) ValidateObjectUpdate(update StateObjectUpdate) error

ValidateObjectUpdate validates that the update conforms to the module schema.

type ReferenceType added in v0.3.0 

type ReferenceType interface {
	Type
	// contains filtered or unexported methods
}

ReferenceType is a marker interface that all types that can be the target of Field.ReferencedType implement. Currently, this is only EnumType.

type StateObjectType added in v0.3.0 

type StateObjectType struct {
	// Name is the name of the object type. It must be unique within the module schema amongst all object and enum
	// types and conform to the NameFormat regular expression.
	Name string `json:"name"`

	// KeyFields is a list of fields that make up the primary key of the object.
	// It can be empty in which case indexers should assume that this object is
	// a singleton and only has one value. Field names must be unique within the
	// object between both key and value fields.
	// Key fields CANNOT be nullable and Float32Kind, Float64Kind, and JSONKind types
	// are not allowed.
	KeyFields []Field `json:"key_fields,omitempty"`

	// ValueFields is a list of fields that are not part of the primary key of the object.
	// It can be empty in the case where all fields are part of the primary key.
	// Field names must be unique within the object between both key and value fields.
	ValueFields []Field `json:"value_fields,omitempty"`

	// RetainDeletions is a flag that indicates whether the indexer should retain
	// deleted rows in the database and flag them as deleted rather than actually
	// deleting the row. For many types of data in state, the data is deleted even
	// though it is still valid in order to save space. Indexers will want to have
	// the option of retaining such data and distinguishing from other "true" deletions.
	RetainDeletions bool `json:"retain_deletions,omitempty"`
}

StateObjectType describes an object type a module schema.

func (StateObjectType) TypeName added in v0.3.0 

func (o StateObjectType) TypeName() string

TypeName implements the Type interface.

func (StateObjectType) Validate added in v0.3.0 

func (o StateObjectType) Validate(typeSet TypeSet) error

Validate validates the object type.

func (StateObjectType) ValidateObjectUpdate added in v0.3.0 

func (o StateObjectType) ValidateObjectUpdate(update StateObjectUpdate, typeSet TypeSet) error

ValidateObjectUpdate validates that the update conforms to the object type.

type StateObjectUpdate added in v0.3.0 

type StateObjectUpdate struct {
	// TypeName is the name of the object type in the module's schema.
	TypeName string

	// Key returns the value of the primary key of the object and must conform to these constraints with respect
	// that the schema that is defined for the object:
	// - if key represents a single field, then the value must be valid for the first field in that
	// 	field list. For instance, if there is one field in the key of type String, then the value must be of
	//  type string
	// - if key represents multiple fields, then the value must be a slice of values where each value is valid
	//  for the corresponding field in the field list. For instance, if there are two fields in the key of
	//  type String, String, then the value must be a slice of two strings.
	// If the key has no fields, meaning that this is a singleton object, then this value is ignored and can be nil.
	Key interface{}

	// Value returns the non-primary key fields of the object and can either conform to the same constraints
	// as StateObjectUpdate.Key or it may be and instance of ValueUpdates. ValueUpdates can be used as a performance
	// optimization to avoid copying the values of the object into the update and/or to omit unchanged fields.
	// If this is a delete operation, then this value is ignored and can be nil.
	Value interface{}

	// Delete is a flag that indicates whether this update is a delete operation. If true, then the Value field
	// is ignored and can be nil.
	Delete bool
}

StateObjectUpdate represents an update operation on an object in a module's state.

type Type added in v0.2.0 

type Type interface {
	// TypeName returns the type's name.
	TypeName() string

	// Validate validates the type.
	Validate(TypeSet) error
	// contains filtered or unexported methods
}

Type is an interface that all types in the schema implement. Currently, these are StateObjectType and EnumType.

type TypeSet added in v0.2.0 

type TypeSet interface {
	// LookupType looks up a type by name.
	LookupType(name string) (t Type, found bool)

	// LookupEnumType is a convenience method that looks up an EnumType by name.
	LookupEnumType(name string) (t EnumType, found bool)

	// LookupStateObjectType is a convenience method that looks up an StateObjectType by name.
	LookupStateObjectType(name string) (t StateObjectType, found bool)

	// AllTypes calls the given function for each type in the type set.
	// This function is compatible with go 1.23 iterators and can be used like this:
	// for t := range types.AllTypes {
	//     // do something with t
	// }
	AllTypes(f func(Type) bool)

	// EnumTypes calls the given function for each EnumType in the type set.
	// This function is compatible with go 1.23 iterators.
	EnumTypes(f func(EnumType) bool)

	// StateObjectTypes calls the given function for each StateObjectType in the type set.
	// This function is compatible with go 1.23 iterators.
	StateObjectTypes(f func(objectType StateObjectType) bool)
	// contains filtered or unexported methods
}

TypeSet represents something that has types and allows them to be looked up by name. Currently, the only implementation is ModuleSchema.

func EmptyTypeSet added in v0.2.0 

func EmptyTypeSet() TypeSet

EmptyTypeSet is a schema that contains no types. It can be used in Validate methods when there is no schema needed or available.

type ValueUpdates 

type ValueUpdates interface {
	// Iterate iterates over the fields and values in the object update. The function should return
	// true to continue iteration or false to stop iteration. Each field value should conform
	// to the requirements of that field's type in the schema. Iterate returns an error if
	// it was unable to decode the values properly (which could be the case in lazy evaluation).
	Iterate(func(col string, value interface{}) bool) error
}

ValueUpdates is an interface that represents the value fields of an object update. fields that were not updated may be excluded from the update. Consumers should be aware that implementations may not filter out fields that were unchanged. However, if a field is omitted from the update it should be considered unchanged.

Source Files

View all Source files

Directories

Path Synopsis
Package diff provides the CompareModuleSchemas function which compares module schemas and their types and generates a structured diff.
 Package diff provides the CompareModuleSchemas function which compares module schemas and their types and generates a structured diff.
Package logutil defines the Logger interface expected by indexer implementations.
 Package logutil defines the Logger interface expected by indexer implementations.
Package view defines interfaces for viewing the data stored in an indexer target or an app's original data store.
 Package view defines interfaces for viewing the data stored in an indexer target or an app's original data store.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.