schema

package
v1.0.0-beta.122 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 28, 2023 License: Apache-2.0 Imports: 27 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	SearchId           = "id"
	TigrisFieldsPrefix = "_tigris_"
)
View Source 
const (
	PrimaryKeyIndexName = "pkey"
	AutoPrimaryKeyF     = "id"
	PrimaryKeySchemaK   = "primary_key"
	// DateTimeFormat represents the supported date time format.
	DateTimeFormat  = time.RFC3339Nano
	CollectionTypeF = "collection_type"

	SecondaryKeyIndexName = "skey"
)
View Source 
const (
	ObjFlattenDelimiter = "."
)

Variables

View Source 
var (
	MsgFieldNameInvalidPattern = "" /* 131-byte string literal not displayed */
	ValidFieldNamePattern      = regexp.MustCompile(`^[a-zA-Z_$][a-zA-Z0-9_$]*$`)

	ErrInvalidType               = fmt.Errorf("invalid type. expected string or array of strings")
	ErrOnlyOneNonNullTypeAllowed = fmt.Errorf("only one non-null type allowed for the field")
)
View Source 
var (
	ErrIncompatibleSchema = fmt.Errorf("error incompatible schema")
	ErrExpectedString     = fmt.Errorf("expected string type")
	ErrExpectedNumber     = fmt.Errorf("expected json.Number")
	ErrUnsupportedType    = fmt.Errorf("unsupported type")
)

Supported subtypes.

View Source 
var (
	ErrCollectionNameMismatch = errors.InvalidArgument("mismatch in the collection name")
	ErrIndexNameMismatch      = errors.InvalidArgument("mismatch in the index name")
)
View Source 
var FieldNames = [...]string{
	UnknownType:  "unknown",
	NullType:     "null",
	BoolType:     "bool",
	Int32Type:    "int32",
	Int64Type:    "int64",
	DoubleType:   "double",
	StringType:   "string",
	ByteType:     "byte",
	UUIDType:     "uuid",
	DateTimeType: "datetime",
	ArrayType:    "array",
	ObjectType:   "object",
	VectorType:   "vector",
}
View Source 
var ReservedFields = [...]string{
	CreatedAt:           TigrisFieldsPrefix + "created_at",
	UpdatedAt:           TigrisFieldsPrefix + "updated_at",
	Metadata:            TigrisFieldsPrefix + "metadata",
	IdToSearchKey:       TigrisFieldsPrefix + "id",
	DateSearchKeyPrefix: TigrisFieldsPrefix + "date_",
	SearchArrNullItem:   TigrisFieldsPrefix + "null",
	SearchNullKeys:      TigrisFieldsPrefix + "null_keys",
}
View Source 
var SearchIndexStateNames = [...]string{
	UnknownSearchState:   "Search Status Unknown",
	SearchIndexActive:    "Search Active",
	SearchIndexWriteMode: "Search Write Mode",
	SearchIndexBuilding:  "Search Building",
	NoSearchIndex:        "No Search Index",
}
View Source 
var SupportedFieldProperties = container.NewHashSet(
	"type",
	"format",
	"items",
	"maxLength",
	"description",
	"contentEncoding",
	"properties",
	"autoGenerate",
	"sorted",
	"sort",
	"index",
	"facet",
	"searchIndex",
	"default",
	"createdAt",
	"updatedAt",
	"title",
	"required",
	"index",
	"facet",
	"searchIndex",
	"maxItems",
	"additionalProperties",
	"dimensions",
	"id",
)

Functions

func ApplyBackwardCompatibilityRules 

func ApplyBackwardCompatibilityRules(existing *DefaultCollection, current *Factory) error

ApplyBackwardCompatibilityRules is to validate incoming collection request against the existing present collection. It performs following validations,

  • Primary Key Changed, or order of fields part of the primary key is changed
  • Collection name change
  • A validation on field property is also applied like for instance if existing field has some property, but it is removed in the new schema
  • Any index exist on the collection will also have same checks like type, etc

func ApplySearchIndexBackwardCompatibilityRules 

func ApplySearchIndexBackwardCompatibilityRules(existing *SearchIndex, current *SearchFactory) error

ApplySearchIndexBackwardCompatibilityRules is to validate incoming index schema against the existing index version. It performs following validations,

  • Collection name change.
  • A validation on field like changing the type is not allowed.
  • A validation on index source, it is defined during index creation and should not change afterwards.

func DefaultFacetableType 

func DefaultFacetableType(fieldType FieldType) bool

DefaultFacetableType are the types for which faceting is automatically enabled for database collections.

func DefaultSortableType 

func DefaultSortableType(fieldType FieldType) bool

DefaultSortableType are the types for which sorting is automatically enabled.

func Generate 

func Generate(jsonSchema []byte, format string) ([]byte, error)

Generate schema in the requested format.

func HasIndex 

func HasIndex(indexes []*Index, idx *Index) bool

func Infer 

func Infer(sch *schema.Schema, name string, docs [][]byte, primaryKey []string, autoGenerate []string,
	depth int,
) error

func IsPrimitiveType 

func IsPrimitiveType(fieldType FieldType) bool

func IsReservedField 

func IsReservedField(name string) bool

func IsSearchID 

func IsSearchID(name string) bool

func IsValidKeyType 

func IsValidKeyType(t FieldType) bool

func SupportedFacetableType 

func SupportedFacetableType(fieldType FieldType, subType FieldType) bool

func SupportedIndexableType 

func SupportedIndexableType(fieldType FieldType) bool

func SupportedSearchIndexableType 

func SupportedSearchIndexableType(fieldType FieldType, subType FieldType) bool

func ToSearchDateKey 

func ToSearchDateKey(key string) string

ToSearchDateKey can be used to generate storage field for search backend Original date strings are persisted as it is under this field.

func ValidateFieldAttributes 

func ValidateFieldAttributes(isSearch bool, field *Field) error

ValidateFieldAttributes is validated when a schema request is received. Each builder(collection/search) calls this method to validate if field is properly formed. This is done outside the fieldBuilder to avoid doing validation during reloading of schemas. This method recursively checks each field that whether the attributes are properly set or not.

func ValidateFieldBuilder 

func ValidateFieldBuilder(f FieldBuilder) error

ValidateFieldBuilder is to validate before building a field. This is done because there are some validation that can't be caught in the "ValidateFieldAttributes" like the values for default fields. Anything that needs to be validated on builder and can't be done on field has to be done here. Similar to ValidateFieldAttributes, this is also only done during incoming requests and ignored during reloading of schemas.

func ValidateSupportedProperties 

func ValidateSupportedProperties(v []byte) error

ValidateSupportedProperties to validate what all JSON tagging is allowed on a field. We can extend this to also validate based on whether the schema is for search or for collection.

Types

type CollectionType 

type CollectionType string
const (
	DocumentsType CollectionType = "documents"
)

func GetCollectionType 

func GetCollectionType(_ jsoniter.RawMessage) (CollectionType, error)

type DefaultCollection 

type DefaultCollection struct {
	// Id is the dictionary encoded value for this collection.
	Id uint32
	// SchVer returns the schema version
	SchVer uint32
	// Name is the name of the collection.
	Name string
	// EncodedName is the encoded name of the collection.
	EncodedName []byte
	// EncodedTableIndexName is the encoded name of the collection's Secondary Index.
	EncodedTableIndexName []byte
	// Fields are derived from the user schema.
	Fields []*Field
	// Indexes is a wrapper on the indexes part of this collection.
	// Primary Key contains the fields used to make up the primary key
	PrimaryKey *Index
	// Secondary SecondaryIndexes for this collection
	SecondaryIndexes *Indexes
	// Validator is used to validate the JSON document. As it is expensive to create this, it is only created once
	// during constructor of the collection.
	Validator *jsonschema.Schema
	// JSON schema
	Schema jsoniter.RawMessage
	// SchemaDeltas contains incompatible schema changes from version to version
	SchemaDeltas []VersionDelta
	// FieldVersions contains the list of schema versions at which the field had incompatible change
	FieldVersions map[string]*FieldVersions
	// ImplicitSearchIndex is created by the Tigris to use a search index for in-memory indexes. This is needed till we move
	// to secondary indexes which will be stored in FDB.
	ImplicitSearchIndex *ImplicitSearchIndex
	// search indexes are indexes that are explicitly created by the user and tagged Tigris as source. Collection will be
	// responsible for ensuring these indexes are in sync when any mutation happens to this collection.
	SearchIndexes map[string]*SearchIndex
	// QueryableFields are similar to Fields but these are flattened forms of fields. For instance, a simple field
	// will be one to one mapped to queryable field but complex fields like object type field there may be more than
	// one queryableFields. As queryableFields represent a flattened state these can be used as-is to index in memory.
	QueryableFields []*QueryableField
	// CollectionType is the type of the collection. Only two types of collections are supported "messages" and "documents"
	CollectionType CollectionType

	// This is the existing fields in search
	FieldsInSearch []tsApi.Field
	// contains filtered or unexported fields
}

DefaultCollection is used to represent a collection. The tenant in the metadata package is responsible for creating the collection.

func NewDefaultCollection 

func NewDefaultCollection(id uint32, schVer uint32, factory *Factory, schemas Versions,
	implicitSearchIndex *ImplicitSearchIndex,
) (*DefaultCollection, error)

func (*DefaultCollection) AddSearchIndex 

func (d *DefaultCollection) AddSearchIndex(index *SearchIndex)

func (*DefaultCollection) CompatibleSchemaSince 

func (d *DefaultCollection) CompatibleSchemaSince(version uint32) bool

CompatibleSchemaSince determines if there was incompatible schema change since given version.

func (*DefaultCollection) GetActiveIndexedFields 

func (d *DefaultCollection) GetActiveIndexedFields() []*QueryableField

GetActiveIndexedFields returns indexes that can be used for queries.

func (*DefaultCollection) GetField 

func (d *DefaultCollection) GetField(name string) *Field

func (*DefaultCollection) GetFields 

func (d *DefaultCollection) GetFields() []*Field

func (*DefaultCollection) GetImplicitSearchIndex 

func (d *DefaultCollection) GetImplicitSearchIndex() *ImplicitSearchIndex

func (*DefaultCollection) GetIndexedFields 

func (d *DefaultCollection) GetIndexedFields() []*QueryableField

func (*DefaultCollection) GetIndexes 

func (d *DefaultCollection) GetIndexes() *Indexes

func (*DefaultCollection) GetInt64FieldsPath 

func (d *DefaultCollection) GetInt64FieldsPath() map[string]struct{}

func (*DefaultCollection) GetName 

func (d *DefaultCollection) GetName() string

func (*DefaultCollection) GetPrimaryIndexedFields 

func (d *DefaultCollection) GetPrimaryIndexedFields() []*QueryableField

func (*DefaultCollection) GetPrimaryKey 

func (d *DefaultCollection) GetPrimaryKey() *Index

func (*DefaultCollection) GetQueryableField 

func (d *DefaultCollection) GetQueryableField(name string) (*QueryableField, error)

func (*DefaultCollection) GetQueryableFields 

func (d *DefaultCollection) GetQueryableFields() []*QueryableField

func (*DefaultCollection) GetSearchState 

func (d *DefaultCollection) GetSearchState() SearchIndexState

func (*DefaultCollection) GetVersion 

func (d *DefaultCollection) GetVersion() uint32

func (*DefaultCollection) GetWriteModeIndexes 

func (d *DefaultCollection) GetWriteModeIndexes() []*QueryableField

func (*DefaultCollection) LookupFieldVersion 

func (d *DefaultCollection) LookupFieldVersion(keyPath []string) []*FieldVersion

LookupFieldVersion returns the list of the schema versions for the specified keyPath, when this field had incompatible change. Returns nil if the field has never been changed, meaning that it has version 1.

func (*DefaultCollection) SecondaryIndexKeyword 

func (*DefaultCollection) SecondaryIndexKeyword() string

SecondaryIndexKeyword is the subspace within a collection where the secondary index information is stored.

func (*DefaultCollection) SecondaryIndexMetadata 

func (d *DefaultCollection) SecondaryIndexMetadata() bool

func (*DefaultCollection) TaggedDefaultsForInsert 

func (d *DefaultCollection) TaggedDefaultsForInsert() map[string]struct{}

func (*DefaultCollection) TaggedDefaultsForUpdate 

func (d *DefaultCollection) TaggedDefaultsForUpdate() map[string]struct{}

func (*DefaultCollection) Type 

func (d *DefaultCollection) Type() CollectionType

func (*DefaultCollection) UpdateRowSchema 

func (d *DefaultCollection) UpdateRowSchema(doc map[string]any, version uint32)

UpdateRowSchema fixes the schema of provided, unmarshalled, document from given Version to the latest collection schema.

func (*DefaultCollection) UpdateRowSchemaRaw 

func (d *DefaultCollection) UpdateRowSchemaRaw(doc []byte, version uint32) ([]byte, error)

UpdateRowSchemaRaw fixes the schema of provided document from given Version to the latest collection schema.

func (*DefaultCollection) Validate 

func (d *DefaultCollection) Validate(document any) error

Validate expects an unmarshalled document which it will validate again the schema of this collection.

type Factory 

type Factory struct {
	// Name is the collection name of this schema.
	Name string
	// Fields are derived from the user schema.
	Fields []*Field
	// Primary Key points to the fields used for the primary index. At this point the dictionary encoded value is not
	// set for these indexes which is set as part of collection creation.
	PrimaryKey *Index
	// Indexes is a wrapper on the secondary indexes part of this collection.
	Indexes *Indexes
	// Schema is the raw JSON schema received as part of CreateOrUpdateCollection request. This is stored as-is in the
	// schema subspace.
	Schema jsoniter.RawMessage
	// CollectionType is the type of the collection. Only two types of collections are supported "messages" and "documents"
	CollectionType CollectionType
	Version        uint32
}

Factory is used as an intermediate step so that collection can be initialized with properly encoded values.

func (*Factory) SecondaryIndexes 

func (f *Factory) SecondaryIndexes() []*Index

type FactoryBuilder 

type FactoryBuilder struct {
	// contains filtered or unexported fields
}

func NewFactoryBuilder 

func NewFactoryBuilder(onUserRequest bool) *FactoryBuilder

func (*FactoryBuilder) Build 

func (fb *FactoryBuilder) Build(collection string, reqSchema jsoniter.RawMessage) (*Factory, error)

Build is used to deserialize the user json schema into a schema factory.

func (*FactoryBuilder) BuildSearch 

func (fb *FactoryBuilder) BuildSearch(index string, reqSchema jsoniter.RawMessage) (*SearchFactory, error)

type Field 

type Field struct {
	FieldName       string
	Defaulter       *FieldDefaulter
	DataType        FieldType
	MaxLength       *int32
	FillCreatedAt   *bool
	FillUpdatedAt   *bool
	UniqueKeyField  *bool
	PrimaryKeyField *bool
	AutoGenerated   *bool
	Sorted          *bool
	Indexed         *bool
	Faceted         *bool
	SearchIndexed   *bool
	SearchIdField   *bool
	Dimensions      *int
	// Nested fields are the fields where we know the schema of nested attributes like if properties are
	Fields               []*Field
	AdditionalProperties *bool
}

func GetField 

func GetField(fields []*Field, name string) *Field

func (*Field) GetDefaulter 

func (f *Field) GetDefaulter() *FieldDefaulter

func (*Field) GetDimensions 

func (f *Field) GetDimensions() int

func (*Field) GetNestedField 

func (f *Field) GetNestedField(name string) *Field

func (*Field) IsAutoGenerated 

func (f *Field) IsAutoGenerated() bool

func (*Field) IsCompatible 

func (f *Field) IsCompatible(keyPath string, f1 *Field) error

func (*Field) IsFaceted 

func (f *Field) IsFaceted() bool

func (*Field) IsIndexable 

func (f *Field) IsIndexable() bool

func (*Field) IsIndexed 

func (f *Field) IsIndexed() bool

func (*Field) IsMap 

func (f *Field) IsMap() bool

func (*Field) IsPrimaryKey 

func (f *Field) IsPrimaryKey() bool

func (*Field) IsSearchId 

func (f *Field) IsSearchId() bool

func (*Field) IsSearchIndexed 

func (f *Field) IsSearchIndexed() bool

func (*Field) IsSorted 

func (f *Field) IsSorted() bool

func (*Field) Name 

func (f *Field) Name() string

func (*Field) Type 

func (f *Field) Type() FieldType

type FieldBuilder 

type FieldBuilder struct {
	FieldName            string
	Description          string                `json:"description,omitempty"`
	TypeRaw              schema.FieldMultiType `json:"type,omitempty"`
	Format               string                `json:"format,omitempty"`
	Encoding             string                `json:"contentEncoding,omitempty"`
	Default              any                   `json:"default,omitempty"`
	CreatedAt            *bool                 `json:"createdAt,omitempty"`
	UpdatedAt            *bool                 `json:"updatedAt,omitempty"`
	MaxLength            *int32                `json:"maxLength,omitempty"`
	MaxItems             *int32                `json:"maxItems,omitempty"`
	Auto                 *bool                 `json:"autoGenerate,omitempty"`
	Sorted               *bool                 `json:"sort,omitempty"`
	Index                *bool                 `json:"index,omitempty"`
	Facet                *bool                 `json:"facet,omitempty"`
	ID                   *bool                 `json:"id,omitempty"`
	SearchIndex          *bool                 `json:"searchIndex,omitempty"`
	Dimensions           *int                  `json:"dimensions,omitempty"`
	Items                *FieldBuilder         `json:"items,omitempty"`
	Properties           jsoniter.RawMessage   `json:"properties,omitempty"`
	Primary              *bool
	Fields               []*Field
	AdditionalProperties *bool `json:"additionalProperties,omitempty"`
}

func (*FieldBuilder) Build 

func (f *FieldBuilder) Build(setSearchDefaults bool) (*Field, error)

func (*FieldBuilder) JSONType 

func (f *FieldBuilder) JSONType() string

func (*FieldBuilder) Type 

func (f *FieldBuilder) Type() FieldType

type FieldDefaulter 

type FieldDefaulter struct {
	// contains filtered or unexported fields
}

func (*FieldDefaulter) GetValue 

func (defaulter *FieldDefaulter) GetValue() any

GetValue returns the value if there is default tag set for a field. For functions, it will execute it and return the value. If the function is now() then it is converted to createdAt during the construction of the defaulter.

func (*FieldDefaulter) TaggedWithCreatedAt 

func (defaulter *FieldDefaulter) TaggedWithCreatedAt() bool

func (*FieldDefaulter) TaggedWithUpdatedAt 

func (defaulter *FieldDefaulter) TaggedWithUpdatedAt() bool

type FieldSchemaValidator 

type FieldSchemaValidator struct{}

func (*FieldSchemaValidator) Validate 

func (v *FieldSchemaValidator) Validate(existing *DefaultCollection, current *Factory) error

func (*FieldSchemaValidator) ValidateIndex 

func (v *FieldSchemaValidator) ValidateIndex(existing *SearchIndex, current *SearchFactory) error

type FieldType 

type FieldType int
const (
	UnknownType FieldType = iota
	NullType
	BoolType
	Int32Type
	Int64Type
	DoubleType
	StringType
	// ByteType is a base64 encoded characters, this means if this type is used as key then we need to decode it
	// and then use it as key.
	ByteType
	UUIDType
	// DateTimeType is a valid date representation as defined by RFC 3339, see https://datatracker.ietf.org/doc/html/rfc3339#section-5.6
	DateTimeType
	ArrayType
	ObjectType
	VectorType
	// For internal querying usage.
	MaxType
)

func ToFieldType 

func ToFieldType(jsonType string, encoding string, format string) FieldType

type FieldVersion 

type FieldVersion struct {
	Change  *VersionDeltaField
	Version uint32
}

FieldVersion contains individual field change, along with the schema version at which this change has happened.

type FieldVersions 

type FieldVersions struct {
	// contains filtered or unexported fields
}

FieldVersions contains all the changes of the field, across all the schema versions.

type ImplicitSearchIndex 

type ImplicitSearchIndex struct {
	// Name is the name of the index.
	Name string
	// StoreSchema is the search schema of the underlying search engine.
	StoreSchema *tsApi.CollectionSchema
	// QueryableFields are similar to Fields but these are flattened forms of fields. For instance, a simple field
	// will be one to one mapped to queryable field but complex fields like object type field there may be more than
	// one queryableFields. As queryableFields represent a flattened state these can be used as-is to index in memory.
	QueryableFields []*QueryableField
	// contains filtered or unexported fields
}

ImplicitSearchIndex is a search index that is automatically created by Tigris when a collection is created. Lifecycle of this index is tied to the collection.

func NewImplicitSearchIndex 

func NewImplicitSearchIndex(name string, searchStoreName string, fields []*Field, prevVersionInSearch []tsApi.Field) *ImplicitSearchIndex

func (*ImplicitSearchIndex) GetSearchDeltaFields 

func (s *ImplicitSearchIndex) GetSearchDeltaFields(existingFields []*QueryableField, incomingFields []*Field) []tsApi.Field

func (*ImplicitSearchIndex) GetState 

func (s *ImplicitSearchIndex) GetState() SearchIndexState

func (*ImplicitSearchIndex) HasUserTaggedSearchIndexes 

func (s *ImplicitSearchIndex) HasUserTaggedSearchIndexes() bool

func (*ImplicitSearchIndex) SetState 

func (s *ImplicitSearchIndex) SetState(state SearchIndexState)

func (*ImplicitSearchIndex) StoreIndexName 

func (s *ImplicitSearchIndex) StoreIndexName() string

type Index 

type Index struct {
	// Fields that are part of this index. An index can have a single or composite fields.
	Fields []*Field
	// Name is used by dictionary encoder for this index.
	Name string
	// Id is assigned to this index by the dictionary encoder.
	Id uint32
	// Secondary indexes can be in 4 states:
	// 1. UNKNOWN = Have not fetched the index metadata yet so the state is unknown
	// 2. NOT_INDEXED = field is not indexed
	// 3. INDEX_WRITE_MODE = index needs to be built in the background and cannot be used for queries
	// 4. INDEX_WRITE_MODE_BUILDING = index is being built in the background and cannot be used for queries
	// 5. INDEX_ACTIVE = index can be used for queries
	// Note: this is not used for primary key indexes
	State IndexState
	// Either a PrimaryKey index or a Secondary Key index
	IdxType IndexType
}

Index can be composite, so it has a list of fields, each index has name and encoded id. The encoded is used for key building.

func FindIndex 

func FindIndex(indexes []*Index, name string) *Index

func (*Index) IndexType 

func (i *Index) IndexType() IndexType

func (*Index) IsCompatible 

func (i *Index) IsCompatible(i1 *Index) error

func (*Index) IsSecondaryIndex 

func (i *Index) IsSecondaryIndex() bool

func (*Index) StateString 

func (i *Index) StateString() string

type IndexState 

type IndexState uint8
const (
	UNKNOWN IndexState = iota
	NOT_INDEXED
	INDEX_DELETED
	INDEX_WRITE_MODE
	INDEX_WRITE_MODE_BUILDING
	INDEX_ACTIVE
)

type IndexType 

type IndexType uint8
const (
	PRIMARY_INDEX IndexType = iota
	SECONDARY_INDEX
)

type Indexes 

type Indexes struct {
	All []*Index

	// Schema level set if the collection should index the `created_at` and `updated_at` metadata
	IndexMetadata bool
}

Indexes is to wrap different index that a collection can have.

func (*Indexes) IsActiveIndex 

func (i *Indexes) IsActiveIndex(name string) bool

type JSONSchema 

type JSONSchema struct {
	Name           string              `json:"title,omitempty"`
	Description    string              `json:"description,omitempty"`
	Properties     jsoniter.RawMessage `json:"properties,omitempty"`
	PrimaryKeys    []string            `json:"primary_key,omitempty"`
	CollectionType string              `json:"collection_type,omitempty"`
	Version        uint32              `json:"version,omitempty"`
}

type PrimaryIndexSchemaValidator 

type PrimaryIndexSchemaValidator struct{}

func (*PrimaryIndexSchemaValidator) Validate 

func (*PrimaryIndexSchemaValidator) Validate(existing *DefaultCollection, current *Factory) error

type QueryableField 

type QueryableField struct {
	FieldName      string
	InMemoryAlias  string
	UnFlattenName  string
	Indexed        bool // Secondary Index
	PrimaryIndexed bool // Primary Index
	Faceted        bool
	SearchIndexed  bool
	Sortable       bool
	DataType       FieldType
	SubType        FieldType
	SearchType     string

	DoNotFlatten  bool
	Dimensions    *int
	SearchIdField bool
	// This is not stored in flattened form in search
	// but will allow filtering on array of objects.
	// ToDo: With secondary indexes on array of objects we need to revisit this.
	AllowedNestedQFields []*QueryableField
	// contains filtered or unexported fields
}

QueryableField is internal structure used after flattening the fields i.e. the representation of the queryable field is of the following form "field" OR "parent.field". This allows us to perform look faster by just checking in this structure. Object that doesn't have any nested fields will be same as any other top level field.

func NewDynamicQueryableField 

func NewDynamicQueryableField(name string, unFlattenName string, ft FieldType) *QueryableField

NewDynamicQueryableField returns a QueryableField which may not be present in the schema. This allows querying fields that are dynamically added to the document like in case of free flow object or Array of free flow object.

func (*QueryableField) InMemoryName 

func (q *QueryableField) InMemoryName() string

InMemoryName returns key name that is used to index this field in the indexing store. For example, an "id" key is indexed with "_tigris_id" name.

func (*QueryableField) IsReserved 

func (q *QueryableField) IsReserved() bool

IsReserved returns true if the queryable field is internal field.

func (*QueryableField) KeyPath 

func (q *QueryableField) KeyPath() []string

func (*QueryableField) Name 

func (q *QueryableField) Name() string

Name returns the name of this field as defined in the schema.

func (*QueryableField) ShouldPack 

func (q *QueryableField) ShouldPack() bool

ShouldPack returns true if we need to pack this field before sending to indexing store.

func (*QueryableField) Type 

func (q *QueryableField) Type() FieldType

Type returns the data type of this field.

type QueryableFieldsBuilder 

type QueryableFieldsBuilder struct{}

func NewQueryableFieldsBuilder 

func NewQueryableFieldsBuilder() *QueryableFieldsBuilder

func (*QueryableFieldsBuilder) BuildQueryableFields 

func (builder *QueryableFieldsBuilder) BuildQueryableFields(fields []*Field, fieldsInSearch []tsApi.Field, indexMetadata bool) []*QueryableField

func (*QueryableFieldsBuilder) NewQueryableField 

func (*QueryableFieldsBuilder) NewQueryableField(name string, f *Field, fieldsInSearch []tsApi.Field) *QueryableField

type ReservedField 

type ReservedField uint8
const (
	CreatedAt ReservedField = iota
	UpdatedAt
	Metadata
	IdToSearchKey
	DateSearchKeyPrefix
	SearchArrNullItem
	SearchNullKeys
)

type SearchFactory 

type SearchFactory struct {
	// Name is the index name.
	Name string
	// Fields are derived from the user schema.
	Fields []*Field
	// Schema is the raw JSON schema received
	Schema  jsoniter.RawMessage
	Sub     string
	Source  SearchSource
	Options SearchSchemaOptions
}

SearchFactory is used as an intermediate step so that collection can be initialized with properly encoded values.

type SearchIdSchemaValidator 

type SearchIdSchemaValidator struct{}

func (*SearchIdSchemaValidator) IdField 

func (v *SearchIdSchemaValidator) IdField(current []*Field) *Field

func (*SearchIdSchemaValidator) ValidateIndex 

func (v *SearchIdSchemaValidator) ValidateIndex(existing *SearchIndex, current *SearchFactory) error

type SearchIndex 

type SearchIndex struct {
	// Name is the name of the index.
	Name string
	// index version
	Version uint32
	// Fields are derived from the user schema.
	Fields []*Field
	// JSON schema.
	Schema jsoniter.RawMessage
	// StoreSchema is the search schema of the underlying search engine.
	StoreSchema *tsApi.CollectionSchema
	// QueryableFields are similar to Fields but these are flattened forms of fields. For instance, a simple field
	// will be one to one mapped to queryable field but complex fields like object type field there may be more than
	// one queryableFields. As queryableFields represent a flattened state these can be used as-is to index in memory.
	QueryableFields []*QueryableField
	// TokenSeparators is a list of symbols or special characters to be used for splitting the text into individual
	// words in addition to space and new-line characters.
	TokenSeparators []string
	// Source of this index
	Source        SearchSource
	SearchIDField *QueryableField
	// contains filtered or unexported fields
}

SearchIndex is to manage search index created by the user.

func NewSearchIndex 

func NewSearchIndex(ver uint32, searchStoreName string, factory *SearchFactory, fieldsInSearch []tsApi.Field) *SearchIndex

func (*SearchIndex) GetField 

func (s *SearchIndex) GetField(name string) *Field

func (*SearchIndex) GetInt64FieldsPath 

func (s *SearchIndex) GetInt64FieldsPath() map[string]struct{}

func (*SearchIndex) GetQueryableField 

func (s *SearchIndex) GetQueryableField(name string) (*QueryableField, error)

func (*SearchIndex) GetSearchDeltaFields 

func (s *SearchIndex) GetSearchDeltaFields(existingFields []*QueryableField, fieldsInSearch []tsApi.Field) []tsApi.Field

func (*SearchIndex) StoreIndexName 

func (s *SearchIndex) StoreIndexName() string

func (*SearchIndex) Validate 

func (s *SearchIndex) Validate(doc map[string]any) error

type SearchIndexSourceValidator 

type SearchIndexSourceValidator struct{}

func (*SearchIndexSourceValidator) ValidateIndex 

func (*SearchIndexSourceValidator) ValidateIndex(existing *SearchIndex, current *SearchFactory) error

type SearchIndexState 

type SearchIndexState uint8

SearchIndexState represents the search state of collection search. 

const (
	UnknownSearchState SearchIndexState = iota
	SearchIndexActive
	SearchIndexWriteMode
	SearchIndexBuilding
	NoSearchIndex
)

type SearchIndexValidator 

type SearchIndexValidator interface {
	ValidateIndex(existing *SearchIndex, current *SearchFactory) error
}

SearchIndexValidator is a backward compatibility validator i.e. when an update request for a search index is received then a set of validators are run to ensure that search index is safe to upgrade from state A to B.

type SearchJSONSchema 

type SearchJSONSchema struct {
	Name        string               `json:"title,omitempty"`
	Description string               `json:"description,omitempty"`
	Properties  jsoniter.RawMessage  `json:"properties,omitempty"`
	Source      *SearchSource        `json:"source,omitempty"`
	Options     *SearchSchemaOptions `json:"options,omitempty"`
}

type SearchSchemaOptions 

type SearchSchemaOptions struct {
	TokenSeparators *[]string `json:"token_separators,omitempty"`
}

func (*SearchSchemaOptions) GetTokenSeparators 

func (s *SearchSchemaOptions) GetTokenSeparators() []string

type SearchSource 

type SearchSource struct {
	// Type is the source type i.e. either it is Tigris or the index will be maintained by the user.
	Type SearchSourceType `json:"type,omitempty"`
	// CollectionName is the source name i.e. collection name in case of Tigris otherwise it is optional.
	CollectionName string `json:"collection,omitempty"`
	// DatabaseBranch is in case the collection is part of a database branch. Only applicable if Type is Tigris.
	DatabaseBranch string `json:"branch,omitempty"`
}

type SearchSourceType 

type SearchSourceType string
const (
	// SearchSourceTigris is when the source type is Tigris for the search index.
	SearchSourceTigris SearchSourceType = "tigris"
	// SearchSourceExternal is when the source type is external for the search index.
	SearchSourceExternal SearchSourceType = "external"
)

type Validator 

type Validator interface {
	Validate(existing *DefaultCollection, current *Factory) error
}

Validator is a backward compatibility validator i.e. when an update request for a collection is received then a set of validators are run to ensure that collection is safe to upgrade from state A to B.

Note: Validators have no role during initial collection creation as there is no backward compatibility check at that point.

type Version 

type Version struct {
	Version uint32
	Schema  []byte
}

Version is schema associated with it version.

type VersionDelta 

type VersionDelta struct {
	Version uint32
	Fields  []*VersionDeltaField
}

VersionDelta contains all fields schema changes in particular schema Version.

type VersionDeltaField 

type VersionDeltaField struct {
	KeyPath []string  // Key path of the field
	From    FieldType // The type is changing from this
	To      FieldType // to this

	MaxLength int
}

VersionDeltaField describes field schema change.

If field is deleted then To is equal to `UnknownType`. If field is added then From is equal to `UnknownType`.

type Versions 

type Versions []Version

Versions is an array of all collections schemas. Sorted in ascending schema version order.

func (Versions) Latest 

func (x Versions) Latest() Version

func (Versions) Len 

func (x Versions) Len() int

func (Versions) Less 

func (x Versions) Less(i, j int) bool

func (Versions) Swap 

func (x Versions) Swap(i, j int)

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.