myjson

package module
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 9, 2023 License: Apache-2.0 Imports: 43 Imported by: 0

README

myjson GoDoc

Coverage

███    ███ ██    ██      ██ ███████  ██████  ███    ██ 
████  ████  ██  ██       ██ ██      ██    ██ ████   ██ 
██ ████ ██   ████        ██ ███████ ██    ██ ██ ██  ██ 
██  ██  ██    ██    ██   ██      ██ ██    ██ ██  ██ ██ 
██      ██    ██     █████  ███████  ██████  ██   ████

MyJSON is an embedded relational document store built on top of pluggable key value storage

go get -u github.com/autom8ter/myjson

Use Case

Build powerful applications on top of simple key value storage.

Features:

Architecture
Feature Description Implemented
Single Node (in-memory) Run the embedded database with no persistance on a single node [x]
Single Node (on-disk) Run the embedded database with persistance on a single node [x]
Distributed (distributed kv-store) Run the embedded database with horizontal scalability using tikv distributed key value storage [x]
Database
Feature Description Implemented
JSON Documents Records are stored as JSON documents [x]
Collections Records are stored in Collections which can hold any number of JSON documents [x]
Collection Schema Collections define a JSON Schema which enforces the schema of JSON documents in a collection [x]
Transactions Cross Collection transactions can be used to persist/rollback changes to the database [x]
Change Streams Built in Change-Data-Capture collection can be queried & streamed for triggering realtime events [x]
Scripting Javascript scripts can be executed with full access to database functionality [x]
Triggers Javascript triggers can be configured at the collection level to add custom business logic based on when events occur [x]
Migrations Built in support for atomic database migrations written in javascript [x]
Relationships Built in support for relationships with foreign keys - Joins and cascade deletes are also supported [x]
Secondary Indexes Multi-field secondary indexes may be used to boost query performance (eq/gt/lt/gte/lte) [x]
Unique Fields Unique fields can be configured which ensure the uniqueness of a field value in a collection [x]
Complex Queries Complex queries can be executed with support for select/where/join/having/orderby/groupby/limit/page clauses [x]
Aggregate Queries Complex aggregate queries can be executed for analytical purposes [x]
Time Travel Find the value of a document at a specific timestamp [x]
Revert Revert the value of a document to it's value at a specific timestamp [x]
Storage Providers
Provider Description Implemented
Badger persistant, embedded LSM database written in Go [x]
Tikv persistant, distributed LSM database written in Rust [x]
RocksDB persistant, embedded LSM database written in C++

Getting Started

go get -u github.com/autom8ter/myjson

Before getting started, take a look at the examples and Godoc

Collection Schemas

Collection schemas are custom JSON Schema documents that declaratively configure a collection of documents. The schema defines the structure of the JSON documents in the collection and the constraints that should be enforced on the documents. The schema is used to validate the documents in the collection and to enforce the constraints on the documents. The schema is also used to define secondary indexes, foreign keys, authorization, and triggers.

Custom Root Level Properties

MyJSON supports a number of custom root level properties that can be used to configure the schema of a collection.

x-collection

x-collection configures the unique name of the collection. The name of the collection must be unique across all collections in the database. x-collection is a required property.

x-authorization

x-authorization is a set of javascript authorization rules to enforce against actions taken against the database. x-authorization is an optional property.

x-triggers

x-triggers are a set of javascript triggers that execute when certain actions are taken against the database. x-triggers is an optional property.

x-read-only

x-readonly indicates that the collection is updated internally and may only be read. x-readonly is an optional property.

x-prevent-deletes

x-prevent-deletes indicates that documents in the collection may not be deleted. x-prevent-deletes is an optional property.

x-immutable

x-immutable indicates that documents in the collection may not be updated or deleted.

Custom Field Level Properties

MyJSON supports a number of custom field level properties that can be used to configure the schema of a collection.

x-primary

x-primary configures the documents primary key (exactly 1 field must specify x-primary). The value of the field must be unique across all documents in the collection.

x-foreign

x-foreign configures a foreign key relationship. The value of the field must be the primary key of another document in the database. Foreign keys are enforced by the database at runtime and boost query join performance.

x-unique

x-unique configures a unique field for the document. The value of the field must be unique across all documents in the collection. a unique field is enforced by the database and is not configurable. For example, if a document has a field called _secondary_id with x-unique set to true, then the value of the field must be unique across all documents in the collection. Unique indexes also boost query performance.

x-index

x-index configures a secondary index. The value of the field is indexed and can be used to boost query performance. For example, if a document has a field called _secondary_id with x-index set to true, then the value of the field will be indexed and can be used to boost query performance.

x-immutable

x-immutable indicates that the field is immutable and all edits will be replaced with it's original value. For example, if a document has a field called _secondary_id with x-immutable set to true, then the value of the field will be set to it's permanent, immutable value on creattion. If the document is updated, the value of the field will not be changed. Immutable properties override the x-compute and default properties.

x-compute

x-compute is a javascript expression that is evaluated when the document is created or updated. The expression is evaluated in the context of the document and the result is stored in the field. The expression is evaluated before the document is validated against the schema. The expression may be used to compute a value for a field based on the values of other fields in the document. Computed properties override the default property.

Example Schema

account.yaml

type: object
x-collection: account
required:
  - _id
  - name
properties:
  _id:
    type: string
    description: The account's id.
    x-primary: true
  name:
    type: string
    description: The accounts's name.
x-authorization:
  rules:
    ## allow super users to do anything
    - effect: allow
      ## match on any action
      action:
      - "*"
      ## context metadata must have role super_user set to true
      match: |
        contains(meta.Get('roles'), 'super_user')

      ## dont allow read-only users to create/update/delete/set accounts
    - effect: deny
      ## match on document mutations
      action:
        - create
        - update
        - delete
        - set
        ## context metadata must have a read_only role set to true
      match: |
        contains(meta.Get('roles'), 'read_only')

      ## only allow users to update their own account
    - effect: allow
        ## match on document mutations
      action:
        - create
        - update
        - delete
        - set
        ## the account's _id must match the user's account_id
      match: |
        doc.Get('_id') == meta.Get('account_id')

      ## only allow users to query their own account
    - effect: allow
        ## match on document queries (includes ForEach and other Query based methods)
      action:
        - query
        ## user must have a group matching the account's _id
      match: |
        query.where?.length > 0 && query.where[0].field == '_id' && query.where[0].op == 'eq' && contains(meta.Get('groups'), query.where[0].value)

user.yaml

type: object
# x-collection specifies the name of the collection the object will be stored in
x-collection: user
# required specifies the required attributes
required:
  - _id
  - name
  - age
  - contact
  - gender
  - account_id
properties:
  _id:
    type: string
    description: The user's id.
    # x-primary indicates that the property is the primary key for the object - only one primary key may be specified
    x-primary: true
  name:
    type: string
    description: The user's name.
  contact:
    type: object
    properties:
      email:
        type: string
        description: The user's email.
        `x-unique`: true
  age:
    description: Age in years which must be equal to or greater than zero.
    type: integer
    minimum: 0
  account_id:
    type: string
    # x-foreign indicates that the property is a foreign key - foreign keys are automatically indexed
    x-foreign:
      collection: account
      field: _id
      cascade: true
    # `x-index` specifies a secondary index which can have 1-many fields
    `x-index`:
      account_email_idx:
        additional_fields:
          - contact.email
  language:
    type: string
    description: The user's first language.
    `x-index`:
      language_idx: { }
  gender:
    type: string
    description: The user's gender.
    enum:
      - male
      - female
  timestamp:
    type: string
  annotations:
    type: object

# x-triggers are javascript functions that execute based on certain events
x-triggers:
  # name of the trigger
  set_timestamp:
    # order determines the order in which the functions are executed - lower ordered triggers are executed first
    order: 1
    # events configures the trigger to execute on certain events
    events:
      - on_create
      - on_update
      - on_set
    # script is the javascript to execute
    script: |
      doc.Set('timestamp', new Date().toISOString())

task.yaml

type: object
x-collection: task
required:
  - _id
  - user
  - content
properties:
  _id:
    type: string
    description: The user's id.
    x-primary: true
  user:
    type: string
    description: The id of the user who owns the task
    x-foreign:
      collection: user
      field: _id
      cascade: true
  content:
    type: string
    description: The content of the task
Authorization

TODO

Opening a database instance
Single Node in Memory (badger)
db, err := myjson.Open(context.Background(), "badger", map[string]any{
	"storage_path": "",
})
Single Node w/ Persistance (badger)
db, err := myjson.Open(context.Background(), "badger", map[string]any{
	"storage_path": "./tmp",
})
Multi Node w/ Persistance (tikv)
db, err := myjson.Open(context.Background(), "tikv", map[string]any{
    "pd_addr":    []string{"http://pd0:2379"},
    "redis_addr": "localhost:6379",
    "redis_user": "admin", //change me
    "redis_password": "123232", //change me
})
Configuring a database instance

Collection schemas can be configured at runtime or at startup. Collection schemas are declarative - any changes to indexing or validation happen within the database when Configure is called


var (
    //go:embed account.yaml
    accountSchema string
    //go:embed user.yaml
    userSchema string
    //go:embed task.yaml
    taskSchema string
)

if err := db.Configure(ctx, "", []string{accountSchema, userSchema, taskSchema}); err != nil {
    panic(err)
})
Working with JSON documents
Creating a JSON document
document, err := myjson.NewDocumentFrom(map[string]any{
    "name": "acme.com",
})

doc := myjson.NewDocument()
doc.Set("name", "acme.com")
Setting JSON values
doc := myjson.NewDocument()
doc.Set("name", "acme.com")

SJSON syntax is supported: https://github.com/tidwall/sjson#path-syntax

doc := myjson.NewDocument()
doc.Set("contact.email", "info@acme.com")
Getting JSON values
doc := myjson.NewDocument()
doc.Set("name", "acme.com")

GJSON syntax is supported: https://github.com/tidwall/sjson#path-syntax

value := doc.Get("contact.email")

additional GJSON modifiers are available:

  • @camelCase - convert a json string field to camel case doc.Get("project|@camelCase")
  • @snakeCase - convert a json string field to snake case doc.Get("project|@snakeCase")
  • @kebabCase - convert a json string field to kebab case doc.Get("project|@kebabCase")
  • @replaceAll - replace a substring within a json string field with another string
  • @unix - get the unix timestamp of the json time field doc.Get("timestamp|@unix")
  • @unixMilli - get the unix millisecond timestamp of the json time field doc.Get("timestamp|@unixMilli")
  • @unixNano - get the unix nanosecond timestamp of the json time field doc.Get("timestamp|@unixNano")
  • @dateTrunc - truncate a date to day, month, or year ex: doc.GetString("timestamp|@dateTrunc:month"), doc.GetString("timestamp|@dateTrunc:year"), doc.GetString("timestamp|@dateTrunc:day")
Transactions

Most database functionality is made available via the Tx interface which has read/write methods across 1-many collections.

Writable
if err := db.Tx(ctx, kv.TxOpts{IsReadOnly: false}, func(ctx context.Context, tx myjson.Tx) error {
	// do stuff ...tx.Set(...)
	// return error to rollback
	// return no error to commit
}
Read Only
if err := db.Tx(ctx, kv.TxOpts{IsReadOnly: true}, func(ctx context.Context, tx myjson.Tx) error {
	// ...tx.Get(...)
}
Adding documents to a collection
if err := db.Tx(ctx, kv.TxOpts{}, func(ctx context.Context, tx myjson.Tx) error {
    doc := myjson.NewDocument()
    doc.Set("name", "acme.com")
	id, err := tx.Create(ctx, "account", document)
	if err != nil {
		return err
    }
}
Queries
results, err := tx.Query(ctx, "user", myjson.Q().
    Select(myjson.Select{Field: "*"}).
	OrderBy(myjson.OrderBy{Field: "age", Direction: myjson.OrderByDirectionDesc}).
    Query())
Joins

1-many joins are

results, err := db.Query(ctx, "user", myjson.Q().
    Select(
        myjson.Select{Field: "acc._id", As: "account_id"},
        myjson.Select{Field: "acc.name", As: "account_name"},
		myjson.Select{Field: "_id", As: "user_id"},
    ).
    Join(myjson.Join{
        Collection: "account",
        On: []myjson.Where{
            {
				Field: "_id",
				Op:    myjson.WhereOpEq,
                Value: "$account_id", //self reference the account_id on the user document
            },
    },
        As: "acc",
    }).
Query())
Iterating through documents in a collection
_, err := tx.ForEach(ctx, "user", myjson.ForEachOpts{}, func(d *myjson.Document) (bool, error) {
    fmt.Println(d)
    return true, nil
})
Reading documents in a collection
doc, err := tx.Get(ctx, "user", "$id")
Change Streams
Stream Changes in a given collection

CDC persistance must be enabled for change streams to work. See the database Options for more info.

ctx, cancel := context.WithCancel(context.Background())
defer cancel()
err := db.ChangeStream(ctx, "user", func(cdc myjson.CDC) (bool, error) {
    fmt.Println(cdc)
    return true, nil
})
Aggregation
query := myjson.Query{
	Select: []myjson.Select{
		{
            Field: "account_id",
		},
		{
            Field:     "age",
            Aggregate: myjson.AggregateFunctionSum,
            As:        "age_sum",
		},
	},
	GroupBy: []string{"account_id"},
}
results, err := db.Query(ctx, "user", query)
Triggers

add a triggers block to your JSON schema ex: update timestamp on set/update/create with a javascript expression

triggers:
  set_timestamp:
    order: 1
    events:
      - on_create
      - on_update
      - on_set
    script: |
      doc.Set('timestamp', new Date().toISOString())

javascript variables are injected at runtime:

  • doc - the JSON document that is being changed
  • db - the global database instance(all methods are available)
  • ctx - the context when the trigger was called
  • metadata - the context metadata when the script is called
  • tx - the current transaction instance
Scripts

Scripts are javascript expressions/functions that can be called in Go - this may be used when embedded/dynamic functionality is required.

getAccountScript:

function setAccount(ctx, db, params) {
    db.tx(ctx, {isReadOnly: false}, (ctx, tx) => {
        tx.set(ctx, "account", params.doc)
    })
}

execute with custom paramaters:

id := ksuid.New().String()
doc, err := myjson.NewDocumentFrom(map[string]any{
    "_id":  id,
    "name": gofakeit.Company(),
})
_, err = db.RunScript(ctx, "setAccount", getAccountScript, map[string]any{
    "doc": doc,
})

javascript variables are injected at runtime:

  • db - the global database instance(all methods are available lowercased)
  • ctx - the context when the script is called
  • metadata - the context metadata when the script is called
  • newDocument - function to intialize a new JSON document
  • newDocumentFrom - function to initialize a new JSON document from a javascript object

Tikv Setup Guide (full scale)

WIP - see tikv foldeer w/ Makefile for running tikv locally

Contributing

Install Dependencies

go mod download

Checkout Branch

git checkout -b ${issueNumber}

Run Tests

go test -race -covermode=atomic -coverprofile=coverage.out -v .

Run Benchmarks

go test -bench=. -benchmem -run=^#

Lint Repository

golangci-lint run

Check Coverage

go tool cover -func=coverage.out

Documentation

Index

Examples

Constants

This section is empty.

Variables

View Source 
var (
	// MetadataKeyNamespace is the key for the database namespace - it will return as "default" if not set
	MetadataKeyNamespace = "namespace"
	// MetadataKeyUserID is the key for the user id	for use in x-authorizers (optional)
	MetadataKeyUserID = "userId"
	// MetadataKeyRoles is the key for the user roles([]string)	for use in x-authorizers (optional)
	MetadataKeyRoles = "roles"
	// MetadataKeyGroups is the key for the user groups([]string) for use in x-authorizers (optional)
	MetadataKeyGroups = "groups"
)
View Source 
var JavascriptBuiltIns = map[string]any{

	"newDocumentFrom": NewDocumentFrom,

	"newDocument": NewDocument,

	"ksuid": func() string {
		return ksuid.New().String()
	},

	"uuid": func() string {
		return uuid.New().String()
	},

	"now": time.Now,

	"chunk": funk.Chunk,

	"map": funk.Map,

	"filter": funk.Filter,

	"find": funk.Find,

	"contains": func(v any, e any) bool {
		if v == nil {
			return false
		}
		switch v := v.(type) {
		case string:
			return strings.Contains(v, cast.ToString(e))
		default:
			return funk.Contains(v, e)
		}
	},

	"equal": funk.Equal,

	"deepEqual": reflect.DeepEqual,

	"keys": funk.Keys,

	"values": funk.Values,

	"flatten": funk.Flatten,

	"uniq": funk.Uniq,

	"drop": funk.Drop,

	"last": funk.Last,

	"empty": funk.IsEmpty,

	"notEmpty": funk.NotEmpty,

	"difference": funk.Difference,

	"isZero": funk.IsZero,

	"sum": funk.Sum,

	"set": funk.Set,

	"getOr": funk.GetOrElse,

	"prune": funk.Prune,

	"len": func(v any) int { return len(cast.ToSlice(v)) },

	"toSlice": cast.ToSlice,

	"toMap": cast.ToStringMap,

	"toStr": cast.ToString,

	"toInt": cast.ToInt,

	"toFloat": cast.ToFloat64,

	"toBool": cast.ToBool,

	"toTime": cast.ToTime,

	"toDuration": cast.ToDuration,

	"asDoc": func(v any) *Document {
		d, _ := NewDocumentFrom(v)
		return d
	},

	"indexOf": funk.IndexOf,

	"join": strings.Join,

	"split": strings.Split,

	"replace": strings.ReplaceAll,

	"lower": strings.ToLower,

	"upper": strings.ToUpper,

	"trim": strings.TrimSpace,

	"trimLeft": strings.TrimLeft,

	"trimRight": strings.TrimRight,

	"trimPrefix": strings.TrimPrefix,

	"trimSuffix": strings.TrimSuffix,

	"startsWith": strings.HasPrefix,

	"endsWith": strings.HasSuffix,

	"camelCase": xstrings.ToCamelCase,

	"snakeCase": xstrings.ToSnakeCase,

	"kebabCase": xstrings.ToKebabCase,

	"quote": strconv.Quote,

	"unquote": strconv.Unquote,

	"parseTime": time.Parse,

	"since": time.Since,

	"until": time.Until,

	"after": time.After,

	"unixMicro": time.UnixMicro,

	"unixMilli": time.UnixMilli,

	"unix": time.Unix,

	"date": time.Date,

	"sha1": func(v any) string {
		h := sha1.New()
		h.Write([]byte(cast.ToString(v)))
		return hex.EncodeToString(h.Sum(nil))
	},

	"sha256": func(v any) string {
		h := sha256.New()
		h.Write([]byte(cast.ToString(v)))
		return hex.EncodeToString(h.Sum(nil))
	},

	"sha512": func(v any) string {
		h := sha512.New()
		h.Write([]byte(cast.ToString(v)))
		return hex.EncodeToString(h.Sum(nil))
	},

	"md5": func(v any) string {
		h := md5.New()
		h.Write([]byte(cast.ToString(v)))
		return hex.EncodeToString(h.Sum(nil))
	},

	"base64Encode": func(v any) string {
		return base64.StdEncoding.EncodeToString([]byte(cast.ToString(v)))
	},

	"base64Decode": func(v any) (string, error) {
		b, err := base64.StdEncoding.DecodeString(cast.ToString(v))
		return string(b), err
	},

	"jsonEncode": func(v any) (string, error) {
		b, err := json.Marshal(v)
		return string(b), err
	},

	"jsonDecode": func(v string) (any, error) {
		var out any
		err := json.Unmarshal([]byte(v), &out)
		return out, err
	},

	"fetch": func(request map[string]any) (map[string]any, error) {
		method := cast.ToString(request["method"])
		if method == "" {
			return nil, fmt.Errorf("missing 'method'")
		}
		url := cast.ToString(request["url"])
		if url == "" {
			return nil, fmt.Errorf("missing 'url'")
		}
		req, err := http.NewRequest(method, url, nil)
		if err != nil {
			return nil, err
		}
		if headers, ok := request["headers"]; ok {
			for k, v := range cast.ToStringMap(headers) {
				req.Header.Set(k, cast.ToString(v))
			}
		}
		if queryParams, ok := request["query"]; ok {
			q := req.URL.Query()
			for k, v := range cast.ToStringMap(queryParams) {
				q.Set(k, cast.ToString(v))
			}
			req.URL.RawQuery = q.Encode()
		}
		if body, ok := request["body"]; ok {
			req.Body = ioutil.NopCloser(strings.NewReader(cast.ToString(body)))
		}

		resp, err := http.DefaultClient.Do(req)
		if err != nil {
			return nil, err
		}
		defer resp.Body.Close()
		b, err := ioutil.ReadAll(resp.Body)
		if err != nil {
			return nil, err
		}
		return map[string]any{
			"status":  resp.StatusCode,
			"headers": resp.Header,
			"body":    string(b),
		}, nil
	},
}

Functions

func GetMetadataValue added in v0.2.0 

func GetMetadataValue(ctx context.Context, key string) any

GetMetadataValue gets a metadata value from the context if it exists

func SetIsInternal added in v0.2.1 

func SetIsInternal(ctx context.Context) context.Context

SetIsInternal sets a context value to indicate that the request is internal (it should only be used to bypass things like authorization, validation, etc)

func SetMetadataGroups added in v0.3.0 

func SetMetadataGroups(ctx context.Context, groups []string) context.Context

SetMetadataGroups sets the metadata user groups for targeting in the collections x-authorizers

Example 
package main

import (
	"context"
	"fmt"

	"github.com/autom8ter/myjson"
)

func main() {
	ctx := context.Background()
	ctx = myjson.SetMetadataGroups(ctx, []string{"group1", "group2"})
	fmt.Println(myjson.ExtractMetadata(ctx).GetArray("groups"))
}

Output:

func SetMetadataNamespace added in v0.3.0 

func SetMetadataNamespace(ctx context.Context, namespace string) context.Context

SetMetadataNamespace sets the metadata namespace

func SetMetadataRoles added in v0.3.0 

func SetMetadataRoles(ctx context.Context, roles []string) context.Context

SetMetadataRoles sets the metadata user roles for targeting in the collections x-authorizers

Example 
package main

import (
	"context"
	"fmt"

	"github.com/autom8ter/myjson"
)

func main() {
	ctx := context.Background()
	ctx = myjson.SetMetadataRoles(ctx, []string{"super_user"})
	fmt.Println(myjson.ExtractMetadata(ctx).GetArray("roles"))
}

Output:

func SetMetadataUserID added in v0.3.0 

func SetMetadataUserID(ctx context.Context, userID string) context.Context

SetMetadataUserID sets the metadata userID for targeting in the collections x-authorizers

func SetMetadataValues added in v0.2.0 

func SetMetadataValues(ctx context.Context, data map[string]any) context.Context

SetMetadataValues sets metadata key value pairs in the context

Types

type Action 

type Action string

Action is an action that causes a mutation to the database 

const (
	// CreateAction creates a document
	CreateAction Action = "create"
	// SetAction sets a document's values in place
	SetAction Action = "set"
	// UpdateAction updates a set of fields on a document
	UpdateAction Action = "update"
	// DeleteAction deletes a document
	DeleteAction Action = "delete"
	// QueryAction queries documents
	QueryAction Action = "query"
	// ConfigureAction configures a collection of documents
	ConfigureAction Action = "configure"
	// ChangeStreamAction creates a change stream
	ChangeStreamAction Action = "changeStream"
)

type AggregateFunction 

type AggregateFunction string

AggregateFunction is an agggregate function applied to a list of documents 

const AggregateFunctionCount AggregateFunction = "count"

AggregateFunctionCount gets the count of a set of documents 

const AggregateFunctionMax AggregateFunction = "max"

AggregateFunctionMax gets the max value in a set of documents 

const AggregateFunctionMin AggregateFunction = "min"

AggregateFunctionMin gets the min value in a set of documents 

const AggregateFunctionSum AggregateFunction = "sum"

AggregateFunctionSum gets the sum of values in a set of documents

type Authz added in v0.2.0 

type Authz struct {
	Rules []AuthzRule `json:"rules" validate:"min=1,required"`
}

Authz is a serializable authz object which represents the x-authorization section of a collection schema It is used to define the authorization rules for a collection When any rule matches that has the effect "deny" the request is denied When no rules match the request is denied When any rule matches that has the effect "allow" the request is allowed (as long as no deny rules match)

type AuthzEffect added in v0.2.0 

type AuthzEffect string

AuthzEffect is an effect of an authz rule 

const (
	// Allow is the allow effect
	Allow AuthzEffect = "allow"
	// Deny is the deny effect
	Deny AuthzEffect = "deny"
)

type AuthzRule added in v0.2.0 

type AuthzRule struct {
	// Effect is the effect of the rule - allow or deny
	Effect AuthzEffect `json:"effect" validate:"required"`
	// Action is the action to apply the rule to - create, read, update, delete, query, configure, changeStream
	Action []Action `json:"action" validate:"min=1,required"`
	// Match is a javscript boolean expression to match the rule against
	Match string `json:"match" validate:"required"`
}

AuthzRule

type CDC 

type CDC struct {
	// ID is the unique id of the cdc
	ID string `json:"_id" validate:"required"`
	// Collection is the collection the change was applied to
	Collection string `json:"collection" validate:"required"`
	// Action is the action applied to the document
	Action Action `json:"action" validate:"required,oneof='create' 'update' 'delete' 'set'"`
	// DocumentID is the ID of the document that was changed
	DocumentID string `json:"documentID" validate:"required"`
	// Diff is the difference between the previous and new version of the document
	Diff []JSONFieldOp `json:"diff,omitempty"`
	// Timestamp is the nanosecond timestamp the cdc was created at
	Timestamp int64 `json:"timestamp" validate:"required"`
	// Metadata is the context metadata when the change was made
	Metadata *Document `json:"metadata" validate:"required"`
}

CDC is a change data capture object used for tracking changes to documents over time

type ChangeStreamHandler added in v0.3.0 

type ChangeStreamHandler func(ctx context.Context, cdc CDC) (bool, error)

ChangeStreamHandler handles changes to documents which are emitted as a change data capture stream

type CollectionConfiguration added in v0.3.0 

type CollectionConfiguration map[string]string

CollectionConfiguration is a map of collection names to collection schemas - it declarative represents the database collection configuration

type CollectionPlan added in v0.4.0 

type CollectionPlan struct {
	Collection string        `json:"collection,omitempty" validate:"required"`
	Diff       []JSONFieldOp `json:"diff,omitempty"`
}

type CollectionSchema 

type CollectionSchema interface {
	// Collection is the collection name
	Collection() string
	// ValidateDocument validates the input document against the collection's JSON schema
	ValidateDocument(ctx context.Context, doc *Document) error
	// Indexing returns a copy the schemas indexing
	Indexing() map[string]Index
	// PrimaryIndex returns the collection's primary index
	PrimaryIndex() Index
	// PrimaryKey returns the collection's primary key
	PrimaryKey() string
	// GetPrimaryKey gets the document's primary key
	GetPrimaryKey(doc *Document) string
	// SetPrimaryKey sets the document's primary key
	SetPrimaryKey(doc *Document, id string) error
	// RequireQueryIndex returns whether the collection requires that queries are appropriately indexed
	RequireQueryIndex() bool
	// Properties returns a map of the schema's properties
	Properties() map[string]SchemaProperty
	// PropertyPaths returns a flattened map of the schema's properties - nested properties will be keyed in dot notation
	PropertyPaths() map[string]SchemaProperty
	// Triggers returns a map of triggers keyed by name that are assigned to the collection
	Triggers() []Trigger
	// IsReadOnly returns whether the collection is read only
	IsReadOnly() bool
	// Authz returns the collection's authz if it exists
	Authz() Authz
	// Immutable returns whether the collection is immutable
	Immutable() bool
	// PreventDeletes returns whether the collection prevents deletes
	PreventDeletes() bool
	// Equals returns whether the given collection schema is equal to the current schema
	Equals(schema CollectionSchema) bool
	// MarshalYAML returns the collection schema as yaml bytes
	MarshalYAML() ([]byte, error)
	// UnmarshalYAML refreshes the collection schema with the given json bytes
	UnmarshalYAML(bytes []byte) error
	json.Marshaler
	json.Unmarshaler
}

CollectionSchema is a database collection configuration

type CommitCmd added in v0.2.0 

type CommitCmd struct{}

CommitCmd is a serializable commit command

type ComputedField added in v0.3.8 

type ComputedField struct {
	// Read indicates if the field is computed on read
	Read bool `json:"read"`
	// Write indicates if the field is computed on write
	Write bool `json:"write"`
	// Expr is the javascript expression to compute the value of the field at runtime
	Expr string `json:"expr" validate:"required"`
}

ComputedField is a computed field

type ConfigurationPlan added in v0.4.0 

type ConfigurationPlan struct {
	PreviousHash string            `json:"previousHash,omitempty"`
	ToDelete     []*CollectionPlan `json:"toDelete,omitempty"`
	ToCreate     []*CollectionPlan `json:"toCreate,omitempty"`
	ToReplace    []*CollectionPlan `json:"toReplace,omitempty"`
}

func (*ConfigurationPlan) String added in v0.4.0 

func (c *ConfigurationPlan) String() string

type CreateCmd added in v0.1.4 

type CreateCmd struct {
	// Collection is the collection the document belongs to
	Collection string `json:"collection" validate:"required"`
	// Document is the document to set
	Document *Document `json:"document" validate:"required"`
}

CreateCmd is a serializable create command

type DBOpt 

type DBOpt func(d *defaultDB)

DBOpt is an option for configuring a collection

func WithGlobalJavascriptFunctions added in v0.3.6 

func WithGlobalJavascriptFunctions(scripts []string) DBOpt

WithGlobalJavasciptFunctions adds global javascript functions to the embedded javascript vm(s)

func WithJavascriptOverrides 

func WithJavascriptOverrides(overrides map[string]any) DBOpt

WithJavascriptOverrides adds global variables or methods to the embedded javascript vm(s)

func WithOptimizer 

func WithOptimizer(o Optimizer) DBOpt

WithOptimizer overrides the default query optimizer provider

type Database 

type Database interface {
	// Collections returns a list of collection names that are registered in the database
	Collections(ctx context.Context) []string
	// GetSchema gets a collection schema by name (if it exists)
	GetSchema(ctx context.Context, collection string) CollectionSchema
	// HasCollection reports whether a collection exists in the database
	HasCollection(ctx context.Context, collection string) bool
	// Plan returns a configuration plan for the given values yaml(optional for templating) and collectionSchemas yaml schemas.
	// The plan will contain the changes that need to be made to the database in order to match the given configuration.
	Plan(ctx context.Context, valuesYaml string, collectionSchemas []string) (*ConfigurationPlan, error)
	// ConfigurePlan applies the given configuration plan to the database
	// This method is useful for applying a plan that was generated by the Plan method
	// If the plan is empty, this method will do nothing
	ConfigurePlan(ctx context.Context, plan ConfigurationPlan) error
	// Configure sets the database collection configurations. It will create/update/delete the necessary collections and indexes to
	// match the given configuration. Each element in the config should be a YAML string representing a CollectionSchema.
	Configure(ctx context.Context, valuesYaml string, yamlSchemas []string) error
	// Tx executes the given function against a new transaction.
	// if the function returns an error, all changes will be rolled back.
	// otherwise, the changes will be commited to the database
	Tx(ctx context.Context, opts kv.TxOpts, fn TxFunc) error
	// NewTx returns a new transaction. a transaction must call Commit method in order to persist changes
	NewTx(opts kv.TxOpts) (Txn, error)
	// ChangeStream streams changes to documents in the given collection. CDC Persistence must be enabled to use this method.
	ChangeStream(ctx context.Context, collection string, filter []Where, fn ChangeStreamHandler) error
	// Get gets a single document by id
	Get(ctx context.Context, collection, id string) (*Document, error)
	// ForEach scans the optimal index for a collection's documents passing its filters.
	// results will not be ordered unless an index supporting the order by(s) was found by the optimizer
	// Query should be used when order is more important than performance/resource-usage
	ForEach(ctx context.Context, collection string, opts ForEachOpts, fn ForEachFunc) (Explain, error)
	// Query queries a list of documents
	Query(ctx context.Context, collection string, query Query) (Page, error)
	// RunScript executes a javascript function within the script
	// The following global variables will be injected:
	// 'db' - a database instance,
	// 'ctx' - the context passed to RunScript,
	// and 'params' - the params passed to RunScript
	RunScript(ctx context.Context, script string, params map[string]any) (any, error)
	// RawKV returns the database key value provider - it should be used with caution and only when standard database functionality is insufficient.
	RawKV() kv.DB
	// Serve serves the database over the given transport
	Serve(ctx context.Context, t Transport) error
	// NewDoc creates a new document builder instance
	NewDoc() *DocBuilder
	// Close closes the database
	Close(ctx context.Context) error
}

Database is a NoSQL database built on top of key value storage

func Open 

func Open(ctx context.Context, provider string, providerParams map[string]any, opts ...DBOpt) (Database, error)

Open opens a new database instance from the given config

Example 
package main

import (
	"context"

	"github.com/autom8ter/myjson"
	"github.com/autom8ter/myjson/kv"
)

func main() {
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()
	db, err := myjson.Open(ctx, "badger", map[string]any{
		// leave empty for in-memory
		"storage_path": "",
	})
	if err != nil {
		panic(err)
	}
	defer db.Close(ctx)
	var accountSchema = `
type: object
# collection name
x-collection: account
required:
  - _id
  - name
properties:
  _id:
    type: string
    description: The account's id.
    x-primary: true
  name:
    type: string
    description: The accounts's name.
x-authorization:
  rules:
    ## allow super users to do anything
    - effect: allow
      ## match on any action
      action:
      - "*"
      ## context metadata must have is_super_user set to true
      match: |
        contains(meta.Get('roles'), 'super_user')

      ## dont allow read-only users to create/update/delete/set accounts
    - effect: deny
      ## match on document mutations
      action:
        - create
        - update
        - delete
        - set
        ## context metadata must have is_read_only set to true
      match: |
        contains(meta.Get('roles'), 'read_only')

      ## only allow users to update their own account
    - effect: allow
        ## match on document mutations
      action:
        - create
        - update
        - delete
        - set
        ## the account's _id must match the user's account_id
      match: |
        doc.Get('_id') == meta.Get('account_id')

      ## only allow users to query their own account
    - effect: allow
        ## match on document queries (includes ForEach and other Query based methods)
      action:
        - query
        ## user must have a group matching the account's _id
      match: |
        query.where?.length > 0 && query.where[0].field == '_id' && query.where[0].op == 'eq' && contains(meta.Get('groups'), query.where[0].value) 

`
	if err := db.Configure(ctx, "", []string{accountSchema}); err != nil {
		panic(err)
	}
	if err := db.Tx(ctx, kv.TxOpts{IsReadOnly: false}, func(ctx context.Context, tx myjson.Tx) error {
		// create a new account document
		account, err := myjson.NewDocumentFrom(map[string]any{
			"name": "acme.com",
		})
		if err != nil {
			return err
		}
		// create the account
		_, err = tx.Create(ctx, "account", account)
		if err != nil {
			return err
		}
		return nil
	}); err != nil {
		panic(err)
	}
}

Output:

type DeleteCmd added in v0.1.4 

type DeleteCmd struct {
	// Collection is the collection the document belongs to
	Collection string `json:"collection" validate:"required"`
	// ID is the unique id of the document
	ID string `json:"id" validate:"required"`
}

DeleteCmd is a serializable delete command

type DocBuilder added in v0.1.0 

type DocBuilder struct {
	// contains filtered or unexported fields
}

DocBuilder is a builder for creating a document

func D added in v0.1.0 

func D() *DocBuilder

D creates a new document builder

Example 
package main

import (
	"fmt"

	"github.com/autom8ter/myjson"
)

func main() {
	doc := myjson.D().Set(map[string]any{
		"name":  "John Doe",
		"email": "johndoe@gmail.com",
	}).Doc()
	fmt.Println(doc.String())
}

Output:

func (*DocBuilder) Doc added in v0.1.0 

func (b *DocBuilder) Doc() *Document

Doc returns the document

func (*DocBuilder) Err added in v0.1.0 

func (b *DocBuilder) Err() error

Err returns an error if one exists

func (*DocBuilder) From added in v0.1.0 

func (b *DocBuilder) From(d *Document) *DocBuilder

From creates a new document builder from the input document

func (*DocBuilder) Set added in v0.1.0 

func (b *DocBuilder) Set(values map[string]any) *DocBuilder

DocBuilder sets the key value pairs on the document

type Document 

type Document struct {
	// contains filtered or unexported fields
}

Document is a concurrency safe JSON document

func ExtractMetadata added in v0.2.0 

func ExtractMetadata(ctx context.Context) *Document

ExtractMetadata extracts metadata from the context and returns it

Example 
package main

import (
	"context"
	"fmt"

	"github.com/autom8ter/myjson"
)

func main() {
	ctx := context.Background()
	meta := myjson.ExtractMetadata(ctx)
	fmt.Println(meta.String())
}

Output:

{"namespace":"default"}

func NewDocument 

func NewDocument() *Document

NewDocument creates a new json document

func NewDocumentFrom 

func NewDocumentFrom(value any) (*Document, error)

NewDocumentFrom creates a new document from the given value - the value must be json compatible

Example 
package main

import (
	"fmt"

	"github.com/autom8ter/myjson"
)

func main() {
	doc, _ := myjson.NewDocumentFrom(map[string]any{
		"name": "autom8ter",
	})
	fmt.Println(doc.String())
}

Output:

{"name":"autom8ter"}

func NewDocumentFromBytes 

func NewDocumentFromBytes(json []byte) (*Document, error)

NewDocumentFromBytes creates a new document from the given json bytes

func (*Document) ApplyOps added in v0.3.2 

func (d *Document) ApplyOps(ops []JSONFieldOp) error

ApplyOps applies the given JSON field operations to the document

func (*Document) Bytes 

func (d *Document) Bytes() []byte

Bytes returns the document as json bytes

func (*Document) Clone 

func (d *Document) Clone() *Document

Clone allocates a new document with identical values

func (*Document) Del 

func (d *Document) Del(field string) error

Del deletes a field from the document. SJSON systax is supported For information on sjson syntax, check out https://github.com/tidwall/sjson#path-syntax

func (*Document) DelAll 

func (d *Document) DelAll(fields ...string) error

DelAll deletes a field from the document. SJSON systax is supported For information on sjson syntax, check out https://github.com/tidwall/sjson#path-syntax

func (*Document) Diff 

func (d *Document) Diff(before *Document) []JSONFieldOp

Diff calculates a json diff between the document and the input document

func (*Document) Encode 

func (d *Document) Encode(w io.Writer) error

Encode encodes the json document to the io writer

func (*Document) Exists 

func (d *Document) Exists(field string) bool

Exists returns true if the fieldPath has a value in the json document. Exists has GJSON syntax support // For information on gjson syntax, check out https://github.com/tidwall/gjson/blob/master/SYNTAX.md

func (*Document) FieldPaths 

func (d *Document) FieldPaths() []string

FieldPaths returns the paths to fields & nested fields in dot notation format

func (*Document) Get 

func (d *Document) Get(field string) any

Get gets a field on the document. Get has GJSON syntax support For information on gjson syntax, check out https://github.com/tidwall/gjson/blob/master/SYNTAX.md

Example 
package main

import (
	"fmt"

	"github.com/autom8ter/myjson"
)

func main() {

	doc := myjson.NewDocument()
	doc.Set("name", "autom8ter")
	doc.Set("contact.email", "coleman@autom8ter.com")

	fmt.Println(doc.String())
}

Output:

{"name":"autom8ter","contact":{"email":"coleman@autom8ter.com"}}

func (*Document) GetArray 

func (d *Document) GetArray(field string) []any

GetArray gets an array field on the document. Get has GJSON syntax support For information on gjson syntax, check out https://github.com/tidwall/gjson/blob/master/SYNTAX.md

func (*Document) GetBool 

func (d *Document) GetBool(field string) bool

GetBool gets a bool field value on the document. GetBool has GJSON syntax support For information on gjson syntax, check out https://github.com/tidwall/gjson/blob/master/SYNTAX.md

func (*Document) GetFloat 

func (d *Document) GetFloat(field string) float64

GetFloat gets a float64 field value on the document. GetFloat has GJSON syntax support For information on gjson syntax, check out https://github.com/tidwall/gjson/blob/master/SYNTAX.md

func (*Document) GetInt added in v0.3.8 

func (d *Document) GetInt(field string) int

GetInt gets an int field value on the document. GetInt has GJSON syntax support For information on gjson syntax, check out https://github.com/tidwall/gjson/blob/master/SYNTAX.md

func (*Document) GetString 

func (d *Document) GetString(field string) string

GetString gets a string field value on the document. GetString has GJSON syntax support For information on gjson syntax, check out https://github.com/tidwall/gjson/blob/master/SYNTAX.md

func (*Document) GetTime 

func (d *Document) GetTime(field string) time.Time

GetTime gets a time.Time field value on the document. GetTime has GJSON syntax support For information on gjson syntax, check out https://github.com/tidwall/gjson/blob/master/SYNTAX.md

func (*Document) MarshalJSON 

func (d *Document) MarshalJSON() ([]byte, error)

MarshalJSON satisfies the json Marshaler interface

func (*Document) Merge 

func (d *Document) Merge(with *Document) error

Merge merges the document with the provided document.

func (*Document) MergeJoin 

func (d *Document) MergeJoin(with *Document, alias string) error

MergeJoin merges the document with the input document with each key from the input document prefixed with the given alias

func (*Document) Overwrite 

func (d *Document) Overwrite(values map[string]any) error

Overwrite resets the document with the given values. SJSON systax is supported For information on sjson syntax, check out https://github.com/tidwall/sjson#path-syntax

func (*Document) RevertOps added in v0.3.2 

func (d *Document) RevertOps(diff []JSONFieldOp) error

RevertOps reverts the given JSON field operations to the document

func (*Document) Scan 

func (d *Document) Scan(value any) error

Scan scans the json document into the value

Example 
package main

import (
	"fmt"

	"github.com/autom8ter/myjson"
)

func main() {
	type User struct {
		Name string `json:"name"`
	}
	doc, _ := myjson.NewDocumentFrom(map[string]any{
		"name": "autom8ter",
	})

	var usr User

	doc.Scan(&usr)
	fmt.Println(usr.Name)
}

Output:

autom8ter

func (*Document) Set 

func (d *Document) Set(field string, val any) error

Set sets a field on the document. Set has SJSON syntax support (dot notation) For information on sjson syntax, check out https://github.com/tidwall/sjson#path-syntax

Example 
package main

import (
	"fmt"

	"github.com/autom8ter/myjson"
)

func main() {

	doc := myjson.NewDocument()
	doc.Set("name", "autom8ter")
	doc.Set("contact.email", "coleman@autom8ter.com")

	fmt.Println(doc.String())
}

Output:

{"name":"autom8ter","contact":{"email":"coleman@autom8ter.com"}}

func (*Document) SetAll 

func (d *Document) SetAll(values map[string]any) error

SetAll sets all fields on the document. SJSON systax is supported For information on sjson syntax, check out https://github.com/tidwall/sjson#path-syntax

func (*Document) String 

func (d *Document) String() string

String returns the document as a json string

func (*Document) UnmarshalJSON 

func (d *Document) UnmarshalJSON(bytes []byte) error

UnmarshalJSON satisfies the json Unmarshaler interface

func (*Document) Valid 

func (d *Document) Valid() bool

Valid returns whether the document is valid

func (*Document) Value 

func (d *Document) Value() map[string]any

Value returns the document as a map

func (*Document) Where 

func (d *Document) Where(wheres []Where) (bool, error)

Where executes the where clauses against the document and returns true if it passes the clauses. If the value of a where clause is prefixed with $. it will compare where.field to the same document's $.{field}.

type Documents 

type Documents []*Document

Documents is an array of documents

func (Documents) Filter 

func (documents Documents) Filter(predicate func(document *Document, i int) bool) Documents

Filter applies the filter function against the documents

func (Documents) ForEach 

func (documents Documents) ForEach(fn func(next *Document, i int))

ForEach applies the function to each document in the documents

func (Documents) Map 

func (documents Documents) Map(mapper func(t *Document, i int) *Document) Documents

Map applies the mapper function against the documents

func (Documents) Slice 

func (documents Documents) Slice(start, end int) Documents

Slice slices the documents into a subarray of documents

type EventType 

type EventType string
const (
	OnSet    EventType = "onSet"
	OnDelete EventType = "onDelete"
	OnUpdate EventType = "onUpdate"
	OnCreate EventType = "onCreate"
)

type Explain added in v0.1.0 

type Explain struct {
	// Collection
	Collection string `json:"collection"`
	// Index is the index the query optimizer chose
	Index Index `json:"index"`
	// MatchedFields are the fields that matched the index
	MatchedFields []string `json:"matchedFields"`
	// MatchedValues are the values that were matched to the index
	MatchedValues map[string]any `json:"matchedValues,omitempty"`
	// SeekFields indicates that the given fields will be seeked
	SeekFields []string `json:"seek,omitempty"`
	// SeekValues are the values to seek
	SeekValues map[string]any `json:"seekValues,omitempty"`
	// Reverse indicates that the index should be scanned in reverse
	Reverse bool `json:"reverse,omitempty"`
}

Explain is the optimizer's output for a query

type ForEachFunc 

type ForEachFunc func(d *Document) (bool, error)

ForEachFunc returns false to stop scanning and an error if one occurred

type ForEachOpts 

type ForEachOpts struct {
	// Where are the query conditions
	Where []Where `json:"where,omitempty"`
	// Join are the join conditions
	Join []Join `json:"join,omitempty"`
}

ForEachOpts are options when executing db.ForEach against a collection

type ForeignKey 

type ForeignKey struct {
	// Collection is the foreign collection
	Collection string `json:"collection"`
	// Cascade indicates that the document should be deleted when the foreign key is deleted
	Cascade bool `json:"cascade"`
}

ForeignKey is a reference/relationship to another collection by primary key

type GetCmd added in v0.1.4 

type GetCmd struct {
	// Collection is the collection the document belongs to
	Collection string `json:"collection" validate:"required"`
	// ID is the unique id of the document
	ID string `json:"id" validate:"required"`
}

GetCmd is a serializable get command

type Index 

type Index struct {
	// Name is the indexes unique name in the collection
	Name string `json:"name" validate:"required,min=1"`
	// Fields to index - order matters
	Fields []string `json:"fields" validate:"required,min=1"`
	// Unique indicates that it's a unique index which will enforce uniqueness
	Unique bool `json:"unique"`
	// Unique indicates that it's a primary index
	Primary bool `json:"primary"`
	// ForeignKey indecates that it's an index for a foreign key
	ForeignKey *ForeignKey `json:"foreign_key,omitempty"`
}

Index is a database index used to optimize queries against a collection

type JSONFieldOp 

type JSONFieldOp struct {
	// Path is the path to the field within the document
	Path string `json:"path"`
	// Op is the operation applied
	Op JSONOp `json:"op"`
	// Value is the value applied with the operation
	Value any `json:"value,omitempty"`
	// BeforeValue is the value before the operation was applied
	BeforeValue any `json:"beforeValue,omitempty"`
}

JSONFieldOp is an operation against a JSON field

type JSONOp 

type JSONOp string

JSONOp is an json field operation type 

const (
	// JSONOpRemove removes a field from a json document
	JSONOpRemove JSONOp = "remove"
	// JSONOpAdd adds a json field to a json document
	JSONOpAdd JSONOp = "add"
	// JSONOpReplace replaces an existing field in a json document
	JSONOpReplace JSONOp = "replace"
)

type JavascriptGlobal added in v0.3.0 

type JavascriptGlobal string

JavascriptGlobal is a global variable injected into a javascript function (triggers/authorizers/etc) 

const (
	// JavascriptGlobalDB is the global variable for the database instance - it is injected into all javascript functions
	// All methods, fields are available within the script
	JavascriptGlobalDB JavascriptGlobal = "db"
	// JavascriptGlobalCtx is the context of the request when the function is called - it is injected into all javascript functions
	// All methods, fields are available within the script
	JavascriptGlobalCtx JavascriptGlobal = "ctx"
	// JavascriptGlobalMeta is the context metadta of the request when the function is called - it is injected into all javascript functions
	// All methods, fields are available within the script
	JavascriptGlobalMeta JavascriptGlobal = "meta"
	// JavascriptGlobalTx is the transaction instance - it is injected into all javascript functions except ChangeStream Authorizers
	// All methods, fields are available within the script
	JavascriptGlobalTx JavascriptGlobal = "tx"
	// JavascriptGlobalSchema is the collection schema instance - it is injected into all javascript functions
	// All methods, fields are available within the script
	JavascriptGlobalSchema JavascriptGlobal = "schema"
	// JavascriptGlobalQuery is the query instance - it is injected into only Query-based based javascript functions (including foreach)
	// All methods, fields are available within the script
	JavascriptGlobalQuery JavascriptGlobal = "query"
	// JavascriptGlobalFilter is an array of where clauses - it is injected into ChangeStream based javascript functions
	// All methods, fields are available within the script
	JavascriptGlobalFilter JavascriptGlobal = "filter"
	// JavascriptGlobalDoc is a myjson document - it is injected into document based javascript functions
	// All methods, fields are available within the script
	JavascriptGlobalDoc JavascriptGlobal = "doc"
)

type Join 

type Join struct {
	Collection string  `json:"collection" validate:"required"`
	On         []Where `json:"on" validate:"required,min=1"`
	As         string  `json:"as,omitempty"`
}

Join is a join against another collection

type Optimizer 

type Optimizer interface {
	// Optimize selects the optimal index to use based on the given where clauses
	Optimize(c CollectionSchema, where []Where) (Explain, error)
}

Optimizer selects the best index from a set of indexes based on where clauses

type OrderBy 

type OrderBy struct {
	Direction OrderByDirection `json:"direction" validate:"oneof='desc' 'asc'"`
	Field     string           `json:"field"`
}

OrderBy indicates how to order results returned from a query

type OrderByDirection 

type OrderByDirection string

OrderByDirection is the direction of an order by clause 

const OrderByDirectionAsc OrderByDirection = "asc"

OrderByDirectionAsc is ascending order 

const OrderByDirectionDesc OrderByDirection = "desc"

OrderByDirectionDesc is descending order

type Page 

type Page struct {
	// Documents are the documents that make up the page
	Documents Documents `json:"documents"`
	// Next page
	NextPage int `json:"nextPage"`
	// Document count
	Count int `json:"count"`
	// Stats are statistics collected from a document aggregation query
	Stats PageStats `json:"stats,omitempty"`
}

Page is a page of documents

type PageStats 

type PageStats struct {
	// ExecutionTime is the execution time to get the page
	ExecutionTime time.Duration `json:"executionTime,omitempty"`
	// Explain is the optimizer's output for the query that returned a page
	Explain *Explain `json:"explain,omitempty"`
}

PageStats are statistics collected from a query returning a page

type PropertyIndex 

type PropertyIndex struct {
	AdditionalFields []string `json:"additional_fields,omitempty"`
}

PropertyIndex is an index attached to a json schema property

type Query 

type Query struct {
	// Select selects fields - at least 1 select is required.
	// 1 select with Field: * gets all fields
	Select []Select `json:"select" validate:"min=1,required"`
	// Join joins the results to another collection
	Join []Join `json:"join,omitempty" validate:"dive"`
	// Where filters results. The optimizer will select the appropriate index based on where clauses
	Where []Where `json:"where,omitempty" validate:"dive"`
	// GroupBy groups results by a given list of fields
	GroupBy []string `json:"groupBy,omitempty"`
	// Page is the page of results - it is used with Limit for pagination
	Page int `json:"page" validate:"min=0"`
	// Limit is used to limit the number of results returned
	Limit int `json:"limit,omitempty" validate:"min=0"`
	// OrderBy orders results
	OrderBy []OrderBy `json:"orderBy,omitempty" validate:"dive"`
	// Having applies a final filter after any aggregations have occured
	Having []Where `json:"having,omitempty" validate:"dive"`
}

Query is a query against the NOSQL database

func (Query) String 

func (q Query) String() string

String returns the query as a json string

func (Query) Validate 

func (q Query) Validate(ctx context.Context) error

Validate validates the query and returns a validation error if one exists

type QueryBuilder 

type QueryBuilder struct {
	// contains filtered or unexported fields
}

QueryBuilder is a utility for creating queries via chainable methods

func Q 

func Q() *QueryBuilder

Q creates a new QueryBuilder instance

Example 
package main

import (
	"fmt"

	"github.com/autom8ter/myjson"
)

func main() {
	query := myjson.Q().
		Select(myjson.Select{
			Field: "*",
		}).
		Where(myjson.Where{
			Field: "description",
			Op:    myjson.WhereOpContains,
			Value: "testing",
		}).Query()
	fmt.Println(query.String())
}

Output:

{"select":[{"field":"*"}],"where":[{"field":"description","op":"contains","value":"testing"}],"page":0}

func (*QueryBuilder) GroupBy 

func (q *QueryBuilder) GroupBy(groups ...string) *QueryBuilder

GroupBy adds the GroupBy clause(s) to the query

func (*QueryBuilder) Having 

func (q *QueryBuilder) Having(where ...Where) *QueryBuilder

Having adds the Where clause(s) to the query - they execute after all other clauses have resolved

func (*QueryBuilder) Join 

func (q *QueryBuilder) Join(join ...Join) *QueryBuilder

Join adds the Join clause(s) to the query

func (*QueryBuilder) Limit 

func (q *QueryBuilder) Limit(limit int) *QueryBuilder

Limit adds the Limit clause(s) to the query

func (*QueryBuilder) OrderBy 

func (q *QueryBuilder) OrderBy(ob ...OrderBy) *QueryBuilder

OrderBy adds the OrderBy clause(s) to the query

func (*QueryBuilder) Page 

func (q *QueryBuilder) Page(page int) *QueryBuilder

Page adds the Page clause(s) to the query which controls pagination results

func (*QueryBuilder) Query 

func (q *QueryBuilder) Query() Query

Query returns the built query

func (*QueryBuilder) Select 

func (q *QueryBuilder) Select(fields ...Select) *QueryBuilder

Select adds the SelectFiel(s) to the query

func (*QueryBuilder) Where 

func (q *QueryBuilder) Where(where ...Where) *QueryBuilder

Where adds the Where clause(s) to the query

type QueryCmd added in v0.1.4 

type QueryCmd struct {
	// Collection is the collection the document belongs to
	Collection string `json:"collection" validate:"required"`
	// Query is the query to execute
	Query Query `json:"query,omitempty"`
}

QueryCmd is a serializable query command

type RevertCmd added in v0.3.2 

type RevertCmd struct {
	// Collection is the collection the document belongs to
	Collection string `json:"collection" validate:"required"`
	// ID is the unique id of the document
	ID string `json:"id" validate:"required"`
	// Timestamp is the timestamp to revert to
	Timestamp time.Time `json:"timestamp" validate:"required"`
}

RevertCmd is a serializable revert command

type RollbackCmd added in v0.2.0 

type RollbackCmd struct{}

RollbackCmd is a serializable rollback command

type SchemaProperty 

type SchemaProperty struct {
	// Primary indicates the property is the primary key
	Primary bool `json:"x-primary,omitempty"`
	// Name is the name of the property
	Name string `json:"name" validate:"required"`
	// Description is the description of the property
	Description string `json:"description,omitempty"`
	// Type is the type of the property
	Type string `json:"type" validate:"required"`
	// Path is a dot notation path to the property
	Path string `json:"path" validate:"required"`
	// Immutable indicates the field value is immutable - it will be ignored on updates
	Immutable bool `json:"x-immutable"`
	// Unique indicates the field value is unique
	Unique bool `json:"x-unique"`
	// ForeignKey is a relationship to another collection
	ForeignKey *ForeignKey `json:"x-foreign,omitempty"`
	// Index is a secondary index mapped by index name
	Index map[string]PropertyIndex `json:"x-index,omitempty"`
	// Properties are object properties
	Properties map[string]SchemaProperty `json:"properties,omitempty"`
	// Compute is a computed field that is evaluated at runtime
	Compute *ComputedField `json:"x-compute,omitempty"`
	// Default is the default value of the field
	Default *any `json:"default,omitempty"`
}

SchemaProperty is a property belonging to a JSON Schema

type Select 

type Select struct {
	Aggregate AggregateFunction `json:"aggregate,omitempty" validate:"oneof='count' 'max' 'min' 'sum'"`
	As        string            `json:"as,omitempty"`
	Field     string            `json:"field"`
}

Select is a field to select

type SetCmd added in v0.1.4 

type SetCmd struct {
	// Collection is the collection the document belongs to
	Collection string `json:"collection" validate:"required"`
	// Document is the document to set
	Document *Document `json:"document" validate:"required"`
}

SetCmd is a serializable set command

type TimeTravelCmd added in v0.3.2 

type TimeTravelCmd struct {
	// Collection is the collection the document belongs to
	Collection string `json:"collection" validate:"required"`
	// ID is the unique id of the document
	ID string `json:"id" validate:"required"`
	// Timestamp is the timestamp to travel to
	Timestamp time.Time `json:"timestamp" validate:"required"`
}

TimeTravelCmd is a serializable time travel command

type Transport added in v0.1.0 

type Transport interface {
	Serve(ctx context.Context, db Database) error
}

Transport serves the database over a network (optional for integration with different transport mechanisms)

type Trigger 

type Trigger struct {
	// Name is the unique name of the trigger
	Name string `json:"name" validate:"required"`
	// Order is used to sort triggers into an array where the lower order #s are executed before larger order #s
	Order int `json:"order"`
	// Events is an array of events that the trigger executes on
	Events []EventType `json:"events" validate:"min=1,required"`
	// Script is the javascript script to execute
	Script string `json:"script" validate:"required"`
}

Trigger is a javasript function executed after a database event occurs

type Tx 

type Tx interface {
	// Cmd is a generic command that can be used to execute any command against the database
	Cmd(ctx context.Context, cmd TxCmd) TxResponse
	// Query executes a query against the database
	Query(ctx context.Context, collection string, query Query) (Page, error)
	// Get returns a document by id
	Get(ctx context.Context, collection string, id string) (*Document, error)
	// Create creates a new document - if the documents primary key is unset, it will be set as a sortable unique id
	Create(ctx context.Context, collection string, document *Document) (string, error)
	// Update updates a value in the database
	Update(ctx context.Context, collection, id string, document map[string]any) error
	// Set sets the specified key/value in the database
	Set(ctx context.Context, collection string, document *Document) error
	// Delete deletes the specified key from the database
	Delete(ctx context.Context, collection string, id string) error
	// ForEach scans the optimal index for a collection's documents passing its filters.
	// results will not be ordered unless an index supporting the order by(s) was found by the optimizer
	// Query should be used when order is more important than performance/resource-usage
	ForEach(ctx context.Context, collection string, opts ForEachOpts, fn ForEachFunc) (Explain, error)
	// TimeTravel sets the document to the value it was at the given timestamp.
	// If the document did not exist at the given timestamp, it will return the first version of the document
	TimeTravel(ctx context.Context, collection string, documentID string, timestamp time.Time) (*Document, error)
	// Revert reverts the document to the value it was at the given timestamp.
	// If the document did not exist at the given timestamp, it will persist the first version of the document
	Revert(ctx context.Context, collection string, documentID string, timestamp time.Time) error
	// DB returns the transactions underlying database
	DB() Database
}

Tx is a database transaction interface - it holds the primary methods used while using a transaction

type TxCmd added in v0.1.4 

type TxCmd struct {
	// Create is a create command
	Create *CreateCmd `json:"create,omitempty"`
	// Get is a get command
	Get *GetCmd `json:"get,omitempty"`
	// Set is a set command
	Set *SetCmd `json:"set,omitempty"`
	// Update is an update command
	Update *UpdateCmd `json:"update,omitempty"`
	// Delete is a delete command
	Delete *DeleteCmd `json:"delete,omitempty"`
	// Query is a query command
	Query *QueryCmd `json:"query,omitempty"`
	// Revert is a revert command
	Revert *RevertCmd `json:"revert,omitempty"`
	// TimeTravel is a time travel command
	TimeTravel *TimeTravelCmd `json:"timeTravel,omitempty"`
	// Commit is a commit command - it ends the transaction
	Commit *CommitCmd `json:"commit,omitempty"`
	// Rollback is a rollback command - it ends the transaction
	Rollback *RollbackCmd `json:"rollback,omitempty"`
}

TxCmd is a serializable transaction command

type TxFunc 

type TxFunc func(ctx context.Context, tx Tx) error

TxFunc is a function executed against a transaction - if the function returns an error, all changes will be rolled back. Otherwise, the changes will be commited to the database

type TxResponse added in v0.1.4 

type TxResponse struct {
	// Create is a create response - it returns the created document
	Create *Document `json:"create,omitempty"`
	// Get is a get response - it returns the document from the get request (if it exists)
	Get *Document `json:"get,omitempty"`
	// Set is a set response - it returns the document after the set was applied
	Set *Document `json:"set,omitempty"`
	// Update is an update response - it contains the document after the update was applied
	Update *Document `json:"update,omitempty"`
	// Delete is an empty delete response
	Delete *struct{} `json:"delete,omitempty"`
	// Query is a query response - it contains the documents returned from the query
	Query *Page `json:"page,omitempty"`
	// Revert is a revert response - it contains the document after the revert was applied
	Revert *struct{} `json:"revert,omitempty"`
	// TimeTravel is a time travel response - it contains the document after the time travel was applied
	TimeTravel *Document `json:"timeTravel,omitempty"`
	// Commit is an empty commit response
	Commit *struct{} `json:"commit,omitempty"`
	// Rollback is an empty rollback response
	Rollback *struct{} `json:"rollback,omitempty"`
	// Error is an error response if an error was encountered
	Error *errors.Error `json:"error,omitempty"`
}

TxResponse is a serializable transaction response

type Txn 

type Txn interface {
	// Commit commits the transaction to the database
	Commit(ctx context.Context) error
	// Rollback rollsback all changes made to the datbase
	Rollback(ctx context.Context) error
	// Close closes the transaction - it should be deferred after
	Close(ctx context.Context)
	Tx
}

Txn is a database transaction interface - it holds the methods used while using a transaction + commit,rollback,and close functionality

type UpdateCmd added in v0.1.4 

type UpdateCmd struct {
	// Collection is the collection the document belongs to
	Collection string `json:"collection" validate:"required"`
	// ID is the unique id of the document
	ID string `json:"id" validate:"required"`
	// Update is the set of fields to set
	Update map[string]any `json:"update,omitempty"`
}

UpdateCmd is a serializable update command

type Where 

type Where struct {
	Field string      `json:"field" validate:"required"`
	Op    WhereOp     `json:"op" validate:"oneof='eq' 'neq' 'gt' 'gte' 'lt' 'lte' 'contains' 'containsAny' 'containsAll' 'in'"`
	Value interface{} `json:"value" validate:"required"`
}

Where is a filter against documents returned from a query

type WhereOp 

type WhereOp string

WhereOp is an operation belonging to a where clause 

const WhereOpContains WhereOp = "contains"

WhereOpContains checks if a string field contains subtext 

const WhereOpContainsAll WhereOp = "containsAll"

WhereOpContainsAll is a check whether an array contains all of the given array values 

const WhereOpContainsAny WhereOp = "containsAny"

WhereOpContainsAny is a check whether an array contains any of the given array values 

const WhereOpEq WhereOp = "eq"

WhereOpEq is an equality check. 

const WhereOpGt WhereOp = "gt"

WhereOpGt is a check whether a value is greater than another 

const WhereOpGte WhereOp = "gte"

WhereOpGte is a check whether a value is greater than or equal to another 

const WhereOpHasPrefix WhereOp = "hasPrefix"

WhereOpHasPrefix is a check whether a string value has a prefix 

const WhereOpHasSuffix WhereOp = "hasSuffix"

WhereOpHasSuffix is a check whether a string value has a suffix 

const WhereOpIn WhereOp = "in"

WhereOpIn is a check whether a value is one of a list of values 

const WhereOpLt WhereOp = "lt"

WhereOpLt is a check whether a value is less than another 

const WhereOpLte WhereOp = "lte"

WhereOpLte is a check whether a values is less than or equal to another 

const WhereOpNeq WhereOp = "neq"

WhereOpNeq is a non-equality check 

const WhereOpRegex WhereOp = "regex"

WhereOpRegex is a check whtether a string value matches a regex expression

Source Files

View all Source files

Directories

Path Synopsis
kv
badger
kvutil
registry
tikv

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.