dgx

README

We have been using Dgraph for over a year at Mooncamp and while loving the product we found some issues that we needed to write additional tools for.

Query Rendering

dgx allows you to render GraphQL+- queries based on its data representation. The types for the data representation are mostly copied from the github.com/dgraph-io/dgraph/gql package.

Start testing the translation between the data and string representation of the query by starting the querybuilder tool.

$ go get -u mooncamp.com/dgx/cmd/dgx
$ dgx querybuilder

We find that using the data representation over the string form gives us several advantages. When defining queries, we realized that composing queries can speed up development and reduce maintenance. However, in the string representation queries are hard to compose, as string concatenations are messy and error prone. When the query is in the data form compositions are substantially easier. Using a typed language like Go brings additional safety, as the query structure can be verified using the language's type system.

GraphQL+- Query

query {
  bladerunner (func: eq(name@en, "Blade Runner")) {
	name@en
	initial_release_date
   }
}

Data Representation in Go

gql.GraphQuery{
	Alias: "bladerunner",
	Func: &gql.Function{
		Attr: "name",
		Lang: "en",
		Name: "eq",
		Args: []gql.Arg{
			{Value: "Blade Runner"},
		},
	},
	Children: []gql.GraphQuery{
		{Attr: "Name", Langs: []string{"en"}},
		{Attr: "initial_release_date"},
	},
}

Moving Query Ownership to the Frontend

While it is standard practice for GraphQL clients to take ownership of the queries, with Dgraph this is usually not an option as there is no safe way for the backend to implement access restriction based on the string representation of the query. However, moving the queries to the client offers great flexibility and should therefore be the preferable option. Using the data representation the backend now has access to every aspect of the query.

This project comes with an example implementation of how minimal access restriction could be implemented. The approach assumes that any identity accessing the graph is stored in the graph itself and that there are no edges connecting the network of the identity itself and the network the identity doesn't have access to.

In the example the approach assumes that there is no connection between the networks {0,1,2,3,4} and {5,6,7,8,9,10,12}. This is often the case when you implement multi tenancy with a single database instance. For example a user John Doe has access to the data of his own company Goggles Inc, but no access to the other company Amazing Corp. At the same time there is no connection between the data of Goggles Inc and Amazing Corp.

The approach works by only allowing uid as the root function, where the corresponding uid is the users identity. Additionally, if the uid is not the users identity a separate query can be provided that proofs the connection between the input uid and the users identity.

Dgraph Extensions

When taking control over the query language we have the ability to extend the language itself. For example we sometimes want to set a default value for a non found relation in dgraph. Therefore the GraphQuery type was enhanced with a Default interface{} property. Extensions are applied on the response of dgraph. Another extension is planned, which will allow setting the cardinality between two nodes. This solves the issue of representing one-to-one relationships within dgraph.

Don't trust us

Although the rendering code is pretty well tested using the actual dgraph tests as an inspiration there could still be bugs which could potentially lead to security issues. Luckily we can use the dgraph query parser as a source of truth, meaning any bug in the rendering code will be detected right away. We are actually testing any query coming from the client for potential issues and return 500 if the query couldn't be verified.

Example Application

Checkout example/webapp/main.go for an example usage of all components described above in a web application setting. The example makes use of the gokit architecture, so if you are confused about what is going on checkout gokit.io.

Documentation

Index

• type ComputedPropertiesHandler
• type ComputedProperty
• type Dogma
• type IdentityProvider
• type IdentityProviderRequest
• type Model
• type ModelHandler
• type MutationVerifier
• type QueryFn
• type QueryHardener
• type QueryMiddleware
• type QueryRequest
  • func (u QueryRequest) ProvideIdentity() uint64
  • func (q QueryRequest) ProvideLanguages() []string
• type QueryResponse
• type QueryVerifier
• type ResourceFilter
• type ResourceFilterProvider
• type Unauthorized
  • func (Unauthorized) Error() string

Types

type ComputedPropertiesHandler

type ComputedPropertiesHandler interface {
	Prepare([]gql.GraphQuery) []gql.GraphQuery
	Compute(context.Context, []gql.GraphQuery, map[string]interface{}) (resp map[string]interface{}, err error)
}

type ComputedProperty

type ComputedProperty struct {
	Attributes        []gql.GraphQuery
	Function          func(context.Context, interface{}, map[string]interface{}) (interface{}, error)
	AttributeDefaults map[string]interface{}
}

type Dogma

type Dogma interface {
	// NewTxn opens a new dgraph transaction and returns a new
	// Dogma instance. If the Dogma instance didn't opena a
	// transaction, everything is immediately commited.
	NewTxn(ctx context.Context) Dogma

	// Commit commits the dgraph transaction
	Commit(ctx context.Context) error

	// Discard discards the dgraph transaction
	Discard(ctx context.Context)

	// Query queries for the requested resources and stores them
	// into the value pointed to by `in`. Therefore `in` needs to
	// be a pointer. Also, the query response needs to match the
	// inserted type, (such that the json result can be
	// unmarshalled into `in`). The root query alias, or query
	// name is ignored and the output is directly copied to the
	// value pointed to by `in`. Nodes that have an edge to a
	// `deletedAt` field are automatically filtered.
	Query(ctx context.Context, queries []gql.GraphQuery, in interface{}) error

	// QueryMultipleResults behaves similar to Dogma.Query and is useful
	// when you expect multiple result objects, which is the case
	// if you have multiple non-var queries, such as
	//
	// query {
	//   person (func: uid(0x01)) {
	//     uid
	//     name
	//   }
	//   office (func: uid(0x02)) {
	//     uid
	//     address
	//   }
	// }
	//
	// as a result you could then use the following type:
	//
	// var input struct {
	// 	Person struct {
	// 		ID string `json:"uid"`
	// 		Name string `json:"name"`
	// 	} `json:"person"`
	// 	Office struct {
	// 		ID string `json:"uid"`
	// 		Address string `json:"address"`
	// 	} `json:'office"`
	// }
	QueryMultipleResults(ctx context.Context, queries []gql.GraphQuery, in interface{}) error

	// Select allows for simple querying of entities and stores
	// them into the value pointed to by `in`. Therefore `in`
	// needs to be a pointer.
	Select(ctx context.Context, ids []uint64, in interface{}, attributes ...gql.GraphQuery) error

	// SelectSingle behaves the same as Select, but fetches only a single item
	SelectSingle(ctx context.Context, id uint64, in interface{}, attributes ...gql.GraphQuery) error

	// Update updates the value pointed to by `in`. Therefore `in`
	// needs to be a pointer. Similar to a mutation in a GraphQL
	// Environment you can query for the updated data using query
	// provided by the `query` argument. If the type of `in` has
	// dgx.Model as a promoted field, the ModifiedAt
	// property is automatically updated.
	Update(ctx context.Context, in interface{}, attributes ...gql.GraphQuery) error

	// UpdateSingle behaves like Update but operators on a single entity
	UpdateSingle(ctx context.Context, in interface{}, attributes ...gql.GraphQuery) error

	// Create inserts the value pointed to by `in`. Therefore `in`
	// needs to be a pointer. Similar to a mutation in a GraphQL
	// Environment you can query for the inserted data using query
	// provided by the `query` argument. If the type of `in` has
	// dgx.Model as a promoted field, the CreatedAt and
	// ModifedAt properties are automatically set.
	Create(ctx context.Context, in interface{}, attributes ...gql.GraphQuery) error

	// CreateSingle behaves like Create but operates on a single entity
	CreateSingle(ctx context.Context, in interface{}, attributes ...gql.GraphQuery) error

	// Delete requires the type of `in` to have dgx.Model
	// as a promoted field. It sets the DeletedAt property to
	// time.Now(), updates the value and exists. It does not
	// delete the record from Dgraph.
	Delete(ctx context.Context, in interface{}) error

	// DeleteSingle behaves like Delete but operates on a single entity
	DeleteSingle(ctx context.Context, in interface{}) error
}

Dogma (Dgraph Object Graph Mapping Architecture)

type IdentityProvider

type IdentityProvider interface {
	Identity(ctx context.Context) (uint64, bool)
}

type IdentityProviderRequest

type IdentityProviderRequest interface {
	ProvideIdentity() uint64
}

type Model

type Model struct {
	ID uint64 `json:"uid,omitempty" yaml:"uid"`

	CreatedAt  *time.Time `json:"createdAt,omitempty" yaml:"createdAt"`
	DeletedAt  *time.Time `json:"deletedAt,omitempty" yaml:"deletedAt"`
	ModifiedAt *time.Time `json:"modifiedAt,omitempty" yaml:"modifiedAt"`
}

Model should be used as a promoted field in any business model entity

type ModelHandler

type ModelHandler interface {
	// UpdateModel sets the ModifiedAt and CreatedAt fields
	// according to the state of the entity. If the ID field is
	// not set, the entity is considered new and CreatedAt,
	// ModifiedAt is set. If the ID is not set and all other
	// fields are empty the entity is ignored. If any fields in
	// addition to the ID field are set the ModifiedAt field is
	// updated. The update is applied recursively.
	UpdateModel(time.Time, interface{})

	// SetDeletedAt sets the deleted field of any type that
	// hosts `Model` as a promoted field.
	SetDeletedAt(time.Time, interface{})

	// FilterDeleted applies a filter to every node of the
	// provided queries that automatically filters for any node
	// that has the deletedAt field set.
	FilterDeleted([]gql.GraphQuery) []gql.GraphQuery
}

ModelHandler provides functionality based on any type that includes `Model` as a promoted field. For internal usages only.

type MutationVerifier

type MutationVerifier interface {
	Verify(context.Context, QueryHardener, interface{}) (bool, error)
}

type QueryFn

type QueryFn func(context.Context, []gql.GraphQuery) (map[string]interface{}, error)

QueryFn is the function querying the dgraph database

type QueryHardener

type QueryHardener interface {
	Harden(ctx context.Context, identity uint64, queries []gql.GraphQuery) ([]gql.GraphQuery, error)
}

type QueryMiddleware

type QueryMiddleware func(QueryFn) QueryFn

QueryMiddleware allows modifying either the response or the queries when quering the dgraph database

type QueryRequest

type QueryRequest struct {
	Identity int `json:"identity"`

	Queries   []gql.GraphQuery       `json:"queries"`
	Alias     string                 `json:"alias"`
	Variables map[string]string      `json:"variables"`
	Proof     map[int]gql.GraphQuery `json:"proof"`

	Languages []string `json:"languages"`
}

func (QueryRequest) ProvideIdentity

func (u QueryRequest) ProvideIdentity() uint64

func (QueryRequest) ProvideLanguages

func (q QueryRequest) ProvideLanguages() []string

type QueryResponse

type QueryResponse struct {
	Response map[string]interface{}
	Error    error
}

type QueryVerifier

type QueryVerifier interface {
	QueryAllowed(ctx context.Context, queries []gql.GraphQuery, userID int, proofs map[int]gql.GraphQuery) (bool, error)
}

type ResourceFilter

type ResourceFilter struct {
	Dependencies []ResourceFilter
	Filter       []gql.GraphQuery
}

type ResourceFilterProvider

type ResourceFilterProvider interface {
	Provide(ctx context.Context, identity uint64) (filter map[string]ResourceFilter, ignoredModels []string, err error)
}

type Unauthorized

type Unauthorized struct{}

func (Unauthorized) Error

func (Unauthorized) Error() string