v2

README

MongoDB Module

This module provides the capability to persist SCIM resources in MongoDB.

💡 Usage

To get this package:

# make sure Go 1.13
go get github.com/imulab/go-scim/mongo/v2

💾 Persistence

This basic db.DB implementation in this module assumes one-to-one mapping between a SCIM resource type and a MongoDB collection.

Index

MongoDB indexes are automatically created for attributes whose uniqueness=server or uniqueness=global, and for attributes who were annotated with @MongoIndex. When uniqueness is not none, a unique index is created; otherwise, just the single field index is created. This module does not support the creation of composite index, or geo-spatial indexes. In addition, index creation failures are logged as warning to the logger, instead of being returned as an error. A particular failure situation to watch out for is that, after MongoDB 4.2, creating an already existing index will actually return an error, in contrast to just returning an implicit success in versions before. Such failure can still be considered an implicit success in our case as the indexes will be there.

Metadata

The domain space of SCIM path characters and MongoDB path characters do not completely overlap. Some characters legal in the SCIM world may not be legal in the MongoDB world. One example is the $ character, typically seen in the $ref attribute. This module provides a way to correlate MongoDB specific metadata to SCIM attributes so the attribute name and attribute path can be override.

The module provides stock metadata in the public folder which covers User, Group and Enterprise User extension. A typical metadata looks like:

{
  "metadata": [
    {
      "id": "urn:ietf:params:scim:schemas:core:2.0:User:groups.$ref",
      "mongoName": "ref",
      "mongoPath": "groups.ref"
    }
  ]
}

The id field is the unique attribute id. mongoName specifies the field name within its containing object. mongoPath specifies the path name from the document root. Use ReadMetadata to register bytes of the metadata file.

Atomicity

The module utilizes the atomicity of MongoDB and safely performs modification operations without explicitly locking the resource. Replace and Delete operations would only perform data modification if the id and meta.version fields matches the record in MongoDB. If no match was found, a conflict error is returned to indicate some current process must have modified the resource in between.

Projection

The projection feature of the Query method is not completely fool-proof. It does not check for the returned property of the target attribute pointed to by the attributes or excludedAttributes specified in crud.Projection parameter. This is done intentionally, as db.DB is not a client facing component with respect to SCIM API. Any projection parameters supplied should have been pre-sanitized so that it does not contradict with the returned property.

There are also cases where callers may wish to carry out operations after the query on fields not requested by the client. In this case, callers can use Options.IgnoreProjection() to disable projection altogether so the database always return the full version of the resource.

✒ Serialization

This module provides direct serialization and de-deserialization between SCIM resource and MongoDB BSON format, without the transition of an intermediary data format.

The serialization of the less significant components like filter, sort, pagination and projection are still being converted to bson.D, before being serialized to BSON and sent to MongoDB.

🚧 Testing

This module uses org/dockertest to setup testing docker containers at test run time. The environment variables to customize the local docker connection are:

# shows the environment variable and their default value
TEST_DOCKER_ENDPOINT=""
TEST_DOCKER_MONGO_IMAGE="bitnami/mongodb"
TEST_DOCKER_MONGO_TAG=latest
TEST_DOCKER_MONGO_USERNAME=testUser
TEST_DOCKER_MONGO_SECRET=s3cret
TEST_DOCKER_MONGO_DB_NAME=mongo_database_test_suite

Documentation

Overview

This package provides MongoDB implementation of db.DB interface and necessary tools to help persisting resources in MongoDB.

Index

• Constants
• func DB(resourceType *spec.ResourceType, coll *mongo.Collection, opt *DBOptions) db.DB
• func ReadMetadata(raw []byte) error
• func ReadMetadataFromReader(reader io.Reader) error
• func TransformCompiledFilter(root *expr.Expression, resourceType *spec.ResourceType) (bson.D, error)
• func TransformFilter(scimFilter string, resourceType *spec.ResourceType) (bson.D, error)
• type DBOptions
  • func Options() *DBOptions
  • func (opt *DBOptions) IgnoreProjection() *DBOptions
• type Metadata

Constants

const (
	// @MongoIndex annotates a field so that a corresponding index is generated in MongoDB. If uniqueness=server on
	// the annotated field, a unique index is generated. Otherwise, an ordinary index is generated.
	AnnotationMongoIndex = "@MongoIndex"
)

Functions

func DB

func DB(resourceType *spec.ResourceType, coll *mongo.Collection, opt *DBOptions) db.DB

Create a db.DB implementation that persists data in MongoDB. This implementation supports one-to-one correspondence of a SCIM resource type to a MongoDB collection.

The database will attempt to create MongoDB indexes on attributes whose uniqueness is global or server, or that has been annotated with "@MongoIndex". For unique attributes, a unique MongoDB index will be created, otherwise, it is just an ordinary index. Any index creation error are treated as non-error and simply ignored.

This implementation has limited capability of correctly performing field projection according to the specification. It dumbly treats the *crud.Projection parameter as it is without performing any sanitation. As a result, if any returned=always field is excluded, it will not be returned; similarly, if any returned=never field is included, it will be returned. It is expected by downstream calls to perform a pre-sanitation on the parameters or perform a post-guard operation to ensure no sensitive information is leaked.

The "github.com/imulab/go-scim/pkg/v2/json" provides a post-guard operation in its serialization function to ensure returned=never parameters are never leaked. When used with this database, the only situation needs to be worried about is that returned=always parameter may not be returned at all when included intentionally in the "attributes" parameter list. This behaviour might be acceptable. If not, pre-sanitation of the projection list is required.

If so desired, use Options().IgnoreProjection() to ignore projection altogether and return a complete version of the result every time.

This implementation also has limited capability to correctly performing sorting according to the specification. The control is not as fine grained as in crud.SeekSortTarget. It can sort on singular type, but may fail if asked to sort on multiValued type, or a singular type within a multiValued type.

This implementation do not directly use the SCIM attribute path to persist into MongoDB. Instead, it uses a concept of MongoDB persistence paths (or mongo paths). These mongo paths are introduced to provide an alternative name to SCIM path when SCIM path consists characters illegal to MongoDB. For instance, group.$ref attribute consists a dollar sign that cannot be used as part of field names in MongoDB. Similarly, schema extensions will almost always introduce illegal characters because schemas such as "urn:ietf:params:scim:schemas:extension:enterprise:2.0:User" contain dot which is used as path separators in MongoDB. When this is the case, this package allows used to register metadata (see metadata.go) that can be associated with the target attribute in order to provide an alias to the SCIM path suitable to be persisted in MongoDB. When a metadata is associated to a target attribute, the metadata's MongoName or MongoPath will be used; otherwise, the attribute's Name and Path will be used.

The atomicity of MongoDB is utilized to avoid explicit locking when modifying the resource. When performing Replace (which provides service to SCIM replace and SCIM patch) and Delete operations, the resources id and version is used as the criteria to match a document in store before carrying out the operation. If the provided id and version failed to match a document, a conflict error is returned instead of a notFound error. This is because caller already provided a resource as argument which was fetched from the database, hence, the resource by the id must have existed. The only reason that id and version failed to match would then because another process modified the resource concurrently. Therefore, conflict seems to be a reasonable error code.

func ReadMetadata

func ReadMetadata(raw []byte) error

Read metadata and add all metadata to the hub

func ReadMetadataFromReader

func ReadMetadataFromReader(reader io.Reader) error

ReadMetadataFromReader reads and registered the JSON encoded metadata from reader.

func TransformCompiledFilter

func TransformCompiledFilter(root *expr.Expression, resourceType *spec.ResourceType) (bson.D, error)

Compile and transform a compiled SCIM filter to bsonx.Val that contains the original filter in MongoDB compatible format. This slight optimization allow the caller to pre-compile frequently used queries and save the trip to the filter parser and compiler.

func TransformFilter

func TransformFilter(scimFilter string, resourceType *spec.ResourceType) (bson.D, error)

Compile and transform a SCIM filter string to a bsonx.Val that contains the original filter in MongoDB compatible format.

Types

type DBOptions

type DBOptions struct {
	// contains filtered or unexported fields
}

func Options

func Options() *DBOptions

DB options

func (*DBOptions) IgnoreProjection

func (opt *DBOptions) IgnoreProjection() *DBOptions

Ask the database to ignore any projection parameters. This might be reasonable when the downstream services wish to perform further actions on the complete version of the resource.

type Metadata

type Metadata struct {
	Id        string `json:"id"`
	MongoName string `json:"mongoName"`
	MongoPath string `json:"mongoPath"`
}

Mongo package extension to spec.Attribute. Here we define a MongoDB property alias to override the attribute name when saving to or reading from MongoDB. This is necessary because some valid SCIM field names are not valid in MongoDB.

To define metadata to be supplied to ReadMetadataFromReader, compose something similar to:

{
	"metadata": [
		{
			"id": "urn:ietf:params:scim:schemas:core:2.0:User:groups.$ref",
			"mongoName": "ref",
			"mongoPath": "groups.ref"
		}
	]
}