stencil

README

Stencil

Stencil is a schema registry that provides schema management and validation dynamically, efficiently, and reliably to ensure data compatibility across applications.

Usage

sinks:
  name: stencil
  config:
    host: https://stencil.com
    namespace_id: test-namespace
    schema_id: example
    format: json
    send_format_header: false

Config Definition

Key	Value	Example	Description
host	string	https://stencil.com	The hostname of the stencil service	required
namespace_id	string	myNamespace	The namespace ID of the stencil service	required
schema_id	string	mySchmea	The schema ID which will be created in the above-mentioned namespace	required
format	string	json	The schema format in which data will sink to stencil	optional
send_format_header	bool	false	If schema format needs to be changed. Suppose changing format from json to avro,
provide below config value as true and schema format in format config.	optional

Contributing

Refer to the contribution guidelines for information on contributing to this module.

Documentation

Index

• Constants
• func New(c httpClient, logger log.Logger) plugins.Syncer
• type AvroFields
• type AvroSchema
• type AvroType
• type Config
• type JSONType
• type JsonProperty
• type JsonSchema
• type Sink
  • func (*Sink) Close() error
  • func (s *Sink) Init(ctx context.Context, config plugins.Config) error
  • func (s *Sink) Sink(ctx context.Context, batch []models.Record) error

Constants

const (
	JSONTypeObject  JSONType = "object"
	JSONTypeString  JSONType = "string"
	JSONTypeNumber  JSONType = "number"
	JSONTypeArray   JSONType = "array"
	JSONTypeBoolean JSONType = "boolean"
	JSONTypeNull    JSONType = "null"

	AvroTypeNull    AvroType = "null"
	AvroTypeBoolean AvroType = "boolean"
	AvroTypeInteger AvroType = "int"
	AvroTypeLong    AvroType = "long"
	AvroTypeFloat   AvroType = "float"
	AvroTypeDouble  AvroType = "double"
	AvroTypeBytes   AvroType = "bytes"
	AvroTypeString  AvroType = "string"
	AvroTypeRecord  AvroType = "record"
	AvroTypeArray   AvroType = "array"
	AvroTypeMap     AvroType = "map"
)

Functions

func New

func New(c httpClient, logger log.Logger) plugins.Syncer

New returns a pointer to an initialized Sink Object

Types

type AvroFields

type AvroFields struct {
	Name string      `json:"name"`
	Type interface{} `json:"type"`
}

type AvroSchema

type AvroSchema struct {
	Type      string       `json:"type"`
	Namespace string       `json:"namespace"`
	Name      string       `json:"name"`
	Fields    []AvroFields `json:"fields"`
}

type AvroType

type AvroType string

type Config

type Config struct {
	Host        string `mapstructure:"host" validate:"required"`
	NamespaceID string `mapstructure:"namespace_id" validate:"required"`
	Format      string `mapstructure:"format" validate:"oneof=json avro" default:"json"`
}

Config holds the set of configuration options for the sink

type JSONType

type JSONType string

type JsonProperty

type JsonProperty struct {
	Type        []JSONType `json:"type"`
	Description string     `json:"description"`
}

type JsonSchema

type JsonSchema struct {
	Id         string                  `json:"$id"`
	Schema     string                  `json:"$schema"`
	Title      string                  `json:"title"`
	Type       JSONType                `json:"type"`
	Properties map[string]JsonProperty `json:"properties"`
}

type Sink

type Sink struct {
	plugins.BasePlugin
	// contains filtered or unexported fields
}

Sink manages the sinking of data to Stencil

func (*Sink) Close

func (*Sink) Close() error

Close will be called once after everything is done

func (*Sink) Init

func (s *Sink) Init(ctx context.Context, config plugins.Config) error

Init initializes the sink

func (*Sink) Sink

func (s *Sink) Sink(ctx context.Context, batch []models.Record) error

Sink helps to sink record to stencil