jtdinfer

README

JSON typedef infer

This is a port of json-typedef-infer for Go. The reason for porting this is that I was in need of JTD inference from code and not as a CLI tool.

For more information about JSON Typedef and its RFC and how to use different kind of hints see json-typedef-infer.

Usage

See examples directory for runnable examples and how to infer JTD.

schema := NewInferrer(WithoutHints()).
    Infer("my-string").
    IntoSchema()
// {
//   "type": "string"
// }

If you have multiple rows of objects or lists as strings you can pass them to the shorthand function InferStrings.

rows := []string{
    `{"name":"Joe", "age": 52, "something_optional": true, "something_nullable": 1.1}`,
    `{"name":"Jane", "age": 48, "something_nullable": null}`,
}
schema := InferStrings(rows, WithoutHints()).IntoSchema()
// {
//   "properties": {
//     "age": {
//       "type": "uint8"
//     },
//     "name": {
//       "type": "string"
//     },
//     "something_nullable": {
//       "nullable": true,
//       "type": "float64"
//     }
//   },
//   "optionalProperties": {
//     "something_optional": {
//       "type": "boolean"
//     }
//   }
// }

Documentation

Index

• Constants
• type Discriminator
• type HintSet
  • func NewHintSet() HintSet
  • func (h HintSet) Add(v []string) HintSet
  • func (h HintSet) IsActive() bool
  • func (h HintSet) PeekActive() (string, bool)
  • func (h HintSet) SubHints(key string) HintSet
• type Hints
  • func WithoutHints() Hints
  • func (h Hints) IsEnumActive() bool
  • func (h Hints) IsValuesActive() bool
  • func (h Hints) PeekActiveDiscriminator() (string, bool)
  • func (h Hints) SubHints(key string) Hints
• type InferredNumber
  • func NewNumber() *InferredNumber
  • func (i *InferredNumber) ContainedBy(nt NumType) bool
  • func (i *InferredNumber) Infer(n float64) *InferredNumber
  • func (i *InferredNumber) IntoType(defaultType NumType) jtd.Type
• type InferredSchema
  • func NewInferredSchema() *InferredSchema
  • func (i *InferredSchema) Infer(value any, hints Hints) *InferredSchema
  • func (i *InferredSchema) IntoSchema(hints Hints) Schema
• type Inferrer
  • func InferStrings(rows []string, hints Hints) *Inferrer
  • func NewInferrer(hints Hints) *Inferrer
  • func (i *Inferrer) Infer(value any) *Inferrer
  • func (i *Inferrer) IntoSchema() Schema
• type NumType
  • func (n NumType) AsRange() (float64, float64)
  • func (n NumType) IntoType() jtd.Type
  • func (n NumType) IsFloat() bool
• type Properties
• type Schema
• type SchemaType

Constants

const Wildcard = "-"

Wildcard represents the character that matches any value for hints.

Types

type Discriminator

type Discriminator struct {
	Discriminator string
	Mapping       map[string]*InferredSchema
}

Discriminator represents discriminators for the schema.

type HintSet

type HintSet struct {
	Values [][]string
}

HintSet represents a list of paths (lists) to match for hints.

func NewHintSet

func NewHintSet() HintSet

NewHintSet creates a new empty `HintSet`.

func (HintSet) Add

func (h HintSet) Add(v []string) HintSet

Add will add a path (slice) to the `HintSet`.

func (HintSet) IsActive

func (h HintSet) IsActive() bool

IsActive returns true if any set in the hint set his active.

func (HintSet) PeekActive

func (h HintSet) PeekActive() (string, bool)

PeekActive returns the currently active value if any. The returned boolean tells if a value was found.

func (HintSet) SubHints

func (h HintSet) SubHints(key string) HintSet

SubHints will filter all the current sets and keep those who's first element matches the passed key or wildcard.

type Hints

type Hints struct {
	DefaultNumType NumType
	Enums          HintSet
	Values         HintSet
	Discriminator  HintSet
}

Hints contains the default number type to use and all the hints for enums, values and discriminators.

func WithoutHints

func WithoutHints() Hints

WithoutHints is a shorthand to return empty hints.

func (Hints) IsEnumActive

func (h Hints) IsEnumActive() bool

IsEnumActive checks if the enum hint set is active.

func (Hints) IsValuesActive

func (h Hints) IsValuesActive() bool

IsValuesActive checks if the values hint set is active.

func (Hints) PeekActiveDiscriminator

func (h Hints) PeekActiveDiscriminator() (string, bool)

PeekActiveDiscriminator will peek the currently active discriminator, if any. The returned boolean tells if there is an active discriminator.

func (Hints) SubHints

func (h Hints) SubHints(key string) Hints

SubHints will return the sub hints for all hint sets for the passed key.

type InferredNumber

type InferredNumber struct {
	Min       float64
	Max       float64
	IsInteger bool
}

InferredNumber represents the state for a column that is a number. It holds the seen maximum and minimum value together with information about if all seen numbers are integers.

func NewNumber

func NewNumber() *InferredNumber

NewNumber will return a new `InferredNumber`.

func (*InferredNumber) ContainedBy

func (i *InferredNumber) ContainedBy(nt NumType) bool

ContainedBy checks if an inferred number column can be contained within the passed `NumType`, meaning it is above the minimum and below the maximum value for the number type.

func (*InferredNumber) Infer

func (i *InferredNumber) Infer(n float64) *InferredNumber

Infer will infer a value, updating the state for the `InferredNumber`.

func (*InferredNumber) IntoType

func (i *InferredNumber) IntoType(defaultType NumType) jtd.Type

IntoType will convert an `InferredNumber` to a `jtd.Type`.

type InferredSchema

type InferredSchema struct {
	SchemaType    SchemaType
	Number        *InferredNumber
	Enum          map[string]struct{}
	Array         *InferredSchema
	Properties    Properties
	Values        *InferredSchema
	Discriminator Discriminator
	Nullable      *InferredSchema
}

InferredSchema is the schema while being inferred that holds information about fields.

func NewInferredSchema

func NewInferredSchema() *InferredSchema

NewInferredSchema will return a new, empty, `InferredSchema`.

func (*InferredSchema) Infer

func (i *InferredSchema) Infer(value any, hints Hints) *InferredSchema

Infer will infer the schema by trying to mimic the way it's implemented in the Rust implementation at https://github.com/jsontypedef/json-typedef-infer/blob/master/src/inferred_schema.rs. Since we don't have enums of this kind in Go we're using a struct with pointers to a schema instead of wrapping the enums.

func (*InferredSchema) IntoSchema

func (i *InferredSchema) IntoSchema(hints Hints) Schema

IntoSchema will convert an `InferredSchema` to a final `Schema`.

type Inferrer

type Inferrer struct {
	Inference *InferredSchema
	Hints     Hints
}

Inferrer represents the `InferredSchema` with its state combined with the hints used when inferring.

func InferStrings

func InferStrings(rows []string, hints Hints) *Inferrer

InferStrings accepts a slice of strings and will try to JSON unmarshal each row. If an error occurs the inferrer will return with the state it had when the error occurred. If you already have the type of your data such as a slice of numbers or a map of strings you can pass them directly to `Infer`. This is just a convenience method if all you got is strings.

func NewInferrer

func NewInferrer(hints Hints) *Inferrer

NewInferrer will create a new inferrer with a default `InferredSchema`.

func (*Inferrer) Infer

func (i *Inferrer) Infer(value any) *Inferrer

Infer will infer the schema.

func (*Inferrer) IntoSchema

func (i *Inferrer) IntoSchema() Schema

IntoSchema will convert the `InferredSchema` into a final `Schema`.

type NumType

type NumType uint8

NumType represents the type of number a number should be represented in the JTD.

const (
	NumTypeUint8 NumType = iota
	NumTypeInt8
	NumTypeUint16
	NumTypeInt16
	NumTypeUint32
	NumTypeInt32
	NumTypeFloat32
	NumTypeFloat64
)

Available number types.

func (NumType) AsRange

func (n NumType) AsRange() (float64, float64)

AsRange returns the maximum and minimum value for a `NumType`.

func (NumType) IntoType

func (n NumType) IntoType() jtd.Type

IntoType will convert a `NumType` to a `jtd.Type`.

func (NumType) IsFloat

func (n NumType) IsFloat() bool

IsFloat returns true if the `NumType` is a float.

type Properties

type Properties struct {
	Required map[string]*InferredSchema
	Optional map[string]*InferredSchema
}

Properties represents all required and optional properties which is the same as JSON objects.

type Schema

type Schema struct {
	Definitions          map[string]Schema      `json:"definitions,omitempty"`
	Metadata             map[string]interface{} `json:"metadata,omitempty"`
	Nullable             bool                   `json:"nullable,omitempty"`
	Ref                  *string                `json:"ref,omitempty"`
	Type                 jtd.Type               `json:"type,omitempty"`
	Enum                 []string               `json:"enum,omitempty"`
	Elements             *Schema                `json:"elements,omitempty"`
	Properties           map[string]Schema      `json:"properties,omitempty"`
	OptionalProperties   map[string]Schema      `json:"optionalProperties,omitempty"`
	AdditionalProperties bool                   `json:"additionalProperties,omitempty"`
	Values               *Schema                `json:"values,omitempty"`
	Discriminator        string                 `json:"discriminator,omitempty"`
	Mapping              map[string]Schema      `json:"mapping,omitempty"`
}

Schema represents the JTD schema that will get inferred. It's a hard copy of the upstream type since we want and need to be able to omit empty fields. Ref: https://github.com/jsontypedef/json-typedef-go/issues/7

type SchemaType

type SchemaType int8

SchemaType represents the type of schema element. It will map to the available types for JTD.

const (
	SchemaTypeUnknown SchemaType = iota
	SchemaTypeAny
	SchemaTypeBoolean
	SchemaTypeNumber
	SchemaTypeString
	SchemaTypeTimestmap
	SchemaTypeEnum
	SchemaTypeArray
	SchemaTypeProperties
	SchemaTypeValues
	SchemaTypeDiscriminator
	SchemaTypeNullable
)

Available schema types.