protoc_gen_jsonschema

package module
v1.5.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 14, 2023 License: Apache-2.0 Imports: 5 Imported by: 0

README

Protobuf to JSON-Schema compiler

This takes protobuf definitions and converts them into JSONSchemas, which can be used to dynamically validate JSON messages.

Useful for people who define their data using ProtoBuf, but use JSON for the "wire" format.

"Heavily influenced" by Google's protobuf-to-BigQuery-schema compiler.

Generated Schemas

  • One JSONSchema file is generated for each root-level proto message and ENUM. These are intended to be stand alone self-contained schemas which can be used to validate a payload derived from their source proto message
  • Nested message schemas become referenced "definitions". This means that you know the name of the proto message they came from, and their schema is not duplicated (within the context of one JSONSchema file at least)

Logic

  • For each proto file provided
    • Generates schema for each ENUM
      • JSONSchema filename deried from ENUM name
    • Generates schema for each Message
      • Builds a list of every nested message and converts them to JSONSchema
      • Recursively converts attributes and nested messages within the root message
        • Optionally makes all fields required
        • Optionally allows NULL values
        • Optionally allows additional properties
        • Optionally marks all fields required
        • Specially marked fields are labelled required (options.proto)
        • Specially marked fields are omitted (options.proto)
        • Special handling for "OneOf"
        • Special handling for arrays
        • Special handling for maps
      • Injects references to nested messages
      • JSONSchema filename derived from Message name
    • Bundles these into a protoc generator response

Installation

Note: This tool requires Go 1.11+ to be installed.

Install this plugin using Go:

go install github.com/ImSingee/protoc-gen-jsonschema/cmd/protoc-gen-jsonschema@latest

Usage

Note: This plugin requires the protoc CLI to be installed.

protoc-gen-jsonschema is designed to run like any other proto generator. The following examples show how to use options flags to enable different generator behaviours (more examples in the Makefile too).

protoc \ # The protobuf compiler
--jsonschema_out=. \ # jsonschema out directory
--proto_path=testdata/proto testdata/proto/ArrayOfPrimitives.proto # proto input directories and folders

Configuration Parameters

The following configuration parameters are supported. They should be added to the protoc command and can be combined as a comma-delimited string. Some examples are included in the following Examples section.

Options can also be provided in this format (which is easier on the eye):

protoc \
  --plugin=${HOME}/go/bin/protoc-gen-jsonschema \
  --jsonschema_opt=enforce_oneof
  --jsonschema_opt=file_extension=schema.json \
  --jsonschema_opt=disallow_additional_properties \
  --jsonschema_out=schemas \
  --proto_path=proto
CONFIG DESCRIPTION
all_fields_required Require all fields in schema
allow_null_values Allow null values in schema
info Enable info logging
debug Enable debug logging
disallow_additional_properties Disallow additional properties in schema
disallow_bigints_as_strings Disallow big integers as strings
enforce_oneof Interpret Proto "oneOf" clauses
enums_as_strings_only Only include strings in the allowed values for enums
file_extension Specify a custom file extension for generated schemas
json_fieldnames Use JSON field names only
prefix_schema_files_with_package Prefix the output filename with package
proto_and_json_fieldnames Use proto and JSON field names

Custom Proto Options

If you don't want to use the configuration parameters (admittedly quite a nasty cli syntax) then some of the generator behaviour can be controlled using custom proto options. These are defined in options.proto, and your protoc command will need to include this file. See the sample protos and generator commands in the Makefile.

Enum Options

These apply to specifically marked enums, giving you more finely-grained control than with the CLI flags.

Field Options

These apply to specifically marked fields, giving you more finely-grained control than with the CLI flags.

  • ignore: Ignore (omit) a specific field
  • required: Mark a specific field as being REQUIRED
File Options

These options apply to an entire proto file.

  • ignore: Ignore (skip) a specific file
  • extension: Specify a custom file-extension for the generated schema for this file
Message Options

These options apply to a specific proto message.

Validation Options

We are also beginning to support validation options from protoc-gen-validate.

At the moment the following are supported (but export more in the future):

  • Arrays
    • MaxItems
    • MinItems
  • Strings
    • MaxLength
    • MinLength
    • Pattern

Examples

Require all fields

Because proto3 doesn't accommodate this.

protoc \
--jsonschema_out=all_fields_required:. \
--proto_path=testdata/proto testdata/proto/ArrayOfPrimitives.proto
Allow NULL values

By default, JSONSchemas will reject NULL values unless we explicitly allow them

protoc \
--jsonschema_out=allow_null_values:. \
--proto_path=testdata/proto testdata/proto/ArrayOfPrimitives.proto
Enable debug logging
protoc \
--jsonschema_out=debug:. \
--proto_path=testdata/proto testdata/proto/ArrayOfPrimitives.proto
Disallow additional properties

JSONSchemas won't validate JSON containing extra parameters

protoc \
--jsonschema_out=disallow_additional_properties:. \
--proto_path=testdata/proto testdata/proto/ArrayOfPrimitives.proto
Disallow permissive validation of big-integers as strings

(eg scientific notation)

protoc \
--jsonschema_out=disallow_bigints_as_strings:. \
--proto_path=testdata/proto testdata/proto/ArrayOfPrimitives.proto
Prefix generated schema files with their package name (as a directory)
protoc \
--jsonschema_out=prefix_schema_files_with_package:. \
--proto_path=testdata/proto testdata/proto/ArrayOfPrimitives.proto
Target specific messages within a proto file
# Generates MessageKind10.jsonschema and MessageKind11.jsonschema
# Use this to generate json schema from proto files with multiple messages
# Separate schema names with '+'
protoc \
--jsonschema_out=messages=[MessageKind10+MessageKind11]:. \
--proto_path=testdata/proto testdata/proto/TwelveMessages.proto
Generate fields with JSON names
protoc \
--jsonschema_out=json_fieldnames:. \
--proto_path=testdata/proto testdata/proto/ArrayOfPrimitives.proto
Custom schema file extension

The default file extension is json. You can override that with the "file_extension" parameter.

protoc \
--jsonschema_out=file_extension=jsonschema:. \
--proto_path=internal/converter/testdata/proto internal/converter/testdata/proto/ArrayOfPrimitives.proto

Sample protos (for testing)

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	// optional protoc.gen.jsonschema.EnumOptions enum_options = 1128;
	E_EnumOptions = &file_options_proto_extTypes[3]
)

Extension fields to descriptorpb.EnumOptions.

View Source 
var (
	// optional protoc.gen.jsonschema.FieldOptions field_options = 1125;
	E_FieldOptions = &file_options_proto_extTypes[0]
)

Extension fields to descriptorpb.FieldOptions.

View Source 
var (
	// optional protoc.gen.jsonschema.FileOptions file_options = 1126;
	E_FileOptions = &file_options_proto_extTypes[1]
)

Extension fields to descriptorpb.FileOptions.

View Source 
var (
	// optional protoc.gen.jsonschema.MessageOptions message_options = 1127;
	E_MessageOptions = &file_options_proto_extTypes[2]
)

Extension fields to descriptorpb.MessageOptions.

View Source 
var File_options_proto protoreflect.FileDescriptor

Functions

This section is empty.

Types

type EnumOptions 

type EnumOptions struct {

	// Enums tagged with this will have be encoded to use constants instead of simple types (supports value annotations):
	EnumsAsConstants bool `protobuf:"varint,1,opt,name=enums_as_constants,json=enumsAsConstants,proto3" json:"enums_as_constants,omitempty"`
	// Enums tagged with this will only provide string values as options (not their numerical equivalents):
	EnumsAsStringsOnly bool `protobuf:"varint,2,opt,name=enums_as_strings_only,json=enumsAsStringsOnly,proto3" json:"enums_as_strings_only,omitempty"`
	// Enums tagged with this will have enum name prefix removed from values:
	EnumsTrimPrefix bool `protobuf:"varint,3,opt,name=enums_trim_prefix,json=enumsTrimPrefix,proto3" json:"enums_trim_prefix,omitempty"`
	// Enums tagged with this will not be processed
	Ignore bool `protobuf:"varint,4,opt,name=ignore,proto3" json:"ignore,omitempty"`
	// contains filtered or unexported fields
}

Custom EnumOptions

func (*EnumOptions) Descriptor deprecated 

func (*EnumOptions) Descriptor() ([]byte, []int)

Deprecated: Use EnumOptions.ProtoReflect.Descriptor instead.

func (*EnumOptions) GetEnumsAsConstants 

func (x *EnumOptions) GetEnumsAsConstants() bool

func (*EnumOptions) GetEnumsAsStringsOnly 

func (x *EnumOptions) GetEnumsAsStringsOnly() bool

func (*EnumOptions) GetEnumsTrimPrefix 

func (x *EnumOptions) GetEnumsTrimPrefix() bool

func (*EnumOptions) GetIgnore 

func (x *EnumOptions) GetIgnore() bool

func (*EnumOptions) ProtoMessage 

func (*EnumOptions) ProtoMessage()

func (*EnumOptions) ProtoReflect 

func (x *EnumOptions) ProtoReflect() protoreflect.Message

func (*EnumOptions) Reset 

func (x *EnumOptions) Reset()

func (*EnumOptions) String 

func (x *EnumOptions) String() string

type FieldOptions 

type FieldOptions struct {

	// Fields tagged with this will be omitted from generated schemas
	Ignore bool `protobuf:"varint,1,opt,name=ignore,proto3" json:"ignore,omitempty"`
	// Fields tagged with this will be marked as "required" in generated schemas
	Required bool `protobuf:"varint,2,opt,name=required,proto3" json:"required,omitempty"`
	// Fields tagged with this will constrain strings using the "minLength" keyword in generated schemas
	MinLength int32 `protobuf:"varint,3,opt,name=min_length,json=minLength,proto3" json:"min_length,omitempty"`
	// Fields tagged with this will constrain strings using the "maxLength" keyword in generated schemas
	MaxLength int32 `protobuf:"varint,4,opt,name=max_length,json=maxLength,proto3" json:"max_length,omitempty"`
	// Fields tagged with this will constrain strings using the "pattern" keyword in generated schemas
	Pattern string `protobuf:"bytes,5,opt,name=pattern,proto3" json:"pattern,omitempty"`
	// contains filtered or unexported fields
}

Custom FieldOptions

func (*FieldOptions) Descriptor deprecated 

func (*FieldOptions) Descriptor() ([]byte, []int)

Deprecated: Use FieldOptions.ProtoReflect.Descriptor instead.

func (*FieldOptions) GetIgnore 

func (x *FieldOptions) GetIgnore() bool

func (*FieldOptions) GetMaxLength 

func (x *FieldOptions) GetMaxLength() int32

func (*FieldOptions) GetMinLength 

func (x *FieldOptions) GetMinLength() int32

func (*FieldOptions) GetPattern 

func (x *FieldOptions) GetPattern() string

func (*FieldOptions) GetRequired 

func (x *FieldOptions) GetRequired() bool

func (*FieldOptions) ProtoMessage 

func (*FieldOptions) ProtoMessage()

func (*FieldOptions) ProtoReflect 

func (x *FieldOptions) ProtoReflect() protoreflect.Message

func (*FieldOptions) Reset 

func (x *FieldOptions) Reset()

func (*FieldOptions) String 

func (x *FieldOptions) String() string

type FileOptions 

type FileOptions struct {

	// Files tagged with this will not be processed
	Ignore bool `protobuf:"varint,1,opt,name=ignore,proto3" json:"ignore,omitempty"`
	// Override the default file extension for schemas generated from this file
	Extension string `protobuf:"bytes,2,opt,name=extension,proto3" json:"extension,omitempty"`
	// contains filtered or unexported fields
}

Custom FileOptions

func (*FileOptions) Descriptor deprecated 

func (*FileOptions) Descriptor() ([]byte, []int)

Deprecated: Use FileOptions.ProtoReflect.Descriptor instead.

func (*FileOptions) GetExtension 

func (x *FileOptions) GetExtension() string

func (*FileOptions) GetIgnore 

func (x *FileOptions) GetIgnore() bool

func (*FileOptions) ProtoMessage 

func (*FileOptions) ProtoMessage()

func (*FileOptions) ProtoReflect 

func (x *FileOptions) ProtoReflect() protoreflect.Message

func (*FileOptions) Reset 

func (x *FileOptions) Reset()

func (*FileOptions) String 

func (x *FileOptions) String() string

type MessageOptions 

type MessageOptions struct {

	// Messages tagged with this will not be processed
	Ignore bool `protobuf:"varint,1,opt,name=ignore,proto3" json:"ignore,omitempty"`
	// Messages tagged with this will have all fields marked as "required":
	AllFieldsRequired bool `protobuf:"varint,2,opt,name=all_fields_required,json=allFieldsRequired,proto3" json:"all_fields_required,omitempty"`
	// Messages tagged with this will additionally accept null values for all properties:
	AllowNullValues bool `protobuf:"varint,3,opt,name=allow_null_values,json=allowNullValues,proto3" json:"allow_null_values,omitempty"`
	// Messages tagged with this will have all fields marked as not allowing additional properties:
	DisallowAdditionalProperties bool `` /* 148-byte string literal not displayed */
	// Messages tagged with this will have all nested enums encoded to use constants instead of simple types (supports value annotations):
	EnumsAsConstants bool `protobuf:"varint,5,opt,name=enums_as_constants,json=enumsAsConstants,proto3" json:"enums_as_constants,omitempty"`
	// contains filtered or unexported fields
}

Custom MessageOptions

func (*MessageOptions) Descriptor deprecated 

func (*MessageOptions) Descriptor() ([]byte, []int)

Deprecated: Use MessageOptions.ProtoReflect.Descriptor instead.

func (*MessageOptions) GetAllFieldsRequired 

func (x *MessageOptions) GetAllFieldsRequired() bool

func (*MessageOptions) GetAllowNullValues 

func (x *MessageOptions) GetAllowNullValues() bool

func (*MessageOptions) GetDisallowAdditionalProperties 

func (x *MessageOptions) GetDisallowAdditionalProperties() bool

func (*MessageOptions) GetEnumsAsConstants 

func (x *MessageOptions) GetEnumsAsConstants() bool

func (*MessageOptions) GetIgnore 

func (x *MessageOptions) GetIgnore() bool

func (*MessageOptions) ProtoMessage 

func (*MessageOptions) ProtoMessage()

func (*MessageOptions) ProtoReflect 

func (x *MessageOptions) ProtoReflect() protoreflect.Message

func (*MessageOptions) Reset 

func (x *MessageOptions) Reset()

func (*MessageOptions) String 

func (x *MessageOptions) String() string

Source Files

View all Source files

Directories

Path Synopsis
cmd
protoc-gen-jsonschema
protoc plugin which converts .proto to JSON schema It is called by protoc and generates JSON-schema files.
 protoc plugin which converts .proto to JSON schema It is called by protoc and generates JSON-schema files.
internal
converter

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.