specter

package module
v0.0.0-...-53808f6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 12, 2023 License: MIT Imports: 18 Imported by: 1

README

Specter

Go

Specter is a development toolkit in Go that allows you to develop configuration file processors based on HashiCorp Configuration Language (HCL). With Specter, you can define your own Domain-Specific Language (DSL) using HCL and create a processing pipeline to validate, lint, resolve dependencies, and generate code or output files from these DSL configuration files.

Features

  • Develop your own DSL using HCL
  • Validate and lint configuration files
  • Resolve dependencies between configuration files
  • Generate code or output files from configuration files

Getting Started

To start using Specter, you need to install Go and set up your Go workspace. Then, you can install Specter using the following command:

go get github.com/morebec/specter

Next, you can create a new configuration file processor by defining your DSL in HCL and implementing the processing pipeline. You can find more detailed instructions and examples in the documentation.

Examples

Here are some examples of what you can do with Specter:

Contributions

We welcome contributions to Specter! If you have an idea for a new feature or have found a bug, please open an issue to discuss it. If you want to contribute code, please follow our contribution guidelines and open a pull request.

License

Specter is licensed under the MIT License.

Documentation

Index

Constants

View Source 
const InvalidHCLErrorCode = "invalid_hcl"
View Source 
const LintingErrorCode = "linting_error"
View Source 
const (
	// Unknown is used for attributes where the actual type is unknown.
	Unknown = "any"
)
View Source 
const UnsupportedSpecificationLoaderCode = "unsupported_spec_loader"

UnsupportedSpecificationLoaderCode ErrorSeverity code returned by a SpecificationLoader when a given loader does not support a certain source.

View Source 
const WriteFileOutputsProcessorErrorCode = "write_file_outputs_processor_error"

Variables

This section is empty.

Functions

func MapSpecGroup 

func MapSpecGroup[T any](g SpecificationGroup, p func(s Specification) T) []T

MapSpecGroup performs a map operation on a SpecificationGroup

func UnexpectedSpecTypeError 

func UnexpectedSpecTypeError(actual SpecificationType, expected SpecificationType) error

Types

type AttributeType 

type AttributeType string

AttributeType represents the type of an attribute

type AttributeValue 

type AttributeValue interface {
	IsAttributeValue()
}

type ColoredOutputLogger 

type ColoredOutputLogger struct {
	// contains filtered or unexported fields
}

func NewColoredOutputLogger 

func NewColoredOutputLogger(c ColoredOutputLoggerConfig) *ColoredOutputLogger

func (ColoredOutputLogger) Error 

func (l ColoredOutputLogger) Error(msg string)

func (ColoredOutputLogger) Info 

func (l ColoredOutputLogger) Info(msg string)

func (ColoredOutputLogger) Log 

func (l ColoredOutputLogger) Log(msg string)

func (ColoredOutputLogger) Success 

func (l ColoredOutputLogger) Success(msg string)

func (ColoredOutputLogger) Trace 

func (l ColoredOutputLogger) Trace(msg string)

func (ColoredOutputLogger) Warning 

func (l ColoredOutputLogger) Warning(msg string)

type ColoredOutputLoggerConfig 

type ColoredOutputLoggerConfig struct {
	EnableColors bool
	Writer       io.Writer
}

type DependencyGraph 

type DependencyGraph []Specification

func NewDependencyGraph 

func NewDependencyGraph(specifications ...Specification) DependencyGraph

func (DependencyGraph) Merge 

func (g DependencyGraph) Merge(o DependencyGraph) DependencyGraph

Merge Allows merging this dependency graph with another one and returns the result.

func (DependencyGraph) Resolve 

func (g DependencyGraph) Resolve() (ResolvedDependencies, error)

type DependencySet 

type DependencySet map[SpecificationName]struct{}

func NewDependencySet 

func NewDependencySet(dependencies ...SpecificationName) DependencySet

func (DependencySet) Names 

func (s DependencySet) Names() []SpecificationName

type ExecutionMode 

type ExecutionMode string
const FullMode ExecutionMode = "full"

FullMode will cause a Specter instance to be run fully. 

const LintMode ExecutionMode = "lint"

LintMode will cause a Specter instance to run until the lint step only. 

const PreviewMode ExecutionMode = "preview"

PreviewMode will cause a Specter instance to run until the processing step only, no output will be processed.

type FileOutput 

type FileOutput struct {
	Path string
	Data []byte
	Mode os.FileMode
}

FileOutput is a data structure that can be used by a SpecificationProcessor to output files that can be written by tje WriteFileOutputsProcessor.

type GenericSpecAttribute 

type GenericSpecAttribute struct {
	Name  string
	Value AttributeValue
}

GenericSpecAttribute represents an attribute of a specification. It relies on cty.Value to represent the loaded value.

func (GenericSpecAttribute) AsGenericValue 

func (a GenericSpecAttribute) AsGenericValue() GenericValue

func (GenericSpecAttribute) AsObjectValue 

func (a GenericSpecAttribute) AsObjectValue() ObjectValue

type GenericSpecification 

type GenericSpecification struct {
	Attributes []GenericSpecAttribute
	// contains filtered or unexported fields
}

GenericSpecification is a generic implementation of a Specification that saves its attributes in a list of attributes for introspection. these can be useful for loaders that are looser in what they allow.

func NewGenericSpecification 

func NewGenericSpecification(name SpecificationName, typ SpecificationType, source Source, dependencies []SpecificationName) *GenericSpecification

func (GenericSpecification) Attribute 

func (s GenericSpecification) Attribute(name string) *GenericSpecAttribute

Attribute returns an attribute by its FilePath or nil if it was not found.

func (GenericSpecification) Dependencies 

func (s GenericSpecification) Dependencies() []SpecificationName

func (GenericSpecification) Description 

func (s GenericSpecification) Description() string

func (GenericSpecification) HasAttribute 

func (s GenericSpecification) HasAttribute(name string) bool

HasAttribute indicates if a specification has a certain attribute or not.

func (GenericSpecification) Name 

func (s GenericSpecification) Name() SpecificationName

func (GenericSpecification) SetSource 

func (s GenericSpecification) SetSource(src Source)

func (GenericSpecification) Source 

func (s GenericSpecification) Source() Source

func (GenericSpecification) Type 

func (s GenericSpecification) Type() SpecificationType

type GenericValue 

type GenericValue struct {
	cty.Value
}

GenericValue represents a generic value that is mostly unknown in terms of type and intent.

func (GenericValue) IsAttributeValue 

func (d GenericValue) IsAttributeValue()

type HCLFileConfig 

type HCLFileConfig interface {
	Specifications() []Specification
}

type HCLGenericSpecLoader 

type HCLGenericSpecLoader struct {
	hclparse.Parser
}

HCLGenericSpecLoader this SpecificationLoader loads Specifications as GenericSpecification.

func NewHCLGenericSpecLoader 

func NewHCLGenericSpecLoader() *HCLGenericSpecLoader

NewHCLGenericSpecLoader this SpecificationLoader will load all Specifications to instances of GenericSpecification.

func (HCLGenericSpecLoader) Load 

func (l HCLGenericSpecLoader) Load(s Source) ([]Specification, error)

func (HCLGenericSpecLoader) SupportsSource 

func (l HCLGenericSpecLoader) SupportsSource(s Source) bool

type HCLSpecLoader 

type HCLSpecLoader struct {
	// contains filtered or unexported fields
}

HCLSpecLoader this loader allows to load Specifications to typed structs by providing a HCLFileConfig.

func NewHCLFileConfigSpecLoader 

func NewHCLFileConfigSpecLoader(fileConfigProvider HCLSpecLoaderFileConfigurationProvider) *HCLSpecLoader

func (HCLSpecLoader) Load 

func (l HCLSpecLoader) Load(s Source) ([]Specification, error)

func (HCLSpecLoader) SupportsSource 

func (l HCLSpecLoader) SupportsSource(s Source) bool

type HCLSpecLoaderFileConfigurationProvider 

type HCLSpecLoaderFileConfigurationProvider func() HCLFileConfig

type HCLVariableConfig 

type HCLVariableConfig struct {
	Name        string    `hcl:"FilePath,label"`
	Description string    `hcl:"description,optional"`
	Value       cty.Value `hcl:"value"`
}

HCLVariableConfig represents a block configuration that allows defining variables.

type LinterResult 

type LinterResult struct {
	Severity LinterResultSeverity
	Message  string
}

type LinterResultSet 

type LinterResultSet []LinterResult

LinterResultSet represents a set of LinterResult.

func (LinterResultSet) Errors 

func (s LinterResultSet) Errors() errors.Group

Errors returns a list of LinterResult as errors.

func (LinterResultSet) HasErrors 

func (s LinterResultSet) HasErrors() bool

HasErrors returns if this result set has any result representing an error.

func (LinterResultSet) HasWarnings 

func (s LinterResultSet) HasWarnings() bool

HasWarnings returns if this result set has any result representing a warning.

func (LinterResultSet) Warnings 

func (s LinterResultSet) Warnings() LinterResultSet

Warnings returns another LinterResultSet with only warnings.

type LinterResultSeverity 

type LinterResultSeverity string
const (
	ErrorSeverity   LinterResultSeverity = "error"
	WarningSeverity LinterResultSeverity = "warning"
)

type LocalFileSourceLoader 

type LocalFileSourceLoader struct{}

LocalFileSourceLoader is an implementation of a SourceLoader that loads files from the local system.

func NewLocalFileSourceLoader 

func NewLocalFileSourceLoader() LocalFileSourceLoader

func (LocalFileSourceLoader) Load 

func (l LocalFileSourceLoader) Load(location string) ([]Source, error)

func (LocalFileSourceLoader) Supports 

func (l LocalFileSourceLoader) Supports(target string) bool

type Logger 

type Logger interface {
	// Trace should only be used for debugging purposes.
	Trace(msg string)

	// Info is used to indicate informative messages.
	Info(msg string)

	// Warning is used to indicate events that could be problematic but that do not constitute errors.
	Warning(msg string)

	// Error is used to indicate that an error has occurred within specter.
	Error(msg string)

	// Success is used to indicate that a given action was performed successfully.
	// This can be used in stdout for example to format specific colors for successful actions as opposed to Info.
	Success(msg string)
}

Logger interface to be used by specter and processors to perform logging. implementations can be made for different scenarios, such as outputting to a file, stderr, silencing the logger etc. The logger only provides contextual logging.

type ObjectValue 

type ObjectValue struct {
	Type       AttributeType
	Attributes []GenericSpecAttribute
}

ObjectValue represents a type of attribute value that is a nested data structure as opposed to a scalar value.

func (ObjectValue) IsAttributeValue 

func (o ObjectValue) IsAttributeValue()

type Option 

type Option func(s *Specter)

Option represents an option to configure a specter instance.

func WithExecutionMode 

func WithExecutionMode(m ExecutionMode) Option

WithExecutionMode configures the ExecutionMode of a Specter instance.

func WithLinters 

func WithLinters(linters ...SpecificationLinter) Option

WithLinters configures the SpecificationLinter of a Specter instance.

func WithLoaders 

func WithLoaders(loaders ...SpecificationLoader) Option

WithLoaders configures the SpecificationLoader of a Specter instance.

func WithLogger 

func WithLogger(l Logger) Option

WithLogger configures the Logger of a Specter instance.

func WithOutputProcessors 

func WithOutputProcessors(processors ...OutputProcessor) Option

WithOutputProcessors configures the OutputProcessor of a Specter instance.

func WithProcessors 

func WithProcessors(processors ...SpecificationProcessor) Option

WithProcessors configures the SpecProcess of a Specter instance.

func WithSourceLoaders 

func WithSourceLoaders(loaders ...SourceLoader) Option

WithSourceLoaders configures the SourceLoader of a Specter instance.

type OutputFileRegistry 

type OutputFileRegistry struct {
	GeneratedAt time.Time `json:"generatedAt"`
	Files       []string  `json:"files"`
	FilePath    string
}

OutputFileRegistry allows tracking on the file system the files that were written by the last execution of the WriteFileOutputsProcessor to perform cleaning operations on next executions.

func NewOutputFileRegistry 

func NewOutputFileRegistry(fileName string) OutputFileRegistry

NewOutputFileRegistry returns a new output file registry.

func (*OutputFileRegistry) Clean 

func (r *OutputFileRegistry) Clean() error

Clean will delete all files listed in the registry.

func (*OutputFileRegistry) Load 

func (r *OutputFileRegistry) Load() error

Load loads the registry.

func (*OutputFileRegistry) Write 

func (r *OutputFileRegistry) Write() error

Write will write the registry file to disk.

type OutputProcessingContext 

type OutputProcessingContext struct {
	DependencyGraph ResolvedDependencies
	Outputs         []ProcessingOutput
	Logger          Logger
}

type OutputProcessor 

type OutputProcessor interface {
	// Process performs the processing of outputs generated by SpecificationProcessor.
	Process(ctx OutputProcessingContext) error

	// Name returns the name of this processor.
	Name() string
}

OutputProcessor are services responsible for processing outputs of SpecProcessors.

type ProcessingContext 

type ProcessingContext struct {
	DependencyGraph ResolvedDependencies
	Outputs         []ProcessingOutput
	Logger          Logger
}

func (ProcessingContext) Output 

func (c ProcessingContext) Output(outputName string) ProcessingOutput

Output returns the output associated with a given processor.

type ProcessingOutput 

type ProcessingOutput struct {
	// Name of the Output
	Name string

	// Value of the output
	Value any
}

ProcessingOutput represents an output generated by a SpecificationProcessor.

type ResolvedDependencies 

type ResolvedDependencies SpecificationGroup

ResolvedDependencies represents an ordered list of Specification that should be processed in that specific order to avoid unresolved types. TODO Remove specification group and add its methods here.

type Source 

type Source struct {
	// Location of the source, this can be a local file or a remote file.
	Location string

	// RAW content of the source
	Data []byte

	// Detected format of the source.
	Format SourceFormat
}

Source represents the source code that was used to load a given specification.

type SourceFormat 

type SourceFormat string
const (
	HCLSourceFormat SourceFormat = "hcl"
)

type SourceLoader 

type SourceLoader interface {
	Load(location string) ([]Source, error)
	Supports(location string) bool
}

SourceLoader are services responsible for loading sources.

type SpecBase 

type SpecBase struct {
	// contains filtered or unexported fields
}

func (SpecBase) Description 

func (c SpecBase) Description() string

func (SpecBase) Name 

func (c SpecBase) Name() SpecificationName

func (SpecBase) Source 

func (c SpecBase) Source() Source

func (SpecBase) Type 

func (c SpecBase) Type() SpecificationType

type Specification 

type Specification interface {
	// Name returns the unique Name of this specification.
	Name() SpecificationName

	// Type returns the type of this specification.
	Type() SpecificationType

	// Description of this specification.
	Description() string

	// Source returns the source of this specification.
	Source() Source

	// SetSource sets the source of the specification.
	// This method should only be used by loaders.
	SetSource(s Source)

	// Dependencies returns a list of the Names of the specifications this one depends on.
	Dependencies() []SpecificationName
}

Specification is a general purpose data structure to represent a specification as loaded from a file regardless of the loader used. It is the responsibility of a Module to convert a specification to an appropriate data structure representing the intent of a given Specification.

type SpecificationGroup 

type SpecificationGroup []Specification

SpecificationGroup Represents a list of Specification.

func NewSpecGroup 

func NewSpecGroup(s ...Specification) SpecificationGroup

func (SpecificationGroup) Exclude 

func (g SpecificationGroup) Exclude(p func(s Specification) bool) SpecificationGroup

func (SpecificationGroup) ExcludeType 

func (g SpecificationGroup) ExcludeType(t SpecificationType) SpecificationGroup

func (SpecificationGroup) Merge 

func (g SpecificationGroup) Merge(group SpecificationGroup) SpecificationGroup

Merge Allows merging a group with another one.

func (SpecificationGroup) Select 

func (g SpecificationGroup) Select(p func(s Specification) bool) SpecificationGroup

Select allows filtering the group for certain specifications.

func (SpecificationGroup) SelectName 

func (g SpecificationGroup) SelectName(t SpecificationName) Specification

func (SpecificationGroup) SelectType 

func (g SpecificationGroup) SelectType(t SpecificationType) SpecificationGroup

type SpecificationLinter 

type SpecificationLinter interface {
	Lint(specifications SpecificationGroup) LinterResultSet
}

SpecificationLinter represents a function responsible for linting specifications.

type SpecificationLinterFunc 

type SpecificationLinterFunc func(specifications SpecificationGroup) LinterResultSet

SpecificationLinterFunc implementation of a SpecificationLinter that relies on a func

func CompositeSpecificationLinter 

func CompositeSpecificationLinter(linters ...SpecificationLinter) SpecificationLinterFunc

CompositeSpecificationLinter A Composite linter is responsible for running multiple linters as one.

func SpecificationMustNotHaveUndefinedNames 

func SpecificationMustNotHaveUndefinedNames() SpecificationLinterFunc

SpecificationMustNotHaveUndefinedNames ensures that no specification has an undefined type FilePath

func SpecificationsMustHaveDescriptionAttribute 

func SpecificationsMustHaveDescriptionAttribute() SpecificationLinterFunc

SpecificationsMustHaveDescriptionAttribute ensures that all specifications have a description.

func SpecificationsMustHaveLowerCaseNames 

func SpecificationsMustHaveLowerCaseNames() SpecificationLinterFunc

SpecificationsMustHaveLowerCaseNames ensures that all specification type names are lower case.

func SpecificationsMustHaveUniqueNames 

func SpecificationsMustHaveUniqueNames() SpecificationLinterFunc

SpecificationsMustHaveUniqueNames ensures that names are unique amongst specifications.

func (SpecificationLinterFunc) Lint 

func (l SpecificationLinterFunc) Lint(specifications SpecificationGroup) LinterResultSet

type SpecificationLoader 

type SpecificationLoader interface {
	// Load loads a slice of Specification from a Source, or returns an error if it encountered a failure.
	Load(s Source) ([]Specification, error)

	// SupportsSource indicates if this loader supports a certain source or not.
	SupportsSource(s Source) bool
}

SpecificationLoader is a service responsible for loading Specifications from Sources.

type SpecificationName 

type SpecificationName string
const UndefinedSpecificationName SpecificationName = ""

UndefinedSpecificationName constant used to test against undefined SpecificationName.

type SpecificationProcessor 

type SpecificationProcessor interface {
	// Name returns the unique FilePath of this processor.
	Name() string

	// Process processes a group of specifications.
	Process(ctx ProcessingContext) ([]ProcessingOutput, error)
}

SpecificationProcessor are services responsible for performing work using Specifications.

type SpecificationType 

type SpecificationType string

type Specter 

type Specter struct {
	SourceLoaders    []SourceLoader
	Loaders          []SpecificationLoader
	Processors       []SpecificationProcessor
	Linters          []SpecificationLinter
	OutputProcessors []OutputProcessor
	Logger           Logger
	ExecutionMode    ExecutionMode
}

Specter is the service responsible to run a specter pipeline.

func New 

func New(opts ...Option) *Specter

New allows creating a new specter instance using the provided options.

func (Specter) LintSpecifications 

func (s Specter) LintSpecifications(specifications []Specification) LinterResultSet

LintSpecifications runes all Linters against a list of Specifications.

func (Specter) LoadSources 

func (s Specter) LoadSources(sourceLocations []string) ([]Source, error)

LoadSources only performs the Load sources step.

func (Specter) LoadSpecifications 

func (s Specter) LoadSpecifications(sources []Source) ([]Specification, error)

LoadSpecifications performs the loading of Specifications.

func (Specter) ProcessOutputs 

func (s Specter) ProcessOutputs(specifications ResolvedDependencies, outputs []ProcessingOutput) error

ProcessOutputs sends a list of ProcessingOutputs to the registered OutputProcessors.

func (Specter) ProcessSpecifications 

func (s Specter) ProcessSpecifications(specifications ResolvedDependencies) ([]ProcessingOutput, error)

ProcessSpecifications sends the specifications to processors.

func (Specter) ResolveDependencies 

func (s Specter) ResolveDependencies(specifications []Specification) (ResolvedDependencies, error)

ResolveDependencies resolves the dependencies between specifications.

func (Specter) Run 

func (s Specter) Run(sourceLocations []string) error

Run the pipeline from start to finish.

type Stats 

type Stats struct {
	StartedAt         time.Time
	EndedAt           time.Time
	NbSourceLocations int
	NbSources         int
	NbSpecifications  int
	NbOutputs         int
}

func (Stats) ExecutionTime 

func (s Stats) ExecutionTime() time.Duration

type WriteFileOutputsProcessor 

type WriteFileOutputsProcessor struct {
	// contains filtered or unexported fields
}

WriteFileOutputsProcessor is a processor responsible for writing ProcessingOutput referring to files. To perform its work this processor looks at the processing context for any FileOutput.

func NewWriteFilesProcessor 

func NewWriteFilesProcessor(conf WriteFileOutputsProcessorConfig) *WriteFileOutputsProcessor

func (WriteFileOutputsProcessor) Name 

func (f WriteFileOutputsProcessor) Name() string

func (WriteFileOutputsProcessor) Process 

func (f WriteFileOutputsProcessor) Process(ctx OutputProcessingContext) error

type WriteFileOutputsProcessorConfig 

type WriteFileOutputsProcessorConfig struct {
	// Indicates if a registry file should be used to clean up generated files when running the WriteFileOutputsProcessor.
	UseRegistry bool
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.