inspector

package module
v1.4.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 28, 2024 License: MIT Imports: 22 Imported by: 8

README

Inspector

Inspector is a code-generation framework of special wrappers of arbitrary types and data structures that allows to read/write/iterate fields without using reflection and produce minimum or zero allocations.

Each type-wrapper (next "inspector" or "type-inspector") has name of original type with suffix Inspector and implements Inspector interface.

Intro

The main idea: you may read/write/iterate (inspect) structure of arbitrary type in common way. The historical reason of development that framework was dyntpl package and many others. Usually this problem solves using reflection, but that way is extremely slow in general and produces huge amount of allocs on every use of reflect.Value type. There is good library reflect2 that solves 'reflect.Value' problem, but it is also slow. The perfect way if to use type assertion together with hardcoded combinations of all possible paths to type fields. Unfortunately this way isn't a pure "dynamic solution". Let's consider that problem using example:

Let we have type T:

type T struct {
    L1 *L1
}

type L1 struct {
    L2 *L2
}

type L2 struct {
    L3 *L3
}

type L3 struct {
    S string
    I int64
    F float64
}

with many nested subtypes. And we need to read data of fields for arbitrary path, eg:

  • obj.L1.L2.L3.S
  • obj.L1.L2.L3.F
  • ...

The hard requirement is a "dynamic" paths to fields - at any time may require to read value of any field.

Let's check how that problem may be solved ob showcase:

reflect

Solution https://github.com/koykov/versus/blob/master/inspector2/reflect.go#L8

It's a pure dynamic solution, it solves the problem, but benchmark looks bad:

BenchmarkReflect/obj.L1.L2.L3.S-8         	 3119121	       375.6 ns/op	      64 B/op	       5 allocs/op

Speed is not acceptable and too many allocations - they will trigger problems with GC.

reflect2

Solution https://github.com/koykov/versus/blob/master/inspector2/reflect2.go#L12

It's also pure dynamic solution, and it should solve problem with allocations (cost of use reflect.Value). It's true, but benchmarks shows problems:

BenchmarkReflect2/obj.L1.L2.L3.S-8         	 2973918	       391.3 ns/op	       0 B/op	       0 allocs/op

Allocations problem is solved, but speed is much worse than native reflect due to internal design of reflect2.frozenConfig (it uses sync.Map inside and speed reduces due to sync operations).

inspector

Solution https://github.com/koykov/versus/blob/master/inspector2/inspector_test.go#L13 Benchmark https://github.com/koykov/versus/blob/master/inspector2/inspector_test.go#L18

BenchmarkInspector/obj.L1.L2.L3.S-8         	159301698	         7.596 ns/op	       0 B/op	       0 allocs/op

And that speed is acceptable. See explanation is the next chapter.

How it works

As mentioned, the perfect way is using type assertion together with hardcoded combinations of all possible paths to type fields, i.e. zero reflection. Type T allows the following paths combinations:

  • obj.L1.L2.L3.S
  • obj.L1.L2.L3.I
  • obj.L1.L2.L3.F
  • obj.L1.L2.L3
  • obj.L1.L2
  • obj.L1

Type is simple, thus has only 6 combinations. The code considers these combinations looks the following https://github.com/koykov/versus/blob/master/inspector2/inspector2_ins/t_ins.go#L31. So primitive and thus so fast.

But what about big types? They provide hundreds/thousands combinations and write code manually is so big and boring work. The further support is a problem as well. Therefore, this work was automatized writing special code-generation tool.

Additional features

Fields values reading is so important feature, but inspectors also provide features:

Basic types support

static

For support of basic types (int, uint, float64, ...) was developed special static inspector. It uses in dyntpl and decoder packages for primitive types.

strings

Special for types string and [][]byte was developed inspector strings. See test/bench.

map[string]any

Popular type uses together with encoding/json supported by inspector map[string]any. See test/bench https://github.com/koykov/inspector/blob/master/test/stranymap_test.go

Documentation

Index

Constants

View Source 
const FloatPrecision = 1e-3
View Source 
const Nil = "nil"

Variables

View Source 
var (
	ErrNoConfig            = errors.New("no config provided")
	ErrUnknownTarget       = errors.New("unknown target provided")
	ErrUnknownInspector    = errors.New("unknown inspector")
	ErrNoGOPATH            = errors.New("no GOPATH variable found")
	ErrDstNotExists        = errors.New("destination directory doesn't exists")
	ErrUnknownEncodingType = errors.New("unknown encoding type")
	ErrUnsupportedType     = errors.New("unsupported type")
	ErrMustPointerType     = errors.New("variable must have pointer type")
	ErrNotImplement        = errors.New("not implemented")
)
View Source 
var (
	ErrNoConvFunc = errors.New("convert function doesn't exists")
)

Functions

func Assign 

func Assign(dst, src any) (ok bool)

Assign is a generic assign function.

func AssignBuf 

func AssignBuf(dst, src any, buf AccumulativeBuffer) (ok bool)

AssignBuf is a generic buffered assign function.

Walks other registered assign callbacks and try to execute each of them. Stops when current callback return true.

func AssignToBool 

func AssignToBool(dst, src any, _ AccumulativeBuffer) (ok bool)

AssignToBool assign(converts) source to bool destination.

func AssignToBytes 

func AssignToBytes(dst, src any, buf AccumulativeBuffer) (ok bool)

AssignToBytes assigns(converts) source to bytes destination.

func AssignToFloat 

func AssignToFloat(dst, src any, _ AccumulativeBuffer) (ok bool)

AssignToFloat assigns(converts) source to float destination.

func AssignToInt 

func AssignToInt(dst, src any, _ AccumulativeBuffer) (ok bool)

AssignToInt assigns(converts) source to int destination.

func AssignToStr 

func AssignToStr(dst, src any, buf AccumulativeBuffer) (ok bool)

AssignToStr assigns(converts) source to string destination.

func AssignToUint 

func AssignToUint(dst, src any, _ AccumulativeBuffer) (ok bool)

AssignToUint assigns(converts) source to unsigned int destination.

func Bufferize 

func Bufferize(buf, p []byte) ([]byte, []byte)

Bufferize appends p to the buffer and returns both pointers to buffer and buffered data.

func BufferizeString 

func BufferizeString(buf []byte, s string) ([]byte, string)

BufferizeString appends s to the buffer and returns both pointers to buffer and buffered data.

func DEQMustCheck 

func DEQMustCheck(path string, options *DEQOptions) bool

DEQMustCheck makes a decision must field (by given path) checks or not during DeepEqual().

func EqualFloat32 

func EqualFloat32(a, b float32, opts *DEQOptions) bool

func EqualFloat64 

func EqualFloat64(a, b float64, opts *DEQOptions) bool

func RegisterAssignFn 

func RegisterAssignFn(fn AssignFn)

RegisterAssignFn registers new assign function to the registry.

func RegisterInspector 

func RegisterInspector(name string, ins Inspector)

RegisterInspector saves inspector to the registry.

func RegisterStrToXFn added in v1.4.2 

func RegisterStrToXFn(typ, snippet string, imports []string)

RegisterStrToXFn registers new snippet.

func StrConvSnippet 

func StrConvSnippet(s, typn, typu, _var string) (string, []string, error)

StrConvSnippet builds a conversion snippet according arguments.

Types

type AccumulativeBuffer 

type AccumulativeBuffer interface {
	// AcquireBytes returns more space to use.
	AcquireBytes() []byte
	// ReleaseBytes returns space to the buffer.
	ReleaseBytes([]byte)
	// Bufferize makes a copy of p to buffer and returns pointer to copy.
	Bufferize(p []byte) []byte
	// BufferizeString makes a copy of s to buffer and returns pointer to copy.
	BufferizeString(s string) string
	// Reset all accumulated data.
	Reset()
}

AccumulativeBuffer describes buffer that accumulates bytes data. Collects data during inspector functions work.

type AssignFn 

type AssignFn func(dst, src any, buf AccumulativeBuffer) bool

AssignFn describes signature of assign function.

Assigns src to dst with dynamic typecasting. Returns true if typecasting success, false otherwise.

type BaseInspector 

type BaseInspector struct{}

BaseInspector describes base struct.

func (BaseInspector) Capacity added in v1.4.5 

func (i BaseInspector) Capacity(_ any, _ *int, _ ...string) error

func (BaseInspector) Compare added in v1.4.5 

func (i BaseInspector) Compare(_ any, _ Op, _ string, _ *bool, _ ...string) error

func (BaseInspector) Copy added in v1.4.5 

func (i BaseInspector) Copy(_ any) (dst any, err error)

func (BaseInspector) CopyTo added in v1.4.5 

func (i BaseInspector) CopyTo(_, _ any, _ AccumulativeBuffer) error

func (BaseInspector) DeepEqual added in v1.4.5 

func (i BaseInspector) DeepEqual(l, r any) bool

func (BaseInspector) DeepEqualWithOptions added in v1.4.5 

func (i BaseInspector) DeepEqualWithOptions(_, _ any, _ *DEQOptions) bool

func (BaseInspector) Get added in v1.4.5 

func (i BaseInspector) Get(_ any, _ ...string) (any, error)

func (BaseInspector) GetTo added in v1.4.5 

func (i BaseInspector) GetTo(_ any, _ *any, _ ...string) error

func (BaseInspector) Length added in v1.4.5 

func (i BaseInspector) Length(_ any, _ *int, _ ...string) error

func (BaseInspector) Loop added in v1.4.5 

func (i BaseInspector) Loop(_ any, _ Iterator, _ *[]byte, _ ...string) error

func (BaseInspector) Reset added in v1.4.5 

func (i BaseInspector) Reset(_ any) error

func (BaseInspector) Set added in v1.4.5 

func (i BaseInspector) Set(_, _ any, _ ...string) error

func (BaseInspector) SetWithBuffer added in v1.4.5 

func (i BaseInspector) SetWithBuffer(_, _ any, _ AccumulativeBuffer, _ ...string) error

func (BaseInspector) TypeName added in v1.4.5 

func (i BaseInspector) TypeName() string

func (BaseInspector) Unmarshal added in v1.4.5 

func (i BaseInspector) Unmarshal(_ []byte, _ Encoding) (any, error)

type ByteBuffer 

type ByteBuffer struct {
	// contains filtered or unexported fields
}

func NewByteBuffer 

func NewByteBuffer(size int) *ByteBuffer

func (*ByteBuffer) AcquireBytes 

func (b *ByteBuffer) AcquireBytes() []byte

func (*ByteBuffer) Bufferize added in v1.4.2 

func (b *ByteBuffer) Bufferize(p []byte) []byte

func (*ByteBuffer) BufferizeString added in v1.4.2 

func (b *ByteBuffer) BufferizeString(s string) string

func (*ByteBuffer) ReleaseBytes 

func (b *ByteBuffer) ReleaseBytes(p []byte)

func (*ByteBuffer) Reset 

func (b *ByteBuffer) Reset()

type ByteStringWriter 

type ByteStringWriter interface {
	Write(p []byte) (n int, err error)
	WriteByte(c byte) error
	WriteString(s string) (n int, err error)
	Bytes() []byte
	String() string
	Reset()
}

ByteStringWriter is an extended writer interface.

type Compiler 

type Compiler struct {
	// contains filtered or unexported fields
}

func NewCompiler 

func NewCompiler(conf *Config) (*Compiler, error)

func (*Compiler) Compile 

func (c *Compiler) Compile() error

func (*Compiler) GetTotal 

func (c *Compiler) GetTotal() int

GetTotal returns the total number of compiled types.

func (*Compiler) WriteXML added in v1.4.3 

func (c *Compiler) WriteXML() error

type Config added in v1.4.3 

type Config struct {
	// Target points to source for parsing (package, directory or file).
	Target Target
	// Package indicates source directory relatives to GOPATH (or modules) directory.
	Package string
	// Directory indicates source directory contains go files.
	Directory string
	// File indicates path to Go file.
	File string
	// Destination directory path.
	Destination string
	// Package to import inspector code.
	// Required for directory or file targets.
	Import string
	// Registry of blacklisted types.
	BlackList map[string]struct{}
	// Flag denies destination directory cleaning.
	NoClean bool
	// Flag denies compiling inspectors to separate files.
	// Reserved, still not works.
	NoSplit bool
	// Flag if destination equals source.
	InPlace bool
	// Continue even if error occurred.
	Force bool
	// Path to build debug data in XML format.
	XML string
	// Output buffer.
	Buf ByteStringWriter
	// Logger of debug messages.
	Logger Logger
}

func (*Config) Copy added in v1.4.3 

func (c *Config) Copy() *Config

type DEQOptions 

type DEQOptions struct {
	// Float fields comparison precision.
	Precision float64
	// List of fields to exclude from check.
	Exclude map[string]struct{}
	// Limit fields to check.
	Filter map[string]struct{}
}

DEQOptions describes DeepEqual options.

type Encoding 

type Encoding uint
const (
	EncodingJSON Encoding = iota
)

type Inspector 

type Inspector interface {
	// TypeName returns name of underlying type.
	TypeName() string
	// Get returns value from src according path.
	Get(src any, path ...string) (any, error)
	// GetTo writes value from src to buf according path.
	GetTo(src any, buf *any, path ...string) error
	// Set sets value to dst according path.
	Set(dst, value any, path ...string) error
	// SetWithBuffer is a buffered version of Set().
	SetWithBuffer(dst, value any, buf AccumulativeBuffer, path ...string) error
	// Compare applies condition cond to value in src by path and right.
	// Result will set to result buffer.
	Compare(src any, cond Op, right string, result *bool, path ...string) error
	// Loop iterates in src value taking by path using iterator iter.
	Loop(src any, iter Iterator, buf *[]byte, path ...string) error
	// DeepEqual compares l and r.
	DeepEqual(l, r any) bool
	// DeepEqualWithOptions compares l and r corresponding options.
	DeepEqualWithOptions(l, r any, options *DEQOptions) bool
	// Unmarshal parses encoded data according encoding type.
	Unmarshal(p []byte, typ Encoding) (any, error)
	// Copy makes a copy of x.
	Copy(x any) (any, error)
	// CopyTo makes a copy of src to dst using buffer.
	CopyTo(src, dst any, buf AccumulativeBuffer) error
	// Length puts length of field value by path in src to the buffer.
	Length(src any, result *int, path ...string) error
	// Capacity puts capacity of field value by path in src to the buffer.
	Capacity(src any, result *int, path ...string) error
	// Reset resets x.
	Reset(x any) error
}

Inspector signature.

func GetInspector 

func GetInspector(name string) (Inspector, error)

GetInspector returns inspector from the registry.

type Iterator added in v1.4.2 

type Iterator interface {
	// RequireKey checks set key requirement.
	RequireKey() bool
	// SetKey sets the key value and inspector to hidden context.
	SetKey(val any, ins Inspector)
	// SetVal sets the value and inspector to context.
	SetVal(val any, ins Inspector)
	// Iterate performs the iteration.
	Iterate() LoopCtl
}

Iterator signature.

type Logger 

type Logger interface {
	Print(v ...any)
	Println(v ...any)
}

Logger interface.

type LoopCtl 

type LoopCtl int

LoopCtl describes loop control type. 

const (
	LoopCtlNone LoopCtl = iota
	LoopCtlBrk
	LoopCtlCnt
)

type Op 

type Op int

Op describes operation type. 

const (
	OpUnk Op = iota
	OpEq
	OpNq
	OpGt
	OpGtq
	OpLt
	OpLtq
	OpInc
	OpDec
)

type ReflectInspector 

type ReflectInspector struct {
	BaseInspector
}

func (ReflectInspector) Capacity added in v1.4.2 

func (i ReflectInspector) Capacity(_ any, _ *int, _ ...string) error

func (ReflectInspector) Compare added in v1.4.2 

func (i ReflectInspector) Compare(_ any, _ Op, _ string, _ *bool, _ ...string) error

func (ReflectInspector) Copy 

func (i ReflectInspector) Copy(x any) (any, error)

func (ReflectInspector) CopyTo 

func (i ReflectInspector) CopyTo(_, _ any, _ AccumulativeBuffer) error

func (ReflectInspector) DeepEqual 

func (i ReflectInspector) DeepEqual(l, r any) bool

func (ReflectInspector) DeepEqualWithOptions 

func (i ReflectInspector) DeepEqualWithOptions(l, r any, _ *DEQOptions) bool

func (ReflectInspector) Get 

func (i ReflectInspector) Get(src any, path ...string) (any, error)

func (ReflectInspector) GetTo 

func (i ReflectInspector) GetTo(src any, buf *any, path ...string) error

func (ReflectInspector) Length added in v1.4.2 

func (i ReflectInspector) Length(_ any, _ *int, _ ...string) error

func (ReflectInspector) Loop 

func (i ReflectInspector) Loop(_ any, _ Iterator, _ *[]byte, _ ...string) (err error)

func (ReflectInspector) Reset 

func (i ReflectInspector) Reset(_ any) error

func (ReflectInspector) Set 

func (i ReflectInspector) Set(_, _ any, _ ...string) error

func (ReflectInspector) SetWithBuffer added in v1.4.2 

func (i ReflectInspector) SetWithBuffer(_, _ any, _ AccumulativeBuffer, _ ...string) error

func (ReflectInspector) TypeName 

func (i ReflectInspector) TypeName() string

func (ReflectInspector) Unmarshal 

func (i ReflectInspector) Unmarshal(p []byte, typ Encoding) (any, error)

type StaticInspector 

type StaticInspector struct {
	BaseInspector
}

func (StaticInspector) Capacity added in v1.4.2 

func (i StaticInspector) Capacity(x any, result *int, _ ...string) error

func (StaticInspector) Compare added in v1.4.2 

func (i StaticInspector) Compare(src any, cond Op, right string, result *bool, _ ...string) error

func (StaticInspector) Copy 

func (i StaticInspector) Copy(x any) (dst any, err error)

func (StaticInspector) CopyTo 

func (i StaticInspector) CopyTo(src, dst any, buf AccumulativeBuffer) error

func (StaticInspector) DeepEqual 

func (i StaticInspector) DeepEqual(l, r any) bool

func (StaticInspector) DeepEqualWithOptions 

func (i StaticInspector) DeepEqualWithOptions(l, r any, _ *DEQOptions) bool

func (StaticInspector) Get 

func (i StaticInspector) Get(src any, _ ...string) (any, error)

func (StaticInspector) GetTo 

func (i StaticInspector) GetTo(src any, buf *any, _ ...string) error

func (StaticInspector) Length added in v1.4.2 

func (i StaticInspector) Length(x any, result *int, _ ...string) error

func (StaticInspector) Loop 

func (i StaticInspector) Loop(_ any, _ Iterator, _ *[]byte, _ ...string) error

func (StaticInspector) Reset 

func (i StaticInspector) Reset(x any) error

func (StaticInspector) Set 

func (i StaticInspector) Set(_, _ any, _ ...string) error

func (StaticInspector) SetWithBuffer added in v1.4.2 

func (i StaticInspector) SetWithBuffer(_, _ any, _ AccumulativeBuffer, _ ...string) error

func (StaticInspector) TypeName 

func (i StaticInspector) TypeName() string

func (StaticInspector) Unmarshal 

func (i StaticInspector) Unmarshal(p []byte, typ Encoding) (any, error)

type StrConv 

type StrConv struct {
	// Code snippet.
	Snippet string
	// Imports list.
	Import []string
}

StrConv describes string to X conversion object.

type StringAnyMapInspector added in v1.4.4 

type StringAnyMapInspector struct {
	BaseInspector
}

func (StringAnyMapInspector) Capacity added in v1.4.4 

func (i StringAnyMapInspector) Capacity(x any, result *int, path ...string) error

func (StringAnyMapInspector) Compare added in v1.4.4 

func (i StringAnyMapInspector) Compare(src any, cond Op, right string, result *bool, path ...string) (err error)

func (StringAnyMapInspector) Copy added in v1.4.4 

func (i StringAnyMapInspector) Copy(x any) (dst any, err error)

func (StringAnyMapInspector) CopyTo added in v1.4.4 

func (i StringAnyMapInspector) CopyTo(src, dst any, buf AccumulativeBuffer) (err error)

func (StringAnyMapInspector) DeepEqual added in v1.4.4 

func (i StringAnyMapInspector) DeepEqual(l, r any) bool

func (StringAnyMapInspector) DeepEqualWithOptions added in v1.4.4 

func (i StringAnyMapInspector) DeepEqualWithOptions(l, r any, opts *DEQOptions) bool

func (StringAnyMapInspector) Get added in v1.4.4 

func (i StringAnyMapInspector) Get(src any, path ...string) (any, error)

func (StringAnyMapInspector) GetTo added in v1.4.4 

func (i StringAnyMapInspector) GetTo(src any, buf *any, path ...string) error

func (StringAnyMapInspector) Length added in v1.4.4 

func (i StringAnyMapInspector) Length(x any, result *int, path ...string) error

func (StringAnyMapInspector) Loop added in v1.4.4 

func (i StringAnyMapInspector) Loop(src any, it Iterator, buf *[]byte, path ...string) error

func (StringAnyMapInspector) Reset added in v1.4.4 

func (i StringAnyMapInspector) Reset(x any) error

func (StringAnyMapInspector) Set added in v1.4.4 

func (i StringAnyMapInspector) Set(dst, value any, path ...string) error

func (StringAnyMapInspector) SetWithBuffer added in v1.4.4 

func (i StringAnyMapInspector) SetWithBuffer(dst, value any, buf AccumulativeBuffer, path ...string) (err error)

func (StringAnyMapInspector) TypeName added in v1.4.4 

func (i StringAnyMapInspector) TypeName() string

func (StringAnyMapInspector) Unmarshal added in v1.4.4 

func (i StringAnyMapInspector) Unmarshal(p []byte, typ Encoding) (any, error)

type StringsInspector added in v1.4.2 

type StringsInspector struct{}

StringsInspector is a built-in inspector for []string/[][]byte types.

func (StringsInspector) Capacity added in v1.4.2 

func (i StringsInspector) Capacity(src any, result *int, path ...string) error

func (StringsInspector) Compare added in v1.4.2 

func (i StringsInspector) Compare(src any, cond Op, right string, result *bool, path ...string) error

func (StringsInspector) Copy added in v1.4.2 

func (i StringsInspector) Copy(x any) (any, error)

func (StringsInspector) CopyTo added in v1.4.2 

func (i StringsInspector) CopyTo(src, dst any, buf AccumulativeBuffer) error

func (StringsInspector) DeepEqual added in v1.4.2 

func (i StringsInspector) DeepEqual(l, r any) bool

func (StringsInspector) DeepEqualWithOptions added in v1.4.2 

func (i StringsInspector) DeepEqualWithOptions(l, r any, _ *DEQOptions) bool

func (StringsInspector) Get added in v1.4.2 

func (i StringsInspector) Get(src any, path ...string) (any, error)

func (StringsInspector) GetTo added in v1.4.2 

func (i StringsInspector) GetTo(src any, buf *any, path ...string) error

func (StringsInspector) Length added in v1.4.2 

func (i StringsInspector) Length(src any, result *int, path ...string) error

func (StringsInspector) Loop added in v1.4.2 

func (i StringsInspector) Loop(src any, l Iterator, buf *[]byte, path ...string) error

func (StringsInspector) Reset added in v1.4.2 

func (i StringsInspector) Reset(x any) error

func (StringsInspector) Set added in v1.4.2 

func (i StringsInspector) Set(dst, value any, path ...string) error

func (StringsInspector) SetWithBuffer added in v1.4.2 

func (i StringsInspector) SetWithBuffer(dst, value any, buf AccumulativeBuffer, path ...string) error

func (StringsInspector) TypeName added in v1.4.2 

func (i StringsInspector) TypeName() string

func (StringsInspector) Unmarshal added in v1.4.2 

func (i StringsInspector) Unmarshal(p []byte, typ Encoding) (any, error)

type Target added in v1.4.3 

type Target int
const (
	TargetPackage Target = iota
	TargetDirectory
	TargetFile
)

Source Files

View all Source files

Directories

Path Synopsis
inspc module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.