dyntpl

package module
v1.1.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 10, 2023 License: MIT Imports: 19 Imported by: 2

README

Dynamic templates

Dynamic replacement for quicktemplate template engine.

Retrospective

We're used for a long time quicktemplate for building JSON to interchange data between microservices on high-load project, and we were happy. But now we need to be able to change existing templates or add new templates on the fly. Unfortunately quicktemplates doesn't support this and this package was developed as replacement.

It reproduces many of qtpl features and syntax.

How it works

The biggest problem during development was how to get data from arbitrary structure without using reflection, since reflect package produces a lot of allocations by design and is extremely slowly in general.

To solve that problem was developed inspector framework. It takes as argument path to the package with structures signatures and build an exact primitive methods to get data of any fields in them, loop over fields that support loops, etc...

You may check example of inspectors in subdirectory testobj_ins that represents testing structures in testobj.

Usage

The typical usage of dyntpl looks like this:

package main

import (
    "bytes"

    "github.com/koykov/dyntpl"
    "path/to/inspector_lib_ins"
)

var (
    // Test data.
    data = &Data{
        // ...
    }

    // Template code.
    tplData = []byte(`...`)
)

func init() {
    // Parse the template and register it.
    tree, _ := dyntpl.Parse(tplData, false)
    dyntpl.RegisterTpl("tplData", tree)
}

func main() {
    // Prepare output buffer
    buf := bytes.Buffer{}
    // Prepare dyntpl context.
    ctx := dyntpl.AcquireCtx()
    ctx.Set("data", data, &inspector_lib_ins.DataInspector{})
    // Execute the template and write result to buf.
    _ = dyntpl.Write(&buf, "tplData", ctx)
    // Use result as buf.Bytes() or buf.String() ...
    // Release context.
    dyntpl.ReleaseCtx(ctx)
}

Content of init() function may be moved to scheduler and periodically take fresh template code from source data, e.g. DB table and update it on the fly.

Content of main() function is how to use dyntpl in general way. Of course, byte buffer should take from the pool.

Benchmarks

See bench.md for result of internal benchmarks.

Highly recommend to check *_test.go files in the project, since them contains a lot of typical language constructions that supports this engine.

See versus/dyntpl for comparison benchmarks with quicktemplate and native marshaler/template.

As you can see, dyntpl in ~3-4 times slower than quicktemplates. That is a cost for dynamics. There is no way to write template engine that will faster than native Go code.

Syntax

Print

The most general syntax construction is printing a variable or structure field:

This is a simple statis variable: {%= var0 %}
This is a field of struct: {%= obj.Parent.Name %}

Construction {%= ... %} prints data as is, independent of its type.

There are special directives before = that modifies output before printing:

  • h - HTML escape.
  • a - HTML attribute escape.
  • j - JSON escape.
  • q - JSON quote.
  • J - JS escape.
  • u - URL encode.
  • l - Link escape.
  • c - CSS escape.
  • f.<num> - float with precision, example: {%f.3= 3.1415 %} will output 3.141.
  • F.<num> - ceil rounded float with precision, example: {%F.3= 3.1415 %} will output 3.142.

Note, that none of these directives doesn't apply by default. It's your responsibility to controls what and where you print.

All directives supports multipliers, like {%jj= ... %}, {%uu= ... %}, {%uuu= ... %}, ...

For example, the following instruction {%uu= someUrl %} will print double url-encoded value of someUrl.

Also, you may combine directives, eg {%Ja= var1 %}. In this example JS escape and HTML attribute escape will apply consecutively before output var1.

Print construction supports prefix and suffix attributes, it may be handy when you print HTML or XML:

<ul>
{%= var prefix <li> suffix </li> %}
</ul>

Prefix and suffix will print only if var isn't empty. Prefix/suffix has shorthands pfx and sfx.

Also print supports data modifiers. They calls typically for any template languages:

Name: {%= obj.Name|default("anonymous") %}

and may contains variadic list of arguments or doesn't contain them at all. See the full list of built-in modifiers in init.go (calls of RegisterModFn()). You may register your own modifiers, see section Modifier helpers.

Conditions

Conditions in dyntpl is pretty simple and supports only two types of record:

  • {% if leftVar [=|!=|>|>=|<|<=] rightVar %}...{% endif %}
  • {% if conditionHelper(var0, obj.Name, "foo") %}...{% endif %}

First type is for the simplest case, like:

{% if user.Id == 0 %}
You should <a href="#">log in</a>.
{% endif %}

Left side or right side or both may be a variable. But you can't specify a condition with static values on both sides, since it's senseless.

Second type of condition is for more complex conditions when any side of condition should contain Go code, like:

Welcome, {% if len(user.Name) > 0 %}{%= user.Name %}{% else %}anonymous{%endif%}!

Dyntpl can't handle that kind of records, but it supports special functions that may make a decision is given args suitable or not and return true/false. See the full list of built-in condition helpers in init.go (calls of RegisterCondFn). Of course, you can register your own handlers to implement your logic.

For multiple conditions you can use switch statement, example 1:

<item type="{% switch item.Type %}
{% case 0 %}
    deny
{% case 1 %}
    allow
{% case 2 %}
    allow-by-permission
{% default %}
    unknown
{% endswitch %}">foo</item>

, example 2:

<item type="{% switch %}
{% case item.Type == 0 %}
    deny
{% case item.Type == 1 %}
    allow
{% case item.Type == 2 %}
    allow-by-permission
{% default %}
    unknown
{% endswitch %}">foo</item>

Switch can handle only primitive cases, condition helpers doesn't support.

Loops

Dyntpl supports both types of loops:

  • conditional loop from three components separated by semicolon, like {% for i:=0; i<5; i++ %}...{% endfor %}
  • range-loop, like {% for k, v := range obj.Items %}...{% endfor %}

Edge cases like for k < 2000 {...} or for ; i < 10 ; {...} isn't supported. Also, you can't make infinite loop by using for {...}.

There is a special attribute separator that made special to build JSON output. Example of use:

[
  {% for _, a := range user.History separator , %}
    {
      "id": {%q= a.Id %},
      "date": {%q= a.Date %},
      "comment": {%q= a.Note %}
    }
  {% endfor %}
]

The output that will produced:

[
  {"id":1, "date": "2020-01-01", "comment": "success"},
  {"id":2, "date": "2020-01-01", "comment": "failed"},
  {"id":3, "date": "2020-01-01", "comment": "rejected"}
]

As you see, commas between 2nd and last elements was added by dyntpl without any additional handling like ...{% if i>0 %},{% endif %}{% endfor %}. Separator has shorthand variant sep.

Include sub-templates

Just call {% include subTplID %} (example {% include sidebar/right %}) to render and include output of that template inside current template. Sub-template will use parent template's context to access the data.

Modifier helpers

Modifiers is a special functions that may perform modifications over the data during print. These function have signature:

func(ctx *Ctx, buf *any, val any, args []any) error

and should be registered using function dyntpl.RegisterModFn(). See init.go for examples. See mod.go for explanation of arguments.

Modifiers calls using pipeline symbol after a variable, example: {%= var0|default(0) %}.

You may specify a sequence of modifiers: {%= var0|roundPrec(4)|default(1) %}.

Condition helpers

If you want to make a condition more complex than simple condition, you may declare a special function with signature:

func(ctx *Ctx, args []any) bool

and register it using function dyntpl.RegisterCondFn(). See init.go for examples. See cond.go for explanation of arguments.

After declaring and registering you can use the helper in conditions:

{% if <condFnName>(var0, var1, "static val", 0, 15.234) %}...{% endif %}

Function will make a decision according arguments you take and will return true or false.

Bound tags

Dyntpl support special tags to escape/quote the output. Currently, allows three types:

  • {% jsonquote %}...{% endjsonquote %} apply JSON escape for all text data.
  • {% htmlescape %}...{% endhtmlescape %} apply HTML escape.
  • {% urlencode %}...{% endurlencode %} URL encode all text data.

Note, these tags escapes only text data inside. All variables should be escaped using corresponding modifiers. Example:

{"key": "{% jsonquote %}Lorem ipsum "dolor sit amet", {%j= var0 %}.{%endjsonquote%}"}

Here, {% end/jsonquote %} applies only for text data Lorem ipsum "dolor sit amet",, whereas var0 prints using JSON-escape printing prefix.

{% end/htmlescape %} and {% end/urlencode %} works the same.

I18n

Internationalization support provides by i18n package.

I18n must be enabled on context level using method ctx.I18n() before start templating.

For simple translate use function template or shorthand t:

{%= t("key", "default value", {"!placeholder0": "replacement", "!placeholder1": object.Label, ...}) %}

You may omit default value and replacements, only first argument is required.

For plural translation use function translatePlural or shorthand tp:

{%= tp("key", "default value", 15, {...}) %}

Third argument is a count for a plural formula. It's required as a key argument.

Documentation

Index

Constants

View Source 
const (
	TypeRaw       Type = 0
	TypeTpl       Type = 1
	TypeCond      Type = 2
	TypeCondOK    Type = 3
	TypeCondTrue  Type = 4
	TypeCondFalse Type = 5
	TypeLoopRange Type = 6
	TypeLoopCount Type = 7
	TypeBreak     Type = 8
	TypeLBreak    Type = 9
	TypeContinue  Type = 10
	TypeCtx       Type = 11
	TypeCounter   Type = 12
	TypeSwitch    Type = 13
	TypeCase      Type = 14
	TypeDefault   Type = 15
	TypeDiv       Type = 16
	TypeJsonQ     Type = 17
	TypeEndJsonQ  Type = 18
	TypeHtmlE     Type = 19
	TypeEndHtmlE  Type = 20
	TypeUrlEnc    Type = 21
	TypeEndUrlEnc Type = 22
	TypeInclude   Type = 23
	TypeLocale    Type = 24
	TypeExit      Type = 99

	// Must be in sync with inspector.Op type.
	OpUnk Op = 0
	OpEq  Op = 1
	OpNq  Op = 2
	OpGt  Op = 3
	OpGtq Op = 4
	OpLt  Op = 5
	OpLtq Op = 6
	OpInc Op = 7
	OpDec Op = 8
)

Variables

View Source 
var (
	ErrUnexpectedEOF = errors.New("unexpected end of file: control structure couldn't be closed")
	ErrUnknownCtl    = errors.New("unknown ctl")

	ErrSenselessCond   = errors.New("comparison of two static args")
	ErrCondHlpNotFound = errors.New("condition helper not found")

	ErrTplNotFound = errors.New("template not found")
	ErrInterrupt   = errors.New("tpl processing interrupted")
	ErrModNoArgs   = errors.New("empty arguments list")
	ErrModPoorArgs = errors.New("arguments list is too small")
	ErrModNoStr    = errors.New("argument is not string or bytes")

	ErrWrongLoopLim  = errors.New("wrong count loop limit argument")
	ErrWrongLoopCond = errors.New("wrong loop condition operation")
	ErrWrongLoopOp   = errors.New("wrong loop operation")
	ErrBreakLoop     = errors.New("break loop")
	ErrLBreakLoop    = errors.New("lazybreak loop")
	ErrContLoop      = errors.New("continue loop")

	ErrUnknownPool = errors.New("unknown pool")
)

Functions

func ConvBool 

func ConvBool(val any) (b bool, ok bool)

ConvBool tries to convert value ti boolean.

func ConvBytes 

func ConvBytes(val any) (b []byte, ok bool)

ConvBytes tries to convert value to bytes.

func ConvBytesSlice 

func ConvBytesSlice(val any) (b [][]byte, ok bool)

ConvBytesSlice tries to convert value to slice of bytes.

func ConvFloat 

func ConvFloat(val any) (f float64, ok bool)

ConvFloat tries to convert value to float.

func ConvInt 

func ConvInt(val any) (i int64, ok bool)

ConvInt tries to convert value to integer.

func ConvStr 

func ConvStr(val any) (s string, ok bool)

ConvStr tries to convert value to string.

func ConvStrSlice 

func ConvStrSlice(val any) (s []string, ok bool)

ConvStrSlice tries to convert value to string slice.

func ConvUint 

func ConvUint(val any) (u uint64, ok bool)

ConvUint tries to convert value to uint.

func EmptyCheck 

func EmptyCheck(ctx *Ctx, val any) bool

EmptyCheck tries to apply all known helpers over the val.

First acceptable helper will break next attempts.

func EmptyCheckBool 

func EmptyCheckBool(_ *Ctx, val any) bool

EmptyCheckBool checks is val is an empty bool.

func EmptyCheckBytes 

func EmptyCheckBytes(_ *Ctx, val any) bool

EmptyCheckBytes checks is val is an empty bytes array.

func EmptyCheckBytesSlice 

func EmptyCheckBytesSlice(_ *Ctx, val any) bool

EmptyCheckBytesSlice checks is val is an empty slice of bytes.

func EmptyCheckFloat 

func EmptyCheckFloat(_ *Ctx, val any) bool

EmptyCheckFloat checks is val is an empty float number.

func EmptyCheckInt 

func EmptyCheckInt(_ *Ctx, val any) bool

EmptyCheckInt checks is val is an empty integer.

func EmptyCheckStr 

func EmptyCheckStr(_ *Ctx, val any) bool

EmptyCheckStr checks is val is an empty string.

func EmptyCheckStrSlice 

func EmptyCheckStrSlice(_ *Ctx, val any) bool

EmptyCheckStrSlice checks is val is an empty slice of strings.

func EmptyCheckUint 

func EmptyCheckUint(_ *Ctx, val any) bool

EmptyCheckUint checks is val is an empty unsigned integer.

func GetInsByVarName 

func GetInsByVarName(varName string) (inspector.Inspector, bool)

GetInsByVarName gets inspector by variable name.

func GetInspector 

func GetInspector(varName, name string) (ins inspector.Inspector, err error)

GetInspector gets inspector by both variable name or inspector name.

func RegisterCondFn 

func RegisterCondFn(name string, cond CondFn)

RegisterCondFn registers new condition helper in registry.

func RegisterCondOKFn 

func RegisterCondOKFn(name string, cond CondOKFn)

RegisterCondOKFn registers new condition-OK helper in registry.

func RegisterEmptyCheckFn 

func RegisterEmptyCheckFn(name string, cond EmptyCheckFn)

RegisterEmptyCheckFn registers new empty check helper.

func RegisterModFn 

func RegisterModFn(name, alias string, mod ModFn)

RegisterModFn registers new modifier function.

func RegisterPool added in v1.1.6 

func RegisterPool(key string, pool Pool) error

RegisterPool adds new internal pool to the registry by given key.

func RegisterTpl 

func RegisterTpl(id int, key string, tree *Tree)

RegisterTpl saves template by ID and key in the registry.

You may use to access to the template both ID or key. This function can be used in any time to register new templates or overwrite existing to provide dynamics.

func RegisterTplID 

func RegisterTplID(id int, tree *Tree)

RegisterTplID saves template using only ID.

See RegisterTpl().

func RegisterTplKey 

func RegisterTplKey(key string, tree *Tree)

RegisterTplKey saves template using only key.

See RegisterTpl().

func RegisterVarInsPair 

func RegisterVarInsPair(varName string, ins inspector.Inspector)

RegisterVarInsPair registers new variable-inspector pair.

func ReleaseCtx 

func ReleaseCtx(ctx *Ctx)

ReleaseCtx puts object back to default pool.

func Render 

func Render(key string, ctx *Ctx) ([]byte, error)

Render template with given key according given context.

See Write(). Recommend to use Write() together with byte buffer pool to avoid redundant allocations.

func RenderByID 

func RenderByID(id int, ctx *Ctx) ([]byte, error)

RenderByID renders template with given ID according context.

See WriteByID(). Recommend to use WriteByID() together with byte buffer pool to avoid redundant allocations.

func RenderFallback 

func RenderFallback(key, fbKey string, ctx *Ctx) ([]byte, error)

RenderFallback renders template using one of keys: key or fallback key.

See WriteFallback(). Using this func you can handle cases when some objects have custom templates and all other should use default templates. Example: template registry: * tplUser * tplUser-15 user object with id 15 Call of dyntpl.RenderFallback("tplUser-15", "tplUser", ctx) will take template tplUser-15 from registry. In other case, for user #4: call of dyntpl.WriteFallback("tplUser-4", "tplUser", ctx) will take default template tplUser from registry. Recommend to user WriteFallback().

func Write 

func Write(w io.Writer, key string, ctx *Ctx) (err error)

Write template with given key to given writer object.

Using this function together with byte buffer pool reduces allocations.

func WriteByID 

func WriteByID(w io.Writer, id int, ctx *Ctx) (err error)

WriteByID writes template with given ID to given writer object.

Using this function together with byte buffer pool reduces allocations.

func WriteFallback 

func WriteFallback(w io.Writer, key, fbKey string, ctx *Ctx) (err error)

WriteFallback writes template using fallback key logic and write result to writer object.

See RenderFallback(). Use this function together with byte buffer pool to reduce allocations.

Types

type CondFn 

type CondFn func(ctx *Ctx, args []any) bool

CondFn describes helper func signature.

func GetCondFn 

func GetCondFn(name string) *CondFn

GetCondFn returns condition helper from the registry.

type CondOKFn 

type CondOKFn func(ctx *Ctx, v *any, ok *bool, args []any)

CondOKFn describes helper func signature.

func GetCondOKFn 

func GetCondOKFn(name string) *CondOKFn

GetCondOKFn returns condition-OK helper from the registry.

type Ctx 

type Ctx struct {

	// External buffers to use in modifier and condition helpers.
	BufAcc bytebuf.AccumulativeBuf
	// todo remove as unused later
	Buf, Buf1, Buf2 bytebuf.ChainBuf

	BufB bool
	BufI int64
	BufU uint64
	BufF float64

	Err error
	// contains filtered or unexported fields
}

Ctx is a context object. Contains list of variables available to inspect. In addition, has buffers to help develop new helpers without allocations.

func AcquireCtx 

func AcquireCtx() *Ctx

AcquireCtx gets object from the default context pool.

func NewCtx 

func NewCtx() *Ctx

NewCtx makes new context object.

func (*Ctx) AcquireFrom added in v1.1.6 

func (ctx *Ctx) AcquireFrom(pool string) (any, error)

AcquireFrom receives new variable from given pool and register it to return batch after finish template processing.

func (*Ctx) BufModOut 

func (ctx *Ctx) BufModOut(buf *any, p []byte)

BufModOut buffers mod output bytes.

func (*Ctx) BufModStrOut 

func (ctx *Ctx) BufModStrOut(buf *any, s string)

BufModStrOut buffers mod output string.

func (*Ctx) Defer added in v1.1.6 

func (ctx *Ctx) Defer(fn func() error)

Defer registers new deferred function.

Function will call after finishing template. todo: find a way how to avoid closure allocation.

func (*Ctx) Get 

func (ctx *Ctx) Get(path string) any

Get arbitrary value from the context by path.

See Ctx.get(). Path syntax: <ctxVrName>[.<Field>[.<NestedField0>[....<NestedFieldN>]]] Examples: * user.Bio.Birthday * staticVar

func (*Ctx) GetCounter 

func (ctx *Ctx) GetCounter(key string) int

GetCounter gets int counter value.

func (*Ctx) I18n 

func (ctx *Ctx) I18n(locale string, db *i18n.DB)

I18n sets i18n locale and database.

func (*Ctx) Reset 

func (ctx *Ctx) Reset()

Reset the context.

Made to use together with pools.

func (*Ctx) Set 

func (ctx *Ctx) Set(key string, val any, ins inspector.Inspector)

Set the variable to context. Inspector ins should be corresponded to variable val.

func (*Ctx) SetBytes 

func (ctx *Ctx) SetBytes(key string, val []byte)

SetBytes sets bytes as static variable.

See Ctx.Set(). This is a special case to improve speed.

func (*Ctx) SetCounter 

func (ctx *Ctx) SetCounter(key string, val int)

SetCounter sets int counter as static variable.

See Ctx.Set(). This is a special case to support counters.

func (*Ctx) SetStatic 

func (ctx *Ctx) SetStatic(key string, val any)

SetStatic sets static variable to context.

func (*Ctx) SetString 

func (ctx *Ctx) SetString(key, val string)

SetString sets string as static variable.

type CtxPool 

type CtxPool struct {
	// contains filtered or unexported fields
}

CtxPool is a context pool. 

var CP CtxPool

CP is a default instance of context pool. You may use it directly as dyntpl.CP.Get()/Put() or using functions AcquireCtx()/ReleaseCtx().

func (*CtxPool) Get 

func (p *CtxPool) Get() *Ctx

Get context object from the pool or make new object if pool is empty.

func (*CtxPool) Put 

func (p *CtxPool) Put(ctx *Ctx)

Put the object to the pool.

type EmptyCheckFn 

type EmptyCheckFn func(ctx *Ctx, val any) bool

EmptyCheckFn describes empty check helper func signature.

func GetEmptyCheckFn 

func GetEmptyCheckFn(name string) *EmptyCheckFn

GetEmptyCheckFn gets empty check helper from the registry.

type ModFn 

type ModFn func(ctx *Ctx, buf *any, val any, args []any) error

ModFn describes signature of the modifier functions.

Arguments description: * ctx provides access to additional variables and various buffers to reduce allocations. * buf is a storage for final result after finishing modifier work. * val is a left side variable that preceded to call of modifier func, example: {%= val|mod(...) %} * args is a list of all arguments listed on modifier call.

func GetModFn 

func GetModFn(name string) *ModFn

GetModFn gets modifier from the registry.

type Node 

type Node struct {
	// contains filtered or unexported fields
}

Node is a description of template part. Every piece of the template, beginning from static text and finishing of complex structures (switch, loop, ...) Represents by this type.

type Op 

type Op int

Op is a type of the operation in conditions and loops.

func (Op) String 

func (o Op) String() string

String view of the opertion.

func (Op) Swap 

func (o Op) Swap() Op

Swap inverts itself.

type Parser 

type Parser struct {
	// contains filtered or unexported fields
}

Parser object.

type Pool added in v1.1.6 

type Pool interface {
	Get() any
	Put(any)
	// Reset cleanups data before putting to the pool.
	Reset(any)
}

Pool represents internal pool. In addition to native sync.Pool requires Reset() method.

type RangeLoop 

type RangeLoop struct {
	// contains filtered or unexported fields
}

RangeLoop is a object that injects to inspector to perform range loop execution.

func NewRangeLoop 

func NewRangeLoop(node Node, tpl *Tpl, ctx *Ctx, w io.Writer) *RangeLoop

NewRangeLoop makes new RL.

func (*RangeLoop) Iterate 

func (rl *RangeLoop) Iterate() inspector.LoopCtl

Iterate performs the iteration.

func (*RangeLoop) RequireKey 

func (rl *RangeLoop) RequireKey() bool

RequireKey checks if node requires a key to store in the context.

func (*RangeLoop) Reset 

func (rl *RangeLoop) Reset()

Reset clears all data in the list of RL.

func (*RangeLoop) SetKey 

func (rl *RangeLoop) SetKey(val any, ins inspector.Inspector)

SetKey saves key to the context.

func (*RangeLoop) SetVal 

func (rl *RangeLoop) SetVal(val any, ins inspector.Inspector)

SetVal saves value to the context.

type Tpl 

type Tpl struct {
	ID  int
	Key string
	// contains filtered or unexported fields
}

Tpl is a main template object. Template contains only parsed template and evaluation logic. All temporary and intermediate data should be store in context object to make using of templates thread-safe.

type Tree 

type Tree struct {
	// contains filtered or unexported fields
}

Tree structure that represents parsed template as list of nodes with childrens.

func Parse 

func Parse(tpl []byte, keepFmt bool) (tree *Tree, err error)

Parse initializes parser and parse the template body.

func ParseFile 

func ParseFile(fileName string, keepFmt bool) (tree *Tree, err error)

ParseFile initializes parser and parse file contents.

func (*Tree) HumanReadable 

func (t *Tree) HumanReadable() []byte

HumanReadable builds human readable view of the tree (currently in XML format).

type Type 

type Type int

Type of the node.

func (Type) String 

func (typ Type) String() string

String view of the type.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.