W2

README

Codegen for Web Framework

This framework aims to reduce common task that happened during backend API development which are:

1. generating ORM and migrating model
2. generate for other type of input and output encoding/presentation format (JSON, MsgPack, YAML, JSON5, GraphQL, etc), command line, or different transport (REST, WebSocket, gRPC, CLI, etc)
3. generate API documentation and API clients for different programming languages
4. testable business logic (by untangling domain/ and decorator/adapter/presentation)

This codegen currently tied to Tarantool (for OLTP use cases), Clickhouse (for OLAP use cases), and Fiber (for web routes) and only tested for generating json using REST/GraphQL transport and plain JS API docs. But you can always create a new codegen for other databases or other web frameworks or other API clients.

FAQ

Why is it tied to Tarantool?

• because it's currently the fastest SQL-OLTP database that also works as in-memory cache, so I can do integration testing without noticable overhead (can do ~200K writes per second), your first scalability issue would mostly always be the database, unless you're using slow untyped programming language.

Why is it tied to Clickhouse?

• because it's currently the fastest OLAP database (can do ~600K inserts per second).

Why is it tied to Fiber?

• because it's currently the fastest minimalist golang framework, ones that are faster but not more painful to use is C#'s asp.net core

Can I replace the components/database (so instead of using Tarantool/Clickhouse/Fiber)?

• yes, write your own codegen for database and api generator, it's possible

Why using github.com/goccy/go-json?

• because I couldn't find any other faster alternative (other than slower encoding/json) that can properly parse int64/uint64 (already tried jsoniter and easyjson, both give wrong result for {"id":"89388457092187654"} with json:"id,string" tag).

Why tying between business logic and data store (not using Repository pattern)?

• because we are rarely changing database product, like 90% of the time we'll never switch database since it's too costly (time consuming), if read part is the bottleneck we usually cache it (or precalculate or create static snapshot on every update), but if the write is the bottleneck, we usually shard it to different machine (either manual sharding, eg. A to L on node 1, K to Z on node 2, or automatic like with 2 level hash), even when we have to switch database, we usually have to do double-write first, which using interface{anything} totally have a very little benefit, compared to the cost (reduced agility when doing source code navigation, since you cannot jump properly to the declaration/implementation/usage if using interface), I still believe that only good time to use interface is for third party (to replace it with fakes for testing 3rd party that usually have a very slow I/O)
• because the data store itself are fast (in-mem) and can be tested using docker-compose or dockertest, no noticable difference between fake/mock and integration test with this database product.
• because we shouldn't separate between memory (data structure) and business logic (algorithm), since they are both the brain of our application, without memory there's nothing to think, without thinking/processing it's just uninformative raw data blob.
• because most entity/repository have 1 to 1 relationship with the persistent/data store adapter, so it would be a overengineering to split when you not yet need it (YAGNI).

How to scale out or load balance? you can use strategies in this blog

Usage

See example/ directory for minimum framework template, or if you want to do codegen manually:

1. create a test file 0_generator_test.go inside your domain/ project folder

package domain_test

import (
	"testing"
	`github.com/kokizzu/gotro/W2`
)

//go:generate go test -run=XXX -bench=Benchmark_Generate_WebApiRoutes_CliArgs
//go:generate go test -run=XXX -bench=Benchmark_Generate_SvelteApiDocs

func Benchmark_Generate_WebApiRoutes_CliArgs(b *testing.B) {
	W2.GenerateFiberAndCli(&WW.GeneratorConfig{
		ProjectName: PROJECT_NAME,
	})
	b.SkipNow()
}

func Benchmark_Generate_SvelteApiDocs(b *testing.B) {
	W2.GenerateApiDocs(&WW.GeneratorConfig{
		ProjectName: PROJECT_NAME,
	})
	b.SkipNow()
}

1. create a makefile to do the codegen

gen-route:
	cd domain ; rm -f *MSG.GEN.go 
	cd domain ; go test -bench=Benchmark_Generate_WebApiRoutes_CliArgs 0_generator_test.go
	cd domain ; cat *.go | grep '//go:generate ' | cut -d ' ' -f 2- | sh -x
	cd domain ; go test -bench=Benchmark_Generate_SvelteApiDocs 0_generator_test.go

1. run make gen-route

this would create few generated file:

main_cli_args.GEN.go --> cli arguments handler
main_restApi_routes.GEN.go --> used to generating fiber route handlers
svelte/src/pages/api.js --> used for generating API client

Example Generated API Docs

TODO

• censor fields from GraphQL codegen (eg. password, access tokens)
• add gRPC codegen

Documentation

Index

• Constants
• func GenerateApiDocs(c *GeneratorConfig)
• func GenerateFiberAndCli(c *GeneratorConfig)
• type CallList
  • func NewCallList(old *CallList) *CallList
  • func (c *CallList) Add(funName string) *CallList
  • func (c *CallList) AddCallParam(args []ast.Expr)
  • func (c *CallList) IncCallCount(funName string)
  • func (c *CallList) SortedKeys() []string
  • func (c *CallList) ZeroCallCount(funName string)
• type CallParam
  • func (p *CallParam) ToString() string
• type ErrMap
  • func (m *ErrMap) Add(pair ErrPair) *ErrMap
• type ErrPair
  • func (e ErrPair) String() string
• type GeneratorConfig
  • func (c *GeneratorConfig) ParseRoutes(parseModelAndCalls bool) *RoutesArgs
• type ModelArgs
  • func (r *ModelArgs) ParseModel(path string)
  • func (r *ModelArgs) Visit(n ast.Node) (w ast.Visitor)
• type RoutesArgs
  • func (r *RoutesArgs) IncStatisticsCalls(methodName string, param *CallParam)
  • func (r *RoutesArgs) ParseDomain(path string)
  • func (r *RoutesArgs) Visit(n ast.Node) ast.Visitor
  • func (r *RoutesArgs) WriteApiDocs(path string)
  • func (r *RoutesArgs) WriteCliArgs(path string)
  • func (r *RoutesArgs) WriteGraphql(path string)
  • func (r *RoutesArgs) WriteWebRoutes(path string)
• type StructField
  • func (f StructField) ApiComment() string
  • func (f StructField) LowerName() string

Constants

const WARNING = `
// WARNING: DO NOT edit this file, it would be overwritten by domain/0_generator_test.go
`

Functions

func GenerateApiDocs

func GenerateApiDocs(c *GeneratorConfig)

func GenerateFiberAndCli

func GenerateFiberAndCli(c *GeneratorConfig)

Types

type CallList

type CallList struct {
	List map[string]*CallParam
}

func NewCallList

func NewCallList(old *CallList) *CallList

func (*CallList) Add

func (c *CallList) Add(funName string) *CallList

func (*CallList) AddCallParam

func (c *CallList) AddCallParam(args []ast.Expr)

func (*CallList) IncCallCount

func (c *CallList) IncCallCount(funName string)

func (*CallList) SortedKeys

func (c *CallList) SortedKeys() []string

func (*CallList) ZeroCallCount

func (c *CallList) ZeroCallCount(funName string)

type CallParam

type CallParam struct {
	CallCount  int
	FirstParam string
}

func (*CallParam) ToString

func (p *CallParam) ToString() string

type ErrMap

type ErrMap struct {
	List map[string]ErrPair
}

func (*ErrMap) Add

func (m *ErrMap) Add(pair ErrPair) *ErrMap

type ErrPair

type ErrPair struct {
	Code int
	Msg  string
}

func (ErrPair) String

func (e ErrPair) String() string

type GeneratorConfig

type GeneratorConfig struct {
	ProjectName string // must equal go.mod header

	ModelPath string // model directory

	WebRoutesFile  string // fiber web route generated file
	WebGraphqlFile string // fiber /graphql generated handler
	CliArgsFile    string // cli args handler generated file
	ApiDocsFile    string // apidocs generated file

	GenGraphQl bool

	ThirdParties []string
}

func (*GeneratorConfig) ParseRoutes

func (c *GeneratorConfig) ParseRoutes(parseModelAndCalls bool) *RoutesArgs

type ModelArgs

type ModelArgs struct {
	// contains filtered or unexported fields
}

func (*ModelArgs) ParseModel

func (r *ModelArgs) ParseModel(path string)

func (*ModelArgs) Visit

func (r *ModelArgs) Visit(n ast.Node) (w ast.Visitor)

type RoutesArgs

type RoutesArgs struct {
	ProjectName  string
	ThirdParties []string
	// contains filtered or unexported fields
}

func (*RoutesArgs) IncStatisticsCalls

func (r *RoutesArgs) IncStatisticsCalls(methodName string, param *CallParam)

func (*RoutesArgs) ParseDomain

func (r *RoutesArgs) ParseDomain(path string)

func (*RoutesArgs) Visit

func (r *RoutesArgs) Visit(n ast.Node) ast.Visitor

func (*RoutesArgs) WriteApiDocs

func (r *RoutesArgs) WriteApiDocs(path string)

func (*RoutesArgs) WriteCliArgs

func (r *RoutesArgs) WriteCliArgs(path string)

func (*RoutesArgs) WriteGraphql

func (r *RoutesArgs) WriteGraphql(path string)

func (*RoutesArgs) WriteWebRoutes

func (r *RoutesArgs) WriteWebRoutes(path string)

type StructField

type StructField struct {
	Name    string
	Type    string
	Tags    string
	Comment string
	IsArray bool
	IsMap   bool
}

func (StructField) ApiComment

func (f StructField) ApiComment() string

func (StructField) LowerName

func (f StructField) LowerName() string