README
¶
English | 中文
CORE
Package core is a service container that elegantly bootstrap and coordinate twelve-factor apps in Go.
Background
The twelve-factor methodology has proven its worth over the years. Since its invention many fields in technology have changed, many among them are shining and exciting. In the age of Kubernetes, service mesh and serverless architectures, the twelve-factor methodology has not faded away, but rather has happened to be a good fit for nearly all of those powerful platforms.
Scaffolding a twelve-factor go app may not be a difficult task for experienced engineers, but certainly presents some challenges to juniors. For those who are capable of setting things up, there are still many decisions left to make, and choices to be agreed upon within the team.
Package core was created to bootstrap and coordinate such services.
Feature
Package core shares the common concerns of your application:
- Configuration management: env, flags, files, etc.
- Pluggable transports: HTTP, gRPC, etc.
- Dependency injection
- Job management: Cron, long-running, one-off commandline, etc.
- Events and Queues
- Metrics
- Distributed Tracing
- Database migrations and seedings
- Distributed transactions
- Leader election
Overview
Whatever the app is, the bootstrapping phase is roughly composed by:
-
Read the configuration from out of the binary. Namely, flags, environment variables, and/or configuration files.
-
Initialize dependencies. Databases, message queues, service discoveries, etc.
-
Define how to run the app. HTTP, RPC, command-lines, cronjobs, or more often mixed.
Package core abstracts those repeated steps, keeping them concise, portable yet explicit. Let's see the following snippet:
package main
import (
"context"
"net/http"
"github.com/DoNewsCode/core"
"github.com/DoNewsCode/core/observability"
"github.com/DoNewsCode/core/otgorm"
"github.com/gorilla/mux"
)
func main() {
// Phase One: create a core from a configuration file
c := core.New(core.WithYamlFile("config.yaml"))
// Phase two: bind dependencies
c.Provide(otgorm.Providers())
// Phase three: define service
c.AddModule(core.HttpFunc(func(router *mux.Router) {
router.HandleFunc("/", func(writer http.ResponseWriter, request *http.Request) {
writer.Write([]byte("hello world"))
})
}))
// Phase Four: run!
c.Serve(context.Background())
}
In a few lines, an HTTP service is bootstrapped in the style outlined above. It is simple, explicit, and to some extent, declarative.
The service demonstrated above uses an inline handler function to highlight the point. Normally, for real projects, we will use modules instead. The "module" in package Core's glossary is not necessarily a go module (though it can be). It is simply a group of services.
You may note that the HTTP service doesn't really consume the dependency. That's true.
Let's rewrite the HTTP service to consume the above dependencies.
package main
import (
"context"
"net/http"
"github.com/DoNewsCode/core"
"github.com/DoNewsCode/core/otgorm"
"github.com/DoNewsCode/core/srvhttp"
"github.com/gorilla/mux"
"gorm.io/gorm"
)
type User struct {
Id string
Name string
}
type Repository struct {
DB *gorm.DB
}
func (r Repository) Find(id string) (*User, error) {
var user User
if err := r.DB.First(&user, id).Error; err != nil {
return nil, err
}
return &user, nil
}
type Handler struct {
R Repository
}
func (h Handler) ServeHTTP(writer http.ResponseWriter, request *http.Request) {
encoder := srvhttp.NewResponseEncoder(writer)
encoder.Encode(h.R.Find(request.URL.Query().Get("id")))
}
type Module struct {
H Handler
}
func New(db *gorm.DB) Module {
return Module{Handler{Repository{db}}}
}
func (m Module) ProvideHTTP(router *mux.Router) {
router.Handle("/", m.H)
}
func main() {
// Phase One: create a core from a configuration file
c := core.New(core.WithYamlFile("config.yaml"))
// Phase two: bind dependencies
c.Provide(otgorm.Providers())
// Phase three: define service
c.AddModuleFunc(New)
// Phase four: run!
c.Serve(context.Background())
}
Phase three has been replaced by the c.AddModuleFunc(New)
. AddModuleFunc
populates the arguments to New
from dependency containers
and add the returned module instance to the internal module registry.
When c.Serve() is called, all registered modules will be scanned for implemented interfaces. The module in the example implements interface:
type HTTPProvider interface {
ProvideHTTP(router *mux.Router)
}
Therefore, the core knows this module wants to expose HTTP service and subsequently invokes the ProvideHTTP
with a router. You can register multiple modules, and each module can implement one or more services.
Now we have a fully workable project, with layers of handler, repository, and entity. Had this been a DDD workshop, we would be expanding the example even further.
That being said, let's redirect our attention to other goodies package core has offered:
-
Package core natively supports multiplexing modules. You could start you project as a monolith with multiple modules, and gradually migrate them into microservices.
-
Package core doesn't lock in transport or framework. For instance, you can use go kit to construct your services, and bring in transports like gRPC, AMPQ, thrift, etc. Non-network services like CLI and Cron are also supported.
-
Package core also babysits the services after initialization. The duty includes but not limited to distributed tracing, metrics exporting, error handling, event-dispatching, and leader election.
Be sure to checkout the documentation section to learn more.
Documentation
Design Principles
- No package global state.
- Promote dependency injection.
- Testable code.
- Minimalist interface design. Easy to decorate and replace.
- Work with the Go ecosystem rather than reinventing the wheel.
- End to end Context passing.
Non-Goals
- Tries to be a Spring, Laravel, or Ruby on Rails.
- Tries to care about service details.
- Tries to reimplement the functionality provided by modern platforms.
Suggested service framework
Documentation
¶
Overview ¶
Package core is a service container that elegantly bootstrap and coordinate twelve-factor apps in Go.
Checkout the README.md for an overview of this package.
Index ¶
- func NewServeModule(in serveIn) serveModule
- func ProvideAppName(conf contract.ConfigAccessor) contract.AppName
- func ProvideConfig(configStack []config.ProviderSet, configWatcher contract.ConfigWatcher) contract.ConfigAccessor
- func ProvideEnv(conf contract.ConfigAccessor) contract.Env
- func ProvideEventDispatcher(conf contract.ConfigAccessor) contract.Dispatcher
- func ProvideLogger(conf contract.ConfigAccessor, appName contract.AppName, env contract.Env) log.Logger
- func WithYamlFile(path string) (CoreOption, CoreOption)
- type AppNameProvider
- type C
- type ConfParser
- type ConfProvider
- type ConfigProvider
- type CoreOption
- func SetAppNameProvider(provider AppNameProvider) CoreOption
- func SetConfigProvider(provider ConfigProvider) CoreOption
- func SetDiProvider(provider DiProvider) CoreOption
- func SetEnvProvider(provider EnvProvider) CoreOption
- func SetEventDispatcherProvider(provider EventDispatcherProvider) CoreOption
- func SetLoggerProvider(provider LoggerProvider) CoreOption
- func WithConfigStack(provider ConfProvider, parser ConfParser) CoreOption
- func WithConfigWatcher(watcher contract.ConfigWatcher) CoreOption
- func WithInline(key, entry string) CoreOption
- type DiContainer
- type DiProvider
- type EnvProvider
- type EventDispatcherProvider
- type HttpFunc
- type LoggerProvider
- type OnGRPCServerShutdown
- type OnGRPCServerStart
- type OnHTTPServerShutdown
- type OnHTTPServerStart
Examples ¶
Constants ¶
Variables ¶
Functions ¶
func NewServeModule ¶
func NewServeModule(in serveIn) serveModule
func ProvideAppName ¶
func ProvideAppName(conf contract.ConfigAccessor) contract.AppName
ProvideAppName is the default AppNameProvider for package Core.
func ProvideConfig ¶
func ProvideConfig(configStack []config.ProviderSet, configWatcher contract.ConfigWatcher) contract.ConfigAccessor
ProvideConfig is the default ConfigProvider for package Core.
func ProvideEnv ¶
func ProvideEnv(conf contract.ConfigAccessor) contract.Env
ProvideEnv is the default EnvProvider for package Core.
func ProvideEventDispatcher ¶
func ProvideEventDispatcher(conf contract.ConfigAccessor) contract.Dispatcher
ProvideEventDispatcher is the default EventDispatcherProvider for package Core.
func ProvideLogger ¶
func ProvideLogger(conf contract.ConfigAccessor, appName contract.AppName, env contract.Env) log.Logger
ProvideLogger is the default LoggerProvider for package Core.
func WithYamlFile ¶
func WithYamlFile(path string) (CoreOption, CoreOption)
WithYamlFile is a two-in-one coreOption. It uses the configuration file as the source of configuration, and watches the change of that file for hot reloading.
Types ¶
type AppNameProvider ¶
type AppNameProvider func(conf contract.ConfigAccessor) contract.AppName
AppNameProvider provides the contract.AppName to the core.
type C ¶
type C struct { AppName contract.AppName Env contract.Env contract.ConfigAccessor logging.LevelLogger contract.Container contract.Dispatcher // contains filtered or unexported fields }
C stands for the core of the application. It contains service definitions and dependencies. C is mean to be used in the boostrap phase of the application. Do not pass C into services and use it as a service locator.
func Default ¶
func Default(opts ...CoreOption) *C
Default creates a core.C under its default state. Core dependencies are already provided, and the config module and serve module are bundled.
func (*C) AddModule ¶
func (c *C) AddModule(modules ...interface{})
AddModule adds one or more module(s) to the core. If any of the variadic arguments is an error, it would panic. This makes it easy to consume constructors directly, so instead of writing:
component, err := components.New() if err != nil { panic(err) } c.AddModule(component)
You can write:
c.AddModule(component.New())
A Module is a group of functionality. It must provide some runnable stuff: http handlers, grpc handlers, cron jobs, one-time command, etc.
func (*C) AddModuleFunc ¶
func (c *C) AddModuleFunc(constructor interface{})
AddModuleFunc add the module after Invoking its' constructor. Clean up functions and errors are handled automatically.
func (*C) Invoke ¶
func (c *C) Invoke(function interface{})
Invoke runs the given function after instantiating its dependencies. Any arguments that the function has are treated as its dependencies. The dependencies are instantiated in an unspecified order along with any dependencies that they might have. The function may return an error to indicate failure. The error will be returned to the caller as-is.
It internally calls uber's dig library. Consult dig's documentation for details. (https://pkg.go.dev/go.uber.org/dig)
func (*C) Provide ¶
Provide adds a dependencies provider to the core. Note the dependency provider must be a function in the form of:
func(foo Foo) Bar
where foo is the upstream dependency and Bar is the provided type. The order for providers doesn't matter. They are only executed lazily when the Invoke is called.
This method internally calls uber's dig library. Consult dig's documentation for details. (https://pkg.go.dev/go.uber.org/dig)
The difference is, core.Provide has been made to accommodate the convention from google/wire (https://github.com/google/wire). All "func()" returned by constructor are treated as clean up functions. It also respect the core's unique "di.Module" annotation.
func (*C) ProvideEssentials ¶
func (c *C) ProvideEssentials()
ProvideEssentials adds the default core dependencies to the core.
type ConfParser ¶
type ConfParser interface { Unmarshal([]byte) (map[string]interface{}, error) Marshal(map[string]interface{}) ([]byte, error) }
ConfParser models a parser for configuration. For example, yaml.Parser.
type ConfProvider ¶
ConfProvider models a configuration provider. For example, file.Provider.
type ConfigProvider ¶
type ConfigProvider func(configStack []config.ProviderSet, configWatcher contract.ConfigWatcher) contract.ConfigAccessor
ConfigProvider provides contract.ConfigAccessor to the core.
type CoreOption ¶
type CoreOption func(*coreValues)
CoreOption is the option to modify core attribute.
func SetAppNameProvider ¶
func SetAppNameProvider(provider AppNameProvider) CoreOption
SetAppNameProvider is a CoreOption to replaces the default AppNameProvider.
func SetConfigProvider ¶
func SetConfigProvider(provider ConfigProvider) CoreOption
SetConfigProvider is a CoreOption to replaces the default ConfigProvider.
func SetDiProvider ¶
func SetDiProvider(provider DiProvider) CoreOption
SetDiProvider is a CoreOption to replaces the default DiContainer.
func SetEnvProvider ¶
func SetEnvProvider(provider EnvProvider) CoreOption
SetEnvProvider is a CoreOption to replaces the default EnvProvider.
func SetEventDispatcherProvider ¶
func SetEventDispatcherProvider(provider EventDispatcherProvider) CoreOption
SetEventDispatcherProvider is a CoreOption to replaces the default EventDispatcherProvider.
func SetLoggerProvider ¶
func SetLoggerProvider(provider LoggerProvider) CoreOption
SetLoggerProvider is a CoreOption to replaces the default LoggerProvider.
func WithConfigStack ¶
func WithConfigStack(provider ConfProvider, parser ConfParser) CoreOption
WithConfigStack is a CoreOption that defines a configuration layer. See package config for details.
func WithConfigWatcher ¶
func WithConfigWatcher(watcher contract.ConfigWatcher) CoreOption
WithConfigWatcher is a CoreOption that adds a config watcher to the core (for hot reloading configs).
func WithInline ¶
func WithInline(key, entry string) CoreOption
WithInline is a CoreOption that creates a inline config in the configuration stack.
type DiContainer ¶
type DiContainer interface { Provide(constructor interface{}) error Invoke(function interface{}) error }
DiContainer is a container roughly modeled after dig.Container
func ProvideDi ¶
func ProvideDi(conf contract.ConfigAccessor) DiContainer
ProvideDi is the default DiProvider for package Core.
type DiProvider ¶
type DiProvider func(conf contract.ConfigAccessor) DiContainer
DiProvider provides the DiContainer to the core.
type EnvProvider ¶
type EnvProvider func(conf contract.ConfigAccessor) contract.Env
EnvProvider provides the contract.Env to the core.
type EventDispatcherProvider ¶
type EventDispatcherProvider func(conf contract.ConfigAccessor) contract.Dispatcher
EventDispatcherProvider provides contract.Dispatcher to the core.
type HttpFunc ¶
HttpFunc converts a function to a module provides http.
func (HttpFunc) ProvideHTTP ¶
ProvideHTTP implements container.HTTPProvider
type LoggerProvider ¶
type LoggerProvider func(conf contract.ConfigAccessor, appName contract.AppName, env contract.Env) log.Logger
LoggerProvider provides the log.Logger to the core.
type OnGRPCServerShutdown ¶
OnGRPCServerShutdown is an event triggered when the http server is shutting down. traffic. At this point The traffic can no longer reach the server, but the database and other infrastructures are not closed yet. This event is useful to unregister service to service discovery.
type OnGRPCServerStart ¶
OnGRPCServerStart is an event triggered when the grpc server is ready to serve traffic. At this point the module is already wired up. This event is useful to register service to service discovery.
type OnHTTPServerShutdown ¶
OnHTTPServerShutdown is an event triggered when the http server is shutting down. traffic. At this point The traffic can no longer reach the server, but the database and other infrastructures are not closed yet. This event is useful to unregister service to service discovery.
type OnHTTPServerStart ¶
OnHTTPServerStart is an event triggered when the http server is ready to serve traffic. At this point the module is already wired up. This event is useful to register service to service discovery.
Directories
¶
Path | Synopsis |
---|---|
Package clihttp adds opentracing support to http client.
|
Package clihttp adds opentracing support to http client. |
Package config provides supports to a fully customizable layered configuration stack.
|
Package config provides supports to a fully customizable layered configuration stack. |
Package container includes the Container type, witch contains a collection of modules.
|
Package container includes the Container type, witch contains a collection of modules. |
Package contract defines a set of common interfaces for all packages in this repository.
|
Package contract defines a set of common interfaces for all packages in this repository. |
Package cronopts contains the options for cron.
|
Package cronopts contains the options for cron. |
Package di is a thin wrapper around dig.
|
Package di is a thin wrapper around dig. |
Package dtx contains a variety of patterns in the field of distributed transaction.
|
Package dtx contains a variety of patterns in the field of distributed transaction. |
sagas
Package sagas implements the orchestration based saga pattern.
|
Package sagas implements the orchestration based saga pattern. |
sagas/mysqlstore
Package mysqlstore provides a mysql store implementation for sagas.
|
Package mysqlstore provides a mysql store implementation for sagas. |
Package events provides a simple and effective implementation of event system.
|
Package events provides a simple and effective implementation of event system. |
Package key provides a way to distribute labels to component.
|
Package key provides a way to distribute labels to component. |
Package leader provides a simple leader election implementation.
|
Package leader provides a simple leader election implementation. |
leaderetcd
Package leaderetcd provides a etcd driver for package leader
|
Package leaderetcd provides a etcd driver for package leader |
leaderredis
Package leaderredis provides a redis driver for package leader
|
Package leaderredis provides a redis driver for package leader |
Package logging provides a kitlog compatible logger.
|
Package logging provides a kitlog compatible logger. |
Package observability provides a tracer and a histogram to measure all incoming requests to the system.
|
Package observability provides a tracer and a histogram to measure all incoming requests to the system. |
Package otes provides es client with opentracing.
|
Package otes provides es client with opentracing. |
Package otetcd provides etcd client with opentracing.
|
Package otetcd provides etcd client with opentracing. |
Package otgorm provides gorm with opentracing.
|
Package otgorm provides gorm with opentracing. |
Package otkafka contains the opentracing integrated a kafka transport for package Core.
|
Package otkafka contains the opentracing integrated a kafka transport for package Core. |
Package otmongo provides mongo client with opentracing.
|
Package otmongo provides mongo client with opentracing. |
Package otredis provides redis client with opentracing.
|
Package otredis provides redis client with opentracing. |
Package ots3 provides a s3 uploader with opentracing capabilities.
|
Package ots3 provides a s3 uploader with opentracing capabilities. |
Package queue provides a persistent queue implementation that interplays with the Dispatcher in the contract package.
|
Package queue provides a persistent queue implementation that interplays with the Dispatcher in the contract package. |
Package srvhttp groups helpers for server side http use cases.
|
Package srvhttp groups helpers for server side http use cases. |
Package text provides utilities to generate textual output.
|
Package text provides utilities to generate textual output. |
Package unierr presents an unification error model between gRPC transport and HTTP transport, and between server and client.
|
Package unierr presents an unification error model between gRPC transport and HTTP transport, and between server and client. |