kit

module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 7, 2016 License: MIT

README

Go kit Circle CI Drone.io Travis CI GoDoc Coverage Status Go Report Card

Go kit is a distributed programming toolkit for building microservices in large organizations. We solve common problems in distributed systems, so you can focus on your business logic.

Documentation

Examples

Perhaps the best way to understand Go kit is to follow along as we build an example service from first principles. This can serve as a blueprint for your own new service, or demonstrate how to adapt your existing service to use Go kit components.

Endpoint

Go kit primarily deals in the RPC messaging pattern. We use an abstraction called an endpoint to model individual RPCs. An endpoint can be implemented by a server, and called by a client. It's the fundamental building block of many Go kit components.

Circuit breaker

The circuitbreaker package provides endpoint adapters to several popular circuit breaker libraries. Circuit breakers prevent thundering herds, and improve resiliency against intermittent errors. Every client-side endpoint should be wrapped in a circuit breaker.

Rate limiter

The ratelimit package provides endpoint adapters to rate limiting packages. Rate limiters are equally applicable to both server- and client-side endpoints. Use rate limiters to enforce upper thresholds on incoming or outgoing request throughput.

Transport

The transport package provides helpers to bind endpoints to specific serialization mechanisms. At the moment, Go kit just provides helpers for simple JSON over HTTP. If your organization uses a fully-featured transport, bindings are typically provided by the Go library for the transport, and there's not much for Go kit to do. In those cases, see the examples to understand how to write adapters for your endpoints. For now, see the addsvc to understand how transport bindings work. We have specific examples for Thrift, gRPC, net/rpc, and JSON over HTTP. JSON/RPC and Swagger support is planned.

Logging

Services produce logs to be consumed later, either by humans or machines. Humans might be interested in debugging errors, or tracing specific requests. Machines might be interested in counting interesting events, or aggregating information for offline processing. In both cases, it's important that the log messages be structured and actionable. Go kit's log package is designed to encourage both of these best practices.

Metrics (Instrumentation)

Services can't be considered production-ready until they're thoroughly instrumented with metrics that track counts, latency, health, and other periodic or per-request information. Go kit's metrics package provides a robust common set of interfaces for instrumenting your service. Bindings exist for common backends, from expvar to statsd to Prometheus.

Request tracing

As your infrastructure grows, it becomes important to be able to trace a request, as it travels through multiple services and back to the user. Go kit's tracing package provides enhancements for your endpoints and transport bindings to capture information about requests and emit them to request tracing systems. (Currently, Zipkin is supported; Appdash support is planned.)

Service discovery and load balancing

If your service calls another service, it needs to know how to find it, and should intelligently spread its load among those discovered instances. Go kit's loadbalancer package provides client-side endpoint middleware to solve that problem, whether your organization uses static hosts or IPs, DNS SRV records, Consul, etcd, or Zookeeper. And if you use a custom system, it's very easy to write your own Publisher and use Go kit's load balancing strategies. (Currently, static hosts, DNS SRV, etcd, Consul and ZooKeeper are supported)

Motivation

Go has emerged as the language of the server, but it remains underrepresented in large, consumer-focused tech companies like Facebook, Twitter, Netflix, and SoundCloud. These organizations have largely adopted JVM-based stacks for their business logic, owing in large part to libraries and ecosystems that directly support their microservice architectures.

To reach its next level of success, Go needs more than simple primitives and idioms. It needs a comprehensive toolkit, for coherent distributed programming in the large. Go kit is a set of packages and best practices, leveraging years of production experience, and providing a comprehensive, robust, and trustable platform for organizations of any size.

In short, Go kit makes Go a viable choice for business-domain microservices.

For more details, see the motivating blog post and the video of the talk. See also the Go kit talk at GopherCon 2015.

Goals

  • Operate in a heterogeneous SOA — expect to interact with mostly non-Go-kit services
  • RPC as the primary messaging pattern
  • Pluggable serialization and transport — not just JSON over HTTP
  • Operate within existing infrastructures — no mandates for specific tools or technologies

Non-goals

  • Supporting messaging patterns other than RPC (for now) — e.g. MPI, pub/sub, CQRS, etc.
  • Re-implementing functionality that can be provided by adapting existing software
  • Having opinions on operational concerns: deployment, configuration, process supervision, orchestration, etc.

Contributing

Please see CONTRIBUTING.md. Thank you, contributors!

Dependency management

Go kit is a library, designed to be imported into a binary package. Vendoring is currently the best way for binary package authors to ensure reliable, reproducible builds. Therefore, we strongly recommend our users use vendoring for all of their dependencies, including Go kit.
To avoid compatibility and availability issues, Go kit doesn't vendor its own dependencies, and doesn't recommend use of third-party import proxies.

There are several tools which make vendoring easier, including gb, glide, gvt, govendor, and vendetta. In addition, Go kit uses a variety of continuous integration providers to find and fix compatibility problems as soon as they occur.

Projects with a ★ have had particular influence on Go kit's design (or vice-versa).

Service frameworks
  • gizmo, a microservice toolkit from The New York Times ★
  • go-micro, a microservices client/server library ★
  • gocircuit, dynamic cloud orchestration
  • gotalk, async peer communication protocol & library
  • h2, a microservices framework ★
  • Kite, a micro-service framework
Individual components
Web frameworks

Additional reading

Directories

Path Synopsis
examples
addsvc
Package addsvc implements the business and transport logic for an example service that can sum integers and concatenate strings.
Package addsvc implements the business and transport logic for an example service that can sum integers and concatenate strings.
addsvc/client/grpc
Package grpc provides a gRPC client for the add service.
Package grpc provides a gRPC client for the add service.
addsvc/client/http
Package http provides an HTTP client for the add service.
Package http provides an HTTP client for the add service.
addsvc/client/thrift
Package thrift provides a Thrift client for the add service.
Package thrift provides a Thrift client for the add service.
addsvc/pb
Package pb is a generated protocol buffer package.
Package pb is a generated protocol buffer package.
profilesvc/client
Package client provides a profilesvc client based on a predefined Consul service name and relevant tags.
Package client provides a profilesvc client based on a predefined Consul service name and relevant tags.
shipping/booking
Package booking provides the use-case of booking a cargo.
Package booking provides the use-case of booking a cargo.
shipping/cargo
Package cargo contains the heart of the domain model.
Package cargo contains the heart of the domain model.
shipping/handling
Package handling provides the use-case for registering incidents.
Package handling provides the use-case for registering incidents.
shipping/inspection
Package inspection provides means to inspect cargos.
Package inspection provides means to inspect cargos.
shipping/location
Package location provides the Location aggregate.
Package location provides the Location aggregate.
shipping/repository
Package repository provides implementations of all the domain repositories.
Package repository provides implementations of all the domain repositories.
shipping/routing
Package routing provides the routing domain service.
Package routing provides the routing domain service.
shipping/tracking
Package tracking provides the use-case of tracking a cargo.
Package tracking provides the use-case of tracking a cargo.
shipping/voyage
Package voyage provides the Voyage aggregate.
Package voyage provides the Voyage aggregate.
log
Package log provides basic interfaces for structured logging.
Package log provides basic interfaces for structured logging.
term
Package term provides tools for logging to a terminal.
Package term provides tools for logging to a terminal.
Package metrics provides an extensible framework to instrument your application.
Package metrics provides an extensible framework to instrument your application.
discard
Package discard implements a backend for package metrics that succeeds without doing anything.
Package discard implements a backend for package metrics that succeeds without doing anything.
dogstatsd
Package dogstatsd implements a DogStatsD backend for package metrics.
Package dogstatsd implements a DogStatsD backend for package metrics.
expvar
Package expvar implements an expvar backend for package metrics.
Package expvar implements an expvar backend for package metrics.
graphite
Package graphite implements a Graphite backend for package metrics.
Package graphite implements a Graphite backend for package metrics.
influxdb
Package influxdb implements a InfluxDB backend for package metrics.
Package influxdb implements a InfluxDB backend for package metrics.
prometheus
Package prometheus implements a Prometheus backend for package metrics.
Package prometheus implements a Prometheus backend for package metrics.
statsd
Package statsd implements a statsd backend for package metrics.
Package statsd implements a statsd backend for package metrics.
teststat
Package teststat contains helper functions for statistical testing of metrics implementations.
Package teststat contains helper functions for statistical testing of metrics implementations.
sd
Package sd provides utilities related to service discovery.
Package sd provides utilities related to service discovery.
lb
Package lb deals with client-side load balancing across multiple identical instances of services and endpoints.
Package lb deals with client-side load balancing across multiple identical instances of services and endpoints.
zk
tracing
transport
util
conn
Package conn provides utilities related to connections.
Package conn provides utilities related to connections.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL