config

Documentation

Overview

Package config implements a generic configuration system that may be used to build YARPC Dispatchers from configurations specified in different markup formats.

Usage

To build a Dispatcher, set up a Configurator and inform it about the different transports that it needs to support. This object is re-usable and may be stored as a singleton in your application.

cfg := config.New()
cfg.MustRegisterTransport(http.TransportSpec())
cfg.MustRegisterTransport(redis.TransportSpec())

Use LoadConfigFromYAML to load a yarpc.Config from YAML and pass that to yarpc.NewDispatcher.

c, err := cfg.LoadConfigFromYAML("myservice", yamlConfig)
if err != nil {
	log.Fatal(err)
}
dispatcher := yarpc.NewDispatcher(c)

If you have already parsed your configuration from a different format, pass the parsed data to LoadConfig instead.

var m map[string]interface{} = ...
c, err := cfg.LoadConfig("myservice", m)
if err != nil {
	log.Fatal(err)
}
dispatcher := yarpc.NewDispatcher(c)

NewDispatcher or NewDispatcherFromYAML may be used to get a yarpc.Dispatcher directly instead of a yarpc.Config.

dispatcher, err := cfg.NewDispatcherFromYAML("myservice", yamlConfig)

Configuration parameters for the different transports, inbounds, and outbounds are defined in the TransportSpecs that were registered against the Configurator. A TransportSpec uses this information to build the corresponding Transport, Inbound and Outbound objects.

Configuration

The configuration may be specified in YAML or as any Go-level map[string]interface{}. The examples below use YAML for illustration purposes but other markup formats may be parsed into map[string]interface{} as long as the information provided is the same.

The configuration accepts the following top-level attributes: transports, inbounds, and outbounds.

inbounds:
  # ...
outbounds:
  # ...
transports:
  # ...

See the following sections for details on the transports, inbounds, and outbounds keys in the configuration.

Inbound Configuration

The 'inbounds' attribute configures the different ways in which the service receives requests. It is represented as a mapping between inbound transport type and its configuration. For example, the following states that we want to receive requests over HTTP and Redis.

inbounds:
  redis:
    # ...
  http:
    # ...

(For details on the configuration parameters of individual transport types, check the documentation for the corresponding transport package.)

If you want multiple inbounds of the same type, specify a different name for it and add a 'type' attribute to its configuration:

inbounds:
  http:
    # ...
  http-deprecated:
    type: http
    # ...

Any inbound can be disabled by adding a 'disabled' attribute.

inbounds:
  http:
    # ...
  http-deprecated:
    type: http
    disabled: true
    # ...

Outbound Configuration

The 'outbounds' attribute configures how this service makes requests to other YARPC-compatible services. It is represented as mapping between service name and outbound configuration.

outbounds:
  keyvalue:
    # ..
  anotherservice:
    # ..

(For details on the configuration parameters of individual transport types, check the documentation for the corresponding transport package.)

The outbound configuration for a service has at least one of the following keys: unary, oneway. These specify the configurations for the corresponding RPC types for that service. For example, the following specifies that we make Unary requests to keyvalue service over HTTP and Oneway requests over Redis.

keyvalue:
  unary:
    http:
      # ...
  oneway:
    redis:
      # ...

For convenience, if there is only one outbound configuration for a service, it may be specified one level higher (without the 'unary' or 'oneway' attributes). In this case, all RPC types supported by that transport will be set. For example, the HTTP transport supports both, Unary and Oneway RPC types so the following states that requests for both RPC types must be made over HTTP.

keyvalue:
  http:
    # ...

Similarly, the following states that we only make Oneway requests to the "email" service and those are always made over Redis.

email:
  redis:
    # ...

When the name of the target service differs from the outbound name, it may be overridden with the 'service' key.

keyvalue:
  unary:
    # ...
  oneway:
    # ...
keyvalue-staging:
  service: keyvalue
  unary:
    # ...
  oneway:
    # ...

Transport Configuration

The 'transports' attribute configures the Transport objects that are shared between all inbounds and outbounds of that transport type. It is represented as a mapping between the transport name and its configuration.

transports:
  redis:
    # ...
  http:
    # ...

(For details on the configuration parameters of individual transport types, check the documentation for the corresponding transport package.)

Defining a Transport

To teach a Configurator about a Transport, register a TransportSpec against it.

cfg.RegisterTransport(TransportSpec{
	Name: "mytransport",
	BuildTransport: func(*myTransportConfig) (transport.Transport, error) {
		// ...
	},
	...
})

This transport will be configured under the 'mytransport' key in the parsed configuration data. See documentation for TransportSpec for details on what each field of TransportSpec means and how it behaves.

Index

• type BinderSpec
• type ChooserSpec
• type Configurator
  • func New(opts ...Option) *Configurator
  • func (c *Configurator) LoadConfig(serviceName string, data interface{}) (yarpc.Config, error)
  • func (c *Configurator) LoadConfigFromYAML(serviceName string, r io.Reader) (yarpc.Config, error)
  • func (c *Configurator) MustRegisterBinder(s BinderSpec)
  • func (c *Configurator) MustRegisterChooser(s ChooserSpec)
  • func (c *Configurator) MustRegisterTransport(t TransportSpec)
  • func (c *Configurator) NewDispatcher(serviceName string, data interface{}) (*yarpc.Dispatcher, error)
  • func (c *Configurator) NewDispatcherFromYAML(serviceName string, r io.Reader) (*yarpc.Dispatcher, error)
  • func (c *Configurator) RegisterBinder(s BinderSpec) error
  • func (c *Configurator) RegisterChooser(s ChooserSpec) error
  • func (c *Configurator) RegisterTransport(t TransportSpec) error
• type Kit
  • func (k *Kit) ServiceName() string
• type Option
  • func InterpolationResolver(f func(k string) (v string, ok bool)) Option
• type TransportSpec

Types

type BinderSpec

type BinderSpec struct {
	// Name of the peer selection strategy
	Name string

	// A function in the shape,
	//
	//  func(C, *config.Kit) (peer.Binder, error)
	//
	// Where C is a struct or pointer to a struct defining the configuration
	// parameters accepted by this peer chooser.
	//
	// This function will be called with the parsed configuration to build a
	// peer chooser for an outbound that uses a peer chooser.
	//
	// For example, the HTTP and TChannel outbound configurations embed a peer
	// chooser configuration. Peer choosers support a single peer or arrays of
	// peers.  Using the "with" property, an outbound can use an alternate peer
	// chooser registered by name on a YARPC Configurator using a ChooserSpec.
	//
	// BuildBinder is required.
	BuildBinder interface{}
}

BinderSpec specifies the configuration parameters for an outbound peer binding (like DNS). These specifications are registered against a Configurator to teach it how to parse the configuration for that peer binder and build instances of it.

Every BinderSpec MUST have a BuildBinder function.

For example, if we register a "dns-srv" peer list binder and a "random" peer chooser, we can use "with: dns-srv" and "choose: random" to select a random task (by host and port) from DNS A and SRV records for each outbound request.

myoutbound:
 peers:
  with: dns-srv
  choose: random
  service: fortune.yarpc.io

type ChooserSpec

type ChooserSpec struct {
	Name string

	// A function in the shape,
	//
	//  func(C, *config.Kit) (peer.List, error)
	//
	// Where C is a struct or pointer to a struct defining the configuration
	// parameters accepted by this peer chooser.
	//
	// BuildChooser is required.
	BuildChooser interface{}
}

ChooserSpec specifies the configuration parameters for an outbound peer chooser (load balancer or sharding). These specifications are registered against a Configurator to teach it how to parse the configuration for that peer chooser and build instances of it.

For example, if we register a "dns-srv" peer list binder and a "random" peer chooser, we can use "with: dns-srv" and "choose: random" to select a random task (by host and port) from DNS A and SRV records for each outbound request.

myoutbound:
 peers:
  with: dns-srv
  choose: random
  service: fortune.yarpc.io

type Configurator

type Configurator struct {
	// contains filtered or unexported fields
}

Configurator helps build Dispatchers using runtime configuration.

An empty Configurator does not know about any transports. Inform it about the different transports and their configuration parameters using the RegisterTransport function.

func New

func New(opts ...Option) *Configurator

New sets up a new empty Configurator. The returned Configurator does not know about any Transports, peer Chooser lists, or peer list Binders. Individual TransportSpecs, ChooserSpecs, and BinderSpecs must be registered against it using the RegisterTransport, RegisterChooser, and RegisterBinder functions.

func (*Configurator) LoadConfig

func (c *Configurator) LoadConfig(serviceName string, data interface{}) (yarpc.Config, error)

LoadConfig loads a yarpc.Config from a map[string]interface{} or map[interface{}]interface{}.

See the module documentation for the shape the map[string]interface{} is expected to conform to.

func (*Configurator) LoadConfigFromYAML

func (c *Configurator) LoadConfigFromYAML(serviceName string, r io.Reader) (yarpc.Config, error)

LoadConfigFromYAML loads a yarpc.Config from YAML. Use LoadConfig if you have your own map[string]interface{} or map[interface{}]interface{} to provide.

func (*Configurator) MustRegisterBinder

func (c *Configurator) MustRegisterBinder(s BinderSpec)

MustRegisterBinder registers the given BinderSpec with the Configurator. This function panics if the BinderSpec is invalid.

func (*Configurator) MustRegisterChooser

func (c *Configurator) MustRegisterChooser(s ChooserSpec)

MustRegisterChooser registers the given ChooserSpec with the Configurator. This function panics if the ChooserSpec is invalid.

func (*Configurator) MustRegisterTransport

func (c *Configurator) MustRegisterTransport(t TransportSpec)

MustRegisterTransport registers the given TransportSpec with the Configurator. This function panics if the TransportSpec was invalid.

func (*Configurator) NewDispatcher

func (c *Configurator) NewDispatcher(serviceName string, data interface{}) (*yarpc.Dispatcher, error)

NewDispatcher builds a new Dispatcher from the given configuration data.

func (*Configurator) NewDispatcherFromYAML

func (c *Configurator) NewDispatcherFromYAML(serviceName string, r io.Reader) (*yarpc.Dispatcher, error)

NewDispatcherFromYAML builds a Dispatcher from the given YAML configuration.

func (*Configurator) RegisterBinder

func (c *Configurator) RegisterBinder(s BinderSpec) error

RegisterBinder registers a BinderSpec with the given Configurator. Returns an error if the BinderSpec is invalid.

A binder enables custom peer list bindings, like DNS with SRV + A records or a task list file watcher.

If a binder with the same name already exists, it will be replaced.

Use MustRegisterBinder to panic if the registration fails.

func (*Configurator) RegisterChooser

func (c *Configurator) RegisterChooser(s ChooserSpec) error

RegisterChooser registers a ChooserSpec with the given Configurator. Returns an error if the ChooserSpec is invalid.

If a chooser with the same name already exists, it will be replaced.

Use MustRegisterChooser to panic in the case of registration failure.

func (*Configurator) RegisterTransport

func (c *Configurator) RegisterTransport(t TransportSpec) error

RegisterTransport registers a TransportSpec with the given Configurator. An error is returned if the TransportSpec was invalid.

If a transport with the same name was already registered, it will be overwritten.

Use MustRegisterTransport if you want to panic in case of registration failure.

type Kit

type Kit struct {
	// contains filtered or unexported fields
}

Kit carries internal dependencies for building peer choosers. The kit gets threaded through transport, outbound, and inbound builders so they can thread the kit through functions like BuildChooser on a ChooserConfig.

func (*Kit) ServiceName

func (k *Kit) ServiceName() string

ServiceName returns the name of the service for which components are being built.

type Option

type Option func(*Configurator)

Option customizes a Configurator.

func InterpolationResolver

func InterpolationResolver(f func(k string) (v string, ok bool)) Option

InterpolationResolver changes how interpolated variables in the configuration are resolved. By default, environment variables are used.

Variables will be interpolated using this function if a parsed field inside a configuration structure was annotated with config:",interpolate" or config:"$name,interpolate" where $name is encoded name of that field.

See TransportSpec documentation for more information.

type TransportSpec

type TransportSpec struct {
	// Name of the transport
	Name string

	// A function in the shape,
	//
	// 	func(C, *config.Kit) (transport.Transport, error)
	//
	// Where C is a struct or pointer to a struct defining the configuration
	// parameters accepted by this transport.
	//
	// This function will be called with the parsed configuration to build
	// Transport defined by this spec.
	BuildTransport interface{}

	// A function in the shape,
	//
	// 	func(C, transport.Transport, *config.Kit) (transport.Inbound, error)
	//
	// Where C is a struct or pointer to a struct defining the configuration
	// parameters for the inbound.
	//
	// This may be nil if this transport does not support inbounds.
	//
	// This function will be called with the parsed configuration and the
	// transport built by BuildTransport to build the inbound for this
	// transport.
	BuildInbound interface{}

	// The following two are functions in the shapes,
	//
	// 	func(C, transport.Transport, *config.Kit) (transport.UnaryOutbound, error)
	// 	func(C, transport.Transport, *config.Kit) (transport.OnewayOutbound, error)
	//
	// Where C is a struct or pointer to a struct defining the configuration
	// parameters for outbounds of that RPC type.
	//
	// Either value may be nil to indicate that the transport does not support
	// unary or oneway outbounds.
	//
	// These functions will be called with the parsed configurations and the
	// transport built by BuildTransport to build the unary and oneway
	// outbounds for this transport.
	BuildUnaryOutbound  interface{}
	BuildOnewayOutbound interface{}
}

TransportSpec specifies the configuration parameters for a transport. These specifications are registered against a Configurator to teach it how to parse the configuration for that transport and build instances of it.

Every TransportSpec MUST have a BuildTransport function. The spec may provide BuildInbound, BuildUnaryOutbound, and BuildOnewayOutbound functions if the Transport supports that functoinality. For example, if a transport only supports incoming and outgoing Oneway requests, its spec will provide a BuildTransport, BuildInbound, and BuildOnewayOutbound function.

The signature of BuildTransport must have the shape:

func(C, *config.Kit) (T, error)

Where C is a struct defining the configuration parameters for the transport, the kit carries information and tools from the configurator to this and other builders, and T is the transport type.

The remaining Build* functions must have a similar interface, but also carry the transport instance.

Each Build* function has the shape:

func(C, transport.Transport, *config.Kit) (X, error)

Where X is the entity type, albeit an inbound or a unary or oneway outbound. For example,

func(HttpOutboundConfig, transport.Transport) (transport.UnaryOutbound, error)

Is a function to build an HTTP unary outbound from its outbound configuration and the corresponding transport.

The Configurator will decode and fill the requested struct type from the input configuration. For example, given,

type HttpOutboundConfig struct {
	URL string
}

Configurator expects the outbound configuration for HTTP to have a 'url' field. In YAML, the following,

outbounds:
  myservice:
    http:
      url: http://localhost:8080

Will be decoded into,

HttpOutboundConfig{URL: "http://localhost:8080"}

A case-insensitive match is performed to map fields from configuration data to structs.

Configuration structs can use standard Go primitive types, time.Duration, maps, slices, and other similar structs. For example only, an outbound might accept a config containing an array of host:port structs (In practice, an outbound would use a ChooserConfig to build a peer.Chooser).

type Peer struct {
	Host string
	Port int
}

type MyOutboundConfig struct{ Peers []Peer }

Will expect the following YAML.

myoutbound:
  peers:
	- host: localhost
	  port: 8080
	- host: anotherhost
	  port: 8080

Customizing Field Names

If a field name differs from the name of the field inside the configuration data, a `config` tag may be added to the struct to specify a different name.

type MyInboundConfig struct {
	Peer string `config:"peer"`
}

The configuration for this struct will be in the shape,

myinbound:
  peer: foo

Runtime Variables

In addition to specifying the field name, the `config` tag may also include an `interpolate` option to request interpolation of variables in the form ${NAME} or ${NAME:default} at the time the value is decoded. By default, environment variables are used to fill the variables; this may be changed with the InterpolationResolver option. The `interpolate` option may be applied to primitive types (strings, integers, booleans, floats, and time.Duration) only.

For example in,

type MyInboundConfig struct {
	QueueName string `config:"queue,interpolate"`
	Timeout time.Duration `config:",interpolate"`
	// If the name is left empty, the default name is used.
}

The values for both QueueName and Timeout may contain strings in the form ${NAME} to be replaced with the value of that environment variable. The form ${NAME:default} may be used to provide a default value if the environment variable is not set.

myinbound:
  queue: inbound-requests-${STAGE:dev}
  timeout: ${REQUEST_TIMEOUT:5s}

The above states that the queue inbound-requests-dev should be used by default, but if the STAGE environment variable is set, the queue inbound-requests-${STAGE} should be used. Similarly, it also states that a timeout of 5 seconds should be used by default, unless the REQUEST_TIMEOUT environment variable is set in which case the timeout specified in the environment variable should be used.