Documentation

Overview

Package http implements a YARPC transport based on the HTTP/1.1 protocol. The HTTP transport provides first class support for Unary RPCs and experimental support for Oneway RPCs.

Usage

An HTTP Transport must be constructed to use this transport.

httpTransport := http.NewTransport()

To serve your YARPC application over HTTP, pass an HTTP inbound in your yarpc.Config.

myInbound := httpTransport.NewInbound(":8080")
dispatcher := yarpc.NewDispatcher(yarpc.Config{
	Name: "myservice",
	Inbounds: yarpc.Inbounds{myInbound},
})

To make requests to a YARPC application that supports HTTP, pass an HTTP outbound in your yarpc.Config.

myserviceOutbound := httpTransport.NewSingleOutbound("http://127.0.0.1:8080")
dispatcher := yarpc.NewDispatcher(yarpc.Config{
	Name: "myclient",
	Outbounds: yarpc.Outbounds{
		"myservice": {Unary: myserviceOutbound},
	},
})

Note that stopping an HTTP transport does NOT immediately terminate ongoing requests. Connections will remain open until all clients have disconnected.

Configuration

An HTTP Transport may be configured using YARPC's configuration system. See TransportConfig, InboundConfig, and OutboundConfig for details on the different configuration parameters supported by this transport.

Wire Representation

YARPC requests and responses are sent as plain HTTP requests and responses. YARPC metadata is sent inside reserved HTTP headers. Application headers for requests and responses are sent as HTTP headers with the header names prefixed with a pre-defined string. See Constants for more information on the names of these headers. The request and response bodies are sent as-is in the HTTP request or response body.

See Also

YARPC Properties: https://github.com/yarpc/yarpc/blob/master/properties.md

Index

Examples

Constants

View Source
const (
	// Name of the service sending the request. This corresponds to the
	// Request.Caller attribute.
	CallerHeader = "Rpc-Caller"

	// Name of the procedure of the caller sending the request. This corresponds to the
	// Request.CallerProcedure attribute.
	CallerProcedureHeader = "Rpc-Caller-Procedure"

	// Name of the encoding used for the request body. This corresponds to the
	// Request.Encoding attribute.
	EncodingHeader = "Rpc-Encoding"

	// Amount of time (in milliseconds) within which the request is expected
	// to finish.
	TTLMSHeader = "Context-TTL-MS"

	// Name of the procedure being called. This corresponds to the
	// Request.Procedure attribute.
	ProcedureHeader = "Rpc-Procedure"

	// Name of the service to which the request is being sent. This
	// corresponds to the Request.Service attribute. This header is also used
	// in responses to ensure requests are processed by the correct service.
	ServiceHeader = "Rpc-Service"

	// Shard key used by the destined service to shard the request. This
	// corresponds to the Request.ShardKey attribute.
	ShardKeyHeader = "Rpc-Shard-Key"

	// The traffic group responsible for handling the request. This
	// corresponds to the Request.RoutingKey attribute.
	RoutingKeyHeader = "Rpc-Routing-Key"

	// A service that can proxy the destined service. This corresponds to the
	// Request.RoutingDelegate attribute.
	RoutingDelegateHeader = "Rpc-Routing-Delegate"

	// Whether the response body contains an application error.
	ApplicationStatusHeader = "Rpc-Status"

	// ErrorCodeHeader contains the string representation of the error code.
	ErrorCodeHeader = "Rpc-Error-Code"

	// ErrorNameHeader contains the name of a user-defined error.
	ErrorNameHeader = "Rpc-Error-Name"

	// ErrorMessageHeader contains the message of an error, if the
	// BothResponseError feature is enabled.
	ErrorMessageHeader = "Rpc-Error-Message"

	// ErrorDetailsHeader contains the marshaled bytes of an error. Because
	// error details are mainly a gRPC concept, we use the header key name that
	// gRPC uses.
	// https://github.com/grpc/grpc-go/blob/04ea82009cdb9ecdefc6289f4c93ec919a10b3b6/internal/transport/handler_server.go#L218
	ErrorDetailsHeader = "Grpc-Status-Details-Bin"

	// AcceptsBothResponseErrorHeader says that the BothResponseError
	// feature is supported on the client. If the value is "true",
	// this indicates true.
	AcceptsBothResponseErrorHeader = "Rpc-Accepts-Both-Response-Error"

	// BothResponseErrorHeader says that the BothResponseError
	// feature is supported on the server. If any non-empty value is set,
	// this indicates true.
	BothResponseErrorHeader = "Rpc-Both-Response-Error"
)

    HTTP headers used in requests and responses to send YARPC metadata.

    View Source
    const (
    	// The request was successful.
    	ApplicationSuccessStatus = "success"
    
    	// An error occurred. The response body contains an application header.
    	ApplicationErrorStatus = "error"
    
    	// AcceptTrue is the true value used for accept headers.
    	AcceptTrue = "true"
    )

      Valid values for the Rpc-Status header.

      View Source
      const ApplicationHeaderPrefix = "Rpc-Header-"

        ApplicationHeaderPrefix is the prefix added to application header keys to send them in requests or responses.

        View Source
        const TransportName = "http"

          TransportName is the name of the transport.

          This value is what is used as transport.Request#Transport and transport.Namer for Outbounds.

          Variables

          This section is empty.

          Functions

          func TransportSpec

          func TransportSpec(opts ...Option) yarpcconfig.TransportSpec

            TransportSpec returns a TransportSpec for the HTTP transport.

            See TransportConfig, InboundConfig, and OutboundConfig for details on the different configuration parameters supported by this Transport.

            Any Transport, Inbound or Outbound option may be passed to this function. These options will be applied BEFORE configuration parameters are interpreted. This allows configuration parameters to override Option provided to TransportSpec.

            Types

            type Inbound

            type Inbound struct {
            	// contains filtered or unexported fields
            }

              Inbound receives YARPC requests using an HTTP server. It may be constructed using the NewInbound method on the Transport.

              Example
              Output:
              
              

              func (*Inbound) Addr

              func (i *Inbound) Addr() net.Addr

                Addr returns the address on which the server is listening. Returns nil if Start has not been called yet.

                func (*Inbound) Introspect

                func (i *Inbound) Introspect() introspection.InboundStatus

                  Introspect returns the state of the inbound for introspection purposes.

                  func (*Inbound) IsRunning

                  func (i *Inbound) IsRunning() bool

                    IsRunning returns whether the inbound is currently running

                    func (*Inbound) SetRouter

                    func (i *Inbound) SetRouter(router transport.Router)

                      SetRouter configures a router to handle incoming requests. This satisfies the transport.Inbound interface, and would be called by a dispatcher when it starts.

                      func (*Inbound) Start

                      func (i *Inbound) Start() error

                        Start starts the inbound with a given service detail, opening a listening socket.

                        func (*Inbound) Stop

                        func (i *Inbound) Stop() error

                          Stop the inbound using Shutdown.

                          func (*Inbound) Tracer

                          func (i *Inbound) Tracer(tracer opentracing.Tracer) *Inbound

                            Tracer configures a tracer on this inbound.

                            func (*Inbound) Transports

                            func (i *Inbound) Transports() []transport.Transport

                              Transports returns the inbound's HTTP transport.

                              type InboundConfig

                              type InboundConfig struct {
                              	// Address to listen on. This field is required.
                              	Address string `config:"address,interpolate"`
                              	// The additional headers, starting with x, that should be
                              	// propagated to handlers. This field is optional.
                              	GrabHeaders []string `config:"grabHeaders"`
                              	// The maximum amount of time to wait for the inbound to shutdown.
                              	ShutdownTimeout *time.Duration `config:"shutdownTimeout"`
                              }

                                InboundConfig configures an HTTP inbound.

                                inbounds:
                                  http:
                                    address: ":80"
                                    grabHeaders:
                                      - x-foo
                                      - x-bar
                                    shutdownTimeout: 5s
                                

                                type InboundOption

                                type InboundOption func(*Inbound)

                                  InboundOption customizes the behavior of an HTTP Inbound constructed with NewInbound.

                                  func GrabHeaders

                                  func GrabHeaders(headers ...string) InboundOption

                                    GrabHeaders specifies additional headers that are not prefixed with ApplicationHeaderPrefix that should be propagated to the caller.

                                    All headers given must begin with x- or X- or the Inbound that the returned option is passed to will return an error when Start is called.

                                    Headers specified with GrabHeaders are case-insensitive. https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2

                                    func Interceptor

                                    func Interceptor(interceptor func(yarpcHandler http.Handler) http.Handler) InboundOption

                                      Interceptor specifies a function which can wrap the YARPC handler. If provided, this function will be called with an http.Handler which will route requests through YARPC. The http.Handler returned by this function may delegate requests to the provided YARPC handler to route them through YARPC.

                                      Example
                                      Output:
                                      
                                      hello, world!
                                      

                                      func Mux

                                      func Mux(pattern string, mux *http.ServeMux) InboundOption

                                        Mux specifies that the HTTP server should make the YARPC endpoint available under the given pattern on the given ServeMux. By default, the YARPC service is made available on all paths of the HTTP server. By specifying a ServeMux, users can narrow the endpoints under which the YARPC service is available and offer their own non-YARPC endpoints.

                                        Example
                                        Output:
                                        
                                        hello from /health
                                        

                                        func ShutdownTimeout

                                        func ShutdownTimeout(timeout time.Duration) InboundOption

                                          ShutdownTimeout specifies the maximum duration the inbound should wait for closing idle connections, and pending calls to complete.

                                          Set to 0 to wait for a complete drain.

                                          Defaults to 5 seconds.

                                          type Option

                                          type Option interface {
                                          	// contains filtered or unexported methods
                                          }

                                            Option allows customizing the YARPC HTTP transport. Any InboundOption, OutboundOption, or TransportOption is a valid Option.

                                            type Outbound

                                            type Outbound struct {
                                            	// contains filtered or unexported fields
                                            }

                                              Outbound sends YARPC requests over HTTP. It may be constructed using the NewOutbound function or the NewOutbound or NewSingleOutbound methods on the HTTP Transport. It is recommended that services use a single HTTP transport to construct all HTTP outbounds, ensuring efficient sharing of resources across the different outbounds.

                                              Example
                                              Output:
                                              
                                              

                                              func NewOutbound

                                              func NewOutbound(chooser peer.Chooser, opts ...OutboundOption) *Outbound

                                                NewOutbound builds an HTTP outbound that sends requests to peers supplied by the given peer.Chooser. The URL template for used for the different peers may be customized using the URLTemplate option.

                                                The peer chooser and outbound must share the same transport, in this case the HTTP transport. The peer chooser must use the transport's RetainPeer to obtain peer instances and return those peers to the outbound when it calls Choose. The concrete peer type is private and intrinsic to the HTTP transport.

                                                func (*Outbound) Call

                                                func (o *Outbound) Call(ctx context.Context, treq *transport.Request) (*transport.Response, error)

                                                  Call makes a HTTP request

                                                  func (*Outbound) CallOneway

                                                  func (o *Outbound) CallOneway(ctx context.Context, treq *transport.Request) (transport.Ack, error)

                                                    CallOneway makes a oneway request

                                                    func (*Outbound) Chooser

                                                    func (o *Outbound) Chooser() peer.Chooser

                                                      Chooser returns the outbound's peer chooser.

                                                      func (*Outbound) Introspect

                                                      func (o *Outbound) Introspect() introspection.OutboundStatus

                                                        Introspect returns basic status about this outbound.

                                                        func (*Outbound) IsRunning

                                                        func (o *Outbound) IsRunning() bool

                                                          IsRunning returns whether the Outbound is running.

                                                          func (*Outbound) RoundTrip

                                                          func (o *Outbound) RoundTrip(hreq *http.Request) (*http.Response, error)

                                                            RoundTrip implements the http.RoundTripper interface, making a YARPC HTTP outbound suitable as a Transport when constructing an HTTP Client. An HTTP client is suitable only for relative paths to a single outbound service. The HTTP outbound overrides the host:port portion of the URL of the provided request.

                                                            Sample usage:

                                                            client := http.Client{Transport: outbound}
                                                            

                                                            Thereafter use the Golang standard library HTTP to send requests with this client.

                                                            ctx, cancel := context.WithTimeout(context.Background(), time.Second)
                                                            defer cancel()
                                                            req, err := http.NewRequest("GET", "http://example.com/", nil /* body */)
                                                            req = req.WithContext(ctx)
                                                            res, err := client.Do(req)
                                                            

                                                            All requests must have a deadline on the context. The peer chooser for raw HTTP requests will receive a YARPC transport.Request with no body.

                                                            OpenTracing information must be added manually, before this call, to support context propagation.

                                                            func (*Outbound) Start

                                                            func (o *Outbound) Start() error

                                                              Start the HTTP outbound

                                                              func (*Outbound) Stop

                                                              func (o *Outbound) Stop() error

                                                                Stop the HTTP outbound

                                                                func (*Outbound) TransportName

                                                                func (o *Outbound) TransportName() string

                                                                  TransportName is the transport name that will be set on `transport.Request` struct.

                                                                  func (*Outbound) Transports

                                                                  func (o *Outbound) Transports() []transport.Transport

                                                                    Transports returns the outbound's HTTP transport.

                                                                    type OutboundConfig

                                                                    type OutboundConfig struct {
                                                                    	yarpcconfig.PeerChooser
                                                                    
                                                                    	// URL to which requests will be sent for this outbound. This field is
                                                                    	// required.
                                                                    	URL string `config:"url,interpolate"`
                                                                    
                                                                    	// HTTP headers that will be added to all requests made through this
                                                                    	// outbound.
                                                                    	//
                                                                    	//  http:
                                                                    	//    url: "http://localhost:8080/yarpc"
                                                                    	//    addHeaders:
                                                                    	//      X-Caller: myserice
                                                                    	//      X-Token: foo
                                                                    	AddHeaders map[string]string `config:"addHeaders"`
                                                                    }

                                                                      OutboundConfig configures an HTTP outbound.

                                                                      outbounds:
                                                                        keyvalueservice:
                                                                          http:
                                                                            url: "http://127.0.0.1:80/"
                                                                      

                                                                      The HTTP outbound supports both, Unary and Oneway transport types. To use it for only one of these, nest the section inside a "unary" or "onewy" section.

                                                                      outbounds:
                                                                        keyvalueservice:
                                                                          unary:
                                                                            http:
                                                                              url: "http://127.0.0.1:80/"
                                                                      

                                                                      An HTTP outbound can also configure a peer list. In this case, there can still be a "url" and it serves as a template for the HTTP client, expressing whether to use "http:" or "https:" and what path to use. The address gets replaced with peers from the peer list.

                                                                      outbounds:
                                                                        keyvalueservice:
                                                                          unary:
                                                                            http:
                                                                              url: "https://address/rpc"
                                                                              round-robin:
                                                                                peers:
                                                                                  - 127.0.0.1:8080
                                                                                  - 127.0.0.1:8081
                                                                      

                                                                      type OutboundOption

                                                                      type OutboundOption func(*Outbound)

                                                                        OutboundOption customizes an HTTP Outbound.

                                                                        func AddHeader

                                                                        func AddHeader(key, value string) OutboundOption

                                                                          AddHeader specifies that an HTTP outbound should always include the given header in outgoung requests.

                                                                          httpTransport.NewOutbound(chooser, http.AddHeader("X-Token", "TOKEN"))
                                                                          

                                                                          Note that headers starting with "Rpc-" are reserved by YARPC. This function will panic if the header starts with "Rpc-".

                                                                          func URLTemplate

                                                                          func URLTemplate(template string) OutboundOption

                                                                            URLTemplate specifies the URL this outbound makes requests to. For peer.Chooser-based outbounds, the peer (host:port) spection of the URL may vary from call to call but the rest will remain unchanged. For single-peer outbounds, the URL will be used as-is.

                                                                            type Transport

                                                                            type Transport struct {
                                                                            	// contains filtered or unexported fields
                                                                            }

                                                                              Transport keeps track of HTTP peers and the associated HTTP client. It allows using a single HTTP client to make requests to multiple YARPC services and pooling the resources needed therein.

                                                                              func NewTransport

                                                                              func NewTransport(opts ...TransportOption) *Transport

                                                                                NewTransport creates a new HTTP transport for managing peers and sending requests

                                                                                func (*Transport) IsRunning

                                                                                func (a *Transport) IsRunning() bool

                                                                                  IsRunning returns whether the HTTP transport is running.

                                                                                  func (*Transport) NewInbound

                                                                                  func (t *Transport) NewInbound(addr string, opts ...InboundOption) *Inbound

                                                                                    NewInbound builds a new HTTP inbound that listens on the given address and sharing this transport.

                                                                                    func (*Transport) NewOutbound

                                                                                    func (t *Transport) NewOutbound(chooser peer.Chooser, opts ...OutboundOption) *Outbound

                                                                                      NewOutbound builds an HTTP outbound that sends requests to peers supplied by the given peer.Chooser. The URL template for used for the different peers may be customized using the URLTemplate option.

                                                                                      The peer chooser and outbound must share the same transport, in this case the HTTP transport. The peer chooser must use the transport's RetainPeer to obtain peer instances and return those peers to the outbound when it calls Choose. The concrete peer type is private and intrinsic to the HTTP transport.

                                                                                      func (*Transport) NewSingleOutbound

                                                                                      func (t *Transport) NewSingleOutbound(uri string, opts ...OutboundOption) *Outbound

                                                                                        NewSingleOutbound builds an outbound that sends YARPC requests over HTTP to the specified URL.

                                                                                        The URLTemplate option has no effect in this form.

                                                                                        func (*Transport) ReleasePeer

                                                                                        func (a *Transport) ReleasePeer(pid peer.Identifier, sub peer.Subscriber) error

                                                                                          ReleasePeer releases a peer from the peer.Subscriber and removes that peer from the Transport if nothing is listening to it

                                                                                          func (*Transport) RetainPeer

                                                                                          func (a *Transport) RetainPeer(pid peer.Identifier, sub peer.Subscriber) (peer.Peer, error)

                                                                                            RetainPeer gets or creates a Peer for the specified peer.Subscriber (usually a peer.Chooser)

                                                                                            func (*Transport) Start

                                                                                            func (a *Transport) Start() error

                                                                                              Start starts the HTTP transport.

                                                                                              func (*Transport) Stop

                                                                                              func (a *Transport) Stop() error

                                                                                                Stop stops the HTTP transport.

                                                                                                type TransportConfig

                                                                                                type TransportConfig struct {
                                                                                                	// Specifies the keep-alive period for all HTTP clients. This field is
                                                                                                	// optional.
                                                                                                	KeepAlive             time.Duration       `config:"keepAlive"`
                                                                                                	MaxIdleConns          int                 `config:"maxIdleConns"`
                                                                                                	MaxIdleConnsPerHost   int                 `config:"maxIdleConnsPerHost"`
                                                                                                	IdleConnTimeout       time.Duration       `config:"idleConnTimeout"`
                                                                                                	DisableKeepAlives     bool                `config:"disableKeepAlives"`
                                                                                                	DisableCompression    bool                `config:"disableCompression"`
                                                                                                	ResponseHeaderTimeout time.Duration       `config:"responseHeaderTimeout"`
                                                                                                	ConnTimeout           time.Duration       `config:"connTimeout"`
                                                                                                	ConnBackoff           yarpcconfig.Backoff `config:"connBackoff"`
                                                                                                }

                                                                                                  TransportConfig configures the shared HTTP Transport. This is shared between all HTTP outbounds and inbounds of a Dispatcher.

                                                                                                  transports:
                                                                                                    http:
                                                                                                      keepAlive: 30s
                                                                                                      maxIdleConns: 2
                                                                                                      maxIdleConnsPerHost: 2
                                                                                                      idleConnTimeout: 90s
                                                                                                      disableKeepAlives: false
                                                                                                      disableCompression: false
                                                                                                      responseHeaderTimeout: 0s
                                                                                                      connTimeout: 500ms
                                                                                                      connBackoff:
                                                                                                        exponential:
                                                                                                          first: 10ms
                                                                                                          max: 30s
                                                                                                  

                                                                                                  All parameters of TransportConfig are optional. This section may be omitted in the transports section.

                                                                                                  type TransportOption

                                                                                                  type TransportOption func(*transportOptions)

                                                                                                    TransportOption customizes the behavior of an HTTP transport.

                                                                                                    func ConnBackoff

                                                                                                    func ConnBackoff(s backoffapi.Strategy) TransportOption

                                                                                                      ConnBackoff specifies the connection backoff strategy for delays between connection attempts for each peer.

                                                                                                      The default is exponential backoff starting with 10ms fully jittered, doubling each attempt, with a maximum interval of 30s.

                                                                                                      func ConnTimeout

                                                                                                      func ConnTimeout(d time.Duration) TransportOption

                                                                                                        ConnTimeout is the time that the transport will wait for a connection attempt. If a peer has been retained by a peer list, connection attempts are performed in a goroutine off the request path.

                                                                                                        The default is half a second.

                                                                                                        func DialContext

                                                                                                        func DialContext(f func(ctx context.Context, network, addr string) (net.Conn, error)) TransportOption

                                                                                                          DialContext specifies the dial function for creating TCP connections on the outbound. This will override the default dial context, which has a 30 second timeout and respects the KeepAlive option.

                                                                                                          See https://golang.org/pkg/net/http/#Transport.DialContext for details.

                                                                                                          func DisableCompression

                                                                                                          func DisableCompression() TransportOption

                                                                                                            DisableCompression if true prevents the Transport from requesting compression with an "Accept-Encoding: gzip" request header when the Request contains no existing Accept-Encoding value. If the Transport requests gzip on its own and gets a gzipped response, it's transparently decoded in the Response.Body. However, if the user explicitly requested gzip it is not automatically uncompressed.

                                                                                                            func DisableKeepAlives

                                                                                                            func DisableKeepAlives() TransportOption

                                                                                                              DisableKeepAlives prevents re-use of TCP connections between different HTTP requests.

                                                                                                              func IdleConnTimeout

                                                                                                              func IdleConnTimeout(t time.Duration) TransportOption

                                                                                                                IdleConnTimeout is the maximum amount of time an idle (keep-alive) connection will remain idle before closing itself. Zero means no limit.

                                                                                                                Defaults to 15 minutes.

                                                                                                                func InnocenceWindow

                                                                                                                func InnocenceWindow(d time.Duration) TransportOption

                                                                                                                  InnocenceWindow is the duration after the peer connection management loop will suspend suspicion for a peer after successfully checking whether the peer is live with a fresh TCP connection.

                                                                                                                  The default innocence window is 5 seconds.

                                                                                                                  A timeout does not necessarily indicate that a peer is unavailable, but it could indicate that the connection is half-open, that the peer died without sending a TCP FIN packet. In this case, the peer connection management loop attempts to open a TCP connection in the background, once per innocence window, while suspicious of the connection, leaving the peer available until it fails.

                                                                                                                  func KeepAlive

                                                                                                                  func KeepAlive(t time.Duration) TransportOption

                                                                                                                    KeepAlive specifies the keep-alive period for the network connection. If zero, keep-alives are disabled.

                                                                                                                    Defaults to 30 seconds.

                                                                                                                    func Logger

                                                                                                                    func Logger(logger *zap.Logger) TransportOption

                                                                                                                      Logger sets a logger to use for internal logging.

                                                                                                                      The default is to not write any logs.

                                                                                                                      func MaxIdleConns

                                                                                                                      func MaxIdleConns(i int) TransportOption

                                                                                                                        MaxIdleConns controls the maximum number of idle (keep-alive) connections across all hosts. Zero means no limit.

                                                                                                                        func MaxIdleConnsPerHost

                                                                                                                        func MaxIdleConnsPerHost(i int) TransportOption

                                                                                                                          MaxIdleConnsPerHost specifies the number of idle (keep-alive) HTTP connections that will be maintained per host. Existing idle connections will be used instead of creating new HTTP connections.

                                                                                                                          Defaults to 2 connections.

                                                                                                                          func ResponseHeaderTimeout

                                                                                                                          func ResponseHeaderTimeout(t time.Duration) TransportOption

                                                                                                                            ResponseHeaderTimeout if non-zero specifies the amount of time to wait for a server's response headers after fully writing the request (including its body, if any). This time does not include the time to read the response body.

                                                                                                                            func Tracer

                                                                                                                            func Tracer(tracer opentracing.Tracer) TransportOption

                                                                                                                              Tracer configures a tracer for the transport and all its inbounds and outbounds.