rpc

Documentation

Overview

Package rpc provides a framework for implementing RPC servers and clients.

RPC wire protocol

The protocol for a particular method is to send an HTTP request with the appropriate request message to an "api" URL with the server type and method name; the response will then be returned. Both request and response are sent as binary protocol buffers, as declared by upspin.io/upspin/proto. The encoding procedure is described in more detail below.

As an example, to call the Put method of the Store server running at store.example.com, send to the URL,

https://store.example.com/api/Store/Put

a POST request with, as payload, an encoded StorePutRequest. The response will be a StorePutResponse. There is such a pair of protocol buffer messages for each method.

For streaming RPC methods, the requests are the same, but the response is sent as the bytes "OK" followed by a series of encoded protocol buffers. Each encoded message is preceded by a four byte, big-endian-encoded int32 that describes the length of the following encoded protocol buffer. The stream is considered closed when the HTTP response stream ends.

If an error occurs while processing a request, the server returns a 500 Internal Server Error status code and the response body contains the error string.

Authentication

The client authenticates itself to the server using special HTTP headers.

In its first request to a given Upspin server, the client presents a signed authentication request as a series of HTTP headers with key 'Upspin-Auth-Request'. The header values are, in order:

the user name,
the server host name,
the current time, and
the R part of an upspin.Signature,
the S part of an upspin.Signature.

The current time is formatted using Go's time.ANSIC presentation:

"Mon Jan _2 15:04:05 2006"

To generate the upspin.Signature, the client concatenates the user name, host name, and formatted time, each string prefixed by its length as a big endian-encoded uint32:

[len(username)][username][len(hostname)][hostname][len(time)][time]

The client then SHA256-hashes that string and signs it using its Factotum.

The server checks the signature and, if valid, returns an authentication token that represents the current session in the 'Upspin-Auth-Token' response header.

In subsequent requests, the client presents that authentication token to the server using the 'Upspin-Auth-Token' request header.

If there is an error validating an authentication request or token, the server returns an error message in the 'Upspin-Auth-Error' response header.

TODO: document the 'Upspin-Proxy-Request' header.

Encoding

The arguments to each method, and the returned values, are encoded as protocol buffers at the top level. These protocol buffers are declared in the package upspin.io/upspin/proto and its .proto definition file, upspin.io/upspin/proto/upspin.proto.

Unlike some other systems that use protocol buffers, Upspin does not use the types defined in upspin.proto internally. The protocol buffer types are used for transport only, and so the internal types must be transcoded for use on the wire.

Some of the transcoding is straightforward. The scalar types Transport, NetAddr, Reference, UserName, PathName, and so on are transmitted as their underlying scalar types, int or string. (Strings are considered scalars here as they are basic types in both Go and protocol buffers.)

The structured types Endpoint, Location, Refdata and User themselves contain only scalar types and so are converted field by field. There are helper functions in the upspin.io/upspin/proto package to automate the conversion of these types. For instance, the function proto.UpspinLocations converts from a slice of protocol buffer Location structs to a slice of the internal type, upspin.Location.

The DirEntry type and its component DirBlock are treated a little differently. In the upspin.io/upspin package, which defines these types, there are Marshal and Unmarshal methods that encode and decode these types to byte slices. (The details of these encodings are listed below.) Thus within the protocol buffer types, these appear as proto type "bytes".

These types are handled differently so the DirServer can serialize them easily for storage.

The errors.Error type is also handled specially so its full functionality can be accessed across the wire. Methods defined in the errors package do the marshaling and unmarshaling of Error values.

When these types appear in other protocol buffer types, such as proto.Event, they are encoded as the byte slices generated by the process described below.

How to Marshal

The fields of a structured type that marshals to a byte slice (one of DirBlock, DirEntry, or Error) are encoded as follows using the methods of the Go package encoding/binary to marshal their underlying type:

int:
	varint
string:
	varint count n followed by n bytes
byte slice:
	varint count n followed by n bytes

Encoding of DirBlock

The fields of a DirBlock are encoded using the rules above in the following order:

Location.Endpoint.Transport: 1 byte
Location.Endpoint.NetAddr: as string
Location.Reference: as string
Offset: as int
Size: as int
Packdata: as bytes

Encoding of DirEntry

The fields of a DirEntry are encoded using the rules above in the following order:

SignedName: as string
Packing: 1 byte
Time: as int
Blocks:
	len(Blocks): as int
	Blocks: n DirBlocks as described above.
Packdata: as bytes
Link: as string
Writer: as string
Name: if different from SignedName, as string
	Otherwise, 1 byte with value 0.
Attr: 1 byte
Sequence: as int

Encoding of Error

The fields of an Error are encoded using the rules above in the following order.

Path: as string
User: as string
Op: as string
Kind: as int
Error:
	If of type Error, as described here.
	Otherwise the value of Error.Error() as string.

The Error value is always marshaled by address (*Error); if the pointer is nil, nothing is marshaled. Similarly, for a nil Error.Err field, nothing is marshaled.

Index

• Constants
• func CertPoolFromConfig(cfg upspin.Config) (*x509.CertPool, error)
• func ClearSession(authToken string)
• func NewServer(cfg upspin.Config, svc Service) http.Handler
• func PublicUserKeyService(cfg upspin.Config) func(userName upspin.UserName) (upspin.PublicKey, error)
• type Client
  • func NewClient(cfg upspin.Config, netAddr upspin.NetAddr, security SecurityLevel, ...) (Client, error)
• type Method
• type ResponseChan
• type SecurityLevel
• type Service
• type Session
  • func GetSession(authToken string) Session
  • func NewSession(user upspin.UserName, expiration time.Time, authToken string, ...) Session
• type Stream
• type UnauthenticatedMethod

Constants

const (
	// Secure as the security argument to NewClient requires TLS
	// connections using CA certificates.
	Secure = SecurityLevel(iota + 1)

	// NoSecurity as the security argument to NewClient requires
	// connections with no authentication or encryption.
	NoSecurity
)

Functions

func CertPoolFromConfig

func CertPoolFromConfig(cfg upspin.Config) (*x509.CertPool, error)

CertPoolFromConfig returns the TLS certificate pool for this config.

func ClearSession

func ClearSession(authToken string)

ClearSession removes a session associated with a given auth token from the cache.

func NewServer

func NewServer(cfg upspin.Config, svc Service) http.Handler

NewServer returns a new Server that uses the given ServerConfig.

func PublicUserKeyService

func PublicUserKeyService(cfg upspin.Config) func(userName upspin.UserName) (upspin.PublicKey, error)

PublicUserKeyService returns a Lookup function that looks up user's public keys. The lookup function returned is bound to a well-known public Upspin user service.

Types

type Client

type Client interface {
	Close()

	// Invoke calls the given RPC method ("Server/Method") with the
	// given request message and decodes the response into the given
	// response message.
	// For regular one-shot methods, the stream and done channels must be nil.
	// For streaming RPC methods, the caller should provide a nil response
	// and non-nil stream and done channels.
	// TODO: remove stream param and add method InvokeStream.
	Invoke(method string, req, resp pb.Message, stream ResponseChan, done <-chan struct{}) error

	// InvokeUnauthenticated invokes an unauthenticated one-shot RPC method
	// ("Server/Method") with request body req. Upon success, resp, if nil,
	// contains the server's reply, if any.
	InvokeUnauthenticated(method string, req, resp pb.Message) error
}

Client is a partial upspin.Service that uses HTTP as a transport and implements authentication using out-of-band headers.

func NewClient

func NewClient(cfg upspin.Config, netAddr upspin.NetAddr, security SecurityLevel, proxyFor upspin.Endpoint) (Client, error)

NewClient returns a new client that speaks to an HTTP server at a net address. The address is expected to be a raw network address with port number, as in domain.com:5580. The security level specifies the expected security guarantees of the connection. If proxyFor is an assigned endpoint, it indicates that this connection is being used to proxy request to that endpoint.

type Method

type Method func(s Session, reqBytes []byte) (pb.Message, error)

Method describes an authenticated RPC method.

type ResponseChan

type ResponseChan interface {
	// Send sends a proto-encoded message to the client.
	// If done is closed, the send should abort.
	Send(b []byte, done <-chan struct{}) error

	// Error sends an error condition to the client.
	Error(error)

	// Close closes the response channel.
	Close()
}

ResponseChan describes a mechanism to report streamed messages to a client (the caller of Client.Invoke). Typically this interface should wrap a channel that carries decoded protocol buffers.

type SecurityLevel

type SecurityLevel int

SecurityLevel defines the security required of a connection.

type Service

type Service struct {
	// The name of the service, which forms the first path component of any
	// HTTP request.
	Name string

	// The RPC methods to serve.
	Methods map[string]Method

	// The RPC methods to serve without validating sessions.
	UnauthenticatedMethods map[string]UnauthenticatedMethod

	// The streaming RPC methods to serve.
	Streams map[string]Stream

	// Lookup is KeyServer.Lookup function that should be used for key
	// lookups during authentication.
	// If nil, PublicUserKeyService will be used.
	Lookup func(userName upspin.UserName) (upspin.PublicKey, error)
}

Service describes an RPC service.

type Session

type Session interface {
	// User returns the user name present in the session. It may be empty. Note: user, even when present,
	// might not be authenticated.
	User() upspin.UserName

	// Err reports the status of the session.
	Err() error

	// Expires returns the expiration time of the session, in UTC.
	Expires() time.Time

	// ProxiedEndpoint returns the endpoint for which this session is a proxy.
	// If we aren't proxying it returns an Unassigned endpoint.
	ProxiedEndpoint() upspin.Endpoint
}

Session contains information about the connection and the authenticated user, if any.

func GetSession

func GetSession(authToken string) Session

GetSession returns a session associated with an auth token, if one has been cached or nil otherwise.

func NewSession

func NewSession(user upspin.UserName, expiration time.Time, authToken string, proxyFor *upspin.Endpoint, err error) Session

NewSession creates a new session with the given contents.

type Stream

type Stream func(s Session, reqBytes []byte, done <-chan struct{}) (<-chan pb.Message, error)

Stream describes an authenticated streaming RPC method.

type UnauthenticatedMethod

type UnauthenticatedMethod func(reqBytes []byte) (pb.Message, error)

UnauthenticatedMethod describes an RPC method that does not require server-side authentication.