storagehelper

README

ccache-go-storage-helper

ccache remote storage helpers are implemented as standalone programs. This module provides the common Go implementation of their IPC protocol and server.

Functionality

• ccache remote storage helper IPC protocol handling
• Unix socket and Windows named-pipe listeners
• Request dispatch for GET, PUT, REMOVE, EXISTS, INFO and STOP requests
• Common CRSH_* environment configuration parsing
• Idle timeout handling and diagnostic reporting
• Optional file logging

The protocol is specified in the ccache remote storage helper specification.

Using the library

logger := storagehelper.NewLogger(os.Getenv("CRSH_LOGFILE"))
defer logger.Close()

config, err := storagehelper.ParseConfig(logger)
// ...
identity := "ccache-storage-example 1.0"
storage, err := newStorageClient(...) // implementation of the storagehelper.Storage interface
// ...
server := storagehelper.NewServer(config, identity, storage, logger)
server.Run()

Development

make test
make format

Documentation

Index

• Constants
• type Attribute
• type Config
  • func ParseConfig(logger *Logger) (*Config, error)
• type Logger
  • func NewLogger(logFile string) *Logger
  • func (l *Logger) Close()
  • func (l *Logger) Logf(format string, args ...any)
• type Server
  • func NewServer(config *Config, identity string, storage Storage, logger *Logger) *Server
  • func (s *Server) Run() error
• type Storage

Constants

const (
	// ProtocolVersion is the ccache remote storage helper protocol version.
	ProtocolVersion = 0x01

	// CapabilityGetPutRemove identifies the GET, PUT and REMOVE capability.
	CapabilityGetPutRemove = 0x00
	// CapabilityInfo identifies the INFO capability.
	CapabilityInfo = 0x01
	// CapabilityExists identifies the EXISTS capability.
	CapabilityExists = 0x02

	// RequestGet identifies a GET request.
	RequestGet = 0x00
	// RequestPut identifies a PUT request.
	RequestPut = 0x01
	// RequestRemove identifies a REMOVE request.
	RequestRemove = 0x02
	// RequestStop identifies a STOP request.
	RequestStop = 0x03
	// RequestInfo identifies an INFO request.
	RequestInfo = 0x04
	// RequestExists identifies an EXISTS request.
	RequestExists = 0x05

	// ResponseOK identifies a successful response.
	ResponseOK = 0x00
	// ResponseNoop identifies a successful response with no operation performed.
	ResponseNoop = 0x01
	// ResponseErr identifies an error response.
	ResponseErr = 0x02

	// PutFlagOverwrite requests that PUT overwrite an existing value.
	PutFlagOverwrite = 0x01
)

Types

type Attribute

type Attribute struct {
	// Key is the attribute name without the leading @ character.
	Key string
	// Value is the attribute value.
	Value string
}

Attribute is a backend-specific key-value attribute supplied by ccache.

type Config

type Config struct {
	// IPCEndpoint is the Unix socket path or Windows named-pipe name on which
	// the helper listens.
	IPCEndpoint string
	// URL is the unparsed CRSH_URL value.
	URL string
	// IdleTimeout is the duration after which an idle helper will exit. Zero
	// disables the idle timeout.
	IdleTimeout time.Duration
	// Attributes contains the backend-specific attributes supplied by ccache
	// (order preserved).
	Attributes []Attribute
	// Diagnostics are sent in response to an INFO request.
	Diagnostics []string
}

Config contains settings common to all storage helpers. Backends interpret URL and Attributes themselves and append any INFO diagnostics to Diagnostics.

func ParseConfig

func ParseConfig(logger *Logger) (*Config, error)

ParseConfig parses the CRSH_* environment variables common to all storage helpers. It validates CRSH_URL, CRSH_IDLE_TIMEOUT and CRSH_NUM_ATTR, but deliberately leaves URL and Attributes for the backend to interpret.

type Logger

type Logger struct {
	// contains filtered or unexported fields
}

Logger writes timestamped diagnostic messages to an optional file.

func NewLogger

func NewLogger(logFile string) *Logger

NewLogger creates a Logger that appends to logFile. Logging is disabled if logFile is empty or cannot be opened.

func (*Logger) Close

func (l *Logger) Close()

Close closes the log file if logging is enabled.

func (*Logger) Logf

func (l *Logger) Logf(format string, args ...any)

Logf writes a formatted, timestamped message. It is safe for concurrent use.

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server serves the ccache remote storage helper protocol over the configured IPC endpoint.

func NewServer

func NewServer(config *Config, identity string, storage Storage, logger *Logger) *Server

NewServer creates a Server using config, identity, storage, and logger. The identity is returned to ccache in response to INFO requests.

func (*Server) Run

func (s *Server) Run() error

Run listens on Config.IPCEndpoint and blocks until the helper receives a STOP request or its idle timeout expires. It returns an error if the IPC listener cannot be created.

type Storage

type Storage interface {
	// Exists reports whether key is present.
	Exists(key []byte) (present bool, err error)
	// Get looks up key. The body return value supplies the stored value, size is
	// the number of bytes in that value and found reports whether key exists.
	// When found is false, Server ignores body and size. When found is true, body
	// must supply the complete value (Server closes it after reading). A negative
	// size means that the size is unknown, so Server buffers the value before
	// responding.
	Get(key []byte) (body io.ReadCloser, size int64, found bool, err error)
	// Put stores value under key. value is valid only for the duration of the
	// call and contains at most size bytes. Server drains any unread bytes when
	// Put returns. If overwrite is false it is only a performance hint: a backend
	// may still overwrite an existing value. Put reports whether it stored a
	// value.
	Put(key []byte, value io.Reader, size int64, overwrite bool) (bool, error)
	// Remove removes key and reports whether it was present.
	Remove(key []byte) (present bool, err error)
}

Storage is implemented by a storage backend. Keys are opaque byte strings supplied by ccache.