transparent

README

transparent

Transparent cache and distributed commit to key-value store written in Go.

Documentation

Installation

go get github.com/juntaki/transparent

Basic usage

First, create layers and stack them with Stack.Stack(). This example adds LRU memory cache and filesystem cache to dummy source layer.

	cacheLayer1, _ := lru.NewCache(10, 100)
	cacheLayer2 := filesystem.NewCache(10, "/tmp")
	sourceLayer := test.NewSource(10)

	stack := transparent.NewStack()
	stack.Stack(sourceLayer)
	stack.Stack(cacheLayer2)
	stack.Stack(cacheLayer1)

If you manipulate the Stack, the value will be transmitted from top layer to the bottom layer transparently.

	stack.Set("key", []byte("value"))
	stack.Sync()
    
    // value, _ = cacheLayer1.Get("key") // "value"
	// value, _ = cacheLayer2.Get("key") // "value"
	// value, _ = sourceLayer.Get("key") // "value"

	value, _ := stack.Get("key")
	fmt.Printf("%s\n", value)            // "value"

For details, please refer to [Godoc] (https://godoc.org/github.com/juntaki/transparent).

Documentation

Overview

Package transparent is a library that provides transparent operations for key-value stores. Transparent Layer is tearable on Stack. In addition to caching, it is also possible to transparently use a layer of synchronization between distributed systems. See subpackage for implementation.

Example

package main

import (
	"fmt"

	"github.com/juntaki/transparent"
	"github.com/juntaki/transparent/filesystem"
	"github.com/juntaki/transparent/lru"
	"github.com/juntaki/transparent/test"
)

func main() {
	var err error
	cacheLayer1, _ := lru.NewCache(10, 100)
	cacheLayer2 := filesystem.NewCache(10, "/tmp")
	sourceLayer := test.NewSource(10)

	stack := transparent.NewStack()
	stack.Stack(sourceLayer)
	stack.Stack(cacheLayer2)
	stack.Stack(cacheLayer1)

	stack.Start()
	defer stack.Stop()

	stack.Set("key", []byte("value"))
	stack.Sync()

	value, _ := stack.Get("key")
	fmt.Printf("%s\n", value)

	value, _ = cacheLayer1.Get("key")
	fmt.Printf("%s\n", value)

	value, err = cacheLayer2.Get("key")
	if err != nil {
		fmt.Println(err)
	}
	fmt.Printf("%s\n", value)
	value, err = sourceLayer.Get("key")
	if err != nil {
		fmt.Println(err)
	}
	fmt.Printf("%s\n", value)
}

Output:

value
value
value
value

Index

• type BackendReceiver
• type BackendStorage
• type BackendTransmitter
• type KeyNotFoundError
  • func (e *KeyNotFoundError) Error() string
• type Layer
  • func NewLayerCache(bufferSize int, storage BackendStorage) (Layer, error)
  • func NewLayerConsensus(t BackendTransmitter) (Layer, error)
  • func NewLayerReceiver(Receiver BackendReceiver) Layer
  • func NewLayerSource(storage BackendStorage) (Layer, error)
  • func NewLayerTransmitter(Transmitter BackendTransmitter) Layer
• type Message
• type MessageType
• type Stack
  • func NewStack() *Stack
  • func (s *Stack) Stack(l Layer) error
  • func (s *Stack) Start() error
  • func (s *Stack) Stop() error

Examples

• Package

Types

type BackendReceiver

type BackendReceiver interface {
	Start() error
	Stop() error
	SetCallback(func(m *Message) (*Message, error)) error
}

BackendReceiver is interface from another system Callback function will executed with received Message.

type BackendStorage

type BackendStorage interface {
	Get(key interface{}) (value interface{}, err error)
	Add(key interface{}, value interface{}) error
	Remove(key interface{}) error
}

BackendStorage defines the interface that backend data storage.

type BackendTransmitter

type BackendTransmitter interface {
	Request(operation *Message) (*Message, error)
	Start() error
	Stop() error
	SetCallback(func(m *Message) (*Message, error)) error
}

BackendTransmitter is interface to another system Request transfer an operation as Message If request will be processed asynchronously, callback function should executed with reply Message.

type KeyNotFoundError

type KeyNotFoundError struct {
	Key interface{}
}

KeyNotFoundError means specified key is not found in the layer

func (*KeyNotFoundError) Error

func (e *KeyNotFoundError) Error() string

type Layer

type Layer interface {
	Set(key interface{}, value interface{}) error
	Get(key interface{}) (value interface{}, err error)
	Remove(key interface{}) error
	Sync() error
	// contains filtered or unexported methods
}

Layer is stackable function

func NewLayerCache

func NewLayerCache(bufferSize int, storage BackendStorage) (Layer, error)

NewLayerCache returns LayerCache. LayerCache wraps BackendStorage. It Get/Set key-value to BackendStorage, and asynchronously apply same operation to Next Layer. It must be Stacked on a Layer.

func NewLayerConsensus

func NewLayerConsensus(t BackendTransmitter) (Layer, error)

NewLayerConsensus returns LayerConsensus. LayerConsensus wraps BackendTransmitter. It send Set operation and key-value to multiple Stacks asynchronously and Get key-value from Next Layer. It must be Stacked on a Layer.

User program A       User program B
      |                    |

----------------------------------------- | transparent.Consensus | -------------------- -------------------- |transparent.Source| |transparent.Source| -------------------- --------------------

func NewLayerReceiver

func NewLayerReceiver(Receiver BackendReceiver) Layer

NewLayerReceiver returns LayerReceiver. LayerReceiver wraps BackendReceiver. It receive operation and key-value from another Stack. This layer must be the top of Stack.

func NewLayerSource

func NewLayerSource(storage BackendStorage) (Layer, error)

NewLayerSource returns LayerSource. LayerSource wraps BackendStorage. It Get/Set key-value to BackendStorage. This layer must be the bottom of Stack.

func NewLayerTransmitter

func NewLayerTransmitter(Transmitter BackendTransmitter) Layer

NewLayerTransmitter returns LayerTransmitter. LayerTransmitter wraps BackendTransmitter. It send operation and key-value to another Stack. This layer must be the bottom of Stack.

type Message

type Message struct {
	Key     interface{}
	Value   interface{}
	Message MessageType
	UUID    string
}

Message is layer operation

type MessageType

type MessageType int

MessageType of operation

const (
	MessageSet MessageType = iota
	MessageGet
	MessageRemove
	MessageSync
)

MessageType of operation

type Stack

type Stack struct {
	Layer
	// contains filtered or unexported fields
}

Stack is stacked layer

func NewStack

func NewStack() *Stack

NewStack returns Stack

func (*Stack) Stack

func (s *Stack) Stack(l Layer) error

Stack add the layer to Stack

func (*Stack) Start

func (s *Stack) Start() error

Start initialize all stacked layers

func (*Stack) Stop

func (s *Stack) Stop() error

Stop clean up all stacked layers