types

package
v0.1.15 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 27, 2016 License: GPL-3.0 Imports: 15 Imported by: 47

Documentation

Overview

Package types describes few of the essential types used throughout the application.

Package types describes few of the essential types used throughout the application.

Index

Constants

View Source 
const ObjectIDHashSize = sha1.Size

ObjectIDHashSize is the size of the byte array that contains the object hash.

View Source 
const ObjectIndexHashSize = ObjectIDHashSize + 4 // sizeof(uint32)

ObjectIndexHashSize is the size of the byte array that contains the object hash.

Variables

View Source 
var (
	ErrAlreadyInCache = errors.New("Object already in cache")
)

Exported errors

Functions

This section is empty.

Types

type App 

type App interface {
	// Stats returns application wide stats
	Stats() AppStats

	// Started returns the time at which the app was started
	Started() time.Time

	// Version returns a string representation of the version of the app
	Version() AppVersion

	// GetLocationFor returns the Location that mathes the provided host and path
	GetLocationFor(host, path string) *Location

	// GetUpstream gets an upstream by it's id, nil is returned if no such is defined
	GetUpstream(id string) Upstream
}

App is an interface for an application to implement

type AppStats 

type AppStats struct {
	Requests, Responded, NotConfigured uint64
}

AppStats are stats for the whole application

type AppVersion 

type AppVersion struct {
	Dirty     bool
	Version   string
	GitHash   string
	GitTag    string
	BuildTime time.Time
}

AppVersion is struct representing an App version

func (AppVersion) String 

func (a AppVersion) String() string

type BytesSize 

type BytesSize uint64

BytesSize represents size written in string format. Examples: "1m", "20g" etc. Its main purpose is to be stored and loaded from json.

func BytesSizeFromString 

func BytesSizeFromString(str string) (BytesSize, error)

BytesSizeFromString parses bytes size such as "1m", "15g" to BytesSize struct.

func (BytesSize) Bytes 

func (b BytesSize) Bytes() uint64

Bytes returns bytes number as uint64

func (*BytesSize) MarshalJSON 

func (b *BytesSize) MarshalJSON() ([]byte, error)

MarshalJSON is needed for automatic marshalling of BytesSize fields in the JSON configuration.

func (*BytesSize) UnmarshalJSON 

func (b *BytesSize) UnmarshalJSON(buff []byte) error

UnmarshalJSON is needed for automatic unmarshalling of BytesSize fields in the JSON configuration.

type CacheAlgorithm 

type CacheAlgorithm interface {

	// Lookup returns wheather this object is in the cache or not
	Lookup(*ObjectIndex) bool

	// ShouldKeep is called to signal that this ObjectIndex has been stored
	ShouldKeep(*ObjectIndex) bool

	// AddObject adds this ObjectIndex to the cache. Returns an error when
	// the object is in the cache already.
	AddObject(*ObjectIndex) error

	// PromoteObject is called every time this part of a file has been used
	// to satisfy a client request
	PromoteObject(*ObjectIndex)

	// ConsumedSize returns the full size of all files currently in the cache
	ConsumedSize() BytesSize

	// Stats returns statistics for this cache algorithm
	Stats() CacheStats

	// Remove all of the provided object indexes from the cache.
	Remove(...*ObjectIndex)

	// ChangeConfig changes the changeable parts of the a CacheAlgorithm:
	// the timeout and count for removing objects in bulk
	// and the count of objects it contains. Automatically resizing the algorithm
	// if it's required
	ChangeConfig(bulkTimeout, bulkCount, objectCount uint64)

	// SetLogger changes the Logger of the CacheAlgorithm
	SetLogger(Logger)
}

CacheAlgorithm interface defines how a cache should behave

type CacheStats 

type CacheStats interface {

	// CacheHitPrc returns a string such as '53%' which represents the cache hit
	// ratio of this cache. Basically this number is (Hits()/Requests()) * 100.
	CacheHitPrc() string

	// ID is a way of identifing this particular cache zone
	ID() string

	// Hits returns the number of cache hits this cache has generated
	Hits() uint64

	// Requests returns the number of lookups in the cache
	Requests() uint64

	// Objects returns the number of cache object at the moment. These are the actual
	// objects on the disk. Not maximum possible objects
	Objects() uint64

	// Size returns the consumed space in bytes for this cache
	Size() BytesSize
}

CacheStats is the common interface that is used so every cache can generate a Stats object which is to be used in the server status page.

type CacheZone 

type CacheZone struct {
	ID        string
	PartSize  BytesSize
	Algorithm CacheAlgorithm
	Scheduler Scheduler
	Storage   Storage
}

CacheZone is the combination of a Storage for storing object parts and an `CacheAlgorithm` which determines what should be stored.

type FilePath 

type FilePath string

FilePath represents the string path to a file on the local filesystem. It is mainly used for flag arguments because it implements the flag.Value interface and resolves the absolute path to the file when the Set() method is used. Thus, if the current working directory is changed during the execution of the program, the path still remains valid.

func (*FilePath) Set 

func (fp *FilePath) Set(path string) error

Set converts the supplied argument to an absolute path. It does NOT check a file or folder exists at this path.

func (*FilePath) String 

func (fp *FilePath) String() string

type IncomingConn added in v0.1.11 

type IncomingConn interface {
	// see net.Conn.RemoteAddr
	ID() string
	// will throttle the connection at the given speed
	SetThrottle(speed BytesSize)
	// stop throttling
	RemoveThrottling()
}

IncomingConn is a type that represents incoming connections

type Location 

type Location struct {
	Name                  string
	Handler               RequestHandler
	CacheKey              string
	CacheDefaultDuration  time.Duration
	CacheKeyIncludesQuery bool
	Cache                 *CacheZone //!TODO: move to the cache handler settings (plus all Cache* settings)
	Upstream              Upstream
	Logger                Logger
}

Location links a config location to its cache algorithm and a storage object.

func (*Location) NewObjectIDForURL 

func (l *Location) NewObjectIDForURL(u *url.URL) *ObjectID

NewObjectIDForURL returns new ObjectID from the provided URL

func (*Location) String 

func (l *Location) String() string

type Logger 

type Logger interface {
	// Log emits this message to the log stream.
	Log(v ...interface{})

	// Logf emits this message to the log stream. Supports fmt.Printf formating.
	Logf(format string, args ...interface{})

	// Debug emits this message to the debug stream.
	Debug(v ...interface{})

	// Debuggerebugf emits this message to the debug stream. Supports fmt.Printf formatting.
	Debugf(format string, args ...interface{})

	// Error emits this message to the error stream.
	Error(v ...interface{})

	// Errorf emits this message to the error stream. Supports fmt.Printf formatting.
	Errorf(format string, args ...interface{})

	// Fatal will print the message to the error stream and halt the program.
	Fatal(v ...interface{})

	// Fatalf is similar to Fatal but supports formatting. Support fmt.Printf formating.
	Fatalf(format string, v ...interface{})
}

Logger is the common interface that all nedomi loggers should implement.

type NilNextHandler 

type NilNextHandler string

NilNextHandler is an error type to be returned when a next handler is required but it's nil

func (NilNextHandler) Error 

func (n NilNextHandler) Error() string

type NotNilNextHandler 

type NotNilNextHandler string

NotNilNextHandler is an error type to be returned when a next handler is not nil but it should be

func (NotNilNextHandler) Error 

func (n NotNilNextHandler) Error() string

type ObjectID 

type ObjectID struct {
	// contains filtered or unexported fields
}

ObjectID represents a cached file.

func NewObjectID 

func NewObjectID(cacheKey, path string) *ObjectID

NewObjectID creates and returns a new ObjectID.

func (*ObjectID) CacheKey 

func (oid *ObjectID) CacheKey() string

CacheKey returns the object's cache key.

func (*ObjectID) Hash 

func (oid *ObjectID) Hash() ObjectIDHash

Hash returns the pre-calculated sha1 hash of the object id.

func (*ObjectID) MarshalJSON 

func (oid *ObjectID) MarshalJSON() ([]byte, error)

MarshalJSON is used to help the JSON library marshal the unexported vars.

func (*ObjectID) Path 

func (oid *ObjectID) Path() string

Path returns the object's path.

func (*ObjectID) StrHash 

func (oid *ObjectID) StrHash() string

StrHash returns the sha1 hash of the object id in hex format.

func (*ObjectID) String 

func (oid *ObjectID) String() string

func (*ObjectID) UnmarshalJSON 

func (oid *ObjectID) UnmarshalJSON(buf []byte) error

UnmarshalJSON is used to help the JSON library unmarshal the unexported vars.

type ObjectIDHash 

type ObjectIDHash [ObjectIDHashSize]byte

ObjectIDHash is the fixed-width byte array that represents an ObjectID hash.

type ObjectIndex 

type ObjectIndex struct {
	ObjID *ObjectID
	Part  uint32
}

ObjectIndex represents a particular index in a file

func (*ObjectIndex) Hash 

func (oi *ObjectIndex) Hash() ObjectIndexHash

Hash returns the pre-calculated sha1 of the ObjectID with concatinated the 4 bytes identifying the part

func (*ObjectIndex) HashStr 

func (oi *ObjectIndex) HashStr() string

HashStr returns a hash

func (*ObjectIndex) String 

func (oi *ObjectIndex) String() string

type ObjectIndexHash 

type ObjectIndexHash [ObjectIndexHashSize]byte

ObjectIndexHash is the fixed-width byte array that represents an ObjectID hash.

type ObjectMetadata 

type ObjectMetadata struct {

	// The ObjectID for this file. It is used for identifying the object throughout
	// the cache layers in nedomi.
	ID *ObjectID

	// The time of the first request/response for this object as unix timestamp.
	ResponseTimestamp int64

	// Status code of the first proxied response for this object.
	Code int

	// The object size in bytes. Normally this should correspond to the
	// upstream's Content-Length header.
	Size uint64

	// HTTP headers which were received from the upstream and which we should
	// pass down for this object for any subsequent request.
	Headers http.Header

	// The time at wich this object can be considered stale. After this time
	// the object must be revalidated or discarded. This value is a unix timestamp.
	ExpiresAt int64
}

ObjectMetadata represents all the needed metadata of a cacheable object.

type RequestHandler 

type RequestHandler interface {

	// RequestHandle is function similar to the http.ServeHTTP. It differs only
	// in that it has context as an extra argument.
	RequestHandle(context.Context, http.ResponseWriter, *http.Request)
}

RequestHandler interface defines the RequestHandle funciton. All nedomi handle modules must implement this interface.

type RequestHandlerFunc 

type RequestHandlerFunc func(context.Context, http.ResponseWriter, *http.Request)

The RequestHandlerFunc type is an adapter to allow the use of ordinary functions as request handlers. If f is a function with the appropriate signature, HandlerFunc(f) is a Handler object that calls f.

func (RequestHandlerFunc) RequestHandle 

func (rhf RequestHandlerFunc) RequestHandle(ctx context.Context, w http.ResponseWriter, req *http.Request)

RequestHandle with rhf(ctx, w, req)

type RequestID added in v0.1.9 

type RequestID []byte

RequestID represent a request id

type ScheduledCallback added in v0.1.14 

type ScheduledCallback func(Logger)

ScheduledCallback is the type of the function that Scheduler will callback when scheduled

type Scheduler 

type Scheduler interface {
	// AddEvent schedules the passed callback to be executed at the supplied time.
	AddEvent(key ObjectIDHash, callback ScheduledCallback, in time.Duration)

	// Contains checks whether an event with the supplied key is scheduled.
	Contains(key ObjectIDHash) bool

	// SetLogger changes the logger of the scheduler
	SetLogger(Logger)
}

Scheduler efficiently manages and executes callbacks at specified times.

type Storage 

type Storage interface {
	// Returns the maximum part size for the storage.
	PartSize() uint64

	// Returns the metadata for this object, it it is present. If the requested
	// metadata is not on the storage, it returns os.ErrNotExist.
	GetMetadata(id *ObjectID) (*ObjectMetadata, error)

	// Returns an io.ReadCloser instance that will read the specified part of
	// the object, if it is present. If the requested part is not on the
	// storage, it will return os.ErrNotExist.
	GetPart(id *ObjectIndex) (io.ReadCloser, error)

	// GetAvailableParts returns types.ObjectIndexMap including all the available
	// parts of for the object specified by the provided objectMetadata
	GetAvailableParts(id *ObjectID) ([]*ObjectIndex, error)

	// Saves the supplied metadata to the storage.
	SaveMetadata(m *ObjectMetadata) error

	// Saves the contents of the supplied object part to the storage.
	SavePart(index *ObjectIndex, data io.Reader) error

	// Discard an object and its metadata from the storage.
	Discard(id *ObjectID) error

	// Discard the specified part of an Object from the storage.
	DiscardPart(index *ObjectIndex) error

	// Iterate iterates over the storage objects and passes them and information
	// about their parts to the supplied callback function. It is used for
	// restoring the state after the service has been restarted. When the
	// callback returns false, the iteration stops.
	Iterate(callback func(*ObjectMetadata, ...*ObjectIndex) bool) error

	// SetLogger changes the Logger of the Storage
	SetLogger(Logger)
}

Storage represents a single unit of storage.

type SyncLogger added in v0.1.14 

type SyncLogger struct {
	// contains filtered or unexported fields
}

SyncLogger is a container for a logger that can be setted and getted safely from multiple goroutines

func (*SyncLogger) GetLogger added in v0.1.14 

func (s *SyncLogger) GetLogger() (l Logger)

GetLogger get the Logger

func (*SyncLogger) SetLogger added in v0.1.14 

func (s *SyncLogger) SetLogger(l Logger)

SetLogger set the Logger

type Upstream 

type Upstream interface {
	Do(ctx context.Context, req *http.Request) (*http.Response, error)

	CancelRequest(*http.Request)

	GetAddress(string) (*UpstreamAddress, error)
}

Upstream represents an object that is used by the proxy handler for making requests to the configured upstream server or servers.

type UpstreamAddress 

type UpstreamAddress struct {
	url.URL
	Hostname    string
	Port        string
	OriginalURL *url.URL
	Weight      uint32
}

UpstreamAddress represents a resolved upstream address

func (*UpstreamAddress) String added in v0.1.2 

func (ua *UpstreamAddress) String() string

type UpstreamBalancingAlgorithm 

type UpstreamBalancingAlgorithm interface {

	// Set is used to specify all the possible upstream addresses the algorithm
	// has to choose from.
	Set([]*UpstreamAddress)

	// Get returns a specific address, according to the supplied path.
	Get(string) (*UpstreamAddress, error)
}

UpstreamBalancingAlgorithm encapulates thread-safe methods that are used for balancing user requests between a set of upstream addresses. That is done according to the specific balancing algorithm implementation.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.