Datablock Build Status GoDoc Report Card

Module for reading and caching files into a fixed size cache buffer.

Can also cache calls to os.Stat, for cases of extreme optimization.

Coding style

  • The code shall always be formatted with go fmt.

Online API Documentation

General information

  • Version: 1.0
  • License: MIT
  • Alexander F Rødseth <>
Expand ▾ Collapse ▴




This section is empty.


View Source
var (
	// ErrRemoval is used if a filename that does not exist is attempted to be removed
	ErrRemoval = errors.New("can't remove a file ID that does not exist")

	// ErrNoData is used if no data is attempted to be stored in the cache
	ErrNoData = errors.New("no data")

	// ErrAlreadyStored is used if a given filename has already been stored in the cache
	ErrAlreadyStored = errors.New("file ID is already stored")

	// ErrLargerThanCache is used if the given data is larger than the total cache size
	ErrLargerThanCache = errors.New("data is larger than the the total cache size")

	// ErrEntityTooLarge is used if a maximum size per entity has been set
	ErrEntityTooLarge = errors.New("data is larger than the allowed size")

	// ErrGivenDataSizeTooLarge is returned if the uncompressed size of the given data is too large
	ErrGivenDataSizeTooLarge = errors.New("size of given data is larger than allowed")
View Source
var (
	// EmptyDataBlock is an empty data block
	EmptyDataBlock = &DataBlock{[]byte{}, false, 0, true}


This section is empty.


type DataBlock

type DataBlock struct {
	// contains filtered or unexported fields

    DataBlock represents a block of data that may be compressed

    func NewDataBlock

    func NewDataBlock(data []byte, compressionSpeed bool) *DataBlock

      NewDataBlock creates a new uncompressed data block. compressionSpeed is if speedy compression should be used over compact compression

      func (*DataBlock) Compress

      func (b *DataBlock) Compress() error

        Compress this data block

        func (*DataBlock) Decompress

        func (b *DataBlock) Decompress() error

          Decompress this data block

          func (*DataBlock) Gzipped

          func (b *DataBlock) Gzipped() ([]byte, int, error)

            Gzipped returns the compressed data, length and an error. Will compress if needed.

            func (*DataBlock) HasData

            func (b *DataBlock) HasData() bool

              HasData returns true if there is data present

              func (*DataBlock) IsCompressed

              func (b *DataBlock) IsCompressed() bool

                IsCompressed checks if this data block is compressed

                func (*DataBlock) Length

                func (b *DataBlock) Length() int

                  Length returns the lentgth of the current data (not the length of the original data, but in the current state)

                  func (*DataBlock) MustData

                  func (b *DataBlock) MustData() []byte

                    MustData returns the uncompressed data or an empty byte slice

                    func (*DataBlock) String

                    func (b *DataBlock) String() string

                      String returns the uncompressed data as a string or as an empty string. Same as MustData, but converted to a string.

                      func (*DataBlock) StringLength

                      func (b *DataBlock) StringLength() string

                        StringLength returns the length of the data, represented as a string

                        func (*DataBlock) ToClient

                        func (b *DataBlock) ToClient(w http.ResponseWriter, req *http.Request, name string, canGzip bool, gzipThreshold int)

                          ToClient writes the data to the client. Also sets the right headers and compresses the data with gzip if needed. Set canGzip to true if the http client can handle gzipped data. gzipThreshold is the threshold (in bytes) for when it makes sense to compress the data with gzip

                          func (*DataBlock) UncompressedData

                          func (b *DataBlock) UncompressedData() ([]byte, int, error)

                            UncompressedData returns the the original, uncompressed data, the length of the data and an error. Will decompress if needed.

                            type FileCache

                            type FileCache struct {
                            	// contains filtered or unexported fields

                              FileCache manages a set of bytes as a cache

                              func NewFileCache

                              func NewFileCache(cacheSize uint64, compress bool, maxEntitySize uint64, compressionSpeed bool, maxGivenDataSize uint64) *FileCache

                                NewFileCache creates a new FileCache struct. cacheSize is the total cache size, in bytes. compress is for enabling compression of cache data. maxEntitySize is for setting a per-file maximum size. (0 to disable) compressionSpeed is if speedy compression should be used over compact compression. maxGivenDataSize is the maximum amount of bytes that can be given at once. (0 to disable, 1 MiB is recommended)

                                func (*FileCache) Clear

                                func (cache *FileCache) Clear()

                                  Clear the entire cache

                                  func (*FileCache) IsEmpty

                                  func (cache *FileCache) IsEmpty() bool

                                    IsEmpty checks if the cache is empty

                                    func (*FileCache) Read

                                    func (cache *FileCache) Read(filename string, cached bool) (*DataBlock, error)

                                      For reading files, with optional caching.

                                      func (*FileCache) Stats

                                      func (cache *FileCache) Stats() string

                                        Stats returns formatted cache statistics

                                        type FileStat

                                        type FileStat struct {
                                        	// contains filtered or unexported fields

                                          FileStat can cache calls to os.Stat. This requires that the user wants to assume that no files are removed from the server directory while the server is running, to gain some additional speed (and a tiny bit of memory use for the cache).

                                          func NewFileStat

                                          func NewFileStat(useCache bool, clearStatCacheDelay time.Duration) *FileStat

                                            NewFileStat creates a new FileStat struct, with optional caching. Only use the caching if it is not critical that os.Stat is always correct.

                                            func (*FileStat) Exists

                                            func (fs *FileStat) Exists(path string) bool

                                              Exists check if a given path exists

                                              func (*FileStat) IsDir

                                              func (fs *FileStat) IsDir(path string) bool

                                                IsDir checks if a given path is a directory

                                                func (*FileStat) Sleep

                                                func (fs *FileStat) Sleep(extraSleep time.Duration)

                                                  Sleep for an entire stat cache clear cycle + optional extra sleep time