almostio

README

Almostio

Almost usable utility classes that help with io and byte processing.

Marshal

Reader and writer for a given format.

FixedSizeWriter

Forwards at most maxCapacity byte to the underlying writer

MultiWriteCloser

MultiWriteCloser is similar to the io.MultiWriter, but with the Close() method.

Overlay

Overlay is an extra layer between the filesystem (io) and the user code. For example, when you want to keep files with extra long filenames with uncommon characters. Allows to workaround several filesystem limitations while keeping files stored in the system as is and accessible by other programs.

Example 1: Http downloader cache

package main

import (
        "fmt"
        "io"
        "os"

        ol "github.com/lanseg/golang-commons/almostio"
)

func main() {
        lo, err := ol.NewLocalOverlay("cache", ol.NewJsonMarshal[ol.OverlayMetadata]())
        if err != nil {
                fmt.Printf("Cannot create overlay: %s\n", err)
                os.Exit(-1)
        }

        afile := "http://someurl.domain/привет こんにちは"
        ow, _ := lo.OpenWrite(afile)
        ow.Write([]byte("Hello world"))
        ow.Close()

        r, _ := lo.OpenRead(afile)
        data, _ := io.ReadAll(r)
        fmt.Printf("Data: %s\n", string(data))
}

And in the result you will get a new folder with the following file structure:

./cache
./cache/8bfb0bf7_http_someurl.domain_
./cache/.overlay
./cache/.overlay/metadata.json

And the metadata.json with the system information:

{
    "fileMetadata": {
        "http://someurl.domain/привет こんにちは": {
            "name": "http://someurl.domain/привет こんにちは",
            "localName": "8bfb0bf7_http_someurl.domain_",
            "sha256": "64ec88ca00b268e5ba1a35678a1b5316d212f4f366b2477232534a8aeca37f3c",
            "mime": "text/plain; charset=utf-8"
        }
    }
}

Documentation

Index

• func BenchmarkOverlayPerformance(bt *testing.B)
• func FixedSizeWriter(w io.Writer, maxCapacity int) io.Writer
• func NopWriteCloser(w io.Writer) io.WriteCloser
• type FileMetadata
• type Marshaller
  • func NewJsonMarshal[T any]() *Marshaller[T]
• type MultiWriteCloser
  • func NewMultiWriteCloser() *MultiWriteCloser
  • func (mw *MultiWriteCloser) AddWriteCloser(w io.WriteCloser) *MultiWriteCloser
  • func (mw *MultiWriteCloser) AddWriter(w io.Writer) *MultiWriteCloser
  • func (mw *MultiWriteCloser) Close() error
  • func (mw *MultiWriteCloser) SetOnClose(onClose func()) *MultiWriteCloser
  • func (mw *MultiWriteCloser) Write(b []byte) (int, error)
• type Overlay
  • func NewLocalOverlay(root string, marshaller *Marshaller[OverlayMetadata]) (Overlay, error)
• type OverlayMetadata

Functions

func BenchmarkOverlayPerformance

func BenchmarkOverlayPerformance(bt *testing.B)

func FixedSizeWriter

func FixedSizeWriter(w io.Writer, maxCapacity int) io.Writer

FixedSizeWriter creates writer that forwards at most maxCapacity byte to the underlying writer.

func NopWriteCloser

func NopWriteCloser(w io.Writer) io.WriteCloser

Types

type FileMetadata

type FileMetadata struct {
	Name      string `json:"name"`
	LocalName string `json:"localName"`
	Sha256    string `json:"sha256"`
	Mime      string `json:"mime"`
}

type Marshaller

type Marshaller[T any] struct {
	Marshal   func(*T) ([]byte, error)
	Unmarshal func([]byte) (*T, error)
}

Marshaller contains reader and writer for a given format. Used when you need to write same data into different file types (e.g. json, binary, text...)

func NewJsonMarshal

func NewJsonMarshal[T any]() *Marshaller[T]

NewJsonMarshal creates a reader/writer for Json file type

type MultiWriteCloser

type MultiWriteCloser struct {
	io.WriteCloser
	// contains filtered or unexported fields
}

MultiWriteCloser is similar to the io.MultiWriter, but with the Close() method.

func NewMultiWriteCloser

func NewMultiWriteCloser() *MultiWriteCloser

NewMultiWriteCloser creates default MultiWriteCloser with empty writers and no onClose function.

func (*MultiWriteCloser) AddWriteCloser

func (mw *MultiWriteCloser) AddWriteCloser(w io.WriteCloser) *MultiWriteCloser

AddWriterCloser adds another writerCloser that will accept data written to the MultiWriteCloser. Its Close() method invoked when parent onCall is invoked.

func (*MultiWriteCloser) AddWriter

func (mw *MultiWriteCloser) AddWriter(w io.Writer) *MultiWriteCloser

AddWriter adds another writer that will accept data written to the MultiWriteCloser.

func (*MultiWriteCloser) Close

func (mw *MultiWriteCloser) Close() error

Close invokes "Close" for all underlying WriteClosers, returns an error if any of them fails.

func (*MultiWriteCloser) SetOnClose

func (mw *MultiWriteCloser) SetOnClose(onClose func()) *MultiWriteCloser

SetOnClose configures a function that is called after all child WriteClosers successfuly closed.

func (*MultiWriteCloser) Write

func (mw *MultiWriteCloser) Write(b []byte) (int, error)

Write writes given bytes to all the underlying writers.

type Overlay

type Overlay interface {
	OpenRead(name string) (io.ReadCloser, error)
	OpenWrite(name string) (io.WriteCloser, error)
	GetMetadata(names []string) []*FileMetadata
}

Overlay is an extra layer between the filesystem (io) and the user code.

func NewLocalOverlay

func NewLocalOverlay(root string, marshaller *Marshaller[OverlayMetadata]) (Overlay, error)

type OverlayMetadata

type OverlayMetadata struct {
	FileMetadata map[string]*FileMetadata `json:"fileMetadata"`
}

OverlayMetadata contains a system information for the overlay (e.g. file list)