commons

README

go-aws-commons - JakartaCommons meets Go, for lack of better naming

Henry's Golang multi-module workspace containing various libraries to make using AWS just a little bit more fun.

Available as their own module:

• config-cache (configcache): single AWS config cache to make using package-level methods easier.
• ddb-mapper: add optimistic locking and auto-generated timestamps via struct tags.
  • gin-sessions: replaces github.com/gin-contrib/sessions with ddb-mapper integration.
• gin-caching-response-headers (cachingheaders): sets caching response headers (Cache-Control, ETag, and/or Last-Modified) on the gin response.
• gin-json-abort (abort): provides package-level methods to help abort a gin request using JSON response as well as logging.
• gin-metrics (ginmetrics): replaces gin.Logger and gin.Recovery with metrics.Metrics integration.
• gin-preconditions (preconditions): provides helper methods to parse and compare conditional headers such as If-Match, If-None-Match, If-Modified-Since, and If-Unmodified-Since.
• gin-s3-proxy (proxy): a very simple Gin middleware acting as an S3 proxy.
• lambda: Lambda handler wrappers with sensible defaults and metrics integration.
  • function-url (functionurl): provides Lambda wrappers for Function URL gin handlers in either BUFFERED or STREAMING mode.
  • getenv: decouple how to retrieve a variable of any type (usually string or binary) via AWS Parameter Store and Secrets Lambda extension.
• metrics: logging latency metrics and other custom counters.
• opaque-token (token): convert DynamoDB last evaluated key to opaque token; create and validate CSRF tokens.
• s3reader: implements io.ReadSeeker, io.ReaderAt, and io.WriterTo using S3 ranged GetObject.
• s3writer: implements io.Writer and io.ReaderFrom for uploading to S3.
• scale-in-protection (sip): protect EC2 AutoScaling instances from being scaled down while busy.
• slogging: attach/retrieve to/from context; JSON and error (with stack trace) slog.Value implementations.
• sri: Subresource Integrity (SRI) computation and verification.
• tspb: Terminal-Safe Progress Bar (TSPB); when you want your program to show progress bar in interactive mode (with terminal), but log normally otherwise.

Available as package in this module (commons):

• args: iterator to scan text lines from multiple sources.
• errors: convenient methods to extract status code and other metadata from AWS errors.
• executor: Java's Executor for Go.
• fmt: provides fmt.Formatter implementations for printing/logging any data as JSON.
• must: for when you're tired of typing if a, err := someFunction(); err != nil and just want to panic instead.

Documentation

Index

• func Any[H any](m map[string]H) (k string, v H, ok bool)
• func CopyBufferWithContext(ctx context.Context, dst io.Writer, src io.Reader, buf []byte) (written int64, err error)
• func DirBase(name string) string
• func First[H any](s []H) (v H, ok bool)
• func Last[H any](s []H) (v H, ok bool)
• func MkExclDir(parent, stem string, perm os.FileMode) (name string, err error)
• func NewContextReader(ctx context.Context, r io.Reader) io.Reader
• func NewContextWriter(ctx context.Context, w io.Writer) io.Writer
• func OpenExclFile(parent, stem, ext string, perm os.FileMode) (file *os.File, err error)
• func StemExt(path string) (stem, ext string)
• func StemExtWithSize(path string, maxExtSize int) (stem, ext string)
• type Sizer
  • func (s *Sizer) Write(p []byte) (n int, err error)

Functions

func Any

func Any[H any](m map[string]H) (k string, v H, ok bool)

Any returns the first (any) key-value entry from looping over the map.

If the map is empty, the returned bool value will be false.

Useful if the map contains only one entry. If it has more than one, this method may return different entries upon subsequent calls.

func CopyBufferWithContext

func CopyBufferWithContext(ctx context.Context, dst io.Writer, src io.Reader, buf []byte) (written int64, err error)

CopyBufferWithContext is a custom implementation of io.CopyBuffer that is cancellable via context.

Similar to io.CopyBuffer, if buf is nil, a new buffer of size 32*1024 is created. Unlike io.CopyBuffer, it does not matter if src implements io.WriterTo or dst implements io.ReaderFrom because those interfaces do not support context.

The context is checked for done status after every write. As a result, having too small a buffer may introduce too much overhead, while having a very large buffer may cause context cancellation to have a delayed effect.

func DirBase

func DirBase(name string) string

DirBase joins both filepath.Dir and filepath.Base for the given file name.

The idea is that sometimes the working directory is not clear so by printing both the directory and the basename of a file, it is clearer where the file is.

func First

func First[H any](s []H) (v H, ok bool)

First returns the first element in the slice.

If the slice is empty, the returned bool value will be false.

func Last

func Last[H any](s []H) (v H, ok bool)

Last returns the last element in the slice.

If the slice is empty, the returned bool value will be false.

func MkExclDir

func MkExclDir(parent, stem string, perm os.FileMode) (name string, err error)

MkExclDir creates a new child directory that did not exist prior to this invocation.

Stem is the desired name of the directory. The actual directory that is created might have numeric suffixes such as stem-1, stem-2, etc. The return value "name" is the actual path to the newly created directory.

This method gives you a more predictable name over os.MkdirTemp at the cost of performance and concurrency.

func NewContextReader

func NewContextReader(ctx context.Context, r io.Reader) io.Reader

NewContextReader wraps the given io.Reader so that if the context is cancelled, io.Reader.Read always returns the error from the context.

func NewContextWriter

func NewContextWriter(ctx context.Context, w io.Writer) io.Writer

NewContextWriter wraps the given io.Writer so that if the context is cancelled, io.Writer.Write always returns the // error from the context.

func OpenExclFile

func OpenExclFile(parent, stem, ext string, perm os.FileMode) (file *os.File, err error)

OpenExclFile creates a new file for writing with the condition that the file did not exist prior to this call.

The first argument is the parent directory of the file to be created. The second argument is the stem of the file, the third the extension. For example, the stem of "hello-world.txt" is "hello-world", its ext ".txt". But with "hello-world.txt.s3", filepath.Ext will think ".s3" is the ext while this method allows you to choose ".txt.s3" as extension instead. If you use ".txt.s3" as extension, the naming is more natural: it will be "hello-world-1.txt.s3" or "hello-world-2.txt.s3" instead of "hello-world.txt-1.s3". See StemAndExt for a variant of filepath.Ext that allows up to 6 characters to be counted as ext.

The file is opened with flag `os.O_RDWR|os.O_CREATE|os.O_EXCL`. Caller is responsible for closing the file upon a successful return. See MkExclDir for a dir equivalent.

This method gives you a more predictable name over os.CreateTemp at the cost of performance and concurrency.

func StemExt

func StemExt(path string) (stem, ext string)

StemExt is a variant of filepath.Ext that allows extended extension to be detected while also returning the stem.

For example, `filepath.Ext("/path/to/file.tar.gz")` would return ".gz", but `xy3.StemAndExt("/path/to/file.tar.gz")` would return ".tar.gz" for the extension, "file" for the stem. This is useful when passed to OpenExclFile: "file-1.tar.gz" is more natural than "file.tar-1.gz".

StemExt will only accept file extensions of 5 characters or fewer, so if there is no `.` in the last 6 characters, the returned ext will be empty string unlike filepath.Ext which will keep searching until the last path separator or `.` is found. That means longer extensions such as ".jfif-tbnl" or ".turbot" will not be found by StemExt but can be found by filepath.Ext. Use StemExtWithSize if you need to customise the extension's size.

func StemExtWithSize

func StemExtWithSize(path string, maxExtSize int) (stem, ext string)

StemExtWithSize is a variant of StemExt that allows customisation of the extension's size.

If maxExtSize is 3, ".doc" may be returned but ".docx" will not. Similarly, ".doc.gz" may be returned, but if path ends in ".docx.gz", only ".gz" is returned.

Types

type Sizer

type Sizer struct {
	// Size is the total number of bytes that have been written to this io.Writer.
	Size int64
}

Sizer implements io.Writer that tallies that number of bytes written.

func (*Sizer) Write

func (s *Sizer) Write(p []byte) (n int, err error)