spadelib

package module
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 17, 2026 License: MIT Imports: 8 Imported by: 0

README

spadelib

The Go library for authoring Spade blocks. Define a handler function, pass it to Run(), and the library handles parameter loading, input discovery, and output writing.

Installation

go get github.com/krbundy/spadelib

import "github.com/krbundy/spadelib"

The import path is the full module path, but the package identifier stays spadelib, so calls read spadelib.Run(...), spadelib.Input[...], and so on.

Quick Start

package main

import "github.com/krbundy/spadelib"

func handler(args *spadelib.Args) (spadelib.RasterFile, error) {
	source, err := spadelib.Input[spadelib.RasterFile](args, "source")
	if err != nil {
		return spadelib.RasterFile{}, err
	}
	resolution, err := spadelib.Param[float64](args, "resolution")
	if err != nil {
		return spadelib.RasterFile{}, err
	}
	// process source.Path at resolution...
	return spadelib.NewRasterFile("result.tif"), nil
}

func main() {
	spadelib.Run(handler)
}

Run():

  1. Loads scalar parameters from params.yaml
  2. Scans inputs/ for file-based arguments, matched by name
  3. Builds an *Args and calls your handler
  4. Writes the returned value to outputs/

On any error Run() prints it to stderr and exits non-zero.

Reading Arguments

A handler receives an *Args and pulls typed values out of it by name. Because Go generics can't be methods, use the package-level Input and Param functions:

source, err := spadelib.Input[spadelib.RasterFile](args, "source")  // file input
resolution, err := spadelib.Param[float64](args, "resolution")      // scalar

if args.HasInput("mask") {
	mask, err := spadelib.Input[spadelib.VectorFile](args, "mask")  // optional input
}

args.HasInput(name) and args.HasParam(name) test for optional arguments before reading.

Types

Construct file types with the New* helpers (e.g. spadelib.NewRasterFile("path.tif")); the location is on the Path field.

File Types
Type Description
File Generic single file
RasterFile Raster data (e.g., GeoTIFF)
VectorFile Vector data (e.g., GeoJSON)
TabularFile Tabular data (e.g., CSV)
JsonFile JSON data
Directory Type
Type Description
Directory Directory-based input (e.g., shapefiles)
Collection Types
Type Description
FileCollection Collection of files
RasterFileCollection Collection of raster files
VectorFileCollection Collection of vector files
TabularFileCollection Collection of tabular files

Each holds a Paths []string field.

Scalars

string, float64, int, bool, and the other basic types are read directly with spadelib.Param[T].

How Inputs Are Resolved

File-based inputs are discovered from subdirectories in inputs/, where each subdirectory name matches the name you pass to Input:

params.yaml              # scalar args: {resolution: 10, method: "nearest"}
inputs/
  reference/
    data.tif             # -> spadelib.Input[spadelib.RasterFile](args, "reference")
  target/
    data.tif             # -> spadelib.Input[spadelib.RasterFile](args, "target")

The target type determines how each input is constructed:

  • a File type expects a single file in the subdirectory
  • Directory uses the subdirectory path itself
  • a *Collection type gathers all files in the subdirectory

Output Handling

Return a typed value and Run() writes it to outputs/:

func handler(args *spadelib.Args) (spadelib.RasterFile, error) {
	return spadelib.NewRasterFile("result.tif"), nil
}

For multiple named outputs, return an *Outputs:

func handler(args *spadelib.Args) (*spadelib.Outputs, error) {
	out := spadelib.NewOutputs()
	out.Add("raster", spadelib.NewRasterFile("result.tif"))
	out.Add("stats", spadelib.NewJsonFile("stats.json"))
	return out, nil
}

For a handler that produces no output, use spadelib.RunNoOutput, whose handler returns just an error.

Secrets

Named secrets provisioned to the block are read with GetSecret():

token, err := spadelib.GetSecret("api_token")

Run() scrubs the secrets from the environment before your handler runs.

Building a Manifest

Describe a block's inputs and outputs with the manifest builder:

b := spadelib.NewManifestBuilder().Description("Resample a raster.")
spadelib.ManifestInput[spadelib.RasterFile](b, "source")
spadelib.ManifestOutput[spadelib.RasterFile](b, "raster")
manifest := b.Build()

Testing

go test ./...

Learn More

See the Spade homepage for the full system documentation, the block specification, and the other language runtimes (Python, Rust, TypeScript, and R).

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func GetSecret 

func GetSecret(name string) (string, error)

GetSecret returns the secret bound to a logical name for this block. The mapping from logical name to a stored secret is declared in the pipeline (spec/secrets.md §3.2); the value is injected by the worker (cloud) or CLI (local). Returns an error if the name was not provided.

func Input 

func Input[T interface {
	*E
	FromInput
}, E any](args *Args, name string) (E, error)

Input retrieves a typed file/directory input by name from Args. T must be a pointer to a type implementing FromInput.

func LoadParams 

func LoadParams(base string) (map[string]any, error)

LoadParams reads and parses params.yaml from the given base directory. Returns an empty map if the file does not exist or is empty.

func Param 

func Param[T any](args *Args, name string) (T, error)

Param retrieves a typed scalar parameter by name from Args.

func ReadBlockManifest 

func ReadBlockManifest(base string) map[string]any

ReadBlockManifest reads the block manifest to get output declarations. Checks SPADE_BLOCK_MANIFEST env var first, then block.yaml in base dir. Returns nil if no manifest found.

func Run 

func Run[O IntoOutput](handler func(*Args) (O, error))

Run executes a handler function as a Spade block. It loads inputs and params, calls the handler, and writes outputs. On any error it prints to stderr and exits.

func RunAt 

func RunAt[O IntoOutput](base string, handler func(*Args) (O, error)) error

RunAt executes a handler at a specific base path. Used for testing.

func RunNoOutput 

func RunNoOutput(handler func(*Args) error)

RunNoOutput executes a handler that produces no output.

func RunNoOutputAt 

func RunNoOutputAt(base string, handler func(*Args) error) error

RunNoOutputAt executes a no-output handler at a specific base path. Used for testing.

func ScanInputs 

func ScanInputs(base string) (map[string]InputEntry, error)

ScanInputs reads the inputs/ directory at the given base path and returns a map of input name to InputEntry.

func WriteOutputs 

func WriteOutputs(result IntoOutput, base string, manifestOutputs map[string]any) error

WriteOutputs writes handler output(s) to <base>/outputs/.

Types

type Args 

type Args struct {
	// contains filtered or unexported fields
}

Args holds merged parameters and inputs for handler consumption.

func BuildArgs 

func BuildArgs(base string) (*Args, error)

BuildArgs constructs an Args struct from the given base path by loading params.yaml and scanning the inputs/ directory.

func (*Args) HasInput 

func (a *Args) HasInput(name string) bool

HasInput checks whether an input with the given name exists.

func (*Args) HasParam 

func (a *Args) HasParam(name string) bool

HasParam checks whether a parameter with the given name exists.

type BoolType 

type BoolType struct{}

BoolType is a manifest-only type representing a boolean parameter.

func (BoolType) DefaultOutputName 

func (BoolType) DefaultOutputName() string

func (BoolType) ManifestEntry 

func (BoolType) ManifestEntry() ManifestInfo

func (BoolType) TypeName 

func (BoolType) TypeName() string

type Directory 

type Directory struct {
	Path string
}

Directory represents a directory-based input or output.

func NewDirectory 

func NewDirectory(path string) Directory

func (*Directory) DefaultOutputName 

func (d *Directory) DefaultOutputName() string

func (*Directory) FromDirectory 

func (d *Directory) FromDirectory(path string) error

func (*Directory) FromMultipleFiles 

func (d *Directory) FromMultipleFiles(_ []string) error

func (*Directory) FromSingleFile 

func (d *Directory) FromSingleFile(_ string) error

func (*Directory) ManifestEntry 

func (d *Directory) ManifestEntry() ManifestInfo

func (*Directory) TypeName 

func (d *Directory) TypeName() string

func (*Directory) WriteTo 

func (d *Directory) WriteTo(outputDir string) error

type ErrEmptyInputDir 

type ErrEmptyInputDir struct {
	Name string
}

ErrEmptyInputDir is returned when an input subdirectory exists but contains no files.

func (*ErrEmptyInputDir) Error 

func (e *ErrEmptyInputDir) Error() string

type ErrInputNotFound 

type ErrInputNotFound struct {
	Name string
}

ErrInputNotFound is returned when Args.Input references a name not present in inputs/.

func (*ErrInputNotFound) Error 

func (e *ErrInputNotFound) Error() string

type ErrParamNotFound 

type ErrParamNotFound struct {
	Name string
}

ErrParamNotFound is returned when Args.Param references a name not present in params.yaml.

func (*ErrParamNotFound) Error 

func (e *ErrParamNotFound) Error() string

type ErrTypeMismatch 

type ErrTypeMismatch struct {
	Name     string
	Expected string
	Found    string
}

ErrTypeMismatch is returned when an input value cannot be converted to the requested type.

func (*ErrTypeMismatch) Error 

func (e *ErrTypeMismatch) Error() string

type File 

type File struct {
	Path string
}

File represents a generic single-file input or output.

func NewFile 

func NewFile(path string) File

func (*File) DefaultOutputName 

func (f *File) DefaultOutputName() string

func (*File) FromDirectory 

func (f *File) FromDirectory(_ string) error

func (*File) FromMultipleFiles 

func (f *File) FromMultipleFiles(paths []string) error

func (*File) FromSingleFile 

func (f *File) FromSingleFile(path string) error

func (*File) ManifestEntry 

func (f *File) ManifestEntry() ManifestInfo

func (*File) TypeName 

func (f *File) TypeName() string

func (*File) WriteTo 

func (f *File) WriteTo(outputDir string) error

type FileCollection 

type FileCollection struct {
	Paths []string
}

FileCollection represents a collection of generic files.

func NewFileCollection 

func NewFileCollection(paths []string) FileCollection

func (*FileCollection) DefaultOutputName 

func (c *FileCollection) DefaultOutputName() string

func (*FileCollection) FromDirectory 

func (c *FileCollection) FromDirectory(_ string) error

func (*FileCollection) FromMultipleFiles 

func (c *FileCollection) FromMultipleFiles(paths []string) error

func (*FileCollection) FromSingleFile 

func (c *FileCollection) FromSingleFile(path string) error

func (*FileCollection) ManifestEntry 

func (c *FileCollection) ManifestEntry() ManifestInfo

func (*FileCollection) TypeName 

func (c *FileCollection) TypeName() string

func (*FileCollection) WriteTo 

func (c *FileCollection) WriteTo(outputDir string) error

type FromInput 

type FromInput interface {
	FromSingleFile(path string) error
	FromMultipleFiles(paths []string) error
	FromDirectory(path string) error
}

FromInput constructs a typed value from raw filesystem input data.

type InputEntry 

type InputEntry struct {
	Kind  InputKind
	Path  string   // populated for InputSingle
	Paths []string // populated for InputMultiple
}

InputEntry represents a scanned input from the inputs/ directory.

type InputKind 

type InputKind int

InputKind distinguishes single-file from multi-file input entries. 

const (
	InputSingle InputKind = iota
	InputMultiple
)

type IntoOutput 

type IntoOutput interface {
	WriteTo(outputDir string) error
	DefaultOutputName() string
}

IntoOutput writes a typed value to an output subdirectory.

type JsonFile 

type JsonFile struct {
	Path string
}

JsonFile represents a JSON data file.

func NewJsonFile 

func NewJsonFile(path string) JsonFile

func (*JsonFile) DefaultOutputName 

func (f *JsonFile) DefaultOutputName() string

func (*JsonFile) FromDirectory 

func (f *JsonFile) FromDirectory(_ string) error

func (*JsonFile) FromMultipleFiles 

func (f *JsonFile) FromMultipleFiles(paths []string) error

func (*JsonFile) FromSingleFile 

func (f *JsonFile) FromSingleFile(path string) error

func (*JsonFile) ManifestEntry 

func (f *JsonFile) ManifestEntry() ManifestInfo

func (*JsonFile) TypeName 

func (f *JsonFile) TypeName() string

func (*JsonFile) WriteTo 

func (f *JsonFile) WriteTo(outputDir string) error

type ManifestBuilder 

type ManifestBuilder struct {
	// contains filtered or unexported fields
}

ManifestBuilder provides a fluent API for generating block manifest dictionaries.

func ManifestInput 

func ManifestInput[T SpadeType](b *ManifestBuilder, name string) *ManifestBuilder

ManifestInput declares an input on the builder using a SpadeType's manifest metadata.

func ManifestOutput 

func ManifestOutput[T SpadeType](b *ManifestBuilder, name string) *ManifestBuilder

ManifestOutput declares an output on the builder using a SpadeType's manifest metadata.

func NewManifestBuilder 

func NewManifestBuilder() *ManifestBuilder

NewManifestBuilder creates a new empty manifest builder.

func (*ManifestBuilder) Build 

func (b *ManifestBuilder) Build() map[string]any

Build returns the manifest as a map suitable for YAML serialization.

func (*ManifestBuilder) Description 

func (b *ManifestBuilder) Description(desc string) *ManifestBuilder

Description sets the block description.

type ManifestInfo 

type ManifestInfo struct {
	TypeName string
	Format   string // empty string = not set
	ItemType string // empty string = not set
}

ManifestInfo holds metadata returned by SpadeType.ManifestEntry().

type NumberType 

type NumberType struct{}

NumberType is a manifest-only type representing a numeric parameter.

func (NumberType) DefaultOutputName 

func (NumberType) DefaultOutputName() string

func (NumberType) ManifestEntry 

func (NumberType) ManifestEntry() ManifestInfo

func (NumberType) TypeName 

func (NumberType) TypeName() string

type Outputs 

type Outputs struct {
	// contains filtered or unexported fields
}

Outputs is a collection of named outputs for handlers that produce multiple results.

func NewOutputs 

func NewOutputs() *Outputs

NewOutputs creates an empty Outputs collection.

func (*Outputs) Add 

func (o *Outputs) Add(name string, value IntoOutput)

Add appends a named output to the collection.

func (*Outputs) DefaultOutputName 

func (o *Outputs) DefaultOutputName() string

DefaultOutputName returns a sentinel value for internal routing.

func (*Outputs) ManifestEntry 

func (o *Outputs) ManifestEntry() ManifestInfo

ManifestEntry returns a sentinel ManifestInfo.

func (*Outputs) TypeName 

func (o *Outputs) TypeName() string

TypeName returns a sentinel value for internal routing.

func (*Outputs) WriteTo 

func (o *Outputs) WriteTo(outputDir string) error

WriteTo writes all entries to subdirectories under outputDir.

type RasterFile 

type RasterFile struct {
	Path string
}

RasterFile represents a raster data file (e.g., GeoTIFF).

func NewRasterFile 

func NewRasterFile(path string) RasterFile

func (*RasterFile) DefaultOutputName 

func (f *RasterFile) DefaultOutputName() string

func (*RasterFile) FromDirectory 

func (f *RasterFile) FromDirectory(_ string) error

func (*RasterFile) FromMultipleFiles 

func (f *RasterFile) FromMultipleFiles(paths []string) error

func (*RasterFile) FromSingleFile 

func (f *RasterFile) FromSingleFile(path string) error

func (*RasterFile) ManifestEntry 

func (f *RasterFile) ManifestEntry() ManifestInfo

func (*RasterFile) TypeName 

func (f *RasterFile) TypeName() string

func (*RasterFile) WriteTo 

func (f *RasterFile) WriteTo(outputDir string) error

type RasterFileCollection 

type RasterFileCollection struct {
	Paths []string
}

RasterFileCollection represents a collection of raster data files.

func NewRasterFileCollection 

func NewRasterFileCollection(paths []string) RasterFileCollection

func (*RasterFileCollection) DefaultOutputName 

func (c *RasterFileCollection) DefaultOutputName() string

func (*RasterFileCollection) FromDirectory 

func (c *RasterFileCollection) FromDirectory(_ string) error

func (*RasterFileCollection) FromMultipleFiles 

func (c *RasterFileCollection) FromMultipleFiles(paths []string) error

func (*RasterFileCollection) FromSingleFile 

func (c *RasterFileCollection) FromSingleFile(path string) error

func (*RasterFileCollection) ManifestEntry 

func (c *RasterFileCollection) ManifestEntry() ManifestInfo

func (*RasterFileCollection) TypeName 

func (c *RasterFileCollection) TypeName() string

func (*RasterFileCollection) WriteTo 

func (c *RasterFileCollection) WriteTo(outputDir string) error

type SpadeType 

type SpadeType interface {
	TypeName() string
	DefaultOutputName() string
	ManifestEntry() ManifestInfo
}

SpadeType provides metadata about a Spade type for manifest generation and output naming.

type StringType 

type StringType struct{}

StringType is a manifest-only type representing a string parameter.

func (StringType) DefaultOutputName 

func (StringType) DefaultOutputName() string

func (StringType) ManifestEntry 

func (StringType) ManifestEntry() ManifestInfo

func (StringType) TypeName 

func (StringType) TypeName() string

type TabularFile 

type TabularFile struct {
	Path string
}

TabularFile represents a tabular data file (e.g., CSV).

func NewTabularFile 

func NewTabularFile(path string) TabularFile

func (*TabularFile) DefaultOutputName 

func (f *TabularFile) DefaultOutputName() string

func (*TabularFile) FromDirectory 

func (f *TabularFile) FromDirectory(_ string) error

func (*TabularFile) FromMultipleFiles 

func (f *TabularFile) FromMultipleFiles(paths []string) error

func (*TabularFile) FromSingleFile 

func (f *TabularFile) FromSingleFile(path string) error

func (*TabularFile) ManifestEntry 

func (f *TabularFile) ManifestEntry() ManifestInfo

func (*TabularFile) TypeName 

func (f *TabularFile) TypeName() string

func (*TabularFile) WriteTo 

func (f *TabularFile) WriteTo(outputDir string) error

type TabularFileCollection 

type TabularFileCollection struct {
	Paths []string
}

TabularFileCollection represents a collection of tabular data files.

func NewTabularFileCollection 

func NewTabularFileCollection(paths []string) TabularFileCollection

func (*TabularFileCollection) DefaultOutputName 

func (c *TabularFileCollection) DefaultOutputName() string

func (*TabularFileCollection) FromDirectory 

func (c *TabularFileCollection) FromDirectory(_ string) error

func (*TabularFileCollection) FromMultipleFiles 

func (c *TabularFileCollection) FromMultipleFiles(paths []string) error

func (*TabularFileCollection) FromSingleFile 

func (c *TabularFileCollection) FromSingleFile(path string) error

func (*TabularFileCollection) ManifestEntry 

func (c *TabularFileCollection) ManifestEntry() ManifestInfo

func (*TabularFileCollection) TypeName 

func (c *TabularFileCollection) TypeName() string

func (*TabularFileCollection) WriteTo 

func (c *TabularFileCollection) WriteTo(outputDir string) error

type VectorFile 

type VectorFile struct {
	Path string
}

VectorFile represents a vector data file (e.g., GeoJSON).

func NewVectorFile 

func NewVectorFile(path string) VectorFile

func (*VectorFile) DefaultOutputName 

func (f *VectorFile) DefaultOutputName() string

func (*VectorFile) FromDirectory 

func (f *VectorFile) FromDirectory(_ string) error

func (*VectorFile) FromMultipleFiles 

func (f *VectorFile) FromMultipleFiles(paths []string) error

func (*VectorFile) FromSingleFile 

func (f *VectorFile) FromSingleFile(path string) error

func (*VectorFile) ManifestEntry 

func (f *VectorFile) ManifestEntry() ManifestInfo

func (*VectorFile) TypeName 

func (f *VectorFile) TypeName() string

func (*VectorFile) WriteTo 

func (f *VectorFile) WriteTo(outputDir string) error

type VectorFileCollection 

type VectorFileCollection struct {
	Paths []string
}

VectorFileCollection represents a collection of vector data files.

func NewVectorFileCollection 

func NewVectorFileCollection(paths []string) VectorFileCollection

func (*VectorFileCollection) DefaultOutputName 

func (c *VectorFileCollection) DefaultOutputName() string

func (*VectorFileCollection) FromDirectory 

func (c *VectorFileCollection) FromDirectory(_ string) error

func (*VectorFileCollection) FromMultipleFiles 

func (c *VectorFileCollection) FromMultipleFiles(paths []string) error

func (*VectorFileCollection) FromSingleFile 

func (c *VectorFileCollection) FromSingleFile(path string) error

func (*VectorFileCollection) ManifestEntry 

func (c *VectorFileCollection) ManifestEntry() ManifestInfo

func (*VectorFileCollection) TypeName 

func (c *VectorFileCollection) TypeName() string

func (*VectorFileCollection) WriteTo 

func (c *VectorFileCollection) WriteTo(outputDir string) error

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.