salmodule

Documentation

Index

• Constants
• func FetchOntologyDocument(iri string) (document []byte, mediaType string, commitHash string, err error)
• func IsModuleIRI(iri string) bool
• func IsTaskBaseClass(iri string) bool
• type CommandRunner
• type ContainerRunner
• type ModuleOntology
  • func Inspect(ctx context.Context, reference string) (*ModuleOntology, error)
  • func (o *ModuleOntology) GraphFromTaskOutput(output []byte) (*rdflibgo.Graph, error)
  • func (o *ModuleOntology) IsTaskClass(iri string) bool
  • func (o *ModuleOntology) TaskInstance(graph *rdflibgo.Graph, subject rdflibgo.Subject, classIRI string) (string, error)
• type ModuleRef
  • func ParseModuleIRI(iri string) (ModuleRef, error)
  • func (r ModuleRef) ImageTagFor(commit string) string
• type Resolver
  • func Default() *Resolver
  • func (r *Resolver) CommitHash(ctx context.Context, ref ModuleRef) (string, error)
  • func (r *Resolver) Downloaded() []string
  • func (r *Resolver) Ontology(ctx context.Context, ref ModuleRef) (*ModuleOntology, error)
  • func (r *Resolver) Reset()
  • func (r *Resolver) RunTask(ctx context.Context, ref ModuleRef, envVar string, taskInstance string) ([]byte, error)
  • func (r *Resolver) UsePinnedCommit(namespace string, commit string)
• type SalModuleCmd
  • func (cmd *SalModuleCmd) Run() error
• type TaskError
  • func (e TaskError) Error() string

Constants

const (
	// Namespace is the IRI of the SAL Module ontology itself.
	Namespace = "https://w3id.org/sal/cgs-earth/sal-module-spec/salmodule#"
	// ProtocolScheme is the URI scheme used to reference a SAL module from RDF.
	ProtocolScheme = "salmodule"

	BaseCommand     = "salmodule"
	OntologyCommand = "ontology"
	RunCommand      = "run"

	// DefaultTaskInstanceEnvVar is used when a module ontology does not declare
	// its own salmodule:taskInstanceEnvVar.
	DefaultTaskInstanceEnvVar = "SALMODULE_TASK_INSTANCE"

	// IcebergTableProperty is the Iceberg table property a build records the
	// JSON list of every SAL module it downloaded under.
	IcebergTableProperty = "sal.salmodules"
)

Constants describing the SAL Module specification. The command line conventions mirror the salmodule:baseCommand, salmodule:ontologyCommand, salmodule:runCommand, and salmodule:taskInstanceEnvVar ontology properties declared in build/testdata/reference/public/salmodule.ttl.

Functions

func FetchOntologyDocument

func FetchOntologyDocument(iri string) (document []byte, mediaType string, commitHash string, err error)

FetchOntologyDocument dereferences a salmodule:// IRI to the module's ontology document so that RDF validation can resolve the module's terms, and to the git commit hash of the module repository it was built from, which is what a salmodule:// vocabulary is pinned at.

func IsModuleIRI

func IsModuleIRI(iri string) bool

IsModuleIRI reports whether iri uses the salmodule protocol scheme.

func IsTaskBaseClass

func IsTaskBaseClass(iri string) bool

IsTaskBaseClass reports whether iri is one of the SAL Module ontology's own task classes.

Types

type CommandRunner

type CommandRunner func(ctx context.Context, dir string, name string, args ...string) ([]byte, error)

CommandRunner runs an external command and returns its combined output, allowing git usage to be faked in tests.

type ContainerRunner

type ContainerRunner interface {
	BuildImage(ctx context.Context, contextDir string, tag string) error
	// ImageExists reports whether the daemon already holds an image under tag,
	// which is how a module built by a previous invocation is found and reused.
	ImageExists(ctx context.Context, tag string) (bool, error)
	RunContainer(ctx context.Context, image string, env []string, cmd []string) (stdout []byte, stderr []byte, err error)
}

ContainerRunner builds and runs the container images that back SAL modules.

type ModuleOntology

type ModuleOntology struct {
	// Namespace is the vocabulary base the ontology's relative terms resolve against.
	Namespace string
	// Document is the raw JSON-LD the module printed.
	Document []byte
	// Context is the ontology's @context, injected into the JSON a module task
	// writes so that its keys resolve to the module's vocabulary.
	Context json.RawMessage
	// TaskInstanceEnvVar names the environment variable carrying the task
	// instance passed to the run command.
	TaskInstanceEnvVar string
	// contains filtered or unexported fields
}

ModuleOntology is the JSON-LD vocabulary a SAL module prints in response to its ontology command.

func Inspect

func Inspect(ctx context.Context, reference string) (*ModuleOntology, error)

Inspect clones the module's repository, builds its Dockerfile, and runs its ontology command, returning the vocabulary the module published. Docker's own layer cache makes a repeated inspection of an unchanged module cheap, and the shared resolver keeps a module referenced twice in one invocation to a single clone and build.

func (*ModuleOntology) GraphFromTaskOutput

func (o *ModuleOntology) GraphFromTaskOutput(output []byte) (*rdflibgo.Graph, error)

GraphFromTaskOutput turns the newline delimited JSON a module task wrote to stdout into RDF. A task emits plain JSON, so the module ontology's @context is injected to resolve the keys against the module's vocabulary.

func (*ModuleOntology) IsTaskClass

func (o *ModuleOntology) IsTaskClass(iri string) bool

IsTaskClass reports whether the ontology declares iri as a subclass of one of the SAL Module ontology's task classes.

func (*ModuleOntology) TaskInstance

func (o *ModuleOntology) TaskInstance(graph *rdflibgo.Graph, subject rdflibgo.Subject, classIRI string) (string, error)

TaskInstance renders the JSON-LD node object SAL passes to a module through its task instance environment variable. A module is configured in RDF rather than with an embedded JSON literal, so the node object is built from the instance's own properties: every predicate the module's vocabulary defines is an input to the task, and predicates from any other vocabulary are left out since only the module knows what configures it.

type ModuleRef

type ModuleRef struct {
	// Namespace is the vocabulary base that the module's ontology terms resolve
	// against. It always ends in a slash.
	Namespace string
	// CloneURL is the git repository holding the module's Dockerfile.
	CloneURL string
	// ImageRepository is the local docker repository name SAL builds the module
	// under. The image itself is tagged with the git commit hash of the module
	// repository it was built from; see ImageTagFor.
	ImageRepository string
}

ModuleRef is a SAL module dereferenced from a salmodule:// IRI.

func ParseModuleIRI

func ParseModuleIRI(iri string) (ModuleRef, error)

ParseModuleIRI resolves a salmodule://[HOST/]OWNER/REPO IRI into the git repository that provides the module and the local image tag SAL builds it as. Any fragment or term suffix on the IRI is ignored so that both a vocabulary base and an individual term IRI resolve to the same module.

func (ModuleRef) ImageTagFor

func (r ModuleRef) ImageTagFor(commit string) string

ImageTagFor returns the docker tag a module built from commit is stored under. Tagging by commit is what lets a later invocation reuse the image a previous one built: the commit a project pins in .sal/config.jsonld names the exact tag to look for.

type Resolver

type Resolver struct {
	// Runner runs docker operations. A client for the local docker daemon is
	// created on first use when this is nil.
	Runner ContainerRunner
	// Command runs git. os/exec is used when this is nil.
	Command CommandRunner
	// contains filtered or unexported fields
}

Resolver dereferences salmodule:// IRIs by cloning the module's git repository, building the Dockerfile in its root, and invoking the SAL Module command line interface inside the resulting image.

A module is cloned and built at most once per resolver.

func Default

func Default() *Resolver

Default returns the resolver shared by validation and build so that a module referenced from several places is only cloned and built once per invocation.

func (*Resolver) CommitHash

func (r *Resolver) CommitHash(ctx context.Context, ref ModuleRef) (string, error)

CommitHash returns the git commit hash of the HEAD of the module repository the last time it was cloned, cloning and building it first if it has not been referenced yet. A salmodule:// vocabulary is pinned by this rather than by the digest of its ontology document, since code in the module that changes what a task does is not necessarily a change to the ontology itself.

func (*Resolver) Downloaded

func (r *Resolver) Downloaded() []string

Downloaded returns the salmodule:// URI of every module the resolver has resolved to an image, whether it cloned and built the module or reused a prebuilt image, and whether it was dereferenced for its vocabulary or run as a task. A build records these so that a table says which modules produced it.

func (*Resolver) Ontology

func (r *Resolver) Ontology(ctx context.Context, ref ModuleRef) (*ModuleOntology, error)

Ontology returns the vocabulary the module publishes through its ontology command.

func (*Resolver) Reset

func (r *Resolver) Reset()

Reset drops every module the resolver has already cloned, built, and dereferenced so that the next reference resolves from scratch.

func (*Resolver) RunTask

func (r *Resolver) RunTask(ctx context.Context, ref ModuleRef, envVar string, taskInstance string) ([]byte, error)

RunTask invokes the module's run command with taskInstance supplied through the environment variable the module's ontology declares, and returns the newline delimited JSON the task wrote to stdout.

Whatever the task wrote is returned even when the container fails, because a task reports its own failures as salmodule:Error nodes on stdout before exiting non-zero; those messages describe the failure far better than the container's exit status does.

func (*Resolver) UsePinnedCommit

func (r *Resolver) UsePinnedCommit(namespace string, commit string)

UsePinnedCommit tells the resolver which git commit the project pins the module at, so that the image a previous invocation built and tagged with that commit can be reused instead of cloning and building the module again.

type SalModuleCmd

type SalModuleCmd struct {
	// Ontology is needed so that the sal cli itself is a sal module
	Ontology *ontologyCmd `arg:"subcommand:ontology" help:"Print the ontology of the sal cli itself"`
	SalRun   *runCmd      `arg:"subcommand:run" help:"Run a sal project"`
	Inspect  *inspectCmd  `arg:"subcommand:inspect" help:"Print the ontology that a remote SAL module publishes"`
}

func (*SalModuleCmd) Run

func (cmd *SalModuleCmd) Run() error

type TaskError

type TaskError struct {
	Namespace string
	Messages  []string
}

TaskError reports the salmodule:Error nodes a module task emitted instead of data.

func (TaskError) Error

func (e TaskError) Error() string