adapter

Documentation

Overview

Package adapter provides a golang-native Martian job adapter and utilities.

A stage executable should should look something like

package main

import (
	"github.com/martian-lang/martian/martian/adapter"
	"github.com/martian-lang/martian/martian/core"
)

func main() {
	adapter.RunStage(split, chunk, join)
}

func split(metadata *core.Metadata) (*core.StageDefs, error) {
	...
}

func chunk(metadata *core.Metadata) (interface{}, error) {
	...
}

func join(metadata *core.Metadata) (interface{}, error) {
	...
}

One executable handles all 3 phases. Stages which do not split may pass nil for the split and join arguments to RunStage.

Stage code should NEVER directly write to the log, errors, or assert files through the metadata object, but should instead return an error. For an assertion error, use the StageAssertion method. For logging, use util.LogInfo and friends.

Index

• func GetJobInfo() *core.JobInfo
• func IsAssertion(err error) bool
• func RunStage(split SplitFunc, main MainFunc, join MainFunc)
• func StageAssertion(message string) error
• func UpdateProgress(metadata *core.Metadata, message string) error
• type MainFunc
• type SplitFunc

Functions

func GetJobInfo

func GetJobInfo() *core.JobInfo

Get the cached version of the JobInfo object for this job.

func IsAssertion

func IsAssertion(err error) bool

Tests whether the given error is an assertion.

func RunStage

func RunStage(split SplitFunc, main MainFunc, join MainFunc)

Parses the command line and stage inputs, runs the appropriate given stage code, and saves the outputs. split and join may be nil if the stage does not split.

This should be the main entry point for all stage executables.

func StageAssertion

func StageAssertion(message string) error

Creates an error that is interpreted as an assertion.

func UpdateProgress

func UpdateProgress(metadata *core.Metadata, message string) error

Write stage progress information. This information will be bubbled up to the mrp log, unless it is overwritten by a more recent update first.

Types

type MainFunc

type MainFunc func(metadata *core.Metadata) (interface{}, error)

A function for a stage's chunk or join phase. The returned object, if any, is saved to the stage _outs. Stage args, jobinfo, and so on can be read with metadata.ReadInto().

type SplitFunc

type SplitFunc func(metadata *core.Metadata) (*core.StageDefs, error)

A function for a stage's split phase. Must return a StageDefs object. Stage Args, jobinfo, and so on can be read with metadata.ReadInto().