computespark

README

compute-spark

_{part of the togo-framework}

A togo compute backend that submits jobs to Apache Spark via spark-submit. Registers into compute's slot.

togo install togo-framework/compute-spark
togo provider:use compute spark
togo config:set SPARK_MASTER yarn

Key	Default	Purpose
SPARK_SUBMIT_BIN	spark-submit	submitter binary
SPARK_MASTER	—	--master value
SPARK_OPTS	—	extra opts

MIT © fadymondy

Documentation

Overview

Package computespark is a togo compute backend that submits jobs to Apache Spark via spark-submit. Registers into the compute plugin's `compute` slot. Select with `togo provider:use compute spark`.

Config (dynamic — CLI/.env/settings):

SPARK_SUBMIT_BIN  spark-submit binary   (default "spark-submit")
SPARK_MASTER      --master value        (e.g. yarn, k8s://…, local[*])
SPARK_OPTS        extra spark-submit opts (space-separated)