computedatabricks

README

compute-databricks

_{part of the togo-framework}

A togo compute backend that submits one-time runs to Databricks via the Jobs REST API (/api/2.1/jobs/runs/submit).

togo install togo-framework/compute-databricks
togo provider:use compute databricks
togo config:set DATABRICKS_HOST https://xxx.cloud.databricks.com
togo config:set DATABRICKS_CLUSTER_ID 0101-… 
# DATABRICKS_TOKEN in .env (secret)

Key	Purpose
DATABRICKS_HOST	workspace URL
DATABRICKS_TOKEN	PAT (secret)
DATABRICKS_CLUSTER_ID	existing cluster

MIT © fadymondy

Documentation

Overview

Package computedatabricks is a togo compute backend that submits one-time runs to Databricks via the Jobs REST API (/api/2.1/jobs/runs/submit). Select with `togo provider:use compute databricks`.

DATABRICKS_HOST        workspace URL (https://…azuredatabricks.net)
DATABRICKS_TOKEN       PAT (secret)
DATABRICKS_CLUSTER_ID  existing cluster to run on