Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func New ¶
func New(env *Options) destination.Destination
New returns a valid Blacksmith destination.Destination for a document store.
Types ¶
type Docstore ¶
type Docstore struct {
// contains filtered or unexported fields
}
Docstore implements the Blacksmith destination.Destination interface for working with NoSQL document stores.
func (*Docstore) Actions ¶
func (d *Docstore) Actions() map[string]destination.Action
Actions return a list of actions the destination Docstore is able to handle.
func (*Docstore) Init ¶
func (d *Docstore) Init(tk *destination.Toolkit) error
Init is part of the destination.WithHooks interface. It allows to properly open the connection with the document store. It is called when starting the scheduler service.
func (*Docstore) Options ¶
func (d *Docstore) Options() *destination.Options
Options returns common destination options for a blob storage. They will be shared across every actions of this destination, except when overridden.
type Driver ¶
type Driver string
Driver is a custom type allowing the user to only pass supported drivers when creating the destination.
var DriverAWSDynamoDB Driver = "aws/dynamodb"
DriverAWSDynamoDB is used to leverage AWS DynamoDB as the destination's driver.
Environment variables:
- AWS_ACCESS_KEY_ID (required)
- AWS_SECRET_ACCESS_KEY (required)
- AWS_REGION
var DriverAzureCosmosDB Driver = "azure/cosmosdb"
DriverAzureCosmosDB is used to leverage Azure CosmosDB as the destination's driver.
Environment variables:
- MONGO_SERVER_URL (required)
var DriverGoogleFirestore Driver = "google/firestore"
DriverGoogleFirestore is used to leverage Google Firestore as the destination's driver.
Environment variables:
- GOOGLE_APPLICATION_CREDENTIALS (required)
var DriverMongoDB Driver = "mongodb"
DriverMongoDB is used to leverage MongoDB as the destination's driver.
Environment variables:
- MONGO_SERVER_URL (required)
type Options ¶
type Options struct {
// Realtime indicates if the pubsub adapter of the Blacksmith application shall
// be used to load events to the destination in realtime or not. When false, the
// Interval will be used.
Realtime bool
// Interval represents an interval or a CRON string at which a job shall be
// loaded to the destination. It is used as the time-lapse between retries in
// case of a job failure.
//
// Defaults to "@every 1h".
Interval string
// MaxRetries indicates the maximum number of retries per job the scheduler will
// attempt to execute before it succeed. When the limit is reached, the job is
// marked as "discarded".
//
// Defaults to 72.
MaxRetries uint16
// Name indicates the identifier of the document store to use in Blacksmith.
// The computed name is "docstore(<name>)". This does not have any consequences
// on the store's name used in the cloud provider.
//
// Examples: "mynosql"
// Required.
Name string
// Driver is the driver to leverage for using this destination.
//
// Required.
Driver Driver
// Connection is the driver's specific connection string to use for writing
// into the document store.
//
// Format for AWS DynamoDB: "<table>"
// Format for Azure CosmosDB and MongoDB: "<db>/<collection>"
// Format for Google Firestore: "projects/<project>/databases/(default)/documents/<collection>"
Connection string
// Params can be used to add specific configuration per driver.
//
// Supported fields for AWS DynamoDB:
// url.Values{
// "region": {"<region>"}, // Required if environment variable 'AWS_REGION' is not set.
// "partition_key": {"<key>"}, // Required. The path to the partition key of a table or an index.
// "sort_key": {"<key>"}, // Optional. The path to the sort key of a table or an index.
// }
//
// Supported fields for Azure CosmosDB and MongoDB:
// url.Values{
// "id_field": {"<field>"}, // Optional. The field name to use for the "_id" field.
// }
//
// Supported fields for Google Firestore:
// url.Values{
// "name_field": {"<field>"}, // Required. The designated field for the primary key.
// }
Params url.Values
}
Options is the options the destination can take as an input to be configured.
type Put ¶
type Put struct {
Document map[string]interface{} `json:"document"`
// contains filtered or unexported fields
}
Put implements the Blacksmith destination.Action interface for the action "put". It holds the complete job's structure to load into the destination.
func (Put) Load ¶
func (a Put) Load(tk *destination.Toolkit, queue *store.Queue, then chan<- destination.Then)
Load is the function being run by the scheduler to load the data into the destination. It is in charge of the "L" in the ETL process.
func (Put) Marshal ¶
func (a Put) Marshal(tk *destination.Toolkit) (*destination.Job, error)
Marshal is the function being run when the action receives data into the Put receiver. It allows to transform and enrich the data before saving it in the store adapter.
func (Put) Schedule ¶
func (a Put) Schedule() *destination.Schedule
Schedule allows the action to override the schedule options of its destination. Do not override.
Source Files