connect

package
v0.1.33 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 23, 2020 License: Apache-2.0, MIT Imports: 11 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// StatusUnprocessableEntity is the status code returned when sending a
	// request with invalid fields.
	StatusUnprocessableEntity = 422
)

Variables

This section is empty.

Functions

func IsAPIError 

func IsAPIError(err error) bool

IsAPIError indicates if the error is an struct of type APIError

func IsNotFound 

func IsNotFound(err error) bool

IsNotFound indicates if the error represents an HTTP 404 status code

func IsRetryable added in v0.1.6 

func IsRetryable(err error) bool

IsRetryable indicates if the error could be retryed. See https://github.com/apache/kafka/blob/master/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/resources/ConnectorsResource.java#L299-L325

Types

type APIError 

type APIError struct {
	Code     int            `json:"error_code"`
	Message  string         `json:"message"`
	Response *http.Response // HTTP response that caused this error
}

APIError holds information returned from a Kafka Connect API instance about why an API call failed.

func (APIError) Error 

func (e APIError) Error() string

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client manages communication with the Kafka Connect REST API.

func NewClient 

func NewClient(host string, opts ...Option) (*Client, error)

NewClient returns a new Kafka Connect API client that communicates host.

func (*Client) CreateConnector 

func (c *Client) CreateConnector(conn Connector) (*http.Response, error)

CreateConnector creates a new connector instance. If successful, conn is updated with the connector's state returned by the API, including Tasks.

Passing an object that already contains Tasks produces an error.

See: http://docs.confluent.io/current/connect/userguide.html#post--connectors

func (*Client) DeleteConnector 

func (c *Client) DeleteConnector(name string) (*http.Response, error)

DeleteConnector deletes a connector with the given name, halting all tasks and deleting its configuration.

See: http://docs.confluent.io/current/connect/userguide.html#delete--connectors-(string-name)-

func (*Client) Do 

func (c *Client) Do(req *http.Request, v interface{}) (*http.Response, error)

Do sends an API request and returns the API response. The API response is JSON-decoded and stored in the value pointed to by v, or returned as an error if an API or HTTP error has occurred.

func (*Client) GetClusterInfo added in v0.1.31 

func (c *Client) GetClusterInfo() (*ClusterInfo, *http.Response, error)

GetClusterInfo retrieves information about a cluster

See: https://docs.confluent.io/current/connect/references/restapi.html#kconnect-cluster

func (*Client) GetConnector 

func (c *Client) GetConnector(name string) (*Connector, *http.Response, error)

GetConnector retrieves information about a connector with the given name.

See: http://docs.confluent.io/current/connect/userguide.html#get--connectors-(string-name)

func (*Client) GetConnectorConfig 

func (c *Client) GetConnectorConfig(name string) (ConnectorConfig, *http.Response, error)

GetConnectorConfig retrieves configuration for a connector with the given name.

See: http://docs.confluent.io/current/connect/userguide.html#get--connectors-(string-name)-config

func (*Client) GetConnectorStatus 

func (c *Client) GetConnectorStatus(name string) (*ConnectorStatus, *http.Response, error)

GetConnectorStatus gets current status of the connector, including whether it is running, failed or paused, which worker it is assigned to, error information if it has failed, and the state of all its tasks.

See: http://docs.confluent.io/current/connect/userguide.html#get--connectors-(string-name)-status

func (*Client) GetConnectorTaskStatus 

func (c *Client) GetConnectorTaskStatus(name string, taskID int) (*TaskState, *http.Response, error)

GetConnectorTaskStatus gets the status of task for a connector.

See: https://docs.confluent.io/current/connect/references/restapi.html#get--connectors-(string-name)-tasks-(int-taskid)-status

func (*Client) GetConnectorTasks 

func (c *Client) GetConnectorTasks(name string) ([]Task, *http.Response, error)

GetConnectorTasks retrieves a list of tasks currently running for a connector with the given name.

See: http://docs.confluent.io/current/connect/userguide.html#get--connectors-(string-name)-tasks

func (*Client) Host 

func (c *Client) Host() string

Host returns the API root URL the Client is configured to talk to.

func (*Client) ListConnectors 

func (c *Client) ListConnectors() ([]string, *http.Response, error)

ListConnectors retrieves a list of active connector names.

See: http://docs.confluent.io/current/connect/userguide.html#get--connectors

func (*Client) ListPlugins 

func (c *Client) ListPlugins() ([]*Plugin, *http.Response, error)

ListPlugins retrieves a list of the installed plugins. Note that the API only checks for connectors on the worker that handles the request, which means it is possible to see inconsistent results, especially during a rolling upgrade if you add new connector jars See: https://docs.confluent.io/current/connect/references/restapi.html#get--connector-plugins-

func (*Client) NewRequest 

func (c *Client) NewRequest(method, path string, body interface{}) (*http.Request, error)

NewRequest creates an API request. A relative URL can be provided in path, in which case it is resolved relative to the BaseURL of the Client. Relative URLs should always be specified without a preceding slash. If specified, the value pointed to by body is JSON-encoded and included as the request body.

func (*Client) PauseConnector 

func (c *Client) PauseConnector(name string) (*http.Response, error)

PauseConnector pauses a connector and its tasks, which stops message processing until the connector is resumed. Tasks will transition to PAUSED state asynchronously.

See: http://docs.confluent.io/current/connect/userguide.html#put--connectors-(string-name)-pause

func (*Client) RestartConnector 

func (c *Client) RestartConnector(name string) (*http.Response, error)

RestartConnector restarts a connector and its tasks.

See http://docs.confluent.io/current/connect/userguide.html#post--connectors-(string-name)-restart

func (*Client) RestartConnectorTask 

func (c *Client) RestartConnectorTask(name string, taskID int) (*http.Response, error)

RestartConnectorTask restarts a tasks for a connector.

See https://docs.confluent.io/current/connect/references/restapi.html#post--connectors-(string-name)-tasks-(int-taskid)-restart

func (*Client) ResumeConnector 

func (c *Client) ResumeConnector(name string) (*http.Response, error)

ResumeConnector resumes a paused connector. Tasks will transition to RUNNING state asynchronously.

See: http://docs.confluent.io/current/connect/userguide.html#put--connectors-(string-name)-resume

func (*Client) UpdateConnectorConfig 

func (c *Client) UpdateConnectorConfig(name string, config ConnectorConfig) (*Connector, *http.Response, error)

UpdateConnectorConfig updates configuration for an existing connector with the given name, returning the new state of the Connector.

If the connector does not exist, it will be created, and the returned HTTP response will indicate a 201 Created status.

See: http://docs.confluent.io/current/connect/userguide.html#put--connectors-(string-name)-config

func (*Client) ValidatePlugins added in v0.1.33 

func (c *Client) ValidatePlugins(config ConnectorConfig) (*ConfigValidation, *http.Response, error)

ValidatePlugins validates the provided configuration values against the configuration definition. See: https://docs.confluent.io/current/connect/references/restapi.html#put--connector-plugins-(string-name)-config-validate

type ClusterInfo added in v0.1.31 

type ClusterInfo struct {
	Version        string `json:"version"`
	Commit         string `json:"commit"`
	KafkaClusterID string `json:"kafka_cluster_id"`
}

ClusterInfo - this is new and not from the original author

type ConfigValidation added in v0.1.33 

type ConfigValidation struct {
	Name       string            `json:"name"`
	ErrorCount int               `json:"error_count"`
	Groups     []string          `json:"groups"`
	Configs    []FieldValidation `json:"configs"`
}

type Connector 

type Connector struct {
	Name   string          `json:"name"`
	Config ConnectorConfig `json:"config,omitempty"`
	Tasks  []TaskID        `json:"tasks,omitempty"`
}

Connector represents a Kafka Connect connector instance.

See: http://docs.confluent.io/current/connect/userguide.html#connectors-tasks-and-workers

func (*Connector) ConfigEqual 

func (c *Connector) ConfigEqual(other Connector) bool

ConfigEqual will compare 2 instances of a connector config and return true or false if they are equal.

Note: tasks are ignored in the comparison

type ConnectorConfig 

type ConnectorConfig map[string]string

ConnectorConfig is a key-value mapping of configuration for connectors, where keys are in the form of Java properties.

See: http://docs.confluent.io/current/connect/userguide.html#configuring-connectors

type ConnectorState 

type ConnectorState struct {
	State    string `json:"state"`
	WorkerID string `json:"worker_id"`
}

ConnectorState reflects the running state of a Connector and the worker where it is running.

type ConnectorStatus 

type ConnectorStatus struct {
	Name      string         `json:"name"`
	Connector ConnectorState `json:"connector"`
	Tasks     []TaskState    `json:"tasks"`
}

ConnectorStatus reflects the status of a Connector and state of its Tasks.

Having connector name and a "connector" object at top level is a little awkward and produces stuttering, but it's their design, not ours.

type FieldDefinition added in v0.1.33 

type FieldDefinition struct {
	Name          string                   `json:"name"`
	Type          string                   `json:"type"`
	Required      bool                     `json:"required"`
	DefaultValue  *string                  `json:"default_value"`
	Importance    string                   `json:"importance"`
	Documentation string                   `json:"documentation"`
	Group         string                   `json:"group"`
	Width         string                   `json:"width"`
	DisplayName   string                   `json:"display_name"`
	Dependents    []map[string]interface{} `json:"dependents"` //unknown type
	Order         int                      `json:"order"`
}

type FieldValidation added in v0.1.33 

type FieldValidation struct {
	Definition FieldDefinition `json:"definition"`
	Value      FieldValue      `json:"value"`
}

type FieldValue added in v0.1.33 

type FieldValue struct {
	Name              string    `json:"name"`
	Value             *string   `json:"value"`
	RecommendedValues []*string `json:"recommended_values"`
	Errors            []string  `json:"errors"`
	Visible           bool      `json:"visible"`
}

type Option added in v0.1.6 

type Option func(c *Client)

Option can be supplied that override the default Clients properties

func WithHTTPClient added in v0.1.6 

func WithHTTPClient(httpClient *http.Client) Option

WithHTTPClient allows a specific http.Client to be set

func WithUserAgent added in v0.1.6 

func WithUserAgent(userAgent string) Option

WithUserAgent allows the userAgent to be overridden

type Plugin 

type Plugin struct {
	Class   string `json:"class"`
	Type    string `json:"type"`
	Version string `json:"version"`
}

Plugin represents a Kafka Connect connector plugin

type Task 

type Task struct {
	ID     TaskID            `json:"id"`
	Config map[string]string `json:"config"`
}

Task is a unit of work dispatched by a Connector to parallelize the work of a data copy job.

See: http://docs.confluent.io/current/connect/userguide.html#connectors-tasks-and-workers

type TaskID 

type TaskID struct {
	ConnectorName string `json:"connector"`
	ID            int    `json:"task"`
}

TaskID NOTE: Code originally from https://github.com/go-kafka/connect as two components, a numerical ID and a connector name by which the ID is scoped.

type TaskState 

type TaskState struct {
	ID       int    `json:"id"`
	State    string `json:"state"`
	WorkerID string `json:"worker_id"`
	Trace    string `json:"trace,omitempty"`
}

TaskState reflects the running state of a Task and the worker where it is running.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.