vulcanizer

README

vulcanizer

GitHub's ops focused Elasticsearch library

This project is a golang library for interacting with an Elasticsearch cluster. It's goal is to provide a high level API to help with common tasks that are associated with operating an Elasticsearch cluster such as querying health status of the cluster, migrating data off of nodes, updating cluster settings, etc.

This project does not aim to be a fully-featured API client for querying or indexing to Elasticsearch.

Go API

You can perform custom operations in your Go application.

import "github.com/github/vulcanizer"

v = vulcanizer.NewClient("localhost", 9200)
oldSetting, newSetting, err := v.SetSetting("indices.recovery.max_bytes_per_sec", "1000mb")

Command line application

This project produces a vulcanizer binary that is a command line application that can be used to manage your Elasticsearch cluster.

$ vulcanizer -h
Usage:
  vulcanizer [command]

Available Commands:
  allocation  Set shard allocation on the cluster.
  drain       Drain a server or see what servers are draining.
  fill        Fill servers with data, removing shard allocation exclusion rules.
  health      Display the health of the cluster.
  help        Help about any command
  indices     Display the indices of the cluster.
  nodes       Display the nodes of the cluster.
  setting     Interact with cluster settings.
  settings    Display all the settings of the cluster.
  snapshot    Interact with a specific snapshot.
  snapshots   Display the snapshots of the cluster.

Flags:
  -c, --cluster string      Cluster to connect to defined in config file
  -f, --configFile string   Configuration file to read in (default to "~/.vulcanizer.yaml")
  -h, --help                help for vulcanizer
      --host string         Host to connect to (default "localhost")
  -p, --port int            Port to connect to (default 9200)

Use "vulcanizer [command] --help" for more information about a command.

Commands to be implemented

• Listing repositories
• Verifying repositories
• Taking snapshots
• Deleting snapshots
• Displaying index settings
• Updating index settings

Configuration and connection information

All commands take --cluster <name> to look up information in a configuration file in ~/.vulcanizer.yaml. The configuration should be in the form of

local:
  host: localhost
  port: 9200
staging:
  host: 10.10.2.1
  port: 9201
production:
  host: 10.10.1.1
  port: 9202

Alternatively, all commands take --host and --port for the connection information.

For example:

# Query for cluster health on the "local" cluster
vulcanizer health --cluster local

# Query for nodes against the node 10.10.2.1 and port 9202
vulcanizer nodes --host 10.10.2.1 --port 9202

Development

./script/build will compile the project and install the vulcanizer binary to $GOPATH/bin.

./script/test will run the tests in the project.

Supported Elasticsearch versions

To be determined

Name

Vulcanization is the process of making rubber more elastic, so vulcanizer is the library that makes Elasticsearch easier to work with!

Project status

This project is under active development.

Contributing

This repository is open to contributions. Please also see code of conduct

To get up and running, install the project into your $GOPATH and run the set up scripts.

go get github.com/github/vulcanizer

cd $GOPATH/src/github.com/github/vulcanizer

./script/bootstrap
./script/test

And the test suite should execute correctly.

License

This project is released under the MIT LICENSE. Please note it includes 3rd party dependencies release under their own licenses; these are found under vendor.

Authors

Authored by GitHub Engineering

Documentation

Index

• type Client
  • func NewClient(host string, port int) *Client
  • func (c *Client) DrainServer(serverToDrain string) (ExcludeSettings, error)
  • func (c *Client) FillAll() (ExcludeSettings, error)
  • func (c *Client) FillOneServer(serverToFill string) (ExcludeSettings, error)
  • func (c *Client) GetClusterExcludeSettings() (ExcludeSettings, error)
  • func (c *Client) GetHealth() ([]ClusterHealth, error)
  • func (c *Client) GetIndices() ([]Index, error)
  • func (c *Client) GetNodes() ([]Node, error)
  • func (c *Client) GetSettings() (ClusterSettings, error)
  • func (c *Client) GetSnapshotStatus(repository string, snapshot string) (Snapshot, error)
  • func (c *Client) GetSnapshots(repository string) ([]Snapshot, error)
  • func (c *Client) SetAllocation(allocation string) (string, error)
  • func (c *Client) SetSetting(setting string, value string) (string, string, error)
• type ClusterHealth
• type ClusterSetting
• type ClusterSettings
• type ExcludeSettings
• type Index
• type Node
• type Snapshot

Types

type Client

type Client struct {
	Host string
	Port int
}

func NewClient

func NewClient(host string, port int) *Client

func (*Client) DrainServer

func (c *Client) DrainServer(serverToDrain string) (ExcludeSettings, error)

func (*Client) FillAll

func (c *Client) FillAll() (ExcludeSettings, error)

func (*Client) FillOneServer

func (c *Client) FillOneServer(serverToFill string) (ExcludeSettings, error)

func (*Client) GetClusterExcludeSettings

func (c *Client) GetClusterExcludeSettings() (ExcludeSettings, error)

Get current cluster settings for exclusion

func (*Client) GetHealth

func (c *Client) GetHealth() ([]ClusterHealth, error)

func (*Client) GetIndices

func (c *Client) GetIndices() ([]Index, error)

func (*Client) GetNodes

func (c *Client) GetNodes() ([]Node, error)

func (*Client) GetSettings

func (c *Client) GetSettings() (ClusterSettings, error)

func (*Client) GetSnapshotStatus

func (c *Client) GetSnapshotStatus(repository string, snapshot string) (Snapshot, error)

func (*Client) GetSnapshots

func (c *Client) GetSnapshots(repository string) ([]Snapshot, error)

func (*Client) SetAllocation

func (c *Client) SetAllocation(allocation string) (string, error)

func (*Client) SetSetting

func (c *Client) SetSetting(setting string, value string) (string, string, error)

type ClusterHealth

type ClusterHealth struct {
	Cluster                string `json:"cluster"`
	Status                 string `json:"status"`
	RelocatingShards       int    `json:"relo,string"`
	InitializingShards     int    `json:"init,string"`
	UnassignedShards       int    `json:"unassign,string"`
	ActiveShardsPercentage string `json:"active_shards_percent"`
	Message                string
}

type ClusterSetting

type ClusterSetting struct {
	Setting, Value string
}

type ClusterSettings

type ClusterSettings struct {
	PersistentSettings []ClusterSetting
	TransientSettings  []ClusterSetting
}

type ExcludeSettings

type ExcludeSettings struct {
	Ips, Hosts, Names []string
}

type Index

type Index struct {
	Health        string `json:"health"`
	Status        string `json:"status"`
	Name          string `json:"index"`
	PrimaryShards int    `json:"pri,string"`
	ReplicaCount  int    `json:"rep,string"`
	IndexSize     string `json:"store.size"`
	DocumentCount int    `json:"docs.count,string"`
}

type Node

type Node struct {
	Name   string `json:"name"`
	Ip     string `json:"ip"`
	Id     string `json:"id"`
	Role   string `json:"role"`
	Master string `json:"master"`
}

type Snapshot

type Snapshot struct {
	State          string    `json:"state"`
	Name           string    `json:"snapshot"`
	StartTime      time.Time `json:"start_time,string"`
	EndTime        time.Time `json:"end_time,string"`
	DurationMillis int       `json:"duration_in_millis"`
	Indices        []string  `json:"indices"`
	Shards         struct {
		Total      int `json:"total"`
		Failed     int `json:"failed"`
		Successful int `json:"successful"`
	} `json:"shards"`
	Failures []struct {
		Index   string `json:"index"`
		ShardID int    `json:"shard_id"`
		Reason  string `json:"reason"`
		NodeID  string `json:"node_id"`
		Status  string `json:"status"`
	} `json:"failures"`
}