elastic

package
Version: v0.23.6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 24, 2016 License: MIT, Apache-2.0 Imports: 20 Imported by: 0

README

Elastic

Elastic is an Elasticsearch client for the Go programming language.

Build Status Godoc license

See the wiki for additional information about Elastic.

Releases

Notice that the master branch always refers to the latest version of Elastic. If you want to use stable versions of Elastic, you should use the packages released via gopkg.in.

Here's the version matrix:

Elasticsearch version Elastic version - Package URL
2.x 3.0 beta gopkg.in/olivere/elastic.v3-unstable (source doc)
1.x 2.0 gopkg.in/olivere/elastic.v2 (source doc)
0.9-1.3 1.0 gopkg.in/olivere/elastic.v1 (source doc)

Example:

You have Elasticsearch 1.6.0 installed and want to use Elastic. As listed above, you should use Elastic 2.0. So you first install Elastic 2.0.

$ go get gopkg.in/olivere/elastic.v2

Then you use it via the following import path:

import "gopkg.in/olivere/elastic.v2"
Elastic 3.0

Elastic 3.0 targets Elasticsearch 2.x and is currently under active development. It is not published to gokpg yet.

There are a lot of breaking changes in Elasticsearch 2.0 and we will use this as an opportunity to clean up and refactor Elastic as well.

Elastic 2.0

Elastic 2.0 targets Elasticsearch 1.x and published via gopkg.in/olivere/elastic.v2.

Elastic 1.0

Elastic 1.0 is deprecated. You should really update Elasticsearch and Elastic to a recent version.

However, if you cannot update for some reason, don't worry. Version 1.0 is still available. All you need to do is go-get it and change your import path as described above.

Status

We use Elastic in production since 2012. Although Elastic is quite stable from our experience, we don't have a stable API yet. The reason for this is that Elasticsearch changes quite often and at a fast pace. At this moment we focus on features, not on a stable API.

Having said that, there have been no big API changes that required you to rewrite your application big time. More often than not it's renaming APIs and adding/removing features so that we are in sync with the Elasticsearch API.

Elastic has been used in production with the following Elasticsearch versions: 0.90, 1.0, 1.1, 1.2, 1.3, 1.4, and 1.5. Furthermore, we use Travis CI to test Elastic with the most recent versions of Elasticsearch and Go. See the .travis.yml file for the exact matrix and Travis for the results.

Elasticsearch has quite a few features. A lot of them are not yet implemented in Elastic (see below for details). I add features and APIs as required. It's straightforward to implement missing pieces. I'm accepting pull requests :-)

Having said that, I hope you find the project useful.

Usage

The first thing you do is to create a Client. The client connects to Elasticsearch on http://127.0.0.1:9200 by default.

You typically create one client for your app. Here's a complete example.

// Create a client
client, err := elastic.NewClient()
if err != nil {
    // Handle error
}

// Create an index
_, err = client.CreateIndex("twitter").Do()
if err != nil {
    // Handle error
    panic(err)
}

// Add a document to the index
tweet := Tweet{User: "olivere", Message: "Take Five"}
_, err = client.Index().
    Index("twitter").
    Type("tweet").
    Id("1").
    BodyJson(tweet).
    Do()
if err != nil {
    // Handle error
    panic(err)
}

// Search with a term query
termQuery := elastic.NewTermQuery("user", "olivere")
searchResult, err := client.Search().
    Index("twitter").   // search in index "twitter"
    Query(&termQuery).  // specify the query
    Sort("user", true). // sort by "user" field, ascending
    From(0).Size(10).   // take documents 0-9
    Pretty(true).       // pretty print request and response JSON
    Do()                // execute
if err != nil {
    // Handle error
    panic(err)
}

// searchResult is of type SearchResult and returns hits, suggestions,
// and all kinds of other information from Elasticsearch.
fmt.Printf("Query took %d milliseconds\n", searchResult.TookInMillis)

// Each is a convenience function that iterates over hits in a search result.
// It makes sure you don't need to check for nil values in the response.
// However, it ignores errors in serialization. If you want full control
// over iterating the hits, see below.
var ttyp Tweet
for _, item := range searchResult.Each(reflect.TypeOf(ttyp)) {
    if t, ok := item.(Tweet); ok {
        fmt.Printf("Tweet by %s: %s\n", t.User, t.Message)
    }
}
// TotalHits is another convenience function that works even when something goes wrong.
fmt.Printf("Found a total of %d tweets\n", searchResult.TotalHits())

// Here's how you iterate through results with full control over each step.
if searchResult.Hits != nil {
    fmt.Printf("Found a total of %d tweets\n", searchResult.Hits.TotalHits)

    // Iterate through results
    for _, hit := range searchResult.Hits.Hits {
        // hit.Index contains the name of the index

        // Deserialize hit.Source into a Tweet (could also be just a map[string]interface{}).
        var t Tweet
        err := json.Unmarshal(*hit.Source, &t)
        if err != nil {
            // Deserialization failed
        }

        // Work with tweet
        fmt.Printf("Tweet by %s: %s\n", t.User, t.Message)
    }
} else {
    // No hits
    fmt.Print("Found no tweets\n")
}

// Delete the index again
_, err = client.DeleteIndex("twitter").Do()
if err != nil {
    // Handle error
    panic(err)
}

See the wiki for more details.

API Status

Here's the current API status.

APIs
  • Search (most queries, filters, facets, aggregations etc. are implemented: see below)
  • Index
  • Get
  • Delete
  • Delete By Query
  • Update
  • Multi Get
  • Bulk
  • Bulk UDP
  • Term vectors
  • Multi term vectors
  • Count
  • Validate
  • Explain
  • Search
  • Search shards
  • Search template
  • Facets (most are implemented, see below)
  • Aggregates (most are implemented, see below)
  • Multi Search
  • Percolate
  • More like this
  • Benchmark
Indices
  • Create index
  • Delete index
  • Get index
  • Indices exists
  • Open/close index
  • Put mapping
  • Get mapping
  • Get field mapping
  • Types exist
  • Delete mapping
  • Index aliases
  • Update indices settings
  • Get settings
  • Analyze
  • Index templates
  • Warmers
  • Status
  • Indices stats
  • Indices segments
  • Indices recovery
  • Clear cache
  • Flush
  • Refresh
  • Optimize
  • Upgrade
Snapshot and Restore
  • Snapshot
  • Restore
  • Snapshot status
  • Monitoring snapshot/restore progress
  • Partial restore
Cat APIs

Not implemented. Those are better suited for operating with Elasticsearch on the command line.

Cluster
  • Health
  • State
  • Stats
  • Pending cluster tasks
  • Cluster reroute
  • Cluster update settings
  • Nodes stats
  • Nodes info
  • Nodes hot_threads
  • Nodes shutdown
  • Inner hits (for ES >= 1.5.0; see docs)
Query DSL
Queries
  • match
  • multi_match
  • bool
  • boosting
  • common_terms
  • constant_score
  • dis_max
  • filtered
  • fuzzy_like_this_query (flt)
  • fuzzy_like_this_field_query (flt_field)
  • function_score
  • fuzzy
  • geo_shape
  • has_child
  • has_parent
  • ids
  • indices
  • match_all
  • mlt
  • mlt_field
  • nested
  • prefix
  • query_string
  • simple_query_string
  • range
  • regexp
  • span_first
  • span_multi_term
  • span_near
  • span_not
  • span_or
  • span_term
  • term
  • terms
  • top_children
  • wildcard
  • minimum_should_match
  • multi_term_query_rewrite
  • template_query
Filters
  • and
  • bool
  • exists
  • geo_bounding_box
  • geo_distance
  • geo_distance_range
  • geo_polygon
  • geoshape
  • geohash
  • has_child
  • has_parent
  • ids
  • indices
  • limit
  • match_all
  • missing
  • nested
  • not
  • or
  • prefix
  • query
  • range
  • regexp
  • script
  • term
  • terms
  • type
Facets
  • Terms
  • Range
  • Histogram
  • Date Histogram
  • Filter
  • Query
  • Statistical
  • Terms Stats
  • Geo Distance
Aggregations
  • min
  • max
  • sum
  • avg
  • stats
  • extended stats
  • value count
  • percentiles
  • percentile ranks
  • cardinality
  • geo bounds
  • top hits
  • scripted metric
  • global
  • filter
  • filters
  • missing
  • nested
  • reverse nested
  • children
  • terms
  • significant terms
  • range
  • date range
  • ipv4 range
  • histogram
  • date histogram
  • geo distance
  • geohash grid
Sorting
  • Sort by score
  • Sort by field
  • Sort by geo distance
  • Sort by script
Scan

Scrolling through documents (e.g. search_type=scan) are implemented via the Scroll and Scan services. The ClearScroll API is implemented as well.

How to contribute

Read the contribution guidelines.

Credits

Thanks a lot for the great folks working hard on Elasticsearch and Go.

LICENSE

MIT-LICENSE. See LICENSE or the LICENSE file provided in the repository for details.

Documentation

Overview

Package elastic provides an interface to the Elasticsearch server (http://www.elasticsearch.org/).

The first thing you do is to create a Client. If you have Elasticsearch installed and running with its default settings (i.e. available at http://127.0.0.1:9200), all you need to do is:

client, err := elastic.NewClient()
if err != nil {
	// Handle error
}

If your Elasticsearch server is running on a different IP and/or port, just provide a URL to NewClient:

// Create a client and connect to http://192.168.2.10:9201
client, err := elastic.NewClient(elastic.SetURL("http://192.168.2.10:9201"))
if err != nil {
  // Handle error
}

You can pass many more configuration parameters to NewClient. Review the documentation of NewClient for more information.

If no Elasticsearch server is available, services will fail when creating a new request and will return ErrNoClient.

A Client provides services. The services usually come with a variety of methods to prepare the query and a Do function to execute it against the Elasticsearch REST interface and return a response. Here is an example of the IndexExists service that checks if a given index already exists.

exists, err := client.IndexExists("twitter").Do()
if err != nil {
	// Handle error
}
if !exists {
	// Index does not exist yet.
}

Look up the documentation for Client to get an idea of the services provided and what kinds of responses you get when executing the Do function of a service. Also see the wiki on Github for more details.

Index

Constants

View Source
const (
	// Version is the current version of Elastic.
	Version = "2.0.12"

	// DefaultUrl is the default endpoint of Elasticsearch on the local machine.
	// It is used e.g. when initializing a new Client without a specific URL.
	DefaultURL = "http://127.0.0.1:9200"

	// DefaultScheme is the default protocol scheme to use when sniffing
	// the Elasticsearch cluster.
	DefaultScheme = "http"

	// DefaultHealthcheckEnabled specifies if healthchecks are enabled by default.
	DefaultHealthcheckEnabled = true

	// DefaultHealthcheckTimeoutStartup is the time the healthcheck waits
	// for a response from Elasticsearch on startup, i.e. when creating a
	// client. After the client is started, a shorter timeout is commonly used
	// (its default is specified in DefaultHealthcheckTimeout).
	DefaultHealthcheckTimeoutStartup = 5 * time.Second

	// DefaultHealthcheckTimeout specifies the time a running client waits for
	// a response from Elasticsearch. Notice that the healthcheck timeout
	// when a client is created is larger by default (see DefaultHealthcheckTimeoutStartup).
	DefaultHealthcheckTimeout = 1 * time.Second

	// DefaultHealthcheckInterval is the default interval between
	// two health checks of the nodes in the cluster.
	DefaultHealthcheckInterval = 60 * time.Second

	// DefaultSnifferEnabled specifies if the sniffer is enabled by default.
	DefaultSnifferEnabled = true

	// DefaultSnifferInterval is the interval between two sniffing procedures,
	// i.e. the lookup of all nodes in the cluster and their addition/removal
	// from the list of actual connections.
	DefaultSnifferInterval = 15 * time.Minute

	// DefaultSnifferTimeoutStartup is the default timeout for the sniffing
	// process that is initiated while creating a new client. For subsequent
	// sniffing processes, DefaultSnifferTimeout is used (by default).
	DefaultSnifferTimeoutStartup = 5 * time.Second

	// DefaultSnifferTimeout is the default timeout after which the
	// sniffing process times out. Notice that for the initial sniffing
	// process, DefaultSnifferTimeoutStartup is used.
	DefaultSnifferTimeout = 2 * time.Second

	// DefaultMaxRetries is the number of retries for a single request after
	// Elastic will give up and return an error. It is zero by default, so
	// retry is disabled by default.
	DefaultMaxRetries = 0
)

Variables

View Source
var (
	// ErrNoClient is raised when no Elasticsearch node is available.
	ErrNoClient = errors.New("no Elasticsearch node available")

	// ErrRetry is raised when a request cannot be executed after the configured
	// number of retries.
	ErrRetry = errors.New("cannot connect after several retries")

	// ErrTimeout is raised when a request timed out, e.g. when WaitForStatus
	// didn't return in time.
	ErrTimeout = errors.New("timeout")
)
View Source
var (
	// ErrMissingIndex is returned e.g. from DeleteService if the index is missing.
	ErrMissingIndex = errors.New("elastic: index is missing")

	// ErrMissingType is returned e.g. from DeleteService if the type is missing.
	ErrMissingType = errors.New("elastic: type is missing")

	// ErrMissingId is returned e.g. from DeleteService if the document identifier is missing.
	ErrMissingId = errors.New("elastic: id is missing")
)
View Source
var (
	// End of stream (or scan)
	EOS = errors.New("EOS")

	// No ScrollId
	ErrNoScrollId = errors.New("no scrollId")
)

Functions

func SetDecoder

func SetDecoder(decoder Decoder) func(*Client) error

SetDecoder sets the Decoder to use when decoding data from Elasticsearch. DefaultDecoder is used by default.

func SetErrorLog

func SetErrorLog(logger *log.Logger) func(*Client) error

SetErrorLog sets the logger for critical messages like nodes joining or leaving the cluster or failing requests. It is nil by default.

func SetInfoLog

func SetInfoLog(logger *log.Logger) func(*Client) error

SetInfoLog sets the logger for informational messages, e.g. requests and their response times. It is nil by default.

func SetMaxRetries

func SetMaxRetries(maxRetries int) func(*Client) error

SetMaxRetries sets the maximum number of retries before giving up when performing a HTTP request to Elasticsearch.

func SetTraceLog

func SetTraceLog(logger *log.Logger) func(*Client) error

SetTraceLog specifies the log.Logger to use for output of HTTP requests and responses which is helpful during debugging. It is nil by default.

Types

type Aggregation

type Aggregation interface {
	Source() interface{}
}

Aggregations can be seen as a unit-of-work that build analytic information over a set of documents. It is (in many senses) the follow-up of facets in Elasticsearch. For more details about aggregations, visit: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-aggregations.html

type AggregationBucketFilters

type AggregationBucketFilters struct {
	Aggregations

	Buckets      []*AggregationBucketKeyItem          //`json:"buckets"`
	NamedBuckets map[string]*AggregationBucketKeyItem //`json:"buckets"`
}

AggregationBucketFilters is a multi-bucket aggregation that is returned with a filters aggregation.

func (*AggregationBucketFilters) UnmarshalJSON

func (a *AggregationBucketFilters) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes JSON data and initializes an AggregationBucketFilters structure.

type AggregationBucketHistogramItem

type AggregationBucketHistogramItem struct {
	Aggregations

	Key         int64   //`json:"key"`
	KeyAsString *string //`json:"key_as_string"`
	DocCount    int64   //`json:"doc_count"`
}

AggregationBucketHistogramItem is a single bucket of an AggregationBucketHistogramItems structure.

func (*AggregationBucketHistogramItem) UnmarshalJSON

func (a *AggregationBucketHistogramItem) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes JSON data and initializes an AggregationBucketHistogramItem structure.

type AggregationBucketHistogramItems

type AggregationBucketHistogramItems struct {
	Aggregations

	Buckets []*AggregationBucketHistogramItem //`json:"buckets"`
}

AggregationBucketHistogramItems is a bucket aggregation that is returned with a date histogram aggregation.

func (*AggregationBucketHistogramItems) UnmarshalJSON

func (a *AggregationBucketHistogramItems) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes JSON data and initializes an AggregationBucketHistogramItems structure.

type AggregationBucketKeyItem

type AggregationBucketKeyItem struct {
	Aggregations

	Key       interface{} //`json:"key"`
	KeyNumber json.Number
	DocCount  int64 //`json:"doc_count"`
}

AggregationBucketKeyItem is a single bucket of an AggregationBucketKeyItems structure.

func (*AggregationBucketKeyItem) UnmarshalJSON

func (a *AggregationBucketKeyItem) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes JSON data and initializes an AggregationBucketKeyItem structure.

type AggregationBucketKeyItems

type AggregationBucketKeyItems struct {
	Aggregations

	DocCountErrorUpperBound int64                       //`json:"doc_count_error_upper_bound"`
	SumOfOtherDocCount      int64                       //`json:"sum_other_doc_count"`
	Buckets                 []*AggregationBucketKeyItem //`json:"buckets"`
}

AggregationBucketKeyItems is a bucket aggregation that is e.g. returned with a terms aggregation.

func (*AggregationBucketKeyItems) UnmarshalJSON

func (a *AggregationBucketKeyItems) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes JSON data and initializes an AggregationBucketKeyItems structure.

type AggregationBucketKeyedRangeItems

type AggregationBucketKeyedRangeItems struct {
	Aggregations

	DocCountErrorUpperBound int64                                  //`json:"doc_count_error_upper_bound"`
	SumOfOtherDocCount      int64                                  //`json:"sum_other_doc_count"`
	Buckets                 map[string]*AggregationBucketRangeItem //`json:"buckets"`
}

AggregationBucketKeyedRangeItems is a bucket aggregation that is e.g. returned with a keyed range aggregation.

func (*AggregationBucketKeyedRangeItems) UnmarshalJSON

func (a *AggregationBucketKeyedRangeItems) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes JSON data and initializes an AggregationBucketRangeItems structure.

type AggregationBucketRangeItem

type AggregationBucketRangeItem struct {
	Aggregations

	Key          string   //`json:"key"`
	DocCount     int64    //`json:"doc_count"`
	From         *float64 //`json:"from"`
	FromAsString string   //`json:"from_as_string"`
	To           *float64 //`json:"to"`
	ToAsString   string   //`json:"to_as_string"`
}

AggregationBucketRangeItem is a single bucket of an AggregationBucketRangeItems structure.

func (*AggregationBucketRangeItem) UnmarshalJSON

func (a *AggregationBucketRangeItem) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes JSON data and initializes an AggregationBucketRangeItem structure.

type AggregationBucketRangeItems

type AggregationBucketRangeItems struct {
	Aggregations

	DocCountErrorUpperBound int64                         //`json:"doc_count_error_upper_bound"`
	SumOfOtherDocCount      int64                         //`json:"sum_other_doc_count"`
	Buckets                 []*AggregationBucketRangeItem //`json:"buckets"`
}

AggregationBucketRangeItems is a bucket aggregation that is e.g. returned with a range aggregation.

func (*AggregationBucketRangeItems) UnmarshalJSON

func (a *AggregationBucketRangeItems) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes JSON data and initializes an AggregationBucketRangeItems structure.

type AggregationBucketSignificantTerm

type AggregationBucketSignificantTerm struct {
	Aggregations

	Key      string  //`json:"key"`
	DocCount int64   //`json:"doc_count"`
	BgCount  int64   //`json:"bg_count"`
	Score    float64 //`json:"score"`
}

AggregationBucketSignificantTerm is a single bucket of an AggregationBucketSignificantTerms structure.

func (*AggregationBucketSignificantTerm) UnmarshalJSON

func (a *AggregationBucketSignificantTerm) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes JSON data and initializes an AggregationBucketSignificantTerm structure.

type AggregationBucketSignificantTerms

type AggregationBucketSignificantTerms struct {
	Aggregations

	DocCount int64                               //`json:"doc_count"`
	Buckets  []*AggregationBucketSignificantTerm //`json:"buckets"`
}

AggregationBucketSignificantTerms is a bucket aggregation returned with a significant terms aggregation.

func (*AggregationBucketSignificantTerms) UnmarshalJSON

func (a *AggregationBucketSignificantTerms) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes JSON data and initializes an AggregationBucketSignificantTerms structure.

type AggregationExtendedStatsMetric

type AggregationExtendedStatsMetric struct {
	Aggregations

	Count        int64    // `json:"count"`
	Min          *float64 //`json:"min,omitempty"`
	Max          *float64 //`json:"max,omitempty"`
	Avg          *float64 //`json:"avg,omitempty"`
	Sum          *float64 //`json:"sum,omitempty"`
	SumOfSquares *float64 //`json:"sum_of_squares,omitempty"`
	Variance     *float64 //`json:"variance,omitempty"`
	StdDeviation *float64 //`json:"std_deviation,omitempty"`
}

AggregationExtendedStatsMetric is a multi-value metric, returned by an ExtendedStats aggregation.

func (*AggregationExtendedStatsMetric) UnmarshalJSON

func (a *AggregationExtendedStatsMetric) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes JSON data and initializes an AggregationExtendedStatsMetric structure.

type AggregationGeoBoundsMetric

type AggregationGeoBoundsMetric struct {
	Aggregations

	Bounds struct {
		TopLeft struct {
			Latitude  float64 `json:"lat"`
			Longitude float64 `json:"lon"`
		} `json:"top_left"`
		BottomRight struct {
			Latitude  float64 `json:"lat"`
			Longitude float64 `json:"lon"`
		} `json:"bottom_right"`
	} `json:"bounds"`
}

AggregationGeoBoundsMetric is a metric as returned by a GeoBounds aggregation.

func (*AggregationGeoBoundsMetric) UnmarshalJSON

func (a *AggregationGeoBoundsMetric) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes JSON data and initializes an AggregationGeoBoundsMetric structure.

type AggregationPercentilesMetric

type AggregationPercentilesMetric struct {
	Aggregations

	Values map[string]float64 // `json:"values"`
}

AggregationPercentilesMetric is a multi-value metric, returned by a Percentiles aggregation.

func (*AggregationPercentilesMetric) UnmarshalJSON

func (a *AggregationPercentilesMetric) UnmarshalJSON(data []byte) error

UnmarshalJSON decodes JSON data and initializes an AggregationPercentilesMetric structure.