elasticsearch

package module
Version: v0.0.0-...-6b39f24 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 30, 2013 License: BSD-2-Clause Imports: 12 Imported by: 1

README

elasticsearch

This is an opinionated library for ElasticSearch in Go. Its opinions are:

  • Builders are bad: construct queries declaratively, using nested structures
  • Cleverness is bad: when in doubt, be explicit and dumb

Build Status

Usage

First, it helps to import the package with a short name (package alias).

import es "github.com/peterbourgon/elasticsearch"

Create a Cluster, which is an actively-managed handle to a set of nodes.

endpoints := []string{"http://host1:9200", "http://host2:9200"}
pingInterval, pingTimeout := 30*time.Second, 3*time.Second
c := es.NewCluster(endpoints, pingInterval, pingTimeout)

Construct queries declaratively, and fire them against the cluster.

q := es.QueryWrapper(
	es.TermQuery(es.TermQueryParams{
		Query: &es.Wrapper{
			Name:    "user",
			Wrapped: "kimchy",
		},
	}),
)

request := &es.SearchRequest{
	Params: es.SearchParams{
		Indices: []string{"twitter"},
		Types:   []string{"tweet"},
	},
	Query:   q,
}

response, err := c.Search(request)
if err != nil {
	// Fatal
}
fmt.Printf("got %d hit(s)", response.HitsWrapper.Total)

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type BoolQueryParams

type BoolQueryParams struct {
	Must                     SubQuery `json:"must,omitempty"` // can be slice!
	Should                   SubQuery `json:"should,omitempty"`
	MustNot                  SubQuery `json:"must_not,omitempty"`
	MinimumNumberShouldMatch int      `json:"minimum_number_should_match,omitempty"`
	Boost                    float32  `json:"boost,omitempty"`
}

http://www.elasticsearch.org/guide/reference/query-dsl/bool-query.html

type BooleanFiltersParams

type BooleanFiltersParams struct {
	AndFilters []FilterSubQuery
	OrFilters  []FilterSubQuery
}

type BulkIndexable

type BulkIndexable interface {
	EncodeBulkHeader(*json.Encoder) error
	EncodeSource(*json.Encoder) error
}

type BulkItemResponse

type BulkItemResponse IndexResponse

func (*BulkItemResponse) UnmarshalJSON

func (r *BulkItemResponse) UnmarshalJSON(data []byte) error

Bulk responses are wrapped in an extra object whose only key is the operation performed (create, delete, or index). BulkItemResponse response is an alias for IndexResponse, but deals with this extra indirection.

type BulkParams

type BulkParams struct {
	Consistency string
	Refresh     string
	Replication string
}

func (BulkParams) Values

func (p BulkParams) Values() url.Values

type BulkRequest

type BulkRequest struct {
	Params   BulkParams
	Requests []BulkIndexable
}

func (BulkRequest) Request

func (r BulkRequest) Request(uri *url.URL) (*http.Request, error)

type BulkResponse

type BulkResponse struct {
	Took int `json:"took"` // ms

	Items []BulkItemResponse `json:"items"`
}

type Cluster

type Cluster struct {
	// contains filtered or unexported fields
}

A Cluster is an actively-managed collection of Nodes. Cluster implements Searcher, so you can treat it as a single entity. Its Search method chooses the best Node to receive the Request.

func NewCluster

func NewCluster(endpoints []string, pingInterval, pingTimeout time.Duration) *Cluster

NewCluster returns a new, actively-managed Cluster, representing the passed endpoints as Nodes. Each endpoint should be of the form scheme://host:port, for example http://es001:9200.

The Cluster will ping each Node on a schedule dictated by pingInterval. Each node has pingTimeout to respond before the ping is marked as failed.

TODO node discovery from the list of seed-nodes.

func (*Cluster) Bulk

func (c *Cluster) Bulk(r BulkRequest) (response BulkResponse, err error)

func (*Cluster) Create

func (c *Cluster) Create(r CreateRequest) (response IndexResponse, err error)

func (*Cluster) Delete

func (c *Cluster) Delete(r DeleteRequest) (response IndexResponse, err error)

func (*Cluster) Execute

func (c *Cluster) Execute(f Fireable, response interface{}) error

Executes the request against a suitable node and decodes server's reply into response.

func (*Cluster) Index

func (c *Cluster) Index(r IndexRequest) (response IndexResponse, err error)

func (*Cluster) MultiSearch

func (c *Cluster) MultiSearch(r MultiSearchRequest) (response MultiSearchResponse, err error)

MultiSearch implements the MultiSearcher interface for a Cluster. It executes the search request against a suitable node.

func (*Cluster) Search

func (c *Cluster) Search(r SearchRequest) (response SearchResponse, err error)

Search implements the Searcher interface for a Cluster. It executes the request against a suitable node.

func (*Cluster) Shutdown

func (c *Cluster) Shutdown()

Shutdown terminates the Cluster's event dispatcher.

func (*Cluster) Update

func (c *Cluster) Update(r UpdateRequest) (response IndexResponse, err error)

type ConstantScoreQueryParams

type ConstantScoreQueryParams struct {
	Query  SubQuery       `json:"query,omitempty"`
	Filter FilterSubQuery `json:"filter,omitempty"`
	Boost  float32        `json:"boost,omitempty"`
}

type CreateRequest

type CreateRequest struct {
	Params IndexParams
	Source interface{}
}

func (CreateRequest) EncodeBulkHeader

func (r CreateRequest) EncodeBulkHeader(enc *json.Encoder) error

func (CreateRequest) EncodeSource

func (r CreateRequest) EncodeSource(enc *json.Encoder) error

func (CreateRequest) Request

func (r CreateRequest) Request(uri *url.URL) (*http.Request, error)

type CustomScoreQueryParams

type CustomScoreQueryParams struct {
	Script string                 `json:"script"`
	Lang   string                 `json:"lang"`
	Params map[string]interface{} `json:"params"`
	Query  SubQuery               `json:"query"`
}

http://www.elasticsearch.org/guide/reference/query-dsl/custom-score-query.html

type DeleteRequest

type DeleteRequest struct {
	Params IndexParams
}

func (DeleteRequest) EncodeBulkHeader

func (r DeleteRequest) EncodeBulkHeader(enc *json.Encoder) error

func (DeleteRequest) EncodeSource

func (r DeleteRequest) EncodeSource(enc *json.Encoder) error

func (DeleteRequest) Request

func (r DeleteRequest) Request(uri *url.URL) (*http.Request, error)

type DisMaxQueryParams

type DisMaxQueryParams struct {
	Queries    []SubQuery `json:"queries"`
	Boost      float32    `json:"boost,omitempty"`
	TieBreaker float32    `json:"tie_breaker,omitempty"`
}

http://www.elasticsearch.org/guide/reference/query-dsl/dis-max-query.html

type FacetResponse

type FacetResponse struct {
	Type    string `json:"_type"`
	Missing int64  `json:"missing"`
	Total   int64  `json:"total"`
	Other   int64  `json:"other"`
	Terms   []struct {
		Term  string `json:"term"`
		Count int64  `json:"count"`
	} `json:"terms"`
}

type FacetSubQuery

type FacetSubQuery SubQuery

func NamedFacet

func NamedFacet(name string, q FacetSubQuery) FacetSubQuery

NamedFacet wraps any FooFacet SubQuery so that it can be used wherever a facet is called for.

func TermsFacet

func TermsFacet(p TermsFacetParams) FacetSubQuery

type FieldedFilterParams

type FieldedFilterParams struct {
	Value string `json:"value"`
}

http://www.elasticsearch.org/guide/reference/query-dsl/type-filter.html

type FilterSubQuery

type FilterSubQuery SubQuery

func FieldedFilter

func FieldedFilter(fieldName string, p FieldedFilterParams) FilterSubQuery

TODO I guess remove all Fielded* functions except FieldedGenericQuery? TODO and rename FieldedGenericQuery to like FieldedSubObject, maybe?

func MakeFilter

func MakeFilter(filter SubQuery) FilterSubQuery

func MakeFilters

func MakeFilters(filters []SubQuery) []FilterSubQuery

func QueryFilter

func QueryFilter(p QueryFilterParams) FilterSubQuery

func RangeFilter

func RangeFilter(q RangeSubQuery) FilterSubQuery

func TermFilter

func TermFilter(p TermFilterParams) FilterSubQuery

func TermsFilter

func TermsFilter(p TermsFilterParams) FilterSubQuery

type Fireable

type Fireable interface {
	Request(uri *url.URL) (*http.Request, error)
}

Fireable defines anything which can be fired against the search cluster.

type GenericQueryParams

type GenericQueryParams struct {
	Query              string  `json:"query,omitempty"`
	Analyzer           string  `json:"analyzer,omitempty"`
	Type               string  `json:"type,omitempty"`
	MaxExpansions      string  `json:"max_expansions,omitempty"`
	Boost              float32 `json:"boost,omitempty"`
	Operator           string  `json:"operator,omitempty"`
	MinimumShouldMatch string  `json:"minimum_should_match,omitempty"`
	CutoffFrequency    float32 `json:"cutoff_frequency,omitempty"`
}

GenericQueryParams marshal to a valid query object for a large number of query types. You generally use them applied to a particular field, ie. scope; see FieldedGenericQuery.

type Health

type Health int

Health is some encoding of the perceived state of a Node. A Cluster should favor sending queries against healthier nodes.

const (
	Green Health = iota // resemblance to cluster health codes is coincidental
	Yellow
	Red
)

func (Health) Degrade

func (h Health) Degrade() Health

func (Health) Improve

func (h Health) Improve() Health

func (Health) String

func (h Health) String() string

type IndexParams

type IndexParams struct {
	Index string `json:"_index"`
	Type  string `json:"_type"`
	Id    string `json:"_id"`

	Consistency string `json:"_consistency,omitempty"`
	Parent      string `json:"_parent,omitempty"`
	Percolate   string `json:"_percolate,omitempty"`
	Refresh     string `json:"_refresh,omitempty"`
	Replication string `json:"_replication,omitempty"`
	Routing     string `json:"_routing,omitempty"`
	TTL         string `json:"_ttl,omitempty"`
	Timestamp   string `json:"_timestamp,omitempty"`
	Version     string `json:"_version,omitempty"`
	VersionType string `json:"_version_type,omitempty"`
}

func (IndexParams) Values

func (p IndexParams) Values() url.Values

type IndexRequest

type IndexRequest struct {
	Params IndexParams
	Source interface{}
}

func (IndexRequest) EncodeBulkHeader

func (r IndexRequest) EncodeBulkHeader(enc *json.Encoder) error

func (IndexRequest) EncodeSource

func (r IndexRequest) EncodeSource(enc *json.Encoder) error

func (IndexRequest) Request

func (r IndexRequest) Request(uri *url.URL) (*http.Request, error)

type IndexResponse

type IndexResponse struct {
	Found   bool   `json:"found"`
	ID      string `json:"_id"`
	Index   string `json:"_index"`
	OK      bool   `json:"ok"`
	Type    string `json:"_type"`
	Version int    `json:"_version"`

	Error    string `json:"error,omitempty"`
	Status   int    `json:"status,omitempty"`
	TimedOut bool   `json:"timed_out,omitempty"`
}

type MultiSearchParams

type MultiSearchParams struct {
	Indices []string
	Types   []string

	SearchType string
}

func (MultiSearchParams) Values

func (p MultiSearchParams) Values() url.Values

type MultiSearchRequest

type MultiSearchRequest struct {
	Params   MultiSearchParams
	Requests []SearchRequest
}

func (MultiSearchRequest) Request

func (r MultiSearchRequest) Request(uri *url.URL) (*http.Request, error)

type MultiSearchResponse

type MultiSearchResponse struct {
	Responses []SearchResponse `json:"responses"`
}

type MultiSearcher

type MultiSearcher interface {
	MultiSearch(MultiSearchRequest) (MultiSearchResponse, error)
}

MultiSearcher is the interface that wraps the MultiSearch method.

type Node

type Node struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

A Node is a structure which represents a single ElasticSearch host.

func NewNode

func NewNode(endpoint string, pingTimeout time.Duration) *Node

NewNode constructs a Node handle. The endpoint should be of the form "scheme://host:port", eg. "http://es001:9200".

The ping interval is dictated at a higher level (the Cluster), but individual ping timeouts are stored with the Nodes themselves, in a custom HTTP client, with a timeout as part of the Transport dialer. This custom pingClient is used exclusively for Ping() calls.

Regular queries are made with the default client http.Client, which has no explicit timeout set in the Transport dialer.

func (*Node) Execute

func (n *Node) Execute(f Fireable, response interface{}) error

Executes the Fireable f against the node and decodes the server's reply into response.

func (*Node) GetHealth

func (n *Node) GetHealth() Health

GetHealth returns the health of the node, for use in the Cluster's GetBest.

func (*Node) Ping

func (n *Node) Ping() bool

Ping attempts to HTTP GET a specific endpoint, parse some kind of status indicator, and returns true if everything was successful.

type Nodes

type Nodes []*Node

type OffsetLimitFacetsFilterQueryParams

type OffsetLimitFacetsFilterQueryParams struct {
	Offset int            `json:"from"`
	Limit  int            `json:"size"`
	Facets FacetSubQuery  `json:"facets,omitempty"`
	Filter FilterSubQuery `json:"filter,omitempty"`
	Query  SubQuery       `json:"query"`
}

Haven't quite figured out how to best represent this. TODO break these up into embeddable query-parts?

type QueryFilterParams

type QueryFilterParams struct {
	Query SubQuery `json:"query"`
}

type RangeFilterParams

type RangeFilterParams struct {
	From         string `json:"from,omitempty"`
	To           string `json:"to,omitempty"`
	IncludeLower bool   `json:"include_lower"`
	IncludeUpper bool   `json:"include_upper"`
}

http://www.elasticsearch.org/guide/reference/query-dsl/range-filter.html

type RangeSubQuery

type RangeSubQuery SubQuery

func FieldedRangeSubQuery

func FieldedRangeSubQuery(field string, p RangeFilterParams) RangeSubQuery

type SearchParams

type SearchParams struct {
	Indices []string `json:"index,omitempty"`
	Types   []string `json:"type,omitempty"`

	Routing    string `json:"routing,omitempty"`
	Preference string `json:"preference,omitempty"`
	SearchType string `json:"search_type,omitempty"`
}

func (SearchParams) Values

func (p SearchParams) Values() url.Values

type SearchRequest

type SearchRequest struct {
	Params SearchParams
	Query  SubQuery
}

func (SearchRequest) EncodeMultiHeader

func (r SearchRequest) EncodeMultiHeader(enc *json.Encoder) error

func (SearchRequest) EncodeQuery

func (r SearchRequest) EncodeQuery(enc *json.Encoder) error

func (SearchRequest) Path

func (r SearchRequest) Path() string

func (SearchRequest) Request

func (r SearchRequest) Request(uri *url.URL) (*http.Request, error)

type SearchResponse

type SearchResponse struct {
	Took int `json:"took"` // ms

	HitsWrapper struct {
		Total int `json:"total"`
		Hits  []struct {
			Index string   `json:"_index"`
			Type  string   `json:"_type"`
			ID    string   `json:"_id"`
			Score *float64 `json:"_score"` // can be 'null' with constant_score
		} `json:"hits,omitempty"`
	} `json:"hits"`

	Facets map[string]FacetResponse `json:"facets,omitempty"`

	TimedOut bool   `json:"timed_out,omitempty"`
	Error    string `json:"error,omitempty"`
	Status   int    `json:"status,omitempty"`
}

SearchResponse represents the response given by ElasticSearch from a search query.

type Searcher

type Searcher interface {
	Search(SearchRequest) (SearchResponse, error)
}

Searcher is the interface that wraps the basic Search method. Search transforms a Request into a SearchResponse (or an error).

type SubQuery

type SubQuery interface{}

func BoolQuery

func BoolQuery(p BoolQueryParams) SubQuery

func BooleanFilters

func BooleanFilters(p BooleanFiltersParams) SubQuery

func ConstantScoreQuery

func ConstantScoreQuery(p ConstantScoreQueryParams) SubQuery

func CustomScoreQuery

func CustomScoreQuery(p CustomScoreQueryParams) SubQuery

func DisMaxQuery

func DisMaxQuery(p DisMaxQueryParams) SubQuery

func FieldedGenericQuery

func FieldedGenericQuery(field string, p GenericQueryParams) SubQuery

FieldedGenericQuery returns a SubQuery representing the passed QueryParams applied to the given scope, ie. field. The returned SubQuery can be used as the Query in a MatchQuery, for example.

func MatchAllQuery

func MatchAllQuery() SubQuery

func MatchQuery

func MatchQuery(p MatchQueryParams) SubQuery

func QueryWrapper

func QueryWrapper(q SubQuery) SubQuery

func TermQuery

func TermQuery(p TermQueryParams) SubQuery

func TermsQuery

func TermsQuery(p TermsQueryParams) SubQuery

type TermFilterParams

type TermFilterParams struct {
	Field string
	Value string // for multiple values, use TermsFilter
}

type TermQueryParams

type TermQueryParams struct {
	Query SubQuery `json:"term"`
}

http://www.elasticsearch.org/guide/reference/query-dsl/term-query.html Typically `Query` would be &Wrapper{Name: "fieldname", Wrapped: "value"}.

type TermsFacetParams

type TermsFacetParams struct {
	Field string `json:"field"`
	Size  int    `json:"size"`
}

http://www.elasticsearch.org/guide/reference/api/search/facets/terms-facet.html

type TermsFilterParams

type TermsFilterParams struct {
	Field     string
	Values    []string
	Execution string
}

type TermsQueryParams

type TermsQueryParams struct {
	Query SubQuery `json:"terms"` // often `{ "field": ["value1", "value2"] }`
}

http://www.elasticsearch.org/guide/reference/query-dsl/terms-query.html "a simpler syntax query for using a `bool` query with several `term` queries in the `should` clauses."

type UpdateRequest

type UpdateRequest struct {
	Params IndexParams
	Source interface{}
}

func (UpdateRequest) Request

func (r UpdateRequest) Request(uri *url.URL) (*http.Request, error)

type Wrapper

type Wrapper struct {
	Name    string
	Wrapped SubQuery
}

Wrapper gives a dynamic name to a SubQuery. For example, Name="foo" Wrapped=`{"bar": 123}` marshals to `{"foo": {"bar": 123}}`.

You *can* use this directly in your business-logic code, but if you do, it's probably a sign you should file an issue (or make a pull request) to give your specific use-case proper, first class support, via a FooQuery[Params] type-set.

func (*Wrapper) MarshalJSON

func (w *Wrapper) MarshalJSON() ([]byte, error)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL