openpointintime

Documentation

Overview

Open a point in time.

A search request by default runs against the most recent visible data of the target indices, which is called point in time. Elasticsearch pit (point in time) is a lightweight view into the state of the data as it existed when initiated. In some cases, it’s preferred to perform multiple search requests using the same point in time. For example, if refreshes happen between `search_after` requests, then the results of those requests might not be consistent as changes happening between searches are only visible to the more recent point in time.

A point in time must be opened explicitly before being used in search requests.

A subsequent search request with the `pit` parameter must not specify `index`, `routing`, or `preference` values as these parameters are copied from the point in time.

Just like regular searches, you can use `from` and `size` to page through point in time search results, up to the first 10,000 hits. If you want to retrieve more hits, use PIT with `search_after`.

IMPORTANT: The open point in time request and each subsequent search request can return different identifiers; always use the most recently received ID for the next search request.

When a PIT that contains shard failures is used in a search request, the missing are always reported in the search response as a `NoShardAvailableActionException` exception. To get rid of these exceptions, a new PIT needs to be created so that shards missing from the previous PIT can be handled, assuming they become available in the meantime.

**Keeping point in time alive**

The `keep_alive` parameter, which is passed to a open point in time request and search request, extends the time to live of the corresponding point in time. The value does not need to be long enough to process all data — it just needs to be long enough for the next request.

Normally, the background merge process optimizes the index by merging together smaller segments to create new, bigger segments. Once the smaller segments are no longer needed they are deleted. However, open point-in-times prevent the old segments from being deleted since they are still in use.

TIP: Keeping older segments alive means that more disk space and file handles are needed. Ensure that you have configured your nodes to have ample free file handles.

Additionally, if a segment contains deleted or updated documents then the point in time must keep track of whether each document in the segment was live at the time of the initial search request. Ensure that your nodes have sufficient heap space if you have many open point-in-times on an index that is subject to ongoing deletes or updates. Note that a point-in-time doesn't prevent its associated indices from being deleted. You can check how many point-in-times (that is, search contexts) are open with the nodes stats API.

Index

• Variables
• type NewOpenPointInTime
  • func NewOpenPointInTimeFunc(tp elastictransport.Interface) NewOpenPointInTime
• type OpenPointInTime
  • func New(tp elastictransport.Interface) *OpenPointInTime
  • func (r *OpenPointInTime) AllowPartialSearchResults(allowpartialsearchresults bool) *OpenPointInTime
  • func (r OpenPointInTime) Do(providedCtx context.Context) (*Response, error)
  • func (r *OpenPointInTime) ErrorTrace(errortrace bool) *OpenPointInTime
  • func (r *OpenPointInTime) ExpandWildcards(expandwildcards ...expandwildcard.ExpandWildcard) *OpenPointInTime
  • func (r *OpenPointInTime) FilterPath(filterpaths ...string) *OpenPointInTime
  • func (r *OpenPointInTime) Header(key, value string) *OpenPointInTime
  • func (r *OpenPointInTime) HttpRequest(ctx context.Context) (*http.Request, error)
  • func (r *OpenPointInTime) Human(human bool) *OpenPointInTime
  • func (r *OpenPointInTime) IgnoreUnavailable(ignoreunavailable bool) *OpenPointInTime
  • func (r *OpenPointInTime) IndexFilter(indexfilter *types.Query) *OpenPointInTime
  • func (r *OpenPointInTime) KeepAlive(duration string) *OpenPointInTime
  • func (r *OpenPointInTime) MaxConcurrentShardRequests(maxconcurrentshardrequests int) *OpenPointInTime
  • func (r OpenPointInTime) Perform(providedCtx context.Context) (*http.Response, error)
  • func (r *OpenPointInTime) Preference(preference string) *OpenPointInTime
  • func (r *OpenPointInTime) Pretty(pretty bool) *OpenPointInTime
  • func (r *OpenPointInTime) Raw(raw io.Reader) *OpenPointInTime
  • func (r *OpenPointInTime) Request(req *Request) *OpenPointInTime
  • func (r *OpenPointInTime) Routing(routing string) *OpenPointInTime
• type Request
  • func NewRequest() *Request
  • func (r *Request) FromJSON(data string) (*Request, error)
• type Response
  • func NewResponse() *Response

Variables

var ErrBuildPath = errors.New("cannot build path, check for missing path parameters")

ErrBuildPath is returned in case of missing parameters within the build of the request.

Types

type NewOpenPointInTime

type NewOpenPointInTime func(index string) *OpenPointInTime

NewOpenPointInTime type alias for index.

func NewOpenPointInTimeFunc

func NewOpenPointInTimeFunc(tp elastictransport.Interface) NewOpenPointInTime

NewOpenPointInTimeFunc returns a new instance of OpenPointInTime with the provided transport. Used in the index of the library this allows to retrieve every apis in once place.

type OpenPointInTime

type OpenPointInTime struct {
	// contains filtered or unexported fields
}

func New

func New(tp elastictransport.Interface) *OpenPointInTime

Open a point in time.

A search request by default runs against the most recent visible data of the target indices, which is called point in time. Elasticsearch pit (point in time) is a lightweight view into the state of the data as it existed when initiated. In some cases, it’s preferred to perform multiple search requests using the same point in time. For example, if refreshes happen between `search_after` requests, then the results of those requests might not be consistent as changes happening between searches are only visible to the more recent point in time.

A point in time must be opened explicitly before being used in search requests.

A subsequent search request with the `pit` parameter must not specify `index`, `routing`, or `preference` values as these parameters are copied from the point in time.

Just like regular searches, you can use `from` and `size` to page through point in time search results, up to the first 10,000 hits. If you want to retrieve more hits, use PIT with `search_after`.

IMPORTANT: The open point in time request and each subsequent search request can return different identifiers; always use the most recently received ID for the next search request.

When a PIT that contains shard failures is used in a search request, the missing are always reported in the search response as a `NoShardAvailableActionException` exception. To get rid of these exceptions, a new PIT needs to be created so that shards missing from the previous PIT can be handled, assuming they become available in the meantime.

**Keeping point in time alive**

The `keep_alive` parameter, which is passed to a open point in time request and search request, extends the time to live of the corresponding point in time. The value does not need to be long enough to process all data — it just needs to be long enough for the next request.

Normally, the background merge process optimizes the index by merging together smaller segments to create new, bigger segments. Once the smaller segments are no longer needed they are deleted. However, open point-in-times prevent the old segments from being deleted since they are still in use.

TIP: Keeping older segments alive means that more disk space and file handles are needed. Ensure that you have configured your nodes to have ample free file handles.

Additionally, if a segment contains deleted or updated documents then the point in time must keep track of whether each document in the segment was live at the time of the initial search request. Ensure that your nodes have sufficient heap space if you have many open point-in-times on an index that is subject to ongoing deletes or updates. Note that a point-in-time doesn't prevent its associated indices from being deleted. You can check how many point-in-times (that is, search contexts) are open with the nodes stats API.

https://www.elastic.co/guide/en/elasticsearch/reference/current/point-in-time-api.html

func (*OpenPointInTime) AllowPartialSearchResults

func (r *OpenPointInTime) AllowPartialSearchResults(allowpartialsearchresults bool) *OpenPointInTime

AllowPartialSearchResults Indicates whether the point in time tolerates unavailable shards or shard failures when initially creating the PIT. If `false`, creating a point in time request when a shard is missing or unavailable will throw an exception. If `true`, the point in time will contain all the shards that are available at the time of the request. API name: allow_partial_search_results

func (OpenPointInTime) Do

func (r OpenPointInTime) Do(providedCtx context.Context) (*Response, error)

Do runs the request through the transport, handle the response and returns a openpointintime.Response

func (*OpenPointInTime) ErrorTrace

func (r *OpenPointInTime) ErrorTrace(errortrace bool) *OpenPointInTime

ErrorTrace When set to `true` Elasticsearch will include the full stack trace of errors when they occur. API name: error_trace

func (*OpenPointInTime) ExpandWildcards

func (r *OpenPointInTime) ExpandWildcards(expandwildcards ...expandwildcard.ExpandWildcard) *OpenPointInTime

ExpandWildcards The type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. It supports comma-separated values, such as `open,hidden`. API name: expand_wildcards

func (*OpenPointInTime) FilterPath

func (r *OpenPointInTime) FilterPath(filterpaths ...string) *OpenPointInTime

FilterPath Comma-separated list of filters in dot notation which reduce the response returned by Elasticsearch. API name: filter_path

func (*OpenPointInTime) Header

func (r *OpenPointInTime) Header(key, value string) *OpenPointInTime

Header set a key, value pair in the OpenPointInTime headers map.

func (*OpenPointInTime) HttpRequest

func (r *OpenPointInTime) HttpRequest(ctx context.Context) (*http.Request, error)

HttpRequest returns the http.Request object built from the given parameters.

func (*OpenPointInTime) Human

func (r *OpenPointInTime) Human(human bool) *OpenPointInTime

Human When set to `true` will return statistics in a format suitable for humans. For example `"exists_time": "1h"` for humans and `"eixsts_time_in_millis": 3600000` for computers. When disabled the human readable values will be omitted. This makes sense for responses being consumed only by machines. API name: human

func (*OpenPointInTime) IgnoreUnavailable

func (r *OpenPointInTime) IgnoreUnavailable(ignoreunavailable bool) *OpenPointInTime

IgnoreUnavailable If `false`, the request returns an error if it targets a missing or closed index. API name: ignore_unavailable

func (*OpenPointInTime) IndexFilter

func (r *OpenPointInTime) IndexFilter(indexfilter *types.Query) *OpenPointInTime

IndexFilter Filter indices if the provided query rewrites to `match_none` on every shard. API name: index_filter

func (*OpenPointInTime) KeepAlive

func (r *OpenPointInTime) KeepAlive(duration string) *OpenPointInTime

KeepAlive Extend the length of time that the point in time persists. API name: keep_alive

func (*OpenPointInTime) MaxConcurrentShardRequests

func (r *OpenPointInTime) MaxConcurrentShardRequests(maxconcurrentshardrequests int) *OpenPointInTime

MaxConcurrentShardRequests Maximum number of concurrent shard requests that each sub-search request executes per node. API name: max_concurrent_shard_requests

func (OpenPointInTime) Perform

func (r OpenPointInTime) Perform(providedCtx context.Context) (*http.Response, error)

Perform runs the http.Request through the provided transport and returns an http.Response.

func (*OpenPointInTime) Preference

func (r *OpenPointInTime) Preference(preference string) *OpenPointInTime

Preference The node or shard the operation should be performed on. By default, it is random. API name: preference

func (*OpenPointInTime) Pretty

func (r *OpenPointInTime) Pretty(pretty bool) *OpenPointInTime

Pretty If set to `true` the returned JSON will be "pretty-formatted". Only use this option for debugging only. API name: pretty

func (*OpenPointInTime) Raw

func (r *OpenPointInTime) Raw(raw io.Reader) *OpenPointInTime

Raw takes a json payload as input which is then passed to the http.Request If specified Raw takes precedence on Request method.

func (*OpenPointInTime) Request

func (r *OpenPointInTime) Request(req *Request) *OpenPointInTime

Request allows to set the request property with the appropriate payload.

func (*OpenPointInTime) Routing

func (r *OpenPointInTime) Routing(routing string) *OpenPointInTime

Routing A custom value that is used to route operations to a specific shard. API name: routing

type Request

type Request struct {

	// IndexFilter Filter indices if the provided query rewrites to `match_none` on every shard.
	IndexFilter *types.Query `json:"index_filter,omitempty"`
}

Request holds the request body struct for the package openpointintime

https://github.com/elastic/elasticsearch-specification/blob/470b4b9aaaa25cae633ec690e54b725c6fc939c7/specification/_global/open_point_in_time/OpenPointInTimeRequest.ts#L26-L127

func NewRequest

func NewRequest() *Request

NewRequest returns a Request

func (*Request) FromJSON

func (r *Request) FromJSON(data string) (*Request, error)

FromJSON allows to load an arbitrary json into the request structure

type Response

type Response struct {
	Id string `json:"id"`
	// Shards_ Shards used to create the PIT
	Shards_ types.ShardStatistics `json:"_shards"`
}

Response holds the response body struct for the package openpointintime

https://github.com/elastic/elasticsearch-specification/blob/470b4b9aaaa25cae633ec690e54b725c6fc939c7/specification/_global/open_point_in_time/OpenPointInTimeResponse.ts#L23-L29

func NewResponse

func NewResponse() *Response

NewResponse returns a Response