streamcompletion

Documentation

Overview

Perform streaming inference. Get real-time responses for completion tasks by delivering answers incrementally, reducing response times during computation. This API works only with the completion task type.

IMPORTANT: The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.

This API requires the `monitor_inference` cluster privilege (the built-in `inference_admin` and `inference_user` roles grant this privilege). You must use a client that supports streaming.

Index

• Variables
• type NewStreamCompletion
  • func NewStreamCompletionFunc(tp elastictransport.Interface) NewStreamCompletion
• type Request
  • func NewRequest() *Request
  • func (r *Request) FromJSON(data string) (*Request, error)
  • func (s *Request) UnmarshalJSON(data []byte) error
• type Response
  • func NewResponse() Response
• type StreamCompletion
  • func New(tp elastictransport.Interface) *StreamCompletion
  • func (r StreamCompletion) Do(providedCtx context.Context) (Response, error)
  • func (r *StreamCompletion) ErrorTrace(errortrace bool) *StreamCompletion
  • func (r *StreamCompletion) FilterPath(filterpaths ...string) *StreamCompletion
  • func (r *StreamCompletion) Header(key, value string) *StreamCompletion
  • func (r *StreamCompletion) HttpRequest(ctx context.Context) (*http.Request, error)
  • func (r *StreamCompletion) Human(human bool) *StreamCompletion
  • func (r *StreamCompletion) Input(inputs ...string) *StreamCompletion
  • func (r StreamCompletion) Perform(providedCtx context.Context) (*http.Response, error)
  • func (r *StreamCompletion) Pretty(pretty bool) *StreamCompletion
  • func (r *StreamCompletion) Raw(raw io.Reader) *StreamCompletion
  • func (r *StreamCompletion) Request(req *Request) *StreamCompletion
  • func (r *StreamCompletion) TaskSettings(tasksettings json.RawMessage) *StreamCompletion

Variables

var ErrBuildPath = errors.New("cannot build path, check for missing path parameters")

ErrBuildPath is returned in case of missing parameters within the build of the request.

Types

type NewStreamCompletion

type NewStreamCompletion func(inferenceid string) *StreamCompletion

NewStreamCompletion type alias for index.

func NewStreamCompletionFunc

func NewStreamCompletionFunc(tp elastictransport.Interface) NewStreamCompletion

NewStreamCompletionFunc returns a new instance of StreamCompletion with the provided transport. Used in the index of the library this allows to retrieve every apis in once place.

type Request

type Request struct {

	// Input The text on which you want to perform the inference task.
	// It can be a single string or an array.
	//
	// NOTE: Inference endpoints for the completion task type currently only support
	// a single string as input.
	Input []string `json:"input"`
	// TaskSettings Optional task settings
	TaskSettings json.RawMessage `json:"task_settings,omitempty"`
}

Request holds the request body struct for the package streamcompletion

https://github.com/elastic/elasticsearch-specification/blob/3a94b6715915b1e9311724a2614c643368eece90/specification/inference/stream_completion/StreamInferenceRequest.ts#L24-L63

func NewRequest

func NewRequest() *Request

NewRequest returns a Request

func (*Request) FromJSON

func (r *Request) FromJSON(data string) (*Request, error)

FromJSON allows to load an arbitrary json into the request structure

func (*Request) UnmarshalJSON

func (s *Request) UnmarshalJSON(data []byte) error

type Response

type Response = []byte

func NewResponse

func NewResponse() Response

NewResponse returns a Response

type StreamCompletion

type StreamCompletion struct {
	// contains filtered or unexported fields
}

func New

func New(tp elastictransport.Interface) *StreamCompletion

Perform streaming inference. Get real-time responses for completion tasks by delivering answers incrementally, reducing response times during computation. This API works only with the completion task type.

IMPORTANT: The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.

This API requires the `monitor_inference` cluster privilege (the built-in `inference_admin` and `inference_user` roles grant this privilege). You must use a client that supports streaming.

https://www.elastic.co/guide/en/elasticsearch/reference/current/stream-inference-api.html

func (StreamCompletion) Do

func (r StreamCompletion) Do(providedCtx context.Context) (Response, error)

Do runs the request through the transport, handle the response and returns a streamcompletion.Response

func (*StreamCompletion) ErrorTrace

func (r *StreamCompletion) ErrorTrace(errortrace bool) *StreamCompletion

ErrorTrace When set to `true` Elasticsearch will include the full stack trace of errors when they occur. API name: error_trace

func (*StreamCompletion) FilterPath

func (r *StreamCompletion) FilterPath(filterpaths ...string) *StreamCompletion

FilterPath Comma-separated list of filters in dot notation which reduce the response returned by Elasticsearch. API name: filter_path

func (*StreamCompletion) Header

func (r *StreamCompletion) Header(key, value string) *StreamCompletion

Header set a key, value pair in the StreamCompletion headers map.

func (*StreamCompletion) HttpRequest

func (r *StreamCompletion) HttpRequest(ctx context.Context) (*http.Request, error)

HttpRequest returns the http.Request object built from the given parameters.

func (*StreamCompletion) Human

func (r *StreamCompletion) Human(human bool) *StreamCompletion

Human When set to `true` will return statistics in a format suitable for humans. For example `"exists_time": "1h"` for humans and `"eixsts_time_in_millis": 3600000` for computers. When disabled the human readable values will be omitted. This makes sense for responses being consumed only by machines. API name: human

func (*StreamCompletion) Input

func (r *StreamCompletion) Input(inputs ...string) *StreamCompletion

Input The text on which you want to perform the inference task. It can be a single string or an array.

NOTE: Inference endpoints for the completion task type currently only support a single string as input. API name: input

func (StreamCompletion) Perform

func (r StreamCompletion) Perform(providedCtx context.Context) (*http.Response, error)

Perform runs the http.Request through the provided transport and returns an http.Response.

func (*StreamCompletion) Pretty

func (r *StreamCompletion) Pretty(pretty bool) *StreamCompletion

Pretty If set to `true` the returned JSON will be "pretty-formatted". Only use this option for debugging only. API name: pretty

func (*StreamCompletion) Raw

func (r *StreamCompletion) Raw(raw io.Reader) *StreamCompletion

Raw takes a json payload as input which is then passed to the http.Request If specified Raw takes precedence on Request method.

func (*StreamCompletion) Request

func (r *StreamCompletion) Request(req *Request) *StreamCompletion

Request allows to set the request property with the appropriate payload.

func (*StreamCompletion) TaskSettings

func (r *StreamCompletion) TaskSettings(tasksettings json.RawMessage) *StreamCompletion

TaskSettings Optional task settings API name: task_settings