enginecache

Documentation

Overview

Package enginecache binds cachemeta's remote invalidation directives to documented serving-engine control endpoints.

Index

• Constants
• func SupportsExactSpan(engine Engine) bool
• type Client
  • func (c Client) Invalidate(ctx context.Context, dirs []cachemeta.ExternalInvalidationDirective) (Result, error)
• type Engine
• type Result

Constants

const (
	EngineSGLang Engine = "sglang"
	EngineVLLM   Engine = "vllm"

	ScopeWholePrefixCache = "whole_prefix_cache"
	ScopeExactSpan        = "exact_span"
)

Functions

func SupportsExactSpan

func SupportsExactSpan(engine Engine) bool

SupportsExactSpan reports whether the documented public engine control plane can evict exactly the K/V or attention-index span named by cachemeta. Current SGLang and vLLM HTTP endpoints reset the whole prefix/radix cache instead.

Types

type Client

type Client struct {
	Engine        Engine
	BaseURL       string
	AdminAPIKey   string
	IdleTimeout   time.Duration
	RequiredScope string
	HTTPClient    *http.Client
}

Client translates cachemeta invalidation directives into remote engine calls. The current SGLang/vLLM public HTTP surfaces expose whole-prefix-cache reset, not exact-span eviction, so one or more directives becomes one auditable full engine-cache invalidation.

func (Client) Invalidate

func (c Client) Invalidate(ctx context.Context, dirs []cachemeta.ExternalInvalidationDirective) (Result, error)

Invalidate applies remote invalidation directives. It is a no-op when the directive set is empty, and returns an error on unsupported engines or non-2xx control endpoint responses.

type Engine

type Engine string

Engine names a remote serving engine with a documented cache reset endpoint.

type Result

type Result struct {
	Engine             Engine
	Endpoint           string
	Scope              string
	ExactSpanSupported bool
	Directives         int
	StatusCode         int
	BodySummary        string
}

Result is the witnessed effect of an engine-cache invalidation attempt.