README
¶
analytics
Query analytics: record what users search for, how well it went, and where results drop off — the signal for tuning retrieval.
Components
QueryRecord— one search event (query, result IDs, scores, timing, outcome signals).QueryLog(NewQueryLog(maxLen)) — bounded in-memory query log withQueryCountaggregation andDropOffQueryanalysis (queries that returned no/weak results).Sinkinterface + implementations:FileSink(NewFileSink(path)) — append NDJSON to a file.HTTPSink(NewHTTPSink(url, client)) — POST records to an endpoint.
InstrumentedAnalyticsStore(NewInstrumentedAnalyticsStore(store, log)) — wraps astore.Storeso every search is automatically recorded.
Documentation
¶
Overview ¶
Package analytics provides query analytics: a bounded, thread-safe QueryLog that records each query's latency and results, plus helpers for trending (popular) queries and drop-off detection (queries that return no good results). Records can be exported to pluggable sinks (file, HTTP, or a custom message-queue sink).
The package is stdlib-only and dependency-free.
Index ¶
- type DropOffQuery
- type FileSink
- type HTTPSink
- type InstrumentedAnalyticsStore
- func (w *InstrumentedAnalyticsStore) Close() error
- func (w *InstrumentedAnalyticsStore) Count() int
- func (w *InstrumentedAnalyticsStore) DeleteChunk(ctx context.Context, id string) error
- func (w *InstrumentedAnalyticsStore) DeleteDocument(ctx context.Context, docID string) error
- func (w *InstrumentedAnalyticsStore) GetChunk(id string) (*core.Chunk, bool)
- func (w *InstrumentedAnalyticsStore) Inner() store.Store
- func (w *InstrumentedAnalyticsStore) Log() *QueryLog
- func (w *InstrumentedAnalyticsStore) Namespaces() []string
- func (w *InstrumentedAnalyticsStore) Search(ctx context.Context, query string, opts index.SearchOptions) ([]index.SearchResult, error)
- func (w *InstrumentedAnalyticsStore) SearchHybrid(ctx context.Context, query string, opts index.SearchOptions) ([]index.SearchResult, error)
- func (w *InstrumentedAnalyticsStore) Upload(ctx context.Context, doc *core.Document, content string) error
- type QueryCount
- type QueryLog
- func (l *QueryLog) Count() int
- func (l *QueryLog) DropOff(threshold float64, limit int) []DropOffQuery
- func (l *QueryLog) LastError() error
- func (l *QueryLog) LogQuery(query string, latency time.Duration, results int, topScore float64, err error)
- func (l *QueryLog) PopularQueries(limit int) []QueryCount
- func (l *QueryLog) Record(rec QueryRecord)
- func (l *QueryLog) Records() []QueryRecord
- func (l *QueryLog) Reset()
- func (l *QueryLog) Since(t time.Time) []QueryRecord
- func (l *QueryLog) WithSink(s Sink) *QueryLog
- type QueryRecord
- type Sink
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type DropOffQuery ¶
type DropOffQuery struct {
// Query is the normalized query text.
Query string
// Count is how many times the query dropped off.
Count int
// AvgTopScore is the mean top score across its drop-offs.
AvgTopScore float64
// LastSeen is the most recent time the query dropped off.
LastSeen time.Time
}
DropOffQuery aggregates a query that repeatedly failed to return good results.
type FileSink ¶
type FileSink struct {
// contains filtered or unexported fields
}
FileSink appends query records to a file as NDJSON (one JSON object per line).
func NewFileSink ¶
NewFileSink opens (creating if needed) the file at path for appending and returns a FileSink writing NDJSON to it.
func (*FileSink) Write ¶
func (s *FileSink) Write(rec QueryRecord) error
Write appends the record as a single JSON line.
type HTTPSink ¶
type HTTPSink struct {
// contains filtered or unexported fields
}
HTTPSink POSTs each query record as a JSON document to a fixed URL. It is suitable for forwarding analytics to a collector or message-queue HTTP bridge.
func NewHTTPSink ¶
NewHTTPSink returns an HTTPSink that POSTs records to url. If client is nil, a client with a 2-second timeout is used so the query path is never blocked for long.
func (*HTTPSink) Write ¶
func (s *HTTPSink) Write(rec QueryRecord) error
Write POSTs the record as JSON to the sink's URL.
type InstrumentedAnalyticsStore ¶
type InstrumentedAnalyticsStore struct {
// contains filtered or unexported fields
}
InstrumentedAnalyticsStore wraps a store.Store and records a QueryRecord for each search. It is a drop-in replacement for store.Store.
func NewInstrumentedAnalyticsStore ¶
func NewInstrumentedAnalyticsStore(s store.Store, log *QueryLog) *InstrumentedAnalyticsStore
NewInstrumentedAnalyticsStore wraps s, recording query analytics to log. If log is nil, no analytics are recorded (the wrapper still delegates).
func (*InstrumentedAnalyticsStore) Close ¶
func (w *InstrumentedAnalyticsStore) Close() error
Close implements store.Store.
func (*InstrumentedAnalyticsStore) Count ¶
func (w *InstrumentedAnalyticsStore) Count() int
Count implements store.Store.
func (*InstrumentedAnalyticsStore) DeleteChunk ¶
func (w *InstrumentedAnalyticsStore) DeleteChunk(ctx context.Context, id string) error
DeleteChunk implements store.Store.
func (*InstrumentedAnalyticsStore) DeleteDocument ¶
func (w *InstrumentedAnalyticsStore) DeleteDocument(ctx context.Context, docID string) error
DeleteDocument implements store.Store.
func (*InstrumentedAnalyticsStore) GetChunk ¶
func (w *InstrumentedAnalyticsStore) GetChunk(id string) (*core.Chunk, bool)
GetChunk implements store.Store.
func (*InstrumentedAnalyticsStore) Inner ¶
func (w *InstrumentedAnalyticsStore) Inner() store.Store
Inner exposes the wrapped store.
func (*InstrumentedAnalyticsStore) Log ¶
func (w *InstrumentedAnalyticsStore) Log() *QueryLog
Log returns the QueryLog being recorded to (may be nil).
func (*InstrumentedAnalyticsStore) Namespaces ¶
func (w *InstrumentedAnalyticsStore) Namespaces() []string
Namespaces implements store.Store.
func (*InstrumentedAnalyticsStore) Search ¶
func (w *InstrumentedAnalyticsStore) Search(ctx context.Context, query string, opts index.SearchOptions) ([]index.SearchResult, error)
Search implements store.Store and records a query record.
func (*InstrumentedAnalyticsStore) SearchHybrid ¶
func (w *InstrumentedAnalyticsStore) SearchHybrid(ctx context.Context, query string, opts index.SearchOptions) ([]index.SearchResult, error)
SearchHybrid implements store.Store and records a query record.
type QueryCount ¶
type QueryCount struct {
// Query is the normalized query text.
Query string
// Count is how many times the query was recorded.
Count int
// AvgLatency is the mean latency across its occurrences.
AvgLatency time.Duration
// LastSeen is the most recent time the query was recorded.
LastSeen time.Time
}
QueryCount aggregates a query's occurrences and performance.
type QueryLog ¶
type QueryLog struct {
// contains filtered or unexported fields
}
QueryLog is a bounded, thread-safe log of query records. When the log is full, the oldest records are overwritten (ring buffer).
func NewQueryLog ¶
NewQueryLog returns a QueryLog retaining at most maxLen records. A non-positive maxLen defaults to 1024.
func (*QueryLog) DropOff ¶
func (l *QueryLog) DropOff(threshold float64, limit int) []DropOffQuery
DropOff returns queries that did not return good results, most frequent first. A query is a drop-off when it errored, returned no results, or had a top score below threshold. If limit is positive, at most limit entries are returned.
func (*QueryLog) LogQuery ¶
func (l *QueryLog) LogQuery(query string, latency time.Duration, results int, topScore float64, err error)
LogQuery is a convenience for recording a query with its outcome.
func (*QueryLog) PopularQueries ¶
func (l *QueryLog) PopularQueries(limit int) []QueryCount
PopularQueries returns the most frequently recorded queries, most frequent first. Queries are grouped after normalizing case and surrounding whitespace. If limit is positive, at most limit entries are returned.
func (*QueryLog) Record ¶
func (l *QueryLog) Record(rec QueryRecord)
Record stores a query record and forwards it to the configured sink. A zero Time or empty ID is filled in automatically. Sink failures are non-fatal; they are captured and retrievable via LastError.
func (*QueryLog) Records ¶
func (l *QueryLog) Records() []QueryRecord
Records returns a copy of the stored records in oldest-to-newest order.
type QueryRecord ¶
type QueryRecord struct {
// ID uniquely identifies the record.
ID string
// Time is when the query was recorded.
Time time.Time
// Query is the raw query text.
Query string
// Latency is how long the query took.
Latency time.Duration
// Results is the number of results returned.
Results int
// TopScore is the best result score (0 when there were no results).
TopScore float64
// Error is the error message if the query failed (empty on success).
Error string
// Namespace optionally scopes the query.
Namespace string
// Metadata holds optional extra tags.
Metadata map[string]string
}
QueryRecord is a single logged query.
type Sink ¶
type Sink interface {
// Write receives a single query record.
Write(rec QueryRecord) error
// Close releases any resources held by the sink.
Close() error
}
Sink receives each recorded query. Implement this to export analytics to a file, an HTTP endpoint, a message queue, etc. Write must be safe for concurrent use and must not block indefinitely.
Source Files