Documentation
¶
Overview ¶
Package enhancer provides query and document enhancement utilities for RAG systems. This file implements context pruning for retrieval result optimization.
Package enhancer provides query and document enhancement utilities for RAG systems. This file implements CrossEncoder-based reranking for retrieval result enhancement.
Package enhancer provides query and document enhancement utilities for RAG systems. It includes components for query rewriting, hypothetical document generation, and step-back prompting to improve retrieval and generation quality.
Package enhancer provides query and document enhancement utilities for RAG systems. It includes components for query rewriting, hypothetical document generation, and step-back prompting to improve retrieval and generation quality.
Package enhancer provides query and document enhancement utilities for RAG systems. This file implements parent document expansion for retrieval result enhancement.
Package enhancer provides query and document enhancement utilities for RAG systems. This file implements sentence window expansion for context enhancement.
Index ¶
- type CRAGEvaluator
- type ContextPruner
- type ContextPrunerOption
- type CrossEncoderReranker
- type CrossEncoderRerankerOption
- type DocumentStore
- type FilterExtractor
- type HyDEGenerator
- type ParentDocExpander
- type ParentDocExpanderOption
- type QueryRewriter
- type RelevanceScore
- type SentenceWindowExpander
- type SentenceWindowExpanderOption
- type StepBackGenerator
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type CRAGEvaluator ¶
type CRAGEvaluator struct {
// contains filtered or unexported fields
}
CRAGEvaluator evaluates retrieval quality and decides corrective actions. It assesses whether retrieved documents are relevant and triggers web search if needed.
func NewCRAGEvaluator ¶
func NewCRAGEvaluator(llm chat.Client) *CRAGEvaluator
NewCRAGEvaluator creates a new CRAG evaluator.
Parameters: - llm: The chat client to use for evaluation
Returns: - A new CRAGEvaluator instance
func (*CRAGEvaluator) Evaluate ¶
func (c *CRAGEvaluator) Evaluate(ctx context.Context, query *entity.Query, chunks []*entity.Chunk) (*retrieval.CRAGEvaluation, error)
Evaluate evaluates the quality of retrieved chunks and returns assessment.
Parameters: - ctx: The context for the operation - query: The query used for retrieval - chunks: The retrieved chunks to evaluate
Returns: - The evaluation result with relevance score and action recommendation - An error if evaluation fails
type ContextPruner ¶
type ContextPruner struct {
// contains filtered or unexported fields
}
ContextPruner prunes irrelevant context to optimize token usage. It evaluates each chunk's relevance and keeps only the most relevant ones.
func NewContextPruner ¶
func NewContextPruner(llm core.Client, opts ...ContextPrunerOption) *ContextPruner
NewContextPruner creates a new context pruner.
Required: llm. Optional (via options): WithPrunerMaxTokens (default: 2000), WithPrunerLogger, WithPrunerCollector.
func (*ContextPruner) Enhance ¶
func (p *ContextPruner) Enhance(ctx context.Context, results *entity.RetrievalResult) (*entity.RetrievalResult, error)
Enhance prunes the retrieval results to keep only the most relevant chunks.
Parameters: - ctx: The context for cancellation and timeouts - results: The retrieval results to prune
Returns: - The pruned retrieval results with only relevant chunks - An error if pruning fails
type ContextPrunerOption ¶
type ContextPrunerOption func(*ContextPruner)
ContextPrunerOption configures a ContextPruner instance.
func WithPrunerCollector ¶
func WithPrunerCollector(collector observability.Collector) ContextPrunerOption
WithPrunerCollector sets an observability collector.
func WithPrunerLogger ¶
func WithPrunerLogger(logger logging.Logger) ContextPrunerOption
WithPrunerLogger sets a structured logger.
func WithPrunerMaxTokens ¶
func WithPrunerMaxTokens(maxTokens int) ContextPrunerOption
WithPrunerMaxTokens sets the maximum number of tokens to keep.
type CrossEncoderReranker ¶
type CrossEncoderReranker struct {
// contains filtered or unexported fields
}
CrossEncoderReranker uses a CrossEncoder model to rerank retrieval results. It scores each (query, chunk) pair and reorders chunks by relevance.
func NewCrossEncoderReranker ¶
func NewCrossEncoderReranker(llm core.Client, opts ...CrossEncoderRerankerOption) *CrossEncoderReranker
NewCrossEncoderReranker creates a new cross-encoder reranker.
Required: llm. Optional (via options): WithRerankTopK (default: 10), WithRerankLogger, WithRerankCollector.
func (*CrossEncoderReranker) Enhance ¶
func (r *CrossEncoderReranker) Enhance(ctx context.Context, results *entity.RetrievalResult) (*entity.RetrievalResult, error)
Enhance reranks the retrieval results using CrossEncoder scoring.
Parameters: - ctx: The context for cancellation and timeouts - results: The retrieval results to rerank
Returns: - The reranked retrieval results with chunks reordered by relevance - An error if reranking fails
type CrossEncoderRerankerOption ¶
type CrossEncoderRerankerOption func(*CrossEncoderReranker)
CrossEncoderRerankerOption configures a CrossEncoderReranker instance.
func WithRerankCollector ¶
func WithRerankCollector(collector observability.Collector) CrossEncoderRerankerOption
WithRerankCollector sets an observability collector.
func WithRerankLogger ¶
func WithRerankLogger(logger logging.Logger) CrossEncoderRerankerOption
WithRerankLogger sets a structured logger.
func WithRerankTopK ¶
func WithRerankTopK(k int) CrossEncoderRerankerOption
WithRerankTopK sets the number of top results to return after reranking.
type DocumentStore ¶
DocumentStore is an interface for retrieving parent documents.
type FilterExtractor ¶
type FilterExtractor struct {
// contains filtered or unexported fields
}
FilterExtractor uses an LLM to parse natural language constraints into key-value filters for precise Vector Database pre-filtering. It helps improve retrieval precision by extracting explicit filtering conditions from user queries.
func NewFilterExtractor ¶
func NewFilterExtractor(llm core.Client) *FilterExtractor
NewFilterExtractor creates a new filter extractor.
Parameters: - llm: The LLM client to use for extraction
Returns: - A new FilterExtractor instance
func (*FilterExtractor) ExtractFilters ¶
func (f *FilterExtractor) ExtractFilters(ctx context.Context, query *entity.Query) (map[string]any, error)
ExtractFilters extracts key-value filters from the user's query.
Parameters: - ctx: The context for the operation - query: The query to extract filters from
Returns: - A map of key-value filters - An error if extraction fails
type HyDEGenerator ¶
type HyDEGenerator struct {
// contains filtered or unexported fields
}
HyDEGenerator generates hypothetical answers to improve dense retrieval. It uses an LLM to create hypothetical documents that are then embedded alongside the original query to improve search results.
func NewHyDEGenerator ¶
func NewHyDEGenerator(llm chat.Client) *HyDEGenerator
NewHyDEGenerator creates a new hypothetical document generator.
Parameters: - llm: The chat client to use for generation
Returns: - A new HyDEGenerator instance
func (*HyDEGenerator) GenerateHypotheticalDocument ¶
func (h *HyDEGenerator) GenerateHypotheticalDocument(ctx context.Context, query *entity.Query) (*entity.Document, error)
GenerateHypotheticalDocument generates a hypothetical document based on the query.
Parameters: - ctx: The context for the operation - query: The query to generate a hypothetical document for
Returns: - The generated hypothetical document - An error if generation fails
type ParentDocExpander ¶
type ParentDocExpander struct {
// contains filtered or unexported fields
}
ParentDocExpander expands retrieved chunks to their full parent documents. It helps balance fine-grained retrieval with complete context understanding.
func NewParentDocExpander ¶
func NewParentDocExpander(docStore DocumentStore, opts ...ParentDocExpanderOption) *ParentDocExpander
NewParentDocExpander creates a new parent document expander.
Required: docStore. Optional (via options): WithParentDocLogger, WithParentDocCollector.
func (*ParentDocExpander) Enhance ¶
func (e *ParentDocExpander) Enhance(ctx context.Context, results *entity.RetrievalResult) (*entity.RetrievalResult, error)
Enhance expands retrieved chunks to their full parent documents.
Parameters: - ctx: The context for cancellation and timeouts - results: The retrieval results to expand
Returns: - The expanded retrieval results with full parent documents - An error if expansion fails
type ParentDocExpanderOption ¶
type ParentDocExpanderOption func(*ParentDocExpander)
ParentDocExpanderOption configures a ParentDocExpander instance.
func WithParentDocCollector ¶
func WithParentDocCollector(collector observability.Collector) ParentDocExpanderOption
WithParentDocCollector sets an observability collector.
func WithParentDocLogger ¶
func WithParentDocLogger(logger logging.Logger) ParentDocExpanderOption
WithParentDocLogger sets a structured logger.
type QueryRewriter ¶
type QueryRewriter struct {
// contains filtered or unexported fields
}
QueryRewriter uses an LLM to rewrite and expand the user's query. It improves query clarity and specificity for better vector database search.
func NewQueryRewriter ¶
func NewQueryRewriter(llm chat.Client) *QueryRewriter
NewQueryRewriter creates a new query rewriter.
Parameters: - llm: The chat client to use for rewriting
Returns: - A new QueryRewriter instance
type RelevanceScore ¶
type RelevanceScore struct {
ChunkIndex int `json:"chunk_index"`
Relevance float32 `json:"relevance"`
Reason string `json:"reason"`
}
RelevanceScore represents the relevance score of a chunk.
type SentenceWindowExpander ¶
type SentenceWindowExpander struct {
// contains filtered or unexported fields
}
SentenceWindowExpander expands retrieved chunks with surrounding sentences. It provides more complete context while maintaining retrieval precision.
func NewSentenceWindowExpander ¶
func NewSentenceWindowExpander(opts ...SentenceWindowExpanderOption) *SentenceWindowExpander
NewSentenceWindowExpander creates a new sentence window expander.
Required: none. Optional (via options): WithWindowSize (default: 2), WithMaxChars (default: 2000), WithExpanderLogger, WithExpanderCollector.
func (*SentenceWindowExpander) Enhance ¶
func (e *SentenceWindowExpander) Enhance(ctx context.Context, results *entity.RetrievalResult) (*entity.RetrievalResult, error)
Enhance expands chunks with surrounding sentences from the original document.
Parameters: - ctx: The context for cancellation and timeouts - results: The retrieval results to expand
Returns: - The expanded retrieval results with sentence windows - An error if expansion fails
type SentenceWindowExpanderOption ¶
type SentenceWindowExpanderOption func(*SentenceWindowExpander)
SentenceWindowExpanderOption configures a SentenceWindowExpander instance.
func WithExpanderCollector ¶
func WithExpanderCollector(collector observability.Collector) SentenceWindowExpanderOption
WithExpanderCollector sets an observability collector.
func WithExpanderLogger ¶
func WithExpanderLogger(logger logging.Logger) SentenceWindowExpanderOption
WithExpanderLogger sets a structured logger.
func WithMaxChars ¶
func WithMaxChars(maxChars int) SentenceWindowExpanderOption
WithMaxChars sets the maximum characters per expanded chunk.
func WithWindowSize ¶
func WithWindowSize(size int) SentenceWindowExpanderOption
WithWindowSize sets the number of sentences to expand (before and after).
type StepBackGenerator ¶
type StepBackGenerator struct {
// contains filtered or unexported fields
}
StepBackGenerator abstracts queries into higher-level background questions. It helps retrieve broader context by generating more general questions that capture the underlying principles behind the original query.
func NewStepBackGenerator ¶
func NewStepBackGenerator(llm chat.Client) *StepBackGenerator
NewStepBackGenerator creates a new step-back query generator.
Parameters: - llm: The chat client to use for generation
Returns: - A new StepBackGenerator instance
func (*StepBackGenerator) GenerateStepBackQuery ¶
func (s *StepBackGenerator) GenerateStepBackQuery(ctx context.Context, query *entity.Query) (*entity.Query, error)
GenerateStepBackQuery generates a step-back query that captures the underlying principles behind the original query.
Parameters: - ctx: The context for the operation - query: The original query
Returns: - The generated step-back query - An error if generation fails
Source Files