document

Documentation

Overview

Package document defines the canonical serializable Document content value shared by extraction, retrieval, and model-facing components.

NewDocument requires text, media, or both. Metadata is JSON-safe and belongs to the document itself; query-specific relevance belongs to the vector-store match value. Clone provides an independent snapshot when ownership crosses a component boundary. Extraction, formatting, splitting, identifier assignment, batching, and loading policy live in the separate etl module.

Example

package main

import (
	"fmt"

	"github.com/Tangerg/scope/core/document"
)

func main() {
	doc, err := document.NewDocument("Scope are wild cats.", nil)
	if err != nil {
		panic(err)
	}
	doc.ID = "doc-1"
	if setErr := doc.Metadata.Set("source", "field-guide"); setErr != nil {
		panic(setErr)
	}
	source, _, err := doc.Metadata.Decode[string]("source")
	if err != nil {
		panic(err)
	}

	fmt.Println(doc.ID, source)
}

Output:
doc-1 field-guide

Index

• Variables
• type Document
  • func NewDocument(text string, payload *media.Media) (*Document, error)
  • func (d *Document) Clone() *Document
  • func (d *Document) Validate() error

Examples

• Package

Variables

var ErrInvalidDocument = errors.New("document: invalid document")

Types

type Document

type Document struct {
	ID string `json:"id,omitempty"`

	// Text is the textual content. May be empty if Media is set.
	Text string `json:"text,omitempty"`

	Media *media.Media `json:"media,omitempty"`

	Metadata metadata.Map `json:"metadata,omitzero"`
}

Document is the canonical content carrier. It holds identity, content, and metadata; query-specific relationships and runtime policies live outside this value. Clone recursively snapshots media and metadata so indexing and retrieval boundaries never retain caller-owned mutable buffers.

func NewDocument

func NewDocument(text string, payload *media.Media) (*Document, error)

func (*Document) Clone

func (d *Document) Clone() *Document

func (*Document) Validate

func (d *Document) Validate() error