Documentation
¶
Index ¶
Constants ¶
const MempalaceDirEnv = "MEMPALACE_PY_DIR"
MempalaceDirEnv is the environment variable that points at the Python reference implementation checkout. It is consulted by NewPythonSubprocessEmbedder when PythonSubprocessOptions.MempalaceDir is empty. There is intentionally no hardcoded fallback — shipping a developer-specific absolute path in source is a portability and information-disclosure hazard.
Variables ¶
var ErrClosed = errors.New("embed: python subprocess closed")
ErrClosed indicates the embedder's subprocess has been shut down.
var ErrMempalaceDirUnset = errors.New("embed: MEMPALACE_PY_DIR unset and no MempalaceDir override provided")
ErrMempalaceDirUnset is returned when neither PythonSubprocessOptions.MempalaceDir nor the MEMPALACE_PY_DIR env var was supplied. Callers should treat this as a signal to fall back to FakeEmbedder.
var ErrProbeFailed = errors.New("embed: python subprocess probe failed")
ErrProbeFailed indicates the subprocess started but failed its readiness handshake (e.g. Python exception, wrong script, missing chromadb).
Functions ¶
func VerifyModel ¶ added in v0.2.0
VerifyModel checks the SHA-256 hash of the ONNX model file at modelPath. Returns nil if the hash matches expectedModelSHA256, or if no expected hash is configured (logs a warning in that case).
Types ¶
type FakeEmbedder ¶
type FakeEmbedder struct {
// contains filtered or unexported fields
}
func NewFakeEmbedder ¶
func NewFakeEmbedder(dim int) *FakeEmbedder
func (*FakeEmbedder) Dimension ¶
func (f *FakeEmbedder) Dimension() int
type HugotEmbedder ¶
type HugotEmbedder struct {
// contains filtered or unexported fields
}
HugotEmbedder is a pure-Go embedder backed by knights-analytics/hugot. It uses the GoMLX backend (no CGO beyond sqlite-vec) and downloads the model on first use.
func NewHugotEmbedder ¶
func NewHugotEmbedder(opts HugotOptions) (*HugotEmbedder, error)
NewHugotEmbedder creates a HugotEmbedder. It initializes a GoSession, resolves/downloads the model, and creates a FeatureExtractionPipeline.
func (*HugotEmbedder) Close ¶
func (h *HugotEmbedder) Close() error
Close destroys the hugot session and frees resources.
func (*HugotEmbedder) Dimension ¶
func (h *HugotEmbedder) Dimension() int
type HugotOptions ¶
type HugotOptions struct {
// ModelName is the HuggingFace model identifier.
// Defaults to "sentence-transformers/all-MiniLM-L6-v2".
ModelName string
// ModelDir is the parent directory where models are stored.
// Defaults to ~/.mempalace/models/.
ModelDir string
// ModelPath overrides auto-download: use this local model directory directly.
ModelPath string
}
HugotOptions configures the HugotEmbedder.
type PythonSubprocessEmbedder
deprecated
type PythonSubprocessEmbedder struct {
// contains filtered or unexported fields
}
Deprecated: Use HugotEmbedder for offline pure-Go embeddings. PythonSubprocessEmbedder is retained for users who need exact Python parity.
PythonSubprocessEmbedder is an Embedder that delegates to a long-lived Python helper process over newline-delimited JSON. It is safe for use from multiple goroutines: all pipe access is serialized with a mutex.
func NewPythonSubprocessEmbedder ¶
func NewPythonSubprocessEmbedder(opts PythonSubprocessOptions) (*PythonSubprocessEmbedder, error)
NewPythonSubprocessEmbedder spawns the helper and runs the readiness probe. On any failure it kills the subprocess and returns a non-nil error — the caller is expected to fall back to FakeEmbedder with a loud log line.
func (*PythonSubprocessEmbedder) Close ¶
func (e *PythonSubprocessEmbedder) Close() error
Close shuts the helper down. It closes stdin so the Python loop exits, then waits up to 5 seconds before force-killing. Safe to call more than once.
Close returns an error only for unexpected termination (signal, non-zero exit other than the benign EOF-exit, or wait failures). A clean exit (code 0) after stdin close is the expected path.
func (*PythonSubprocessEmbedder) Dimension ¶
func (e *PythonSubprocessEmbedder) Dimension() int
Dimension returns the vector dimension advertised by the helper at probe time. For ONNXMiniLM_L6_V2 this is 384.
type PythonSubprocessOptions ¶
type PythonSubprocessOptions struct {
// UVBinary is the path to the uv launcher. Defaults to "uv".
UVBinary string
// MempalaceDir is the working directory of the Python oracle.
// Defaults to DefaultMempalaceDir.
MempalaceDir string
// PythonArgs, when non-empty, fully overrides the default command —
// the first element is the executable and the rest are its args.
// Intended for unit tests driving testdata/fake_embedder.py.
PythonArgs []string
// ProbeTimeout bounds the initial readiness handshake. Defaults to 60s.
ProbeTimeout time.Duration
}
PythonSubprocessOptions controls how NewPythonSubprocessEmbedder spawns the helper process. Zero values fall back to production defaults. Tests should set PythonArgs to swap in a fake script.
Source Files