pii

Documentation

Overview

Package pii runs Settings → Privacy's local model, which finds what the masking rules can't (internal/redact's Model): a name or an address in free text, a password with nothing around it to say so.

It runs one of two models, the user's pick (Choose, Models): AstrLink Guard (astrlink.go), a 39 MB int8 token classifier for Chinese and English, run in this process by internal/pii/guard, the default for a user who never had the other; or Privacy Filter, below, which masks many more ordinary values in code (go.sum hashes, product names; #3578).

Privacy Filter is OpenAI's (Apache-2.0), 1.6 GB as the q8 GGUF LocalAI made of it, run by pf-serve (native/privacy-filter), a program built from privacy-filter.cpp on ggml. Neither ships with magpie: both are downloaded when the user turns the model on, each checked against the sha256 pinned here, and the setting (settings.RedactModel) is saved only once both are. pf-serve is started on the first request that needs it, on the GPU where there is one (Metal, Vulkan) and else the CPU, and is ended after a while with none, which lets its memory go.

A process of its own, not linked in: 1.7 GB the gateway would otherwise hold for good, and a GPU driver's crash that would end the gateway with it. One that dies is started again; one that dies on the GPU, doesn't load there in gpuLoadTimeout, or hangs there twice in a row, is started on the CPU until the GPU is tried again (gpuRetryAfter), with no restart of magpie. One that keeps failing holds requests back at once, not after a load each, until it is tried again (retryAfter).

Index

• Constants
• Variables
• func Bounded(ctx context.Context) context.Context
• func Cancel()
• func Choose(name string, on bool) string
• func Download(done func()) error
• func Find(texts []string) ([][]redact.Found, error)
• func FindContext(ctx context.Context, texts []string) ([][]redact.Found, error)
• func Installed() bool
• func IsBounded(ctx context.Context) bool
• func Remove() error
• func Size() int64
• func Stop()
• func Supported() bool
• func Threshold(set map[string]float64, m Model) float64
• func Warm()
• type Model
  • func Models() []Model
  • func Of(id string) Model
• type Partial
  • func (p *Partial) Error() string
  • func (p *Partial) Unwrap() error
• type Status
  • func Now() Status

Constants

const (
	PrivacyFilter = "privacy-filter"
	AstrLinkGuard = "astrlink-guard"
)

The local models Settings → Privacy offers, by the id the settings keep (settings.RedactModelName).

const Floor = guard.Floor

Floor is the lowest threshold a user can set (settings.RedactModelScore): what either model sends is at least this sure. pf-serve's default threshold is 0.5, and AstrLink Guard's Find keeps what is (guard.Floor).

Variables

var ErrNotInstalled = errors.New("the local masking model isn't downloaded; download it again in Settings → Privacy, or turn it off")

ErrNotInstalled is what a request gets when the model is on in the settings and isn't on disk (magpie's cache was cleared).

Functions

func Bounded

func Bounded(ctx context.Context) context.Context

Bounded is ctx for a request that doesn't wait for every text it has to be read: masking's default, which waits a short while and lets the rules mask alone after it (internal/masking). Of what a request before it already went without the model's reading (an agent's history, resumed or never read, sent again on every turn), it reads backlogCap at most, after what is new, and answers with nothing found for the rest. Before, each request read such a history oldest first on every core, and a long one kept the CPU busy for as long as requests came while what was new in them waited behind it (xinvxueyuan and shensmart on Discord, 10-28: 13 cores of a 32-core Linux box).

func Cancel

func Cancel()

Cancel stops the download under way, keeping what it fetched.

func Choose

func Choose(name string, on bool) string

Choose is the model masking uses: the one the user picked (name); with none picked, AstrLink Guard, a 38 MB download that runs on every CPU and, measured on code (docs/subsystems), masks far fewer ordinary values. Only while the model is on (on) and Privacy Filter is on disk but AstrLink Guard isn't is it Privacy Filter, so a user who had it on before there was a choice isn't held back while AstrLink Guard downloads (masking.picked starts that download).

func Download

func Download(done func()) error

Download fetches pf-serve and the model, each checked against its sha256, in the background, and calls done once both are on disk. A download already under way is left to go on. What was fetched of the model before (a download stopped, magpie quit) is taken up where it stopped.

func Find

func Find(texts []string) ([][]redact.Found, error)

Find is redact's Model: what the model finds in each of texts. A request's history comes back turn after turn, so what was found in a text is kept (by its hash, on disk too: cache.go) and the model is asked only about texts it hasn't seen. Those go to it joined into chunks of up to chunkSize: one pass over a few joined texts takes about what one over a single short text does (100 short messages: 683 ms one by one, 44 ms joined, on an M4 Pro's GPU). A text longer than pieceSize goes in overlapping pieces (pieces), each kept on its own, so a long text is read a piece at a time and what was read is kept even when the rest isn't (#3682).

func FindContext

func FindContext(ctx context.Context, texts []string) ([][]redact.Found, error)

FindContext is Find for a caller that may stop waiting: when ctx ends (the client went away, or the caller's own bound) it returns at once, whether its turn hadn't come or pf-serve is on it, with what was known of the texts by then and a *Partial. The chunk pf-serve is on is finished all the same and its answers kept, so the request sent again finds them; the chunks after it are left. Between chunks the turn is given up and taken again, so a short request waiting for it goes before the rest of a long one.

func Installed

func Installed() bool

Installed says the model and pf-serve are on disk, as downloaded and checked: the model the size it was hashed at.

func IsBounded

func IsBounded(ctx context.Context) bool

IsBounded says whether ctx is Bounded.

func Remove

func Remove() error

Remove ends pf-serve and deletes the model and pf-serve.

func Size

func Size() int64

Size is how much turning the model on downloads.

func Stop

func Stop()

Stop ends pf-serve, if it runs, and a load under way, at once: it waits for no request's turn. A request on it fails, and pf-serve is killed rather than given the 5s an idle one has to end.

func Supported

func Supported() bool

Supported says pf-serve is built for this system.

func Threshold

func Threshold(set map[string]float64, m Model) float64

Threshold is model m's threshold of those the user set (by model id, settings.RedactModelScore), or its own where none is, or one is out of [Floor, 1).

func Warm

func Warm()

Warm starts pf-serve in the background, where the model is on disk, so that the first request doesn't wait for it to load, and Settings shows where it runs.

Types

type Model

type Model interface {
	ID() string
	Supported() bool
	Installed() bool
	// Size is how much turning it on downloads.
	Size() int64
	// MinScore is its default threshold: masking drops what it is less
	// sure of, unless the user set another.
	MinScore() float64
	Find(ctx context.Context, texts []string) ([][]redact.Found, error)
	Now() Status
	// Download fetches it in the background and calls done once it is on
	// disk; Cancel stops that.
	Download(done func()) error
	Cancel()
	// Warm loads it in the background, where it is on disk.
	Warm()
	// Stop lets it go; Remove stops it and deletes it.
	Stop()
	Remove() error
}

Model is one local model.

func Models

func Models() []Model

Models are the local models, Privacy Filter first.

func Of

func Of(id string) Model

Of is the model id names; AstrLink Guard's for an id there's none of.

type Partial

type Partial struct {
	// Missing of Of texts the model hadn't read
	Missing, Of int
	Err         error
}

Partial is Find's error when its caller stopped waiting (ctx ended) before the model had read every text. What Find returns with it is what was known of each text by then, read before or meanwhile, by this request or another's; a text the model hadn't read has nil, as one it found nothing in has. Before, such a wait gave nothing at all, not even for texts read turns ago, and a caller holding a request back for the model held it back whole (xRayySummers on X: a burst of requests with text the model hadn't read queued for up to 290s, and each that stopped waiting was "masking failed, so nothing was sent").

func (*Partial) Error

func (p *Partial) Error() string

func (*Partial) Unwrap

func (p *Partial) Unwrap() error

type Status

type Status struct {
	Model     string `json:"model"`
	Supported bool   `json:"supported"`
	Installed bool   `json:"installed"`
	Size      int64  `json:"size"`
	// a download under way, or the last one's error
	Downloading bool   `json:"downloading,omitempty"`
	Done        int64  `json:"done,omitempty"`
	Total       int64  `json:"total,omitempty"`
	Error       string `json:"error,omitempty"`
	// pf-serve, while it runs: where the model is, and how long it took
	// to start
	Running bool   `json:"running,omitempty"`
	Device  string `json:"device,omitempty"`
	// Via is the runtime AstrLink Guard runs on, "CoreML" on a Mac where
	// it does (Device then the one its plan puts most of the model on, or
	// ""); "" for magpie's own Go code on the CPU
	Via      string `json:"via,omitempty"`
	GPU      bool   `json:"gpu,omitempty"`
	LoadMS   int    `json:"loadMs,omitempty"`
	GPUError string `json:"gpuError,omitempty"`
	// why the last request got no answer
	Failed string `json:"failed,omitempty"`
	// pf-serve is loading the model, and how many requests wait for it
	// (or for the one it is on)
	Loading bool `json:"loading,omitempty"`
	Waiting int  `json:"waiting,omitempty"`
}

Status is what Settings → Privacy shows of the model.

func Now

func Now() Status

Now is the model's status.