pii

Documentation

Overview

Package pii runs Settings → Privacy's local model, which finds what the masking rules can't (internal/redact's Model): a name or an address in free text, a password with nothing around it to say so.

The model is OpenAI's Privacy Filter (Apache-2.0), 1.6 GB as the q8 GGUF LocalAI made of it, run by pf-serve (native/privacy-filter), a program built from privacy-filter.cpp on ggml. Neither ships with magpie: both are downloaded when the user turns the model on, each checked against the sha256 pinned here, and the setting (settings.RedactModel) is saved only once both are. pf-serve is started on the first request that needs it, on the GPU where there is one (Metal, Vulkan) and else the CPU, and is ended after a while with none, which lets its memory go.

A process of its own, not linked in: 1.7 GB the gateway would otherwise hold for good, and a GPU driver's crash that would end the gateway with it. One that dies is started again; one that dies on the GPU is started on the CPU for the rest of magpie's run.

Index

• Variables
• func Cancel()
• func Download(done func()) error
• func Find(texts []string) ([][]redact.Found, error)
• func Installed() bool
• func Remove() error
• func Size() int64
• func Stop()
• func Supported() bool
• func Warm()
• type Status
  • func Now() Status

Variables

var ErrNotInstalled = errors.New("the local masking model isn't downloaded; download it again in Settings → Privacy, or turn it off")

ErrNotInstalled is what a request gets when the model is on in the settings and isn't on disk (magpie's cache was cleared).

Functions

func Cancel

func Cancel()

Cancel stops the download under way, keeping what it fetched.

func Download

func Download(done func()) error

Download fetches pf-serve and the model, each checked against its sha256, in the background, and calls done once both are on disk. A download already under way is left to go on. What was fetched of the model before (a download stopped, magpie quit) is taken up where it stopped.

func Find

func Find(texts []string) ([][]redact.Found, error)

Find is redact's Model: what the model finds in each of texts. A request's history comes back turn after turn, so what was found in a text is kept (by its hash) and the model is asked only about texts it hasn't seen. Those go to it joined into chunks of up to chunkSize: one pass over a few joined texts takes about what one over a single short text does (100 short messages: 683 ms one by one, 44 ms joined, on an M4 Pro's GPU).

func Installed

func Installed() bool

Installed says the model and pf-serve are on disk, as downloaded and checked: the model the size it was hashed at.

func Remove

func Remove() error

Remove ends pf-serve and deletes the model and pf-serve.

func Size

func Size() int64

Size is how much turning the model on downloads.

func Stop

func Stop()

Stop ends pf-serve, if it runs.

func Supported

func Supported() bool

Supported says pf-serve is built for this system.

func Warm

func Warm()

Warm starts pf-serve in the background, where the model is on disk, so that the first request doesn't wait for it to load, and Settings shows where it runs.

Types

type Status

type Status struct {
	Supported bool  `json:"supported"`
	Installed bool  `json:"installed"`
	Size      int64 `json:"size"`
	// a download under way, or the last one's error
	Downloading bool   `json:"downloading,omitempty"`
	Done        int64  `json:"done,omitempty"`
	Total       int64  `json:"total,omitempty"`
	Error       string `json:"error,omitempty"`
	// pf-serve, while it runs: where the model is, and how long it took
	// to start
	Running  bool   `json:"running,omitempty"`
	Device   string `json:"device,omitempty"`
	GPU      bool   `json:"gpu,omitempty"`
	LoadMS   int    `json:"loadMs,omitempty"`
	GPUError string `json:"gpuError,omitempty"`
	// why the last request got no answer
	Failed string `json:"failed,omitempty"`
}

Status is what Settings → Privacy shows of the model.

func Now

func Now() Status

Now is the model's status.