daemon

Documentation

Overview

Package daemon implements the csx local daemon (plan P4.1/P4.4): a localhost-only HTTP API plus a same-mux named-pipe/unix-socket listener, a single-instance lock, background maintenance tickers (evidence upload, shard warm, cache budget, peer announce, idle verification), and the §12.5 dashboard at /ui. Everything it serves is local; the only outbound traffic is the evidence/shard/server plumbing the user's mode allows, and all of it is best-effort offline-tolerant (goal.md §3.9).

Index

• Variables
• func BaseURLFor(home string) (string, error)
• func SocketPath(home string) string
• func StopRunning(ctx context.Context, home string) (alreadyRunning bool, err error)
• type AdoptionRequest
• type Client
  • func EnsureRunning(ctx context.Context, home string, expectedVersion ...string) (*Client, error)
  • func NewClient(home string) (*Client, error)
  • func NewIPCClient(home string) *Client
  • func (c *Client) Adopt(ctx context.Context, req AdoptionRequest) error
  • func (c *Client) Queue(ctx context.Context) (QueuePreview, error)
  • func (c *Client) Sample(ctx context.Context, id string) (SampleInfo, error)
  • func (c *Client) Search(ctx context.Context, req domain.SearchRequest) (LocalSearchResponse, error)
  • func (c *Client) Shutdown(ctx context.Context) error
  • func (c *Client) Stats(ctx context.Context) (Stats, error)
  • func (c *Client) Status(ctx context.Context) (StatusInfo, error)
  • func (c *Client) Sync(ctx context.Context) (SyncResult, error)
• type Daemon
  • func New(home string) (*Daemon, error)
  • func (d *Daemon) BaseURL() string
  • func (d *Daemon) Close() error
  • func (d *Daemon) Handler() http.Handler
  • func (d *Daemon) Ready() <-chan struct{}
  • func (d *Daemon) Run(ctx context.Context) error
  • func (d *Daemon) SearchAndRecord(ctx context.Context, req domain.SearchRequest) LocalSearchResponse
  • func (d *Daemon) StatsNow(ctx context.Context) (Stats, error)
  • func (d *Daemon) SyncNow(ctx context.Context) SyncResult
• type LocalSearchResponse
• type QueuePreview
• type QueuedUpload
• type Readiness
• type SampleInfo
• type Stats
• type StatusInfo
• type SyncResult

Variables

var ErrAlreadyRunning = errors.New("daemon: already running")

ErrAlreadyRunning is returned by Run when another live daemon holds the single-instance lock for this home.

var Version = "dev"

Version is the build stamp shown in /local/v1/status and the UI. The CLI sets it from its own link-time version before starting a daemon.

Functions

func BaseURLFor

func BaseURLFor(home string) (string, error)

BaseURLFor resolves the daemon's TCP base URL for a home: the live daemon.addr file when a daemon is (or recently was) running — which also covers ephemeral ports — falling back to the configured port.

func SocketPath

func SocketPath(home string) string

SocketPath is the unix-domain socket the daemon serves next to its TCP listener: $home/daemon.sock.

func StopRunning

func StopRunning(ctx context.Context, home string) (alreadyRunning bool, err error)

StopRunning gracefully stops the daemon for home and waits until both its listener and single-instance lock are gone. alreadyRunning is false when no daemon answered; that is a successful no-op for init/config transitions.

Types

type AdoptionRequest

type AdoptionRequest struct {
	OfferID   string `json:"offerId"`
	SampleID  string `json:"sampleId"`
	Applied   bool   `json:"applied"`
	BuildPass *bool  `json:"buildPass,omitempty"`
}

AdoptionRequest is the POST /local/v1/adoption body (MCP report_sample_adoption lands here too).

type Client

type Client struct {
	BaseURL string
	HTTP    *http.Client
}

Client talks to a running daemon's local API, either over TCP (BaseURL) or over the platform IPC transport (NewIPCClient).

func EnsureRunning

func EnsureRunning(ctx context.Context, home string, expectedVersion ...string) (*Client, error)

EnsureRunning returns a client for the daemon serving home, spawning "csx daemon run" detached (Windows: new process group + detached; Unix: setsid) when none answers. If expectedVersion is supplied, a live daemon from a different build is stopped gracefully first. That matters on upgrade: replacing the executable does not replace the already-running process, and accepting it here can leave new queue/protocol behavior dormant until the machine reboots.

The spawned daemon inherits home via CSX_HOME and logs to $home/logs/daemon.log.

func NewClient

func NewClient(home string) (*Client, error)

NewClient builds a client for the daemon serving home, and only that home.

The published TCP address when there is one, and this home's own IPC socket otherwise. Both are addresses this home owns; there is deliberately no third case, because the third case was a shared port that belonged to somebody else.

func NewIPCClient

func NewIPCClient(home string) *Client

NewIPCClient builds a client over the Windows named pipe / unix socket for home. The host in BaseURL is cosmetic; the transport dials the pipe.

func (*Client) Adopt

func (c *Client) Adopt(ctx context.Context, req AdoptionRequest) error

Adopt reports POST /local/v1/adoption.

func (*Client) Queue

func (c *Client) Queue(ctx context.Context) (QueuePreview, error)

Queue fetches the privacy preview, GET /local/v1/queue.

func (*Client) Sample

func (c *Client) Sample(ctx context.Context, id string) (SampleInfo, error)

Sample fetches GET /local/v1/samples/{id}.

func (*Client) Search

func (c *Client) Search(ctx context.Context, req domain.SearchRequest) (LocalSearchResponse, error)

Search runs POST /local/v1/search.

func (*Client) Shutdown

func (c *Client) Shutdown(ctx context.Context) error

Shutdown asks the daemon to stop gracefully.

func (*Client) Stats

func (c *Client) Stats(ctx context.Context) (Stats, error)

Stats fetches GET /local/v1/stats.

func (*Client) Status

func (c *Client) Status(ctx context.Context) (StatusInfo, error)

Status fetches GET /local/v1/status.

func (*Client) Sync

func (c *Client) Sync(ctx context.Context) (SyncResult, error)

Sync triggers POST /local/v1/sync (shard warm + upload now).

type Daemon

type Daemon struct {
	Cfg     *config.Config
	Home    string
	DB      *localdb.DB
	Ident   *identity.Identity
	CAS     *cas.Store
	Engine  *search.Engine
	Syncer  *search.Syncer
	Batcher *evidence.Batcher
	Peer    *peer.Node              // nil unless cfg.peerListen
	Cross   *verifier.CrossVerifier // nil unless cfg.idleVerification != off
	HTTP    *http.Client            // server-bound calls; nil = 30s default
	// WantedPublic is the fail-closed package publicness boundary used before
	// a queued miss is allowed to leave the machine.
	WantedPublic func(context.Context, domain.PURL) bool
	// contains filtered or unexported fields
}

Daemon is one local daemon instance bound to a CSX_HOME.

func New

func New(home string) (*Daemon, error)

New wires a Daemon entirely from the persisted config in home: identity, SQLite store, CAS, search engine, shard syncer, evidence batcher, and — when the config enables them — the peer node and the idle cross-verifier.

func (*Daemon) BaseURL

func (d *Daemon) BaseURL() string

BaseURL is the TCP base URL ("http://127.0.0.1:port"). Valid after Ready.

func (*Daemon) Close

func (d *Daemon) Close() error

Close releases the daemon's local stores. Only needed when Run was never started (CLI direct fallback) or has already returned.

func (*Daemon) Handler

func (d *Daemon) Handler() http.Handler

Handler returns the daemon mux served on both listeners.

func (*Daemon) Ready

func (d *Daemon) Ready() <-chan struct{}

Ready is closed once both listeners accept connections.

func (*Daemon) Run

func (d *Daemon) Run(ctx context.Context) error

Run serves the local API on 127.0.0.1:{cfg.daemonPort} and on the platform IPC listener (Windows named pipe / unix socket), starts the background loops, and blocks until ctx is canceled, a shutdown is requested via the API, or a listener fails. Shutdown is graceful.

func (*Daemon) SearchAndRecord

func (d *Daemon) SearchAndRecord(ctx context.Context, req domain.SearchRequest) LocalSearchResponse

SearchAndRecord runs a search and writes the local hit or miss behind csx stats, csx ui and get_local_stats.

It exists because the recording lived only inside this HTTP handler, so every other caller of Engine.Search silently counted nothing: the MCP tools, and the CLI whenever the daemon was down. Those counters are shown to the user as fact and read 0 forever. One entry point is harder to forget than a convention.

Queries never leave the machine — the hits table is local and is never uploaded — and a recording failure must not break a search.

func (*Daemon) StatsNow

func (d *Daemon) StatsNow(ctx context.Context) (Stats, error)

StatsNow computes the dashboard numbers from localdb + CAS.

func (*Daemon) SyncNow

func (d *Daemon) SyncNow(ctx context.Context) SyncResult

SyncNow runs one full sync pass — shard warm plus evidence upload — and reports what happened. Offline failures land in Errors; local state is never harmed (goal.md §3.9, §25.F).

type LocalSearchResponse

type LocalSearchResponse struct {
	domain.SearchResponse
	OfferID string `json:"offerId,omitempty"`
}

LocalSearchResponse extends a public search response with the opaque capability needed to report on this exact local offer. Keeping the field here prevents it from entering domain.SearchResponse, the public v1 schema, shard documents, or upload payloads.

type QueuePreview

type QueuePreview struct {
	SchemaVersion int                       `json:"schemaVersion"`
	Batches       []domain.ObservationBatch `json:"batches"`
	Queued        []QueuedUpload            `json:"queued"`
}

QueuePreview is the GET /local/v1/queue body — the §12.5 privacy preview. Batches is EXACTLY the "batches" array a community-mode upload would POST to /v1/evidence/batches; Queued lists other pending upload payloads verbatim.

type QueuedUpload

type QueuedUpload struct {
	Kind      string          `json:"kind"`
	Payload   json.RawMessage `json:"payload"`
	CreatedAt string          `json:"createdAt,omitempty"`
}

QueuedUpload is one non-batch pending upload with its exact payload.

type Readiness

type Readiness struct {
	FirstRunAt      string `json:"firstRunAt,omitempty"`
	InitAt          string `json:"initAt,omitempty"`
	FirstSyncAt     string `json:"firstSyncAt,omitempty"`
	MCPFirstReadyAt string `json:"mcpFirstReadyAt,omitempty"`
	MCPLastReadyAt  string `json:"mcpLastReadyAt,omitempty"`
	FirstHitAt      string `json:"firstHitAt,omitempty"`
	FirstAdoptionAt string `json:"firstAdoptionAt,omitempty"`
	// SecondsToFirstAnswer is the §5 duration, init (the consent choice) to
	// the first hit. A pointer because nil is "not both endpoints yet" and 0
	// is a real, very fast install; an int with omitempty would collapse the
	// two into the same absence. Computed here so csx stats, csx ui and
	// get_local_stats cannot each pick different endpoints.
	SecondsToFirstAnswer *int64 `json:"secondsToFirstAnswer,omitempty"`
	// Unmeasured names the stages this install reached before anything was
	// recording them, by field key ("initAt", "firstRunAt", ...).
	//
	// A separate list rather than a sentinel inside the stamp fields, because
	// every one of those is documented as an RFC3339 instant or empty and a
	// consumer parsing them must not suddenly meet a word. Empty still means
	// "not reached"; a key listed here means "reached, before this ledger
	// existed", and the two must not render the same — the panel used to tell
	// an install that had been running for days to run csx init.
	Unmeasured []string `json:"unmeasured,omitempty"`
}

Readiness is the local activation ledger rendered for a reader (docs/activation-funnel.md §7). Every field is an RFC3339 instant or empty, and empty means the stage has not been reached — §6: an unmeasured thing renders as a gap, never as a zero, so a consumer can tell "no first answer yet" from "answered at the epoch".

None of it is uploaded in any mode. S1 and S2 happen before `csx init` asks the mode question, so transmitting them would be collecting before consent exists (§2.3), and the daily anonId rotation means the server could not assemble a funnel from them anyway.

type SampleInfo

type SampleInfo struct {
	SchemaVersion int                   `json:"schemaVersion"`
	SampleID      string                `json:"sampleId"`
	CaseID        string                `json:"caseId,omitempty"`
	Status        string                `json:"status"`
	License       string                `json:"license,omitempty"`
	OriginSeeder  string                `json:"originSeeder,omitempty"`
	CreatedAt     string                `json:"createdAt,omitempty"`
	Pinned        bool                  `json:"pinned"`
	HasArtifact   bool                  `json:"hasArtifact"`
	Manifest      domain.SampleManifest `json:"manifest"`
}

SampleInfo is the GET /local/v1/samples/{id} body: the localdb row plus its parsed manifest.

type Stats

type Stats struct {
	SchemaVersion             int     `json:"schemaVersion"`
	Mode                      string  `json:"mode"`
	Hits                      int     `json:"hits"`
	Misses                    int     `json:"misses"`
	Adoptions                 int     `json:"adoptions"`
	PostHitBuildReports       int     `json:"postHitBuildReports"`
	PostHitBuildPassRate      float64 `json:"postHitBuildPassRate"`
	ExactFailureMatches       int     `json:"exactFailureMatches"`
	VerifiedDetoursOffered    int     `json:"verifiedDetoursOffered"`
	VerifiedDetoursApplied    int     `json:"verifiedDetoursApplied"`
	DetourPostHitPass         int     `json:"detourPostHitPass"`
	DetourPostHitFail         int     `json:"detourPostHitFail"`
	DetourPostHitUnknown      int     `json:"detourPostHitUnknown"`
	ReportedFailuresAvoided   int     `json:"reportedFailuresAvoided"`
	EvidenceBatchesSent       int     `json:"evidenceBatchesSent"`
	OriginSeeds               int     `json:"originSeeds"`
	CrossVerifications        int     `json:"crossVerifications"`
	EstimatedReasoningAvoided int     `json:"estimatedReasoningAvoided"`
	Estimated                 bool    `json:"estimated"` // always true: the reasoning-avoided figure is an estimate
	CacheBytes                int64   `json:"cacheBytes"`
	CacheBudgetMB             int     `json:"cacheBudgetMB"`
	Packages                  int     `json:"packages"`
	QueueDepth                int     `json:"queueDepth"`
	// EvidenceRefusedTerminal is evidence the server decided it will never
	// accept. It is not pending and it was not delivered, and until it had a
	// line of its own an operator could not tell a delivery problem from a
	// server decision — the queue simply sat at its cap.
	EvidenceRefusedTerminal int    `json:"evidenceRefusedTerminal,omitempty"`
	LastUpload              string `json:"lastUpload,omitempty"`
	LastUploadAttempt       string `json:"lastUploadAttempt,omitempty"`
	LastUploadError         string `json:"lastUploadError,omitempty"`
	// CountsArePartial reports that adoption and build-report counts were
	// tallied from the newest page of hits rather than the whole store,
	// which happens once there are more hits than one page holds.
	CountsArePartial bool `json:"countsArePartial,omitempty"`
	// Readiness is where THIS install got to between first run and first
	// proven value. It is a statement about one machine and never a count of
	// anything, which is why it can exist at all: the fleet version of the
	// same question has no honest answer (docs/activation-funnel.md §3).
	Readiness Readiness `json:"readiness"`
}

Stats is the GET /local/v1/stats body — the §12.5 dashboard numbers, all computed from the local store. EstimatedReasoningAvoided is an estimate by construction; Estimated is always true so no consumer can present it as measured.

type StatusInfo

type StatusInfo struct {
	SchemaVersion     int    `json:"schemaVersion"`
	Version           string `json:"version"`
	Mode              string `json:"mode"`
	Home              string `json:"home"`
	PeerID            string `json:"peerId"`
	Uptime            string `json:"uptime"`
	QueueDepth        int    `json:"queueDepth"`
	LastUpload        string `json:"lastUpload,omitempty"`
	LastUploadAttempt string `json:"lastUploadAttempt,omitempty"`
	LastUploadError   string `json:"lastUploadError,omitempty"`
}

StatusInfo is the GET /local/v1/status body.

type SyncResult

type SyncResult struct {
	SchemaVersion   int `json:"schemaVersion"`
	WarmedKeys      int `json:"warmedKeys"`
	UploadedBatches int `json:"uploadedBatches"`
	// ReconcileNote is non-empty when the legacy-Windows reconciliation pass
	// did not finish inside its budget. Uploads still ran; the pass resumes
	// on the next sync.
	ReconcileNote string `json:"reconcileNote,omitempty"`
	// UploadedReports is queued items delivered (adoption reports today).
	UploadedReports int `json:"uploadedReports"`
	// SetAsideReports is queued items that stopped being retried because
	// the server rejected them in a way retrying cannot fix.
	SetAsideReports int      `json:"setAsideReports,omitempty"`
	Errors          []string `json:"errors,omitempty"`
}

SyncResult is the POST /local/v1/sync (and csx sync) outcome.