Documentation
¶
Overview ¶
Package transmit posts a scan's NDJSON to a Riskability server.
It is the client half of docs/API.md in the riskability-server repository: open a scan with its manifest, push the records as numbered batches, close it and read the reconciliation verdict back. Everything here exists because of one property of that contract -- the server compares what the manifest declared against what it stored, and disagreement is an error rather than a warning field. This package's job is to make sure a disagreement means the data really is missing, and never that the transport lost it quietly.
File output is not a fallback for this and this is not a replacement for file output. Air-gapped estates are the likeliest audience for the product and they move files by means they already trust; a site that can reach a server gets both.
Index ¶
- Constants
- func IsPermanent(err error) bool
- func SPKIPin(cert *x509.Certificate) string
- type CheckResult
- type Client
- type CloseVerdict
- type Options
- type PermanentError
- type Spool
- func (s *Spool) Ack(index int) error
- func (s *Spool) Done() error
- func (s *Spool) EachBatch(fn func(index int, body []byte, lines int) error) error
- func (s *Spool) Manifest() ([]byte, error)
- func (s *Spool) PayloadPath() string
- func (s *Spool) Records() (int, error)
- func (s *Spool) SetScanID(id string) error
- func (s *Spool) State() SpoolState
- type SpoolState
Constants ¶
const ( // DefaultBatchLines is a compromise between request count and blast // radius. A 15,000-component host becomes eight requests; a batch that // fails and cannot be retried loses at most this many records, and the // server names the batch index in its status so the loss is identifiable // rather than a hole somewhere in the middle. DefaultBatchLines = 2000 // DefaultBatchBytes bounds the request body before compression. Whichever // of the two limits trips first ends the batch: line count alone is not // enough, because a host with large attribute maps can put 2,000 lines // well past any sane body limit, and a 4 MB scan in one request is the // specific thing this exists to prevent. DefaultBatchBytes = 1 << 20 // DefaultAttempts counts the first try. Five attempts with the backoff // below spans roughly half a minute, which covers a rolling restart of the // server without covering an outage long enough that the collector should // simply give up and leave the spool for the next run. DefaultAttempts = 5 // DefaultRequestTimeout bounds one HTTP request, not the whole upload. DefaultRequestTimeout = 60 * time.Second )
Defaults for the tunables. They are exported so the flag layer states the same numbers the help page prints.
const SpoolDirName = ".swinv-spool"
SpoolDirName is the directory, under --out, where scans awaiting upload sit.
A dotted directory next to the reports, matching the convention the heartbeat state already set: a collector globbing *.json or *.ndjson does not pick it up, and removing --out removes the queue with it rather than leaving scans behind that will be uploaded months later as though they were current.
Variables ¶
This section is empty.
Functions ¶
func IsPermanent ¶
IsPermanent reports whether err is one nothing will fix.
func SPKIPin ¶ added in v0.8.0
func SPKIPin(cert *x509.Certificate) string
SPKIPin computes the pin of a certificate, exported so --transmit-check can print the pin it observed - which is exactly the string an operator needs to copy into the flag.
Types ¶
type CheckResult ¶ added in v0.8.0
CheckResult is one line of the preflight report.
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client posts scans to one server.
func New ¶
New builds a client and validates everything that can be validated without touching the network.
Configuration errors surface here rather than after a multi-minute scan: a typo in a certificate path that only appears at upload time costs the whole scan, and it is the sort of mistake that is made once and repeated hourly by a timer.
func (*Client) Check ¶ added in v0.8.0
func (c *Client) Check(ctx context.Context) []CheckResult
Check validates the endpoint, authentication, TLS and clock without scanning or sending anything.
This is the flag that makes a rollout debuggable: without it, diagnosing a broken deployment means running a full scan to find out the token was wrong. The status endpoint of a scan that does not exist is the probe - a 404 proves the server answered and the credentials were accepted, while a 401 or 403 names the actual problem. One attempt, no retries: a preflight that retries for half a minute answers slower than it has to.
func (*Client) Endpoint ¶
Endpoint is the /ingest root this client posts to. It is what a spool records, so a backlog can be matched to the server it was collected for.
func (*Client) NewSpool ¶
func (c *Client) NewSpool(dir, scanID, hostname string, declared int, _, _ os.FileMode, write func(io.Writer) error) (*Spool, error)
NewSpool writes a scan's NDJSON to the spool directory and records the state that makes it resumable.
The payload is written before the state file, and both atomically. A crash between the two leaves an orphan payload with no state, which Pending ignores -- one wasted scan. The other order would leave a state file pointing at a payload that does not exist, and a resume would report a missing file for a scan that was never written.
func (*Client) Send ¶
Send uploads one spooled scan and returns the server's verdict.
The spool is the unit of resumption: it holds the exact bytes that were declared, so a run that dies half way up can be finished by a later one without rescanning the machine and without the counts shifting underneath the manifest.
type CloseVerdict ¶
type CloseVerdict struct {
ScanID string `json:"scan_id"`
Declared int `json:"declared_components"`
Stored int `json:"stored_components"`
Reconciled bool `json:"reconciled"`
Message string `json:"message"`
}
CloseVerdict is the reconciliation the server returns from close.
A mismatch arrives as HTTP 409 with these two numbers, which is the reason the manifest exists at all. It is surfaced verbatim rather than summarised: "declared 3993, stored 15" is the sentence that would have ended a day of debugging in its first minute.
type Options ¶
type Options struct {
// BaseURL is the server's API root, e.g. https://riskability.example/api/v1.
// A trailing /ingest is added; a BaseURL that already ends in /ingest is
// accepted too, because that is the URL an operator is most likely to be
// given.
BaseURL string
// Token is the bearer token, if the estate distributes tokens.
Token string
// ClientCertFile and ClientKeyFile are the client certificate, if the
// estate runs a CA instead. Both mechanisms are supported and both may be
// configured at once: some servers require the certificate for transport
// and the token for the account it maps to.
ClientCertFile string
ClientKeyFile string
// KeyPassphrase decrypts ClientKeyFile when the estate hands out
// passphrase-protected keys, which any estate running a CA does.
KeyPassphrase []byte
// CAFile is a PEM bundle to verify the server against, for an internal CA
// that is not in the system trust store.
CAFile string
// Pins verifies the server by public key instead of by chain: each entry
// is the base64 SHA-256 of a SubjectPublicKeyInfo. Repeatable so a key
// rotation is two pins for a while rather than a flag day. Mutually
// exclusive with InsecureSkipVerify.
Pins []string
// TLSMinVersion raises the TLS floor. Zero means 1.2; there is no value
// that lowers it.
TLSMinVersion uint16
// InsecureSkipVerify disables server certificate verification. It exists
// for a first-day trial against a self-signed endpoint and is loud
// everywhere it appears.
InsecureSkipVerify bool
// Compress controls the request bodies: "auto" (the default - gzip when
// it helps), "always", or "never". "never" exists to diagnose the proxy
// or WAF that mangles gzipped bodies, a real failure mode that is
// miserable to chase when compression cannot be turned off.
Compress string
// RateLimitBytesPerSec caps upload throughput for metered links.
// Zero is unlimited.
RateLimitBytesPerSec int64
BatchLines int
BatchBytes int
Attempts int
RequestTimeout time.Duration
// HTTPClient overrides the constructed client. Tests set it; nothing else
// should need to.
HTTPClient *http.Client
// Logf receives progress. Never nil after New.
Logf func(string, ...any)
}
Options configures one client.
type PermanentError ¶
type PermanentError struct{ Err error }
PermanentError marks a failure that retrying cannot fix.
The distinction is the whole of the retry policy. A 401 retried five times is five identical rejections and a collector that reports "network trouble" for a token nobody renewed; a 503 not retried at all is a scan lost to a rolling restart.
func (*PermanentError) Error ¶
func (e *PermanentError) Error() string
func (*PermanentError) Unwrap ¶
func (e *PermanentError) Unwrap() error
type Spool ¶
type Spool struct {
// contains filtered or unexported fields
}
Spool is one scan's NDJSON, on disk, plus the record of how much of it the server has taken.
func Pending ¶
Pending lists spooled scans for one endpoint that were never finished, oldest first.
Oldest first so a backlog drains in the order it accumulated: a host that was offline for six hours reports its six scans as a history rather than as six copies of "now" in an arbitrary order.
func (*Spool) EachBatch ¶
EachBatch walks the payload's record lines in batches, calling fn with the batch index, its body, and how many lines it holds.
The manifest line is skipped: it was already delivered by the open call, and sending it again as a record would have the server store the scan's own description as though it were part of the inventory.
Boundaries come from the recorded state rather than from current flags, so batch 7 means the same records on a resume as it did on the run that died.
func (*Spool) Manifest ¶
Manifest returns line 1 of the payload: the self-describing heartbeat.
It is an error for it to be missing. The manifest is the only thing that lets the server reconcile what it stored against what was sent, and a scan uploaded without one is precisely the pipeline that cannot tell "nothing to find" from "nothing arrived".
func (*Spool) PayloadPath ¶
PayloadPath is the NDJSON file this spool will upload.
func (*Spool) Records ¶
Records counts the record lines the payload will send, excluding the manifest. It is what the transmitted total is checked against.
type SpoolState ¶
type SpoolState struct {
ScanID string `json:"scan_id"`
Endpoint string `json:"endpoint"`
Hostname string `json:"hostname"`
Created time.Time `json:"created_at"`
BatchLines int `json:"batch_lines"`
BatchBytes int `json:"batch_bytes"`
// Acked is the number of batches the server has confirmed, so batch index
// Acked is the next one to send. Only ever advanced after a 2xx.
Acked int `json:"acked_batches"`
// Declared is what the manifest claims, kept here so a resumed upload can
// state the same number without re-parsing the payload.
Declared int `json:"declared_components"`
}
SpoolState is what survives a process that dies mid-upload.
Everything needed to finish the upload without rescanning the machine. In particular the batch sizes: they define where the batch boundaries fall, so a resume that used different ones would send batch 4 containing records the server already stored as part of its batch 3, and the idempotency key would not save it because the index would look new.
Source Files