escli

package
v0.5.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 17, 2026 License: Apache-2.0 Imports: 19 Imported by: 0

Documentation

Overview

Package escli is Salient's read-only Elasticsearch access layer.

Every field name and index pattern used in a query lives in FieldMap, never as an inline string in query builders. A wrong field name against Elasticsearch fails silently (empty aggregation buckets, not errors), so a version mismatch must always be fixable with --fieldmap custom.yaml rather than a rebuild.

Index

Constants

This section is empty.

Variables

View Source 
var ErrZeroBuckets = errors.New(
	"aggregation returned zero buckets from a non-empty index — this is the signature of a wrong field map; " +
		"run `salient discover` and pin correct names with --fieldmap")

ErrZeroBuckets is the wrong-fieldmap signature: the index holds documents but an aggregation on a mapped field returned no buckets. This must be loud, never silent (SALIENT_PLAN.md §13).

Functions

func DHCPLeasesQuery added in v0.3.0 

func DHCPLeasesQuery(fm FieldMap, window time.Duration) (string, error)

DHCPLeasesQuery aggregates, per assigned IP, the most-seen hostname and MAC from DHCP lease records — real lease identity, not a guess from conn-log traffic shape.

func DatasetCountsQuery 

func DatasetCountsQuery(fm FieldMap, window time.Duration, termsSize int) (string, error)

DatasetCountsQuery aggregates document counts per dataset value within the lookback window.

func EdgeAggQuery 

func EdgeAggQuery(fm FieldMap, window time.Duration, scope []string, pageSize int, afterKey map[string]any) (string, error)

EdgeAggQuery builds one page of the composite conn aggregation (§6.1). Sources: (source.ip, destination.ip, destination.port). Sub-aggs: byte sums, first/last timestamp, sensor terms. doc_count is the connection count. afterKey pages the composite; pass nil for the first page.

func GatewayCandidatesQuery 

func GatewayCandidatesQuery(fm FieldMap, window time.Duration) (string, error)

GatewayCandidatesQuery (§6.3/§8.4 primary): per sensor, terms on the responder MAC with a cardinality of responder IPs. A MAC answering for many IPs on a segment is that segment's gateway/router.

func HasWritePrivilegesQuery 

func HasWritePrivilegesQuery(pattern string) (string, error)

HasWritePrivilegesQuery asks whether the current key holds any write-class privilege on the pattern.

func MACCoverageQuery 

func MACCoverageQuery(fm FieldMap, window time.Duration) (string, error)

MACCoverageQuery counts, within the conn dataset candidates and window, total docs and docs where each MAC field exists. The answer decides whether gateway inference can use MAC convergence (§8.4 primary) or must fall back to the cross-subnet heuristic.

func NodeMACsQuery 

func NodeMACsQuery(fm FieldMap, window time.Duration) (string, error)

NodeMACsQuery aggregates, per responder IP, its most-seen destination MAC, plus a top-level MAC→IP-count agg used to identify (and exclude) gateway MACs that answer for many IPs.

func ResponderCardinalityQuery 

func ResponderCardinalityQuery(fm FieldMap, datasets []string, window time.Duration, scope []string, responderField, clientField string) (string, error)

ResponderCardinalityQuery counts, per responder value, the number of distinct client values within a dataset (§6.2). A plain terms agg suffices: responders (servers) are few, so no composite pagination is needed. responderField/clientField are usually fm.DestinationIP/ fm.SourceIP for conn-shaped datasets, but a dataset with its own ECS mapping (e.g. Zeek dhcp.log's server.address/client.address) passes its own fields — the aggregation shape is identical either way.

func SSHDocsQuery added in v0.3.0 

func SSHDocsQuery(fm FieldMap, window time.Duration) (string, error)

func SensorsQuery 

func SensorsQuery(fm FieldMap, window time.Duration, termsSize int) (string, error)

SensorsQuery aggregates observer.name values within the window.

func TLSServerNamesQuery added in v0.3.0 

func TLSServerNamesQuery(fm FieldMap, window time.Duration, scope []string, afterKey map[string]any) (string, error)

func TemporalQuery 

func TemporalQuery(fm FieldMap, window time.Duration, ip string) (string, error)

TemporalQuery builds the second-pass histogram for one responder (§6.1): a 1h date_histogram over conn docs where destination.ip = ip. Folding into hour-of-day/day-of-week happens client-side in the operator's timezone, so no scripts are needed on the grid.

func X509DocsQuery added in v0.3.0 

func X509DocsQuery(fm FieldMap, window time.Duration) (string, error)

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client wraps the low-level go-elasticsearch client. Salient only ever issues reads: GET/POST search, field_caps, resolve, and privilege checks.

func New 

func New(cfg Config) (*Client, error)

New builds a client. TLS: prefer a grid CA via CACertPath; InsecureSkipVerify is honored but the caller is responsible for the mandatory red warning.

func (*Client) CheckWritePrivileges 

func (c *Client) CheckWritePrivileges(ctx context.Context, pattern string) (WritePrivilegeCheck, error)

CheckWritePrivileges verifies the API key is genuinely read-only against the index pattern (SALIENT_PLAN.md §14).

func (*Client) DatasetCounts 

func (c *Client) DatasetCounts(ctx context.Context, fm FieldMap, window time.Duration, termsSize int) ([]DatasetCount, error)

DatasetCounts runs the dataset discovery aggregation. Returns ErrZeroBuckets when the pattern holds documents but the dataset field produced no buckets — the wrong-fieldmap signature.

func (*Client) FetchDHCPLeases added in v0.3.0 

func (c *Client) FetchDHCPLeases(ctx context.Context, fm FieldMap, window time.Duration) (map[string]DHCPLease, error)

FetchDHCPLeases returns leased IP → lease identity evidence. Absence of the DHCP dataset or its identity fields yields an empty map, not an error — nodes simply carry no hostname/MAC from this source.

func (*Client) FetchEdges 

func (c *Client) FetchEdges(ctx context.Context, fm FieldMap, window time.Duration, scope []string, maxEdges int) ([]graph.Edge, bool, error)

FetchEdges runs the composite aggregation to completion, paging on after_key, and returns typed edges. Stops with a loud warning at maxEdges.

func (*Client) FetchEvidence 

func (c *Client) FetchEvidence(ctx context.Context, fm FieldMap, window time.Duration, scope []string) (graph.Evidence, error)

FetchEvidence runs every role-evidence query needed for inference (§7), using whichever dataset candidates exist. Missing datasets yield empty maps so inference simply won't assert those roles.

func (*Client) FetchGatewayCandidates 

func (c *Client) FetchGatewayCandidates(ctx context.Context, fm FieldMap, window time.Duration) ([]graph.L2Gateway, error)

FetchGatewayCandidates returns MACs answering for ≥K distinct IPs per sensor. Absence of MAC fields yields an empty slice, not an error — the caller records nothing and maps fall back to heuristic gateways.

func (*Client) FetchNodeMACs 

func (c *Client) FetchNodeMACs(ctx context.Context, fm FieldMap, window time.Duration) (map[string]string, error)

FetchNodeMACs returns responder IP → its own NIC MAC. MACs that answer for ≥ GatewayMACMinIPs distinct IPs are gateways forwarding traffic, not host NICs, and are excluded. Absence of MAC fields yields an empty map, not an error — nodes simply carry no vendor.

func (*Client) FetchSSHHostKeys added in v0.3.0 

func (c *Client) FetchSSHHostKeys(ctx context.Context, fm FieldMap, window time.Duration) (map[string][]string, error)

func (*Client) FetchTLSFingerprints added in v0.3.0 

func (c *Client) FetchTLSFingerprints(ctx context.Context, fm FieldMap, window time.Duration, scope []string, allowed map[string]bool) (map[string][]string, error)

FetchTLSFingerprints correlates observed TLS server names to x509 certs by SAN/CN match. ponytail: best-effort SNI→cert matching, not per-connection uid correlation; add stricter joins only if this heuristic proves too weak.

func (*Client) FetchTLSServerNames added in v0.3.0 

func (c *Client) FetchTLSServerNames(ctx context.Context, fm FieldMap, window time.Duration, scope []string, allowed map[string]bool) (map[string][]string, error)

func (*Client) FetchTemporal 

func (c *Client) FetchTemporal(ctx context.Context, fm FieldMap, window time.Duration, ip string, loc *time.Location) (*graph.TemporalProfile, error)

FetchTemporal fetches and folds the activity histogram for one responder into the given timezone, returning a classified profile.

func (*Client) FetchX509Certs added in v0.3.0 

func (c *Client) FetchX509Certs(ctx context.Context, fm FieldMap, window time.Duration) ([]x509Cert, error)

func (*Client) FieldPresence 

func (c *Client) FieldPresence(ctx context.Context, pattern string, fields []string) (map[string]bool, error)

FieldPresence reports, via _field_caps, whether each requested field is mapped anywhere under the pattern.

func (*Client) Info 

func (c *Client) Info(ctx context.Context) (ClusterInfo, error)

Info authenticates and returns cluster identity.

func (*Client) MACCoverage 

func (c *Client) MACCoverage(ctx context.Context, fm FieldMap, window time.Duration) (MACCoverage, error)

MACCoverage runs the L2 presence probe over the conn dataset.

func (*Client) ResolveIndices 

func (c *Client) ResolveIndices(ctx context.Context, pattern string) ([]IndexInfo, error)

ResolveIndices expands the index pattern into concrete indices and data streams via GET _resolve/index.

func (*Client) ResponderCardinality 

func (c *Client) ResponderCardinality(ctx context.Context, fm FieldMap, datasets []string, window time.Duration, scope []string, responderField, clientField string) (map[string]graph.RoleEvidence, error)

ResponderCardinality runs the query for one dataset group and returns per-responder evidence. An empty result (no such dataset) is not an error — callers degrade role inference gracefully.

func (*Client) Sensors 

func (c *Client) Sensors(ctx context.Context, fm FieldMap, window time.Duration, termsSize int) ([]DatasetCount, error)

Sensors runs the observer.name aggregation. Zero buckets over nonzero docs is reported as a warning condition by the caller, not an error: a grid genuinely may not populate observer.name.

type ClusterInfo 

type ClusterInfo struct {
	ClusterName string `json:"cluster_name"`
	Version     struct {
		Number string `json:"number"`
	} `json:"version"`
}

ClusterInfo is the subset of GET / that test-connection reports.

type Config 

type Config struct {
	ESURL              string
	APIKey             string
	CACertPath         string
	InsecureSkipVerify bool
	Timeout            time.Duration
}

Config carries connection settings for the read-only ES client.

type DHCPLease added in v0.3.0 

type DHCPLease struct {
	Hostname string
	MAC      string
}

DHCPLease is per-leased-IP identity evidence: the hostname the client offered in its DHCP request and its NIC MAC, when present. Either field may be empty — not every client sends a hostname, and the MAC field can be absent from the grid entirely (same caveat as node MAC/vendor lookup).

type DatasetCandidates 

type DatasetCandidates struct {
	Conn     []string `yaml:"conn"`
	DNS      []string `yaml:"dns"`
	Kerberos []string `yaml:"kerberos"`
	SMB      []string `yaml:"smb"`
	SSL      []string `yaml:"ssl"`
	HTTP     []string `yaml:"http"`
	DHCP     []string `yaml:"dhcp"`
	LDAP     []string `yaml:"ldap"`
	X509     []string `yaml:"x509"`
	SSH      []string `yaml:"ssh"`
}

DatasetCandidates lists possible dataset values per log type, in preference order. Empty entries inherit the defaults.

type DatasetCount 

type DatasetCount struct {
	Dataset string
	Docs    int64
}

DatasetCount is one observed dataset value and its document count.

type FieldMap 

type FieldMap struct {
	// IndexPattern selects the indices/data streams holding Zeek logs.
	IndexPattern string `yaml:"index_pattern"`
	// Timestamp is the event time field used for window filtering.
	Timestamp string `yaml:"timestamp"`
	// DatasetField discriminates Zeek log types within IndexPattern.
	DatasetField string `yaml:"dataset_field"`
	// ObserverName identifies which sensor recorded the event.
	ObserverName  string `yaml:"observer_name"`
	MessageField  string `yaml:"message_field"`
	SSLServerName string `yaml:"ssl_server_name"`
	SSHHostKey    string `yaml:"ssh_host_key"`

	SourceIP         string `yaml:"source_ip"`
	DestinationIP    string `yaml:"destination_ip"`
	DestinationPort  string `yaml:"destination_port"`
	Transport        string `yaml:"transport"`
	Service          string `yaml:"service"`
	SourceBytes      string `yaml:"source_bytes"`
	DestinationBytes string `yaml:"destination_bytes"`
	SourceMAC        string `yaml:"source_mac"`
	DestinationMAC   string `yaml:"destination_mac"`
	// ConnState is the Zeek conn_state field grading responder
	// participation (SF/S0/REJ/...). Optional: absent on the grid means
	// evidence classification falls back to responder bytes.
	ConnState string `yaml:"conn_state"`
	// DHCPServer and DHCPClient are the ECS-mapped Zeek dhcp.log fields
	// identifying which host answered a lease request. Only populated on
	// ACK/OFFER records (a REQUEST alone has no server field) — that
	// absence is itself the confirmation signal, no message-type filter
	// needed.
	DHCPServer     string `yaml:"dhcp_server"`
	DHCPClient     string `yaml:"dhcp_client"`
	DHCPAssignedIP string `yaml:"dhcp_assigned_ip"`
	DHCPHostname   string `yaml:"dhcp_hostname"`
	DHCPHostMAC    string `yaml:"dhcp_host_mac"`
	DHCPLeaseTime  string `yaml:"dhcp_lease_time"`

	// Datasets holds candidate DatasetField values per Zeek log type.
	// Security Onion releases have shipped both bare ("conn") and
	// prefixed ("zeek.conn") dataset names; `salient discover` reports
	// which candidates actually exist on the grid so the map can be
	// pinned in a custom fieldmap.
	Datasets DatasetCandidates `yaml:"datasets"`
}

FieldMap maps Salient's abstract field concepts onto the concrete ECS/Zeek field names of the target Security Onion deployment.

func DefaultFieldMap 

func DefaultFieldMap() FieldMap

DefaultFieldMap returns the assumed SO 2.4-era mapping.

UNVERIFIED: every value below is an assumption from SALIENT_PLAN.md §6. Phase 0's exit criterion is replacing these with ground truth observed by `salient discover` against a real grid, recorded in docs/FIELDMAP.md.

func LoadFieldMap 

func LoadFieldMap(path string) (FieldMap, error)

LoadFieldMap reads a YAML override file and merges it over the defaults: any field left empty in the file keeps its default value.

func (FieldMap) CoreConnFields 

func (fm FieldMap) CoreConnFields() []string

CoreConnFields returns the fields the sanity probe verifies against the conn dataset. Order matters only for display.

func (FieldMap) L2Fields 

func (fm FieldMap) L2Fields() []string

L2Fields returns the MAC fields whose presence decides whether gateway inference (§8.4) can use its primary method.

type IndexInfo 

type IndexInfo struct {
	Name       string
	DataStream bool
}

IndexInfo is one concrete index or data stream behind the index pattern.

type MACCoverage 

type MACCoverage struct {
	ConnDocs   int64
	SrcMACDocs int64
	DstMACDocs int64
}

MACCoverage summarizes L2 evidence availability in conn logs.

type WritePrivilegeCheck 

type WritePrivilegeCheck struct {
	CanWrite      bool
	Indeterminate bool
	Detail        string
}

WritePrivilegeCheck is the result of asking ES whether the current key can write to the Zeek indices. Indeterminate is true when the security API itself was unavailable to this key.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.