Documentation
¶
Overview ¶
OpenCTEM Sensor - Universal Security Scanner/Collector Sensor
This sensor supports multiple deployment modes:
ONE-SHOT MODE (CI/CD): openctemio-sensor -tool semgrep -target ./src -push
DAEMON MODE (the default; takes platform commands unless -enable-commands=false, sensor.enable_commands: false or -standalone): SENSOR_API_URL=https://... openctemio-sensor openctemio-sensor -config sensor.yaml
Settings from before the agent -> sensor rename (AGENT_* environment variables, -agent-id, the agent: config block) keep working: the SDK (pkg/sensorkit) migrates the environment and flags, settings_migration.go the config file.
The daemon is the SDK's sensor runtime (sdk-go pkg/sensorkit): this binary adds its tools, content and executors; the SDK connects, heartbeats, polls, delivers results, renews the key and drains.
For more details, see: docs/architecture/deployment-modes.md
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
internal
|
|
|
assetctx
Package assetctx gives the SDK's converters the asset their findings belong to.
|
Package assetctx gives the SDK's converters the asset their findings belong to. |
|
cirepo
Package cirepo reads the repository a CI job is building from the CI environment (GitHub Actions, GitLab CI), for converters that must name the repository their findings belong to when the caller did not.
|
Package cirepo reads the repository a CI job is building from the CI environment (GitHub Actions, GitLab CI), for converters that must name the repository their findings belong to when the caller did not. |
|
content
Package content manages the data the sensor's scanners scan with, apart from their binaries: trivy's vulnerability database, the nuclei templates and the semgrep rules (api docs/rfcs/RFC-031-scanner-content-and-managed-sensor-updates.md).
|
Package content manages the data the sensor's scanners scan with, apart from their binaries: trivy's vulnerability database, the nuclei templates and the semgrep rules (api docs/rfcs/RFC-031-scanner-content-and-managed-sensor-updates.md). |
|
egressenv
Package egressenv tells a tool wrapper how a confined task reaches the network (api RFC-060): through its forwarder, whose relay the sandbox names in OPENCTEM_EGRESS_PROXY (sdk-go executor.EnvEgressProxy).
|
Package egressenv tells a tool wrapper how a confined task reaches the network (api RFC-060): through its forwarder, whose relay the sandbox names in OPENCTEM_EGRESS_PROXY (sdk-go executor.EnvEgressProxy). |
|
git
Package git provides git-related utilities for the sensor.
|
Package git provides git-related utilities for the sensor. |
|
lookup
Package lookup runs the passive lookup tools (rdap, asn): T0 tools that ask public registries and datasets about a target and never send a packet to the target hosts (openctem RFC-071, passive discovery).
|
Package lookup runs the passive lookup tools (rdap, asn): T0 tools that ask public registries and datasets about a target and never send a packet to the target hosts (openctem RFC-071, passive discovery). |
|
lookup/asn
Package asn is the asn lookup tool: the origin autonomous system, its holder and country, and the announced range of an address or network, and optionally the other ranges the same system announces.
|
Package asn is the asn lookup tool: the origin autonomous system, its holder and country, and the announced range of an address or network, and optionally the other ranges the same system announces. |
|
lookup/rdap
Package rdap is the rdap lookup tool: the registration data of a root domain (registrar, registrant organization, name servers, registration and expiry dates, status) from the RDAP service of its registry (RFC 9082, RFC 9083), found through the IANA bootstrap registry (RFC 9224, https://data.iana.org/rdap/dns.json).
|
Package rdap is the rdap lookup tool: the registration data of a root domain (registrar, registrant organization, name servers, registration and expiry dates, status) from the RDAP service of its registry (RFC 9082, RFC 9083), found through the IANA bootstrap registry (RFC 9224, https://data.iana.org/rdap/dns.json). |
|
reachobs
Package reachobs carries what the sensor's tools saw while trying to reach a job's targets (an answer, a closed port, no route, a timeout, a name that does not resolve) to the job's reach recorder (sdk-go pkg/reach), so the command result says which targets were not reached instead of reading them as "nothing found".
|
Package reachobs carries what the sensor's tools saw while trying to reach a job's targets (an answer, a closed port, no route, a timeout, a name that does not resolve) to the job's reach recorder (sdk-go pkg/reach), so the command result says which targets were not reached instead of reading them as "nothing found". |
|
reachprobe
Package reachprobe is the sensor's reach probe: a cheap test, on the platform's request, of whether this sensor reaches a job's targets (one TCP connect per address and port until one answers, no payload, no ICMP).
|
Package reachprobe is the sensor's reach probe: a cheap test, on the platform's request, of whether this sensor reaches a job's targets (one TCP connect per address and port until one answers, no payload, no ICMP). |
|
recon
Package recon runs the ProjectDiscovery recon tools (subfinder, dnsx, naabu, httpx, katana) as ordinary scanners.
|
Package recon runs the ProjectDiscovery recon tools (subfinder, dnsx, naabu, httpx, katana) as ordinary scanners. |
|
recon/dnsx
Package dnsx provides a scanner implementation for the dnsx DNS toolkit.
|
Package dnsx provides a scanner implementation for the dnsx DNS toolkit. |
|
recon/httpx
Package httpx provides a scanner implementation for the httpx HTTP probing tool.
|
Package httpx provides a scanner implementation for the httpx HTTP probing tool. |
|
recon/internal/flagcheck
Package flagcheck verifies that the command lines the recon scanners build use only flags the pinned tool version defines.
|
Package flagcheck verifies that the command lines the recon scanners build use only flags the pinned tool version defines. |
|
recon/internal/resolv
Package resolv picks the DNS resolvers the recon tools use: the sensor's own, never the tools' built-in public resolver lists.
|
Package resolv picks the DNS resolvers the recon tools use: the sensor's own, never the tools' built-in public resolver lists. |
|
recon/katana
Package katana provides a scanner implementation for the katana web crawler.
|
Package katana provides a scanner implementation for the katana web crawler. |
|
recon/naabu
Package naabu provides a scanner implementation for the naabu port scanning tool.
|
Package naabu provides a scanner implementation for the naabu port scanning tool. |
|
recon/subfinder
Package subfinder provides a scanner implementation for the subfinder subdomain enumeration tool.
|
Package subfinder provides a scanner implementation for the subfinder subdomain enumeration tool. |
|
scanners
Package scanners provides scanner implementations for various security tools.
|
Package scanners provides scanner implementations for various security tools. |
|
scanners/betterleaks
Package betterleaks provides a scanner implementation for the Betterleaks secret detection tool (https://github.com/betterleaks/betterleaks), the successor to gitleaks.
|
Package betterleaks provides a scanner implementation for the Betterleaks secret detection tool (https://github.com/betterleaks/betterleaks), the successor to gitleaks. |
|
scanners/codeql
Package codeql provides a scanner implementation for GitHub CodeQL.
|
Package codeql provides a scanner implementation for GitHub CodeQL. |
|
scanners/importparse
Package importparse reads the native output of the sensor's scanners (nuclei, semgrep, trivy, betterleaks) with ctis/importer: one parser library for every OpenCTEM component, fuzzed, size- and depth-limited, with a field-by-field mapping spec per format and secret values masked in every field.
|
Package importparse reads the native output of the sensor's scanners (nuclei, semgrep, trivy, betterleaks) with ctis/importer: one parser library for every OpenCTEM component, fuzzed, size- and depth-limited, with a field-by-field mapping spec per format and secret values masked in every field. |
|
scanners/internal/report
Package report decides where a scanner writes the report file it reads back.
|
Package report decides where a scanner writes the report file it reads back. |
|
scanners/internal/reporttest
Package reporttest helps test that scanners keep their reports out of the scanned tree.
|
Package reporttest helps test that scanners keep their reports out of the scanned tree. |
|
scanners/opengrep
Package opengrep runs opengrep (https://github.com/opengrep/opengrep), the SAST engine of the sast.code capability, over a source tree.
|
Package opengrep runs opengrep (https://github.com/opengrep/opengrep), the SAST engine of the sast.code capability, over a source tree. |
|
scanners/semgrep
Package semgrep provides a scanner implementation for the Semgrep SAST tool.
|
Package semgrep provides a scanner implementation for the Semgrep SAST tool. |
|
scanproc
Package scanproc runs the sensor's scanning tools (the platform-mode executors and the content refresh) the way the SDK's own exec helpers run scanners (api RFC-035 §5.3, defect B6): in their own process group, so a canceled context kills a wrapper's children too and nothing outlives the command, and at the scanner priority (nice, best-effort I/O and an OOM score above the sensor's; SENSOR_SCANNER_PRIORITY).
|
Package scanproc runs the sensor's scanning tools (the platform-mode executors and the content refresh) the way the SDK's own exec helpers run scanners (api RFC-035 §5.3, defect B6): in their own process group, so a canceled context kills a wrapper's children too and nothing outlives the command, and at the scanner priority (nice, best-effort I/O and an OOM score above the sensor's; SENSOR_SCANNER_PRIORITY). |
|
toolrun
Package toolrun runs the sensor's tools that are ported to the tool contract (sdk-go pkg/tool) out of process: each scan re-executes the sensor as "<sensor> __openctem-tool <name>" inside the task sandbox, and the runtime checks and stamps what comes back (sdk-go pkg/sensorkit/toolhost).
|
Package toolrun runs the sensor's tools that are ported to the tool contract (sdk-go pkg/tool) out of process: each scan re-executes the sensor as "<sensor> __openctem-tool <name>" inside the task sandbox, and the runtime checks and stamps what comes back (sdk-go pkg/sensorkit/toolhost). |
|
tools
Package tools provides tool installation and management utilities.
|
Package tools provides tool installation and management utilities. |
|
scripts
|
|
|
rename/sensorrename
command
Command sensorrename performs the type-aware "agent" -> "sensor" rename of the sensor binary's own Go code (RFC-023 §9.5).
|
Command sensorrename performs the type-aware "agent" -> "sensor" rename of the sensor binary's own Go code (RFC-023 §9.5). |