skill-detector

README

skill-detector

CLI to spot risky AI skill packages before you install them.

Scans AI skill folders and agent-instruction files — Claude Code, Codex CLI, OpenCode, Cursor, Gemini CLI, GitHub Copilot, Windsurf, and similar file-based formats — for security threats so you can vet third-party skills — e.g. from skills.sh — without reading every line by hand.

⚠️ Status: Early-stage (v0.x). Usable, but rules and flags may change before 1.0.

See CHANGELOG.md for what changed in each release.

Why

Installing a skill from a third-party source means running someone else's code and prompts inside your AI assistant. A malicious skill can exfiltrate credentials, inject prompts, run shell commands, or quietly tamper with files. skill-detector runs security checks over a skill folder and flags anything suspicious, so you get a second opinion before dropping it into your skills directory.

What it checks

Eleven rule categories (25 rules total), purpose-built for AI agent skill packages and the surrounding configuration files:

Category	Catches
Injection	Shell / command injection (incl. bash fences in Markdown), prompt injection
Supply chain	Suspicious deps, unpinned installs, typosquats
Exfiltration	Outbound HTTP to unknown hosts, clipboard / env reads, DNS tunneling
Misconfiguration	Over-broad permissions, unsafe defaults
Integrity	Tampered or unsigned files
Access control	Permission-declaration vs. actual-behavior mismatches
CLAUDE.md	SQL-injection-by-instruction, Comment-and-Control patterns
settings.json	Bash(curl:*)/Bash(curl*) and PowerShell wildcards, unrestricted "*" grant, broad allows constrained by protective denies, unsanctioned hooks
Hooks	Shell metacharacter interpolation in hook command strings (real nested Claude Code schema)
MCP	External-domain reach (raise to High with --strict-mcp) and auto-installed registry packages (npx/uvx/pipx/bunx)
Reverse shell	Reverse-shell payloads in skill scripts and instruction files

Every finding is tagged with one of four trust axes — Security, Permission hygiene, Transparency, Quality — and the scanner emits an A–F grade per axis on every scan.

It also parses the skill manifest YAML, so findings can be weighed against what the skill claims it needs.

Scope — which files are read

By default the scanner inspects only AI-agent configuration files: skill manifests (SKILL.md, skill.yaml), per-harness instruction files (CLAUDE.md, AGENTS.md — Codex CLI/OpenCode, GEMINI.md, .cursorrules, .cursor/rules/*.mdc, .github/copilot-instructions.md, .github/instructions/**/*.instructions.md, .github/agents/*.agent.md, .windsurfrules), and MCP/settings configs (.claude/settings.json, .mcp.json, .claude/mcp.json, .cursor/mcp.json, .vscode/mcp.json), plus repository Copilot hooks (.github/hooks/*.json) and inline hooks in .github/copilot/settings[.local].json — plus arbitrary files inside .claude/, .codex/, .opencode/, .cursor/, .gemini/, .windsurf/ directories, plus anything under a directory containing a SKILL.md — the whole skill subtree is in scope wherever that directory sits, not just when it's installed under .claude/skills/. It honors .gitignore and skips node_modules, vendor, dist, build, target, .next, .git (a SKILL.md inside one of those creates no scope root either). Pass --scan-all to stop honoring .gitignore and walk every other scannable file; the hardcoded skip-dirs above still apply, and a SKILL.md inside one of them still creates no scope root.

A separate bounded check inspects a nested bare Git repository when its directory has a regular HEAD and config, an objects/ directory and core.bare=true. It inventories command-valued core.fsmonitor and diff.external, without running Git, hooks or configured commands. At most 64 candidates and 64 KiB per config are assessed; unsupported syntax, includes, symlink markers and over-limit configs fail the scan without grades. It never walks the object database, host/global Git configuration or excluded/ignored directories. A finding is conditional: repository contents alone cannot prove Git will load the config or that an affected agent/version will execute it.

The content rules above (injection, access control, exfiltration, etc.) run uniformly across every harness's instruction files — the checks aren't Claude-specific. Copilot hook commands are decoded from version-1 event arrays; prompt text and agent/instruction prose are analyzed separately from metadata. Structurally invalid hook files and frontmatter fail without grades; rejected items are disclosed when valid hooks remain. Folder trust, path match, agent selection, effective managed policy and runtime execution are not inferred from a repository scan. Inline settings with or without hooks inventory supported sandbox switches and worktreePathTemplate as conditional declarations; custom-agent MCP frontmatter checks supported direct-exec commands through existing security rules, without executing servers or interpreting arbitrary arguments as shell. Unsupported settings, launch environment, host/plugin inventory and org policies are not assessed. Claude Code's .claude/settings.json retains its structural checks (SD-017..SD-020).

Static findings describe submitted declarations, not effective enforcement. Supported schemas include Claude settings/hooks, Codex TOML and Copilot repository hooks, instructions and agents; applicability still depends on harness version, source precedence, provider, trust and session state. A repository scan cannot see unsupplied managed/user settings, account-synced skills, installed plugins, cloud state or referenced files outside the input. Warnings qualify grades when context is unknown; an A means no applicable finding in the files checked, not a clean running agent. Invalid analyzed configuration fails without a grade. Unreadable supported files are a known coverage gap, not evidence that they were checked.

What it does NOT check (by default)

• Source code files (.ts, .py, .go, etc.) — that's Snyk / Semgrep's lane.
• node_modules/, vendor/, dist/, lock files — always skipped.
• Files matched by your repo's .gitignore.

To scan every file with a known extension instead, pass --scan-all.

Install

# Homebrew (macOS / Linux)
brew install velzepooz/tap/skill-detector

# Go
go install github.com/velzepooz/skill-detector/cmd/skill-detector@latest

Or grab a prebuilt binary from Releases (linux / darwin / windows × amd64 / arm64).

Usage

# Scan a single skill folder (or a whole repo — only agent files inspected)
skill-detector scan ./path/to/some-skill
skill-detector scan ~/.claude/skills

# CI: fail on HIGH+ severity
skill-detector scan ./my-skill --fail-on high

# CI: fail on an axis-grade threshold
skill-detector scan ./my-skill --fail-on-axis security=B
# repeatable — combines with --fail-on (worst wins)
skill-detector scan . --fail-on-axis security=B --fail-on-axis permission_hygiene=C

# JSON output (for piping into other tools)
skill-detector scan ./my-skill --format json

# Just the 4-axis Trust Score on stdout (text format only; findings go to stderr)
skill-detector scan ./my-skill --axes-only

# Treat external MCP server URLs as High severity (default: Medium)
skill-detector scan ./my-skill --strict-mcp

# Quiet mode — exit code only
skill-detector scan ./my-skill --quiet

# Bypass scope tightening + .gitignore filtering (walks every scannable file)
skill-detector scan . --scan-all

Exit codes

Code	Meaning
0	No findings
1	Findings, all below your --fail-on / --fail-on-axis threshold
2	Finding at or above threshold (worst of severity OR axis-grade)
3	Tool error (bad arguments, unreadable path, internal failure)

Code 1 is a warning state, not a failure: findings exist, but none crossed the threshold you set. Shells and CI runners don't know that — under set -e, inside && chains, or as a GitHub Actions step, any non-zero code fails the build. Getting the warn-without-failing behavior takes one line:

# Minimal: succeed on 0 and 1, fail on 2 and 3
skill-detector scan . --fail-on high || [ $? -eq 1 ]

The minimal form still fails on 2 and 3, but collapses both into 1 — use the explicit form when the caller needs the original code:

# Explicit, and annotates the PR (GitHub Actions)
set -euo pipefail
skill-detector scan . --fail-on high && code=0 || code=$?
case $code in
  0) ;;
  1) echo "::warning::skill-detector: findings below threshold" ;;
  *) exit "$code" ;;   # 2 = threshold breach, 3 = tool error
esac

Don't collapse this to || true — that swallows 3 as well, and a scan that could not run is not a passing scan.

Configuration

Drop a .skill-detector.yml (or .skill-detectorrc — checked first, for backward compatibility) next to the skill (or pass --config) to toggle rules and allowlist known-safe patterns. Defaults are sensible; most users will only need config to suppress false positives.

Trust Score (sample output)

Trust Score
  Security             D   High-severity issue: outbound network reference detected
  Permission hygiene   D   High-severity issue: broad shell permission granted: Bash(curl *)
  Transparency         A   no findings on this axis
  Quality              A   no findings on this axis

Per-axis grade is set by the worst finding on that axis (worst-finding-wins). Each grade ships with a one-line human-readable rationale plus the rule IDs that drove it — so disagreements become rule-tuning conversations, not credibility hits.

How it compares

Plenty of great security scanners already exist — why another one?

Tool	Why not just use it for skills?
semgrep	Generic pattern engine — powerful, but you write the rules. skill-detector ships with skill-aware rules.
gitleaks	Narrower — only secrets. Doesn't cover prompt injection, permission mismatches, exfiltration.
trivy	CVEs in containers / OS packages — a different problem from skill semantics.
gosec	Scans Go source. Skills are YAML + Markdown + shell, not Go.

Short version: use skill-detector when the thing you're scanning is an AI skill package and you want rules that understand skill-manifest semantics out of the box.

Contributing

Issues are very welcome — bug reports, false positives, rule ideas, new skill formats I haven't covered.

For pull requests, please open an issue first so we can agree on the approach. This is a spare-time pet project; I'd rather not have anyone sink effort into a PR that won't land.

Build / test / lint instructions: docs/development-guide.md.

Reporting security issues

If you've found a vulnerability in skill-detector itself (not in a skill it scanned), please file a private security advisory rather than a public issue.

License

MIT — do whatever, no warranty.