jsweb

README

JSWeb - JavaScript Secret Scanner

JSWeb is a tool that scans web pages for JavaScript files and checks them for potential secrets and sensitive information. It uses the Gitleaks configuration format and InvisiblePlaywright (stealth-patched Firefox) for browser automation.

Features

• Scans web pages for JavaScript files using InvisiblePlaywright Firefox
• Scans the page's browser localStorage for secrets
• Uses Gitleaks rules for secret detection
• Supports entropy-based detection with configurable thresholds
• Advanced allowlist functionality with regex and stopword support
• Provides code snippets with context around matches
• Outputs findings in JSON format
• Rate limiting to avoid overwhelming servers
• Skips third-party domains to reduce noise
• Automatic InvisiblePlaywright engine fetch when missing

Prerequisites

• Go 1.16 or later
• Python 3 with invisible-playwright installed
• One-time engine download: python -m invisible_playwright fetch

Installation

1. Clone the repository:

git clone https://github.com/nautical/jsweb.git
cd jsweb

1. Install dependencies:

go mod download
pip install invisible-playwright
python -m invisible_playwright fetch

Usage

Basic usage:

go run main.go https://example.com

The tool will:

1. Ensure the Playwright-go driver is present (no Chromium download)
2. Resolve the InvisiblePlaywright Firefox binary + stealth prefs
3. Download the Gitleaks configuration if not present
4. Launch headless stealth Firefox
5. Navigate to the specified URL
6. Find all JavaScript files
7. Scan each file for potential secrets
8. Scan the page's localStorage entries for potential secrets
9. Output findings in JSON format

Secrets found in localStorage are reported with a file value of localStorage://<key>, so you can tell which storage key the match came from.

You can also inject localStorage entries before scanning (useful for authenticated sessions):

go run main.go --localstorage 'token=abc123; apiKey=secret' https://example.com

InvisiblePlaywright environment overrides

Useful in containers where Python may not regenerate prefs at runtime:

Variable	Purpose
INVISIBLE_PLAYWRIGHT_EXECUTABLE	Absolute path to the patched Firefox binary
INVISIBLE_PLAYWRIGHT_PREFS	Path to a JSON file of firefoxUserPrefs
INVISIBLE_PLAYWRIGHT_SEED	Seed used when generating prefs via Python (default 1)
INVISIBLE_PLAYWRIGHT_PYTHON	Python interpreter for python -m invisible_playwright

Generate a prefs file (regenerate when upgrading invisible-playwright):

python -c "import json;from invisible_playwright import get_default_stealth_prefs;print(json.dumps(get_default_stealth_prefs(seed=1, humanize=True)))" > prefs.json
export INVISIBLE_PLAYWRIGHT_EXECUTABLE="$(python -m invisible_playwright path)"
export INVISIBLE_PLAYWRIGHT_PREFS="$PWD/prefs.json"

Output Format

The tool outputs findings in JSON format with the following structure:

{
  "findings": [
    {
      "description": "Description of the finding",
      "file": "URL of the JavaScript file",
      "rule_id": "ID of the rule that matched",
      "tags": ["list", "of", "tags"],
      "secret": "The matched secret",
      "context": "The full match context",
      "line": "Line number where the secret was found",
      "entropy": 4.5,
      "code_snippet": "Code snippet with context around the match"
    }
  ]
}

Configuration

The tool uses the Gitleaks configuration format. The configuration file (gitleaks.toml) will be downloaded automatically if not present. You can also provide your own configuration file.

Rule Structure

[[rules]]
id = "rule-id"
description = "Description of the rule"
regex = "regex pattern"
secretGroup = 1
entropy = 3.5
path = "path pattern"
keywords = ["keyword1", "keyword2"]
tags = ["javascript", "api-key"]

[[rules.allowlists]]
description = "Allowlist description"
regexTarget = "match"  # Can be "match", "secret", or "line"
regexes = ["regex1", "regex2"]
stopwords = ["word1", "word2"]
condition = "OR"  # Can be "OR" or "AND"

Allowlist Features

• Global and rule-specific allowlists
• Multiple allowlist conditions (AND/OR)
• Target-specific matching (match, secret, or line)
• Regex and stopword support
• Rule targeting for global allowlists

Third-Party Domains

The tool automatically skips JavaScript files from common third-party domains to reduce noise. This includes:

• CDN services (Cloudflare, jsDelivr, etc.)
• Analytics services (Google Analytics, etc.)
• Social media services (Facebook, Twitter, etc.)
• Cloud services (AWS, Google Cloud, etc.)

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the GNU General Public License v3.0 - see the [LICENSE] file for details.