confidential-website-metadata-fetcher

command module
v0.0.5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 5, 2026 License: AGPL-3.0 Imports: 15 Imported by: 0

README

Metadata Fetch

A small self-hosted service that extracts Open Graph metadata from a URL so a chat app can render link preview cards without hitting CORS or leaking a third-party API key. Built in Go around github.com/otiai10/opengraph/v2 and shipped as a Tinfoil enclave image.

The response exposes the resolved og:title, og:description, og:site_name, og:image, and the page's favicon as inlined bytes so the client never has to make a follow-up GET to an external host. Each Open Graph field is null when the source page does not advertise it; the favicon falls back to DuckDuckGo's icon service when the page does not declare one.

Quick Start

go run .

The service listens on :8089 by default.

Environment Variables

Variable Default Description
LISTEN_ADDR :8089 Address to listen on
FETCH_TIMEOUT 5s Per-request timeout when scraping
MAX_REDIRECTS 5 Maximum redirects to follow
MAX_BODY_BYTES 5242880 Reject pages larger than this
MAX_CONCURRENT 32 Concurrent in-flight scrape limit
USER_AGENT Mozilla/5.0 (compatible; MetadataFetchBot/1.0; ...) UA sent to targets
CACHE_MAX_ENTRIES 2000 LRU cache capacity
CACHE_TTL 24h Cache entry TTL

API

Fetch Metadata

POST /metadata

curl http://localhost:8089/metadata \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/article"}'

Request:

Field Type Required Description
url string yes Absolute http/https URL to scrape

Response:

{
  "url": "https://example.com/article",
  "title": "Example Article",
  "description": "A short summary of the article.",
  "site_name": "Example",
  "image": "https://example.com/cover.jpg",
  "favicon_bytes": "<base64-encoded image bytes>",
  "favicon_content_type": "image/x-icon",
  "cached": false
}

Every Open Graph field is null when the source page does not expose the corresponding tag. url reflects the final URL after redirects. favicon_bytes is base64-encoded and favicon_content_type declares the MIME type (typically image/x-icon or image/png) so the client can render the favicon as a data URL or Blob without any further network request. Both favicon fields are omitted when the lookup fails.

Health Check

GET /health returns {"status":"ok"}.

Security

  • URLs are validated before any HTTP request is made. Requests to private, loopback, link-local, multicast, unspecified, and other reserved IP ranges are rejected, as are *.local, *.internal, and *.localhost hostnames. Public hostnames are resolved and every returned address is checked against the same block list, which also applies on every redirect hop.
  • Only http and https URLs on the standard ports (80, 443) are accepted; URLs with embedded credentials (user:pass@host) are rejected.
  • The service is designed to run behind a trusted ingress (for example a Tinfoil shim) that performs caller authentication. Do not expose the upstream port directly to untrusted networks.

Docker

docker build -t metadata-fetch .
docker run -p 8089:8089 metadata-fetch

Reporting Vulnerabilities

Please report security vulnerabilities by either:

We aim to respond to legitimate security reports within 24 hours.

Documentation

The Go Gopher

There is no documentation for this package.

Directories

Path Synopsis
Package favicon proxies favicon lookups through the enclave so the user's browser never talks to the upstream icon provider directly.
Package favicon proxies favicon lookups through the enclave so the user's browser never talks to the upstream icon provider directly.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL