tamala

README

Tamala

Tamala is a work in progress toward a client-side HTTP proxy with advanced support for imperfect connections and impermanent resources. It features flexible aggressive caching with support for offline operation and retrieving earlier versions of content. Planned but not yet implemented are prefetching, automatic request hedging and retries.

Installation

GOBIN=$PWD go install codeberg.org/trigrax/tamala@main
touch config.hcl
./tamala config.hcl

See the file reference.config.hcl for config options. See example configurations below.

Overview

Tamala is an HTTP proxy as defined by RFC 9110. It is not a gateway ("reverse proxy") nor a tunnel. Tamala is also an HTTP cache as defined by RFC 9111: a private cache by default, or a shared cache when the shared setting is in effect.

Tamala receives HTTP requests on the configured TCP addresses (127.0.0.1:8080 by default). Most user agents (Web browsers or other software accessing the Web) can be configured to use one of these addresses as their HTTP proxy. Many have bespoke configuration options for this purpose, such as the -x or --proxy option to curl. Many recognize the quasi-standard environment variable http_proxy=http://127.0.0.1:8080.

When first presented with a request, Tamala forwards it to the origin server specified in the URL, and forwards the response back to the user agent, but may also store the response in its cache, so that it can be reused to satisfy a later request. By default, this follows the standard rules specified in RFC 9111: in essence, the origin server indicates a freshness lifetime during which a response is considered fresh and may be reused. However, Tamala provides multiple settings to enable more aggressive caching for various use cases.

Some minor requirements of RFC 9111 and other standards are not yet implemented.

Supporting https:

Unfortunately, a regular caching proxy is practically useless as of 2025, because most Web resources now use the https: URL scheme, which employs TLS-encrypted connections that are opaque to proxies. Some proxies can operate as tunnels ("HTTPS proxies"), blindly forwarding the encrypted bytes to and from the origin server, but Tamala can't, because it needs to inspect and modify the requests and responses.

To work around this, Tamala uses the technique variously known as TLS bump or SSL bump or MitM (man in the middle). When Tamala receives a tunnel request (CONNECT) from a client configured to use it as its https_proxy, Tamala responds as if switching to a tunnel, but actually presents its own TLS certificate, decrypts requests, and treats them like those in the normal HTTP proxy flow, forwarding them to the origin server (on a separate TLS-secured connection) and re-encrypting the responses back to the client.

In security terms, this is an attack against TLS, which pits Tamala against various defenses deployed by user agents, and may reduce robustness and security. Nevertheless, for lack of better alternatives, this technique is employed by all software in the same position as Tamala, such as Squid and mitmproxy.

On first startup, Tamala creates a new certificate authority (CA) and saves its certificate to the file named ca.pem in the configured store directory. This ca.pem can be imported into the system's or user agent's certificate store in order to trust the temporary server certificates Tamala creates on the fly for TLS bump.

Without trusting Tamala's certificates, it may be possible to instruct the user agent to simply ignore any TLS errors:

• curl provides the --insecure (-k) option
• WebKitGTK provides WEBKIT_TLS_ERRORS_POLICY_IGNORE
• Firefox allows adding an exception in its certificate preferences
• Google Chrome allows typing thisisunsafe on a TLS error page to bypass it

Control panel

The control panel is the built-in Web-based user interface for administering Tamala. It is enabled with the panel_password option in the configuration file (see reference.config.hcl).

The control panel is engaged when Tamala receives a direct HTTP request, as opposed to a proxy request (origin-form as opposed to absolute-form or authority-form). In other words, the panel can be visited as a Web site at the same URL that is used for Tamala as a proxy. With the default addresses = ["127.0.0.1:8080"], this is http://127.0.0.1:8080/. It's best to ensure that the browser doesn't proxy requests to this URL through Tamala itself. In most browsers, requests to loopback are exempt from proxying, so this happens by default.

Access to the control panel is protected by the panel_password specified in config (unless it is set to the empty string). The username is ignored and may be left blank. The password is transmitted in clear (using basic authentication), so it may be dangerous to use over unsecured links such as shared wireless networks: an attacker may be able to discover the password by listening in on the connection.

Settings

For each request, Tamala's behavior depends on settings applied from the following sources, in order:

1. built-in defaults
2. settings of each config rule matching the request, in order of definition
3. any Tamala-Control header fields in the request

See the file reference.config.hcl for a list of settings recognized by Tamala and ways to specify them in config. See example configurations below.

In addition to settings, Tamala honors standard cache request directives, such as Cache-Control: no-cache sent by Web browsers to "force refresh" a page. Where applicable, standard directives should be preferred over Tamala-specific settings for interoperability.

Tamala-Control

If a request has a Tamala-Control header field, it is parsed like the standard Cache-Control field, but each directive is matched to the setting with the same name. Unlike Cache-Control, names in Tamala-Control are case-sensitive and use underscores as word separators. For boolean settings, a missing value is equivalent to true.

For example, the following header field:

Tamala-Control: ignore_no_cache, force_disconnected=false, min_freshness=3600, cacheable_methods="GET,POST"

is equivalent to the following config block:

settings {
  ignore_no_cache = true
  force_disconnected = false
  min_freshness = 3600
  cacheable_methods = ["GET", "POST"]
}

Unlike config, Tamala-Control doesn't support arithmetic or other expressions in values. Unknown setting names in Tamala-Control cause a warning to be logged, but are otherwise ignored.

Sentinels

A sentinel monitors connection quality by sending requests to a given URL, analyzing the latency of responses, and reporting a green or red state, which can be checked in a when block to enable a configuration rule when connectivity is, respectively, good or poor. For example, this can enable reuse of stale responses on a poor connection.

A sentinel is defined and named by a sentinel block in the configuration file (see reference.config.hcl for the syntax). The name is used to refer to the sentinel from when blocks, as well as in diagnostics.

The sentinel sends an HTTP request to the specified target URL every interval seconds, and measures the time to an HTTP response. It does not inspect the response: all status codes, including 5xx server errors, are ignored. On the other hand, network errors or timeouts (absence of HTTP response) are treated as if the time to response is extremely large. Responses taking longer than interval do not block sending further requests. This is similar to the operation of the standard ping command, but using HTTP requests instead of ICMP packets. Sentinel requests are sent directly: they are not affected by Tamala settings and other proxy functionality, and do not appear in history.

At startup, the sentinel is green. Before sending each request, the sentinel computes the specified percentiles of the time to response over the last window of requests. If the value at red_percentile is greater than or equal to the red_threshold, the sentinel switches to red. If the value at green_percentile is less than or equal to the green_threshold, it switches to green. Otherwise, the sentinel remains in its current state, and additionally reports itself as yellow (so, red-yellow or green-yellow), meaning that the latest measurements are inconclusive. A yellow sentinel can be manually switched to red or green via the control panel.

Caching

When Tamala receives a request with a cacheable method (usually GET or HEAD), it computes its cache key from the request method, URL and body. (Most cacheable requests have no body, but one might imagine, for example, a Web search form submitted with method=POST: such queries can be cached by Tamala if the cacheable_methods setting includes POST.)

The cache key corresponds to a directory in Tamala's store. These directories are further grouped by the host (+ port) of the request URL. For example, GET http://example.com/ (with no body) corresponds to the directory example.com/k8QQpr8XV__wV_JmtpffnPF2_aNHRVTEAA.

This directory may contain one or more entries. Each entry is a file containing a request and response.

A typical cache key will contain a single entry. Whenever Tamala stores a new entry, it deletes the old one. But there are two ways by which a single key can accrue multiple entries:

1. Depending on the retain setting, Tamala can retain some of the older entries. These can later be retrieved with the cutoff setting, effectively "rolling back" to an earlier version of the resource.
2. When responses have the Vary header field, it can split the requests into variants that are cached independently. For example, under Vary: Accept, requests with Accept: text/html and with Accept: application/json are distinct variants. Each variant will have its own set of entries to retain.

Cache eviction is not yet implemented: Tamala deletes older entries from the same key, but not from other keys; thus the cache grows unbounded as more and more different resources are accessed. There is also currently no deduplication: even if several entries for a single key have identical response bodies, each is stored in full.

Tamala doesn't implement conditional revalidation with entity tags or timestamps: a cache miss always requires transfering the entire response anew.

Example configurations

Override server cache controls

Many Web servers send Cache-Control directives to prevent caching. This often improves user experience under good connectivity, but also prevents Tamala from fulfilling its purpose under poor connectivity. Thus, most uses of Tamala will require ignoring these directives:

rule {
  settings {
    ignore_no_store = true
    ignore_no_cache = true
    ignore_must_revalidate = true
    heuristic_freshness = 0
  }
}

Setting heuristic_freshness to 0 ensures that Tamala doesn't reuse such responses by default.

In some cases, restrictive cache controls may be important to heed even for Tamala. For example, a live video stream is genuinely not cacheable by its nature, but the above settings may cause it to utilize up to max_cache_entry_size (4 MiB by default) on disk, and even to be reused in offline operation. Unfortunately, there is no simple way to know when this is the case. However, the ignore_* settings may be reset on a case-by-case basis:

rule {
  when { path = "/camera-feed" }
  settings { ignore_no_store = false }
}

rule {
  when { host = "dynamic.example.net" }
  settings { ignore_no_cache = false }
}

Disable caching

To exclude certain resources from caching, the cacheable_methods setting may be set to an empty list:

rule {
  when { host = "localhost:8000" }
  settings { cacheable_methods = [] }
}

Force caching

When a particular resource is known to be seldom updated, or when up-to-date information is not required, Tamala can be configured to treat it as fresh for a prolonged time, avoiding network access and thus reducing latency and data usage:

rule {
  when { host = "example.com" }
  settings { min_freshness = 7 * 24 * 3600 }
}

This can be overridden at any time with a Web browser's "force reload" feature, often invoked by typing Ctrl+F5 or Cmd+Shift+R.

Per-application profiles

To apply different settings depending on the user agent, one way is to use multiple TCP ports. With the following config, any client connecting to Tamala on port 8081 (as with https_proxy=http://127.0.0.1:8081) will get cached responses for at least a week:

addresses = ["127.0.0.1:8080", "127.0.0.1:8081"]

rule {
  when { address = "127.0.0.1:8081" }
  settings { min_freshness = 7 * 24 * 3600 }
}

If the application supports customizing HTTP request headers, it may be more convenient to specify the desired settings directly in the Tamala-Control field:

curl --proxy http://127.0.0.1:8080 --insecure \
     --header 'Tamala-Control: min_freshness=604800' \
     http://example.com/

Offline mode

When the network is unreachable, Tamala may be instructed to serve cached responses even if they have gone stale:

rule {
  # By default, allow any staleness.
  settings { stale_if_error = 999999999 }
}

rule {
  # For some resources, serving overly stale responses might be more confusing than helpful.
  when { path_prefix = "/weather/" }
  settings { stale_if_error = 24 * 3600 }
}

Go offline on a poor connection

When the network is technically reachable but connectivity is too poor to rely on, it may be useful to disconnect Tamala forcibly. This can be accomplished with a sentinel. The following sentinel will attempt an OPTIONS http://example.net/ request every second, and switch to red when at least half of the last 30 probes take longer than 10 seconds (or are lost altogether), then back to green when at least 80% of probes take shorter than 300 ms again. (In between, the sentinel will be yellow and can be toggled manually on the control panel.)

sentinel example {
  method = "OPTIONS"
  target = "http://example.net/"
  interval = 1
  window = 30
  red_percentile = 50
  red_threshold = 10
  green_percentile = 80
  green_threshold = 0.3
}

Now this sentinel can be used in a rule like this:

rule {
  when { red = "example" }
  settings { force_disconnected = true }
}

It's also useful to add this sentinel as a condition for the stale rules shown above, because otherwise they might be triggered by a transient error, when a stale response is undesirable:

rule {
  when { red = "example" }
  settings { stale_if_error = 999999999 }
}

To hedge against potential outages of http://example.net/, it's possible to define another sentinel and configure the rule to fire only when both are red:

rule {
  when { red = "example" }
  when { red = "another" }

Prefetch resources for later use

Of course, the above configuration for offline mode is only useful insofar as Tamala has the necessary resources cached. This can be ensured by fetching them in advance. At present, this is accomplished by manually running a program such as Wget:

http_proxy=http://127.0.0.1:8080 https_proxy=http://127.0.0.1:8080 \
wget --ca-certificate ~/.cache/tamala/ca.pem \
     --directory-prefix /tmp --no-directories --delete-after \
     -e robots=off --timeout 10 --compression gzip \
     --header 'Tamala-Control: force_disconnected=false' \
     --recursive https://example.com/

The last line selects resources to fetch: see the Wget manual for details. Note that --delete-after causes Wget to immediately delete the files it downloads, so they will only remain in Tamala's cache.

Prefetching is complicated by variance between the request headers sent by Wget and by the actual Web browser. Especially problematic is the Cookie field, which may drastically change the responses (e.g. a personalized page for a logged-in user), but content negotiation fields and User-Agent may also pose an obstacle. To work around this, either Wget must be instructed to send the same --cookie, --compression, --user-agent and other --header fields as the Web browser, or else Tamala may be configured to ignore them for purposes of matching requests to responses. As above, this is best enabled only on a poor connection (as reported by a sentinel), and ideally on a per-site or per-resource basis:

rule {
  when {
    host = "example.org"
    red = "example"
  }
  settings { ignore_vary = ["Cookie", "Accept", "Accept-Encoding", "Accept-Language", "Accept-Charset", "User-Agent"] }
}

Replay HTTP traffic

Sometimes a programmer developing an application wishes to capture a snapshot of its HTTP requests and responses and replay them while working on the program, to keep behavior reproducible and avoid hitting the network and the origin servers. Tamala can serve this purpose if the cacheable_methods setting includes all HTTP methods employed by the application:

Tamala-Control: cacheable_methods="GET,POST,PUT,DELETE,OPTIONS", min_freshness=999999999

Once all the desired requests and responses are thus captured, additionally setting force_disconnected=true will prohibit any others.

This is probably best served by a dedicated instance of Tamala whose store will not be used for other purposes.