Documentation
¶
Overview ¶
Command shard spreads a page's asset URLs across a set of hostnames, always putting the same asset on the same one.
<img src="/i/a.png"> -> <img src="https://static2.example/i/a.png"> <img src="/i/b.png"> -> <img src="https://static0.example/i/b.png">
Sharding is an old trick for a real limit - a browser opens only so many connections per host - and the whole value of it depends on being deterministic. An asset that lands on static2 today and static0 tomorrow is an asset every cache downloads twice, so the shard has to come from the URL and from nothing else. This program takes an FNV-1a hash of the path and takes it modulo the number of hosts: same path, same host, on every page and every build, whatever order the elements were in and however the document was chunked.
The tempting alternative is the element's position - every other image on the second host, which a CSS selector can express as img:nth-child(2n). Two reasons not to. It is not stable, because the position of an image changes when the page around it changes; and the position a selector sees is not the one on the page. Structural selectors here are computed against the tokens, so a list written without its end tags has no second child at all: in <ul><li>a<li>b<li>c</ul>, "li:nth-child(2)" matches nothing and "li:first-child" matches all three. See the package documentation on structural selectors.
What it will not do:
- a URL with a host in it is already somewhere, and moving it is not this program's decision
- a URL already on one of the shard hosts is left where it is, so a second run changes nothing - even though the hash would have chosen the same host anyway
- a URL relative to the page rather than to the site - "img/a.png", with no leading slash - is left alone, because where it points depends on the document's own URL and the rewrite does not know that. Putting a host in front of it would silently name a different file
- a srcset is sharded member by member, each by its own path, because a browser picks one member and the point is that the member is cached
Source Files