sprocket

README

sprocket

A Go library that probes, thumbnails, trims, and remuxes video files without ffmpeg.

Status

All four operations work on the ISOBMFF family: mp4, mov, m4v, 3gp, and 3g2, and on Matroska and WebM, in both directions. An mp4 goes into an mkv or a webm, and an mkv or a webm goes into an mp4, a mov, an m4v, a 3gp, or a 3g2. That last direction writes a fragmented file: Matroska has no up-front index, so nothing before the media says how many frames a track holds or how large each one is, and a moov cannot be placed in front of the payload from a single pass. One moof and one mdat per source cluster needs no index and costs a cluster's block headers. See Fragmented output in the package documentation for what that means for the decode times and the edit list the writer has to derive.

MPEG-TS (.ts, and the 192-byte-packet .m2ts) probes, thumbnails, and remuxes in both directions: a TS goes into any of the MP4 family, fragmented for the same reason as Matroska, and an MP4-family file goes into a TS. A transport stream has no header and no index, so Probe scans it, at most 4 MiB from the front and 4 MiB from the end, and a thumbnail seeks by estimate and reads forward to the keyframe. Trimming a TS source is not supported yet; trimming an mp4 into a TS is. See MPEG-TS in the package documentation. Work is tracked in the epic, #13.

A default build decodes VP8 and AV1 keyframes, both royalty-free. H.264 sits behind the h264 build tag and HEVC behind the hevc build tag; build with -tags h264,hevc for both. Without its tag, an H.264 or HEVC sample returns the unsupported-codec error naming the tag. VP9 has no pure-Go decoder and returns the same error. Probe, Trim, and Remux never decode, so they work for every codec in every build.

The tags are there because both codecs are patent encumbered, and a decoder is what the pools count. Via LA's active AVC list still holds a patent mapped to core decoding that runs to November 2030 in the US. HEVC is licensed through several pools and was standardized a decade later, in 2013. Opting in to either is a decision to take deliberately rather than one that arrives with go get, and a downstream that decodes on licensed hardware elsewhere can build sprocket with neither.

file, err := os.Open("clip.mp4")
if err != nil {
	return err
}
defer file.Close()

stat, err := file.Stat()
if err != nil {
	return err
}

info, err := sprocket.Probe(file, stat.Size()) // *os.File is an io.ReaderAt
if err != nil {
	return err
}
fmt.Println(info.Duration, info.Width, info.Height, info.VideoCodec, info.Rotation)

Probe reads box headers and the movie header only, never the media payload, so a 4 GiB file costs a few kilobytes of reads; a transport stream, which has no header, costs at most 8 MiB. Codec names are a documented, stable scheme, and so are the bitrate and frame rate rules; see the package documentation. Input that is not a container this library reads returns ErrUnsupportedContainer with no partial result, and headers that are malformed or cut short return ErrCorrupt.

frame, err := sprocket.Thumbnail(file, stat.Size(), 5*time.Second,
	sprocket.ThumbnailOptions{MaxDimension: 320})
switch {
case errors.Is(err, sprocket.ErrUnsupportedCodec):
	return genericIcon() // the file is fine, this library cannot picture it
case err != nil:
	return err
}
fmt.Println(frame.Time) // the keyframe's own time, which is rarely the 5s asked for
return png.Encode(out, frame.Image)

Thumbnail snaps to a keyframe, so Frame.Time is the time of the frame that came back rather than the one that was asked for. It picks the keyframe nearest the request, breaking a tie towards the earlier one and answering a time past the end of the file with the last keyframe. The track's rotation is applied, the planes are converted to RGB, and MaxDimension caps the longer side of the result without ever upscaling. Frame.Image is an *image.RGBA.

out, err := os.Create("clip-trimmed.mp4")
if err != nil {
	return err
}
defer out.Close()

start, err := sprocket.Trim(file, stat.Size(), out, sprocket.MP4, 30*time.Second, 45*time.Second)
if err != nil {
	return err
}
fmt.Println(start) // where the cut is shown from: 30s, or the keyframe before it into a TS

Trim copies a range of samples into a new file without re-encoding. A cut cannot begin decoding mid-GOP without re-encoding, so the video is copied from the keyframe at or before the start, and the frames between that keyframe and the start are hidden: behind an edit list in the MP4 family, and behind a CodecDelay in Matroska and WebM. The first frame shown is the one at the requested start, and that is the time returned. A TS output cannot hide frames, so it shows them and returns the keyframe's time; the returned time is the one to label the result with either way. Player support for the two mechanisms differs, and the package documentation under "Trim" has the details. The end takes the last sample at or before it. Every track is cut against the same window on its own sample boundaries, so audio lands within one frame of the video, about 21 ms for AAC at 48 kHz, and closer where the lead-in is hidden. The source's edit list is dropped and the composition offsets are normalized in its place. The package documentation writes out what that trades away. Payload moves as byte ranges, so cutting ten seconds out of a 4 GiB file costs the same heap as cutting them out of a small one.

if !sprocket.CanRemux(info, sprocket.MP4) {
	return fmt.Errorf("this file holds %s, which an mp4 cannot", info.VideoCodec)
}
out, err := os.Create("clip.mp4")
if err != nil {
	return err
}
defer out.Close()

return sprocket.Remux(file, stat.Size(), out, sprocket.MP4)

Remux moves the same streams into another container of the MP4 family (mp4, mov, m4v, 3gp, or 3g2) without re-encoding. The moov goes in front of the payload, so the output is progressively playable and probing it costs a read of its head, and the payload is copied as byte ranges through a single buffer, so a multi-gigabyte file costs the same heap as a small one. Sample descriptions, edit lists, and display matrices are copied verbatim, so the output probes and thumbnails to what the input did.

Not every stream fits every container: a MOV carrying ProRes or PCM has no valid representation in an MP4, though it goes into another MOV as it stands. CanRemux answers that from a Probe result, so a caller can offer only the targets that will work, and a remux that cannot happen returns ErrIncompatible naming what blocked it. Both read the same table, which the package documentation writes out in full.

What it does

Four operations:

• Probe: duration, width, height, video codec, audio codec, bitrate, framerate, rotation.
• Thumbnail: the keyframe nearest a requested time, decoded to an image.Image.
• Trim: cut a range by stream copy, hiding the frames between the keyframe and the start.
• Remux: move the same streams into a different container. MOV to MP4 is the headline case.

These are container operations. A container is bookkeeping: boxes, sample tables, timestamps, offsets. The bulk of ffmpeg is codecs, and this library re-encodes nothing, so it needs no encoder and needs a decoder for one purpose only.

Container support, in the order it was built: the ISOBMFF family (mp4, mov, m4v, 3gp, 3g2), then Matroska and WebM, then MPEG-TS. Keyframe decoding covers VP8 and AV1 and, behind the h264 and hevc build tags, H.264 and HEVC. VP9 has no known pure-Go path and returns the unsupported-codec error.

What it does not do

• No re-encoding, scaling, filters, or quality reduction of any kind. That is the line that keeps this from becoming an ffmpeg rewrite.
• No exact-timestamp frame extraction. Thumbnails snap to keyframes.
• No streaming protocols. This library reads files.
• No AVI, WMV, ASF, FLV, OGV, or MPEG-PS.

Guarantees

• No cgo and no external binaries. Decoder backends may be compiled to WASM and run under wazero, which keeps the single-binary and cross-compile properties.
• Streaming with bounded memory. Inputs are user-controlled and unbounded, so the API takes an io.ReaderAt plus a size, or an io.ReadSeeker, never the file as a []byte. Payload moves with io.Copy over byte ranges. Anything that must be read whole, a moov box or one keyframe sample, is capped by a limit this library picks and checks, never by a size the file declares.
• Cross-compiles to linux/arm64 and darwin/arm64. CI builds both.

Package layout

pkg/sprocket holds the public API: the four operations, their result types, CanRemux, and the typed errors. Container parsers and muxers live under internal/ until their API settles and there is a reason to expose them. No Go files sit at the module root.

Development

Needs GNU Make 3.82 or newer. macOS ships 3.81, so use gmake there (brew install make).

The lint tools are pinned in tools/go.mod, a separate module, so the library's go.mod lists only what the library imports.

make check        # lint
make test         # tests and coverage
make build/cross  # linux/arm64 and darwin/arm64, CGO off
make help         # every target

go test -tags h264,hevc ./...  # with the H.264 and HEVC decoders

There are two builds of this library, the default and -tags h264,hevc, so those targets each run twice and both configurations have to pass.

make test/perf times every operation against ffmpeg doing the same work on generated 1080p and 4K files and prints a table of wall times and peak memory. It needs ffmpeg, sits behind the perf build tag so make test never builds it, and runs in the Performance workflow on every pull request and nightly, where the table lands in the run summary.

License

MIT No Attribution. See LICENSE.

Dependencies are listed in THIRD_PARTY_LICENSES.md with their licenses and the notices they carry. One of them, the HEVC decoder, puts it this way: "This software implements a decoder and an encoder. It gives you no special rights on the HEVC patents. HEVC is covered by patents held by several pools and by unpooled holders; if you distribute or use this software you may need a licence from them." No software license grants patent rights in HEVC or H.264, whoever wrote the decoder, so the same goes for sprocket.

H.264 is in the same position, and its decoder carries no patent notice of its own, so this is the only one sprocket's users get. AVC is covered by patents licensed through Via LA's AVC Patent Portfolio License and by holders outside that pool; if you distribute or use this software you may need a license from them. Via LA's published fee schedule counts a decoder on its own as one unit at the same rate as an encoder, so decoding is not a lighter position than encoding.

That is what the h264 and hevc build tags are for: nothing about H.264 or HEVC is compiled in unless you ask for it. A default build decodes only AV1 and VP8, which are royalty-free, and Probe, Trim, and Remux move bytes without decoding any of them.