tmk

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 15, 2026 License: BSD-3-Clause Imports: 2 Imported by: 0

README

tmk

TMK+PDQF, a video-similarity hash, in pure Go with no dependencies and no cgo.

h := tmk.New()
for t, feature := range features {   // one [256]float32 per frame, at 15 fps
    h.Add(t, feature)
}
hash, err := h.Finish()

s, _ := tmk.Level1(a, b)   // cheap filter: cosine of the mean frame
s, _ = tmk.Level2(a, b)    // the real score, 1.0 identical, 0.0 worst

A score of 0.7 or more, at either level, is the reference's validated threshold.

What it is for

TMK answers "are these the same video?" when the two differ in frame rate, container, resolution, light watermarking, or a few seconds of leader and trailer. It is not built to find a clip inside a longer video, nor to survive cropping or heavy overlays.

⭐ Its useful property is that the signature is a fixed size whatever the video's length — about 256 KB, of which the first kilobyte already separates almost every pair. A library of videos costs a predictable amount of memory and can be compared without the originals.

Two levels, and why there are two

Level 1 is the cosine similarity of the mean frame feature. It carries no timing information at all — the same shots in a different order score the same, and a test demonstrates exactly that by hashing a video backwards. It is a filter, cheap enough to index.

Level 2 is the temporal kernel. For each period it slides one video against the other over every whole-frame offset and keeps the best match, then takes the best period. That sliding is what matches a video against itself with the leader removed: a test shifts a video by 97 frames and it still scores above threshold.

The reference computes level 2 only when level 1 passes, because when level-1 scores differ greatly the level-2 scores do too.

⛔ It does not decode video

This package takes frame features, one [256]float32 per frame — which is what go-perceptual/pdq's Float returns. TMK+PDQF is defined as PDQ minus its binarisation step, so the descriptor is the image hash's own float stage.

Decoding belongs to whoever owns the container and the codec. Keeping it out means this package is testable without a single video file, and usable with any decoder you already have.

⛔ Add takes the frame's timestamp, not a counter. The kernel's entire content is where a frame sits in time, so a caller that drops frames must keep the original indices rather than renumbering what it kept.

⛔ Frames must be resampled to 15 fps before hashing. That is part of the algorithm, not a tuning knob: the kernel compares phases at integer frame indices, so two videos hashed at different rates are not comparable even though both hashes look perfectly valid.

Parameters

The four periods {2731, 4391, 9767, 14653} and the 32 Fourier coefficients are the Poullot values used by the reference. NewWith takes your own, and Level1/Level2 refuse a pair built with different ones: a score across mismatched periods is meaningless and looks exactly like a real one.

Where it comes from

Published as The TMK+PDQF video-hashing algorithm and the PDQ image-hashing algorithm (hashing/hashing.pdf in facebook/ThreatExchange), with the kernel from Poullot et al. Ported from the reference C++ in tmk/cpp and from the derivation in its tmkfv.h, both read rather than summarised.

Licence

BSD-3-Clause.

Documentation

Overview

Package tmk implements TMK+PDQF, a video-similarity hash.

TMK answers "are these two videos the same video?" for videos that differ in frame rate, container, resolution, light watermarking or a few seconds of leader and trailer. It is NOT built to find a clip inside a longer video, nor to survive cropping or heavy overlays.

Its useful property is that the signature has a FIXED SIZE whatever the video's length — about 256 KB, of which the first kilobyte already separates almost every pair — so a library of videos costs a predictable amount of memory and can be compared without the originals.

The algorithm is published as "The TMK+PDQF video-hashing algorithm and the PDQ image-hashing algorithm" (hashing/hashing.pdf in github.com/facebook/ThreatExchange), with the kernel from Poullot et al. This package is a port of the reference C++ in tmk/cpp of that repository, and of the derivation written in its tmkfv.h.

⛔ This package does NOT decode video. It takes frame features — one [256]float32 per frame, which is what go-perceptual/pdq's Float returns — at a fixed frame rate. Decoding belongs to whoever owns the container and the codec, and keeping it out means this package is testable without a single video file and usable with any decoder.

Index

Constants

View Source
const DefaultThreshold = 0.7

DefaultThreshold is the reference's validated threshold for both the level-1 and the level-2 score.

View Source
const Dim = 256

Dim is the length of one frame feature: PDQF, the PDQ transform before its binarisation step.

View Source
const FPS = 15

FPS is the frame rate frames must be resampled to before hashing.

⛔ It is part of the algorithm, not a tuning knob: the temporal kernel compares phases at integer frame indices, so two videos hashed at different rates are not comparable even though both hashes look valid.

Variables

View Source
var ErrIncomparable = errors.New("tmk: hashes were built with different parameters")

ErrIncomparable is returned when two hashes were built with different parameters. ⛔ It is an error rather than a number because a score computed across mismatched periods is meaningless and looks exactly like a real one.

View Source
var ErrNoFrames = errors.New("tmk: no frames were ingested")

ErrNoFrames is returned when a hash is finished without any frame.

View Source
var PoullotCoefficients = [...]float64{
	0.0708041893112, 0.13937789309, 0.132897260304,
	0.122765735552, 0.109878684888, 0.09529606433,
	0.0800986647852, 0.0652590650356, 0.0515478238322,
	0.0394851531195, 0.0293374252025, 0.0211492623679,
	0.0147973073245, 0.0100512818746, 0.0066306408014,
	0.00424947117334, 0.0026467615764, 0.00160270959695,
	0.000943882629639, 0.000540841638603, 0.000301633183798,
	0.000163800158855, 8.66454753015e-05, 4.46626303151e-05,
	2.24429442235e-05, 1.09982139799e-05, 5.25823487999e-06,
	2.45358229988e-06, 1.11781474895e-06, 4.97406489221e-07,
	2.16265487234e-07, 9.19087006565e-08,
}

PoullotCoefficients are the 32 Fourier weights, likewise.

View Source
var PoullotPeriods = [...]int{2731, 4391, 9767, 14653}

PoullotPeriods are the four periods of the temporal kernel, from Poullot et al. via the reference implementation.

Functions

func Level1

func Level1(a, b *Hash) (float64, error)

Level1 is the cosine similarity of the two level-1 features: 1.0 for the same video, -1.0 at the opposite end.

It carries NO timing information — it is the mean frame, and two videos with the same shots in a different order score the same. That is exactly why it is only a filter: the reference computes the expensive level-2 score only when this one passes, because when level-1 scores differ greatly the level-2 scores do too.

func Level2

func Level2(a, b *Hash) (float64, error)

Level2 is the TMK pair score: 1.0 for identical hashes, 0.0 at worst.

For each period it slides one video against the other over every whole-frame offset and keeps the best match, then takes the best period. That sliding is what lets it match a video against the same video with a few seconds of leader removed.

The kernel at offset delta is

K = u_c0 . v_c0
  + sum_j cos(j d) (u_cj . v_cj + u_sj . v_sj)
  + sum_j sin(j d) (u_sj . v_cj - u_cj . v_sj)

⭐ The four dot products do not depend on the offset, so they are computed ONCE per coefficient and then combined for each of the thousands of offsets. Recomputing them inside the offset loop — the obvious reading of the formula — multiplies the work by 256 and changes no answer.

Types

type Hash

type Hash struct {
	// Average is the level-1 feature: the plain mean of the frame features,
	// carrying no timing information at all. Comparing two of these is cheap
	// and is what a large index stores.
	Average [Dim]float32
	// Cos and Sin are the level-2 features, indexed [period][coefficient][k].
	Cos [][][]float32
	Sin [][][]float32
	// contains filtered or unexported fields
}

Hash is one video's signature.

func (*Hash) Frames

func (h *Hash) Frames() int

Frames is the number of frames that went into the hash.

type Hasher

type Hasher struct {
	// contains filtered or unexported fields
}

Hasher accumulates frame features into a Hash.

func New

func New() *Hasher

New returns a Hasher using the Poullot parameters.

func NewWith

func NewWith(periods []int, coefficients []float64) *Hasher

NewWith returns a Hasher with explicit parameters. ⛔ Two hashes are only comparable when they were built with the SAME periods and coefficients; Level1 and Level2 refuse a mismatched pair rather than returning a number that means nothing.

func (*Hasher) Add

func (h *Hasher) Add(t int, feature [Dim]float32)

Add ingests the feature of the frame at integer time t, counted in frames at FPS.

⛔ t is the TIMESTAMP, not a counter: the kernel's whole content is where a frame sits in time, so a caller that drops frames must keep the original indices rather than renumbering what it kept.

func (*Hasher) Finish

func (h *Hasher) Finish() (*Hash, error)

Finish normalises the accumulated features and returns the hash.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL