saliency

package
v0.8.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 18, 2026 License: MIT Imports: 5 Imported by: 0

Documentation

Overview

Package saliency is a from-scratch, standard-library-only port of a useful subset of OpenCV's contrib saliency module: algorithms that highlight the regions of an image (or video) most likely to draw human visual attention.

The package sits on top of the root module github.com/malcolmston/opencv (imported as cv) and the Go standard library (only math and math/cmplx). It uses no cgo and no third-party dependencies, and it does not import any of the other cv/* subpackages. Every detector operates on the package's central image type, cv.Mat (8-bit unsigned samples, one or three channels), and static detectors return a single-channel saliency map normalised to the 8-bit range, where brighter samples mark more salient locations.

OpenCV groups saliency algorithms into three families, all represented here.

Static saliency

Static detectors score a single still image and implement the common StaticSaliency interface (one ComputeSaliency method):

  • StaticSaliencySpectralResidual — the spectral-residual method of Hou & Zhang (2007). It works in the Fourier domain: the log-amplitude spectrum minus its own local average (the "spectral residual") is recombined with the original phase and inverse-transformed, so structure that departs from the smooth natural-image spectrum stands out. A small radix-2 2-D FFT is implemented locally in this package.
  • StaticSaliencyFineGrained — a multi-scale center-surround detector after Montabone & Soto (2010). Absolute pixel-versus-surround-mean differences are gathered over several octave scales with summed-area tables and averaged, filling the interior of sizeable salient regions rather than only their edges.
  • StaticSaliencyIttiKochNiebur — the classical bottom-up attention model of Itti, Koch & Niebur (1998): center-surround contrast on Gaussian pyramids of intensity, colour double-opponency and orientation, combined through the N(·) map-promotion operator.
  • MinimumBarrierSaliency — the Minimum Barrier Distance detector of Zhang et al. (ICCV 2015). Saliency is the barrier distance (path max minus min intensity) from each pixel to the image border, computed with fast alternating raster scans.
  • StaticSaliencyFrequencyTuned — the frequency-tuned method of Achanta et al. (CVPR 2009): the Lab distance between a lightly blurred pixel and the whole-image mean colour.
  • StaticSaliencyContextAware — the context-aware detector of Goferman et al. (CVPR 2010), scoring each pixel by its colour dissimilarity to its most similar (position-discounted) context.
  • GMRSaliency — the graph-based manifold-ranking detector of Yang et al. (CVPR 2013): a two-stage ranking of super-pixel regions against the image borders and then against foreground queries.
  • StaticSaliencyBooleanMap — Boolean Map Saliency (BMS) of Zhang & Sclaroff (ICCV 2013), activating the surrounded (border-disconnected) regions of many thresholded Boolean maps.
  • HistogramContrast (HC) and RegionContrast (RC) — the global-contrast detectors of Cheng et al. (CVPR 2011): colour-histogram contrast and spatially-weighted region contrast in Lab space.

ComputeBinaryMap turns any such saliency map into a binary foreground mask with Otsu thresholding; AdaptiveBinaryMap instead thresholds at a multiple of the map mean (Achanta's adaptive threshold). SaliencyToHeatmap renders a map as a jet-coloured image, and CenterBiasPrior/ApplyCenterBias model the human centre-fixation bias.

Evaluation

The standard saliency-benchmark metrics compare a predicted map against a human-fixation record or another map: AUCJudd (area under the ROC), NSS (normalized scanpath saliency), CC (linear correlation), SIM (histogram-intersection similarity) and KLDiv (Kullback-Leibler divergence).

Motion saliency

  • MotionSaliencyBinWangApr2014 — a stateful, per-pixel background-model detector after Wang & Dudek (2014). Fed a sequence of frames, it flags the pixels of moving objects. It is a compact, deterministic rendering of the two-model Bin-Wang scheme (single multi-template model, fixed template replacement).

Objectness

  • ObjectnessBING — a lightweight ("BING-lite") objectness proposer after Cheng et al. (2014). It keeps BING's binarised-normed-gradient front end but scores sliding windows with a fixed boundary-contrast heuristic instead of a learned linear model, returning ranked candidate boxes (ObjectnessBox) without any training data.
  • ObjectnessCascade — a two-stage proposer in the spirit of the BING cascade: a fast normed-gradient first stage, a richer second stage that re-scores survivors with saliency coverage, boundary contrast and a size prior, followed by non-maximum suppression of overlapping boxes.

Determinism

Every detector is fully deterministic: identical input produces identical output, with no randomised sampling. This makes results reproducible and unit-testable.

Relationship to OpenCV and deferred features

The algorithms follow their OpenCV counterparts closely enough to exhibit the same qualitative behaviour. Several detectors substitute regular grid regions for a learned colour segmentation or SLIC super-pixels (GMRSaliency, RegionContrast) and solve their linear systems iteratively; these approximations are documented on each type. The one capability that depends on trained model data is intentionally out of scope:

  • trained deep-learning saliency models (e.g. DeepGaze-style networks), and the offline-learned linear weights of the full BING detector.

ObjectnessBING and ObjectnessCascade therefore return heuristic objectness cues rather than the calibrated scores of the trained detector.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func AUCJudd added in v0.4.0 

func AUCJudd(salMap, fixation *cv.Mat) float64

AUCJudd computes the AUC-Judd area-under-ROC score of a saliency map against a binary fixation map (any non-zero sample marks a fixated pixel). The saliency map is treated as a binary classifier of fixated vs. non-fixated pixels swept over every fixation-valued threshold; the returned value is the area under the resulting ROC curve (0.5 is chance, 1.0 is perfect). It panics if the maps differ in size or are not single-channel, and returns NaN if there are no fixations. Both maps must be the same size.

Example

ExampleAUCJudd scores a saliency map against a binary fixation map. 

img := brightDisk()
sal := saliency.NewStaticSaliencyFineGrained().ComputeSaliency(img)
fix := cv.NewMat(48, 48, 1)
fix.Set(24, 24, 0, 255)
fix.Set(23, 24, 0, 255)
fix.Set(24, 23, 0, 255)
fmt.Println(saliency.AUCJudd(sal, fix) > 0.5)

Output:
true

func AdaptiveBinaryMap added in v0.4.0 

func AdaptiveBinaryMap(saliency *cv.Mat, factor float64) *cv.Mat

AdaptiveBinaryMap thresholds a single-channel saliency map at factor times its mean value (clamped to the 8-bit range), the adaptive threshold of Achanta et al. (CVPR 2009); pixels at or above the threshold become 255 and the rest 0. The customary factor is 2. It panics if saliency is nil, empty or not single-channel, or if factor is not positive.

Example

ExampleAdaptiveBinaryMap thresholds a saliency map at twice its mean. 

img := brightDisk()
sal := saliency.NewMinimumBarrierSaliency().ComputeSaliency(img)
mask := saliency.AdaptiveBinaryMap(sal, 2.0)
fmt.Println(mask.At(24, 24, 0), mask.At(0, 0, 0))

Output:
255 0

func ApplyCenterBias added in v0.4.0 

func ApplyCenterBias(saliency *cv.Mat, sigmaFrac float64) *cv.Mat

ApplyCenterBias multiplies a single-channel saliency map by a Gaussian center-prior (see CenterBiasPrior) and renormalises to the 8-bit range, returning a new map that suppresses off-centre responses. It panics if saliency is nil, empty or not single-channel.

func CC added in v0.4.0 

func CC(a, b *cv.Mat) float64

CC computes the linear (Pearson) correlation coefficient between two saliency maps, in [-1,1]. It is symmetric and invariant to affine rescaling of either map. It panics if the maps differ in size or are not single-channel, and returns NaN if either map is flat. Both maps must be the same size.

func CenterBiasPrior added in v0.4.0 

func CenterBiasPrior(rows, cols int, sigmaFrac float64) *cv.Mat

CenterBiasPrior returns a rows×cols single-channel Gaussian center-prior map (brightest at the image centre, falling off toward the edges), normalised to the 8-bit range. sigmaFrac sets the Gaussian's standard deviation as a fraction of the smaller image dimension (<=0 uses 0.35). Multiplying a saliency map by this prior models the well-known human tendency to fixate near image centres. It panics if either dimension is not positive.

Example

ExampleCenterBiasPrior builds a Gaussian centre-prior map. 

package main

import (
	"fmt"

	"github.com/malcolmston/opencv/saliency"
)

func main() {
	prior := saliency.CenterBiasPrior(32, 32, 0.3)
	fmt.Println(prior.At(16, 16, 0), prior.At(16, 16, 0) > prior.At(0, 0, 0))
}

Output:
255 true

func ComputeBinaryMap 

func ComputeBinaryMap(saliency *cv.Mat) *cv.Mat

ComputeBinaryMap converts a single-channel saliency map into a binary foreground mask (samples are 0 or 255) by thresholding it with Otsu's method, mirroring OpenCV's cv::saliency::StaticSaliency::computeBinaryMap. The threshold is chosen automatically from the map's histogram, so a saliency map with one distinct salient region yields a mask that isolates that region. It panics if saliency is nil or not single-channel.

Example

ExampleComputeBinaryMap turns a saliency map into a binary mask that isolates the salient object. 

package main

import (
	"fmt"

	cv "github.com/malcolmston/opencv"
	"github.com/malcolmston/opencv/saliency"
)

// brightDisk builds a small single-channel image with a flat dark background
// and a bright disk in the middle — a single distinct object.
func brightDisk() *cv.Mat {
	const size, cy, cx, r = 48, 24, 24, 6
	m := cv.NewMat(size, size, 1)
	m.SetTo(30)
	for y := 0; y < size; y++ {
		for x := 0; x < size; x++ {
			if (y-cy)*(y-cy)+(x-cx)*(x-cx) <= r*r {
				m.Set(y, x, 0, 220)
			}
		}
	}
	return m
}

func main() {
	img := brightDisk()
	sal := saliency.NewStaticSaliencyFineGrained().ComputeSaliency(img)
	mask := saliency.ComputeBinaryMap(sal)
	fmt.Println(mask.At(24, 24, 0), mask.At(0, 0, 0))
}

Output:
255 0

func KLDiv added in v0.4.0 

func KLDiv(prediction, groundTruth *cv.Mat) float64

KLDiv computes the Kullback-Leibler divergence of a predicted saliency map from a ground-truth fixation-density map, KL(groundTruth ‖ prediction). Both are normalised to probability distributions; lower is better (0 means the prediction matches the ground truth). It panics if the maps differ in size or are not single-channel, and returns NaN if the ground truth is all zero. Both maps must be the same size.

func NSS added in v0.4.0 

func NSS(salMap, fixation *cv.Mat) float64

NSS computes the Normalized Scanpath Saliency: the saliency map is normalised to zero mean and unit standard deviation, and NSS is the average of the normalised saliency at the fixated pixels (any non-zero sample of fixation). Positive values mean the map predicts fixations above chance. It panics if the maps differ in size or are not single-channel, and returns NaN when the map is flat or there are no fixations. Both maps must be the same size.

func SIM added in v0.4.0 

func SIM(a, b *cv.Mat) float64

SIM computes the similarity (histogram-intersection) metric between two saliency maps: each map is normalised to sum to one and SIM is the sum of the per-pixel minima, in [0,1] (1 means identical distributions). It panics if the maps differ in size or are not single-channel, and returns 0 if either map is all zero. Both maps must be the same size.

func SaliencyToHeatmap added in v0.4.0 

func SaliencyToHeatmap(saliency *cv.Mat) *cv.Mat

SaliencyToHeatmap renders a single-channel saliency map as a three-channel RGB pseudo-colour heatmap using a "jet"-style colormap (low saliency maps to blue, mid to green/yellow, high to red). It is a visualisation aid, analogous to applying cv::applyColorMap with COLORMAP_JET. It panics if saliency is nil, empty or not single-channel.

Example

ExampleSaliencyToHeatmap renders a saliency map as a jet-coloured image. 

package main

import (
	"fmt"

	cv "github.com/malcolmston/opencv"
	"github.com/malcolmston/opencv/saliency"
)

func main() {
	sal := cv.NewMat(2, 2, 1)
	sal.Set(0, 0, 0, 255)
	heat := saliency.SaliencyToHeatmap(sal)
	// Maximum saliency is rendered red (R>B).
	fmt.Println(heat.Channels, heat.At(0, 0, 0) > heat.At(0, 0, 2))
}

Output:
3 true

Types

type GMRSaliency added in v0.4.0 

type GMRSaliency struct {
	// Grid is the number of regions per side (Grid×Grid nodes). The default is
	// 10.
	Grid int
	// Alpha is the manifold-ranking balance parameter in (0,1). The default is
	// 0.99.
	Alpha float64
	// Sigma controls the colour affinity falloff (Lab normalised to [0,1]). The
	// default is 0.1.
	Sigma float64
	// Iterations is the number of Gauss-Seidel sweeps per solve. The default is
	// 60.
	Iterations int
}

GMRSaliency implements the Graph-based Manifold Ranking salient object detector of Yang, Zhang, Lu, Ruan & Yang, "Saliency Detection via Graph-Based Manifold Ranking" (CVPR 2013).

The image is partitioned into compact regions (super-pixels), each a graph node whose feature is its mean CIE L*a*b* colour and whose edges connect spatially adjacent regions and all image-border regions to one another. A closed-form manifold-ranking function propagates label information over this graph: 

f* = (D − αW)⁻¹ y

Stage one ranks every node against the four image borders in turn, treating each border as a set of background queries; the products of the four complementary (1 − rank) maps form a background-based saliency estimate. Stage two binarises that estimate to obtain foreground queries and re-runs the ranking, yielding the final map. Regions that rank far from the borders — a distinct central object — end up bright.

Regular grid regions stand in for SLIC super-pixels and the linear system is solved by Gauss-Seidel iteration rather than explicit inversion, keeping the detector dependency-free and deterministic while preserving the two-stage ranking behaviour.

Construct one with NewGMRSaliency. It satisfies StaticSaliency.

Example

ExampleGMRSaliency ranks image regions against the border to find the salient object. 

img := brightDisk()
sal := saliency.NewGMRSaliency().ComputeSaliency(img)
fmt.Println(sal.At(24, 24, 0) > sal.At(0, 0, 0))

Output:
true

func NewGMRSaliency added in v0.4.0 

func NewGMRSaliency() *GMRSaliency

NewGMRSaliency returns a detector with a 10×10 region grid and the paper's default ranking parameters.

func (*GMRSaliency) ComputeSaliency added in v0.4.0 

func (s *GMRSaliency) ComputeSaliency(img *cv.Mat) *cv.Mat

ComputeSaliency returns the manifold-ranking saliency map of img: a single-channel cv.Mat the same size as img, normalised to [0,255]. It panics if img is nil or empty.

type HistogramContrast added in v0.4.0 

type HistogramContrast struct {
	// Bins is the number of quantisation levels per channel (Bins³ palette
	// entries). The default is 12.
	Bins int
}

HistogramContrast implements the histogram-based global-contrast salient region detector (HC) of Cheng, Mitra, Huang, Torr & Hu, "Global Contrast based Salient Region Detection" (CVPR 2011).

Pixel colours are quantised into a small palette. A colour's saliency is the sum, over all other palette colours, of that colour's population times its Lab-space distance from the query colour: 

S(c) = Σ_j n_j · ‖c − c_j‖

so colours that are both rare and far from the bulk of the image (a distinct object against a dominant background) score highest. The per-colour saliency is smoothed across nearby palette colours to avoid quantisation artefacts and mapped back to the pixels.

Construct one with NewHistogramContrast. It satisfies StaticSaliency.

Example

ExampleHistogramContrast scores rare, contrasting colours as salient. 

img := brightDisk()
sal := saliency.NewHistogramContrast().ComputeSaliency(img)
fmt.Println(sal.At(24, 24, 0) > sal.At(0, 0, 0))

Output:
true

func NewHistogramContrast added in v0.4.0 

func NewHistogramContrast() *HistogramContrast

NewHistogramContrast returns a detector with 12 quantisation levels per channel.

func (*HistogramContrast) ComputeSaliency added in v0.4.0 

func (h *HistogramContrast) ComputeSaliency(img *cv.Mat) *cv.Mat

ComputeSaliency returns the histogram-contrast saliency map of img: a single-channel cv.Mat the same size as img, normalised to [0,255]. It panics if img is nil or empty.

type MinimumBarrierSaliency added in v0.4.0 

type MinimumBarrierSaliency struct {
	// Passes is the number of raster-scan sweeps (each pass alternates
	// direction). More passes tighten the approximation; the default is 3.
	Passes int
	// BlurRadius is the radius of a final box smoothing applied to the distance
	// field (0 disables it). The default is 3.
	BlurRadius int
}

MinimumBarrierSaliency implements the Minimum Barrier Distance (MBD) salient object detector of Zhang, Sclaroff, Lin, Shen, Price & Mech, "Minimum Barrier Salient Object Detection at 80 FPS" (ICCV 2015).

Saliency is the minimum barrier distance from every pixel to the image boundary, where the barrier cost of a path is the difference between the highest and lowest intensity encountered along it (max minus min), not the sum of gradients. Because the image border is treated as a background seed set, pixels that can only be reached by crossing a strong intensity ridge — the interior of a distinct object — receive a large distance and therefore a high saliency, while background connected smoothly to the border stays dark.

The distance field is computed with the paper's fast approximation: a small fixed number of alternating raster-scan (forward/backward) passes that propagate, for each pixel, the lowest barrier cost found so far together with the running max and min of the corresponding path.

Construct one with NewMinimumBarrierSaliency. It satisfies StaticSaliency.

Example

ExampleMinimumBarrierSaliency scores the interior of an object by its minimum barrier distance to the image border. 

img := brightDisk()
sal := saliency.NewMinimumBarrierSaliency().ComputeSaliency(img)
fmt.Println(sal.At(24, 24, 0) > sal.At(0, 0, 0))

Output:
true

func NewMinimumBarrierSaliency added in v0.4.0 

func NewMinimumBarrierSaliency() *MinimumBarrierSaliency

NewMinimumBarrierSaliency returns a detector with three scan passes and a smoothing radius of three.

func (*MinimumBarrierSaliency) ComputeSaliency added in v0.4.0 

func (m *MinimumBarrierSaliency) ComputeSaliency(img *cv.Mat) *cv.Mat

ComputeSaliency returns the minimum-barrier saliency map of img: a single-channel cv.Mat the same size as img, normalised to [0,255]. It panics if img is nil or empty.

type MotionSaliencyBinWangApr2014 

type MotionSaliencyBinWangApr2014 struct {
	// NumTemplates is the number of background samples kept per pixel.
	NumTemplates int
	// Threshold is the maximum absolute intensity difference (0–255) at which a
	// pixel is still considered to match a background template.
	Threshold float64
	// MinMatches is how many templates a pixel must match to count as
	// background.
	MinMatches int
	// LearningRate blends a matched background pixel into its closest template
	// (0 keeps the model frozen, 1 replaces it outright).
	LearningRate float64
	// contains filtered or unexported fields
}

MotionSaliencyBinWangApr2014 detects motion saliency — pixels belonging to moving objects — with a per-pixel background model built up over a sequence of frames, after Wang & Dudek, "A Fast Self-tuning Background Subtraction Algorithm" (CVPR Workshops, April 2014), the method behind OpenCV's cv::saliency::MotionSaliencyBinWangApr2014.

Each pixel keeps a small set of background template samples. A new frame's pixel is classified as background when it matches enough templates within an intensity tolerance, and as foreground (moving) otherwise. Matched templates are nudged toward the observed value so the model tracks slow illumination drift; unmatched pixels receive a slow blind update so that genuinely permanent scene changes are eventually absorbed. This is a compact, deterministic rendering of the two-model Bin-Wang scheme: it uses a single multi-template model and fixed (rather than randomised) template replacement, which keeps results reproducible.

The detector is stateful and processes frames in order. Create one with NewMotionSaliencyBinWangApr2014 and call MotionSaliencyBinWangApr2014.ComputeSaliency once per frame. The first frame seeds the model and returns an all-zero map.

Example

ExampleMotionSaliencyBinWangApr2014 learns a static background from two frames, then flags a blob that appears in the third. 

package main

import (
	"fmt"

	cv "github.com/malcolmston/opencv"
	"github.com/malcolmston/opencv/saliency"
)

func main() {
	const size = 24
	det := saliency.NewMotionSaliencyBinWangApr2014(size, size)

	background := cv.NewMat(size, size, 1)
	background.SetTo(50)
	det.ComputeSaliency(background)         // seed
	det.ComputeSaliency(background.Clone()) // confirm background

	frame := background.Clone()
	for y := 8; y < 16; y++ {
		for x := 8; x < 16; x++ {
			frame.Set(y, x, 0, 200)
		}
	}
	motion := det.ComputeSaliency(frame)
	fmt.Println(motion.At(11, 11, 0), motion.At(0, 0, 0))
}

Output:
255 0

func NewMotionSaliencyBinWangApr2014 

func NewMotionSaliencyBinWangApr2014(rows, cols int) *MotionSaliencyBinWangApr2014

NewMotionSaliencyBinWangApr2014 returns a detector sized for rows×cols frames with sensible defaults (four templates, an intensity tolerance of 40 and a 0.1 learning rate). It panics if either dimension is not positive.

func (*MotionSaliencyBinWangApr2014) ComputeSaliency 

func (m *MotionSaliencyBinWangApr2014) ComputeSaliency(img *cv.Mat) *cv.Mat

ComputeSaliency returns the binary motion-saliency map for one frame: a single-channel cv.Mat the same size as the configured frame size, with moving pixels set to 255 and background to 0. The frame must match the size passed to the constructor. The very first frame seeds the background model and yields an all-zero map. It panics on a size mismatch or empty frame.

func (*MotionSaliencyBinWangApr2014) Reset 

func (m *MotionSaliencyBinWangApr2014) Reset()

Reset discards the learned background model so the next frame re-seeds it.

type ObjectnessBING 

type ObjectnessBING struct {
	// WindowSizes are the side lengths (in pixels) of the square sliding
	// windows evaluated at every position. Windows larger than the image are
	// skipped.
	WindowSizes []int
	// MaxProposals caps the number of boxes returned (0 means no cap).
	MaxProposals int
}

ObjectnessBING is a lightweight ("BING-lite") objectness detector inspired by Cheng et al., "BING: Binarized Normed Gradients for Objectness Estimation at 300fps" (CVPR 2014), the basis of OpenCV's cv::saliency::ObjectnessBING.

The full BING method scores 8×8 binarised normed-gradient (NG) windows with a linear model whose weights are learned offline. This port keeps the normed-gradient front end but replaces the learned classifier with a fixed heuristic: generic objects are gradient-dense regions surrounded by flatter background, so a window is scored by how much stronger the mean normed gradient is inside it than in a surrounding ring. No training data or weights are required, which is why it is a "lite" variant; the trade-off is that scores are relative cues rather than calibrated probabilities.

Construct one with NewObjectnessBING and call ObjectnessBING.ComputeObjectness to obtain ranked proposals.

Example

ExampleObjectnessBING proposes candidate object windows ranked by objectness. 

package main

import (
	"fmt"

	cv "github.com/malcolmston/opencv"
	"github.com/malcolmston/opencv/saliency"
)

// brightDisk builds a small single-channel image with a flat dark background
// and a bright disk in the middle — a single distinct object.
func brightDisk() *cv.Mat {
	const size, cy, cx, r = 48, 24, 24, 6
	m := cv.NewMat(size, size, 1)
	m.SetTo(30)
	for y := 0; y < size; y++ {
		for x := 0; x < size; x++ {
			if (y-cy)*(y-cy)+(x-cx)*(x-cx) <= r*r {
				m.Set(y, x, 0, 220)
			}
		}
	}
	return m
}

func main() {
	img := brightDisk()
	boxes := saliency.NewObjectnessBING().ComputeObjectness(img)
	top := boxes[0]
	overlaps := 24 >= top.X && 24 < top.X+top.W && 24 >= top.Y && 24 < top.Y+top.H
	fmt.Println(len(boxes) > 0, overlaps)
}

Output:
true true

func NewObjectnessBING 

func NewObjectnessBING() *ObjectnessBING

NewObjectnessBING returns a detector with a default range of window sizes and a cap of 64 proposals.

func (*ObjectnessBING) ComputeObjectness 

func (o *ObjectnessBING) ComputeObjectness(img *cv.Mat) []ObjectnessBox

ComputeObjectness returns candidate object windows for img ranked by objectness score (highest first). It panics if img is nil or empty.

type ObjectnessBox 

type ObjectnessBox struct {
	X, Y, W, H int
	Score      float64
}

ObjectnessBox is a candidate object window returned by ObjectnessBING. X, Y are the top-left corner and W, H the size, all in pixels; Score is the objectness measure, higher meaning more object-like. Boxes are returned in descending score order.

type ObjectnessCascade added in v0.4.0 

type ObjectnessCascade struct {
	// WindowSizes are the square window side lengths evaluated in stage one.
	WindowSizes []int
	// StageOneKeep caps how many stage-one windows advance to stage two.
	StageOneKeep int
	// MaxProposals caps the number of boxes returned after suppression (0 means
	// no cap).
	MaxProposals int
	// NMSThreshold is the intersection-over-union above which the lower-scored
	// of two boxes is suppressed.
	NMSThreshold float64
}

ObjectnessCascade is a two-stage objectness proposer in the spirit of the BING cascade of Cheng et al. (CVPR 2014): a fast first stage floods the image with cheaply-scored sliding windows, and a slower second stage re-scores only the survivors with richer cues and suppresses overlaps.

Stage one scores every window by normed-gradient boundary contrast (the mean normed gradient inside the window minus that of a surrounding ring), exactly the ObjectnessBING cue, and keeps the top StageOneKeep windows. Stage two re-ranks each survivor with a combined objectness score:

  • boundary contrast (does the window edge sit on strong gradients?);
  • saliency coverage (how much of a frequency-tuned saliency map the window captures relative to its area — favouring tight windows on the object); and
  • a mild size prior.

Greedy non-maximum suppression by intersection-over-union then removes near-duplicate boxes, yielding a compact ranked proposal set. Because the second stage is a fixed heuristic rather than a trained SVM, scores are relative cues, not calibrated probabilities.

Construct one with NewObjectnessCascade.

Example

ExampleObjectnessCascade proposes ranked object windows with a two-stage scorer and non-maximum suppression. 

img := brightDisk()
boxes := saliency.NewObjectnessCascade().ComputeObjectness(img)
top := boxes[0]
overlaps := 24 >= top.X && 24 < top.X+top.W && 24 >= top.Y && 24 < top.Y+top.H
fmt.Println(len(boxes) > 0, overlaps)

Output:
true true

func NewObjectnessCascade added in v0.4.0 

func NewObjectnessCascade() *ObjectnessCascade

NewObjectnessCascade returns a cascade with a default range of window sizes, 256 stage-one survivors, an IoU suppression threshold of 0.5 and a cap of 32 proposals.

func (*ObjectnessCascade) ComputeObjectness added in v0.4.0 

func (o *ObjectnessCascade) ComputeObjectness(img *cv.Mat) []ObjectnessBox

ComputeObjectness returns candidate object windows for img ranked by objectness score (highest first) after two-stage scoring and non-maximum suppression. It panics if img is nil or empty.

type RegionContrast added in v0.4.0 

type RegionContrast struct {
	// Grid is the number of regions per side (Grid×Grid regions). The default
	// is 12.
	Grid int
	// SpatialSigma is the spatial-distance falloff, as a fraction of the image
	// diagonal. The default is 0.4.
	SpatialSigma float64
}

RegionContrast implements the region-based global-contrast salient region detector (RC) of Cheng, Mitra, Huang, Torr & Hu, "Global Contrast based Salient Region Detection" (CVPR 2011).

The image is divided into regions; each region's saliency is the sum over all other regions of a spatial-distance weight times the other region's pixel count times the Lab colour distance between the two regions: 

S(r_k) = Σ_{i≠k} exp(−D_s(r_k,r_i)/σ²) · w(r_i) · D_c(r_k,r_i)

The spatial term concentrates contrast contributions from nearby regions, so a compact object surrounded by a large uniform background — which contributes both high colour distance and high pixel weight — is highlighted while the background regions, similar to one another, stay dark.

Regular grid regions stand in for a colour segmentation, keeping the detector deterministic; the global-contrast weighting is otherwise faithful to the original. Construct one with NewRegionContrast. It satisfies StaticSaliency.

Example

ExampleRegionContrast measures global region contrast weighted by spatial distance. 

img := brightDisk()
sal := saliency.NewRegionContrast().ComputeSaliency(img)
fmt.Println(sal.At(24, 24, 0) > sal.At(0, 0, 0))

Output:
true

func NewRegionContrast added in v0.4.0 

func NewRegionContrast() *RegionContrast

NewRegionContrast returns a detector with a 12×12 region grid.

func (*RegionContrast) ComputeSaliency added in v0.4.0 

func (rc *RegionContrast) ComputeSaliency(img *cv.Mat) *cv.Mat

ComputeSaliency returns the region-contrast saliency map of img: a single-channel cv.Mat the same size as img, normalised to [0,255]. It panics if img is nil or empty.

type StaticSaliency 

type StaticSaliency interface {
	// ComputeSaliency returns a single-channel saliency map, the same size as
	// img, normalised to the 8-bit range.
	ComputeSaliency(img *cv.Mat) *cv.Mat
}

StaticSaliency is implemented by the static (single-image) saliency detectors. It mirrors OpenCV's cv::saliency::StaticSaliency: a detector consumes one image and produces a single-channel saliency map in which brighter samples mark more visually salient locations.

type StaticSaliencyBooleanMap added in v0.4.0 

type StaticSaliencyBooleanMap struct {
	// Thresholds is the number of evenly spaced threshold levels swept across
	// each channel's [0,255] range. The default is 8.
	Thresholds int
	// BlurRadius smooths the final averaged attention map (0 disables it). The
	// default is 3.
	BlurRadius int
}

StaticSaliencyBooleanMap implements Boolean Map based Saliency (BMS) after Zhang & Sclaroff, "Saliency Detection: A Boolean Map Approach" (ICCV 2013).

The method rests on the Gestalt principle of surroundedness: figures tend to be enclosed regions. Each colour/opponency channel is thresholded at a range of levels to produce a stack of Boolean maps. For every Boolean map an attention map is formed by activating the connected regions that do NOT touch the image border (they are surrounded by the complementary value); this is done for both the map and its inverse. Each attention map is normalised by its own magnitude so that many small surrounded blobs cannot outvote one large one, and the maps are averaged and blurred into the final saliency map.

A distinct object sitting away from the border is enclosed at most threshold levels, so it is repeatedly activated and ends up bright.

Construct one with NewStaticSaliencyBooleanMap. It satisfies StaticSaliency.

Example

ExampleStaticSaliencyBooleanMap activates surrounded regions across many Boolean maps. 

img := brightDisk()
sal := saliency.NewStaticSaliencyBooleanMap().ComputeSaliency(img)
fmt.Println(sal.At(24, 24, 0) > sal.At(0, 0, 0))

Output:
true

func NewStaticSaliencyBooleanMap added in v0.4.0 

func NewStaticSaliencyBooleanMap() *StaticSaliencyBooleanMap

NewStaticSaliencyBooleanMap returns a detector with eight threshold levels.

func (*StaticSaliencyBooleanMap) ComputeSaliency added in v0.4.0 

func (s *StaticSaliencyBooleanMap) ComputeSaliency(img *cv.Mat) *cv.Mat

ComputeSaliency returns the Boolean-map saliency of img: a single-channel cv.Mat the same size as img, normalised to [0,255]. It panics if img is nil or empty.

type StaticSaliencyContextAware added in v0.4.0 

type StaticSaliencyContextAware struct {
	// WorkingSize is the side length the image is resampled to before the
	// pairwise comparison (kept small because the cost is quadratic in pixels).
	// The default is 48.
	WorkingSize int
	// K is how many most-similar references contribute to each pixel's score.
	// The default is 32.
	K int
	// PositionWeight controls how strongly spatial proximity discounts colour
	// dissimilarity (larger means distance matters more). The default is 3.
	PositionWeight float64
}

StaticSaliencyContextAware implements a context-aware saliency detector after Goferman, Zelnik-Manor & Tal, "Context-Aware Saliency Detection" (CVPR 2010).

A region is salient when it is dissimilar in colour to the regions most like it, and that dissimilarity is discounted when the similar regions are nearby. For every pixel the detector measures a colour dissimilarity to a grid of reference locations, weighted down by spatial distance, keeps the K smallest such (colour) dissimilarities — the pixel's most similar context — and maps their mean d through S = 1 − exp(−d). Pixels whose closest matches are still far in colour (a unique object) score high; repetitive background scores low.

The reference set is a fixed sub-sampled grid rather than an exhaustive nearest-patch search over multiple scales, which keeps the detector fast and deterministic; the qualitative single-scale behaviour matches the original.

Construct one with NewStaticSaliencyContextAware. It satisfies StaticSaliency.

Example

ExampleStaticSaliencyContextAware highlights colour-unique regions. 

img := brightDisk()
sal := saliency.NewStaticSaliencyContextAware().ComputeSaliency(img)
fmt.Println(sal.At(24, 24, 0) > sal.At(0, 0, 0))

Output:
true

func NewStaticSaliencyContextAware added in v0.4.0 

func NewStaticSaliencyContextAware() *StaticSaliencyContextAware

NewStaticSaliencyContextAware returns a detector with a 48×48 working size and K=32 similar references.

func (*StaticSaliencyContextAware) ComputeSaliency added in v0.4.0 

func (s *StaticSaliencyContextAware) ComputeSaliency(img *cv.Mat) *cv.Mat

ComputeSaliency returns the context-aware saliency map of img: a single-channel cv.Mat the same size as img, normalised to [0,255]. It panics if img is nil or empty.

type StaticSaliencyFineGrained 

type StaticSaliencyFineGrained struct {
	// Scales is the number of center-surround octaves. The surround window
	// radius doubles each octave (1, 2, 4, … pixels). The OpenCV default is 6.
	Scales int
}

StaticSaliencyFineGrained detects saliency with a multi-scale center-surround scheme in the spirit of Montabone & Soto, "Human detection using a mobile platform and novel features derived from a visual saliency mechanism" (Image and Vision Computing 2010), the algorithm behind OpenCV's cv::saliency::StaticSaliencyFineGrained.

The image is reduced to grayscale, and at each of several octave scales the absolute difference between every pixel and the mean of the surrounding window (an on/off center-surround response) is measured with the help of a summed-area table. Small scales react to fine detail while large scales fill in the interior of sizeable salient regions; the per-scale responses are individually normalised and averaged, so a bright object on a flat background lights up as a whole rather than only along its edges.

Construct one with NewStaticSaliencyFineGrained.

Example

ExampleStaticSaliencyFineGrained detects the same object with the fine-grained center-surround method. 

package main

import (
	"fmt"

	cv "github.com/malcolmston/opencv"
	"github.com/malcolmston/opencv/saliency"
)

// brightDisk builds a small single-channel image with a flat dark background
// and a bright disk in the middle — a single distinct object.
func brightDisk() *cv.Mat {
	const size, cy, cx, r = 48, 24, 24, 6
	m := cv.NewMat(size, size, 1)
	m.SetTo(30)
	for y := 0; y < size; y++ {
		for x := 0; x < size; x++ {
			if (y-cy)*(y-cy)+(x-cx)*(x-cx) <= r*r {
				m.Set(y, x, 0, 220)
			}
		}
	}
	return m
}

func main() {
	img := brightDisk()
	sal := saliency.NewStaticSaliencyFineGrained().ComputeSaliency(img)
	center := sal.At(24, 24, 0)
	corner := sal.At(0, 0, 0)
	fmt.Println(center > corner)
}

Output:
true

func NewStaticSaliencyFineGrained 

func NewStaticSaliencyFineGrained() *StaticSaliencyFineGrained

NewStaticSaliencyFineGrained returns a detector configured with the OpenCV default of six scales.

func (*StaticSaliencyFineGrained) ComputeSaliency 

func (f *StaticSaliencyFineGrained) ComputeSaliency(img *cv.Mat) *cv.Mat

ComputeSaliency returns the fine-grained saliency map of img, a single-channel cv.Mat the same size as img and normalised to [0,255]. It panics if img is nil or empty.

type StaticSaliencyFrequencyTuned added in v0.4.0 

type StaticSaliencyFrequencyTuned struct {
	// BlurKSize is the size of the small separable Gaussian applied before the
	// distance is measured. The default is 5.
	BlurKSize int
	// BlurSigma is that Gaussian's standard deviation (<=0 derives it from the
	// kernel size). The default is 1.5.
	BlurSigma float64
}

StaticSaliencyFrequencyTuned implements the frequency-tuned salient region detector of Achanta, Hemami, Estrada & Süsstrunk, "Frequency-tuned Salient Region Detection" (CVPR 2009).

The image is converted to CIE L*a*b* colour. Saliency at a pixel is the squared Euclidean distance, in Lab, between that pixel's slightly Gaussian- smoothed colour and the arithmetic mean Lab colour of the whole image: 

S(x) = ‖ I_μ − I_ωhc(x) ‖²

The whole-image mean removes the very lowest spatial frequencies (large flat regions) while the small blur removes the very highest (pixel noise and fine texture), leaving well-defined, uniformly highlighted salient objects with crisp boundaries. For a single distinct object the mean colour tracks the dominant background, so the object stands out strongly.

Construct one with NewStaticSaliencyFrequencyTuned. It satisfies StaticSaliency.

Example

ExampleStaticSaliencyFrequencyTuned detects an object via its Lab-colour deviation from the image mean. 

img := brightDisk()
sal := saliency.NewStaticSaliencyFrequencyTuned().ComputeSaliency(img)
fmt.Println(sal.At(24, 24, 0) > sal.At(0, 0, 0))

Output:
true

func NewStaticSaliencyFrequencyTuned added in v0.4.0 

func NewStaticSaliencyFrequencyTuned() *StaticSaliencyFrequencyTuned

NewStaticSaliencyFrequencyTuned returns a detector with a 5×5 pre-blur.

func (*StaticSaliencyFrequencyTuned) ComputeSaliency added in v0.4.0 

func (s *StaticSaliencyFrequencyTuned) ComputeSaliency(img *cv.Mat) *cv.Mat

ComputeSaliency returns the frequency-tuned saliency map of img: a single-channel cv.Mat the same size as img, normalised to [0,255]. It panics if img is nil or empty.

type StaticSaliencyIttiKochNiebur added in v0.4.0 

type StaticSaliencyIttiKochNiebur struct {
	// PyramidLevels is the number of dyadic pyramid levels built for every
	// feature. The default is 7.
	PyramidLevels int
	// CenterLevels are the fine pyramid levels used as center scales.
	CenterLevels []int
	// SurroundDeltas are the level offsets (added to each center) used as
	// surround scales.
	SurroundDeltas []int
}

StaticSaliencyIttiKochNiebur implements the classical bottom-up visual attention model of Itti, Koch & Niebur, "A Model of Saliency-Based Visual Attention for Rapid Scene Analysis" (IEEE TPAMI 1998) — the reference biological saliency architecture that predates and inspired OpenCV's static detectors.

The image is decomposed into three early-vision feature families:

  • intensity (I = (R+G+B)/3);
  • colour double-opponency (red-green and blue-yellow); and
  • local orientation energy (four oriented edge kernels).

Each feature is built into a dyadic Gaussian pyramid, and center-surround contrast is measured as the across-scale absolute difference between a fine "center" level and a coarser "surround" level (each surround is upsampled to the center's size before subtraction). The maps are combined with Itti's normalisation operator N(·) — normalise to a fixed range, then multiply by (1-mean)² so that a map with a few strong peaks is promoted over one with many comparable responses — accumulated into per-feature conspicuity maps, averaged and resized back to the input resolution.

Orientation is approximated with four fixed oriented kernels rather than a full bank of Gabor filters, which keeps the detector dependency-free while preserving the qualitative center-surround behaviour.

Construct one with NewStaticSaliencyIttiKochNiebur. It satisfies StaticSaliency.

Example

ExampleStaticSaliencyIttiKochNiebur highlights a bright object with the classical Itti-Koch-Niebur attention model. 

img := brightDisk()
sal := saliency.NewStaticSaliencyIttiKochNiebur().ComputeSaliency(img)
fmt.Println(sal.At(24, 24, 0) > sal.At(0, 0, 0))

Output:
true

func NewStaticSaliencyIttiKochNiebur added in v0.4.0 

func NewStaticSaliencyIttiKochNiebur() *StaticSaliencyIttiKochNiebur

NewStaticSaliencyIttiKochNiebur returns a detector configured with the classical center scales {2,3}, surround deltas {2,3} and a seven-level pyramid.

func (*StaticSaliencyIttiKochNiebur) ComputeSaliency added in v0.4.0 

func (s *StaticSaliencyIttiKochNiebur) ComputeSaliency(img *cv.Mat) *cv.Mat

ComputeSaliency returns the Itti-Koch-Niebur saliency map of img: a single-channel cv.Mat the same size as img, normalised to [0,255]. It panics if img is nil or empty.

type StaticSaliencySpectralResidual 

type StaticSaliencySpectralResidual struct {
	// ResizedWidth and ResizedHeight are the working dimensions the image is
	// resampled to before the transform. They are rounded up to a power of two
	// internally so the radix-2 FFT applies. Smaller sizes emphasise coarse,
	// large-scale saliency; the OpenCV default is 64×64.
	ResizedWidth  int
	ResizedHeight int
}

StaticSaliencySpectralResidual detects saliency with the spectral-residual method of Hou & Zhang, "Saliency Detection: A Spectral Residual Approach" (CVPR 2007), the same algorithm as OpenCV's cv::saliency::StaticSaliencySpectralResidual.

The image is reduced to grayscale and resampled to a small working size (ResizedWidth×ResizedHeight). Its 2-D Fourier transform is split into a log-amplitude spectrum and a phase spectrum. The spectral residual — the log-amplitude minus its own local average — captures the parts of the spectrum that deviate from the smooth, statistically-expected 1/f falloff and therefore correspond to novel, salient structure. Recombining the residual amplitude with the original phase and inverse-transforming yields a saliency map, which is squared, blurred and resized back to the input dimensions.

Construct one with NewStaticSaliencySpectralResidual. The zero value is not usable; use the constructor so the working size is set.

Example

ExampleStaticSaliencySpectralResidual detects a bright object with the spectral-residual method and confirms it is more salient than the background. 

package main

import (
	"fmt"

	cv "github.com/malcolmston/opencv"
	"github.com/malcolmston/opencv/saliency"
)

// brightDisk builds a small single-channel image with a flat dark background
// and a bright disk in the middle — a single distinct object.
func brightDisk() *cv.Mat {
	const size, cy, cx, r = 48, 24, 24, 6
	m := cv.NewMat(size, size, 1)
	m.SetTo(30)
	for y := 0; y < size; y++ {
		for x := 0; x < size; x++ {
			if (y-cy)*(y-cy)+(x-cx)*(x-cx) <= r*r {
				m.Set(y, x, 0, 220)
			}
		}
	}
	return m
}

func main() {
	img := brightDisk()
	sal := saliency.NewStaticSaliencySpectralResidual().ComputeSaliency(img)
	center := sal.At(24, 24, 0)
	corner := sal.At(0, 0, 0)
	fmt.Println(center > corner)
}

Output:
true

func NewStaticSaliencySpectralResidual 

func NewStaticSaliencySpectralResidual() *StaticSaliencySpectralResidual

NewStaticSaliencySpectralResidual returns a detector configured with the OpenCV default 64×64 working resolution.

func (*StaticSaliencySpectralResidual) ComputeSaliency 

func (s *StaticSaliencySpectralResidual) ComputeSaliency(img *cv.Mat) *cv.Mat

ComputeSaliency returns the spectral-residual saliency map of img, a single-channel cv.Mat the same size as img and normalised to [0,255]. It panics if img is nil or empty.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.