locateimage

README

locateimage (Go)

Performs an (exact or fuzzy) search of a sample image within a larger image, returning the coordinates and similarity scores of the matches.

Installation

This package contains a Go module, so in Go 1.11+ module mode you can import it directly. Otherwise do:

go get -u github.com/andreyvit/locateimage

Example

See godoc.org/github.com/andreyvit/locateimage for a full reference.

All locates all matches:

mm, err := locateimage.All(context.Background(), canvas, sample, 0.04)
if err != nil {
    log.Fatal(err)
}
log.Print(mm)

Find locates a single match:

m, err := locateimage.Find(context.Background(), canvas, sample, 0, locateimage.Fastest)
if err != nil {
    log.Print(err)
} else {
    log.Printf("sample found at %v, similarity = %.*f%%", m.Rect, locateimage.SimilarityDigits-2, 100*m.Similarity)
}

Find returns ErrNotFound if no match is found. Available selection modes are Fastest, Best and Only.

Foreach invokes a callback for each match:

err = locateimage.Foreach(context.Background(), canvas, sample, 0.04, func(m locateimage.Match) error {
    log.Print(m)
    return nil
})
if err != nil {
    log.Fatal(err)
}

Caveats

The package currently only deals with image.RGBA-encoded images. You can use Convert to convert any image to this format. (Note that reading a PNG file returns an NRGBA format, so the conversion will be required. It makes sense to add support for formats like NRGBA in the future.)

The search is currently slow, taking tens or hundreds of milliseconds on large images (screenshots of a 27" screen). This can likely be improved, and contributions are welcome.

Versions

None currently.

Documentation

Overview

Package locateimage performs an (exact or fuzzy) search of a sample image within a larger image, returning the coordinates and similarity scores of the matches.

The package currently only deals with image.RGBA-encoded images. You can use Convert to convert any image to this format. (Note that reading a PNG file returns an NRGBA format, so the conversion will be required. It makes sense to add support for formats like NRGBA in the future.)

The search is currently slow, taking tens or hundreds of milliseconds on large images (screenshots of a 27" screen). This can likely be improved, and contributions are welcome.

Index

• Constants
• Variables
• func Convert(src image.Image) image.Image
• func Foreach(ctx context.Context, canvas, sample image.Image, tolerance float64, ...) error
• func Sort(mm []Match)
• type Match
  • func All(ctx context.Context, canvas, sample image.Image, tolerance float64) ([]Match, error)
  • func Find(ctx context.Context, canvas, sample image.Image, tolerance float64, ...) (Match, error)
  • func (a Match) Before(b Match) bool
  • func (m Match) String() string
• type Selection
  • func (s Selection) String() string

Examples

• Find

Constants

const (
	// SimilarityDigits is the number of fractional digits to be taken into account
	// in similarity and tolerance values. Use this in %.*f when printing.
	SimilarityDigits = 6

	// SimilarityPrecision is 10^(-SimilarityDigits), the minimal difference in
	// the value of similarity or tolerance that should be considered significant.
	SimilarityPrecision float64 = 0.000001
)

const (
	// Fastest asks for whatever match is encountered first, in undefined order.
	Fastest = Selection(iota)

	// Best asks for the match with the best similarity score.
	Best

	// Only asks for the best match like Best mode, but asks to verify that
	// only a single match exists. If multiple matches are found,
	// ErrMultipleFound will be returned together with the best match.
	Only
)

Variables

var (
	// ErrBreak is a convenient error you can use to break out of Foreach loop.
	// There is no special behavior associated with this error; it will be
	// returned from Foreach normally.
	ErrBreak = errors.New("done iterating")

	// ErrUnsupportedImageType is returned when trying to process an image outside
	// of the supported image types, which currently is only image.RGBA.
	ErrUnsupportedImageType = errors.New("unsupported image type")

	// ErrNotFound is returned by Find when no match has been found.
	ErrNotFound = errors.New("not found")

	// ErrMultipleFound is returned by Find in Only selection mode if more than
	// one match has been found.
	ErrMultipleFound = errors.New("multiple matches found")
)

Functions

func Convert

func Convert(src image.Image) image.Image

Convert converts any image to an image format supported by this package. If the image is already in a supported format, it is returned as is.

Currently Convert always converts to image.RGBA.

func Foreach

func Foreach(ctx context.Context, canvas, sample image.Image, tolerance float64, f func(m Match) error) error

Foreach locates matches of the sample image within a canvas image, and invokes the provided callback function for each match.

Tolerance is a value between 0 and 1 specifying how much difference is tolerated between the sample and its match. Pass 0 to find exact matches only. A reasonable value for fuzzy matching is around 0.05.

Searching stops if the callback returns an error. You can use the convenient predefined ErrBreak from this package, or return any other error. Searching also stops if the context is canceled or expired (i.e. the Err() method of the context returns a non-nil value).

Foreach can return one of the following errors:

- ErrUnsupportedImageType when one of the images is not image.RGBA

- any error returned by your callback

- any error returned by ctx.Err()

func Sort

func Sort(mm []Match)

Sort sorts the list of matches in the order established by Match.Before.

Higher-similarity matches come before lower-similarity matches, and equally-similar matches are ordered by Y, then by X, ascending.

Types

type Match

type Match struct {
	// Rect is the part of the canvas that matched the sample. Its size is
	// equal to the size of the sample.
	Rect image.Rectangle

	// Similarity is a score from 0 (completely dissimilar) to 1 (exact match).
	// A returned match will have a similarity of at least (1 - tolerance).
	Similarity float64
}

Match describes a single match of a sample image within a canvas.

func All

func All(ctx context.Context, canvas, sample image.Image, tolerance float64) ([]Match, error)

All locates and returns all matches of the sample image within a canvas image. The returned matches are sorted using the Sort function of this package.

Tolerance is a value between 0 and 1 specifying how much difference is tolerated between the sample and its match. Pass 0 to find exact matches only. A reasonable value for fuzzy matching is around 0.05.

Searching stops if the context is canceled or expired (i.e. the Err() method of the context returns a non-nil value). In this case, the function returns all matches found so far, together with a non-nil error.

This function can return one of the following errors:

- ErrUnsupportedImageType when one of the images is not image.RGBA

- any error returned by ctx.Err()

func Find

func Find(ctx context.Context, canvas, sample image.Image, tolerance float64, selection Selection) (Match, error)

Find locates and returns a single match of the sample image within a canvas image. Depending on the selection argument, this will either be the best match (which requires searching the entire canvas) or the first encountered match (which will stop searching after the match is found).

Tolerance is a value between 0 and 1 specifying how much difference is tolerated between the sample and its match. Pass 0 to find exact matches only. A reasonable value for fuzzy matching is around 0.05.

Searching stops if the context is canceled or expired (i.e. the Err() method of the context returns a non-nil value). In this case, the function returns the best match found so far, together with a non-nil error.

This function can return one of the following errors:

- ErrUnsupportedImageType when one of the images is not image.RGBA

- ErrNotFound is no matches have been found after searching the entire canvas

- any error returned by ctx.Err()

Example

package main

import (
	"context"
	"fmt"
	"image"
	"image/color"
	"image/draw"

	"github.com/andreyvit/locateimage"
)

func main() {
	red := color.RGBA{255, 0, 0, 255}

	// 10x10 uniform-red sample
	sample := image.NewRGBA(image.Rect(0, 0, 10, 10))
	draw.Draw(sample, sample.Bounds(), image.NewUniform(red), image.ZP, draw.Src)

	// expected match rect
	r := image.Rect(15, 15, 25, 25)

	// 100x100 white canvas with a red rectangle at (10,10)
	canvas := image.NewRGBA(image.Rect(0, 0, 100, 100))
	draw.Draw(canvas, canvas.Bounds(), image.NewUniform(color.White), image.ZP, draw.Src)
	draw.Draw(canvas, r, image.NewUniform(red), image.ZP, draw.Src)

	fmt.Printf("Find: ")
	m, err := locateimage.Find(context.Background(), canvas, sample, 0, locateimage.Only)
	if err != nil {
		panic(err)
	}
	fmt.Printf("%v %.*f\n", m.Rect, locateimage.SimilarityDigits, m.Similarity)

	fmt.Printf("All: ")
	mm, err := locateimage.All(context.Background(), canvas, sample, 0.04)
	if err != nil {
		panic(err)
	}
	fmt.Println(mm)

	fmt.Printf("Foreach: ")
	err = locateimage.Foreach(context.Background(), canvas, sample, 0.04, func(m locateimage.Match) error {
		fmt.Println(m)
		return nil
	})
	if err != nil {
		panic(err)
	}

}

Output:

Find: (15,15)-(25,25) 1.000000
All: [(15,15)+(10x10) 100.0000%]
Foreach: (15,15)+(10x10) 100.0000%

func (Match) Before

func (a Match) Before(b Match) bool

Before returns whether this match should sort before the given match.

Higher-similarity matches come before lower-similarity matches, and equally-similar matches are ordered by Y, then by X, ascending.

func (Match) String

func (m Match) String() string

String returns a string description of this match for debugging purposes.

type Selection

type Selection int

Selection determines the way Find selects a match to return.

func (Selection) String

func (s Selection) String() string

String returns a string description of this Selection value for debugging purposes.