lookup

README

Lookup

It is a nice, simple and fast library which helps you to lookup objects on a screen. It also includes OCR functionality. Using Lookup you can do OCR tricks like recognizing any information in your Robot application. Which can be useful for debugging or automating things.

This library is a port of the Java Lookup library to GoLang. Details of NCC (Normalized Cross Correlation), used by this library, can be found in the original library's 'docs' folder (a lot of math).

Usage

Add this library to your project with:

go get github.com/deluan/lookup

To learn how to use it, take a look at the example files for Lookup and OCR. All images used in the examples are available in the testdata folder. For more details check the full documentation.

To Do:

• Add basic LookUp function
• Implement OCR
• Optimize for speed
• Clean-up API
• Better docs
• Implement Scaling

Documentation

Overview

Package lookup implements a nice, simple and fast library which helps you to lookup objects on a screen. It also includes OCR functionality. With Lookup you can do OCR tricks like recognizing any information in a screenshot from your Robot application. Which can be useful for debugging or automating things.

This library is a port of the Java Lookup library (https://gitlab.com/axet/lookup) to GoLang. Details on NCC (Normalized Cross Correlation) used by this library can be found in the original library's docs folder (a lot of math).

Example (Lookup)

package main

import (
	"fmt"
	"image"
	"os"

	_ "image/png"

	"github.com/deluan/lookup"
)

// Helper function to load an image from the filesystem
func loadImage(imgPath string) image.Image {
	imageFile, _ := os.Open(imgPath)
	defer imageFile.Close()
	img, _, _ := image.Decode(imageFile)
	return img
}

func main() {
	// Load full image
	img := loadImage("testdata/cyclopst1.png")

	// Create a lookup for that image
	l := lookup.NewLookup(img)

	// Load a template to search inside the image
	template := loadImage("testdata/cyclopst3.png")

	// Find all occurrences of the template in the image
	pp, _ := l.FindAll(template, 0.9)

	// Print the results
	if len(pp) > 0 {
		fmt.Printf("Found %d matches:\n", len(pp))
		for _, p := range pp {
			fmt.Printf("- (%d, %d) with %f accuracy\n", p.X, p.Y, p.G)
		}
	} else {
		println("No matches found")
	}

}

Output:

Found 1 matches:
- (21, 7) with 0.997942 accuracy

Example (Ocr)

package main

import (
	"fmt"
	"image"
	"os"

	_ "image/png"

	"github.com/deluan/lookup"
)

// Helper function to load an image from the filesystem
func loadImageFromFile(imgPath string) image.Image {
	imageFile, _ := os.Open(imgPath)
	defer imageFile.Close()
	img, _, _ := image.Decode(imageFile)
	return img
}

func main() {
	// Create an OCR object with an accuracy of 0.7
	ocr := lookup.NewOCR(0.7)

	// Load a fontSet
	_ = ocr.LoadFont("testdata/font_1")

	// Load an image to recognize
	img := loadImageFromFile("testdata/test3.png")

	// Recognize text in image
	text, _ := ocr.Recognize(img)

	// Print the results
	fmt.Printf("Text found in image: %s\n", text)

}

Output:

Text found in image: 3662
32€/€

Index

• type GPoint
• type Lookup
  • func NewLookup(image image.Image) *Lookup
  • func NewLookupColor(image image.Image) *Lookup
  • func (l *Lookup) FindAll(template image.Image, threshold float64) ([]GPoint, error)
  • func (l *Lookup) FindAllInRect(template image.Image, rect image.Rectangle, threshold float64) ([]GPoint, error)
• type OCR
  • func NewOCR(threshold float64, numThreads ...int) *OCR
  • func (o *OCR) LoadFont(fontPath string) error
  • func (o *OCR) Recognize(img image.Image) (string, error)

Examples

• Package (Lookup)
• Package (Ocr)

Types

type GPoint

type GPoint struct {
	X, Y int
	G    float64
}

GPoint represents a match of a template inside an image.

type Lookup

type Lookup struct {
	// contains filtered or unexported fields
}

Lookup implements a image search algorithm based on Normalized Cross Correlation. For an overview of the algorithm, see http://www.fmwconcepts.com/imagemagick/similar/index.php

func NewLookup

func NewLookup(image image.Image) *Lookup

NewLookup creates a Lookup object. Lookup objects created by these function will do a gray scale search of the templates. Note that if the image or any template is not GrayScale, it will be converted automatically. This could cause some performance hits

func NewLookupColor

func NewLookupColor(image image.Image) *Lookup

NewLookupColor creates a Lookup object that works with all color channels of the image (RGB). Keep in mind that Lookup objects created with this function can only accept color templates as parameters to the FindAll method

func (*Lookup) FindAll

func (l *Lookup) FindAll(template image.Image, threshold float64) ([]GPoint, error)

FindAll searches for all occurrences of template inside the whole image.

func (*Lookup) FindAllInRect

func (l *Lookup) FindAllInRect(template image.Image, rect image.Rectangle, threshold float64) ([]GPoint, error)

FindAllInRect searches for all occurrences of template only inside a part of the image. This can be used to speed up the search if you know the region of the image that the template should appear in.

type OCR

type OCR struct {
	// contains filtered or unexported fields
}

OCR implements a simple OCR based on the Lookup functions. It allows multiple fontsets, just call LoadFont for each fontset.

If you need to encode special symbols use UNICODE in the file name. For example if you need to have '\' character (which is prohibited in the path and file name) specify %2F.png as a image symbol name.

Sometimes you need to specify two different image for one symbol (if image / font symbol vary too much). To do so add unicode ZERO WIDTH SPACE symbol (%E2%80%8B) to the filename. Ex: %2F%E2%80%8B.png will produce '/' symbol as well.

func NewOCR

func NewOCR(threshold float64, numThreads ...int) *OCR

NewOCR creates a new OCR instance, that will use the given threshold. You can optionally parallelize the processing by specifying the number of threads to use. The optimal number varies and depends on your use case (size of fontset x size of image). Default is use only one thread

func (*OCR) LoadFont

func (o *OCR) LoadFont(fontPath string) error

LoadFont loads a specific fontset from the given folder. Fonts are simple image files containing a PNG/JPEG of the font, and named after the "letter" represented by the image.

This can be called multiple times, with different folders, to load different fontsets.

func (*OCR) Recognize

func (o *OCR) Recognize(img image.Image) (string, error)

Recognize the text in the image using the fontsets previously loaded. If a SubImage is received, the search will be limited by the boundaries of the SubImage