confusables

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 18, 2026 License: MIT Imports: 7 Imported by: 0

README

go-confusables

Unicode Confusable Characters Library for Go.

Go Reference

This library provides utilities to detect and normalize Unicode "confusable" characters (homoglyphs), which are often used to evade text-based security filters. It uses the authoritative data from Unicode TR39.

Installation

go get github.com/disciplinedware/go-confusables

Features

  • Authoritative Data: Embedded latest confusables.txt from unicode.org.
  • Fast: In-memory lookups, zero network/FS calls at runtime.
  • Normalize to ASCII: Easily replace visually similar characters with their ASCII equivalents.
  • Skeleton Algorithm: Full implementation of the TR39 skeleton algorithm for secure string comparison.
  • CLI Tool: Included confusables-gen to update data from the latest Unicode version.

Usage

Simple ASCII Normalization
import "github.com/disciplinedware/go-confusables"

db := confusables.Default()

// Cyrillic 'а' and 'о' look like Latin 'a' and 'o'
text := "hеllо" 
normalized := db.ToASCII(text)
fmt.Println(normalized) // "hello"
Secure String Comparison (Skeleton)

The skeleton algorithm is recommended for detecting spoofed strings (e.g., usernames).

db := confusables.Default()

user1 := "apple"
user2 := "аррle" // Uses Cyrillic 'а' and 'р'

if db.IsConfusable(user1, user2) {
    fmt.Println("Warning: Usernames are confusable!")
}
Metadata Access

Access information about the embedded Unicode dataset:

fmt.Printf("Unicode Version: %s\n", db.UnicodeVersion())
fmt.Printf("Source Date: %s\n", db.SourceDate())

CLI Tool: Updating Data

You can regenerate the embedded data using the provided CLI tool. For CI/reproducible builds, you can provide a deterministic timestamp:

go run ./cmd/confusables-gen --version 16.0.0 --generated-at 2026-02-18T12:00:00Z

License

MIT

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type DB

type DB struct {
	// contains filtered or unexported fields
}

DB is the confusables database. Thread-safe after initialization.

func Default

func Default() *DB

Default returns the embedded database (loaded once via sync.Once).

func Load

func Load(jsonData []byte) (*DB, error)

Load creates a DB from a JSON file (for custom/updated data).

func (*DB) GeneratedAt

func (db *DB) GeneratedAt() time.Time

GeneratedAt returns the timestamp when the database was generated.

func (*DB) IsConfusable

func (db *DB) IsConfusable(a, b string) bool

IsConfusable checks if two strings would produce the same skeleton.

func (*DB) Lookup

func (db *DB) Lookup(r rune) []rune

Lookup returns all target runes for a confusable character. Returns a defensive copy of the internal slice. Returns nil if the character has no mapping.

func (*DB) LookupASCII

func (db *DB) LookupASCII(r rune) (rune, bool)

LookupASCII returns the ASCII equivalent of a rune, if one exists. Returns (replacement, true) for single-char ASCII mappings. Returns (0, false) for characters with no mapping or multi-char/non-ASCII targets. Note: If the input rune is already ASCII (r < 0x80), it returns (0, false) to prevent remapping characters that are already valid ASCII.

func (*DB) Skeleton

func (db *DB) Skeleton(s string) string

Skeleton returns the TR39 skeleton of the string. Maps all confusable characters through the database, regardless of target length. Result is NOT suitable for display — use only for comparison. Implementation: NFD → map → NFD

func (*DB) SourceDate

func (db *DB) SourceDate() string

SourceDate returns the date of the source data.

func (*DB) SourceURL

func (db *DB) SourceURL() string

SourceURL returns the URL of the source data.

func (*DB) ToASCII

func (db *DB) ToASCII(s string) string

ToASCII replaces confusable characters with their ASCII equivalents. Only replaces characters that map to a SINGLE ASCII char (0x20-0x7E). Characters with multi-char targets or non-ASCII targets are kept as-is. Already-ASCII characters (0x00-0x7F) are always returned unchanged.

func (*DB) UnicodeVersion

func (db *DB) UnicodeVersion() string

UnicodeVersion returns the Unicode version of the database.

type Mapping

type Mapping struct {
	Source     int    `json:"source"`
	Target     []int  `json:"target"`
	SourceName string `json:"source_name"`
	TargetName string `json:"target_name"`
}

Mapping represents a single confusable mapping.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL