norm

README

norm

norm normalizes user-entered strings. It is the writing companion to goloop/is: where is reads (is this an email?), norm cleans a value toward the canonical form of that type and validates the result.

value, ok := norm.Email("  Example @ Gmail.com ")
// value == "Example@gmail.com", ok == true

Standard library only, plus goloop/is for validation.

Install

go get github.com/goloop/norm

Typed normalizers

Each mirrors an is predicate and returns (cleaned, valid). The cleaned value is returned even when invalid, so a UI can suggest "did you mean X?".

Function	Normalization
Email	trim, lowercase domain (local part preserved), repair internal spaces
URL	trim, lowercase scheme and host only (no scheme is invented)
UUID	trim, strip {} and urn:uuid:, lowercase
MAC	regroup to lowercase colon pairs (accepts :, -, ., none)
IBAN	remove spaces, uppercase, strict checksum
BankCard	remove spaces and dashes, Luhn (and optional kinds)
E164	remove spaces, dashes, parentheses, dots (no country code invented)
Hex, MD5, SHA1, SHA256, SHA512	trim, lowercase

Every value is first scrubbed of control and invisible characters (a zero-width space in an email is removed before validation). A value is never reported valid unless is agrees, and norm never strips arbitrary punctuation or invents data.

Character toolkit

For input that is not a fixed external format, use the character-group tools. Class is a bitmask (combine with |):

norm.Trim(s, norm.Spaces|norm.Controls) // remove groups from both ends
norm.Remove(s, norm.Invisible)          // remove a group everywhere
norm.Keep(s, norm.Letters|norm.Digits)  // keep only these groups
norm.CollapseSpace(s)                   // runs of whitespace -> one space
norm.Lower(s), norm.Upper(s)            // Unicode case

Groups: Spaces, Controls, Invisible (zero-width / format characters, BOM, soft hyphen, variation selectors), Punct, Symbols, Letters, Digits.

Keep and Remove are the honest replacement for lossy property mirrors: there is deliberately no norm.Alpha or norm.Digit, because "normalizing" to those is really filtering, so you write norm.Keep(s, norm.Letters) instead.

Documentation

• English reference: DOC.md
• Ukrainian reference: DOC.UK.md

License

MIT - see LICENSE.

Documentation

Overview

Package norm normalizes user-entered strings, the writing companion to the predicate module github.com/goloop/is: where is reads (is this an email?), norm cleans a value toward the canonical form of that type and validates the result.

Typed normalizers

Each typed normalizer mirrors an is predicate and returns the cleaned value together with whether it is valid:

value, ok := norm.Email("  Example @ Gmail.com ")
// value == "Example@gmail.com", ok == true

The returned string is the best-effort normalized value even when ok is false, so a UI can offer a "did you mean" suggestion. Covered types: Email, URL, UUID, MAC, IBAN, BankCard, E164 and the hex/hash family (Hex, MD5, SHA1, SHA256, SHA512).

Normalization is conservative and type-specific. Every value is first scrubbed of control and invisible characters (never valid inside these formats); safe canonical steps (trim, lowercasing a case-insensitive domain or scheme) are always applied; a more aggressive repair (removing internal whitespace) is only attempted when the value is still invalid. Every result is confirmed with an is predicate, so a value is never reported valid unless is agrees. norm never strips arbitrary punctuation and never invents data (for example a URL scheme or a phone country code).

Character toolkit

For the cases that are not a fixed external format, norm offers character group tools driven by a Class bitmask:

norm.Trim(s, norm.Spaces|norm.Controls) // remove groups from both ends
norm.Remove(s, norm.Invisible)          // remove a group everywhere
norm.Keep(s, norm.Letters|norm.Digits)  // keep only these groups
norm.CollapseSpace(s)                   // runs of whitespace -> one space
norm.Lower(s), norm.Upper(s)            // Unicode case

These deliberately replace mirrors like a hypothetical norm.Alpha or norm.Digit, which would imply a canonical form but really perform lossy filtering; Keep and Remove make that explicit.

See DOC.md (English) and DOC.UK.md (Ukrainian) for the full reference.

Index

• func BankCard(s string, kinds ...is.CardKind) (string, bool)
• func CollapseSpace(s string) string
• func E164(s string) (string, bool)
• func Email(s string) (string, bool)
• func Hex(s string) (string, bool)
• func IBAN(s string) (string, bool)
• func Keep(s string, class Class) string
• func Lower(s string) string
• func MAC(s string) (string, bool)
• func MD5(s string) (string, bool)
• func Remove(s string, class Class) string
• func SHA1(s string) (string, bool)
• func SHA256(s string) (string, bool)
• func SHA512(s string) (string, bool)
• func Trim(s string, class Class) string
• func TrimLeft(s string, class Class) string
• func TrimRight(s string, class Class) string
• func URL(s string) (string, bool)
• func UUID(s string) (string, bool)
• func Upper(s string) string
• type Class

Examples

• Email
• IBAN
• Keep
• MAC
• Remove

Functions

func BankCard

func BankCard(s string, kinds ...is.CardKind) (string, bool)

BankCard normalizes a payment card number and reports whether the result is valid. It removes spaces and dashes (the usual grouping separators), then validates the number with the Luhn check and, when kinds are given, against those card kinds. No other characters are removed.

Security: the returned value is a full card number (PAN). Do not log or store it unnecessarily, and remember that a Luhn-valid number is not proof of a real or intended card.

func CollapseSpace

func CollapseSpace(s string) string

CollapseSpace trims surrounding whitespace and replaces every internal run of whitespace with a single ASCII space.

func E164

func E164(s string) (string, bool)

E164 normalizes a phone number toward the E.164 format and reports whether the result is valid. It removes spaces, dashes, parentheses and dots. It is non-magical: it never guesses a country code, so a national number without a leading "+" stays invalid.

func Email

func Email(s string) (string, bool)

Email normalizes an email address and reports whether the result is valid.

It trims surrounding whitespace and lowercases the domain (domains are case-insensitive; the local part is preserved, as it is technically case-sensitive). If the address is still invalid, it removes internal whitespace as a best-effort repair. A valid input, including a quoted local part that legitimately contains spaces, is preserved because the repair runs only when the milder result does not validate.

It does not perform IDN/punycode conversion: a Unicode domain is accepted only as far as is.Email accepts it.

Example

package main

import (
	"fmt"

	"github.com/goloop/norm"
)

func main() {
	value, ok := norm.Email("  Example @ Gmail.com ")
	fmt.Println(value, ok)
}

Output:
Example@gmail.com true

func Hex

func Hex(s string) (string, bool)

Hex normalizes a hexadecimal string (trim and lowercase) and reports whether the result is valid hex.

func IBAN

func IBAN(s string) (string, bool)

IBAN normalizes an International Bank Account Number and reports whether the result is valid. It removes spaces and uppercases the value (an IBAN is defined over uppercase letters and digits), then validates it strictly, including the checksum. No other characters are removed.

Example

package main

import (
	"fmt"

	"github.com/goloop/norm"
)

func main() {
	value, ok := norm.IBAN("de89 3704 0044 0532 0130 00")
	fmt.Println(value, ok)
}

Output:
DE89370400440532013000 true

func Keep

func Keep(s string, class Class) string

Keep deletes every character NOT in the given class from s. It is the inverse of Remove and is the honest way to filter to, say, letters: Keep(s, Letters).

Example

package main

import (
	"fmt"

	"github.com/goloop/norm"
)

func main() {
	fmt.Println(norm.Keep("a1-b2 c3!", norm.Letters|norm.Digits))
}

Output:
a1b2c3

func Lower

func Lower(s string) string

Lower returns s with all Unicode letters mapped to lower case. It is the normalizing counterpart of is.Lower and always succeeds, so it returns only a string.

func MAC

func MAC(s string) (string, bool)

MAC normalizes a MAC address to lowercase, colon-separated form and reports whether the result is valid. It accepts the common separators (":", "-", ".") and the separatorless form, and regroups an EUI-48 (12 hex digits) or EUI-64 (16 hex digits) into colon-separated pairs. No other punctuation is stripped.

Example

package main

import (
	"fmt"

	"github.com/goloop/norm"
)

func main() {
	value, ok := norm.MAC("0011.2233.4455")
	fmt.Println(value, ok)
}

Output:
00:11:22:33:44:55 true

func MD5

func MD5(s string) (string, bool)

MD5 normalizes an MD5 digest (trim and lowercase) and reports validity.

func Remove

func Remove(s string, class Class) string

Remove deletes every character of the given class from s. A zero class leaves s unchanged.

Example

package main

import (
	"fmt"

	"github.com/goloop/norm"
)

func main() {
	// Strip zero-width and control characters that break validation.
	clean := norm.Remove("\u200bhi\ufeff", norm.Invisible)
	fmt.Printf("%q\n", clean)
}

Output:
"hi"

func SHA1

func SHA1(s string) (string, bool)

SHA1 normalizes a SHA-1 digest (trim and lowercase) and reports validity.

func SHA256

func SHA256(s string) (string, bool)

SHA256 normalizes a SHA-256 digest (trim and lowercase) and reports validity.

func SHA512

func SHA512(s string) (string, bool)

SHA512 normalizes a SHA-512 digest (trim and lowercase) and reports validity.

func Trim

func Trim(s string, class Class) string

Trim removes characters of the given class from both ends of s. A zero class leaves s unchanged.

func TrimLeft

func TrimLeft(s string, class Class) string

TrimLeft removes characters of the given class from the start of s.

func TrimRight

func TrimRight(s string, class Class) string

TrimRight removes characters of the given class from the end of s.

func URL

func URL(s string) (string, bool)

URL normalizes a URL and reports whether the result is valid.

It trims surrounding whitespace and lowercases the scheme and host only; the userinfo, path, query and fragment are left byte-for-byte untouched (no re-encoding). It never invents a missing scheme, so a schemeless input such as "example.com" stays invalid (and is returned as the best-effort value with false).

func UUID

func UUID(s string) (string, bool)

UUID normalizes a UUID and reports whether the result is valid. It trims surrounding whitespace, strips a surrounding pair of braces and a leading "urn:uuid:" prefix, and lowercases the value. It does not remove any internal characters other than the recognized wrappers.

func Upper

func Upper(s string) string

Upper returns s with all Unicode letters mapped to upper case. It is the normalizing counterpart of is.Upper and always succeeds, so it returns only a string.

Types

type Class

type Class uint

Class is a bitmask selecting one or more groups of characters. Combine groups with the bitwise OR operator, for example Spaces|Controls.

const (
	// Spaces are Unicode White_Space characters, including the regular space,
	// tab, newline and no-break space (U+00A0).
	Spaces Class = 1 << iota

	// Controls are Unicode control characters (category Cc).
	Controls

	// Invisible are zero-width and formatting characters that render to nothing
	// yet can corrupt input or hide content: Unicode format characters
	// (category Cf, such as the zero-width space, byte-order mark, soft hyphen
	// and word joiner), variation selectors and the combining grapheme joiner.
	Invisible

	// Punct are Unicode punctuation characters (category P).
	Punct

	// Symbols are Unicode symbol characters (category S), such as math and
	// currency symbols.
	Symbols

	// Letters are Unicode letters (category L).
	Letters

	// Digits are Unicode decimal digits (category Nd).
	Digits
)