i18n

README

i18n

kanna-i18n compiles a directory of locale files into Go: a typed constructor for every message, and the translations themselves, so nothing is parsed or read at run time.

Part of the kanna family of code generators. Requires Go 1.26.

Install

go get -tool github.com/go-kanna/i18n/cmd/kanna-i18n

The generated code calls the runtime at the root of this module, github.com/go-kanna/i18n, which the line above already requires. The runtime does what depends on run-time values: choosing the language, picking the CLDR plural form, and formatting numbers for the locale.

Use

One YAML file per language, named by its language tag:

# locales/en.yaml
greeting: "Hello!"
hello: "Hello, {name}!"
items_count:
  plural:
    one: "You have {count} item."
    other: "You have {count} items."
total_price: "Total: {price:number}"
user:
  not_found: "User not found."

# locales/ja.yaml
greeting: "こんにちは！"
hello: "こんにちは、{name}さん！"
items_count:
  plural:
    other: "アイテムが{count}個あります。"
total_price: "合計: {price}"
user:
  not_found: "ユーザーが見つかりません。"

//go:generate go tool kanna-i18n

With the defaults, that line reads locales/ and writes messages/i18n_gen.go: a constructor per message, a Localizer function, and the translations of every language.

// Code generated by kanna-i18n. DO NOT EDIT.

package messages

import (
	"github.com/go-kanna/i18n"
)

// Hello returns the message "hello":
//
//	Hello, {name}!
func Hello(name string) i18n.Message {
	return i18n.Message{Key: "hello", Args: []i18n.Arg{
		{Name: "name", Value: name},
	}}
}

// ItemsCount returns the message "items_count", in the form count selects:
//
//	one:   You have {count} item.
//	other: You have {count} items.
func ItemsCount(count int) i18n.Message {
	return i18n.Message{Key: "items_count", Args: []i18n.Arg{
		{Name: "count", Value: count},
	}}
}

// ...

// Localizer renders this package's messages in the language that best matches
// langs, each a language tag or an Accept-Language value, tried in order.
// A message missing from that language falls back to its parent languages and
// then to en.
func Localizer(langs ...string) i18n.Localizer {
	return bundle.Localizer(langs...)
}

// bundle holds every language this package was generated from, en, the
// default, first.
var bundle = i18n.NewBundle(
	i18n.Catalog{Lang: "en", Entries: map[string]i18n.Entry{
		"hello": {Single: i18n.Template{{Text: "Hello, "}, {Param: "name"}, {Text: "!"}}},
		"items_count": {Plural: &i18n.Plural{
			One:   i18n.Template{{Text: "You have "}, {Param: "count"}, {Text: " item."}},
			Other: i18n.Template{{Text: "You have "}, {Param: "count"}, {Text: " items."}},
		}},
		// ...
	}},
	i18n.Catalog{Lang: "ja", Entries: map[string]i18n.Entry{
		// ...
	}},
)

Each constructor's documentation carries what the default language says, so an editor shows the text of a message where it is called.

A constructor returns an i18n.Message, not a string: code that does not know the reader's language can hand one back — inside an error, say — and the code that does know renders it:

loc := messages.Localizer(r.Header.Get("Accept-Language"))
fmt.Fprintln(w, loc.Localize(messages.Hello("Alice")))      // Hello, Alice! or こんにちは、Aliceさん！
fmt.Fprintln(w, loc.Localize(messages.ItemsCount(1)))       // You have 1 item. or アイテムが1個あります。
fmt.Fprintln(w, loc.Localize(messages.TotalPrice(1234.56))) // Total: 1,234.56 or 合計: 1,234.56

A translation that does not fit the default language stops generation, rather than the first request that meets it. Had ja.yaml above misspelled a placeholder and a key, and given the price a kind of its own:

$ go generate ./...
locales/ja.yaml:2:8: error: parameter "namae" of key "hello" is not in en
  did you mean "name"?
locales/ja.yaml:6:14: error: parameter "price" of key "total_price" is int, but number in en
locales/ja.yaml:8:3: error: key "user.not_fund" is not in en
  did you mean "user.not_found"?
locales/ja.yaml: warning: missing keys: user.not_found

Locale files

The locale directory is flat — one file per language, no recursion — and the file name without its extension is the language: en.yaml, pt-BR.yaml, ja.yml. Nothing named like a language is skipped in silence: a subdirectory such as locales/en/ or locales/en.yaml/, or a file in another format such as en.json or en.toml, stops generation. Other files are skipped with a warning, so a stray config.yaml does not fail the run, and names starting with a dot are skipped without one.

Nested mappings become dot-joined keys, and keys become constructor names: user.not_found generates UserNotFound(). Each part of a key matches [a-z][a-z0-9_]*. Two keys that generate one name, or a key that generates a name the destination package already declares, stop generation; so do the key localizer, whose name the generated file takes for itself, and a Localizer the destination package declares by hand.

Every value is a string or a mapping. Quote a value that YAML would read as something else, such as "true" or "123". A message cannot hold a control character other than a line break or a tab, nor a byte order mark.

Plural forms

A message declares its plural forms under an explicit plural mapping, keyed by CLDR category (zero, one, two, few, many, other) and always defining other. Keys that merely happen to be named one or other are ordinary nesting: the only name a locale file cannot use freely is a mapping-valued plural, which takes no other keys beside it. At the root of a file, where there is no message for it to be the forms of, plural is ordinary nesting too.

The constructor takes count int first, and the count picks the form under the plural rules of the language rendering it: 1 is one in English, while Japanese has no one and uses other for every count. A plural message that lacks a form its language uses gets a warning, because those counts would render with other: Russian with only other lacks one, few, and many, while Japanese with only other lacks nothing.

A count of 0 renders with zero whenever the message has it, as in Rails, so zero: "You have no items." in en.yaml works although English has no zero category. In a language whose rules do have one, such as Latvian, zero takes the other counts of that category too: 10 through 20, for one.

Placeholders

Form	Parameter	Rendered
{name}	string	as it is
{name:int}	int	as it is: counts, IDs
{name:number}	float64	in the language's format, three decimals at most: 1,234.56 in en, 1.234,56 in de
{{, }}		literal braces

{name:string} is the same as {name}, and count in a plural message is always an int. A number is written as CLDR's default pattern for the language has it, which keeps three fraction digits at most: 1234.5678 renders as 1,234.568. Within a message, a placeholder written without a kind takes the one written for it elsewhere — in another plural form, say — and two different kinds are an error. The default language decides each parameter's type: a translation may leave the kind off, as ja.yaml above does for the price, and one that writes a different kind stops generation.

Parameters follow the order in which they first appear, count first and the forms of a plural message read in the order of their categories, zero to other, and are named in camelCase: {user_name} becomes userName. A name Go reserves or predeclares gets Arg appended: {type} becomes typeArg, and {error} becomes errorArg.

Languages

The default language — -default, en unless said otherwise — defines the constructors: their names, parameters, and types come from its file, and every other file is checked against it. A file for a regional variant serves a bare default when there is no exact one: -default en takes en-US.yaml in the absence of en.yaml, and warns when en-GB.yaml could have been meant as well. A default written with a region or script, such as -default en-US, is served by its own file alone: neither en.yaml nor en-GB.yaml stands in for it.

Localizer takes language tags such as "ja" or "pt-BR", Accept-Language values such as "fr, ja;q=0.8", or both, and tries them in order, value by value, until one is close to a language the package was generated with, as CLDR judges closeness: a request for en-US is served by en, and one for nb by da when the package has Danish but no Norwegian. So messages.Localizer(cookie.Value, r.Header.Get("Accept-Language")) prefers the language the user picked, then the ones the browser asks for. A value that does not parse is skipped as a whole, one bad entry and all, and when nothing matches, the default language is used.

A message the chosen language lacks falls back through its parent languages — en-GB to en — and then to the default language, and is rendered under the plural rules and number format of the language it is found in. Localize never fails: the zero Localizer renders every message as its key, and a placeholder whose argument is missing renders as {name}.

Validation errors

kanna-validate and kanna-value report a broken rule as a code with params, such as too_long with count, and leave its text to the caller, as Rails' errors.details does. i18n.FullMessages renders them as sentences naming their fields, and i18n.ErrorMessages by field, as Rails' errors.full_messages and errors.messages do:

loc := messages.Localizer(lang)
if errs, ok := errors.AsType[validate.Errors](err); ok {
	for _, m := range i18n.FullMessages(loc, errs) {
		fmt.Println(m)
	}
}
// Title can't be blank
// Tags is too long (maximum is 20 characters)
// Confirm doesn't match Password
//
// タイトルを入力してください
// Tagsは20文字以内で入力してください
// 確認用パスワードとパスワードの入力が一致しません

The texts of every code kanna-validate and kanna-value report are built in, in English as Rails writes them and in Japanese as rails-i18n does. A locale file names the fields, overrides any built-in text, and adds the codes of rules of your own:

# locales/ja.yaml
errors:
  attributes:
    title: タイトル
    confirm: 確認用パスワード
    password: パスワード
    address:
      zip: 郵便番号
  messages:
    corp_domain: は会社のアドレスにしてください

• errors.format puts a field's name and its message together: "{attribute} {message}" in English, "{attribute}{message}" in Japanese.
• A field is named by errors.attributes: its path with any index dropped, address.zip or tags for tags[1], then the last part of it, zip, and failing both, the last part written out: Completed at for completed_at. A confirmation's attribute, the field it is checked against, is named the same way.
• A problem with the value as a whole, at base, reads as its message alone.
• A message is looked up in the reader's language and its parents, the application's text first and then the built-in one, before the default language: a reader of Japanese gets the built-in Japanese rather than the application's English. A translation may override a built-in text that the default language leaves as it is, and name a field that the default language leaves to be written out: Japanese names title, while English reads Title anyway.

Anything with Path, Code, and Params methods renders: this package imports no validation package. For one message without its field, i18n.Coded("errors.messages", code, params) builds it. A *rule.Error of go-kanna/value, from building or decoding a value type, has Code and Params as well.

A code no language has renders as its key, errors.messages.taken, so a rule without a text shows rather than vanishes.

The kind of a placeholder types the generated constructor and nothing else: the runtime never sees it, so the params of a code render by their Go type whatever the locale file says, and a kind that does not match fails nothing. {count:number} does not write an int in the language's number format, nor does {count:int} keep a float64 out of it. What a count's type does decide is the plural form: only an int picks one, and any other count takes other. The lengths of too_short, too_long, and wrong_length are always an int, so write them {count:int}, or in plural forms. The bound of greater_than and the other numeric codes has the type of the field it was checked on, an int, a float64, or an int64, so leave its kind off and its message single.

A param that is neither a string, an int, nor a float64 renders through fmt, as i18n.Arg says: a time.Time a hook passed as count reads 2000-01-01 00:00:00 +0000 UTC. To write one otherwise, replace it in the params before building the message; Params of either package returns a copy.

Validation

Errors stop generation:

• a key a translation has but the default language does not
• a message that is plural in one language and not in the other
• a placeholder the default language's message does not have, or one written with a different kind
• two different kinds for one placeholder within a message
• a malformed file, key, or placeholder, and names that collide

Warnings do not:

• keys a translation lacks. The runtime falls back, so the generated code never breaks when a translation lags behind; it renders the fallback until the translation lands.
• plural forms a language uses that a message does not provide
• an ambiguous default language, and files skipped in the locale directory

A misspelled key, placeholder, plural category, or kind comes with the name it was probably meant to be.

Flags

kanna-i18n [flags]

Flag	Meaning
-locales <dir>	directory holding the locale files (default locales)
-default <lang>	language that defines the constructors and is the last fallback (default en)
-destination <dir>	directory to write i18n_gen.go into (default messages)
-package <name>	package name for a destination without Go files (default: the directory's name)
-check	verify the output is up to date instead of writing it
-v	list the file written or left unchanged
-version	print the version

Paths are relative to the directory go generate runs in, the one holding the //go:generate line. The package of the generated file is the one the Go files already in the destination declare; -package names it when there are none yet, and one that disagrees with them is an error. A run that succeeds prints nothing but its warnings, which Validation lists, such as the keys a translation lacks. A file named i18n_gen.go without the generated-code header is never touched.

Code calling the messages is checked with what a run is about to write in place: the destination's own, and that of the packages of its module importing it, tests included, which a run loads through the go command. A key removed or renamed, or a placeholder changed, stops the run while code still calls the message as it was, and so does a mistake that a call to a message added hid; nothing is written then. An error that code had before the run does not stop it, but one in the destination does, the output's own package, as does a destination that does not parse. Code may call a message before the first run has written it.

Example

example renders the same messages in English and Japanese — plurals, locale-formatted numbers, a fallback for a missing translation, and a language chosen from an Accept-Language value — with nothing to set up at run time. CI regenerates it and fails if the output would change.

Development

make test       # unit tests
make lint       # golangci-lint
make example    # regenerate the example, then build and run it

License

MIT

Documentation

Overview

Package i18n is the runtime that code generated by kanna-i18n calls, and what renders its messages.

kanna-i18n compiles a directory of locale files into a package of typed constructors, each returning a Message, together with the translations as Catalogs. Catalog, Entry, Plural, Template, and Segment are what it writes those as, in literals, and CountParam the name it gives the count of a plural message: a program need build none of them. Nothing is parsed at run time: what is left here is what depends on run-time values, which is choosing the language, picking the CLDR plural form a count needs, and formatting numbers the way the language writes them.

The generator itself is the command in cmd/kanna-i18n.

Index

• Constants
• func ErrorMessages[P Problem](l Localizer, problems []P) map[string][]string
• func FullMessages[P Problem](l Localizer, problems []P) []string
• func IsBuiltin(key string) bool
• type Arg
• type Bundle
  • func NewBundle(def Catalog, rest ...Catalog) Bundle
  • func (b Bundle) Localizer(langs ...string) Localizer
• type Catalog
• type Entry
• type Localizer
  • func (l Localizer) Localize(m Message) string
• type Message
  • func Coded(prefix, code string, params map[string]any) Message
• type Plural
• type Problem
• type Segment
• type Template

Constants

const CountParam = "count"

CountParam is the argument whose value, an int, picks the form of a plural message. It is exported for kanna-i18n, which names the count of every plural constructor so.

Functions

func ErrorMessages

func ErrorMessages[P Problem](l Localizer, problems []P) map[string][]string

ErrorMessages renders problems as Rails renders errors.messages: by path, each the message keyed errors.messages.<code>, in the order they come. It suits an API response, which shows each message beside its field.

func FullMessages

func FullMessages[P Problem](l Localizer, problems []P) []string

FullMessages renders problems as Rails renders errors.full_messages: each a sentence naming its field, put together by errors.format, such as "Title can't be blank" or "タイトルを入力してください". A problem at base reads as its message alone.

The field is named by errors.attributes: the path with any index dropped, address.zip or tags, then its last part, zip, and failing both, the last part written out, Completed at for completed_at. A confirmation's attribute, the other field it names, is named the same way.

func IsBuiltin

func IsBuiltin(key string) bool

IsBuiltin reports whether key is a built-in text, which the catalog of any language may override, the default language's included or not. It is exported for kanna-i18n, which lets a translation hold such a key alone.

Types

type Arg

type Arg struct {
	Name  string
	Value any
}

Arg is the value of the placeholder Name.

A string or an int is rendered as it is, and a float64 in the number format of the language rendering it, under CLDR's default pattern, which keeps three fraction digits at most: that covers everything a generated constructor passes. Any other value is rendered by fmt.

type Bundle

type Bundle struct {
	// contains filtered or unexported fields
}

Bundle holds the catalogs of every language a package of messages was generated from. It never changes once built, so it and the Localizers it hands out can be used from any number of goroutines.

The zero Bundle has no languages, and its Localizers render every message as its key.

func NewBundle

func NewBundle(def Catalog, rest ...Catalog) Bundle

NewBundle builds a Bundle from def, the catalog of the default language, and rest, those of the other languages. It copies what the catalogs hold, so a change to one afterwards changes nothing the Bundle renders.

It panics on a language that is not a valid tag and on two catalogs of one language. Generated code passes neither, so meeting one means a Bundle put together by hand, and failing at once shows the mistake where it was made.

func (Bundle) Localizer

func (b Bundle) Localizer(langs ...string) Localizer

Localizer returns a Localizer for the language of b closest to langs.

Each of langs is a language tag, such as "ja" or "pt-BR", or an Accept-Language value, such as "fr, ja;q=0.8". They are tried in order until one is close to a language of b, as CLDR judges closeness: en-US is served by en, and nb by da when b has Danish but no Norwegian. So a language the user picked can come before the ones the browser asks for. A value that does not parse is skipped as a whole, one bad entry and all, and when nothing matches, the default language is used.

type Catalog

type Catalog struct {
	// Lang is the language as a BCP 47 tag, such as "en" or "pt-BR".
	Lang    string
	Entries map[string]Entry
}

Catalog holds the messages of one language. It and what it is made of, Entry, Plural, Template, and Segment, are what kanna-i18n compiles a locale file into, written into the generated code as literals; the generator has checked them against every other language already, so nothing here checks them again.

type Entry

type Entry struct {
	Single Template
	Plural *Plural
}

Entry is one message of one language: Single holds a message with one form, and Plural, when set, one whose form depends on its count.

type Localizer

type Localizer struct {
	// contains filtered or unexported fields
}

Localizer renders messages in the language a Bundle chose for it. The zero Localizer renders every message as its key.

func (Localizer) Localize

func (l Localizer) Localize(m Message) string

Localize renders m. A message missing from the language of l is looked up in its parent languages and then in the default language, and is rendered under the plural rules and number format of the language it is found in.

Localize never fails: a message no language has renders as its key.

type Message

type Message struct {
	Key  string
	Args []Arg
}

Message is a message to render, with the arguments its placeholders take. The constructors kanna-i18n generates return one, so that code that does not know the reader's language can hand it to code that does.

func Coded

func Coded(prefix, code string, params map[string]any) Message

Coded returns the message keyed prefix.code, such as "errors.messages.too_long", with params as its arguments. It is how an error reported as a code and params, as kanna-validate and kanna-value report one, gets its text from the locale files; FullMessages and ErrorMessages use it for every problem they render. An empty prefix leaves the key code alone.

The arguments come sorted by name, so that one map always makes the same Message. Each renders by its type, as Arg says: the kind a locale file gives a placeholder types only the generated constructor, which Coded bypasses.

type Plural

type Plural struct {
	Zero, One, Two, Few, Many, Other Template
}

Plural holds the forms of a plural message, one for each CLDR plural category the language tells apart. Other is always there, and a count whose category has no form, left nil, renders with it. Zero, when set, is the form of a count of 0 in every language, English and others whose rules have no zero included.

type Problem

type Problem interface {
	Path() string
	Code() string
	Params() map[string]any
}

Problem is a broken rule as a validation reports it: the path of the field it broke on, such as "address.zip" or "tags[1]", or "base" for the value as a whole, and the code and params naming the rule. validate.FieldError of go-kanna/validate is one, and so is anything with these methods: this package imports no validation package.

type Segment

type Segment struct {
	Text  string
	Param string
}

Segment is a run of text or, when Param is set, the placeholder of the argument named Param.

type Template

type Template []Segment

Template is a message split into segments: "Hello, {name}!" is the text "Hello, ", the placeholder name, and the text "!".