extract

Documentation

Overview

Package extract wraps GNU xgettext (and xgotext for Go) for extracting translatable strings from source files. Supports all languages that xgettext supports plus Go via xgotext.

The package auto-detects source files by extension (and shebang for extensionless scripts), then delegates extraction to the appropriate tool.

Go AST-based string extractor for gettext-style wrapper functions.

This extracts translatable strings from Go source files by scanning the AST for calls to specified functions (e.g. T("..."), N("...", "...", n)). Unlike xgotext, this handles arbitrary wrapper functions, not just direct gotext.Get() calls.

Produces a standard PO/POT file compatible with GNU gettext tools.

Index

• Variables
• func DescribeFiles(files []string) string
• func DetectedLanguages(files []string) []string
• func FileLanguage(path string) string
• func FilesByLanguage(files []string) map[string][]string
• func FindSources(dirs []string) ([]string, error)
• func FindSourcesIn(dir string) ([]string, error)
• func MergePOTFiles(file1, file2, outFile string) error
• func SplitGoFiles(files []string) (goFiles, otherFiles []string)
• func SupportedExtensionsList() []string
• func SupportedLanguagesList() []string
• type ExtractResult
  • func RunGoExtract(dirs []string, potFile, domain string, keywords []string) (*ExtractResult, error)
  • func RunXgettext(files []string, potFile, pkgName, pkgVersion, bugsEmail string, ...) (*ExtractResult, error)
  • func RunXgotext(dirs []string, potFile, domain string) (*ExtractResult, error)
• type GoKeyword
  • func ParseGoKeyword(spec string) GoKeyword

Variables

var SupportedExtensions = map[string]string{
	".py":   "Python",
	".c":    "C",
	".h":    "C",
	".cc":   "C++",
	".cpp":  "C++",
	".cxx":  "C++",
	".hh":   "C++",
	".hpp":  "C++",
	".m":    "ObjectiveC",
	".sh":   "Shell",
	".bash": "Shell",
	".js":   "JavaScript",
	".jsx":  "JavaScript",
	".ts":   "JavaScript",
	".tsx":  "JavaScript",
	".pl":   "Perl",
	".pm":   "Perl",
	".php":  "PHP",
	".java": "Java",
	".cs":   "C#",
	".awk":  "awk",
	".tcl":  "Tcl",
	".el":   "EmacsLisp",
	".scm":  "Scheme",
	".lisp": "Lisp",
	".rb":   "Ruby",
	".lua":  "Lua",
	".vala": "Vala",
	".go":   "Go",
}

SupportedExtensions maps file extensions to xgettext language names. xgettext can also auto-detect from extensions, but explicit mapping lets us know which files to collect.

Functions

func DescribeFiles

func DescribeFiles(files []string) string

DescribeFiles returns a human-readable summary of the source files found.

func DetectedLanguages

func DetectedLanguages(files []string) []string

DetectedLanguages returns the set of programming languages found in the file list.

func FileLanguage

func FileLanguage(path string) string

FileLanguage returns the programming language for a source file, checking the extension first and falling back to shebang detection.

func FilesByLanguage

func FilesByLanguage(files []string) map[string][]string

FilesByLanguage groups source files by their detected language.

func FindSources

func FindSources(dirs []string) ([]string, error)

FindSources recursively finds all source files with known extensions in dirs. Also detects extensionless files by shebang (e.g. bash scripts without .sh). Skips common non-source directories (node_modules, .git, __pycache__, etc.) and nested git repositories (directories containing a .git entry).

func FindSourcesIn

func FindSourcesIn(dir string) ([]string, error)

FindSourcesIn is a convenience function that scans a single directory.

func MergePOTFiles

func MergePOTFiles(file1, file2, outFile string) error

MergePOTFiles merges two POT files into one using msgcat. The result is written to outFile.

func SplitGoFiles

func SplitGoFiles(files []string) (goFiles, otherFiles []string)

SplitGoFiles separates Go files from non-Go files in a file list. Returns (goFiles, otherFiles).

func SupportedExtensionsList

func SupportedExtensionsList() []string

SupportedExtensionsList returns a sorted list of supported file extensions.

func SupportedLanguagesList

func SupportedLanguagesList() []string

SupportedLanguagesList returns a sorted list of unique xgettext language names.

Types

type ExtractResult

type ExtractResult struct {
	// SourceFiles is the list of source files scanned.
	SourceFiles []string
	// Languages is the set of detected programming languages.
	Languages []string
	// POTFile is the path to the generated .pot file.
	POTFile string
}

ExtractResult holds the outcome of an extraction.

func RunGoExtract

func RunGoExtract(dirs []string, potFile, domain string, keywords []string) (*ExtractResult, error)

RunGoExtract scans Go source files for calls matching the given keywords and produces a .pot file in standard gettext format.

Parameters:

• dirs: directories to scan recursively for .go files
• potFile: output .pot file path
• domain: gettext domain name (for POT header)
• keywords: keyword specs (xgettext syntax, e.g. "T", "N:1,2")

Returns an ExtractResult with the list of scanned files and output path.

func RunXgettext

func RunXgettext(files []string, potFile, pkgName, pkgVersion, bugsEmail string, keywords []string, workDir string) (*ExtractResult, error)

RunXgettext runs xgettext on the given source files and produces a .pot file. It auto-detects languages from file extensions (xgettext does this natively).

Parameters:

• files: source files to extract from (must NOT contain .go files)
• potFile: output .pot file path
• pkgName: package name for the POT header
• pkgVersion: package version for the POT header
• bugsEmail: bug report email for the POT header
• keywords: xgettext keyword functions; if empty, defaultKeywords are used
• workDir: working directory for xgettext (empty = current process cwd)

Returns an ExtractResult on success. xgettext warnings are suppressed; only errors cause failure.

func RunXgotext

func RunXgotext(dirs []string, potFile, domain string) (*ExtractResult, error)

RunXgotext runs xgotext on Go source directories to produce a .pot file. xgotext is the string extraction tool for Go projects using the gotext library.

xgotext only accepts a single -in directory, so when multiple directories are provided we find their common ancestor and use -exclude to skip irrelevant subdirectories.

Parameters:

• dirs: directories containing Go source files
• potFile: output .pot file path
• domain: gettext domain name (used as the POT file basename)

Returns an ExtractResult on success.

type GoKeyword

type GoKeyword struct {
	// FuncName is the function name to match (e.g. "T", "N", "Get").
	// Can be a bare name (matches any package) or "pkg.Func" (matches specific selector).
	FuncName string
	// MsgIDArg is the 1-based argument index for msgid (default 1).
	MsgIDArg int
	// PluralArg is the 1-based argument index for plural msgid (0 = none).
	PluralArg int
	// ContextArg is the 1-based argument index for msgctxt (0 = none).
	ContextArg int
}

GoKeyword defines a function call to scan for and how to extract arguments. Follows xgettext --keyword syntax:

"T"       — single-argument: T(msgid)
"N:1,2"   — positional: N(singular, plural, n) — args 1 and 2 are strings
"pgettext:1c,2" — with context: arg 1 is context, arg 2 is msgid

func ParseGoKeyword

func ParseGoKeyword(spec string) GoKeyword

ParseGoKeyword parses an xgettext-style keyword spec into a GoKeyword. Examples:

"T"        → GoKeyword{FuncName:"T", MsgIDArg:1}
"N:1,2"    → GoKeyword{FuncName:"N", MsgIDArg:1, PluralArg:2}
"pgettext:1c,2" → GoKeyword{FuncName:"pgettext", ContextArg:1, MsgIDArg:2}