README
¶
Parser Package
Package parser provides configuration parsing utilities for various file formats and sources.
Overview
The parser package offers a unified interface for parsing configuration from multiple sources including JSON, YAML, TOML, INI, XML files, environment variables, and command-line arguments. All parsers normalize keys and values using the normalize package and return a flattened map[string]string representation.
Supported Formats
JSON Parser
parser := NewJSON("config.json")
config, err := parser.Load()
- Supports .json extension
- Flattens nested objects with dot notation
- Arrays are indexed (e.g.,
array.0,array.1)
YAML Parser
parser := NewYAML("config.yaml")
config, err := parser.Load()
- Supports .yaml and .yml extensions
- Handles all YAML data types
- Supports anchors and aliases
TOML Parser
parser := NewTOML("config.toml")
config, err := parser.Load()
- Supports .toml extension
- Handles tables, arrays, and inline tables
- Preserves TOML date/time types as strings
XML Parser
parser := NewXML("config.xml")
config, err := parser.Load()
- Supports .xml extension
- Attributes stored as
element.attribute - Repeated elements indexed automatically
- Handles CDATA sections
INI Parser
parser := NewINI("config.ini")
config, err := parser.Load()
- Supports .ini, .cfg, and .conf extensions
- Section keys prefixed with section name
- Supports both
=and:separators - Comments with
#or;prefixes
Environment Variables
parser := NewENV("APP_")
config, err := parser.Load()
- Optional prefix filtering
- Strips prefix from keys when filtered
- Normalizes keys (uppercase to lowercase, underscore to dot)
Command-Line Arguments
parser := NewARGS(true) // Skip first arg (program name)
config, err := parser.Load()
- Supports
--key=valueand--key valueformats - Boolean flags set to "true"
- Single and double dash prefixes
Common Methods
All parsers implement these methods:
Type() string- Returns the parser type identifierLoad() (map[string]string, error)- Loads and parses from the configured sourceLoadReader(r io.Reader) (map[string]string, error)- Parses from an io.ReaderLoadBytes(data []byte) (map[string]string, error)- Parses from byte slice
Error Handling
The package uses the kerror package for structured error handling. Common errors include:
ErrInvalidExtension- File has wrong extensionErrFileNotFound- File doesn't existErrReadFailed- IO error during readErrJSONParse,ErrYAMLParse, etc. - Format-specific parse errors
Usage Examples
Loading a JSON Configuration
package main
import (
"fmt"
"log"
"github.com/kitsunium/sdk/pkg/core/config/parser"
)
func main() {
p := parser.NewJSON("config.json")
config, err := p.Load()
if err != nil {
log.Fatal(err)
}
// Access nested values with dot notation
dbHost := config["database.host"]
dbPort := config["database.port"]
fmt.Printf("Database: %s:%s\n", dbHost, dbPort)
}
Parsing Environment Variables with Prefix
p := parser.NewENV("MYAPP_")
config, err := p.Load()
// MYAPP_DATABASE_URL becomes "database.url" in config
Parsing Command-Line Arguments
p := parser.NewARGS(true) // Skip program name
config, err := p.Load()
// --database-url=localhost becomes {"database.url": "localhost"}
// --verbose becomes {"verbose": "true"}
Loading from Reader
file, err := os.Open("config.yaml")
if err != nil {
log.Fatal(err)
}
defer file.Close()
p := parser.NewYAML("")
config, err := p.LoadReader(file)
Parser Options
Some parsers support options via the ParserOption type:
parser := NewJSON("config.json",
WithBufferSize(16384),
WithPool(false),
)
Available options:
WithBufferSize(size int)- Set buffer size for readingWithPool(enabled bool)- Enable/disable buffer pooling
Key Normalization
All parsers automatically normalize keys:
- Uppercase letters become lowercase
- Underscores become dots
DATABASE_URL→database.url
Value Processing
Values are normalized to:
- Trim surrounding whitespace
- Remove matching quotes
- Convert non-string types to string representation
Documentation
¶
Overview ¶
Package parser provides configuration parsing for command-line arguments.
Package parser provides error definitions for configuration parsers. All errors are defined using the kerror package for enhanced tracking and observability.
Package parser provides configuration parsing for INI files.
Package parser provides a flexible configuration parsing framework with support for multiple formats including JSON, YAML, TOML, XML, INI, environment variables, and command-line arguments.
The package defines common interfaces for all parsers and provides implementations that normalize configuration data into flat key-value maps with dot-notation keys. This design allows for easy configuration merging from multiple sources and consistent access patterns regardless of the original format.
Example usage:
// Parse JSON configuration
jsonParser := parser.NewJSON("config.json")
config, err := jsonParser.Load()
// Parse environment variables
envParser := parser.NewEnv(parser.WithPrefix("APP_"))
envConfig, err := envParser.Load()
// Parse command-line arguments
argsParser := parser.NewArgs(os.Args[1:])
argsConfig, err := argsParser.Load()
Package parser provides configuration parsing for TOML files.
Package parser provides configuration parsing for XML files.
Package parser provides configuration parsing for YAML files.
Index ¶
Constants ¶
const ( // MaxJSONSize defines the maximum size of JSON input (10MB) MaxJSONSize = 10 * 1024 * 1024 // MaxJSONDepth defines maximum nesting depth to prevent stack overflow MaxJSONDepth = 100 )
Variables ¶
var ( // ErrInvalidExtension is returned when a file has an unsupported extension. // Each parser validates specific extensions (e.g., .json, .yaml, .toml). ErrInvalidExtension = kerror.Define(kerror.KConfig{ Code: 1001, Message: "invalid file extension", }) // ErrFileNotFound is returned when a configuration file does not exist. // This error wraps the underlying OS error for additional context. ErrFileNotFound = kerror.Define(kerror.KConfig{ Code: 1002, Message: "file not found", }) // ErrReadFailed is returned when a file or reader cannot be read. // This typically occurs due to I/O errors or permission issues. ErrReadFailed = kerror.Define(kerror.KConfig{ Code: 1003, Message: "failed to read file", }) // ErrJSONParse is returned when JSON content is malformed or invalid. // The error wraps the underlying json.Unmarshal error for details. ErrJSONParse = kerror.Define(kerror.KConfig{ Code: 1010, Message: "failed to parse JSON", }) // ErrYAMLParse is returned when YAML content is malformed or invalid. // Supports both .yaml and .yml file extensions. ErrYAMLParse = kerror.Define(kerror.KConfig{ Code: 1020, Message: "failed to parse YAML", }) // ErrTOMLParse is returned when TOML content is malformed or invalid. // Uses the pelletier/go-toml/v2 library for parsing. ErrTOMLParse = kerror.Define(kerror.KConfig{ Code: 1030, Message: "failed to parse TOML", }) // ErrXMLParse is returned when XML content is malformed or invalid. // Handles attributes, nested elements, and CDATA sections. ErrXMLParse = kerror.Define(kerror.KConfig{ Code: 1040, Message: "failed to parse XML", }) // ErrINIParse is returned when INI content is malformed or invalid. // Supports .ini, .cfg, and .conf file extensions. ErrINIParse = kerror.Define(kerror.KConfig{ Code: 1050, Message: "failed to parse INI", }) // ErrENVParse is returned when environment variables cannot be parsed. // Supports optional prefix filtering for variable selection. ErrENVParse = kerror.Define(kerror.KConfig{ Code: 1060, Message: "failed to parse environment variable", }) // ErrARGSParse is returned when command-line arguments cannot be parsed. // Supports both --key=value and --key value formats. ErrARGSParse = kerror.Define(kerror.KConfig{ Code: 1070, Message: "failed to parse arguments", }) // ErrARGSInvalid is returned when arguments don't follow expected format. // In strict mode, all arguments must start with - or --. ErrARGSInvalid = kerror.Define(kerror.KConfig{ Code: 1071, Message: "invalid argument format", }) )
Error definitions for the parser package. These errors provide structured error handling with unique codes for each error type.
Functions ¶
This section is empty.
Types ¶
type ARGS ¶
type ARGS struct {
SkipFirst bool // Skip first argument (usually program name)
}
ARGS is a parser for command-line arguments. It converts command-line flags into a key-value configuration map.
Supported formats:
- --key=value or -key=value
- --key value or -key value
- --flag (boolean, sets to "true")
- Double dash (--) and single dash (-) prefixes
func NewARGS ¶
NewARGS creates a new command-line argument parser.
Parameters:
- skipFirst: Skip the first argument (typically the program name)
Example:
// Parse args skipping program name
parser := NewARGS(true)
config, _ := parser.Load()
// --database-url=localhost becomes {"database.url": "localhost"}
// --verbose becomes {"verbose": "true"}
func (*ARGS) Load ¶
Load parses command-line arguments from os.Args. Automatically handles the skipFirst setting.
Returns a map where:
- Keys are normalized (lowercase, dash/underscore to dot)
- Boolean flags are set to "true"
- Values are preserved as-is
func (*ARGS) ParseArgs ¶
ParseArgs parses a specific list of arguments. This is the core parsing method used by Load().
Handles:
- Key=value format
- Key value format (space-separated)
- Boolean flags (no value means "true")
- Single and double dash prefixes
func (*ARGS) ParseArgsStrict ¶
ParseArgsStrict parses arguments with strict validation. All arguments must start with - or -- prefix.
Returns ErrARGSInvalid if any argument doesn't follow the expected format. This is useful for CLI tools that require all arguments to be flags.
type ENV ¶
type ENV struct {
Prefix string // Optional prefix to filter environment variables
}
ENV is a parser for environment variables. It can optionally filter variables by prefix.
func NewENV ¶
NewENV creates a new environment variable parser. If prefix is not empty, only variables starting with that prefix are included.
func (*ENV) LoadFiltered ¶
LoadFiltered parses environment variables with a custom filter function. The filter receives the full KEY=VALUE string and returns true to include it.
Example:
parser := NewENV("")
config, _ := parser.LoadFiltered(func(env string) bool {
return strings.HasPrefix(env, "DB_")
})
type FileParser ¶
type FileParser interface {
Parser
// LoadReader parses configuration from an io.Reader source.
LoadReader(r io.Reader) (map[string]string, error)
}
FileParser extends Parser with the ability to parse from io.Reader. This interface is implemented by file-based parsers (JSON, YAML, TOML, XML, INI) and allows for parsing from various sources like files, network streams, or buffers.
type INI ¶
type INI struct {
Path string
// contains filtered or unexported fields
}
INI is a parser for INI configuration files. It supports .ini, .cfg, and .conf file extensions.
The parser handles:
- Sections [section]
- Key-value pairs with = or : separators
- Comments with # or ; prefixes
- Quoted values (single or double quotes)
- Whitespace trimming
- Section-prefixed keys (section.key)
func NewINI ¶
func NewINI(path string, opts ...ParserOption) *INI
NewINI creates a new INI parser instance.
Parameters:
- path: Path to the INI file to parse
- opts: Optional parser configuration options
Example:
parser := NewINI("config.ini")
config, err := parser.Load()
if err != nil {
log.Fatal(err)
}
func (*INI) Load ¶
Load reads and parses an INI file from disk. Validates that the file has a .ini, .cfg, or .conf extension.
Returns a flattened map where section keys are prefixed:
- [database] host=localhost becomes {"database.host": "localhost"}
- Global keys (before any section) have no prefix
Returns an error if:
- The file extension is not .ini, .cfg, or .conf
- The file cannot be read
func (*INI) LoadBytes ¶
LoadBytes parses INI from a byte slice. Implements a single-pass parser with direct byte manipulation.
func (*INI) LoadReader ¶
LoadReader parses INI from an io.Reader. This method reads all data into memory before parsing.
type JSON ¶
type JSON struct {
Path string
// contains filtered or unexported fields
}
JSON implements a high-performance JSON configuration parser that flattens nested structures into a flat key-value map with dot-separated keys. It supports all JSON types including arrays, nested objects, and preserves numeric precision using json.Number.
Features:
- Zero-copy parsing with bytes.Reader
- Stack-based iteration to avoid recursion overhead
- Pre-allocated data structures for performance
- Protection against malicious inputs (size and depth limits)
- Precise number handling without floating-point errors
Example:
parser := NewJSON("config.json")
config, err := parser.Load()
if err != nil {
log.Fatal(err)
}
// Access nested values with dot notation
dbHost := config["database.host"]
port := config["server.port"]
func NewJSON ¶
func NewJSON(path string, opts ...ParserOption) *JSON
NewJSON creates a new JSON parser instance.
Options can be provided to customize the parser behavior:
- WithBufferSize(size): Set the buffer size for reading (default: 8192)
- WithPool(enabled): Enable/disable buffer pooling (default: false)
Example:
parser := NewJSON("config.json", WithBufferSize(16384))
func (*JSON) Load ¶
Load reads and parses a JSON file from disk. The file path must have a .json extension.
Returns a flattened map where nested keys are joined with dots:
- {"a": {"b": "c"}} becomes {"a.b": "c"}
- Arrays are indexed: {"arr": [1, 2]} becomes {"arr.0": "1", "arr.1": "2"}
Returns an error if:
- The file extension is not .json
- The file cannot be read
- The JSON is malformed
func (*JSON) LoadBytes ¶
LoadBytes parses JSON from a byte slice.
This method:
- Validates JSON size to prevent DoS attacks
- Unmarshals JSON into a map structure
- Flattens nested structures using a stack-based approach
- Normalizes all keys to lowercase with dots instead of underscores
- Converts all values to strings
Example:
data := []byte(`{"db": {"host": "localhost"}}`)
config, err := parser.LoadBytes(data)
// config["db.host"] == "localhost"
func (*JSON) LoadReader ¶
LoadReader parses JSON from an io.Reader.
This method reads all data from the reader into memory before parsing. For large files, consider using Load() which reads directly from disk.
type Parser ¶
type Parser interface {
// Type returns a unique identifier for the parser type (e.g., "json", "yaml").
Type() string
// Load parses configuration and returns a flattened key-value map.
// Keys are normalized to lowercase with dots as separators.
Load() (map[string]string, error)
}
Parser is the base interface for all configuration parsers. Implementations must provide a Type identifier and a Load method that returns normalized configuration as a flat key-value map.
type ParserOption ¶
type ParserOption func(*baseParser)
ParserOption is a functional option for configuring parser behavior. Options can be passed to parser constructors to customize their operation.
func WithBufferSize ¶
func WithBufferSize(size int) ParserOption
WithBufferSize sets the buffer size for reading operations. Larger buffers can improve performance for large files by reducing the number of read syscalls. Consider increasing for files > 1MB.
Default: 8192 bytes Recommended: 8192 for small files, 65536 for large files
func WithPool ¶
func WithPool(enabled bool) ParserOption
WithPool enables or disables buffer pooling for memory reuse. Note: Buffer pooling has been found to decrease performance in most cases due to synchronization overhead and is disabled by default. This option is maintained for backward compatibility only.
Default: false Deprecated: Buffer pooling is not recommended for general use
type TOML ¶
type TOML struct {
Path string
// contains filtered or unexported fields
}
TOML is a parser for TOML (Tom's Obvious, Minimal Language) configuration files. It flattens nested structures into a dot-separated key-value map.
The parser handles all TOML data types including:
- Basic strings and literal strings
- Integers, floats, booleans
- Dates and times
- Arrays and inline tables
- Nested tables and table arrays
func NewTOML ¶
func NewTOML(path string, opts ...ParserOption) *TOML
NewTOML creates a new TOML parser instance.
Parameters:
- path: Path to the TOML file to parse
- opts: Optional parser configuration options
Example:
parser := NewTOML("config.toml")
config, err := parser.Load()
if err != nil {
log.Fatal(err)
}
func (*TOML) Load ¶
Load reads and parses a TOML file from disk. Validates that the file has a .toml extension.
Returns a flattened map where nested keys are joined with dots:
- [database] host = "localhost" becomes {"database.host": "localhost"}
- Arrays are indexed: servers = ["a", "b"] becomes {"servers.0": "a", "servers.1": "b"}
Returns an error if:
- The file extension is not .toml
- The file cannot be read
- The TOML is malformed
func (*TOML) LoadBytes ¶
LoadBytes parses TOML from a byte slice. Uses the pelletier/go-toml/v2 library for parsing and normalize.Map for flattening.
func (*TOML) LoadReader ¶
LoadReader parses TOML from an io.Reader. This method reads all data into memory before parsing.
type XML ¶
type XML struct {
Path string
// contains filtered or unexported fields
}
XML is a parser for XML configuration files. It flattens nested XML structures into a dot-separated key-value map.
The parser handles:
- Elements and nested elements
- Attributes (stored as element.attribute)
- Text content
- CDATA sections
- Empty and self-closing tags
- Repeated elements (indexed with .0, .1, etc.)
func NewXML ¶
func NewXML(path string, opts ...ParserOption) *XML
NewXML creates a new XML parser instance.
Parameters:
- path: Path to the XML file to parse
- opts: Optional parser configuration options
Example:
parser := NewXML("config.xml")
config, err := parser.Load()
if err != nil {
log.Fatal(err)
}
func (*XML) Load ¶
Load reads and parses an XML file from disk. Validates that the file has a .xml extension.
Returns a flattened map where:
- Nested elements become dot-separated keys
- Attributes are suffixed to element keys
- Repeated elements are indexed
Example XML:
<config> <database host="localhost" port="5432"/> </config>
Becomes:
{"config.database.host": "localhost", "config.database.port": "5432"}
Returns an error if:
- The file extension is not .xml
- The file cannot be read
- The XML is malformed
func (*XML) LoadBytes ¶
LoadBytes parses XML from a byte slice. Uses a stack-based approach to handle nested elements.
func (*XML) LoadReader ¶
LoadReader parses XML from an io.Reader. This method reads all data into memory before parsing.
type YAML ¶
type YAML struct {
Path string
// contains filtered or unexported fields
}
YAML is a parser for YAML configuration files. It supports both .yaml and .yml file extensions and flattens nested structures into a dot-separated key-value map.
The parser handles all YAML data types including:
- Scalars (strings, numbers, booleans)
- Maps and nested maps
- Arrays and sequences
- Anchors and aliases
- Multi-line strings
func NewYAML ¶
func NewYAML(path string, opts ...ParserOption) *YAML
NewYAML creates a new YAML parser instance.
Parameters:
- path: Path to the YAML file to parse
- opts: Optional parser configuration options
Example:
parser := NewYAML("config.yaml")
config, err := parser.Load()
if err != nil {
log.Fatal(err)
}
func (*YAML) Load ¶
Load reads and parses a YAML file from disk. Validates that the file has a .yaml or .yml extension.
Returns a flattened map where nested keys are joined with dots:
- {"database": {"host": "localhost"}} becomes {"database.host": "localhost"}
- Arrays are indexed: {"servers": ["a", "b"]} becomes {"servers.0": "a", "servers.1": "b"}
Returns an error if:
- The file extension is not .yaml or .yml
- The file cannot be read
- The YAML is malformed
func (*YAML) LoadBytes ¶
LoadBytes parses YAML from a byte slice. Uses the yaml.v3 library for parsing and normalize.Map for flattening.
func (*YAML) LoadReader ¶
LoadReader parses YAML from an io.Reader. This method reads all data into memory before parsing.
Source Files