codecs

Documentation

Overview

Built-in codecs for utf-8, ascii, and latin-1 (iso-8859-1). Each codec is registered at init time so that Lookup("utf-8") etc. resolve without an external search function.

CPython: Lib/encodings/utf_8.py, ascii.py, latin_1.py CPython: Python/codecs.c:L160 PyCodec_Encode / L195 PyCodec_Decode

Error handler registry. Mirrors the Python-level codecs.register_error / codecs.lookup_error pair: handlers are stored by name and called when encode/decode hits an unencodable character or invalid sequence.

CPython: Python/codecs.c:L763 PyCodec_RegisterError

Package codecs is the gopy port of Python/codecs.c. The registry maps codec names to CodecInfo structs; search functions are registered via Register and looked up via Lookup after name normalization (lowercase, hyphens and spaces to underscores).

CPython: Python/codecs.c

Index

• func Decode(input []byte, encoding, errors string) (out string, n int, err error)
• func Encode(input, encoding, errors string) (out []byte, n int, err error)
• func NormalizeName(name string) string
• func Register(fn SearchFunc)
• func RegisterError(name string, fn ErrorHandler)
• type CodecInfo
  • func Lookup(encoding string) (*CodecInfo, error)
• type ErrorHandler
  • func LookupError(name string) (ErrorHandler, error)
• type SearchFunc

Functions

func Decode

func Decode(input []byte, encoding, errors string) (out string, n int, err error)

Decode looks up encoding, then calls its Decode function.

CPython: Python/codecs.c:L195 PyCodec_Decode

func Encode

func Encode(input, encoding, errors string) (out []byte, n int, err error)

Encode looks up encoding, then calls its Encode function.

CPython: Python/codecs.c:L160 PyCodec_Encode

func NormalizeName

func NormalizeName(name string) string

NormalizeName converts an encoding name to the canonical lowercase form with hyphens and spaces replaced by underscores.

CPython: Python/codecs.c:L72 normalizestring

func Register

func Register(fn SearchFunc)

Register appends fn to the codec search path. When Lookup is called it tries each function in order until one returns non-nil.

CPython: Python/codecs.c:L50 PyCodec_Register

func RegisterError

func RegisterError(name string, fn ErrorHandler)

RegisterError registers a named error handler. Overwrites any prior handler registered under the same name.

CPython: Python/codecs.c:L763 PyCodec_RegisterError

Types

type CodecInfo

type CodecInfo struct {
	Name   string
	Encode func(input string, errors string) ([]byte, int, error)
	Decode func(input []byte, errors string) (string, int, error)
}

CodecInfo holds the four callables that implement a codec.

CPython: Modules/_codecsmodule.c:L34 codec_info_new

func Lookup

func Lookup(encoding string) (*CodecInfo, error)

Lookup finds the codec for encoding. The name is normalized before the search path is consulted; a cached hit skips the search.

CPython: Python/codecs.c:L99 _PyCodec_Lookup

type ErrorHandler

type ErrorHandler func(enc string, input []byte, start, end int) (replacement string, newpos int, err error)

ErrorHandler is a function that handles an encode or decode error. It receives the position of the bad input and returns a replacement string plus the new position to resume from.

CPython: Python/codecs.c:L793 call_codec_error_handler

func LookupError

func LookupError(name string) (ErrorHandler, error)

LookupError returns the handler registered under name.

CPython: Python/codecs.c:L793 PyCodec_LookupError

type SearchFunc

type SearchFunc func(name string) (*CodecInfo, error)

SearchFunc is a callable that receives a normalized codec name and returns a CodecInfo or nil if the codec is unknown.

CPython: Python/codecs.c:L50 codec_register