jsonrepair

README

jsonrepair-go

English | 中文

English

jsonrepair-go repairs non-standard JSON commonly returned by LLMs into strict JSON that can be decoded with Go's standard encoding/json package.

LLM responses often look like JSON but are not valid JSON: the model may wrap the payload in Markdown, add explanatory prose, omit quotes or commas, use Python-style values, truncate the output, or return a field in a slightly different shape. jsonrepair-go is designed for this recovery layer between model output and typed Go decoding.

Typical LLM Failure Modes

• JSON embedded in a larger natural-language response
• Markdown fenced JSON blocks
• unquoted object keys
• single-quoted strings
• comments
• trailing or leading commas
• missing commas between object properties or array items
• missing colons or missing object values
• Python-style constants: True, False, None
• truncated objects, arrays, strings, or numbers
• light schema drift, such as []string where the target Go type expects []struct{ Command string }

Installation

go get github.com/silaswei-io/jsonrepair-go

Quick Start

package main

import (
	"encoding/json"
	"fmt"

	jsonrepair "github.com/silaswei-io/jsonrepair-go"
)

func main() {
	modelOutput := `
The result is:
```json
{name: 'Ada', active: True,}
```
`

	fragments := jsonrepair.ExtractJSON(modelOutput)
	if len(fragments) == 0 {
		panic("no JSON found")
	}

	var data map[string]any
	if err := json.Unmarshal([]byte(fragments[0]), &data); err != nil {
		panic(err)
	}

	fmt.Println(data["name"])
}

Usage

Repair a JSON-like Fragment

Use Repair when you already have the JSON-like fragment and only need syntax repair.

repaired, err := jsonrepair.Repair(`{name: 'Ada', active: True,}`)

Extract JSON From an LLM Response

Use ExtractJSON when the response may contain Markdown, prose, or multiple JSON-like fragments.

fragments := jsonrepair.ExtractJSON(`
Here is the result:

```json
{name: 'Ada', age: 36,}
```
`)

Extract, Repair, and Unmarshal Into a Go Type

Use UnmarshalJSONFromText when you have a target Go type and want the first matching repaired fragment.

type Person struct {
	Name string `json:"name"`
	Age  int    `json:"age"`
}

var person Person
err := jsonrepair.UnmarshalJSONFromText(`
The answer is:
{name: 'Ada', age: 36,}
`, &person)

Work With Bytes, Readers, and Writers

repairedBytes, err := jsonrepair.RepairBytes(data)
repairedText, err := jsonrepair.RepairReader(r)
err = jsonrepair.RepairToWriter(w, r)

Supported Syntax Repair

Repair focuses on JSON syntax repair. It can handle:

• Markdown fenced JSON blocks
• JavaScript/Python-style strings and constants
• line comments and block comments
• trailing commas and leading commas
• missing commas between object properties and array items
• missing colons between object keys and values
• missing object values, repaired as null
• unquoted object keys
• raw newlines and control characters inside strings
• truncated objects, arrays, strings, and numbers
• special quotes and special whitespace normalization
• escaped JSON string content
• JSONP wrappers
• MongoDB constructors such as ObjectId(...), ISODate(...), NumberLong(...)
• JavaScript string concatenation
• regular expression literals converted to strings
• newline-delimited or comma-separated root values converted into an array

Supported Type Normalization

Type/schema-aware repair is intentionally limited and only available through UnmarshalJSONFromText. It runs after syntax repair and only when direct unmarshaling fails.

Supported normalization includes:

• string to single-string-field struct
• string array to array of single-string-field structs
• nested arrays of the above form
• case-only object key normalization when the target field is unambiguous
• candidate filtering by exact target JSON field names
• candidate filtering through array items when the target type is a slice or array

Example:

type ValidationCommand struct {
	Command string `json:"command"`
}

type Result struct {
	ValidationCommands []ValidationCommand `json:"validation_commands"`
}

var result Result
err := jsonrepair.UnmarshalJSONFromText(`{
	validation_commands: ["go test ./...", "go vet ./..."],
}`, &result)

The model returned validation_commands as []string; the target Go type makes the intended object shape unambiguous, so the field can be normalized to []ValidationCommand.

Boundaries

• Repair repairs syntax only. It does not infer business intent.
• Already valid JSON values are not rewritten by Repair.
• Type normalization is conservative and target-type-driven.
• Case-insensitive matching is used only during fallback normalization and only when it is unambiguous.
• If a conversion would be ambiguous, the package returns the original unmarshaling error instead of guessing.

License

MIT. See LICENSE.

---

中文

jsonrepair-go 用于将大模型常见的非标准 JSON 输出修复为标准 JSON，使其可以继续交给 Go 标准库 encoding/json 解码。

LLM 的响应经常“看起来像 JSON”，但并不是严格合法的 JSON：模型可能把内容包在 Markdown 代码块里，混入解释性文字，漏掉引号或逗号，使用 Python 风格值，输出被截断，或者把某个字段返回成略有不同的形态。jsonrepair-go 解决的是模型输出到 Go 类型解码之间的恢复层问题。

典型 LLM 失败模式

• JSON 混在自然语言响应中
• Markdown fenced JSON 代码块
• 未加引号的对象 key
• 单引号字符串
• 注释
• 尾逗号或前导逗号
• 对象属性或数组元素之间缺失逗号
• 对象 key 和 value 之间缺失冒号
• 对象值缺失
• True、False、None 这类 Python 风格常量
• 截断的对象、数组、字符串或数字
• 轻微 schema 漂移，例如目标 Go 类型需要 []struct{ Command string }，模型返回了 []string

安装

go get github.com/silaswei-io/jsonrepair-go

快速开始

package main

import (
	"encoding/json"
	"fmt"

	jsonrepair "github.com/silaswei-io/jsonrepair-go"
)

func main() {
	modelOutput := `
The result is:
```json
{name: 'Ada', active: True,}
```
`

	fragments := jsonrepair.ExtractJSON(modelOutput)
	if len(fragments) == 0 {
		panic("no JSON found")
	}

	var data map[string]any
	if err := json.Unmarshal([]byte(fragments[0]), &data); err != nil {
		panic(err)
	}

	fmt.Println(data["name"])
}

用法

修复 JSON-like 片段

当你已经拿到 JSON-like 片段，只需要修复语法时，使用 Repair。

repaired, err := jsonrepair.Repair(`{name: 'Ada', active: True,}`)

从 LLM 响应中提取 JSON

当模型响应里可能包含 Markdown、说明文字或多个 JSON-like 片段时，使用 ExtractJSON。

fragments := jsonrepair.ExtractJSON(`
Here is the result:

```json
{name: 'Ada', age: 36,}
```
`)

提取、修复并反序列化到 Go 类型

当你有目标 Go 类型，并希望使用第一个匹配的修复后片段时，使用 UnmarshalJSONFromText。

type Person struct {
	Name string `json:"name"`
	Age  int    `json:"age"`
}

var person Person
err := jsonrepair.UnmarshalJSONFromText(`
The answer is:
{name: 'Ada', age: 36,}
`, &person)

处理 Bytes、Readers 和 Writers

repairedBytes, err := jsonrepair.RepairBytes(data)
repairedText, err := jsonrepair.RepairReader(r)
err = jsonrepair.RepairToWriter(w, r)

支持的语法修复

Repair 专注于 JSON 语法修复，支持处理：

• Markdown fenced JSON 代码块
• JavaScript/Python 风格字符串和常量
• 行注释和块注释
• 尾逗号和前导逗号
• 对象属性、数组元素之间缺失的逗号
• 对象 key 和 value 之间缺失的冒号
• 缺失的对象值，修复为 null
• 未加引号的对象 key
• 字符串中的原始换行和控制字符
• 截断的对象、数组、字符串和数字
• 特殊引号和特殊空白字符归一化
• 已转义 JSON 字符串内容
• JSONP wrapper
• MongoDB 构造器，例如 ObjectId(...)、ISODate(...)、NumberLong(...)
• JavaScript 字符串拼接
• 正则字面量，转换为字符串
• 根级换行分隔或逗号分隔值，转换为数组

支持的类型归一化

类型/schema-aware 修复是刻意收窄的能力，只在 UnmarshalJSONFromText 中提供。它发生在语法修复之后，并且只会在直接反序列化失败时启用。

当前支持：

• string 到单字符串字段 struct
• string array 到单字符串字段 struct array
• 上述形式的嵌套数组
• 目标字段无歧义时的 key 大小写归一化
• 基于目标 JSON 字段名的精确候选过滤
• 当目标类型是 slice 或 array 时，候选过滤可以检查数组元素

示例：

type ValidationCommand struct {
	Command string `json:"command"`
}

type Result struct {
	ValidationCommands []ValidationCommand `json:"validation_commands"`
}

var result Result
err := jsonrepair.UnmarshalJSONFromText(`{
	validation_commands: ["go test ./...", "go vet ./..."],
}`, &result)

模型将 validation_commands 返回成了 []string；目标 Go 类型让期望的对象形态变得明确，因此该字段可以被归一化为 []ValidationCommand。

边界

• Repair 只修复语法，不推断业务含义。
• Repair 不会重写本来已经合法的 JSON 值。
• 类型归一化是保守的，并且由目标 Go 类型驱动。
• 大小写不敏感匹配只在 fallback 归一化阶段使用，并且必须无歧义。
• 如果转换存在歧义，本包会返回原始反序列化错误，而不是猜测。

许可证

MIT。详见 LICENSE。

Documentation

Overview

Package jsonrepair repairs invalid JSON-like text into strict JSON.

The package is intentionally syntax-focused: it fixes malformed JSON-like text commonly returned by humans, scripts, and LLMs, then returns bytes that can be passed to encoding/json. Repair does not infer application schemas or rewrite already-valid values; UnmarshalJSONFromText may use the target Go type to normalize narrow, recoverable schema drift after syntax repair.

Index

• Variables
• func ExtractJSON(text string) []string
• func Repair(text string) (string, error)
• func RepairBytes(data []byte) ([]byte, error)
• func RepairReader(r io.Reader) (string, error)
• func RepairToWriter(w io.Writer, r io.Reader) error
• func UnmarshalJSONFromText(text string, v any) error
• type Error

Variables

var ErrNoJSONCandidate = errors.New("jsonrepair: no JSON candidate found")

ErrNoJSONCandidate is returned when no JSON-like fragment can be repaired and unmarshaled into the target value.

Functions

func ExtractJSON

func ExtractJSON(text string) []string

ExtractJSON finds JSON-like fragments inside text by regular expression and returns repaired JSON for every fragment that can be repaired into syntactically valid JSON.

func Repair

func Repair(text string) (string, error)

Repair repairs common invalid JSON and returns strict JSON.

func RepairBytes

func RepairBytes(data []byte) ([]byte, error)

RepairBytes repairs common invalid JSON and returns strict JSON bytes.

func RepairReader

func RepairReader(r io.Reader) (string, error)

RepairReader reads all data from r, repairs it, and returns strict JSON.

func RepairToWriter

func RepairToWriter(w io.Writer, r io.Reader) error

RepairToWriter reads all data from r, repairs it, and writes strict JSON to w.

func UnmarshalJSONFromText

func UnmarshalJSONFromText(text string, v any) error

UnmarshalJSONFromText finds JSON-like fragments inside text by regular expression, repairs them with this package, and unmarshals the first fragment compatible with v. If direct unmarshaling fails, it may use the target Go type to normalize narrow schema drift such as []string into []struct{Command string}. The target v must be a non-nil pointer accepted by encoding/json.Unmarshal.

Types

type Error

type Error = repair.Error

Error describes a repair failure at a rune offset in the input.