llmkit-go

README

llmkit-go

Provider-neutral Go primitives for typed LLM programming.

llmkit-go is a small toolkit for code that wants structured LLM output without taking a dependency on a specific model provider SDK. It focuses on three stable boundaries:

• settle: a bounded stable loop primitive.
• llmschema: Go type to structured output JSON Schema projection and decode.
• llmadapter: prompt plus typed output request/value helpers.

Concrete provider callers live in separate modules. This repository does not own provider transport, provider credentials, prompt libraries, tracing backends, or business validation rules.

Status

This project is usable but still pre-v1. The public API is intentionally small; see API compatibility before depending on it from a library with a strict compatibility policy.

Packages

Package	Purpose	Provider dependencies
github.com/ronhuafeng/llmkit-go/settle	Run an operation until its validator accepts the output or a bounded iteration limit is reached.	Standard library only.
github.com/ronhuafeng/llmkit-go/llmschema	Project Go output types to provider-neutral JSON Schema and decode structured JSON responses.	Uses github.com/google/jsonschema-go.
github.com/ronhuafeng/llmkit-go/llmadapter	Build typed LLM requests and decode typed values behind a tiny Caller interface.	Depends on llmschema; no concrete provider SDK.

The internal/ tree contains repository tests and is not public API.

Installation

Requires Go 1.23 or newer.

go get github.com/ronhuafeng/llmkit-go@latest

Quick Start

settle

Use settle.Run when an operation may need a few bounded attempts before the output is acceptable.

package main

import (
	"context"
	"fmt"
	"strings"

	"github.com/ronhuafeng/llmkit-go/settle"
)

type op struct {
	attempt int
}

func (o *op) Run(ctx context.Context, input string) (string, error) {
	o.attempt++
	if o.attempt == 1 {
		return "draft", nil
	}
	return input + " final", nil
}

func (o *op) Validate(ctx context.Context, input, result string) (bool, error) {
	return strings.Contains(result, input), nil
}

func main() {
	got, err := settle.Run(context.Background(), &op{}, "ship", 3)
	if err != nil {
		panic(err)
	}
	fmt.Println(got)
}

llmschema

Use llmschema when you need the JSON Schema for an expected output type or need to decode the provider's final structured JSON.

package main

import (
	"fmt"

	"github.com/ronhuafeng/llmkit-go/llmschema"
)

type Verdict struct {
	Status string `json:"status" jsonschema:"short final status"`
	Score  int    `json:"score,omitempty"`
}

func main() {
	schema, err := llmschema.SchemaJSONFor[Verdict]()
	if err != nil {
		panic(err)
	}
	fmt.Println(string(schema))

	value, err := llmschema.DecodeString[Verdict](`{"status":"pass","score":2}`)
	if err != nil {
		panic(err)
	}
	fmt.Println(value.Status)
}

llmadapter

Use llmadapter to keep provider-specific transport behind a narrow interface. Your provider caller receives a prompt and schema, then returns the final JSON text to decode.

package main

import (
	"context"
	"fmt"

	"github.com/ronhuafeng/llmkit-go/llmadapter"
)

type staticCaller struct{}

func (staticCaller) Call(ctx context.Context, request llmadapter.Request) (llmadapter.Response, error) {
	// A real caller would send request.Prompt and request.OutputSchema to a provider.
	return llmadapter.Response{FinalResponse: `{"answer":"yes"}`}, nil
}

type Answer struct {
	Answer string `json:"answer"`
}

func main() {
	answer, err := llmadapter.Value[Answer](context.Background(), staticCaller{}, "Return yes.")
	if err != nil {
		panic(err)
	}
	fmt.Println(answer.Answer)
}

For a retry/review loop, build an llmadapter.Op with NewOp and pass it to settle.Run.

API Compatibility

Public API is limited to exported identifiers in these packages:

• settle
• llmschema
• llmadapter

Everything under internal/ is private. README examples are illustrative and may change, but exported package behavior should be treated as compatibility surface.

Before v1.0.0, this project follows SemVer with a conservative pre-v1 policy: patch releases should be bug fixes only, minor releases may add API, and any known breaking API change must be called out in CHANGELOG.md and the release notes. After v1.0.0, breaking public API changes require a new major version.

Versioning

Releases should be tagged as standard Go module tags. For a future release, replace the version below with the next intended version:

VERSION=v0.2.0
git tag -a "$VERSION" -m "llmkit-go $VERSION"
git push origin "$VERSION"

See docs/release.md for the release checklist.

Testing

Run the same checks used by CI:

gofmt -w $(find . -name '*.go' -not -path './vendor/*')
test -z "$(gofmt -l $(find . -name '*.go' -not -path './vendor/*'))"
go vet ./...
go test ./...

If this repository is inside a larger local go.work, use GOWORK=off to verify it as a standalone module:

GOWORK=off go test ./...

Security

Do not report vulnerabilities by opening a public issue with exploit details. Use GitHub private vulnerability reporting:

https://github.com/ronhuafeng/llmkit-go/security/advisories/new

See SECURITY.md for supported versions and disclosure handling.

License and Dependency Provenance

llmkit-go is released under the MIT License. See LICENSE.

Dependency provenance is tracked in THIRD_PARTY_NOTICES.md and should be reviewed before each release.

Contributing

Issues and pull requests are welcome. Please read CONTRIBUTING.md and CODE_OF_CONDUCT.md first.

Related Modules

This repository is intentionally provider-neutral. Provider-specific callers, SDK wrappers, application policy, and business validation should live in separate modules that depend on this one.