dblog

README

go-dblog

go-dblog is a multi-module Go toolkit for parsing database change logs. It provides a small common API for orchestration while each database-family backend keeps its own native event model and dependency graph.

中文说明

Modules

Install only the backend you use.

Module	Scope	Status
github.com/Infranite/go-dblog	Common API for multi-source orchestration	Supported
github.com/Infranite/go-dblog/mysql	MySQL-family binlog parser: MySQL, MariaDB, MySQL-compatible dialects	Supported
github.com/Infranite/go-dblog/postgres	PostgreSQL-family logical replication / WAL parser	Planned
github.com/Infranite/go-dblog/mongo	MongoDB-family oplog / change stream parser	Planned
github.com/Infranite/go-dblog/redis	Redis-family AOF / replication stream parser	Planned

Backend modules are split by database family instead of log format names. That keeps imports predictable as each ecosystem grows its own dialects and compatibility layers.

Common API

The root module is intentionally small. It defines the shared event shape used by orchestration code and leaves backend-specific parsing details inside each backend module.

go get github.com/Infranite/go-dblog
go get github.com/Infranite/go-dblog/mysql

package main

import (
	"fmt"

	"github.com/Infranite/go-dblog"
	"github.com/Infranite/go-dblog/mysql/decode/decoder"
)

func main() {
	mysqlDecoder, err := decoder.NewDblogDecoder("./testdata/mysql-bin.000004")
	if err != nil {
		panic(err)
	}
	defer mysqlDecoder.Close()

	for event, err := range dblog.Events(mysqlDecoder) {
		if err != nil {
			panic(err)
		}
		source := dblog.SourceOf(event)
		position := dblog.PositionOf(event)
		fmt.Println(source.Driver, position.Value, event.Kind())
	}
}

Use the common API for multi-source routing, shared filtering, CDC pipelines, and recovery workflows. Use backend-native APIs when you need full database-specific event details.

Backends

The first backend is the MySQL-family module:

go get github.com/Infranite/go-dblog/mysql

See mysql/README.md for installation, examples, compatibility modes, dialect plugins, and supported event tables.

Roadmap

Phase	Scope
1	MySQL-family replication connection reader
2	PostgreSQL-family logical replication backend
3	Shared CDC helpers after at least two real backends exist
4	MongoDB-family oplog / change stream backend
5	Redis-family AOF / replication stream backend
6	Recovery helpers built on stable backend parsers

Development

Run root module tests:

go test ./...

Run MySQL backend tests:

go test ./mysql/...

Contributing

Issues and pull requests are welcome. Keep backend changes focused, preserve native event details, and add parser tests for new log formats.

License

Apache License 2.0. See LICENSE.

Documentation

Overview

Package dblog defines the shared user-facing contracts for database log backends.

Index

• func Bodies[T any](seq iter.Seq2[Event, error]) iter.Seq2[T, error]
• func Events[T Event](decoder Decoder[T]) iter.Seq2[Event, error]
• type Decoder
• type Event
• type Position
  • func PositionOf(event Event) Position
• type Source
  • func SourceOf(event Event) Source

Functions

func Bodies

func Bodies[T any](seq iter.Seq2[Event, error]) iter.Seq2[T, error]

Bodies filters an event iterator by decoded body type.

func Events

func Events[T Event](decoder Decoder[T]) iter.Seq2[Event, error]

Events adapts a typed backend decoder to the common event interface.

Types

type Decoder

type Decoder[T Event] interface {
	Events() iter.Seq2[T, error]
	Close() error
}

Decoder streams events from one database log source.

type Event

type Event interface {
	SourceDriver() string
	SourceName() string
	PositionDriver() string
	PositionString() string
	Kind() string
	Raw() []byte
	Body() any
}

Event is the minimal common shape shared by database log backends.

type Position

type Position struct {
	Driver string
	Value  string
}

Position identifies a backend-specific location in a database log stream.

func PositionOf

func PositionOf(event Event) Position

PositionOf returns the common position value for an event.

type Source

type Source struct {
	Driver string
	Name   string
}

Source identifies the database log producer behind an event stream.

func SourceOf

func SourceOf(event Event) Source

SourceOf returns the common source value for an event.