dblog

package module
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 2, 2026 License: Apache-2.0 Imports: 6 Imported by: 0

README

go-dblog

CI Go Version Go Reference Go Report Card License

go-dblog is a multi-module Go toolkit for parsing database change logs. The root module defines shared event, registry, checkpoint, filtering, and safe flashback/recovery contracts; each backend keeps product-specific parsing, live reading, typed events, and extension hooks in its own module.

中文

Product Index

Install only the backend you use.

Product Module README Features Examples
Common API github.com/Infranite/go-dblog This README Roadmap scope Minimal example below
MySQL family github.com/Infranite/go-dblog/mysql English / 中文 English / 中文 English / 中文
PostgreSQL family github.com/Infranite/go-dblog/postgres English / 中文 English / 中文 English / 中文
MongoDB family github.com/Infranite/go-dblog/mongo English / 中文 English / 中文 English / 中文
Redis family github.com/Infranite/go-dblog/redis English / 中文 English / 中文 English / 中文

Core Features

  • Backend-neutral dblog.Event shape for mixed database log streams.
  • Explicit backend registration through dblog.Registry.
  • Streaming decoders built on Go iterator APIs.
  • Shared source metadata, position, checkpoint resume, filtering, and safe flashback/recovery helpers.
  • Backend-native typed events for database-specific fields.
  • Plugin hooks inside backend decoder packages for compatible dialects and product-specific records.
  • Separate Go modules so callers do not install unused database dependencies.

Install

The current public tag set is v0.4.0.

go get github.com/Infranite/go-dblog@v0.4.0
go get github.com/Infranite/go-dblog/mysql@v0.4.0
go get github.com/Infranite/go-dblog/postgres@v0.4.0
go get github.com/Infranite/go-dblog/mongo@v0.4.0
go get github.com/Infranite/go-dblog/redis@v0.4.0

Minimal Example

package main

import (
	"fmt"
	"strings"

	"github.com/Infranite/go-dblog"
	"github.com/Infranite/go-dblog/redis"
)

func main() {
	var registry dblog.Registry
	if err := redis.Register(&registry); err != nil {
		panic(err)
	}

	decoder, err := registry.Open(redis.Driver,
		dblog.WithReader(strings.NewReader("*2\r\n$4\r\nINCR\r\n$7\r\ncounter\r\n")),
	)
	if err != nil {
		panic(err)
	}
	defer decoder.Close()

	for event, err := range decoder.Events() {
		if err != nil {
			panic(err)
		}
		fmt.Println(event.Kind(), dblog.PositionOf(event).Value)
	}
}

Use the common API for multi-source routing, shared filtering, CDC pipelines, backend registration, and recovery tasks. Use backend-native APIs when you need database-specific event fields.

Project Docs

Topic English 中文
Project overview This README doc/README.zh-CN.md
Recovery cookbook doc/RECOVERY.md doc/RECOVERY.zh-CN.md
CI evidence doc/CI.md doc/CI.zh-CN.md
Roadmap and product scope doc/ROADMAP.md doc/ROADMAP.zh-CN.md
Development and contribution flow doc/DEVELOPMENT.md doc/DEVELOPMENT.zh-CN.md
Security policy doc/SECURITY.md doc/SECURITY.zh-CN.md

GitHub Releases and git tags are the public release record. Git history is the detailed change log; this repository does not maintain separate release notes or changelog files.

License

Apache License 2.0. See LICENSE.

Documentation

Overview

Package dblog defines the shared user-facing contracts for database log backends.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrInvalidBackend is returned when a backend does not expose a driver.
	ErrInvalidBackend = errors.New("invalid backend")
	// ErrBackendExists is returned when a registry already has a driver.
	ErrBackendExists = errors.New("backend already registered")
	// ErrBackendNotFound is returned when a driver is not registered.
	ErrBackendNotFound = errors.New("backend not found")
)

Functions

func Bodies 

func Bodies[T any](seq iter.Seq2[Event, error]) iter.Seq2[T, error]

Bodies filters an event iterator by decoded body type.

func ContextOf 

func ContextOf(options OpenOptions) context.Context

ContextOf returns the context carried by open options, or context.Background.

func Events 

func Events[T Event](decoder Decoder[T]) iter.Seq2[Event, error]

Events adapts a typed backend decoder to the common event interface.

func Filter 

func Filter(seq iter.Seq2[Event, error], predicates ...Predicate) iter.Seq2[Event, error]

Filter yields events accepted by every predicate.

func Flashbacks 

func Flashbacks(seq iter.Seq2[Event, error]) iter.Seq2[any, error]

Flashbacks yields compensating operations from reversible events.

func RecoveryPlan added in v0.3.0 

func RecoveryPlan(seq iter.Seq2[Event, error]) iter.Seq2[RecoveryStep, error]

RecoveryPlan yields safe compensating operations with checkpoints.

func Register 

func Register(backend Backend) error

Register adds a backend to DefaultRegistry.

Types

type Backend 

type Backend interface {
	Driver() string
	Open(OpenOptions) (Decoder[Event], error)
}

Backend opens decoders for one database log driver.

type Checkpoint 

type Checkpoint struct {
	Source   Source
	Position Position
}

Checkpoint stores the last consumed event location for a source.

func CheckpointOf 

func CheckpointOf(event Event) Checkpoint

CheckpointOf returns a portable checkpoint for the event.

type Decoder 

type Decoder[T Event] interface {
	Events() iter.Seq2[T, error]
	Close() error
}

Decoder streams events from one database log source.

func Open 

func Open(driver string, opts ...OpenOption) (Decoder[Event], error)

Open creates a decoder from DefaultRegistry.

type Event 

type Event interface {
	SourceDriver() string
	SourceName() string
	PositionDriver() string
	PositionString() string
	Kind() string
	Raw() []byte
	Body() any
}

Event is the minimal common shape shared by database log backends.

type OpenOption 

type OpenOption func(*openOptions)

OpenOption configures backend decoder creation.

func WithCheckpoint 

func WithCheckpoint(checkpoint Checkpoint) OpenOption

WithCheckpoint resumes an opened backend after a previously consumed event.

func WithContext 

func WithContext(ctx context.Context) OpenOption

WithContext sets the cancellation context used by context-aware backends.

func WithDSN 

func WithDSN(dsn string) OpenOption

WithDSN sets a backend-specific data source name.

func WithPath 

func WithPath(path string) OpenOption

WithPath opens a backend from a local path when supported.

func WithReader 

func WithReader(reader io.Reader) OpenOption

WithReader opens a backend from a stream when supported.

func WithSource 

func WithSource(source Source) OpenOption

WithSource sets the source metadata for opened events.

type OpenOptions 

type OpenOptions interface {
	Source() Source
	Path() string
	DSN() string
	Reader() io.Reader
}

OpenOptions exposes backend decoder creation options to backend packages.

type Position 

type Position struct {
	Driver string
	Value  string
}

Position identifies a backend-specific location in a database log stream.

func PositionOf 

func PositionOf(event Event) Position

PositionOf returns the common position value for an event.

func StartPositionOf 

func StartPositionOf(options OpenOptions) Position

StartPositionOf returns the checkpoint position carried by open options.

type Predicate 

type Predicate func(Event) bool

Predicate filters common events.

func ByDriver 

func ByDriver(driver string) Predicate

ByDriver matches the event source driver.

func ByKind 

func ByKind(kind string) Predicate

ByKind matches the backend event kind.

func BySource 

func BySource(driver, name string) Predicate

BySource matches the event source driver and name.

type RecoveryStep added in v0.3.0 

type RecoveryStep struct {
	Checkpoint Checkpoint
	Operation  any
}

RecoveryStep is one safe compensating operation with its source checkpoint.

type Registry 

type Registry struct {
	// contains filtered or unexported fields
}

Registry stores backend drivers. 

var DefaultRegistry Registry

DefaultRegistry is used by package-level Register and Open.

func (*Registry) Open 

func (r *Registry) Open(driver string, opts ...OpenOption) (Decoder[Event], error)

Open creates a decoder for a registered driver.

func (*Registry) Register 

func (r *Registry) Register(backend Backend) error

Register adds a backend driver to the registry.

type Reversible 

type Reversible interface {
	Reverse() (any, bool)
}

Reversible is implemented by events that can emit a compensating operation.

type SeqDecoder 

type SeqDecoder struct {
	// contains filtered or unexported fields
}

SeqDecoder adapts an event iterator to Decoder.

func NewSeqDecoder 

func NewSeqDecoder(seq iter.Seq2[Event, error], close func() error) *SeqDecoder

NewSeqDecoder creates a decoder from an event iterator.

func (*SeqDecoder) Close 

func (d *SeqDecoder) Close() error

Close releases decoder resources.

func (*SeqDecoder) Events 

func (d *SeqDecoder) Events() iter.Seq2[Event, error]

Events returns the wrapped event iterator.

type Source 

type Source struct {
	Driver string
	Name   string
}

Source identifies the database log producer behind an event stream.

func SourceOf 

func SourceOf(event Event) Source

SourceOf returns the common source value for an event.

Source Files

View all Source files

Directories

Path Synopsis
internal
cireport
cmd/cireport command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.