README
¶
Source · 脚本来源模块
Go Scripts 的脚本加载与变更监听核心接口
概述
Source 模块定义了脚本来源的核心接口，为所有引擎提供统一的脚本加载和热更新能力。它包含三个核心接口、六种内置实现和多个扩展子模块。
核心接口
| 接口 | 说明 |
|---|---|
Reader |
脚本加载接口，Load(ctx, key) 按键加载脚本源码 |
Watcher |
变更监听接口，Watch(ctx, key) 返回变更通知 channel |
ReadWatcher |
Reader + Watcher 组合接口 |
TransformFunc |
源码变换钩子，用于解密/解压/过滤（中间件模式） |
内置实现
| 实现 | 文件 | 热更新机制 | 说明 |
|---|---|---|---|
FileSource |
file.go |
mtime 轮询 | 本地文件系统 |
FileSystemSource |
fs.go |
不支持（不可变源） | 基于 io/fs.FS（embed / zip / os.DirFS） |
MemSource |
memory.go |
内嵌 channel 通知 | 内存存储，用于测试和注入 |
MultiSource |
multiple.go |
转发子 Source 事件 | 多源聚合（Fallback / FirstOK） |
TransformSource |
transform.go |
透传内部源 | 中间件：解密/解压/过滤钩子链 |
CachedSource |
cached.go |
自动失效缓存 | 缓存层：内存缓存 + TTL + Watch 自动失效 |
扩展子模块
| 子模块 | 底层库 | 热更新机制 |
|---|---|---|
| s3 | AWS SDK v2 | ETag 比对轮询 |
| etcd | etcd clientv3 | 原生 Watch API |
| consul | Consul API | ModifyIndex 轮询 |
| redis | go-redis/v9 | 值比对轮询 |
| http | net/http | CRC32 校验和比对 |
| git | go-git/v6 | commit hash 比对轮询 |
| database | database/sql | checksum 列比对轮询 |
接口定义
// Reader - 脚本加载
type Reader interface {
Load(ctx context.Context, key string) (code string, err error)
Close() error
}
// Watcher - 变更监听
type Watcher interface {
Watch(ctx context.Context, key string) (<-chan struct{}, error)
}
// ReadWatcher - 组合接口
type ReadWatcher interface {
Reader
Watcher
}
快速开始
package main
import (
"context"
"fmt"
"os"
"github.com/tx7do/go-scripts/source"
)
func main() {
ctx := context.Background()
// 使用 FileSource
fileSrc := source.NewFileSource()
defer fileSrc.Close()
code, _ := fileSrc.Load(ctx, "/path/to/script.lua")
fmt.Println(code)
// 使用 FileSystemSource（基于 io/fs.FS）
// 适用于 go:embed、archive/zip、os.DirFS 等
fsSrc, _ := source.NewFileSystemSource(os.DirFS("/scripts"))
defer fsSrc.Close()
code, _ = fsSrc.Load(ctx, "hello.lua")
// 使用 MemSource
memSrc := source.NewMemSource()
defer memSrc.Close()
memSrc.Set("hello.lua", `print("hello")`)
code, _ = memSrc.Load(ctx, "hello.lua")
// Fallback 聚合（File → Memory 回退）
multi, _ := source.NewFallbackSource(fileSrc, memSrc)
code, _ = multi.Load(ctx, "backup.lua")
// CachedSource：S3 远程源 + 内存缓存
// cached, _ := source.NewCachedSource(s3Source, source.WithTTL(5*time.Minute))
// defer cached.Close()
// code, _ = cached.Load(ctx, "script.lua")
// TransformSource：解密中间件
// decrypt := source.TransformFunc(func(key, raw string) (string, error) {
// return aesDecrypt(raw, secretKey)
// })
// transformSrc, _ := source.NewTransformSource(cached, decrypt)
// defer transformSrc.Close()
// code, _ = transformSrc.Load(ctx, "script.lua")
}
TransformSource 源码变换中间件
TransformSource 在 Load 之后、返回给引擎之前，对脚本源码执行可配置的变换链。
典型场景：
- 解密：S3 / Git / DB 中存储的 AES/XXTEA 加密脚本
- 解压：gzip / zstd 压缩的脚本
- 过滤：去除敏感信息、编码转换
// 解密钩子
decrypt := source.TransformFunc(func(key string, raw string) (string, error) {
return aesDecrypt(raw, secretKey)
})
// 包装 S3 Source
src, _ := source.NewTransformSource(s3Source, decrypt)
// 链式追加：解密后再解压
src, _ = src.Then(decompress)
CachedSource 缓存层
CachedSource 在远程源（S3 / DB / HTTP）之上添加内存缓存，大幅减少网络 IO。
特性：
- 缓存命中时零网络开销
- 支持 TTL 自动过期（
WithTTL） - 如果远程源实现
Watcher，变更信号自动失效缓存 - 支持手动失效（
Invalidate/InvalidateAll）
// S3 + 内存缓存，TTL 5 分钟
cached, _ := source.NewCachedSource(s3Source, source.WithTTL(5*time.Minute))
// 叠加解密中间件
decrypted, _ := source.NewTransformSource(cached, decrypt)
code, _ := decrypted.Load(ctx, "script.lua")
MultiSource 聚合策略
| 策略 | 构造函数 | 说明 |
|---|---|---|
| Fallback | NewFallbackSource(srcs...) |
按顺序尝试每个 Source，第一个成功即返回 |
| FirstOK | NewFirstOKSource(srcs...) |
并发请求所有 Source，返回最先成功的结果 |
测试
cd source && go test -v ./...
相关文档
- 返回主文档
- S3 Source 文档
- etcd Source 文档
- Consul Source 文档
- Redis Source 文档
- HTTP Source 文档
- Git Source 文档
- Database Source 文档
License
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type CachedOption ¶
type CachedOption func(*CachedSource)
CachedOption configures a CachedSource.
func WithTTL ¶
func WithTTL(ttl time.Duration) CachedOption
WithTTL sets the time-to-live for cached entries. After TTL expires, the next Load fetches from the remote source regardless of Watch status. A zero duration (default) disables TTL-based expiry.
type CachedSource ¶
type CachedSource struct {
// contains filtered or unexported fields
}
CachedSource wraps a remote Reader (e.g. S3, database, HTTP) with an in-memory cache backed by MemSource. On Load, it first checks the cache; only on a cache miss (or after a Watch signal invalidates the entry) does it fetch from the remote source.
This dramatically reduces network round-trips for hot-path script loading, which is critical for game servers and microservices that execute scripts thousands of times per second.
If the remote source implements Watcher, CachedSource automatically starts a background goroutine per key to listen for invalidation signals. When a change is detected, the cached entry is evicted so the next Load fetches the fresh content from the remote.
All exported methods are safe for concurrent use. CachedSource implements the ReadWatcher interface.
func NewCachedSource ¶
func NewCachedSource(remote Reader, opts ...CachedOption) (*CachedSource, error)
NewCachedSource wraps the given remote Reader with an in-memory cache. The remote reader must not be nil.
If the remote implements Watcher, cached entries are automatically invalidated when the remote signals a change. Otherwise, entries persist until TTL expiry (if set) or manual Invalidate.
func (*CachedSource) Close ¶
func (c *CachedSource) Close() error
Close releases the remote reader and stops all background watchers.
func (*CachedSource) Invalidate ¶
func (c *CachedSource) Invalidate(key string)
Invalidate removes a key from the cache, forcing the next Load to fetch from the remote.
func (*CachedSource) InvalidateAll ¶
func (c *CachedSource) InvalidateAll()
InvalidateAll clears the entire cache.
type FSSourceOption ¶
type FSSourceOption func(*FileSystemSource)
FSSourceOption configures a FileSystemSource.
func WithFSPrefix ¶
func WithFSPrefix(prefix string) FSSourceOption
WithFSPrefix sets a prefix prepended to every key before lookup in the underlying fs.FS. Leading slashes are stripped from the prefix; a trailing slash is appended if missing.
Example: WithFSPrefix("scripts") + key "main.lua" -> "scripts/main.lua"
type FileSource ¶
type FileSource struct {
// contains filtered or unexported fields
}
FileSource reads scripts from the local filesystem. It has no extra dependencies and is the default choice for dev/debug.
func (*FileSource) Close ¶
func (fs *FileSource) Close() error
Close is a no-op: FileSource holds no resources that need releasing.
func (*FileSource) Watch ¶
func (fs *FileSource) Watch(ctx context.Context, key string) (<-chan struct{}, error)
Watch returns a channel that signals when the file identified by `key` changes. It polls the file's modification time every second and sends a signal on the channel when a change is detected.
The returned channel is closed when the context is cancelled. Callers should re-Load the script after receiving from the channel.
type FileSystemSource ¶
type FileSystemSource struct {
// contains filtered or unexported fields
}
FileSystemSource reads scripts from any io/fs.FS implementation.
This enables several powerful patterns:
- go:embed: bake scripts into the binary at compile time (zero external file dependencies at runtime).
- archive/zip: read scripts directly from .zip / .pak archives without writing any decompression logic.
- os.DirFS: read from a real directory through the fs.FS abstraction.
- Custom fs.FS implementations (in-memory, remote, etc.).
FileSystemSource implements Reader. It does NOT implement Watcher because most fs.FS implementations (embed.FS, zip.Reader) are immutable. For mutable filesystems with hot-reload support, use FileSource instead.
func NewFileSystemSource ¶
func NewFileSystemSource(fsys fs.FS, opts ...FSSourceOption) (*FileSystemSource, error)
NewFileSystemSource creates a FileSystemSource from the given fs.FS. Returns an error if fsys is nil.
Example with go:embed:
//go:embed scripts/*.lua
var embedFS embed.FS
src, err := NewFileSystemSource(embedFS, WithFSPrefix("scripts"))
func (*FileSystemSource) Close ¶
func (s *FileSystemSource) Close() error
Close is a no-op: the underlying fs.FS is owned by the caller and should be closed (if needed) by its creator.
type MemSource ¶
type MemSource struct {
// contains filtered or unexported fields
}
MemSource keeps scripts in memory. Suitable for dynamic short-lived scripts, unit tests, or RPC-pushed snippets; zero IO overhead.
func (*MemSource) Watch ¶
Watch returns a channel that signals when the script identified by `key` changes. The channel receives a signal whenever Set or Delete is called for that key.
The returned channel is closed when the context is cancelled. Callers should re-Load the script after receiving from the channel.
type MultiSource ¶
type MultiSource struct {
// contains filtered or unexported fields
}
MultiSource aggregates multiple sub-sources under a single Reader interface. The strategy controls how Load selects among them.
func NewFallbackSource ¶
func NewFallbackSource(sources ...Reader) (*MultiSource, error)
NewFallbackSource is a shortcut for NewMultiSource(MultiStrategyFallback, sources...).
func NewFirstOKSource ¶
func NewFirstOKSource(sources ...Reader) (*MultiSource, error)
NewFirstOKSource is a shortcut for NewMultiSource(MultiStrategyFirstOK, sources...).
func NewMultiSource ¶
func NewMultiSource(strategy MultiStrategy, sources ...Reader) (*MultiSource, error)
NewMultiSource creates a MultiSource. At least one sub-source is required.
func (*MultiSource) Close ¶
func (ms *MultiSource) Close() error
Close closes every sub-source and returns the aggregated error.
func (*MultiSource) Watch ¶
func (ms *MultiSource) Watch(ctx context.Context, key string) (<-chan struct{}, error)
Watch delegates to the first sub-source that implements Watcher. It returns an error if no sub-source supports watching.
The strategy is: try each sub-source in order; return the first successful Watch. This allows mixing sources with and without watch support.
type MultiStrategy ¶
type MultiStrategy int
MultiStrategy selects how MultiSource aggregates its sub-sources.
const ( // MultiStrategyFallback tries sub-sources in order; the first success wins and // subsequent sources are skipped. Suitable for "S3 primary + local backup" scenarios. MultiStrategyFallback MultiStrategy = iota // MultiStrategyFirstOK fetches from all sub-sources concurrently and returns the // first successful result. Suitable for low-latency reads across mirrored sources. MultiStrategyFirstOK )
type ReadWatcher ¶
ReadWatcher combines Reader and Watcher into a single interface. Use this when you need both script loading and change notification capabilities from the same source.
type Reader ¶
type Reader interface {
// Load loads the script source code.
Load(ctx context.Context, key string) (code string, err error)
// Close releases underlying resources (s3 client, file handles, etc.).
Close() error
}
Reader represents a script source. key is the unique identifier of a script: path / object key / script id, etc., interpreted by the concrete implementation.
type TransformFunc ¶
TransformFunc is a hook that transforms the raw script source after it is loaded from the underlying Reader but before it is returned to the caller.
Typical use cases:
- Decryption: decrypt AES/XXTEA encrypted scripts stored in S3, DB, etc.
- Decompression: decompress gzip/zstd compressed scripts.
- Validation / sanitization: strip sensitive information.
- Encoding conversion: convert from legacy encoding to UTF-8.
The `key` parameter is the original key passed to Load, useful for determining which transform to apply per-key (e.g., some keys may be encrypted, others not).
If the transform fails, the error propagates to the caller of Load without any fallback.
type TransformSource ¶
type TransformSource struct {
// contains filtered or unexported fields
}
TransformSource wraps a Reader and applies one or more TransformFunc hooks to the loaded source code. Transforms are applied in registration order: the output of transform N becomes the input of transform N+1.
If the wrapped Reader also implements Watcher, TransformSource delegates Watch calls transparently — the transform is only applied to Load, not to Watch signals. Callers should re-Load after receiving a Watch signal to get the freshly transformed content.
All exported methods are safe for concurrent use. TransformSource implements the ReadWatcher interface.
func NewTransformSource ¶
func NewTransformSource(inner Reader, transforms ...TransformFunc) (*TransformSource, error)
NewTransformSource wraps the given Reader with one or more transform functions. At least one transform must be provided.
Transforms are applied in order: if you pass [decrypt, decompress], the raw data is first decrypted, then decompressed.
func (*TransformSource) Close ¶
func (ts *TransformSource) Close() error
Close releases the underlying Reader.
func (*TransformSource) Load ¶
Load fetches the script from the underlying Reader, then applies all registered transforms in sequence.
func (*TransformSource) Then ¶
func (ts *TransformSource) Then(fn TransformFunc) (*TransformSource, error)
Then chains an additional transform to an existing TransformSource. Returns a new TransformSource with the additional transform appended.
type Watcher ¶
type Watcher interface {
// Watch returns a channel that receives a signal whenever the script
// identified by `key` changes. The caller should re-Load the script
// after receiving from the channel.
//
// The returned error indicates whether watching could be established
// (e.g., unsupported key, permission denied). Once Watch succeeds,
// the channel is closed when the context is cancelled or the watcher
// encounters an unrecoverable error.
Watch(ctx context.Context, key string) (<-chan struct{}, error)
}
Watcher defines an optional capability to observe changes for a given key. Implementations that support hot-reload can return a channel that signals when the underlying source has been modified.
Source Files