goplugin

README

Go Plugin System

go-plugin is a go plugin system that combined two most popular golang plugin systems: hashicorp/go-plugin and knqyf263/go-plugin, providing consistent between gRPC and Wasm plugin experience.

Architecture

Plugin

Plugin is packs into a single plg file, essentially a zip file.

Say there is a greeter plugin, the structure is as shown in below

greeter.plg
├── info.yaml
└── Content/

Content is a directory containing all resources used by the plugin, including executable, wasm, and static resources.

info.yaml defines the plugin. It looks like the following.

# Required
Name: com.example/greeter
Version: 1.0.0
Type: grpc             # enum: grpc | wasm
ContractVersion: 1     # host check this for compatibility
Command: $PLUGIN_ROOT/greeter run
# Optional custom metadata
DisplayName: Greeter
Category: demo

In the case of wasm, field Command will be the location of wasm file

You can read an info.yaml file directly with goplugin.ReadInfo. Required fields are mapped onto Info; any other fields are stored in Info.Metadata.

plg file will be extracted to a temporary location everytime before loading the plugin.

$PLUGIN_ROOT will be the location of Content

Host

Plugins ane host are communicated using protobuf3 protocol.

SDKs, or pb files will be generated from proto files. They define interfaces and data structures used to communicate.

Installation

This module uses knqyf263/go-plugin as backend for Wasm. To develop Wasm plugin, you need a compiler.

To develop a plugin, protobuf is required, see documentation for instructions.

And install go module by

go install google.golang.org/protobuf/cmd/protoc-gen-go@latest

Usage

For open box example, see basic example.

Generate interface

If you're not provided pb files (sdk), you need to generate it from proto file, where interfaces are defined.

After that, generate SDK

protoc \
-I. \
--go_out=. --go_opt=paths=source_relative \
--go-grpc_out=. --go-grpc_opt=paths=source_relative \
<my-plugin>.proto

Then import into host

import (
    pb "example.com/my-plugin/proto
)

Initialize manager

Handshake defines some information that will be checked prior to establishing a gRPC connection.

goplugin.HandshakeConfig{
    ProtocolVersion:  1,
    MagicCookieKey:   "GRPC_PLUGIN",
    MagicCookieValue: "hello",
}

Prepare for configs. Here binds services defined in proto

goplugin.GRPCConfig{
    HandshakeConfig: handshake,
    // Set true to prevent the plugin subprocess from inheriting the host environment.
    SkipHostEnv: true,
    Loader: func(_ context.Context, c *grpc.ClientConn) (any, error) {
        return pb.NewPluginClient(c), nil
    },
},

Load the config

mgr, err := goplugin.NewManager(goplugin.Config{
    TempDir: "/tmp",
    GRPC: GRPCConfig,
    WASM: nil,
})

Use plugin

Load the plugin

handle, _ := mgr.Load("my-plugin.plg")
defer mgr.Unload(handle)
// pb.<PluginSDK> is a placeholder, definitions at .proto
client, _ := handle.Client().(pb.<PluginSDK>)

WASM Client is not thread-safe, add a lock

Call plugin methods

resp, _ := client.Hello(ctx, &pb.<Params>{To: "World"})
fmt.Printf("Hello %s", resp.<GetMessage>())

Read plugin resources

Each loaded plugin has a Content root directory. Handle can map resource addresses to this root, so callers can expose a simple read API to plugins.

// Both forms are supported:
// - "/greet.txt"
// - "my-plugin.plg/Content/greet.txt"
data, err := handle.ReadFile("/greet.txt")
if err != nil {
    // handle error
}
fmt.Printf("resource bytes: %d\n", len(data))

Documentation

Index

• type ClientConfig
• type Config
• type GRPCBroker
  • func (b *GRPCBroker) AcceptAndServe(id uint32, register func(*grpc.Server))
  • func (b *GRPCBroker) NextID() uint32
• type GRPCConfig
• type Handle
  • func (h *Handle) Client() any
  • func (h *Handle) Close(ctx context.Context) error
  • func (h *Handle) Info() Info
  • func (h *Handle) PluginPath() string
  • func (h *Handle) ReadFile(resource string) ([]byte, error)
  • func (h *Handle) ResolvePath(resource string) (string, error)
  • func (h *Handle) RootPath() string
• type HandshakeConfig
• type Info
  • func ReadInfo(infoFile string) (Info, error)
• type Manager
  • func NewManager(cfg Config) (*Manager, error)
  • func (m *Manager) Load(path string) (*Handle, error)
  • func (m *Manager) Unload(h *Handle) error
• type Protocol
• type WASMClientConfig
• type WASMConfig

Types

type ClientConfig

type ClientConfig = hcplugin.ClientConfig

type Config

type Config struct {
	TempDir string

	GRPC *GRPCConfig
	WASM *WASMConfig
}

type GRPCBroker

type GRPCBroker struct {
	// contains filtered or unexported fields
}

GRPCBroker wraps hashicorp go-plugin broker for host-side usage.

func (*GRPCBroker) AcceptAndServe

func (b *GRPCBroker) AcceptAndServe(id uint32, register func(*grpc.Server))

AcceptAndServe serves a gRPC server on broker stream id.

func (*GRPCBroker) NextID

func (b *GRPCBroker) NextID() uint32

NextID returns a unique broker stream ID.

type GRPCConfig

type GRPCConfig struct {
	HandshakeConfig HandshakeConfig

	RunAsUser string
	// SkipHostEnv prevents the plugin process from inheriting the host
	// process environment. go-plugin still supplies the environment values
	// required to establish its handshake.
	//
	// It defaults to false to preserve the existing behavior.
	SkipHostEnv bool

	AllowedProtocols []Protocol
	Stderr           io.Writer
	SyncStdout       io.Writer
	SyncStderr       io.Writer

	// Loader is used by the default gRPC preset.
	// If nil, the preset returns *grpc.ClientConn as client.
	Loader func(ctx context.Context, conn *grpc.ClientConn) (any, error)
	// LoaderWithBroker is used by the default gRPC preset.
	// If set, it takes precedence over Loader and receives GRPCBroker.
	LoaderWithBroker func(ctx context.Context, broker *GRPCBroker, conn *grpc.ClientConn) (any, error)

	ClientConfigOverride func(*ClientConfig)
}

type Handle

type Handle struct {
	// contains filtered or unexported fields
}

func (*Handle) Client

func (h *Handle) Client() any

func (*Handle) Close

func (h *Handle) Close(ctx context.Context) error

func (*Handle) Info

func (h *Handle) Info() Info

func (*Handle) PluginPath

func (h *Handle) PluginPath() string

func (*Handle) ReadFile

func (h *Handle) ReadFile(resource string) ([]byte, error)

ReadFile reads bytes from a plugin resource under Content.

func (*Handle) ResolvePath

func (h *Handle) ResolvePath(resource string) (string, error)

ResolvePath maps a plugin resource reference to an absolute file path under Content.

The resource can be one of: - "/greet.txt" - "greet.txt" - "Content/greet.txt" - "<plugin-file>.plg/Content/greet.txt"

func (*Handle) RootPath

func (h *Handle) RootPath() string

RootPath returns the extracted plugin Content directory path.

type HandshakeConfig

type HandshakeConfig struct {
	ProtocolVersion  int
	MagicCookieKey   string
	MagicCookieValue string
}

type Info

type Info struct {
	Name            string         `yaml:"Name"`
	Version         string         `yaml:"Version"`
	Type            string         `yaml:"Type"`
	ContractVersion int            `yaml:"ContractVersion"`
	Command         string         `yaml:"Command"`
	Metadata        map[string]any `yaml:",inline"`
}

func ReadInfo

func ReadInfo(infoFile string) (Info, error)

ReadInfo reads an info.yaml file into Info. Required fields are mapped to Info directly; all other fields are kept in Metadata.

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

func NewManager

func NewManager(cfg Config) (*Manager, error)

func (*Manager) Load

func (m *Manager) Load(path string) (*Handle, error)

func (*Manager) Unload

func (m *Manager) Unload(h *Handle) error

type Protocol

type Protocol string

const (
	ProtocolGRPC Protocol = Protocol(hcplugin.ProtocolGRPC)
)

type WASMClientConfig

type WASMClientConfig struct {
	// NewRuntime creates a new wazero runtime for one plugin instance.
	// It must install any required host modules, such as WASI.
	NewRuntime func(context.Context) (wazero.Runtime, error)

	// ModuleConfig configures the WebAssembly module instance.
	ModuleConfig wazero.ModuleConfig
}

WASMClientConfig contains wazero settings shared with a WASM Loader. A fresh runtime must be created for each loaded plugin.

type WASMConfig

type WASMConfig struct {
	// Loader receives the resolved module path, plugin metadata, and the
	// WASM client configuration after ClientConfigOverride has been applied.
	// It returns:
	// 1) plugin client instance used by caller
	// 2) cleanup function invoked on Unload
	Loader func(ctx context.Context, modulePath string, info Info, clientConfig *WASMClientConfig) (client any, cleanup func(context.Context) error, err error)

	// ClientConfigOverride customizes the wazero runtime and module settings
	// before Loader creates a contract-specific WASM client.
	ClientConfigOverride func(*WASMClientConfig)
}