bento-grpc-plugin

README

Bento gRPC Plugin (Go)

This plugin adds minimal gRPC input/output components for Bento using its public service API.

• Input grpc: subscribes to a server-stream via Subscribe and yields frames as Bento messages.
• Output grpc: sends messages via a client-stream Publish.

Build

Prereqs:

• Go 1.21+
• protoc, protoc-gen-go, protoc-gen-go-grpc in PATH

Commands:

make proto
make build

Usage

Run the binary as a Bento plugin CLI:

go run ./cmd/bento-grpc-plugin --help

Example config using the input:

input:
  grpc:
    address: 127.0.0.1:50051
    consumer_id: demo

output:
  stdout: {}

Example config using the output:

input:
  generate:
    mapping: 'root = "hello"'
    interval: 1s

output:
  grpc:
    address: 127.0.0.1:50051
    max_in_flight: 32

Proto service contract lives in proto/ingest.proto and generates Go into pkg/pb.

End-to-end demo

1. Start the test gRPC server in one terminal:

go run ./cmd/grpc-test-server -addr :50051

1. In another terminal, run input example (receive from server via gRPC and print):

go run ./cmd/bento-grpc-plugin run ./examples/e2e_input.yaml

1. Or run output example (generate messages and send to server via gRPC):

go run ./cmd/bento-grpc-plugin run ./examples/e2e_output.yaml

You should see logs from the test server and printed messages on the client.

Token auth examples

Set your secrets in env, then run the token-auth variants:

export GRPC_BEARER_TOKEN="example-token"
export API_KEY="example-api-key"

# Output with token/headers
go run ./cmd/bento-grpc-plugin run ./examples/e2e_output_token.yaml

# Input with token/headers
go run ./cmd/bento-grpc-plugin run ./examples/e2e_input_token.yaml

These attach Authorization: Bearer $GRPC_BEARER_TOKEN and the headers in auth_headers to each RPC.

References

• Bento docs: https://warpstreamlabs.github.io/bento/
• Public service API: https://pkg.go.dev/github.com/warpstreamlabs/bento/public/service
• gRPC Go quickstart: https://grpc.io/docs/languages/go/