network

package
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 9, 2026 License: Apache-2.0 Imports: 23 Imported by: 0

README

network package

The network package provides a high‑performance, zero‑allocation TCP engine built on top of the gnet event‑driven networking library. It handles a compact, optimized binary wire protocol designed to transport Tellstone frames with maximum throughput and minimum latency.

Features

  • Zero‑allocation by default – Inbound messages are decoded directly out of the live connection ring-buffer views, entirely bypassing the Go heap allocator on the hot path.
  • Edge‑triggered epoll loop – Powered by github.com/panjf2000/gnet/v2 to orchestrate 32 event loops smoothly across available CPU cores.
  • Scatter-Gather I/O – Outbound writes leverage net.Buffers to utilize the kernel-level writev system call, preventing TCP packet fragmentation and minimizing context switches.
  • Synchronous Client – Contains an execution-blocking Client wrapper designed for low-latency microservice pipelines featuring zero-alloc payload extraction.

Protocol Overview


+----------------+----------------+-----------------+
| uint32 length  | uint8 type     | []byte payload  |
+----------------+----------------+-----------------+
  • length – Total size of the type byte + the variable length payload (Big-Endian).
  • type – An 8-bit unsigned integer representing MessageType (MsgPing, MsgPong, MsgRequest, MsgResponse, MsgError, MsgAuth, MsgAuthOk, MsgAuthErr).
  • payload – Optional binary array representing data instructions (e.g., Tellstone raw SQL statements). Failures for the data path ride in MsgError frames; MsgResponse frames carry data values unchanged, so a stored value may begin with "ERR " without being mistaken for an error.

Usage Examples

Server Configuration
package main

import (
	"log"
	"[github.com/Saxy/Tellstone/internal/network](https://github.com/Saxy/Tellstone/internal/network)"
)

func main() {
	// Simple echo handler – replies with a Pong for every Ping.
	handler := func(msg *network.Message) ([]byte, network.MessageType, error) {
		if msg.Type == network.MsgPing {
			return []byte("pong"), network.MsgPong, nil
		}
		return nil, 0, nil
	}

	srv := network.NewServer("127.0.0.1:9988", handler)
	if err := srv.ListenAndServe(); err != nil {
		log.Fatalf("server failed: %v", err)
	}
}
High-Performance Client Execution
package main

import (
	"fmt"
	"time"
	"[github.com/Saxy/Tellstone/internal/network](https://github.com/Saxy/Tellstone/internal/network)"
)

func main() {
	client, err := network.Dial("127.0.0.1:9988", 2*time.Second)
	if err != nil {
		panic(err)
	}
	defer client.Close()

	// Reusable stack buffers to guarantee 0 allocations during calls
	var scratchpad [1024]byte
	var response network.Message

	err = client.Call(network.MsgPing, []byte("ping"), scratchpad[:], &response)
	if err != nil {
		panic(err)
	}

	fmt.Printf("Received response type: %d, payload: %s\n", response.Type, string(response.Payload))
}

Benchmarks

Micro-benchmarks conducted on a bare-metal AMD Ryzen 9 9950X (16-Core / 32-Threads) environment running Linux:

goos: linux
goarch: amd64
pkg: [github.com/Saxy/Tellstone/internal/network](https://github.com/Saxy/Tellstone/internal/network)
cpu: AMD Ryzen 9 9950X 16-Core Processor            

BenchmarkReadMessageZeroAlloc-32         856,039,898        1.464 ns/op        0 B/op        0 allocs/op
BenchmarkGnetServerHandlerParallel-32        522,372         2297 ns/op      149 B/op        6 allocs/op
Insights
  • Decode execution layer: Parsing an active protocol packet takes a mere $1.46\text{ ns}$ with absolute 0 allocations, maximizing memory density and leaving zero traces for the Go Garbage Collector.
  • Parallel Core Concurrency: When executing full network event loops in parallel across all 32 threads, operations settle cleanly at $2297\text{ ns}$. The minimal 149 bytes allocated here reflect the system's runtime framework lifecycles—completely isolating Tellstone's core execution path from runtime pressure.

Documentation

Overview

Package network Tellstone Secure TCP Networking Package File: acl.go Description: Wire codec for the ACL OpCodes. Requests reuse the ROLE primitive — length-prefixed tokens in the message Value field — and ACL LIST responses use the same primitive as ROLE LIST. All lengths are big-endian uint16, so a single token is capped at 64 KiB. ACL is an admin operation, so these helpers allocate freely — they never touch the KV hot path.

Authors: 

Maximilian Hagen

Package network Tellstone Cloud-Native In-Memory Database File: client.go Description: Implements a high-performance, synchronous, zero-allocation TCP client using pre‑allocated buffers for request/response handling.

Authors: 

Maximilian Hagen

Package network Tellstone Cloud-Native In-Memory Database File: client_acl.go Description: Binary-protocol client methods for the ACL command family. Requests carry their arguments in the message Value as length-prefixed tokens (EncodeRoleArgs); the server replies with ResponseOK in a MsgResponse frame on success, the encoded typed ACL LIST payload, or the error detail in a MsgError frame that surfaces as the returned error. Admin ops only — no allocation concerns on the hot path.

Authors: 

Maximilian Hagen

Package network Tellstone Cloud-Native In-Memory Database File: client_role.go Description: Binary-protocol client methods for the ROLE command family. Requests carry their arguments in the message Value as length-prefixed tokens (EncodeRoleArgs); the server replies with ResponseOK in a MsgResponse frame on success, an encoded typed payload (LIST/GETUSER), or the error detail in a MsgError frame that surfaces as the returned error. Admin ops only — no allocation concerns on the hot path.

Authors: 

Maximilian Hagen

Package network Tellstone Secure TCP Networking Package File: protocol.go Description: Defines the binary protocol wire format used by the secure server. Provides Message struct, MessageType constants, and zero‑allocation encode/decode helpers.

Authors: 

Maximilian Hagen

Package network Tellstone Secure TCP Networking Package File: role.go Description: Wire codec for the ROLE OpCodes. Requests pack their argument list into the message Value field as length-prefixed tokens; responses use the same primitive. All lengths are big-endian uint16, so a single token is capped at 64 KiB. ROLE is an admin operation, so these helpers allocate freely — they never touch the KV hot path.

Authors: 

Maximilian Hagen

Package network Tellstone Secure Event-Driven Networking Package File: server.go Description: Implements an ultra‑high‑performance, zero‑allocation TCP server using an edge‑triggered epoll event‑loop (gnet). Handles incoming messages, dispatches them to storage, and writes responses. Supports optional TLS 1.3 transport encryption via the internal TLS library.

Authors: 

Maximilian Hagen

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrResponseBufferTooSmall is returned if the scratchpad buffer cannot hold the incoming server frame.
	ErrResponseBufferTooSmall = errors.New("client: provided scratchpad buffer is too small for the server response")

	// ErrRequestTooLarge is returned if the generated request exceeds the local stack buffer boundaries.
	ErrRequestTooLarge = errors.New("client: key or value size exceeds local client packaging limitations")

	// ErrAuthCredentialsTooLong is returned when an AUTH credential exceeds the
	// uint16 length prefix of the binary auth frame.
	ErrAuthCredentialsTooLong = errors.New("client: auth credential exceeds 65535 byte protocol limit")
)
View Source 
var (
	ResponseOK             = []byte("OK")
	ResponseNotFound       = []byte("NOT_FOUND")
	ResponseEmptyKey       = []byte("EMPTY_KEY")
	ResponseStorageFailure = []byte("STORAGE_FAILURE")
	ResponseInvalidOpCode  = []byte("INVALID_OPCODE")
	ResponseAuthErr        = []byte("INVALID_AUTH")
	ResponseNotAuthorized  = []byte("NOT_AUTHORIZED")
)

Response payloads for the data path. Failures ride in MsgError frames with the bare code — the frame type, not an in-band "ERR " marker, distinguishes them from data, so a stored value that happens to begin with "ERR " survives a round trip untouched.

Functions

func Decode 

func Decode(data []byte, maxMsgSize uint64, out *Message) (int, error)

Decode parses a full protocol frame from an existing byte slice. It returns the payload length and populates the supplied Message struct. Guarantees 0 heap allocations by slicing directly into the network ring buffer.

func DecodeRoleArgs added in v1.1.0 

func DecodeRoleArgs(payload []byte, dst [][]byte) (args [][]byte, ok bool)

DecodeRoleArgs unpacks a role request payload into dst[:0]. ok is false when the payload is truncated or has trailing garbage.

func EncodeACLListResponse added in v1.1.0 

func EncodeACLListResponse(users []ACLUser) ([]byte, bool)

EncodeACLListResponse packs an ACL LIST response. ok is false when the user count or a name, command, or namespace exceeds the 64 KiB length-prefix limit.

func EncodeACLLogResponse added in v1.1.0 

func EncodeACLLogResponse(entries []AuthLogEntry) ([]byte, bool)

EncodeACLLogResponse packs an ACL LOG response. ok is false when the entry count or any field exceeds the 64 KiB length-prefix limit.

func EncodeRoleArgs added in v1.1.0 

func EncodeRoleArgs(args [][]byte) ([]byte, bool)

EncodeRoleArgs packs args into a request payload. Empty args yield a two-byte zero count. ok is false when the argument count or any single argument exceeds the 64 KiB length-prefix limit — encoding it would silently truncate the wire form.

func EncodeRoleGetUserResponse added in v1.1.0 

func EncodeRoleGetUserResponse(u RoleUser) []byte

EncodeRoleGetUserResponse packs a ROLE GETUSER response.

func EncodeRoleListResponse added in v1.1.0 

func EncodeRoleListResponse(entries []RoleListEntry) ([]byte, bool)

EncodeRoleListResponse packs a ROLE LIST response. ok is false when the entry count or a name, command, or namespace exceeds the 64 KiB length-prefix limit.

func Read 

func Read(r io.Reader, buf []byte, out *Message) error

Read extracts a message from an io.Reader stream directly into a pre-allocated scratchpad buffer.

func Write 

func Write(w io.Writer, msgType MessageType, payload []byte) error

Write transmits a message completely allocation-free.

Types

type ACLUser added in v1.1.0 

type ACLUser struct {
	Username   string
	Role       string // empty when the default role applies
	HasPass    bool
	Commands   []string
	Namespaces [][]byte
}

ACLUser is one user from an ACL LIST response.

func DecodeACLListResponse added in v1.1.0 

func DecodeACLListResponse(payload []byte) ([]ACLUser, bool)

DecodeACLListResponse unpacks an ACL LIST response. ok is false on a truncated payload.

type AuthLogEntry added in v1.1.0 

type AuthLogEntry struct {
	Timestamp  string
	Username   string
	RemoteAddr string
	Reason     string
}

AuthLogEntry is one ACL LOG record. Timestamp is an RFC3339 string, the same rendering the RESP ACL LOG handler emits, so both protocols expose identical log content.

func DecodeACLLogResponse added in v1.1.0 

func DecodeACLLogResponse(payload []byte) ([]AuthLogEntry, bool)

DecodeACLLogResponse unpacks an ACL LOG response. ok is false on a truncated payload.

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client represents a high-performance synchronous connection to a Tellstone server.

func Dial 

func Dial(addr string, timeout time.Duration) (*Client, error)

Dial connects to a Tellstone server pool via the specified TCP address.

func DialTLS added in v1.1.0 

func DialTLS(addr string, certPath, keyPath, caPath string, timeout time.Duration) (*Client, error)

DialTLS connects to a Tellstone server with TLS 1.3 encryption. certPath/keyPath are the client certificate and key for mTLS (pass empty for one-way TLS). caPath is the CA certificate to verify the server (pass empty to skip verification).

func DialTLSWithLogger added in v1.1.0 

func DialTLSWithLogger(addr string, certPath, keyPath, caPath string, timeout time.Duration, logger log.Logger) (*Client, error)

DialTLSWithLogger connects like DialTLS and reports connection lifecycle events to logger.

func DialWithLogger added in v1.1.0 

func DialWithLogger(addr string, timeout time.Duration, logger log.Logger) (*Client, error)

DialWithLogger connects like Dial and reports connection lifecycle events to logger.

func (*Client) AclDelUser added in v1.1.0 

func (c *Client) AclDelUser(username string, scratchBuf []byte) error

AclDelUser issues ACL DELUSER <username>.

func (*Client) AclList added in v1.1.0 

func (c *Client) AclList(scratchBuf []byte) ([]ACLUser, error)

AclList issues ACL LIST and decodes the typed response: one user per entry with username, bound role, password presence, and the role's granted commands and namespace whitelist.

func (*Client) AclLog added in v1.1.0 

func (c *Client) AclLog(scratchBuf []byte) ([]AuthLogEntry, error)

AclLog issues ACL LOG and decodes the typed response: the recent auth-failure buffer in chronological order, each entry carrying timestamp, username, remote address, and reason.

func (*Client) AclSetUser added in v1.1.0 

func (c *Client) AclSetUser(username, role string, passOptions [][]byte, scratchBuf []byte) error

AclSetUser issues ACL SETUSER <username> <role> [>password] [nopass], the ACL alias of ROLE SETUSER. The role must already exist.

func (*Client) Auth added in v1.1.0 

func (c *Client) Auth(password string, scratchBuf []byte) error

Auth authenticates the client with a password (single-password mode, username empty). scratchBuf must be large enough to hold the server response. Returns nil on success.

func (*Client) AuthUser added in v1.1.0 

func (c *Client) AuthUser(username, password string, scratchBuf []byte) error

AuthUser authenticates with a username/password pair (RBAC mode). Returns nil on success, an error on wrong credentials or a missing user. The password is transmitted in cleartext unless the connection was made via DialTLS — TLS is an operator opt-in and this payload rides the same transport as every other message.

func (*Client) Call 

func (c *Client) Call(msgType MessageType, reqPayload []byte, buf []byte, out *Message) error

Call executes a synchronous Request-Response cycle completely allocation-free.

func (*Client) Close 

func (c *Client) Close() error

Close gracefully closes the underlying network connection.

func (*Client) Delete 

func (c *Client) Delete(key []byte, scratchBuf []byte) ([]byte, error)

Delete removes a key-value entity permanently from the remote cluster space.

func (*Client) Get 

func (c *Client) Get(key []byte, scratchBuf []byte) ([]byte, error)

Get retrieves a binary value from the remote engine using its key identifier.

func (*Client) RoleCreate added in v1.1.0 

func (c *Client) RoleCreate(role string, rules []string, scratchBuf []byte) error

RoleCreate issues ROLE CREATE <name> <rule>... .

func (*Client) RoleDelUser added in v1.1.0 

func (c *Client) RoleDelUser(username string, scratchBuf []byte) error

RoleDelUser issues ROLE DELUSER <username>.

func (*Client) RoleDelete added in v1.1.0 

func (c *Client) RoleDelete(role string, scratchBuf []byte) error

RoleDelete issues ROLE DELETE <role>.

func (*Client) RoleGetUser added in v1.1.0 

func (c *Client) RoleGetUser(username string, scratchBuf []byte) (RoleUser, error)

RoleGetUser issues ROLE GETUSER <username> and decodes the typed response.

func (*Client) RoleList added in v1.1.0 

func (c *Client) RoleList(scratchBuf []byte) ([]RoleListEntry, error)

RoleList issues ROLE LIST and decodes the typed response.

func (*Client) RoleSetUser added in v1.1.0 

func (c *Client) RoleSetUser(username, role string, passOptions [][]byte, scratchBuf []byte) error

RoleSetUser issues ROLE SETUSER <username> <role> [>password] [nopass].

func (*Client) Set 

func (c *Client) Set(key, value []byte, ttlMs int64, scratchBuf []byte) ([]byte, error)

Set stores a binary key-value pair with a millisecond-based TTL inside the remote engine.

type Message 

type Message struct {
	Type    MessageType
	Op      OpCode
	TTL     int64
	Key     []byte
	Value   []byte
	Payload []byte
}

Message is the atomic execution frame of the Tellstone TCP protocol.

func ReadMessage 

func ReadMessage(r io.Reader) (*Message, error)

func Unmarshal 

func Unmarshal(r io.Reader) (*Message, error)

Unmarshal blocks on an io.Reader and instantiates a freshly allocated Message pointer on success.

func (*Message) Marshal 

func (m *Message) Marshal() []byte

Marshal encodes the Message into its binary wire format. It allocates a new slice – suitable for one‑off operations such as handshakes.

type MessageType 

type MessageType uint8

MessageType defines the kind of message exchanged over the protocol. 

const (
	MsgPing MessageType = iota
	MsgPong
	MsgRequest
	MsgResponse
	MsgAuth
	MsgAuthOk
	MsgAuthErr
	// MsgError is the dedicated error frame for the data path. It is appended
	// after the original set so existing type byte values stay stable.
	MsgError
)

type OpCode 

type OpCode uint8

OpCode defines the backend database operation. 

const (
	OpGet OpCode = iota + 1
	OpSet
	OpDelete
	OpRoleCreate
	OpRoleSetUser
	OpRoleDelUser
	OpRoleDelete
	OpRoleList
	OpRoleGetUser
	OpACLSetUser
	OpACLDelUser
	OpACLList
	OpACLLog
)

func (OpCode) String 

func (o OpCode) String() string

type RoleListEntry added in v1.1.0 

type RoleListEntry struct {
	Name       string
	Commands   []string
	Namespaces [][]byte
}

RoleListEntry is one role from a ROLE LIST response.

func DecodeRoleListResponse added in v1.1.0 

func DecodeRoleListResponse(payload []byte) ([]RoleListEntry, bool)

DecodeRoleListResponse unpacks a ROLE LIST response. ok is false on a truncated payload.

type RoleUser added in v1.1.0 

type RoleUser struct {
	Role    string // empty when the user has no explicit role
	HasPass bool
}

RoleUser is the decoded ROLE GETUSER response.

func DecodeRoleGetUserResponse added in v1.1.0 

func DecodeRoleGetUserResponse(payload []byte) (RoleUser, bool)

DecodeRoleGetUserResponse unpacks a ROLE GETUSER response.

type Server 

type Server struct {
	gnet.BuiltinEventEngine
	// contains filtered or unexported fields
}

func NewServer 

func NewServer(
	addr string,
	maxMsgSize uint64,
	shards []*shard.Shard,
	handler func(msg *Message) ([]byte, MessageType, error),
	logger log.Logger,
	tlsConfigs *tlslib.ConfigStore,
	requirePass string,
	policy *rbac.Store,
	provider oauth.Provider,
	audit *audit.LogEngine) *Server

NewServer initializes an edge-triggered networking server engine instance. It applies defensive configuration defaults before spawning infrastructure. shards is optional — if nil, per-shard metrics are not tracked. tlsConfigs is optional — if nil, plaintext TCP is used. When configured, each accepted connection atomically loads the latest immutable TLS configuration. requirePass is optional — if empty, AUTH is a no-op and connections start authenticated; otherwise it is hashed at startup and clients must AUTH before issuing data commands. policy is optional — if nil, RBAC is disabled and every authenticated op is allowed; otherwise AUTH resolves per-user credentials and sessions gate data ops. provider is optional — if nil, bearer-token AUTH is disabled and AUTH stays password-only; when set it verifies JWT-shaped secrets and maps their claims to roles through the policy store (which must therefore also be set). audit is the shared audit engine; it must be non-nil (pass a disabled engine when audit logging is off) and is always called without a nil guard.

func (*Server) BytesRead 

func (s *Server) BytesRead() uint64

func (*Server) BytesWritten 

func (s *Server) BytesWritten() uint64

func (*Server) ConnectedClients 

func (s *Server) ConnectedClients() uint64

func (*Server) HandlerErrors 

func (s *Server) HandlerErrors() uint64

func (*Server) ListenAndServe 

func (s *Server) ListenAndServe() error

ListenAndServe starts the multi-reactor epoll event loop.

func (*Server) OnBoot 

func (s *Server) OnBoot(eng gnet.Engine) gnet.Action

func (*Server) OnClose 

func (s *Server) OnClose(c gnet.Conn, err error) (action gnet.Action)

func (*Server) OnOpen 

func (s *Server) OnOpen(c gnet.Conn) (out []byte, action gnet.Action)

func (*Server) OnTraffic 

func (s *Server) OnTraffic(c gnet.Conn) gnet.Action

OnTraffic handles incoming bytes on the socket asynchronously and lock-free. When TLS is enabled, encrypted bytes are decrypted via the internal TLS library before protocol parsing. The handshake is driven automatically by the first Read/Write calls on the TLS connection.

func (*Server) ProtocolErrors 

func (s *Server) ProtocolErrors() uint64

func (*Server) Shutdown 

func (s *Server) Shutdown(ctx context.Context) error

Shutdown gracefully stops the event loop, waiting for in-flight connections to drain or ctx to expire. It blocks until ListenAndServe has reached OnBoot, so it is safe to call concurrently with ListenAndServe from another goroutine (e.g. a signal handler).

func (*Server) TotalConnections 

func (s *Server) TotalConnections() uint64

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.