cmdjsonl

README

cmdjsonl

cmdjsonl is a Go package for parsing data in the application/x-cmdjsonl format. This format represents command-style invocations as line-oriented text, making it convenient for streaming, logging, and cross-language/cross-process communication.

Data Format

The application/x-cmdjsonl format is defined as follows:

• UTF-8 encoding;
• Each line consists of a Command followed by a space and a JSON object, with lines separated by a newline character (\n).

Example:

add {"a":1,"b":2}
greet {"name":"Alice"}
log {"level":"info","msg":"hello world"}

Installation

go get github.com/qiniu/x/cmdjsonl

Quick Start

package main

import (
	"strings"

	"github.com/qiniu/x/cmdjsonl"
)

type AddParam struct {
	A int `json:"a"`
	B int `json:"b"`
}

type GreetParam struct {
	Name string `json:"name"`
}

func main() {
	var p cmdjsonl.Parser

	// Register a handler with a non-pointer parameter
	p.HandleFunc("add", func(param AddParam) {
		// ...
	})

	// Register a handler with a pointer parameter that returns an error
	p.HandleFunc("greet", func(param *GreetParam) error {
		// ...
		return nil
	})

	data := `add {"a":1,"b":2}
greet {"name":"Alice"}
`
	if err := p.Parse(strings.NewReader(data), 4096); err != nil {
		panic(err)
	}
}

You can use cmdjsonl to customize a new script. For example:

• demo/cmdjsonl.

API

type Parser

type Parser struct {
	// ...
}

Parser parses data in the cmdjsonl format and dispatches the parsed commands to registered handler functions.

func (*Parser) HandleFunc

func (p *Parser) HandleFunc(cmd string, fn any) error

Registers a handler function fn for the specified command cmd.

The handler function fn must satisfy the following requirements:

• It must be a function;
• It must have exactly one parameter, which can be either a pointer or a non-pointer type;
• It must return zero or one value; if it returns a value, that value's type must be error. If fn does not meet these requirements, HandleFunc returns an *InvalidHandler error.

Examples of parameter types:

// Non-pointer parameter
p.HandleFunc("foo", func(param FooParam) { ... })

// Pointer parameter
p.HandleFunc("bar", func(param *BarParam) error { ... })

// No return value
p.HandleFunc("baz", func(param BazParam) { ... })

func (*Parser) Parse

func (p *Parser) Parse(in io.Reader, maxLineSize int) error

Reads and parses data line by line from in:

• maxLineSize specifies the maximum buffer size (in bytes) for a single line; if a line exceeds this length, an error is returned;
• Each line is split at the first space into a command and a JSON payload;
• The handler registered for the command is looked up, and the JSON payload is unmarshaled into that handler's parameter type;
• The handler function is then invoked; if it returns a non-nil error, parsing stops and that error is returned;
• When io.EOF is reached, parsing completes normally and nil is returned.

func (*Parser) ParseLax

func (p *Parser) ParseLax(in io.Reader, maxLineSize int) error

ParseLax is similar to Parse but ignores lines with unknown commands instead of returning an error. This allows the parser to continue processing valid commands even if some lines contain unrecognized commands.

The only difference from Parse is in the ParseCommand stage: when a line's command has no registered handler, ParseLax skips that line and continues parsing, instead of stopping and returning an error. All other error cases (ReadLine, UnmarshalParam, CallHandler <cmd>) still behave the same as Parse, returning a *ParseError and stopping parsing.

type InvalidHandler

type InvalidHandler struct {
	Cmd    string // The command for which the handler is invalid.
	Reason string // The reason why the handler is invalid.
}

func (e *InvalidHandler) Error() string

Represents an error returned by HandleFunc when a registered handler function does not meet the requirements.

type ParseError

type ParseError struct {
	Line int    // The line number where the error occurred (1-indexed).
	When string // The stage at which the error occurred.
	Msg  string // The error message.
}

func (e *ParseError) Error() string

Represents an error that occurred during Parse, including the line number, the stage (e.g., ReadLine, ParseCommand, UnmarshalParam, CallHandler <cmd>), and a message describing the error, making it easy to pinpoint the issue.

Error Handling

Parse stops and returns a *ParseError in the following cases:

When	Trigger Condition
ReadLine	An I/O error occurred while reading a line, or a line exceeds maxLineSize
ParseCommand	No space found in the line, or the command has no registered handler
UnmarshalParam	Failed to unmarshal the JSON payload
CallHandler <cmd>	The handler function returned a non-nil error

When the end of input (io.EOF) is reached normally, Parse returns nil, indicating successful completion.

ParseLax shares the same error handling table as Parse, with one exception: in the ParseCommand stage, the "command has no registered handler" trigger condition is ignored by ParseLax (the line is skipped instead of producing an error). All other trigger conditions (including "no space found in the line" in the ParseCommand stage) still result in a *ParseError.

Documentation

Index

• type InvalidHandler
  • func (e *InvalidHandler) Error() string
• type ParseError
  • func (p *ParseError) Error() string
• type Parser
  • func (p *Parser) HandleFunc(cmd string, fn any) error
  • func (p Parser) Parse(in io.Reader, maxLineSize int) error
  • func (p Parser) ParseLax(in io.Reader, maxLineSize int) error

Types

type InvalidHandler

type InvalidHandler struct {
	Cmd    string // The command for which the handler is invalid.
	Reason string // The reason why the handler is invalid.
}

func (*InvalidHandler) Error

func (e *InvalidHandler) Error() string

type ParseError

type ParseError struct {
	Line int
	When string
	Msg  string
}

ParseError represents an error that occurred during parsing of the input. It includes the line number where the error occurred, a description of when the error happened, and a message describing the error itself.

func (*ParseError) Error

func (p *ParseError) Error() string

type Parser

type Parser struct {
	// contains filtered or unexported fields
}

Parser parses input data in the cmdjsonl format (MIME type: application/x-cmdjsonl) and dispatches commands to registered handlers. Each line of input is expected to contain a command followed by a JSON object, separated by a space. The parser reads the input line by line, extracts the command and the JSON object, and invokes the corresponding handler.

func (*Parser) HandleFunc

func (p *Parser) HandleFunc(cmd string, fn any) error

HandleFunc registers a handler function for a specific command. The handler function must have exactly one parameter, which can be either a pointer or a non-pointer type. The function can optionally return an error. If the handler function does not meet these requirements, an InvalidHandler error is returned.

func (Parser) Parse

func (p Parser) Parse(in io.Reader, maxLineSize int) error

Parse reads from the provided io.Reader line by line, expecting each line to contain a command followed by a JSON object. It uses the registered handlers to process each command. If any error occurs during reading, parsing, or handling a command, a ParseError is returned with details about the error.

func (Parser) ParseLax

func (p Parser) ParseLax(in io.Reader, maxLineSize int) error

ParseLax is similar to Parse but ignores lines with unknown commands instead of returning an error. This allows the parser to continue processing valid commands even if some lines contain unrecognized commands.