gotextfsm

package module
v1.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 25, 2026 License: Apache-2.0 Imports: 8 Imported by: 11

README

gotextfsm

This is a Golang implementation (or version) of Google's TextFSM Python library.

TextFSM is a template-based state machine for parsing semi-formatted text. It was originally developed to allow programmatic access to information returned from the command line interface (CLI) of networking devices.

The complete documentation of TextFSM can be found at the TextFSM Wiki.

This porting attempts to be 100% compatible to the original TextFSM specification given in the above links.

Get Started

Get the library:

$ go get github.com/sirikothe/gotextfsm
Basic Usage

Import the gotextfsm library:

import "github.com/sirikothe/gotextfsm"

Create a TextFSM Object and parse the template:

  fsm := gotextfsm.TextFSM{}
  # template should hold the string of the Textfsm to be parsed.
  err := fsm.ParseString(template)
  # err will be nil if the parsing of the template is successful. 
  # If will contain error object if the parsing failed.

Parse a raw input string using the fsm object created:

  parser := gotextfsm.ParserOutput{}
  err = parser.ParseTextString(input, fsm, true)
  # err will be nil if the parsing of the input (as per the fsm provided) is successful. 
  # If will contain error object if the parsing failed.

At the end of the parsing of input the parser's Dict object contains the results.

Complete Example Code
package main

import (
	"fmt"

	"github.com/sirikothe/gotextfsm"
)

func main() {
	template := `Value beer (.*)

Start
  ^$beer
`
	input := "Hello World"
	fsm := gotextfsm.TextFSM{}
	err := fsm.ParseString(template)
	if err != nil {
		fmt.Printf("Error while parsing template '%s'\n", err.Error())
		return
	}
	parser := gotextfsm.ParserOutput{}
	err = parser.ParseTextString(input, fsm, true)
	if err != nil {
		fmt.Printf("Error while parsing input '%s'\n", err.Error())
	}
	fmt.Printf("Parsed output: %v\n", parser.Dict)
}

How to read results of parsing

The defined type for ParserOutput.Dict is []map[string]interface{}.

This is an array of maps. Each element in array represents a record of parsed output.

Each record is of type map[string]interface{} where the key is the field name (type string) and value is the field value (type interface{}).

Even though the field value is defined as type interface{}, its concrete type is one of:

  • []string type -> For variables declared as List in the definition. ex. Value List ifnames (\w+)
  • map[string]string type -> For variables declared as scalar, but with nested regexes. ex. Value person ((?P<name>\w+):\s+(?P<age>\d+)\s+(?P<state>\w{2})\s*)
  • []map[string]string type -> For variables declared as List, but with nested regexes. ex. Value List person ((?P<name>\w+):\s+(?P<age>\d+)\s+(?P<state>\w{2})\s*)
  • string type -> For every other variable type. This is most common use case.
Option 1 - Example code to handle the output

Following complete code snippet shows an example of how to process the output of parser.

package main

import (
	"fmt"

	"github.com/sirikothe/gotextfsm"
)

func main() {
	template := `Value continent (.*)
Value List countries (.*)
Value state_abbr ((?P<fullstate>\w+):\s+(?P<abbr>\w{2}))
Value List persons ((?P<name>\w+):\s+(?P<age>\d+)\s+(?P<state>\w{2})\s*)

Start
	^Continent: ${continent}
	^Country: ${countries}
	^State: ${state_abbr}
	^${persons}
`

	input := `Continent: North America
Country: USA
Country: Canada
Country: Mexico
State: California: CA
Siri: 50 CA
Raj: 22 NM
Gandhi: 150 NV
`

	fsm := gotextfsm.TextFSM{}
	err := fsm.ParseString(template)
	if err != nil {
		fmt.Printf("Error while parsing template '%s'\n", err.Error())
		return
	}
	parser := gotextfsm.ParserOutput{}
	err = parser.ParseTextString(input, fsm, true)
	if err != nil {
		fmt.Printf("Error while parsing input '%s'\n", err.Error())
	}
	for _, record := range parser.Dict {
		for key, value := range record {
			switch value.(type) {
			case string:
				// typecast to string and use it.
				// ex: Value continent (.*)
				fmt.Printf("%s: %s\n", key, value.(string))
			case []string:
				// List type variable. typecast to []string and use it.
				// ex: Value List countries (.*)
				fmt.Printf("%s: %s\n", key, value.([]string))

			case map[string]string:
				// Nested scalar variable.
				// ex: Value state_abbr ((?P<fullstate>\w+):\s+(?P<abbr>\w{2}))
				fmt.Printf("%s: %s\n", key, value.(map[string]string))

			case []map[string]string:
				// Nested List variable.
				// ex: Value List persons ((?P<name>\w+):\s+(?P<age>\d+)\s+(?P<state>\w{2})\s*)
				fmt.Printf("%s: %s\n", key, value.([]map[string]string))
			default:
				// Should never happen.
				panic("Really?")
			}
		}
	}
}
Option 2 - Example code to handle the output

You can also marshal the resulting dict to json (or yaml) if that makes it easier for you to handle the output.

	str, err := json.Marshal(parser.Dict)
	if err != nil {
		fmt.Printf("Unable to convert dict to json \n", err)
	} else {
		fmt.Printf("JSON: %s\n", str)
	}

Output from the above example:

JSON: [{"continent":"North America","countries":["USA","Canada","Mexico"],"persons":[{"age":"50","name":"Siri","state":"CA"},{"age":"22","name":"Raj","state":"NM"},{"age":"150","name":"Gandhi","state":"NV"}],"state_abbr":{"abbr":"CA","fullstate":"California"}}]

Highlights

  • Attempts to be 100% compatible with the original TextFSM implementation (See differences section).
  • Very nimble code with zero external dependencies on any other libraries.
  • Well tested (~ 97% code coverage) >1740 Test cases executed!!!
    • All test cases of Python's implementation are ported and executed.
    • More test cases added as well to test corner cases.
  • All the test cases of ntc-templates are executed.
    • Out of 1578 test cases of ntc-templates, 28 of them are failing (All due to reasons listed in Caveats).

Differences with Python's implementation

The following are the differences between this implementation of TextFSM and the original Python implementation:

  • Python's implementation provides 2 ways of getting results.
    • Output as a list of lists.
      • The outer list represents a record and inner list contains the values in the order they were declared.
    • Output as a list of dicts.
      • This Golang implementation provides the output as only slice of maps. It does not provide a slice of slices.
  • [TODO] 🚧 This Golang implementation (currently) implements the core TextFSM functionality. It does not implement the following:
    • clitable
    • terminal
    • texttable

Caveats

There are differences in Golang's regular expression implementation and Python's.

⚠ Because of this reason some templates that are valid in Python fail parsing in gotextfsm.

[!TIP] Supported regular expression sytnax are those provided by the re2 library

[!IMPORTANT] Following are some of the known examples:

  • Golang restricts repeat count to be 1000 while Python allows it to be 2^^16 -1. Because of this reason, the regular expressions like Value SAP_COUNT ([0-9]{1,1500}) throw an error. More details about this are discussed at https://github.com/golang/go/issues/7252
  • Golang does not support negative or positive lookahead or lookbehind nor backreferences.
    • Perl syntax like (?<. The regex like Value NAME (\S.*(?<!\s)) throws an error.

Testing

PS C:\Users\siri\code\nuviso\GitHub\gotextfsm> go test -v
=== RUN   TestParseText
    parsetext_test.go:83: Executed 48 test cases
--- PASS: TestParseText (0.04s)
=== RUN   TestPyTemplate
    pytemplate_test.go:31: Executed 9 test cases
--- PASS: TestPyTemplate (0.00s)
=== RUN   TestRuleParse
    rule_test.go:41: Executed 25 test cases
--- PASS: TestRuleParse (0.00s)
    textfsm_test.go:84: Executed 56 test cases
--- PASS: TestFSMParse (0.01s)
=== RUN   TestValueParse
    value_test.go:56: Executed 27 test cases
--- PASS: TestValueParse (0.01s)
PASS
ok      _/C_/Users/siri/code/nuviso/GitHub/gotextfsm    0.235s
PS C:\Users\siri\code\nuviso\GitHub\gotextfsm> go test -cover
PASS
coverage: 96.9% of statements
ok      _/C_/Users/siri/code/nuviso/GitHub/gotextfsm    0.236s

Documentation

Index

Constants

View Source 
const MAX_NAME_LENG = 48

Variables

View Source 
var LINE_OPERATORS = []string{"Continue", "Next", "Error"}
View Source 
var RECORD_OPERATORS = []string{"Clear", "Clearall", "Record", "NoRecord"}

Functions

func ExecutePythonTemplate 

func ExecutePythonTemplate(pytemplate string, vars_map map[string]interface{}) (string, error)

ExecutePythonTemplate converts a python template into a golang template and executes against the given variable map. This is a quick hack and not does not support all comprehensive list of Python Template package. It does the following:

  • replace all ${varname} to {{.varname}}
  • replace all $varname to {{.varname}} (By sorting the varnames with the longest name first)
  • replace all $$ with $
  • escape all literal {{ and }} with {{"{{"}} and {{"}}"}}

Assumes it is a valid python Template. No validations done to validate the python template syntax. Assumes all the variables are proper variable identifiers Then executes the resulting golang template on the map passed.

func FindIndex 

func FindIndex(arr []string, elem string) int

func GetGroupNames 

func GetGroupNames(r string) ([]string, error)

Given a regular expression with named groups ex. (?P<name>\w+)\s+(?P<age>\d+) Return the names e.g. ["name", "age"]

func GetNamedMatches 

func GetNamedMatches(r *regexp.Regexp, s string) map[string]string

func TrimRightSpace 

func TrimRightSpace(str string) string

Types

type ON_RECORD_TYPE 

type ON_RECORD_TYPE int
const (
	SKIP_RECORD ON_RECORD_TYPE = iota
	SKIP_VALUE
	CONTINUE
)

type ParserOutput 

type ParserOutput struct {
	Dict []map[string]interface{}
	// contains filtered or unexported fields
}

Dict contains a slice of maps. Each element in the slice holds the value of a record. Each record is represented as map of (name,value)

Note that type of value is interface{}. But the concrete type is either 'string' or '[]string'

func (*ParserOutput) ParseTextReader 

func (t *ParserOutput) ParseTextReader(reader *strings.Reader, fsm TextFSM, eof bool) error

func (*ParserOutput) ParseTextScanner 

func (t *ParserOutput) ParseTextScanner(scanner *bufio.Scanner, fsm TextFSM, eof bool) error

func (*ParserOutput) ParseTextString 

func (t *ParserOutput) ParseTextString(text string, fsm TextFSM, eof bool) error

ParseTextString passes CLI output (provided as string) through FSM and 

    Args:
      text: (string), Text to parse with embedded newlines.
		 fsm: (TextFSM), TextFSM object as a result of parsing the text fsm template
      eof: (bool), Set to False if we are parsing only part of the file.
            Suppresses triggering EOF state.
    Returns:
      error if there is any error in parsing

func (*ParserOutput) Reset 

func (t *ParserOutput) Reset(fsm TextFSM)

type TextFSM 

type TextFSM struct {
	COMMENT_RE         *regexp.Regexp
	STATE_RE           *regexp.Regexp
	MAX_STATE_NAME_LEN int
	Values             map[string]TextFSMValue
	States             map[string]TextFSMState
	// contains filtered or unexported fields
}

func (*TextFSM) ParseReader 

func (t *TextFSM) ParseReader(reader *strings.Reader) error

func (*TextFSM) ParseScanner 

func (t *TextFSM) ParseScanner(scanner *bufio.Scanner) error

func (*TextFSM) ParseString 

func (t *TextFSM) ParseString(input string) error

Parses the string passed, into a TextFSM structure. 

Args:
	input string: Valid template as a string.
Retruns:
	error if there is any error. nil otherwise

type TextFSMRule 

type TextFSMRule struct {
	Regex         string
	CompiledRegex *regexp.Regexp
	Match         string
	LineOp        string
	RecordOp      string
	NewState      string
	LineNum       int
}

func (*TextFSMRule) Parse 

func (r *TextFSMRule) Parse(line string, lineNum int, var_map map[string]interface{}) error

func (*TextFSMRule) String 

func (t *TextFSMRule) String() string

type TextFSMState 

type TextFSMState struct {
	// contains filtered or unexported fields
}

type TextFSMValue 

type TextFSMValue struct {
	Regex    string
	Template string
	Name     string
	Options  []string
	// contains filtered or unexported fields
}

func (*TextFSMValue) Parse 

func (value *TextFSMValue) Parse(input string, line_num int) error

func (*TextFSMValue) String 

func (v *TextFSMValue) String() string

String() returns a string representation of the value

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.