grok

README

grok

A simple library to parse grok patterns with Go.

Installation

Make sure you have a working Go environment.

go get github.com/vjeantet/grok

Use in your project

import "github.com/vjeantet/grok"

Usage

Available patterns and custom ones

By default this grok package contains only patterns you can see in patterns/grok-patterns file.

When you want to add a custom pattern, use the grok.AddPattern(nameOfPattern, pattern), see the example folder for an example of usage. You also can load your custom patterns from a file (or folder) using grok.AddPatternsFromPath(path), or PatterndDir configuration.

Parse all or only named captures

g, _ := grok.New()
values, _  := g.Parse("%{COMMONAPACHELOG}", `127.0.0.1 - - [23/Apr/2014:22:58:32 +0200] "GET /index.php HTTP/1.1" 404 207`)

g, _ = grok.NewWithConfig(&grok.Config{NamedCapturesOnly: true})
values2, _ := g.Parse("%{COMMONAPACHELOG}", `127.0.0.1 - - [23/Apr/2014:22:58:32 +0200] "GET /index.php HTTP/1.1" 404 207`)

values is a map with all captured groups values2 contains only named captures

Examples

package main

import (
	"fmt"

	"github.com/vjeantet/grok"
)

func main() {
	g, _ := grok.New()
	values, _ := g.Parse("%{COMMONAPACHELOG}", `127.0.0.1 - - [23/Apr/2014:22:58:32 +0200] "GET /index.php HTTP/1.1" 404 207`)

	for k, v := range values {
		fmt.Printf("%+15s: %s\n", k, v)
	}
}

output:

       response: 404
          bytes: 207
       HOSTNAME: 127.0.0.1
       USERNAME: -
       MONTHDAY: 23
        request: /index.php
      BASE10NUM: 207
           IPV6:
           auth: -
      timestamp: 23/Apr/2014:22:58:32 +0200
           verb: GET
    httpversion: 1.1
           TIME: 22:58:32
           HOUR: 22
COMMONAPACHELOG: 127.0.0.1 - - [23/Apr/2014:22:58:32 +0200] "GET /index.php HTTP/1.1" 404 207
       clientip: 127.0.0.1
             IP:
          ident: -
          MONTH: Apr
           YEAR: 2014
         SECOND: 32
            INT: +0200
           IPV4:
         MINUTE: 58
     rawrequest:

Example 2

package main

import (
  "fmt"

  "github.com/vjeantet/grok"
)

func main() {
  g, _ := grok.NewWithConfig(&grok.Config{NamedCapturesOnly: true})
  values, _ := g.Parse("%{COMMONAPACHELOG}", `127.0.0.1 - - [23/Apr/2014:22:58:32 +0200] "GET /index.php HTTP/1.1" 404 207`)

  for k, v := range values {
    fmt.Printf("%+15s: %s\n", k, v)
  }
}

output:

      timestamp: 23/Apr/2014:22:58:32 +0200
           verb: GET
     rawrequest:
          bytes: 207
           auth: -
        request: /index.php
    httpversion: 1.1
       response: 404
COMMONAPACHELOG: 127.0.0.1 - - [23/Apr/2014:22:58:32 +0200] "GET /index.php HTTP/1.1" 404 207
       clientip: 127.0.0.1
          ident: -

Documentation

Index

• type Config
• type Grok
  • func New() (*Grok, error)
  • func NewWithConfig(config *Config) (*Grok, error)
  • func (g *Grok) AddPattern(name, pattern string) error
  • func (g *Grok) AddPatternsFromMap(m map[string]string) error
  • func (g *Grok) AddPatternsFromPath(path string) error
  • func (g *Grok) Match(pattern, text string) (bool, error)
  • func (g *Grok) Parse(pattern, text string) (map[string]string, error)
  • func (g *Grok) ParseStream(reader *bufio.Reader, pattern string, process func(map[string]string) error) error
  • func (g *Grok) ParseToMultiMap(pattern, text string) (map[string][]string, error)
  • func (g *Grok) ParseTyped(pattern string, text string) (map[string]interface{}, error)

Types

type Config

type Config struct {
	NamedCapturesOnly   bool
	SkipDefaultPatterns bool
	RemoveEmptyValues   bool
	PatternsDir         []string
	Patterns            map[string]string
}

A Config structure is used to configure a Grok parser.

type Grok

type Grok struct {
	// contains filtered or unexported fields
}

Grok object us used to load patterns and deconstruct strings using those patterns.

func New

func New() (*Grok, error)

New returns a Grok object.

func NewWithConfig

func NewWithConfig(config *Config) (*Grok, error)

NewWithConfig returns a Grok object that is configured to behave according to the supplied Config structure.

func (*Grok) AddPattern

func (g *Grok) AddPattern(name, pattern string) error

AddPattern adds a named pattern to grok

func (*Grok) AddPatternsFromMap

func (g *Grok) AddPatternsFromMap(m map[string]string) error

AddPatternsFromMap loads a map of named patterns

func (*Grok) AddPatternsFromPath

func (g *Grok) AddPatternsFromPath(path string) error

AddPatternsFromPath adds new patterns from the files in the specified directory to the list of loaded patterns.

func (*Grok) Match

func (g *Grok) Match(pattern, text string) (bool, error)

Match returns true if the specified text matches the pattern.

func (*Grok) Parse

func (g *Grok) Parse(pattern, text string) (map[string]string, error)

Parse the specified text and return a map with the results.

func (*Grok) ParseStream

func (g *Grok) ParseStream(reader *bufio.Reader, pattern string, process func(map[string]string) error) error

ParseStream will match the given pattern on a line by line basis from the reader and apply the results to the process function

func (*Grok) ParseToMultiMap

func (g *Grok) ParseToMultiMap(pattern, text string) (map[string][]string, error)

ParseToMultiMap parses the specified text and returns a map with the results. Values are stored in an string slice, so values from captures with the same name don't get overridden.

func (*Grok) ParseTyped

func (g *Grok) ParseTyped(pattern string, text string) (map[string]interface{}, error)

ParseTyped returns a inteface{} map with typed captured fields based on provided pattern over the text