kaito

README

Kaito

Kaito (káitəʊ) is auto-detection and decompression tool for Go.

Motivation

These days, I've been working on log files with several formats; some are Gzipped, others are Xz-compressed, and even others are just plain text. OMG! I don't mind whatever compression format is used, but just want plain content.

Usage

Just make kaito.Reader from another io.Reader, then read from it.

Example:

package main

import (
	"bufio"
	"fmt"
	"io"
	"os"

	"github.com/Maki-Daisuke/go-kaito"
)

func main() {
	for _, f := range os.Args[1:] {
		file, err := os.Open(f)
		if err != nil {
			panic(err)
		}
		k := kaito.New(file)
		r := bufio.NewReader(k)
		for {
			line, err := r.ReadString('\n')
			if err != nil {
				if err == io.EOF {
					break
				}
				panic(err)
			}
			fmt.Print(line) // `Here, line is decompressed string if the file is compressed, as-is otherwise.
		}
	}
}

You can make it from any kind of io.Reader. For example, you can easily implement your own filter command with reading STDIN:

package main

import (
	"bufio"
	"fmt"
	"io"
	"os"

	"github.com/Maki-Daisuke/go-kaito"
)

func main() {
	k := kaito.New(os.Stdin)
	r := bufio.NewReader(k)
	for {
		line, err := r.ReadString('\n')
		if err != nil {
			if err == io.EOF {
				break
			}
			panic(err)
		}
		// Do what you want to do.
	}
}

Prerequisites

It is recommended to install gzip, bzip2 and xz commands. Kaito tries to use these commands, and fallbacks to Go-native implementation of those algorithms if the commands fail. Thus, those commands are not mandatory, but they are much faster than Go-native implementations according to my experience.

License

The Simplified BSD License (2-clause)

Author

Daisuke (yet another) Maki

Documentation

Index

• Constants
• func New(r io.Reader) io.Reader
• func NewWithOptions(r io.Reader, o Options) io.Reader
• type Options
  • func (o Options) IsDisableBzip2() bool
  • func (o Options) IsDisableGzip() bool
  • func (o Options) IsDisableXz() bool
  • func (o Options) IsForceNative() bool
• type Reader

Constants

const (
	DisableGzip  = 1 << iota
	DisableBzip2 = 1 << iota
	DisableXz    = 1 << iota
	ForceNative  = 1 << iota
)

Functions

func New

func New(r io.Reader) io.Reader

func NewWithOptions

func NewWithOptions(r io.Reader, o Options) io.Reader

Types

type Options

type Options uint16

func (Options) IsDisableBzip2

func (o Options) IsDisableBzip2() bool

func (Options) IsDisableGzip

func (o Options) IsDisableGzip() bool

func (Options) IsDisableXz

func (o Options) IsDisableXz() bool

func (Options) IsForceNative

func (o Options) IsForceNative() bool

type Reader

type Reader struct {
	io.Reader // Just delegate all Read operation to this Reader
}