ngram

package module
Version: v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 18, 2018 License: MIT Imports: 1 Imported by: 0

README

Ngram for Golang Last release

Go Report Card

Branch Status Coverage
master Build Status Coveralls

an n-gram is a contiguous sequence of n items from a given sequence of text or speech.

import (
    "fmt"

    "github.com/euskadi31/go-ngram"
)

func main() {
    tokens := ngram.New("Hello", ngram.Bigram).Tokenize()

    fmt.Printf("%v\n", tokens) // ["He", "el", "ll", "lo"]
}

License

go-ngram is licensed under the MIT license.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type SizeType

type SizeType int

SizeType type

const (
	Unigram SizeType = 1
	Bigram  SizeType = 2
	Trigram SizeType = 3
)

N-Gram size type

type Tokenizer

type Tokenizer interface {
	Tokenize() []string
}

Tokenizer interface

func New

func New(content string, size SizeType) Tokenizer

New N-gram Tokenizer

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL