text

Documentation

Index

• Variables
• func WrapFunc(fn func(string) string) func(string) (string, error)
• type Cutter
  • func (p Cutter) Describe() string
  • func (Cutter) Once() bool
  • func (p Cutter) Process(s string) (string, error)
• type Processor
  • func CutSpace() Processor
  • func NewMultiProcessor(desc string, once bool, procs ...Processor) Processor
  • func NewProcessor(desc string, once bool, fn func(string) (string, error)) Processor
  • func RemoveParentheses() Processor
  • func TrimSpace() Processor
• type RegexpRemover
  • func (p RegexpRemover) Describe() string
  • func (RegexpRemover) Once() bool
  • func (p RegexpRemover) Process(s string) (string, error)
• type Tasks
  • func NewTasks(tasks ...Processor) *Tasks
  • func (t *Tasks) Append(tasks ...Processor) *Tasks
  • func (t *Tasks) Process(s string) (output string, err error)
  • func (t *Tasks) ProcessAll(inputs []string) ([]string, error)
• type Trimmer
  • func (p Trimmer) Describe() string
  • func (Trimmer) Once() bool
  • func (p Trimmer) Process(s string) (string, error)

Variables

var MaxIter = 100

Functions

func WrapFunc

func WrapFunc(fn func(string) string) func(string) (string, error)

WrapFunc wraps a simple string -> string function into a function matching the Processor signature.

Types

type Cutter

type Cutter struct {
	Sep string
}

Cutter splits the input by the given separator and keeps only the part before it.

func (Cutter) Describe

func (p Cutter) Describe() string

Describe returns a string representation of the Cutter.

func (Cutter) Once

func (Cutter) Once() bool

Once always returns true, meaning this processor should be run only once.

func (Cutter) Process

func (p Cutter) Process(s string) (string, error)

Process cuts the string at the first occurrence of the separator and returns the left part.

type Processor

type Processor interface {
	// Describe returns a short description of this processor.
	Describe() string
	// Once reports whether this processor should run only once.
	Once() bool
	// Process performs the actual text transformation.
	Process(string) (string, error)
}

Processor defines a generic interface for text processors. Each processor can optionally be executed only once (Once == true) and provides a human-readable description for debugging or logging.

func CutSpace

func CutSpace() Processor

CutSpace returns a processor that extracts the first word in the input string.

func NewMultiProcessor

func NewMultiProcessor(desc string, once bool, procs ...Processor) Processor

NewMultiProcessor creates a new MultiProcessor.

desc  - human-readable name
once  - whether this processor should execute only once
procs - list of sub-processors

func NewProcessor

func NewProcessor(desc string, once bool, fn func(string) (string, error)) Processor

NewProcessor creates a new Processor from a function.

desc - short description for debugging
once - whether this processor should be executed only once
fn   - transformation function taking a string and returning a string/error

func RemoveParentheses

func RemoveParentheses() Processor

RemoveParentheses returns a processor that remove both western and full-width parentheses.

func TrimSpace

func TrimSpace() Processor

TrimSpace returns a processor that removes leading and trailing spaces.

type RegexpRemover

type RegexpRemover struct {
	Re *regexp.Regexp
}

RegexpRemover removes substrings that match the given regular expression.

func (RegexpRemover) Describe

func (p RegexpRemover) Describe() string

Describe returns a string representation of the RegexpRemover.

func (RegexpRemover) Once

func (RegexpRemover) Once() bool

Once always returns false, meaning this processor can be applied repeatedly.

func (RegexpRemover) Process

func (p RegexpRemover) Process(s string) (string, error)

Process removes all matches of the regular expression from the input string.

type Tasks

type Tasks struct {
	// contains filtered or unexported fields
}

Tasks represents an ordered list of text processors. Each processor in the list will be executed sequentially, and repeated until the text no longer changes.

func NewTasks

func NewTasks(tasks ...Processor) *Tasks

NewTasks creates a new Tasks instance with the provided processors.

func (*Tasks) Append

func (t *Tasks) Append(tasks ...Processor) *Tasks

Append adds one or more processors to the task list and returns the updated instance.

func (*Tasks) Process

func (t *Tasks) Process(s string) (output string, err error)

Process executes all configured processors on a single string.

func (*Tasks) ProcessAll

func (t *Tasks) ProcessAll(inputs []string) ([]string, error)

ProcessAll applies all processors to a slice of strings, returning the processed results. If any processor returns an error, processing stops and the error is returned.

type Trimmer

type Trimmer struct {
	Cutset string
}

Trimmer removes all leading and trailing characters from the given cutset.

func (Trimmer) Describe

func (p Trimmer) Describe() string

Describe returns a string representation of the Trimmer.

func (Trimmer) Once

func (Trimmer) Once() bool

Once always returns false, meaning this processor can be applied repeatedly.

func (Trimmer) Process

func (p Trimmer) Process(s string) (string, error)

Process trims all leading and trailing characters in Cutset from the input string.