utils

Documentation

Index

• func CombineSentences(sentences []string, delimeter string) string
• func SplitSentences(content string) ([]string, error)

Functions

func CombineSentences

func CombineSentences(sentences []string, delimeter string) string

CombineSentences will take the output from the SplitSentences func and combine it back into a single string using the passed delimiter. This may be used as a preparation step for writing the sentences all back into a single output file.

func SplitSentences

func SplitSentences(content string) ([]string, error)

SplitSentences will use the neurosnap/sentences tokenizer to split the content up into a slice of strings. Each string is a sentence, based on existing punctuation. This util method is hardcoded to english at the moment but the sentence splitter does support multiple languages which can be enabled later.