split

Documentation

Index

• func SplitParagraphs(maxLength int, input string) ([]string, error)
• func SplitSentences(maxLength int, input string) ([]string, error)
• func SplitText(maxLength int, input string) ([]string, error)

Functions

func SplitParagraphs

func SplitParagraphs(maxLength int, input string) ([]string, error)

SplitParagraphs splits the input text into paragraphs. A paragraph is defined as a sequence of characters that ends with two or more newlines. When paragraphs are merged, each paragraph will be separated with a single newline. If a paragraph is too long, it will be split at sentence boundaries.

func SplitSentences

func SplitSentences(maxLength int, input string) ([]string, error)

SplitSentences splits the input text into sentences. A sentence is defined as a sequence of characters that ends with a period, exclamation mark, or question mark, followed by a space or end of input. The ending punctuation/space will be included in the output.

func SplitText

func SplitText(maxLength int, input string) ([]string, error)

SplitText splits the input text into chunks of at most maxLength bytes. The text will be split at paragraph boundaries, if possible. If the paragraph is too long, it will be split at sentence boundaries. If the sentence is too long, it will return an error as the text is considered invalid for this operation.