Documentation
¶
Index ¶
- Constants
- type AlignmentRange
- type NormalizedRange
- type NormalizedString
- func (ns *NormalizedString) Append(s string)
- func (ns *NormalizedString) CoerceRangeToNormalized(r Range) (NormalizedRange, bool)
- func (ns *NormalizedString) CoerceRangeToOriginal(r Range) (OriginalRange, bool)
- func (ns *NormalizedString) Filter(keep func(rune) bool)
- func (ns *NormalizedString) Get() string
- func (ns *NormalizedString) GetOriginal() string
- func (ns *NormalizedString) GetOriginalRange(rng Range) (string, bool)
- func (ns *NormalizedString) GetRange(rng Range) (string, bool)
- func (ns *NormalizedString) IsEmpty() bool
- func (ns *NormalizedString) Len() int
- func (ns *NormalizedString) Map(mapFunc func(rune) rune)
- func (ns *NormalizedString) OriginalAlignments() []AlignmentRange
- func (ns *NormalizedString) OriginalLen() int
- func (ns *NormalizedString) OriginalOffsets() strutils.ByteOffsets
- func (ns *NormalizedString) Prepend(s string)
- func (ns *NormalizedString) Replace(pattern splitpattern.SplitPattern, content string) error
- func (ns *NormalizedString) Slice(rng Range) (*NormalizedString, bool)
- func (ns *NormalizedString) Split(pattern splitpattern.SplitPattern, behaviour SplitDelimiterBehavior) ([]*NormalizedString, error)
- func (ns *NormalizedString) ToLower()
- func (ns *NormalizedString) ToUpper()
- func (ns *NormalizedString) Transform(dest []RuneChange, initialOffset int)
- func (ns *NormalizedString) TransformRange(rng Range, dest []RuneChange, initialOffset int)
- func (ns *NormalizedString) Trim()
- func (ns *NormalizedString) TrimLeft()
- func (ns *NormalizedString) TrimLeftRight(left, right bool)
- func (ns *NormalizedString) TrimRight()
- type OriginalRange
- type Range
- type RuneChange
- type SplitDelimiterBehavior
Constants ¶
const ( SplitDelimiterRemoved SplitDelimiterBehavior = iota SplitDelimiterIsolated = iota SplitDelimiterMergedWithPrevious = iota SplitDelimiterMergedWithNext = iota SplitDelimiterContiguous = iota )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AlignmentRange ¶
type AlignmentRange struct {
// contains filtered or unexported fields
}
AlignmentRange represents a (start, end) range for representing the alignments of a NormalizedString.
type NormalizedRange ¶
type NormalizedRange struct {
// contains filtered or unexported fields
}
NormalizedRange represents a range usable by the NormalizedString to index its content, using indices relative to the "normalized" string.
func NewNormalizedRange ¶
func NewNormalizedRange(start, end int) NormalizedRange
func (NormalizedRange) End ¶
func (r NormalizedRange) End() int
func (NormalizedRange) Len ¶
func (r NormalizedRange) Len() int
func (NormalizedRange) Start ¶
func (r NormalizedRange) Start() int
type NormalizedString ¶
type NormalizedString struct {
// contains filtered or unexported fields
}
NormalizedString takes care of processing an "original" string to modify it and obtain a "normalized" string.
It keeps both versions of the string, alignments information between both of them, and provides an interface to retrieve ranges of each string, using offsets from any of them.
It is possible to retrieve a part of the original string, by indexing it with offsets from the normalized one, and the other way around too. It is also possible to convert offsets from one referential to the other one easily.
func FromString ¶
func FromString(s string) *NormalizedString
FromString returns a new NormalizedString built from the given string.
func New ¶
func New( original string, normalized string, alignments []AlignmentRange, originalShift int, ) *NormalizedString
New returns a new NormalizedString.
func (*NormalizedString) Append ¶
func (ns *NormalizedString) Append(s string)
Append appends the given string to the NormalizedString.
FIXME: Append does nothing if the normalized string is empty
func (*NormalizedString) CoerceRangeToNormalized ¶
func (ns *NormalizedString) CoerceRangeToNormalized(r Range) (NormalizedRange, bool)
CoerceRangeToNormalized coerces the given Range (either an OriginalRange or a NormalizedRange) to a NormalizedRange, performing a conversion if needed. It also returns a flag which reports whether the operation was successful.
The operation is unsuccessful if the range is targeting something out of range.
func (*NormalizedString) CoerceRangeToOriginal ¶
func (ns *NormalizedString) CoerceRangeToOriginal(r Range) (OriginalRange, bool)
CoerceRangeToOriginal coerces the given Range (either an OriginalRange or a NormalizedRange) to a OriginalRange, performing a conversion if needed. It also returns a flag which reports whether the operation was successful.
The operation is unsuccessful if the range is targeting something out of range.
func (*NormalizedString) Filter ¶
func (ns *NormalizedString) Filter(keep func(rune) bool)
Filter applies filtering over the characters of the NormalizedString.
func (*NormalizedString) Get ¶
func (ns *NormalizedString) Get() string
Get returns the "normalized" version of the NormalizedString.
func (*NormalizedString) GetOriginal ¶
func (ns *NormalizedString) GetOriginal() string
GetOriginal returns the "original" version of the NormalizedString.
func (*NormalizedString) GetOriginalRange ¶
func (ns *NormalizedString) GetOriginalRange(rng Range) (string, bool)
GetOriginalRange returns a range of the original string, and a flag which reports whether the operation is successful or not.
func (*NormalizedString) GetRange ¶
func (ns *NormalizedString) GetRange(rng Range) (string, bool)
GetRange returns a range of the normalized string, and a flag which reports whether the operation is successful or not.
func (*NormalizedString) IsEmpty ¶
func (ns *NormalizedString) IsEmpty() bool
IsEmpty reports whether the "normalized" string is empty.
func (*NormalizedString) Len ¶
func (ns *NormalizedString) Len() int
Len returns the length in bytes of the "normalized" string.
func (*NormalizedString) Map ¶
func (ns *NormalizedString) Map(mapFunc func(rune) rune)
Map maps the characters of the NormalizedString.
func (*NormalizedString) OriginalAlignments ¶
func (ns *NormalizedString) OriginalAlignments() []AlignmentRange
OriginalAlignments recalculates the original alignments.
func (*NormalizedString) OriginalLen ¶
func (ns *NormalizedString) OriginalLen() int
OriginalLen returns the length in bytes of the "original" string.
func (*NormalizedString) OriginalOffsets ¶
func (ns *NormalizedString) OriginalOffsets() strutils.ByteOffsets
OriginalOffsets returns the original offsets.
func (*NormalizedString) Prepend ¶
func (ns *NormalizedString) Prepend(s string)
Prepend prepends the given string to the NormalizedString.
FIXME: Prepend does nothing if the normalized string is empty
func (*NormalizedString) Replace ¶
func (ns *NormalizedString) Replace( pattern splitpattern.SplitPattern, content string, ) error
Replace replaces anything that matches the pattern with the given content.
func (*NormalizedString) Slice ¶
func (ns *NormalizedString) Slice(rng Range) (*NormalizedString, bool)
Slice returns a slice of the current NormalizedString. It also returns a flag which reports whether the operation is successful or not.
func (*NormalizedString) Split ¶
func (ns *NormalizedString) Split( pattern splitpattern.SplitPattern, behaviour SplitDelimiterBehavior, ) ([]*NormalizedString, error)
Split splits the current string in many subparts.
func (*NormalizedString) ToLower ¶
func (ns *NormalizedString) ToLower()
ToLower remaps all Unicode letters of the "normalized" string to their lower case. FIXME: see Unicode special casing notes on `NormalizedString.ToUpper()`
func (*NormalizedString) ToUpper ¶
func (ns *NormalizedString) ToUpper()
ToUpper remaps all Unicode letters of the "normalized" string to their upper case. FIXME: Go `unicode` package does not consider Unicode special casing
(see https://www.unicode.org/Public/UCD/latest/ucd/SpecialCasing.txt)
As a result, every single rune is always transformed to a single
new rune. This is not the case in the original rust implementation.
Should we consider using another "better" package?
Explanatory example:
```rust
fn main() {
let a = '\u{00DF}';
println!("{} {:?}", a, a.to_string().chars().count());
// => ß 1
let b = a.to_uppercase();
println!("{} {}", b, b.to_string().chars().count());
// => SS 2
}
```
```go
package main
import (
"fmt"
"unicode"
)
func main() {
a := '\u00DF'
fmt.Println(string(a), len([]rune(string(a))))
// => ß 1
b := unicode.ToUpper(a)
fmt.Println(string(b), len([]rune(string(b))))
// => ß 1
}
```
func (*NormalizedString) Transform ¶
func (ns *NormalizedString) Transform(dest []RuneChange, initialOffset int)
Transform applies transformations to the current normalized version of the string, while updating the alignments.
It is the same as calling TransformRange with the full normalized string range.
func (*NormalizedString) TransformRange ¶
func (ns *NormalizedString) TransformRange( rng Range, dest []RuneChange, initialOffset int, )
TransformRange applies transformations to the current normalized version of the string, while updating the alignments.
This method expect a slice of RuneChange, where each item corresponds to one rune of the new normalized string.
Since it is possible that the normalized string doesn't include some of the characters at the beginning of the original one, we need an initialOffset which represents the number of removed chars at the very beginning.
func (*NormalizedString) Trim ¶
func (ns *NormalizedString) Trim()
Trim removes leading and trailing spaces from the "normalized" string.
func (*NormalizedString) TrimLeft ¶
func (ns *NormalizedString) TrimLeft()
TrimLeft removes leading spaces from the "normalized" string.
func (*NormalizedString) TrimLeftRight ¶
func (ns *NormalizedString) TrimLeftRight(left, right bool)
TrimLeftRight removes leading (left) and/or trailing (right) spaces from the "normalized" string.
func (*NormalizedString) TrimRight ¶
func (ns *NormalizedString) TrimRight()
TrimRight removes trailing spaces from the "normalized" string.
type OriginalRange ¶
type OriginalRange struct {
// contains filtered or unexported fields
}
OriginalRange represents a range usable by the NormalizedString to index its content, using indices relative to the "original" string.
func NewOriginalRange ¶
func NewOriginalRange(start, end int) OriginalRange
func (OriginalRange) End ¶
func (r OriginalRange) End() int
func (OriginalRange) Len ¶
func (r OriginalRange) Len() int
func (OriginalRange) Start ¶
func (r OriginalRange) Start() int
type Range ¶
type Range interface {
// Start returns the start index, inclusive.
Start() int
// End returns the end index, exclusive.
End() int
// Len returns the length of the range (i.e. End - Start).
Len() int
}
Range represents a range which can be used by NormalizedString to index its content.
A Range can use indices relative to either the "original" string (see OriginalRange), or the "normalized" string (see NormalizedRange).
type RuneChange ¶
type RuneChange struct {
// Rune is a rune of the new transformed "normalized" string.
Rune rune
// Values greater than "1" are not allowed. To add multiple runes, each of
// them must be represented by a separate RuneChange value, with Changes
// set to "1".
Change int
}
RuneChange is a single rune-based transformation that can be applied by NormalizedString.TransformRange
type SplitDelimiterBehavior ¶
type SplitDelimiterBehavior uint8
SplitDelimiterBehavior is used by NormalizedString.Split to define the expected behavior for the delimiter.
For example, when splitting on '-' with input `the-final--countdown`: - SplitDelimiterRemoved => [ "the", "final", "countdown" ] - SplitDelimiterIsolated => [ "the", "-", "final", "-", "-", "countdown" ] - SplitDelimiterMergedWithPrevious => [ "the-", "final-", "-", "countdown" ] - SplitDelimiterMergedWithNext => [ "the", "-final", "-", "-countdown" ] - SplitDelimiterContiguous => [ "the", "-", "final", "--", "countdown" ]
Source Files