mark

Documentation

Index

• type Class
• type Mark
  • func Clean(src string) (cleaned string, acted []Mark)
  • func Scan(src string) []Mark

Types

type Class

type Class uint8

Class groups a marker codepoint by what a caller may safely do with it.

const (
	// ClassStrip is removable without reading a single neighbour: the codepoint
	// carries no meaning in Go source or Markdown, whatever surrounds it.
	ClassStrip Class = iota
	// ClassSpace is a space homoglyph, replaced by U+0020 rather than deleted.
	ClassSpace
	// ClassContextual is invisible but may be load-bearing (emoji glue, a script
	// joiner), so it is reported and never rewritten.
	ClassContextual
	// ClassTypographic is visible punctuation an AI model reaches for by default:
	// dashes, ellipsis, curly quotes. Reported, never rewritten - inside a string
	// literal or a fenced block it can be deliberate data.
	ClassTypographic
)

type Mark

type Mark struct {
	Rune   rune
	Class  Class
	Name   string
	Offset int
	Line   int
}

Mark is one marker occurrence, located by byte offset so a caller can report a line or splice the rune out without re-scanning.

func Clean

func Clean(src string) (cleaned string, acted []Mark)

Clean removes every ClassStrip marker and folds every ClassSpace one to a plain space, leaving ClassContextual and ClassTypographic untouched.

why: guessing at emoji glue or at a dash inside a string literal would corrupt data the author meant to keep, after the write tool reported success.

func Scan

func Scan(src string) []Mark

Scan reports every marker in src, in byte order.