bacot

package
v1.2.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 13, 2026 License: MIT Imports: 4 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Bacot 

type Bacot struct {

	// modal scanning
	Dict *Dictionary
	// contains filtered or unexported fields
}

func New 

func New() *Bacot

func (*Bacot) AddFalsePositive added in v1.2.0 

func (b *Bacot) AddFalsePositive(words ...string) *Bacot

func (*Bacot) AddWord 

func (b *Bacot) AddWord(words ...string) *Bacot

func (*Bacot) Config added in v1.2.0 

func (b *Bacot) Config(config *ModalScanConfig) *Bacot

Config

func (*Bacot) DelWord added in v1.2.0 

func (b *Bacot) DelWord(words ...string) *Bacot

func (*Bacot) Raw added in v1.2.0 

func (b *Bacot) Raw(s string) *ModalScan

Mentah, setting sendiri

func (*Bacot) Text 

func (b *Bacot) Text(s string) *ModalScan

Text() menjalankan pipeline preprocessing default: 

Emoji → ReplaceWhiteSpace → SanitizeReadSign → ReplaceWhiteSpace → UnstackChar → Affix(true)

Urutan ini penting karena:

  1. Emoji dihapus dulu biar ga jadi noise
  2. Tanda baca diganti spasi biar tokenisasi akurat
  3. Unstack dilakukan setelah leet speak (default leet tidak aktif, perlu manual)
  4. Affix detection aktif default, bisa dimatikan dengan Affix(false)

Config() bisa override pipeline ini. Lihat ModalScanConfig.Order.

type CraftManConfig added in v1.2.0 

type CraftManConfig struct {
	PrefixNasalFusion bool
}

type DictWords 

type DictWords = map[string]struct{}

func NewDictWord 

func NewDictWord(words ...string) DictWords

type Dictionary 

type Dictionary struct {
	// contains filtered or unexported fields
}

func NewDictionary 

func NewDictionary() *Dictionary

func (*Dictionary) AddFalsePositive added in v1.2.2 

func (d *Dictionary) AddFalsePositive(words ...string)

func (*Dictionary) AddWords 

func (d *Dictionary) AddWords(words ...string)

func (*Dictionary) Contains 

func (d *Dictionary) Contains(word string) bool

func (*Dictionary) DelWords 

func (d *Dictionary) DelWords(words ...string)

func (*Dictionary) GetDict 

func (d *Dictionary) GetDict() DictWords

func (*Dictionary) GetWordsLen 

func (d *Dictionary) GetWordsLen() []int

func (*Dictionary) IsContainLen 

func (d *Dictionary) IsContainLen(n int) bool

func (*Dictionary) IsFalsePositive added in v1.2.0 

func (d *Dictionary) IsFalsePositive(s string) bool

func (*Dictionary) IsStopWord 

func (d *Dictionary) IsStopWord(s string) bool

func (*Dictionary) Majority added in v1.2.2 

func (d *Dictionary) Majority() int

func (*Dictionary) Max 

func (d *Dictionary) Max() int

func (*Dictionary) Min 

func (d *Dictionary) Min() int

type ModalScan 

type ModalScan struct {
	// contains filtered or unexported fields
}

func (*ModalScan) Affix 

func (ms *ModalScan) Affix(v bool) *ModalScan

func (*ModalScan) ClearSpace added in v1.2.0 

func (ms *ModalScan) ClearSpace() *ModalScan

func (*ModalScan) Collect 

func (ms *ModalScan) Collect(v bool) *ModalScan

func (*ModalScan) GetText 

func (ms *ModalScan) GetText() string

func (*ModalScan) RecursiveScan 

func (ms *ModalScan) RecursiveScan() *ScanResult

RecursiveScan() mencari kata kotor di dalam substring token. Berguna untuk kasus seperti "xbabi" di mana kata kotor menempel dengan karakter lain. RecursiveScan akan sliding window dari setiap posisi l di dalam token dan mencocokkan dengan dictionary.

Kompleksitas: O(token_length × dict_word_lengths). Untuk token normal (<20 char) sangat cepat. Untuk token sangat panjang (10K char) perlu diwaspadai.

func (*ModalScan) ReplaceWhiteSpace added in v1.2.0 

func (ms *ModalScan) ReplaceWhiteSpace() *ModalScan

func (*ModalScan) SanitizeEmoji added in v1.2.0 

func (ms *ModalScan) SanitizeEmoji() *ModalScan

func (*ModalScan) SanitizeNewLine added in v1.2.0 

func (ms *ModalScan) SanitizeNewLine() *ModalScan

func (*ModalScan) SanitizeReadSign added in v1.2.0 

func (ms *ModalScan) SanitizeReadSign() *ModalScan

func (*ModalScan) Scan 

func (ms *ModalScan) Scan() *ScanResult

Scan() menggunakan strategi token-based (split by spasi), bukan substring search. Ini mengurangi false positive: "kelas" tidak akan terdeteksi sebagai "elas" karena token exact match.

Optimasi: length histogram pre-filter — jika tidak ada kata di dictionary dengan panjang yang sama, token langsung skip. Ini menghindari map lookup yang tidak perlu.

func (*ModalScan) TrimSpace 

func (ms *ModalScan) TrimSpace() *ModalScan

func (*ModalScan) UnstackChar 

func (ms *ModalScan) UnstackChar() *ModalScan

func (*ModalScan) WithLeetSpeak 

func (ms *ModalScan) WithLeetSpeak() *ModalScan

Leet mengandung angka jika case nya := "babi123" -> "babiire" -> "babire" dengan affix "babire" tidak dianggap kata kotor karena "babi" + sukukata, beda makna maka jika lolos di leetspeak, scan kedua kali yang tanpa sanitasi leetspeak

kesimpulan: jika input mengandung elemen leetspeak dan lolos scan scan ulang dengan mematikan leet

type ModalScanConfig added in v1.2.0 

type ModalScanConfig struct {
	Affix   bool
	Collect bool

	Order []SanitizeOrder
}

type SanitizeOrder added in v1.2.0 

type SanitizeOrder int
const (
	SanitizeNewLine SanitizeOrder = iota
	ClearSpace
	WithLeetSpeak
	UnstackChar
	// new
	TrimSpace
	ReplaceWhiteSpace
	SanitizeReadSign
)

type ScanResult 

type ScanResult struct {
	// contains filtered or unexported fields
}

ScanResult adalah hasil dari proses scanning. Ia menyimpan dua versi teks:

  • text: teks asli input (preserve huruf besar/kecil)
  • praScanText: teks setelah preprocessing (lowercased + sanitized)

WordIndex menyimpan posisi start/end tiap kata kotor yang ditemukan, memungkinkan operasi seperti Censor() tanpa perlu scanning ulang.

func (*ScanResult) Censor 

func (sr *ScanResult) Censor() string

Censor() mengganti character kata kotor dengan '*'. Arsitektur: menggunakan precomputed WordIndex dari Scan().

Strategi:

  1. Replace langsung di []rune praScanText menggunakan WordIndex (O(1) per kata, tanpa perlu scanning ulang).
  2. Jika teks asli punya huruf besar/kecil, petakan kembali dari teks asli ke teks yang sudah dicensor.
  3. Jika teks sudah lowercase semua, tidak perlu mapping.

func (*ScanResult) Count 

func (sr *ScanResult) Count() int

func (*ScanResult) Extract 

func (sr *ScanResult) Extract() []string

func (*ScanResult) First 

func (sr *ScanResult) First() string

func (*ScanResult) Generator 

func (sr *ScanResult) Generator() *WordIndexGenerator

func (*ScanResult) GetText 

func (sr *ScanResult) GetText() string

func (*ScanResult) IsProfane 

func (sr *ScanResult) IsProfane() bool

func (*ScanResult) Last 

func (sr *ScanResult) Last() string

type WordIndex 

type WordIndex struct {
	Word  string
	Start int
	End   int
}

type WordIndexGenerator 

type WordIndexGenerator struct {
	// contains filtered or unexported fields
}

func (*WordIndexGenerator) Yield 

func (wig *WordIndexGenerator) Yield() *WordIndex

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.