processor

Documentation

Overview

Code generated by scripts/include.go using 'go generate'. DO NOT EDIT.

Index

• Constants
• Variables
• func BloomHash(b byte) uint64
• func ConfigureGc()
• func ConfigureLazy(lazy bool)
• func CountStats(fileJob *FileJob)
• func DetectLanguage(name string) ([]string, string)
• func DetectSheBang(content string) (string, error)
• func DetermineLanguage(filename string, fallbackLanguage string, possibleLanguages []string, ...) string
• func EstimateCost(effortApplied float64, averageWage int64, overhead float64) float64
• func EstimateEffort(sloc int64, eaf float64) float64
• func EstimateScheduleMonths(effortApplied float64) float64
• func GetMostSimilarFlags(flagSet *pflag.FlagSet, flag string) []string
• func LanguageDatabase() map[string]Language
• func LoadLanguageFeature(loadName string)
• func LocomoComplexityDensity(complexity, code int64) float64
• func LocomoComplexityFactor(complexityDensity, complexityWeight float64) float64
• func LocomoIterationFactor(complexityDensity, baseIterations, iterationWeight float64) float64
• func PrintLanguages(dst io.Writer)
• func Process()
• func ProcessConstants()
• func StringSimilarRatio(s1, s2 string) float64
• type BaselineFile
• type BaselineObserver
• type BaselineSnapshot
• type Bucketing
  • func NewBucketing(from, to time.Time, n int) Bucketing
  • func (b Bucketing) Index(t time.Time) int
  • func (b Bucketing) Start(i int) time.Time
• type CheckDuplicates
  • func (c *CheckDuplicates) Add(key int64, hash []byte)
  • func (c *CheckDuplicates) Check(key int64, hash []byte) bool
• type CommitInfo
• type CommitObserver
• type FileChange
• type FileJob
  • func (fj *FileJob) FilterContentByType(keepTypes ...byte) []byte
• type FileJobCallback
• type FileReader
  • func NewFileReader() FileReader
  • func (reader *FileReader) ReadFile(path string, size int) ([]byte, error)
• type HeadFile
• type HeadSnapshot
• type HistoryWindow
• type Json2
• type Language
• type LanguageFeature
• type LanguageSummary
  • func ProcessResult() ([]LanguageSummary, error)
• type LineRange
• type LineType
• type LocomoPreset
  • func GetLocomoPreset(name string) LocomoPreset
• type LocomoResult
  • func LocomoEstimate(sumCode, sumComplexity int64) LocomoResult
• type MailmapObserver
• type OpenClose
• type Quote
• type Trie
  • func (root *Trie) Insert(tokenType int, token []byte)
  • func (root *Trie) InsertClose(tokenType int, openToken, closeToken []byte)
  • func (root *Trie) Match(token []byte) (int, int, []byte)

Constants

const (
	TString int = iota + 1
	TSlcomment
	TMlcomment
	TComplexity
	TComplexityPostfix
)

Used by trie structure to store the types

const (
	ByteTypeBlank   byte = 0
	ByteTypeCode    byte = 1
	ByteTypeComment byte = 2
	ByteTypeString  byte = 3
)

ByteType constants for per-byte content classification. When FileJob.ClassifyContent is true, CountStats populates FileJob.ContentByteType with one of these values per byte.

const (
	SBlank             int64 = 1
	SCode              int64 = 2
	SComment           int64 = 3
	SCommentCode       int64 = 4 // Indicates comment after code
	SMulticomment      int64 = 5
	SMulticommentCode  int64 = 6 // Indicates multi comment after code
	SMulticommentBlank int64 = 7 // Indicates multi comment ended with blank afterward
	SString            int64 = 8
	SDocString         int64 = 9
)

The below are used as identifiers for the code state machine

const HotspotsTopN = 20

HotspotsTopN is the cap on rows shown in the tabular hotspots report. CSV / JSON output is not capped.

const SheBang string = "#!"

SheBang is a global constant for indicating a shebang file header

const SimilarStringThreshold float64 = 0.75

Variables

var AllowListExtensions = []string{}

AllowListExtensions is a list of extensions which are allowed to be processed

var AverageWage int64 = 56286

AverageWage is the average wage in dollars used for the COCOMO cost estimate

var BloomTable [256]uint64

var ByAuthor = false

ByAuthor toggles the author-rollup git-history report

var ByteOrderMarks = [][]byte{
	{254, 255},
	{255, 254},
	{0, 0, 254, 255},
	{255, 254, 0, 0},
	{43, 47, 118, 56},
	{43, 47, 118, 57},
	{43, 47, 118, 43},
	{43, 47, 118, 47},
	{43, 47, 118, 56, 45},
	{247, 100, 76},
	{221, 115, 102, 115},
	{14, 254, 255},
	{251, 238, 40},
	{132, 49, 149, 51},
}

ByteOrderMarks are taken from https://en.wikipedia.org/wiki/Byte_order_mark#Byte_order_marks_by_encoding These indicate that we cannot count the file correctly so we can at least warn the user

var Ci = false

Ci indicates if running inside a CI so to disable box drawing characters

var Cocomo = false

Cocomo toggles the COCOMO calculation

var CocomoProjectType = "organic"

CocomoProjectType allows the flipping between project types which impacts the calculation

var Complexity = false

Complexity toggles complexity calculation

var CostComparison = false

CostComparison enables both COCOMO and LOCOMO output for side-by-side comparison

var CountAs = ""

CountAs is a rule for mapping known or new extensions to other rules

var CountIgnore = false

CountIgnore should we count ignore files?

var CurrencySymbol = ""

CurrencySymbol allows setting the currency symbol for cocomo project cost estimation

var Debug = false

Debug enables debug logging output

var DirFilePaths = []string{}

DirFilePaths is not set via flags but by arguments following the flags for file or directory to process

var DirectoryWalkerJobWorkers = 8

DirectoryWalkerJobWorkers is the number of workers which will walk the directory tree

var DisableCheckBinary = false

DisableCheckBinary toggles checking for binary files using NUL bytes

var Dryness = false

Dryness toggles checking for binary files using NUL bytes

var Duplicates = false

Duplicates enables duplicate file detection

var EAF float64 = 1.0

EAF is the effort adjustment factor derived from the cost drivers, i.e. 1.0 if rated nominal

var Exclude = []string{}

Exclude is a regular expression which is used to exclude files from being processed

var ExcludeFilename = []string{}

ExcludeFilename is a list of filenames which should be ignored

var ExcludeListExtensions = []string{}

ExcludeListExtensions is a list of extensions which should be ignored

var ExtensionToLanguage = map[string][]string{}

ExtensionToLanguage is loaded from the JSON that is in constants.go

var FileListQueueSize = runtime.NumCPU()

FileListQueueSize is the queue of files found and ready to be read into memory

var FileOutput = ""

FileOutput sets the file that output should be written to

var FileProcessJobWorkers = runtime.NumCPU() * 4

FileProcessJobWorkers is the number of workers that process the file collecting stats

var FileSummaryJobQueueSize = runtime.NumCPU()

FileSummaryJobQueueSize is the queue used to hold processed file statistics before formatting

var FilenameToLanguage = map[string]string{}

FilenameToLanguage similar to ExtensionToLanguage loaded from the JSON in constants.go

var Files = false

Files indicates if there should be file output or not when formatting

var FoldAuthors = true

FoldAuthors enables the name+domain identity folding fallback applied after the mailmap. Toggled off via --no-fold-authors.

var Format = ""

Format sets the output format of the formatter

var FormatMulti = ""

FormatMulti is a rule for defining multiple output formats

var GcFileCount = 10000

GcFileCount is the number of files to process before turning the GC back on

var Generated = false

Generated enables generated file detection

var GeneratedMarkers []string

GeneratedMarkers defines head markers for generated file detection

var GitIgnore = false

GitIgnore disables .gitignore checks

var GitModuleIgnore = false

GitModuleIgnore disables .gitmodules checks

var HBorder = false

Draw horizontal borders between sections.

var HistoryBuckets = 60

HistoryBuckets is the time-bucket resolution for the timeline reports. Wired to --buckets in main.go; default 60.

var HistoryDepth = 1000

HistoryDepth is the maximum number of commits the history engine walks. 0 means "entire history". Wired to --depth in main.go.

var Hotspots = false

Hotspots toggles the hotspots git-history report

var Ignore = false

Ignore disables ignore file checks

var IgnoreGenerated = false

IgnoreGenerated ignore printing counts for generated files

var IgnoreMinified = false

IgnoreMinified ignore printing counts for minified files

var IgnoreMinifiedGenerate = false

IgnoreMinifiedGenerate printing counts for minified/generated files

var IncludeSymLinks = false

IncludeSymLinks if set true will count symlink files

var LanguageFeatures = map[string]LanguageFeature{}

LanguageFeatures contains the processed languages from processLanguageFeature

var LanguageFeaturesMutex = sync.Mutex{}

LanguageFeaturesMutex is the shared mutex used to control getting and setting of language features used rather than sync.Map because it turned out to be marginally faster

var Languages = false

Languages indicates if the command line should print out the supported languages

var LargeByteCount int64 = 1000000

LargeByteCount number of bytes before being counted as a large file based on https://github.com/pinpt/ripsrc/blob/master/ripsrc/fileinfo/fileinfo.go#L44

var LargeLineCount int64 = 40000

LargeLineCount number of lines before being counted as a large file based on https://github.com/pinpt/ripsrc/blob/master/ripsrc/fileinfo/fileinfo.go#L44

var Locomo = false

Locomo toggles the LOCOMO (LLM Output COst MOdel) calculation

var LocomoBaseInputPerLine float64 = 20

LocomoBaseInputPerLine is the base number of input tokens per output line

var LocomoComplexityWeight float64 = 5

LocomoComplexityWeight is the scaling weight applied to sqrt(complexity density) for input tokens

var LocomoConfig = ""

LocomoConfig is the power-user config string "tokensPerLine,baseInputPerLine,complexityWeight,iterations,iterationWeight"

var LocomoCyclesOverride float64

LocomoCyclesOverride is the user-supplied iteration factor override (--locomo-cycles)

var LocomoCyclesSet = false

LocomoCyclesSet indicates whether --locomo-cycles was explicitly set

var LocomoInputPrice float64

LocomoInputPrice is the cost per 1M input tokens (overrides preset)

var LocomoInputPriceSet = false

var LocomoIterationWeight float64 = 2

LocomoIterationWeight is the scaling weight for complexity-driven retries

var LocomoIterations float64 = 1.5

LocomoIterations is the base number of iteration/retry attempts

var LocomoOutputPrice float64

LocomoOutputPrice is the cost per 1M output tokens (overrides preset)

var LocomoOutputPriceSet = false

var LocomoPresetName = "medium"

LocomoPresetName is the LLM model preset for pricing and throughput defaults

var LocomoReviewMinutesPerLine float64 = 0.01

LocomoReviewMinutesPerLine is the human review time per line of code in minutes

var LocomoTPS float64

LocomoTPS is the output tokens per second (overrides preset)

var LocomoTPSSet = false

var LocomoTokensPerLine float64 = 10

LocomoTokensPerLine is the average number of output tokens per line of code

var MaxMean = false

MaxMean sets the calculation of the max and mean line length

var Minified = false

Minified enables minified file detection

var MinifiedGenerated = false

MinifiedGenerated enables minified/generated file detection

var MinifiedGeneratedLineByteLength = 255

MinifiedGeneratedLineByteLength number of bytes per average line to determine file is minified/generated

var More = false

More enables wider output with more information in formatter

var NoLarge = false

NoLarge if set true will ignore files over a certain number of lines or bytes

var Overhead float64 = 2.4

Overhead is the overhead multiplier for corporate overhead (facilities, equipment, accounting, etc.)

var PathDenyList = []string{}

PathDenyList sets the paths that should be skipped

var Percent = false

Percent toggles checking for binary files using NUL bytes

var RemapAll = ""

RemapAll allows remapping of all files with a string to search the content for

var RemapUnknown = ""

RemapUnknown allows remapping of unknown files with a string to search the content for

var SLOCCountFormat = false

SLOCCountFormat prints a more SLOCCount like COCOMO calculation

var SQLProject = ""

SQLProject is used to store the name for the SQL insert formats but is optional

var SccIgnore = false

SccIgnore disables sccignore file checks

var ShebangLookup = map[string][]string{}

ShebangLookup loaded from the JSON in constants.go contains shebang lookups

var Size = false

Size toggles the Size calculation

var SizeUnit = "si"

SizeUnit determines what size calculation is used for megabytes

var SortBy = ""

SortBy sets which column output in formatter should be sorted by

var Timeline = false

Timeline selects an over-time view. With ByAuthor, runs the author timeline report (plan 04); alone, runs the languages-over-time report (plan 05). With Hotspots set, the combination errors out.

var Trace = false

Trace enables trace logging output which is extremely verbose

var UlocMode = false

UlocMode toggles checking for binary files using NUL bytes

var Verbose = false

Verbose enables verbose logging output

var Version = "3.8.0 (beta)"

Version indicates the version of the application

Functions

func BloomHash

func BloomHash(b byte) uint64

func ConfigureGc

func ConfigureGc()

ConfigureGc needs to be set outside of ProcessConstants because it should only be enabled in command line mode https://github.com/secusolutions/scc/issues/32

func ConfigureLazy

func ConfigureLazy(lazy bool)

ConfigureLazy is a simple setter used to turn on lazy loading used only by command line

func CountStats

func CountStats(fileJob *FileJob)

CountStats will process the fileJob If the file contains anything even just a newline its line count should be >= 1. If the file has a size of 0 its line count should be 0. Newlines belong to the line they started on so a file of \n means only 1 line This is the 'hot' path for the application and needs to be as fast as possible

func DetectLanguage

func DetectLanguage(name string) ([]string, string)

DetectLanguage detects a language based on the filename returns the language extension and error

func DetectSheBang

func DetectSheBang(content string) (string, error)

DetectSheBang given some content attempt to determine if it has a #! that maps to a known language and return the language

func DetermineLanguage

func DetermineLanguage(filename string, fallbackLanguage string, possibleLanguages []string, content []byte) string

DetermineLanguage given a filename, fallback language, possible languages and content make a guess to the type. If multiple possible it will guess based on keywords similar to how https://github.com/vmchale/polyglot does

func EstimateCost

func EstimateCost(effortApplied float64, averageWage int64, overhead float64) float64

EstimateCost calculates the cost in dollars applied using generic COCOMO weighted values based on the average yearly wage

func EstimateEffort

func EstimateEffort(sloc int64, eaf float64) float64

EstimateEffort calculate the effort applied using generic COCOMO weighted values

func EstimateScheduleMonths

func EstimateScheduleMonths(effortApplied float64) float64

EstimateScheduleMonths estimates the effort in months based on the result from EstimateEffort

func GetMostSimilarFlags

func GetMostSimilarFlags(flagSet *pflag.FlagSet, flag string) []string

func LanguageDatabase

func LanguageDatabase() map[string]Language

LanguageDatabase provides access to the internal language database useful for consuming applications wanting to consume and use

func LoadLanguageFeature

func LoadLanguageFeature(loadName string)

LoadLanguageFeature will load a single feature as requested given the name

func LocomoComplexityDensity

func LocomoComplexityDensity(complexity, code int64) float64

LocomoComplexityDensity calculates complexity/code with a guard for division by zero

func LocomoComplexityFactor

func LocomoComplexityFactor(complexityDensity, complexityWeight float64) float64

LocomoComplexityFactor calculates the input token scaling factor based on complexity density Uses sqrt scaling to prevent runaway compounding

func LocomoIterationFactor

func LocomoIterationFactor(complexityDensity, baseIterations, iterationWeight float64) float64

LocomoIterationFactor calculates the iteration/retry multiplier based on complexity density Uses sqrt scaling to prevent runaway compounding

func PrintLanguages

func PrintLanguages(dst io.Writer)

func Process

func Process()

Process is the main entry point of the command line it sets everything up and starts running

func ProcessConstants

func ProcessConstants()

ProcessConstants is responsible for setting up the language features based on the JSON file that is stored in constants Needs to be called at least once in order for anything to actually happen

func StringSimilarRatio

func StringSimilarRatio(s1, s2 string) float64

StringSimilarRatio calculates the similarity ratio between s1 and s2.

The function is based on the Levenshtein distance. The ratio is calculated using the formula:

1 - (Levenshtein Distance / length of the longer string).

It returns a float64 between 0.0 (completely dissimilar) and 1.0 (identical). Based on experience, a ratio >= SimilarStringThreshold can generally be considered to indicate that two strings are highly similar.

Note: The comparison is case-sensitive. For example, "hello" and "HELLO" will be treated as completely different strings (their similarity ratio is 0).

Types

type BaselineFile

type BaselineFile struct {
	Path       string
	Language   string
	LineTypes  []LineType
	Complexity []int // 1-based line numbers that fired a complexity tick
}

BaselineFile is one file from the window's start-commit tree, classified by scc's engine. Carries per-line type and complexity placement so observers can attribute lines that survive untouched from before the window.

type BaselineObserver

type BaselineObserver interface {
	Seed(BaselineSnapshot)
}

BaselineObserver is an optional extension to CommitObserver. When an observer implements it, the engine builds the baseline snapshot before the walk and calls Seed once. Observers that don't need the baseline (e.g. Hotspots) skip the expense by not implementing the interface.

type BaselineSnapshot

type BaselineSnapshot struct {
	Files   map[string]BaselineFile
	Mailmap *mailmap
}

BaselineSnapshot is the optional pre-walk state handed to observers that implement BaselineObserver. Files holds the classified contents of the window's start-commit tree (empty when the window covers all history); Mailmap is the parsed .mailmap from the HEAD tree, if present.

type Bucketing

type Bucketing struct {
	From  time.Time
	To    time.Time
	N     int
	Width time.Duration
}

Bucketing divides [From, To] into N equal time slices. Used by the timeline reports (plans 04 and 05) to map per-commit timestamps to a fixed-resolution per-bucket series independent of terminal width.

func NewBucketing

func NewBucketing(from, to time.Time, n int) Bucketing

NewBucketing constructs a Bucketing covering [from, to] divided into n equal-width slices. n must be > 0; n <= 0 is normalised to 1 so callers can pass user input unchecked. A degenerate window (from == to or to before from) yields Width=0; all commits land in bucket 0 / N-1.

func (Bucketing) Index

func (b Bucketing) Index(t time.Time) int

Index returns the 0..N-1 bucket slot for commit time t. Times before From clamp to 0 (defensive — should not happen given the walk window). Times at or after To clamp to N-1.

func (Bucketing) Start

func (b Bucketing) Start(i int) time.Time

Start returns the wall-clock start time of bucket i. Indexes outside [0, N) are clamped.

type CheckDuplicates

type CheckDuplicates struct {
	// contains filtered or unexported fields
}

CheckDuplicates is used to hold hashes if duplicate detection is enabled it comes with a mutex that should be locked while a check is being performed then added

func (*CheckDuplicates) Add

func (c *CheckDuplicates) Add(key int64, hash []byte)

Add is a non thread safe add a key into the duplicates check need to use mutex inside struct before calling this

func (*CheckDuplicates) Check

func (c *CheckDuplicates) Check(key int64, hash []byte) bool

Check is a non thread safe check to see if the key exists already need to use mutex inside struct before calling this

type CommitInfo

type CommitInfo struct {
	Hash   plumbing.Hash
	Author string
	Email  string
	When   time.Time
}

CommitInfo is the per-commit metadata handed to observers.

type CommitObserver

type CommitObserver interface {
	Observe(c CommitInfo, changes []FileChange)
	Finalise(window HistoryWindow, head HeadSnapshot)
}

CommitObserver is implemented by each report's accumulator. The engine invokes Observe once per commit oldest-first, then Finalise once with the window metadata and a snapshot of the HEAD tree (latest language / complexity per surviving file).

type FileChange

type FileChange struct {
	Path             string
	FromPath         string // != Path on a detected rename; "" on a pure add
	Language         string
	AddedRanges      []LineRange
	RemovedRanges    []LineRange
	LineTypes        []LineType
	RemovedLineTypes []LineType // old-blob line types, for code-filtered removals
	Complexity       []int
	NewBlob          []byte
}

FileChange is one changed file inside a commit. AddedRanges/RemovedRanges describe the diff against the first parent; LineTypes and Complexity are scc's classifier output for the new blob (one LineType per line, one entry in Complexity per line that fired a complexity tick).

type FileJob

type FileJob struct {
	Language             string
	PossibleLanguages    []string // Used to hold potentially more than one language which populates language when determined
	Filename             string
	Extension            string
	Location             string
	Symlocation          string
	Content              []byte `json:"-"`
	Bytes                int64
	Lines                int64
	Code                 int64
	Comment              int64
	Blank                int64
	Complexity           int64
	ComplexityLine       []int64 `json:"-"`
	WeightedComplexity   float64
	Hash                 hash.Hash
	Callback             FileJobCallback `json:"-"`
	Binary               bool
	Minified             bool
	Generated            bool
	EndPoint             int
	Uloc                 int
	LineLength           []int  `json:"-"`
	ClassifyContent      bool   `json:"-"` // When true, CountStats populates ContentByteType
	ContentByteType      []byte `json:"-"` // Per-byte classification, allocated by CountStats when ClassifyContent is true
	TrackComplexityLines bool   `json:"-"` // When true, CountStats populates ComplexityLine
}

FileJob is a struct used to hold all of the results of processing internally before sent to the formatter

func (*FileJob) FilterContentByType

func (fj *FileJob) FilterContentByType(keepTypes ...byte) []byte

FilterContentByType returns a copy of Content with bytes not matching any of the given types replaced by spaces. Newlines are always preserved regardless of type. Returns nil if ContentByteType is nil.

type FileJobCallback

type FileJobCallback interface {
	// ProcessLine should return true to continue processing or false to stop further processing and return
	ProcessLine(job *FileJob, currentLine int64, lineType LineType) bool
}

FileJobCallback is an interface that FileJobs can implement to get a per line callback with the line type

type FileReader

type FileReader struct {
	Buffer *bytes.Buffer
}

FileReader is a struct responsible for reading files into its buffer

func NewFileReader

func NewFileReader() FileReader

NewFileReader creates a new file reader responsible for reading a file

func (*FileReader) ReadFile

func (reader *FileReader) ReadFile(path string, size int) ([]byte, error)

ReadFile actually reads the file into a buffer size controlled by LargeByteCount

type HeadFile

type HeadFile struct {
	Path       string
	Language   string
	Complexity int64
}

HeadFile is one file in the HEAD tree, classified by scc's engine.

type HeadSnapshot

type HeadSnapshot struct {
	Files map[string]HeadFile
}

HeadSnapshot is the set of files in HEAD, keyed by path.

type HistoryWindow

type HistoryWindow struct {
	Depth   int
	Commits int
	From    time.Time
	To      time.Time
	Head    plumbing.Hash
}

HistoryWindow describes the commit window the engine walked.

type Json2

type Json2 struct {
	LanguageSummary         []LanguageSummary `json:"languageSummary"`
	EstimatedCost           float64           `json:"estimatedCost"`
	EstimatedScheduleMonths float64           `json:"estimatedScheduleMonths"`
	EstimatedPeople         float64           `json:"estimatedPeople"`

	// LOCOMO fields (only populated when --locomo or --cost-comparison is enabled)
	EstimatedLLMCost                  *float64 `json:"estimatedLLMCost,omitempty"`
	EstimatedLLMInputTokens           *float64 `json:"estimatedLLMInputTokens,omitempty"`
	EstimatedLLMOutputTokens          *float64 `json:"estimatedLLMOutputTokens,omitempty"`
	EstimatedLLMGenerationSeconds     *float64 `json:"estimatedLLMGenerationSeconds,omitempty"`
	EstimatedLLMReviewHours           *float64 `json:"estimatedLLMReviewHours,omitempty"`
	EstimatedLLMPreset                *string  `json:"estimatedLLMPreset,omitempty"`
	EstimatedLLMAverageComplexityMult *float64 `json:"estimatedLLMAverageComplexityMultiplier,omitempty"`
	EstimatedLLMCycles                *float64 `json:"estimatedLLMCycles,omitempty"`
}

type Language

type Language struct {
	LineComment                     []string   `json:"line_comment"`
	ComplexityChecks                []string   `json:"complexitychecks"`
	ComplexityChecksPostfix         []string   `json:"complexitychecks_postfix"`
	ComplexityChecksPostfixExcludes []string   `json:"complexitychecks_postfix_excludes"`
	Extensions                      []string   `json:"extensions"`
	MultiLine                       [][]string `json:"multi_line"`
	Quotes                          []Quote    `json:"quotes"`
	Keywords                        []string   `json:"keywords"`
	FileNames                       []string   `json:"filenames"`
	SheBangs                        []string   `json:"shebangs"`
	ExtensionFile                   bool       `json:"extensionFile"`
	NestedMultiLine                 bool       `json:"nestedmultiline"`
}

Language is a struct which contains the values for each language stored in languages.json

type LanguageFeature

type LanguageFeature struct {
	Complexity            *Trie
	MultiLineComments     *Trie
	MultiLine             [][]string // in case someone needs the actual value
	SingleLineComments    *Trie
	LineComment           []string // in case someone needs the actual value
	Strings               *Trie
	Tokens                *Trie
	Nested                bool
	PostfixExcludes       [][]byte
	ComplexityCheckMask   byte
	SingleLineCommentMask byte
	MultiLineCommentMask  byte
	StringCheckMask       byte
	ProcessMask           byte
	Keywords              []string
	KeywordBytes          [][]byte
	Quotes                []Quote
}

LanguageFeature is a struct which represents the conversion from Language into what is used for matching

type LanguageSummary

type LanguageSummary struct {
	Name               string
	Bytes              int64
	CodeBytes          int64
	Lines              int64
	Code               int64
	Comment            int64
	Blank              int64
	Complexity         int64
	Count              int64
	WeightedComplexity float64
	Files              []*FileJob
	LineLength         []int
	ULOC               int
}

LanguageSummary is used to hold summarized results for a single language

func ProcessResult

func ProcessResult() ([]LanguageSummary, error)

ProcessResult runs the same pipeline as Process but returns structured results instead of formatting to stdout. Useful for programmatic consumers like MCP servers.

type LineRange

type LineRange struct {
	Start int
	Count int
}

LineRange is a half-open line span [Start, Start+Count) in 1-based line numbers. A FileChange carries one entry per contiguous run of added (or removed) lines emitted by go-git's diff.

type LineType

type LineType int32

LineType what type of line are processing

const (
	LINE_BLANK LineType = iota
	LINE_CODE
	LINE_COMMENT
)

These are not meant to be CAMEL_CASE but as it us used by an external project we cannot change it

type LocomoPreset

type LocomoPreset struct {
	Name        string
	InputPrice  float64 // cost per 1M input tokens
	OutputPrice float64 // cost per 1M output tokens
	TPS         float64 // output tokens per second
}

LocomoPreset defines the pricing and throughput for an LLM tier. Presets are tier-based (large/medium/small/local) rather than model-specific, so they don't go stale as specific models are retired or renamed.

func GetLocomoPreset

func GetLocomoPreset(name string) LocomoPreset

GetLocomoPreset returns the preset for the given name, falling back to medium

type LocomoResult

type LocomoResult struct {
	InputTokens           float64
	OutputTokens          float64
	Cost                  float64
	GenerationSeconds     float64
	ReviewHours           float64
	AverageComplexityMult float64
	IterationFactor       float64
	Preset                string
}

LocomoResult holds the computed estimates from the LOCOMO model

func LocomoEstimate

func LocomoEstimate(sumCode, sumComplexity int64) LocomoResult

LocomoEstimate computes the full LOCOMO estimate for a project

type MailmapObserver

type MailmapObserver interface {
	SetMailmap(*mailmap)
}

MailmapObserver is an optional extension to CommitObserver. The engine always parses the repo's .mailmap from HEAD — one small blob — and hands it to observers that implement this, before the walk. Unlike BaselineObserver it does NOT trigger the expensive start-tree classification, so observers that only need author folding (e.g. Hotspots, the author timeline) can implement it cheaply.

type OpenClose

type OpenClose struct {
	Open  []byte
	Close []byte
}

OpenClose is used to hold an open/close pair for matching such as multi line comments

type Quote

type Quote struct {
	Start        string `json:"start"`
	End          string `json:"end"`
	IgnoreEscape bool   `json:"ignoreEscape"` // To enable turning off the \ check for C# @"\" string examples https://github.com/secusolutions/scc/issues/71
	DocString    bool   `json:"docString"`    // To enable docstring check for Python where "If the triple quote string starts following a newline with only white-space characters in front and ends followed by only a newline or white-space characters it is a comment" https://github.com/secusolutions/scc/issues/62
}

Quote is a struct which holds rules and start/end values for string quotes

type Trie

type Trie struct {
	Type  int
	Close []byte
	Table [256]*Trie
}

Trie is a structure used to store matches efficiently

func (*Trie) Insert

func (root *Trie) Insert(tokenType int, token []byte)

Insert inserts a string into the trie for matching

func (*Trie) InsertClose

func (root *Trie) InsertClose(tokenType int, openToken, closeToken []byte)

InsertClose closes off a string in the trie

func (*Trie) Match

func (root *Trie) Match(token []byte) (int, int, []byte)

Match checks the created trie structure for a match