ftdetect

README

ftdetect

ftdetect is a library for detecting the filetype of source code files (what programming language the file is written in). It primarily uses the file extension and file name to determine the filetype, but also may use the first line of the file (the header) for additional information (for example, #!/bin/bash on the first line is a good indication of a shell file, even if there is no extension).

The library is optimized for very fast detection in the common case and supports saving the detection data structure to a file for very fast loading/startup time.

A default set of detectors for 135 different languages is provided. If you would like to do the embedding yourself you can specify the ftdetect_custom build tag to avoid embedding the default detectors.

The ./cmd/detect directory contains an example tool which uses the default detectors to perform file detection on the first argument to the CLI application.

Documentation

Index

• func MustRegex(s string) *regex
• func Regex(s string) (*regex, error)
• type Detector
  • func LoadDetectorJson(data []byte) (*Detector, error)
• type Detectors
  • func LoadDefaultDetectors() Detectors
  • func LoadDetectors(b []byte) (Detectors, error)
  • func (ds Detectors) Detect(filename string, header []byte) *Detector
  • func (ds Detectors) RegisterDetector(d *Detector)
  • func (ds Detectors) Serialize() ([]byte, error)
  • func (ds Detectors) SetPriority(lang string, priority int)

Functions

func MustRegex

func MustRegex(s string) *regex

MustRegex is equivalent to Regex but panics if the regular expression is invalid.

func Regex

func Regex(s string) (*regex, error)

Regex compiles a serializable regular expression.

Types

type Detector

type Detector struct {
	Exts     []string
	Files    []string
	File     *regex
	Header   *regex
	Priority int // 0 is lowest priority
	Name     string
}

A Detector defines a language and how it should be detected via extensions, special files, a file regex, and a header regex. A language does not need to provide every detection mechanism. In fact, most languages only need to provide an extension, which makes detection very efficient.

func LoadDetectorJson

func LoadDetectorJson(data []byte) (*Detector, error)

LoadDetectorJson loads a detector from a json spec.

type Detectors

type Detectors map[string][]*Detector

Detectors is a set of languages that are supported. It is a map from extension/filename to the list of detectors that are registered for those values.

func LoadDefaultDetectors

func LoadDefaultDetectors() Detectors

LoadDefaultDetectors returns a set of detectors for many programming languages.

func LoadDetectors

func LoadDetectors(b []byte) (Detectors, error)

LoadDetectors loads a set of languages from a serialized Detectors byte slice.

func (Detectors) Detect

func (ds Detectors) Detect(filename string, header []byte) *Detector

Detect returns the language that was detected from the filename and file header (first line of file), or nil if no matching language was found.

func (Detectors) RegisterDetector

func (ds Detectors) RegisterDetector(d *Detector)

RegisterDetector adds a new detector to the set.

func (Detectors) Serialize

func (ds Detectors) Serialize() ([]byte, error)

Serialize writes the detector set to a byte slice so it can be saved.

func (Detectors) SetPriority

func (ds Detectors) SetPriority(lang string, priority int)

SetPriority sets the priority for a certain language. A higher priority will make this language match above others when there are match conflicts.