dialect

README

Dialect

Dialect is a framework for processing source code to determine basis linguistic details such as number of lines of comments, etc.

Dialect implements a language specific set of rules to determine these details.

If Dialect cannot determine the language or doesn't have a specific implementation, it will simply return the number of lines only.

Installation

go get github.com/pinpt/dialect

Usage

The most simple usage is to invoke Examine passing it the language, filename and a io.Reader to the source code.

package main

import (
	"fmt"
	"github.com/pinpt/dialect"
	_ "github.com/pinpt/dialect/pkg/languages"
	"strings"
)

func main() {
	reader := strings.NewReader("var a = 1")
	config := dialect.CreateDefaultConfiguration()
	result, err := dialect.Examine("JavaScript", "test.js", reader, config)
	if err != nil {
		panic(err)
	}
	fmt.Println(result)
}

The result is a struct DialectResult which provides the details of what the examiner found.

type DialectResult struct {
	Blanks     int
	Comments   int
	Loc        int
	Sloc       int
	IsTest     bool
	Copyrights []*copyright.CopyrightResult
}

The fields are the following:

• Blanks the number of blank lines detected
• Comments the number of comment lines detected
• Loc the total number of lines detected (Blanks + Comments + Sloc = Loc)
• Sloc the number of source lines detected (Sloc = Loc - Comments - Blanks)
• IsTest a boolean to indicate if the examiner detected the file to be a test file
• Copyrights an array of copyrights found in the file (only available if DetectCopyrights is set in config)

Detecting line by line

Instead of using a Reader, you can feed data line by line.

config := dialect.CreateDefaultConfiguration()
ex, err := dialect.CreateLineByLineExaminer("JavaScript", "foo.js", config)
if err != nil {
	panic(err)
}
// pass true as 2nd argument to indicate the EOF
_, err = ex.ProcessLine("var a = 1", true)
if err != nil {
	panic(err)
}
// results will be in ex.Results

Detecting Copyrights

You can detect Copyrights in source code by setting the configuration field DetectCopyrights to true.

config := dialect.CreateDefaultConfiguration()
config.DetectCopyrights = true

If any copyrights are detected in the source code, the Copyrights field will be an array of *CopyrightResult. If none are found, the Copyrights will be nil.

Implementing a missing language

To implement your own language or override an existing language, you can use the function RegisterExaminer and implement the interface DialectExaminer. See any of the existing implementations for a good example.

If you do implement a new language or fix an existing one, please consider sending us a Pull Request so that we can merge it!

License

Licensed under the MIT License. Copyright (c) 2016 A Pinpoint PBC

Documentation

Index

• Constants
• Variables
• func RegisterExaminer(language string, examiner DialectExaminer)
• func RegisterFrameworkProcessor(name string, processor DialectFrameworkProcessor)
• type DialectConfiguration
  • func CreateConfigurationWithCallback(callback DialectResultCallback) *DialectConfiguration
  • func CreateDefaultConfiguration() *DialectConfiguration
• type DialectContext
  • func CreateLineByLineExaminer(language string, filename string, config *DialectConfiguration) (*DialectContext, error)
  • func (ctx *DialectContext) ProcessLine(buf []byte, eof bool) (*DialectLine, error)
• type DialectExaminer
• type DialectFramework
  • func DetectFrameworks(directory string) ([]*DialectFramework, error)
• type DialectFrameworkProcessor
• type DialectFrameworkType
• type DialectLine
• type DialectResult
  • func Examine(language string, filename string, reader io.Reader, ...) (*DialectResult, error)
  • func (r *DialectResult) String() string
• type DialectResultCallback

Constants

const EOL = byte('\n')

Variables

var (
	Build   string
	Version string
)

Functions

func RegisterExaminer

func RegisterExaminer(language string, examiner DialectExaminer)

RegisterExaminer is used to register an implementation of the DialectExaminer interface

func RegisterFrameworkProcessor

func RegisterFrameworkProcessor(name string, processor DialectFrameworkProcessor)

RegisterFrameworkProcessor is used to register an implementation of the DialectFrameworkProcessor interface

Types

type DialectConfiguration

type DialectConfiguration struct {
	Callback         DialectResultCallback
	DetectCopyrights bool
}

DialectConfiguration is a struct which is used for specifying configuration

func CreateConfigurationWithCallback

func CreateConfigurationWithCallback(callback DialectResultCallback) *DialectConfiguration

CreateConfigurationWithCallback returns a default configuration with a callback

func CreateDefaultConfiguration

func CreateDefaultConfiguration() *DialectConfiguration

CreateDefaultConfiguration will return a default configuration

type DialectContext

type DialectContext struct {
	Language   string
	Filename   string
	Config     *DialectConfiguration
	Examiner   DialectExaminer
	LineNumber int
	Result     *DialectResult
	Buffer     bytes.Buffer
}

func CreateLineByLineExaminer

func CreateLineByLineExaminer(language string, filename string, config *DialectConfiguration) (*DialectContext, error)

CreateLineByLineExaminer returns an interface which can be called with each line using the ProcessLine function

func (*DialectContext) ProcessLine

func (ctx *DialectContext) ProcessLine(buf []byte, eof bool) (*DialectLine, error)

type DialectExaminer

type DialectExaminer interface {
	NewExaminer() DialectExaminer
	Examine(language string, filename string, line *DialectLine) error
}

DialectExaminer is the interface that language processors must implement to handle details about a specific language

type DialectFramework

type DialectFramework struct {
	Name string               `json:"name"`
	Type DialectFrameworkType `json:"type"`
}

DialectFramework is details about a framework that the project supports

func DetectFrameworks

func DetectFrameworks(directory string) ([]*DialectFramework, error)

Detect will attempt to detect all the frameworks for a given project source directory

type DialectFrameworkProcessor

type DialectFrameworkProcessor interface {
	Detect(directory string) ([]*DialectFramework, error)
}

DialectFramework is the interface that framework processors must implement to handle details about a specific framework

type DialectFrameworkType

type DialectFrameworkType string

const (
	DialectFrameworkBuild     DialectFrameworkType = "build"
	DialectFrameworkLanguage  DialectFrameworkType = "language"
	DialectFrameworkContainer DialectFrameworkType = "container"
)

type DialectLine

type DialectLine struct {
	LineNumber int
	IsComment  bool
	IsCode     bool
	IsBlank    bool
	IsTest     bool
	Contents   string
	EOF        bool
	Buffer     string
	Config     *DialectConfiguration
}

DialectLine is internally used by the dialect framework to communicate results per line by a DialectExaminer implementation

type DialectResult

type DialectResult struct {
	Blanks     int
	Comments   int
	Loc        int
	Sloc       int
	IsTest     bool
	Copyrights []*copyright.CopyrightResult
}

DialectResult is returned from Examine to describe the code

func Examine

func Examine(language string, filename string, reader io.Reader, config *DialectConfiguration) (*DialectResult, error)

Examine is used to detect information about the source code

func (*DialectResult) String

func (r *DialectResult) String() string

type DialectResultCallback

type DialectResultCallback func(language string, line *DialectLine) error

DialectResultCallback is a callback function for receiving per line results as the code is being examined