lookout

package module
v0.11.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 19, 2019 License: AGPL-3.0 Imports: 9 Imported by: 20

README

Service for assisted code review, that allows running custom code Analyzers on pull requests.

GitHub version Build Status Development Code Coverage Go Report Card GoDoc

Website • Documentation • Blog • Slack • Twitter

Introduction

With source{d} Lookout, we’re introducing a service for assisted code review, that allows running custom code analyzers on pull requests.

Jump to the Quickstart section to start using it!

Table of Contents

Motivation and Scope

source{d} is the company driving the Machine Learning on Code (#MLonCode) movement. Doing Machine Learning on Code consists of applying ML techniques to train models that can cluster, identify and predict useful aspects of source code and software repositories.

source{d} Lookout is the first step towards a full suite of Machine Learning on Code applications for AI-assisted coding, but you can also create your own analyzers without an ML approach.

The benefits of using source{d} Lookout are:

  • Keep your code base style/patterns consistent.
  • Language agnostic assisted code reviews.
  • Identify where to focus your attention on code reviews.
  • Automatically warn about common mistakes before human code review.

Current Status

Currently, source{d} Lookout is in development process.

Further Reading

This repository contains the code of source{d} Lookout and the project documentation, which you can also see properly rendered at https://docs.sourced.tech/lookout.

Quickstart

There are several ways to run source{d} Lookout; we recommend to use docker-compose because it's straightforward, but you can learn more about the different ways to run source{d} Lookout.

Please refer to the Configuring source{d} Lookout guide for documentation about the config.yml file, and to know how to configure source{d} Lookout to analyze your repositories, or to use your own analyzers.

There is docker-compose.yml config file for Docker Compose to start source{d} Lookout, its dependencies (bblfsh and PostgreSQL) and a dummy analyzer which will add some stats to the watched pull requests.

To do so, clone this repository or download docker-compose.yml directly.

Create the config.yml file in the same directory where docker-compose.yml is. You can use config.yml.tpl as a template. Make sure that you specify in the config.yml the repositories that will be watched by source{d} Lookout. Then run, passing a valid GitHub user/token:

$ docker-compose pull
$ GITHUB_USER=<user> GITHUB_TOKEN=<token> docker-compose up --force-recreate

Once it is running, source{d} Lookout will start posting the comments returned by dummy analyzer into the pull requests opened at GitHub in the repositories that you configured to be watched.

You can stop it by pressing ctrl+c

If you want to try source{d} Lookout with your own analyzer instead of dummy one, you must run it in advance, then set it into config.yml and then run:

$ docker-compose pull
$ GITHUB_USER=<user> GITHUB_TOKEN=<token> docker-compose up --force-recreate lookout bblfsh postgres

If you need to reset the database to a clean state, you should drop the postgres container. To do so, stop running source{d} Lookout with ctrl+c and then execute:

$ docker rm lookout_postgres_1

Available Analyzers

This is the list of the known implemented analyzers for source{d} Lookout:

Name Description Targeted files Maturity level
style-analyzer Code style analyzer development
terraform Checks if Terraform files are correctly formatted Terraform usable
gometalint Reports gometalinter results on pull requests Go testing and demo
sonarcheck Reports SonarSource checks results on pull requests using bblfsh UAST Java testing and demo
flake8 Reports flake8 results on pull requests Python testing and demo
npm-audit Reports issues with newly added dependencies using npm-audit JavaScript development
function-name analyzer Applies a translation model from function identifiers to function names. development

Create an Analyzer

If you are developing an Analyzer, or you want more info about how they work, please check the documentation about source{d} Lookout analyzers.

Contribute

Contributions are more than welcome, if you are interested please take a look at our Contributing Guidelines.

Community

source{d} has an amazing community of developers and contributors who are interested in Code As Data and/or Machine Learning on Code. Please join us! 👋

Code of Conduct

All activities under source{d} projects are governed by the source{d} code of conduct.

License

Affero GPL v3.0 or later, see LICENSE.

Documentation

Overview

Package lookout provides gRPC APIs to implement analysis servers and clients.

Services

There are two services:

  1. Analyzer: analyzers process source code changes and provide analysis results as text comments, possibly linked to specific parts of the code.

2. Data: provides simple access to source code changes.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// NoErrStopWatcher if a new error of this kind is returned by EventHandler
	// the Watcher.Watch function exits without error.
	NoErrStopWatcher = errors.NewKind("Stop watcher")
)

Functions

func RegisterAnalyzerServer 

func RegisterAnalyzerServer(s *grpc.Server, srv AnalyzerServer)

func RegisterDataServer 

func RegisterDataServer(s *grpc.Server, srv *DataServerHandler)

Types

type AnalysisStatus added in v0.0.3 

type AnalysisStatus int

AnalysisStatus is the status reported to the provider to inform that we are performing an analysis, or that it has finished 

const (

	// ErrorAnalysisStatus represents an error status
	ErrorAnalysisStatus AnalysisStatus
	// FailureAnalysisStatus represents a failure status
	FailureAnalysisStatus
	// PendingAnalysisStatus represents a pending status
	PendingAnalysisStatus
	// SuccessAnalysisStatus represents a success status
	SuccessAnalysisStatus
)

func (AnalysisStatus) String added in v0.0.3 

func (st AnalysisStatus) String() string

type Analyzer 

type Analyzer struct {
	Client AnalyzerClient
	Config AnalyzerConfig
}

Analyzer is a struct of analyzer client and config

type AnalyzerClient 

type AnalyzerClient = pb.AnalyzerClient

func NewAnalyzerClient 

func NewAnalyzerClient(conn *grpc.ClientConn) AnalyzerClient

type AnalyzerComments added in v0.0.4 

type AnalyzerComments struct {
	Config   AnalyzerConfig
	Comments []*Comment
}

AnalyzerComments contains a group of comments and the config for the analyzer that created them

type AnalyzerCommentsGroups added in v0.6.0 

type AnalyzerCommentsGroups []AnalyzerComments

AnalyzerCommentsGroups list of AnalyzerComments

func (AnalyzerCommentsGroups) Count added in v0.8.0 

func (g AnalyzerCommentsGroups) Count() int

Count returns the total number of comments

func (AnalyzerCommentsGroups) Dedup added in v0.8.0 

func (g AnalyzerCommentsGroups) Dedup() AnalyzerCommentsGroups

Dedup filters duplicated comments

func (AnalyzerCommentsGroups) Filter added in v0.6.0 

func (g AnalyzerCommentsGroups) Filter(fn CommentsFilterFn) ([]AnalyzerComments, error)

Filter filters comments groups using CommentsFilterFn

type AnalyzerConfig 

type AnalyzerConfig struct {
	Name string
	// Addr is gRPC URL.
	// can be defined only in global config, repository-scoped configuration is ignored
	Addr string
	// Disabled repository-scoped configuration can accept only true value, false value is ignored
	Disabled bool
	// Feedback is a url to be linked after each comment
	Feedback string
	// Settings any configuration for an analyzer
	Settings map[string]interface{}
}

AnalyzerConfig is a configuration of analyzer

type AnalyzerServer 

type AnalyzerServer = pb.AnalyzerServer

type Change 

type Change = pb.Change

type ChangeGetter 

type ChangeGetter interface {
	// GetChanges returns a ChangeScanner that scans all changes according
	// to the request.
	GetChanges(context.Context, *ChangesRequest) (ChangeScanner, error)
}

ChangeGetter is used to retrieve code changes.

type ChangeScanner 

type ChangeScanner interface {
	// Next advances the scanner to the next change. It returns true if a new
	// change is found, and false otherwise. After the user is done scanning,
	// Err must be called to check if all changes were consumed or there was an
	// error.
	Next() bool
	// Err returns any error found during scanning.
	Err() error
	// Change returns the current change.
	Change() *Change
	// Close closes the scanner.
	Close() error
}

ChangeScanner is a scanner for changes.

type ChangesRequest 

type ChangesRequest = pb.ChangesRequest

type ClientChangeScanner 

type ClientChangeScanner struct {
	// contains filtered or unexported fields
}

func (*ClientChangeScanner) Change 

func (s *ClientChangeScanner) Change() *Change

func (*ClientChangeScanner) Close 

func (s *ClientChangeScanner) Close() error

func (*ClientChangeScanner) Err 

func (s *ClientChangeScanner) Err() error

func (*ClientChangeScanner) Next 

func (s *ClientChangeScanner) Next() bool

type ClientFileScanner 

type ClientFileScanner struct {
	// contains filtered or unexported fields
}

func (*ClientFileScanner) Close 

func (s *ClientFileScanner) Close() error

func (*ClientFileScanner) Err 

func (s *ClientFileScanner) Err() error

func (*ClientFileScanner) File 

func (s *ClientFileScanner) File() *File

func (*ClientFileScanner) Next 

func (s *ClientFileScanner) Next() bool

type Comment 

type Comment = pb.Comment

type CommentsFilterFn added in v0.6.0 

type CommentsFilterFn func(*Comment) (skip bool, err error)

CommentsFilterFn is a function that filters comments

type CommitRevision 

type CommitRevision = pb.CommitRevision

CommitRevision defines a range of commits, from a base to a head

type DataClient 

type DataClient struct {
	// contains filtered or unexported fields
}

func NewDataClient 

func NewDataClient(cc *grpc.ClientConn) *DataClient

func (*DataClient) GetChanges 

func (c *DataClient) GetChanges(ctx context.Context, in *ChangesRequest, opts ...grpc.CallOption) (
	ChangeScanner, error)

func (*DataClient) GetFiles 

func (c *DataClient) GetFiles(ctx context.Context, in *FilesRequest, opts ...grpc.CallOption) (
	FileScanner, error)

type DataServerHandler 

type DataServerHandler struct {
	ChangeGetter ChangeGetter
	FileGetter   FileGetter
}

func (*DataServerHandler) GetChanges 

func (s *DataServerHandler) GetChanges(req *ChangesRequest,
	srv pb.Data_GetChangesServer) (err error)

func (*DataServerHandler) GetFiles 

func (s *DataServerHandler) GetFiles(req *FilesRequest, srv pb.Data_GetFilesServer) (err error)

type Event 

type Event interface {
	// ID returns the EventID.
	ID() EventID
	// Type returns the EventType, in order to identify the concreate type of
	// the event.
	Type() EventType
	// Revision returns a commit revision, containing the head and the base of
	// the changes.
	Revision() *CommitRevision
	// Validate returns an error if the event is malformed
	Validate() error
	// GetProvider returns the name of the provider that created this event
	GetProvider() string
	// GetOrganizationID returns the organization to which this event's repository
	// belongs to
	GetOrganizationID() string
}

Event represents a repository event.

type EventHandler 

type EventHandler func(context.Context, Event) error

EventHandler is the function to be called when a new event happens.

func CachedHandler added in v0.3.0 

func CachedHandler(fn EventHandler) EventHandler

CachedHandler wraps an EventHandler, keeping a cache to skip successfully processed Events

type EventID 

type EventID = pb.EventID

EventID is a unique hash id for an event

type EventResponse 

type EventResponse = pb.EventResponse

type EventType 

type EventType = pb.EventType

EventType defines the supported event types

type File 

type File = pb.File

type FileGetter 

type FileGetter interface {
	// GetFiles returns a FilesScanner that scans all files according
	// to the request.
	GetFiles(context.Context, *FilesRequest) (FileScanner, error)
}

FileGetter is used to retrieve all code for a revision.

type FileScanner 

type FileScanner interface {
	// Next advances the scanner to the next file. It returns true if a new
	// file is found, and false otherwise. After the user is done scanning,
	// Err must be called to check if all files were consumed or there was an
	// error.
	Next() bool
	// Err returns any error found during scanning.
	Err() error
	// File returns the current file.
	File() *File
	// Close closes the scanner.
	Close() error
}

FileScanner is a scanner for files.

type FilesRequest 

type FilesRequest = pb.FilesRequest

type FnChangeScanner 

type FnChangeScanner struct {
	Scanner ChangeScanner
	Fn      func(*Change) (bool, error)
	OnStart func() error
	// contains filtered or unexported fields
}

FnChangeScanner implements ChangeScanner using functions

func (*FnChangeScanner) Change 

func (s *FnChangeScanner) Change() *Change

func (*FnChangeScanner) Close 

func (s *FnChangeScanner) Close() error

func (*FnChangeScanner) Err 

func (s *FnChangeScanner) Err() error

func (*FnChangeScanner) Next 

func (s *FnChangeScanner) Next() bool

type FnFileScanner 

type FnFileScanner struct {
	Scanner FileScanner
	Fn      func(*File) (bool, error)
	OnStart func() error
	// contains filtered or unexported fields
}

FnFileScanner implements FileScanner using functions

func (*FnFileScanner) Close 

func (s *FnFileScanner) Close() error

func (*FnFileScanner) Err 

func (s *FnFileScanner) Err() error

func (*FnFileScanner) File 

func (s *FnFileScanner) File() *File

func (*FnFileScanner) Next 

func (s *FnFileScanner) Next() bool

type Poster 

type Poster interface {
	// Post posts comments about an event.
	// poster should make sure comments weren't posted before if safe is true
	Post(ctx context.Context, e Event, cs []AnalyzerComments, safe bool) error

	// Status sends the current analysis status to the provider
	Status(context.Context, Event, AnalysisStatus) error
}

Poster can post comments about an event.

type PushEvent 

type PushEvent struct {
	pb.PushEvent
	// OrganizationID is the organization to which this event's repository
	// belongs to
	OrganizationID string
}

PushEvent represents a Push to a git repository. It wraps the pb.PushEvent adding information only relevant to lookout, and not for the analyzers.

func (*PushEvent) GetOrganizationID added in v0.8.0 

func (e *PushEvent) GetOrganizationID() string

GetOrganizationID returns the organization to which this event's repository belongs to

func (*PushEvent) GetProvider added in v0.8.0 

func (e *PushEvent) GetProvider() string

GetProvider returns the name of the provider that created this event

type ReferencePointer 

type ReferencePointer = pb.ReferencePointer

ReferencePointer is a pointer to a git refererence in a repository

type RepositoryInfo 

type RepositoryInfo = pb.RepositoryInfo

RepositoryInfo contains information about a repository

type ReviewEvent 

type ReviewEvent struct {
	pb.ReviewEvent
	// OrganizationID is the organization to which this event's repository
	// belongs to
	OrganizationID string
}

ReviewEvent represents a Review (pull request in case of GitHub) being created or updated. It wraps the pb.PushEvent adding information only relevant to lookout, and not for the analyzers.

func (*ReviewEvent) GetOrganizationID added in v0.8.0 

func (e *ReviewEvent) GetOrganizationID() string

GetOrganizationID returns the organization to which this event's repository belongs to

func (*ReviewEvent) GetProvider added in v0.8.0 

func (e *ReviewEvent) GetProvider() string

GetProvider returns the name of the provider that created this event

type Watcher 

type Watcher interface {
	// Watch for new events triggering the EventHandler for each new issue,
	// it stops until an error is returned by the EventHandler. Network errors
	// or other temporal errors are handled as non-fatal errors, just logging it.
	Watch(context.Context, EventHandler) error
}

Watcher watch for new events in given provider.

Source Files

View all Source files

Directories

Path Synopsis
cmd
dummy
lookout-sdk
lookoutd
Package dummy implements an example analyzer.
 Package dummy implements an example analyzer.
provider
github
json
service
bblfsh
Package bblfsh provides access to bblfsh parsing for the data service.
 Package bblfsh provides access to bblfsh parsing for the data service.
enry
git
Package git provides access to git repositories.
 Package git provides access to git repositories.
purge
models
Code generated by https://github.com/src-d/go-kallax.
 Code generated by https://github.com/src-d/go-kallax.
util
cache
cli
cmdtest
ctxlog
grpchelper

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.