README
¶
License Classifier
[TOC]
Introduction
The license classifier is a library and set of tools that can analyze text to
determine what type of license it contains. It searches for license texts in a
file and compares it to an archive of known licenses. These files could be,
e.g., LICENSE files with a single or multiple licenses in it, or source code
files with the license text in a comment.
A "confidence level" is associated with each result indicating how close the
match was. A confidence level of 1.0 indicates an exact match, while a
confidence level of 0.0 indicates that no license was able to match the text.
Adding a new license
Adding a new license is straight-forward:
-
Create a file in
licenses/.- The filename should be the name of the license or its abbreviation. If the license is an Open Source license, use the appropriate identifier specified at https://spdx.org/licenses/.
- If the license is the "header" version of the license, append the suffix
"
.header" to it. Seelicenses/README.mdfor more details.
-
Add the license name to the list in
license_type.go. -
Regenerate the
licenses.dbfile by running the license serializer:license_serializer -output licenseclassifier/licenses -
Create and run appropriate tests to verify that the license is indeed present.
Tools
Identify license
identify_license is a command line tool that can identify the license(s)
within a file.
$ identify_license LICENSE
LICENSE: GPL-2.0 (confidence: 1, offset: 0, extent: 14794)
LICENSE: LGPL-2.1 (confidence: 1, offset: 18366, extent: 23829)
LICENSE: MIT (confidence: 1, offset: 17255, extent: 1059)
License serializer
The license_serializer tool regenerates the licenses.db archive. The archive
contains preprocessed license texts for quicker comparisons against unknown
texts.
license_serializer -output licenseclassifier/licenses
Directories
¶
| Path | Synopsis |
|---|---|
|
Package licenseclassifier provides methods to identify the open source license that most closely matches an unknown license.
|
Package licenseclassifier provides methods to identify the open source license that most closely matches an unknown license. |
|
internal/commentparser
Package commentparser does a basic parse over a source file and returns all of the comments from the code.
|
Package commentparser does a basic parse over a source file and returns all of the comments from the code. |
|
internal/commentparser/language
Package language contains methods and information about the different programming languages the comment parser supports.
|
Package language contains methods and information about the different programming languages the comment parser supports. |
|
internal/sets
Package sets provides sets for storing collections of unique elements.
|
Package sets provides sets for storing collections of unique elements. |
|
serializer
Package serializer normalizes the license text and calculates the hash values for all substrings in the license.
|
Package serializer normalizes the license text and calculates the hash values for all substrings in the license. |
|
tools
command
The identify_license program tries to identify the license type of an unknown license.
|
The identify_license program tries to identify the license type of an unknown license. |
|
Package stringclassifier finds the nearest match between a string and a set of known values.
|
Package stringclassifier finds the nearest match between a string and a set of known values. |
|
internal/pq
Package pq provides a priority queue.
|
Package pq provides a priority queue. |
|
searchset
Package searchset generates hashes for all substrings of a text.
|
Package searchset generates hashes for all substrings of a text. |
|
searchset/tokenizer
Package tokenizer converts a text into a stream of tokens.
|
Package tokenizer converts a text into a stream of tokens. |