Package runner implements a go/analysis runner. It makes heavy use of on-disk caching to reduce overall memory usage and to speed up repeat runs.
Public API ¶
A Runner maps a list of analyzers and package patterns to a list of results. Results provide access to diagnostics, directives, errors encountered, and information about packages. Results explicitly do not contain ASTs or type information. All position information is returned in the form of token.Position, not token.Pos. All work that requires access to the loaded representation of a package has to occur inside analyzers.
Planning and execution ¶
Analyzing packages is split into two phases: planning and execution.
During planning, a directed acyclic graph of package dependencies is computed. We materialize the full graph so that we can execute the graph from the bottom up, without keeping unnecessary data in memory during a DFS and with simplified parallel execution.
During execution, leaf nodes (nodes with no outstanding dependencies) get executed in parallel, bounded by a semaphore sized according to the number of CPUs. Conceptually, this happens in a loop, processing new leaf nodes as they appear, until no more nodes are left. In the actual implementation, nodes know their dependents, and the last dependency of a node to be processed is responsible for scheduling its dependent.
The graph is rooted at a synthetic root node. Upon execution of the root node, the algorithm terminates.
Analyzing a package repeats the same planning + execution steps, but this time on a graph of analyzers for the package. Parallel execution of individual analyzers is bounded by the same semaphore as executing packages.
Actions are executed in parallel where the dependency graph allows. Overall parallelism is bounded by a semaphore, sized according to GOMAXPROCS. Each concurrently processed package takes up a token, as does each analyzer – but a package can always execute at least one analyzer, using the package's token.
Depending on the overall shape of the graph, there may be GOMAXPROCS packages running a single analyzer each, a single package running GOMAXPROCS analyzers, or anything in between.
Total memory consumption grows roughly linearly with the number of CPUs, while total execution time is inversely proportional to the number of CPUs. Overall, parallelism is affected by the shape of the dependency graph. A lot of inter-connected packages will see less parallelism than a lot of independent packages.
The runner caches facts, directives and diagnostics in a content-addressable cache that is designed after Go's own cache. Additionally, it makes use of Go's export data.
This cache not only speeds up repeat runs, it also reduces peak memory usage. When we've analyzed a package, we cache the results and drop them from memory. When a dependent needs any of this information, or when analysis is complete and we wish to render the results, the data gets loaded from disk again.
Data only exists in memory when it is immediately needed, not retained for possible future uses. This trades increased CPU usage for reduced memory usage. A single dependency may be loaded many times over, but it greatly reduces peak memory usage, as an arbitrary amount of time may pass between analyzing a dependency and its dependent, during which other packages will be processed.
- type Diagnostic
- type RelatedInformation
- type Result
- type ResultData
- type Runner
- type SerializedDirective
- type Stats
- type SuggestedFix
- type TextEdit
This section is empty.
This section is empty.
type Diagnostic ¶
type RelatedInformation ¶
RelatedInformation provides additional context for a diagnostic.
A Result describes the result of analyzing a single package.
It holds references to cached diagnostics and directives. They can be loaded on demand with Diagnostics and Directives respectively.
type ResultData ¶
A Runner executes analyzers on packages.
func (*Runner) ActiveWorkers ¶
ActiveWorkers returns the number of currently running workers.
Run loads the packages specified by patterns, runs analyzers on them and returns the results. Each result corresponds to a single package. Results will be returned for all packages, including dependencies. Errors specific to packages will be reported in the respective results.
If cfg is nil, a default config will be used. Otherwise, cfg will be used, with the exception of the Mode field.