nilaway

package module
v0.0.0-...-28b542b Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 19, 2024 License: Apache-2.0 Imports: 4 Imported by: 1

README

NilAway

GoDoc Build Status Coverage Status

[!WARNING]
NilAway is currently under active development: false positives and breaking changes can happen. We highly appreciate any feedback and contributions!

NilAway is a static analysis tool that seeks to help developers avoid nil panics in production by catching them at compile time rather than runtime. NilAway is similar to the standard nilness analyzer, however, it employs much more sophisticated and powerful static analysis techniques to track nil flows within a package as well across packages, and report errors providing users with the nilness flows for easier debugging.

NilAway enjoys three key properties that make it stand out:

  • It is fully-automated: NilAway is equipped with an inference engine, making it require no any additional information from the developers (e.g., annotations) besides standard Go code.

  • It is fast: we have designed NilAway to be fast and scalable, making it suitable for large codebases. In our measurements, we have observed less than 5% build-time overhead when NilAway is enabled. We are also constantly applying optimizations to further reduce its footprint.

  • It is practical: it does not prevent all possible nil panics in your code, but it catches most of the potential nil panics we have observed in production, allowing NilAway to maintain a good balance between usefulness and build-time overhead.

🌟 For more detailed technical discussion, please check our Wiki, Engineering Blog, and paper (WIP).

Running NilAway

NilAway is implemented using the standard go/analysis, making it easy to integrate with existing analyzer drivers (i.e., golangci-lint, nogo, or running as a standalone checker).

[!IMPORTANT]
By default, NilAway analyzes all Go code, including the standard libraries and dependencies. This helps NilAway better understand the code form dependencies and reduce its false negatives. However, this would also incur a significant performance cost (only once for drivers with modular support) and increase the number of non-actionable errors in dependencies, for large Go projects with a lot of dependencies.

We highly recommend using the include-pkgs flag to narrow down the analysis to your project's code exclusively. This directs NilAway to skip analyzing dependencies (e.g., third-party libraries), allowing you to focus solely on potential nil panics reported by NilAway in your first-party code!

Standalone Checker

[!IMPORTANT]
Due to the sophistication of the analyses that NilAway does, NilAway caches its findings about a particular package via the Fact Mechanism from the go/analysis framework. Therefore, it is highly recommended to leverage a driver that supports modular analysis (i.e., bazel/nogo or golangci-lint, but not the standalone checker since it stores all facts in memory) for better performance on large projects. The standalone checker is provided more for evaluation purposes since it is easy to get started.

Install the binary from source by running:

go install go.uber.org/nilaway/cmd/nilaway@latest

Then, run the linter by:

nilaway -include-pkgs="<YOUR_PKG_PREFIX>,<YOUR_PKG_PREFIX_2>" ./...
golangci-lint (>= v1.57.0)

NilAway, in its current form, can report false positives. This unfortunately hinders its immediate merging in golangci-lint and be offered as a linter (see PR#4045). Therefore, you need to build NilAway as a plugin to golangci-lint to be executed as a private linter. There are two plugin systems in golangci-lint, and it is much easier to use the Module Plugin System (introduced since v1.57.0), and it is the only supported approach to run NilAway in golangci-lint.

(1) Create a .custom-gcl.yml file at the root of the repository if you have not done so, add the following content:

# This has to be >= v1.57.0 for module plugin system support.
version: v1.57.0
plugins:
  - module: "go.uber.org/nilaway"
    import: "go.uber.org/nilaway/cmd/gclplugin"
    version: latest # Or a fixed version for reproducible builds.

(2) Add NilAway to the linter configuration file .golangci.yaml:

linters-settings:
  custom:
    nilaway:
      type: "module"
      description: Static analysis tool to detect potential nil panics in Go code.
      settings:
        # Settings must be a "map from string to string" to mimic command line flags: the keys are
        # flag names and the values are the values to the particular flags.
        include-pkgs: "<YOUR_PACKAGE_PREFIXES>"
# NilAway can be referred to as `nilaway` just like any other golangci-lint analyzers in other 
# parts of the configuration file.

(3) Build a custom golangci-lint binary with NilAway included:

# Note that your `golangci-lint` to bootstrap the custom binary must also be version >= v1.57.0.
$ golangci-lint custom

By default, the custom binary will be built at . with the name custom-gcl, which can be further customized in .custom-gcl.yml file (see Module Plugin System for instructions).

[!TIP]
Cache the custom binary to avoid having to build it again to save resources, you can use the hash of the .custom-gcl.yml file as the cache key if you are using a fixed version of NilAway. If you are using latest as NilAway version, you can append the date of build to the cache key to force cache expiration after certain time period.

(4) Run the custom binary instead of golangci-lint:

# Arguments are the same as `golangci-lint`.
$ ./custom-gcl run ./...
Bazel/nogo

Running with bazel/nogo requires slightly more efforts. First follow the instructions from rules_go, gazelle, and nogo to set up your Go project such that it can be built with bazel/nogo with no or default set of linters configured. Then,

(1) Add import _ "go.uber.org/nilaway" to your tools.go file (or other file that you use for configuring tool dependencies, see How can I track tool dependencies for a module? from Go Modules documentation) to avoid go mod tidy from removing NilAway as a tool dependency.

(2) Run the following commands to add NilAway as a tool dependency to your project:

# Get NilAway as a dependency, as well as getting its transitive dependencies in go.mod file.
$ go get go.uber.org/nilaway@latest
# This should not remove NilAway as a dependency in your go.mod file.
$ go mod tidy
# Run gazelle to sync dependencies from go.mod to WORKSPACE file.
$ bazel run //:gazelle -- update-repos -from_file=go.mod

(3) Add NilAway to nogo configurations (usually in top-level BUILD.bazel file):

nogo(
    name = "my_nogo",
    visibility = ["//visibility:public"],  # must have public visibility
    deps = [
+++     "@org_uber_go_nilaway//:go_default_library",
    ],
    config = "config.json",
)

(4) Run bazel build to see NilAway working (any nogo error will stop the bazel build, you can use the --keep_going flag to request bazel to build as much as possible):

$ bazel build --keep_going //...

(5) See nogo documentation on how to pass a configuration JSON to the nogo driver, and see our wiki page on how to pass configurations to NilAway.

Code Examples

Let's look at a few examples to see how NilAway can help prevent nil panics.

// Example 1:
var p *P
if someCondition {
      p = &P{}
}
print(p.f) // nilness reports NO error here, but NilAway does.

In this example, the local variable p is only initialized when someCondition is true. At the field access p.f, a panic may occur if someCondition is false. NilAway is able to catch this potential nil flow and reports the following error showing this nilness flow:

go.uber.org/example.go:12:9: error: Potential nil panic detected. Observed nil flow from source to dereference point:
    - go.uber.org/example.go:12:9: unassigned variable `p` accessed field `f`

If we guard this dereference with a nilness check (if p != nil), the error goes away.

NilAway is also able to catch nil flows across functions. For example, consider the following code snippet:

// Example 2:
func foo() *int {
      return nil
}
func bar() {
     print(*foo()) // nilness reports NO error here, but NilAway does.
}

In this example, the function foo returns a nil pointer, which is directly dereferenced in bar, resulting in a panic whenever bar is called. NilAway is able to catch this potential nil flow and reports the following error, describing the nilness flow across function boundaries:

go.uber.org/example.go:23:13: error: Potential nil panic detected. Observed nil flow from source to dereference point:
    - go.uber.org/example.go:20:14: literal `nil` returned from `foo()` in position 0
    - go.uber.org/example.go:23:13: result 0 of `foo()` dereferenced

Note that in the above example, foo does not necessarily have to reside in the same package as bar. NilAway is able to track nil flows across packages as well. Moreover, NilAway handles Go-specific language constructs such as receivers, interfaces, type assertions, type switches, and more.

Configurations

We expose a set of flags via the standard flag passing mechanism in go/analysis. Please check wiki/Configuration to see the available flags and how to pass them using different linter drivers.

Support

We follow the same version support policy as the Go project: we support and test the last two major versions of Go.

Please feel free to open a GitHub issue if you have any questions, bug reports, and feature requests.

Contributions

We'd love for you to contribute to NilAway! Please note that once you create a pull request, you will be asked to sign our Uber Contributor License Agreement.

License

This project is copyright 2023 Uber Technologies, Inc., and licensed under Apache 2.0.

Documentation

Overview

Package nilaway implements the top-level analyzer that simply retrieves the diagnostics from the accumulation analyzer and reports them.

Index

Constants

This section is empty.

Variables

View Source
var Analyzer = &analysis.Analyzer{
	Name:      "nilaway",
	Doc:       _doc,
	Run:       run,
	FactTypes: []analysis.Fact{},
	Requires:  []*analysis.Analyzer{config.Analyzer, accumulation.Analyzer},
}

Analyzer is the top-level instance of Analyzer - it coordinates the entire dataflow to report nil flow errors in this package. It is needed here for nogo to recognize the package.

Functions

This section is empty.

Types

This section is empty.

Directories

Path Synopsis
Package accumulation coordinates the entire workflow and collects the annotations, full triggers, and then runs inference to generate and return all potential diagnostics for upper-level analyzers to report.
Package accumulation coordinates the entire workflow and collects the annotations, full triggers, and then runs inference to generate and return all potential diagnostics for upper-level analyzers to report.
Package annotation implements annotation-related structs (site, maps, triggers) and methods.
Package annotation implements annotation-related structs (site, maps, triggers) and methods.
Package assertion implements a sub-analyzer that collects full triggers from the sub-analyzers and combine them into a list of full triggers for the entire package.
Package assertion implements a sub-analyzer that collects full triggers from the sub-analyzers and combine them into a list of full triggers for the entire package.
affiliation
Package affiliation implements the affliation analyzer that tries to find the concrete implementation of an interface and create full triggers for them.
Package affiliation implements the affliation analyzer that tries to find the concrete implementation of an interface and create full triggers for them.
anonymousfunc
Package anonymousfunc implements a sub-analyzer to analyze anonymous functions in a package.
Package anonymousfunc implements a sub-analyzer to analyze anonymous functions in a package.
function
Package function implements a sub-analyzer to create full triggers for each function declaration.
Package function implements a sub-analyzer to create full triggers for each function declaration.
function/assertiontree
Package assertiontree contains the node definitions for the assertion tree, as well as the main backpropagation algorithm.
Package assertiontree contains the node definitions for the assertion tree, as well as the main backpropagation algorithm.
function/functioncontracts
Package functioncontracts implements a sub-analyzer to analyze function contracts in a package, i.e., parsing specified function contracts written as special comments before function declarations, or automatically inferring function contracts from the function body.
Package functioncontracts implements a sub-analyzer to analyze function contracts in a package, i.e., parsing specified function contracts written as special comments before function declarations, or automatically inferring function contracts from the function body.
function/preprocess
Package preprocess hosts preprocessing logic for the input (e.g., CFGs etc.) to make it more amenable to analysis.
Package preprocess hosts preprocessing logic for the input (e.g., CFGs etc.) to make it more amenable to analysis.
function/producer
Package producer contains definitions for parsed producers, which are the result of ParseExprAsProducer.
Package producer contains definitions for parsed producers, which are the result of ParseExprAsProducer.
function/trustedfunc
Package trustedfunc implements a trusted function framework where it hooks into different parts of NilAway to provide additional context for certain function calls.
Package trustedfunc implements a trusted function framework where it hooks into different parts of NilAway to provide additional context for certain function calls.
global
Package global implements a sub-analyzer to create full triggers for global variables.
Package global implements a sub-analyzer to create full triggers for global variables.
structfield
Package structfield implements a sub-analyzer that collects struct fields accessed within a function to aid the analysis of the main function analyzer.
Package structfield implements a sub-analyzer that collects struct fields accessed within a function to aid the analysis of the main function analyzer.
cmd
gclplugin
Package gclplugin implements the golangci-lint's module plugin interface for NilAway to be used as a private linter in golangci-lint.
Package gclplugin implements the golangci-lint's module plugin interface for NilAway to be used as a private linter in golangci-lint.
nilaway
main package makes it possible to build NilAway as a standalone code checker that can be independently invoked to check other packages.
main package makes it possible to build NilAway as a standalone code checker that can be independently invoked to check other packages.
Package config implements the configurations for NilAway.
Package config implements the configurations for NilAway.
Package diagnostic hosts the diagnostic engine, which is responsible for collecting the conflicts from annotation-based checks (no-infer mode) and/or inference (full-infer mode) and generating user-friendly diagnostics from those conflicts.
Package diagnostic hosts the diagnostic engine, which is responsible for collecting the conflicts from annotation-based checks (no-infer mode) and/or inference (full-infer mode) and generating user-friendly diagnostics from those conflicts.
Package inference implements the inference algorithm in NilAway to automatically infer the nilability of the annotation sites.
Package inference implements the inference algorithm in NilAway to automatically infer the nilability of the annotation sites.
Package nilawaytest implements utility functions for tests.
Package nilawaytest implements utility functions for tests.
tools module
Package util implements utility functions for AST and types.
Package util implements utility functions for AST and types.
analysishelper
Package analysishelper provides helper functions for the `go/analysis` package.
Package analysishelper provides helper functions for the `go/analysis` package.
asthelper
Package asthelper implements utility functions for AST.
Package asthelper implements utility functions for AST.
orderedmap
Package orderedmap implements a generic ordered map that supports iteration in insertion order.
Package orderedmap implements a generic ordered map that supports iteration in insertion order.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL