regtest

package
v0.12.1-pre.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 1, 2023 License: BSD-3-Clause Imports: 47 Imported by: 0

Documentation

Overview

Package regtest provides a framework for writing gopls regression tests.

User reported regressions are often expressed in terms of editor interactions. For example: "When I open my editor in this directory, navigate to this file, and change this line, I get a diagnostic that doesn't make sense". In these cases reproducing, diagnosing, and writing a test to protect against this regression can be difficult.

The regtest package provides an API for developers to express these types of user interactions in ordinary Go tests, validate them, and run them in a variety of execution modes.

Test package setup

The regression test package uses a couple of uncommon patterns to reduce boilerplate in test bodies. First, it is intended to be imported as "." so that helpers do not need to be qualified. Second, it requires some setup that is currently implemented in the regtest.Main function, which must be invoked by TestMain. Therefore, a minimal regtest testing package looks like this: 

package lsptests

import (
	"fmt"
	"testing"

	"golang.org/x/tools/gopls/internal/hooks"
	. "golang.org/x/tools/gopls/internal/lsp/regtest"
)

func TestMain(m *testing.M) {
	Main(m, hooks.Options)
}

Writing a simple regression test

To run a regression test use the regtest.Run function, which accepts a txtar-encoded archive defining the initial workspace state. This function sets up the workspace in a temporary directory, creates a fake text editor, starts gopls, and initializes an LSP session. It then invokes the provided test function with an *Env handle encapsulating the newly created environment. Because gopls may be run in various modes (as a sidecar or daemon process, with different settings), the test runner may perform this process multiple times, re-running the test function each time with a new environment. 

func TestOpenFile(t *testing.T) {
	const files = `
-- go.mod --
module mod.com

go 1.12
-- foo.go --
package foo
`
	Run(t, files, func(t *testing.T, env *Env) {
		env.OpenFile("foo.go")
	})
}

Configuring Regtest Execution

The regtest package exposes several options that affect the setup process described above. To use these options, use the WithOptions function: 

WithOptions(opts...).Run(...)

See options.go for a full list of available options.

Operating on editor state

To operate on editor state within the test body, the Env type provides access to the workspace directory (Env.SandBox), text editor (Env.Editor), LSP server (Env.Server), and 'awaiter' (Env.Awaiter).

In most cases, operations on these primitive building blocks of the regression test environment expect a Context (which should be a child of env.Ctx), and return an error. To avoid boilerplate, the Env exposes a set of wrappers in wrappers.go for use in scripting: 

env.CreateBuffer("c/c.go", "")
env.EditBuffer("c/c.go", fake.Edit{
	Text: `package c`,
})

These wrappers thread through Env.Ctx, and call t.Fatal on any errors.

Expressing expectations

The general pattern for a regression test is to script interactions with the fake editor and sandbox, and assert that gopls behaves correctly after each state change. Unfortunately, this is complicated by the fact that state changes are communicated to gopls via unidirectional client->server notifications (didOpen, didChange, etc.), and resulting gopls behavior such as diagnostics, logs, or messages is communicated back via server->client notifications. Therefore, within regression tests we must be able to say "do this, and then eventually gopls should do that". To achieve this, the regtest package provides a framework for expressing conditions that must eventually be met, in terms of the Expectation type.

To express the assertion that "eventually gopls must meet these expectations", use env.Await(...): 

env.RegexpReplace("x/x.go", `package x`, `package main`)
env.Await(env.DiagnosticAtRegexp("x/main.go", `fmt`))

Await evaluates the provided expectations atomically, whenever the client receives a state-changing notification from gopls. See expectation.go for a full list of available expectations.

A fundamental problem with this model is that if gopls never meets the provided expectations, the test runner will hang until the test timeout (which defaults to 10m). There are two ways to work around this poor behavior:

  1. Use a precondition to define precisely when we expect conditions to be met. Gopls provides the OnceMet(precondition, expectations...) pattern to express ("once this precondition is met, the following expectations must all hold"). To instrument preconditions, gopls uses verbose progress notifications to inform the client about ongoing work (see CompletedWork). The most common precondition is to wait for gopls to be done processing all change notifications, for which the regtest package provides the AfterChange helper. For example:

    // We expect diagnostics to be cleared after gopls is done processing the // didSave notification. env.SaveBuffer("a/go.mod") env.AfterChange(EmptyDiagnostics("a/go.mod"))

  2. Set a shorter timeout during development, if you expect to be breaking tests. By setting the environment variable GOPLS_REGTEST_TIMEOUT=5s, regression tests will time out after 5 seconds.

Tips & Tricks

Here are some tips and tricks for working with regression tests:

  1. Set the environment variable GOPLS_REGTEST_TIMEOUT=5s during development.
  2. Run tests with -short. This will only run regression tests in the default gopls execution mode.
  3. Use capture groups to narrow regexp positions. All regular-expression based positions (such as DiagnosticAtRegexp) will match the position of the first capture group, if any are provided. This can be used to identify a specific position in the code for a pattern that may occur in multiple places. For example `var (mu) sync.Mutex` matches the position of "mu" within the variable declaration.
  4. Read diagnostics into a variable to implement more complicated assertions about diagnostic state in the editor. To do this, use the pattern OnceMet(precondition, ReadDiagnostics("file.go", &d)) to capture the current diagnostics as soon as the precondition is met. This is preferable to accessing the diagnostics directly, as it avoids races.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// InitialWorkspaceLoad is an expectation that the workspace initial load has
	// completed. It is verified via workdone reporting.
	InitialWorkspaceLoad = CompletedWork(lsp.DiagnosticWorkTitle(lsp.FromInitialWorkspaceLoad), 1, false)
)

Functions

func Main 

func Main(m *testing.M, hook func(*source.Options))

Main sets up and tears down the shared regtest state.

func Run 

func Run(t *testing.T, files string, f TestFunc)

func RunMarkerTests added in v0.12.0 

func RunMarkerTests(t *testing.T, dir string)

RunMarkerTests runs "marker" tests in the given test data directory. (In practice: ../../regtest/marker/testdata)

Use this command to run the tests: 

$ go test ./gopls/internal/regtest/marker [-update]

A marker test uses the '//@' marker syntax of the x/tools/go/expect package to annotate source code with various information such as locations and arguments of LSP operations to be executed by the test. The syntax following '@' is parsed as a comma-separated list of ordinary Go function calls, for example 

//@foo(a, "b", 3),bar(0)

and delegates to a corresponding function to perform LSP-related operations. See the Marker types documentation below for a list of supported markers.

Each call argument is converted to the type of the corresponding parameter of the designated function. The conversion logic may use the surrounding context, such as the position or nearby text. See the Argument conversion section below for the full set of special conversions. As a special case, the blank identifier '_' is treated as the zero value of the parameter type.

The test runner collects test cases by searching the given directory for files with the .txt extension. Each file is interpreted as a txtar archive, which is extracted to a temporary directory. The relative path to the .txt file is used as the subtest name. The preliminary section of the file (before the first archive entry) is a free-form comment.

These tests were inspired by (and in many places copied from) a previous iteration of the marker tests built on top of the packagestest framework. Key design decisions motivating this reimplementation are as follows:

  • The old tests had a single global session, causing interaction at a distance and several awkward workarounds.
  • The old tests could not be safely parallelized, because certain tests manipulated the server options
  • Relatedly, the old tests did not have a logic grouping of assertions into a single unit, resulting in clusters of files serving clusters of entangled assertions.
  • The old tests used locations in the source as test names and as the identity of golden content, meaning that a single edit could change the name of an arbitrary number of subtests, and making it difficult to manually edit golden content.
  • The old tests did not hew closely to LSP concepts, resulting in, for example, each marker implementation doing its own position transformations, and inventing its own mechanism for configuration.
  • The old tests had an ad-hoc session initialization process. The regtest environment has had more time devoted to its initialization, and has a more convenient API.
  • The old tests lacked documentation, and often had failures that were hard to understand. By starting from scratch, we can revisit these aspects.

Special files

There are several types of file within the test archive that are given special treatment by the test runner:

  • "flags": this file is treated as a whitespace-separated list of flags that configure the MarkerTest instance. Supported flags: -min_go=go1.18 sets the minimum Go version for the test; -cgo requires that CGO_ENABLED is set and the cgo tool is available -write_sumfile=a,b,c instructs the test runner to generate go.sum files in these directories before running the test. -skip_goos=a,b,c instructs the test runner to skip the test for the listed GOOS values. TODO(rfindley): using build constraint expressions for -skip_goos would be clearer. TODO(rfindley): support flag values containing whitespace.
  • "settings.json": this file is parsed as JSON, and used as the session configuration (see gopls/doc/settings.md)
  • "env": this file is parsed as a list of VAR=VALUE fields specifying the editor environment.
  • Golden files: Within the archive, file names starting with '@' are treated as "golden" content, and are not written to disk, but instead are made available to test methods expecting an argument of type *Golden, using the identifier following '@'. For example, if the first parameter of Foo were of type *Golden, the test runner would convert the identifier a in the call @foo(a, "b", 3) into a *Golden by collecting golden file data starting with "@a/".
  • proxy files: any file starting with proxy/ is treated as a Go proxy file. If present, these files are written to a separate temporary directory and GOPROXY is set to file://<proxy directory>.

Marker types

The following markers are supported within marker tests:

  • codeaction(kind, start, end, golden): specifies a codeaction to request for the given range. To support multi-line ranges, the range is defined to be between start.Start and end.End. The golden directory contains changed file content after the code action is applied.

  • codeactionerr(kind, start, end, wantError): specifies a codeaction that fails with an error that matches the expectation.

  • complete(location, ...labels): specifies expected completion results at the given location.

  • diag(location, regexp): specifies an expected diagnostic matching the given regexp at the given location. The test runner requires a 1:1 correspondence between observed diagnostics and diag annotations. The diagnostics source and kind fields are ignored, to reduce fuss.

    The specified location must match the start position of the diagnostic, but end positions are ignored.

    TODO(adonovan): in the older marker framework, the annotation asserted two additional fields (source="compiler", kind="error"). Restore them?

  • def(src, dst location): perform a textDocument/definition request at the src location, and check the result points to the dst location.

  • format(golden): perform a textDocument/format request for the enclosing file, and compare against the named golden file. If the formatting request succeeds, the golden file must contain the resulting formatted source. If the formatting request fails, the golden file must contain the error message.

  • hover(src, dst location, g Golden): perform a textDocument/hover at the src location, and checks that the result is the dst location, with hover content matching "hover.md" in the golden data g.

  • implementations(src location, want ...location): makes a textDocument/implementation query at the src location and checks that the resulting set of locations matches want.

  • loc(name, location): specifies the name for a location in the source. These locations may be referenced by other markers.

  • rename(location, new, golden): specifies a renaming of the identifier at the specified location to the new name. The golden directory contains the transformed files.

  • renameerr(location, new, wantError): specifies a renaming that fails with an error that matches the expectation.

  • suggestedfix(location, regexp, kind, golden): like diag, the location and regexp identify an expected diagnostic. This diagnostic must to have exactly one associated code action of the specified kind. This action is executed for its editing effects on the source files. Like rename, the golden directory contains the expected transformed files.

  • refs(location, want ...location): executes a 'references' query at the first location and asserts that the result is the set of 'want' locations. The first want location must be the declaration (assumedly unique).

  • symbol(golden): makes a textDocument/documentSymbol request for the enclosing file, formats the response with one symbol per line, sorts it, and compares against the named golden file. Each line is of the form:

    dotted.symbol.name kind "detail" +n lines

    where the "+n lines" part indicates that the declaration spans several lines. The test otherwise makes no attempt to check location information. There is no point to using more than one @symbol marker in a given file.

  • workspacesymbol(query, golden): makes a workspace/symbol request for the given query, formats the response with one symbol per line, and compares against the named golden file. As workspace symbols are by definition a workspace-wide request, the location of the workspace symbol marker does not matter. Each line is of the form:

    location name kind

Argument conversion

Marker arguments are first parsed by the go/expect package, which accepts the following tokens as defined by the Go spec:

  • string, int64, float64, and rune literals
  • true and false
  • nil
  • identifiers (type expect.Identifier)
  • regular expressions, denoted the two tokens re"abc" (type *regexp.Regexp)

These values are passed as arguments to the corresponding parameter of the test function. Additional value conversions may occur for these argument -> parameter type pairs:

  • string->regexp: the argument is parsed as a regular expressions.
  • string->location: the argument is converted to the location of the first instance of the argument in the partial line preceding the note.
  • regexp->location: the argument is converted to the location of the first match for the argument in the partial line preceding the note. If the regular expression contains exactly one subgroup, the position of the subgroup is used rather than the position of the submatch.
  • name->location: the argument is replaced by the named location.
  • name->Golden: the argument is used to look up golden content prefixed by @<argument>.
  • {string,regexp,identifier}->wantError: a wantError type specifies an expected error message, either in the form of a substring that must be present, a regular expression that it must match, or an identifier (e.g. foo) such that the archive entry @foo exists and contains the exact expected error.

Example

Here is a complete example: 

-- a.go --
package a

const abc = 0x2a //@hover("b", "abc", abc),hover(" =", "abc", abc)
-- @abc/hover.md --
```go
const abc untyped int = 42
```

@hover("b", "abc", abc),hover(" =", "abc", abc)

In this example, the @hover annotation tells the test runner to run the hoverMarker function, which has parameters: 

(mark marker, src, dsc protocol.Location, g *Golden).

The first argument holds the test context, including fake editor with open files, and sandboxed directory.

Argument converters translate the "b" and "abc" arguments into locations by interpreting each one as a regular expression and finding the location of its first match on the preceding portion of the line, and the abc identifier into a dictionary of golden content containing "hover.md". Then the hoverMarker method executes a textDocument/hover LSP request at the src position, and ensures the result spans "abc", with the markdown content from hover.md. (Note that the markdown content includes the expect annotation as the doc comment.)

The next hover on the same line asserts the same result, but initiates the hover immediately after "abc" in the source. This tests that we find the preceding identifier when hovering.

Updating golden files

To update golden content in the test archive, it is easier to regenerate content automatically rather than edit it by hand. To do this, run the tests with the -update flag. Only tests that actually run will be updated.

In some cases, golden content will vary by Go version (for example, gopls produces different markdown at Go versions before the 1.19 go/doc update). By convention, the golden content in test archives should match the output at Go tip. Each test function can normalize golden content for older Go versions.

Note that -update does not cause missing @diag or @loc markers to be added.

TODO

This API is a work-in-progress, as we migrate existing marker tests from internal/lsp/tests.

Remaining TODO:

  • parallelize/optimize test execution
  • reorganize regtest packages (and rename to just 'test'?)
  • Rename the files .txtar.
  • Provide some means by which locations in the standard library (or builtin.go) can be named, so that, for example, we can we can assert that MyError implements the built-in error type.

Existing marker tests (in ../testdata) to port:

  • CallHierarchy
  • CodeLens
  • Diagnostics
  • CompletionItems
  • Completions
  • CompletionSnippets
  • UnimportedCompletions
  • DeepCompletions
  • FuzzyCompletions
  • CaseSensitiveCompletions
  • RankCompletions
  • FoldingRanges
  • Formats
  • Imports
  • SemanticTokens
  • FunctionExtractions
  • MethodExtractions
  • Highlights
  • Renames
  • PrepareRenames
  • InlayHints
  • WorkspaceSymbols
  • Signatures
  • Links
  • AddImport
  • SelectionRanges

func WithOptions 

func WithOptions(opts ...RunOption) configuredRunner

Types

type Awaiter 

type Awaiter struct {
	// contains filtered or unexported fields
}

An Awaiter keeps track of relevant LSP state, so that it may be asserted upon with Expectations.

Wire it into a fake.Editor using Awaiter.Hooks().

TODO(rfindley): consider simply merging Awaiter with the fake.Editor. It probably is not worth its own abstraction.

func NewAwaiter 

func NewAwaiter(workdir *fake.Workdir) *Awaiter

func (*Awaiter) Await 

func (a *Awaiter) Await(ctx context.Context, expectations ...Expectation) error

Await waits for all expectations to simultaneously be met. It should only be called from the main test goroutine.

func (*Awaiter) Hooks 

func (a *Awaiter) Hooks() fake.ClientHooks

Hooks returns LSP client hooks required for awaiting asynchronous expectations.

type DiagnosticFilter added in v0.12.0 

type DiagnosticFilter struct {
	// contains filtered or unexported fields
}

A DiagnosticFilter filters the set of diagnostics, for assertion with Diagnostics or NoDiagnostics.

func AtPosition added in v0.12.0 

func AtPosition(name string, line, character uint32) DiagnosticFilter

AtPosition filters to diagnostics at location name:line:character, for a sandbox-relative path name.

Line and character are 0-based, and character measures UTF-16 codes.

Note: prefer the more readable AtRegexp.

func ForFile added in v0.12.0 

func ForFile(name string) DiagnosticFilter

ForFile filters to diagnostics matching the sandbox-relative file name.

func FromSource added in v0.12.0 

func FromSource(source string) DiagnosticFilter

FromSource filters to diagnostics matching the given diagnostics source.

func WithMessage added in v0.12.0 

func WithMessage(substring string) DiagnosticFilter

WithMessage filters to diagnostics whose message contains the given substring.

type Env 

type Env struct {
	T   testing.TB // TODO(rfindley): rename to TB
	Ctx context.Context

	// Most tests should not need to access the scratch area, editor, server, or
	// connection, but they are available if needed.
	Sandbox *fake.Sandbox
	Server  servertest.Connector

	// Editor is owned by the Env, and shut down
	Editor *fake.Editor

	Awaiter *Awaiter
}

Env holds the building blocks of an editor testing environment, providing wrapper methods that hide the boilerplate of plumbing contexts and checking errors.

func (*Env) AcceptCompletion 

func (e *Env) AcceptCompletion(loc protocol.Location, item protocol.CompletionItem)

AcceptCompletion accepts a completion for the given item at the given position.

func (*Env) AfterChange added in v0.11.0 

func (e *Env) AfterChange(expectations ...Expectation)

AfterChange expects that the given expectations will be met after all state-changing notifications have been processed by the server.

It awaits the completion of all anticipated work before checking the given expectations.

func (*Env) ApplyCodeAction 

func (e *Env) ApplyCodeAction(action protocol.CodeAction)

ApplyCodeAction applies the given code action.

func (*Env) ApplyQuickFixes 

func (e *Env) ApplyQuickFixes(path string, diagnostics []protocol.Diagnostic)

ApplyQuickFixes processes the quickfix codeAction, calling t.Fatal on any error.

func (*Env) AtRegexp added in v0.12.0 

func (e *Env) AtRegexp(name, pattern string) DiagnosticFilter

AtRegexp filters to diagnostics in the file with sandbox-relative path name, at the first position matching the given regexp pattern.

TODO(rfindley): pass in the editor to expectations, so that they may depend on editor state and AtRegexp can be a function rather than a method.

func (*Env) Await 

func (e *Env) Await(expectations ...Expectation)

Await blocks until the given expectations are all simultaneously met.

Generally speaking Await should be avoided because it blocks indefinitely if gopls ends up in a state where the expectations are never going to be met. Use AfterChange or OnceMet instead, so that the runner knows when to stop waiting.

func (*Env) BufferText added in v0.12.0 

func (e *Env) BufferText(name string) string

BufferText returns the current buffer contents for the file with the given relative path, calling t.Fatal if the file is not open in a buffer.

func (*Env) ChangeConfiguration 

func (e *Env) ChangeConfiguration(newConfig fake.EditorConfig)

ChangeConfiguration updates the editor config, calling t.Fatal on any error.

func (*Env) ChangeWorkspaceFolders 

func (e *Env) ChangeWorkspaceFolders(newFolders ...string)

ChangeWorkspaceFolders updates the editor workspace folders, calling t.Fatal on any error.

func (*Env) CheckForFileChanges 

func (e *Env) CheckForFileChanges()

CheckForFileChanges triggers a manual poll of the workspace for any file changes since creation, or since last polling. It is a workaround for the lack of true file watching support in the fake workspace.

func (*Env) Close added in v0.12.0 

func (e *Env) Close()

Close shuts down the editor session and cleans up the sandbox directory, calling t.Error on any error.

func (*Env) CloseBuffer 

func (e *Env) CloseBuffer(name string)

CloseBuffer closes an editor buffer without saving, calling t.Fatal on any error.

func (*Env) CodeAction 

func (e *Env) CodeAction(path string, diagnostics []protocol.Diagnostic) []protocol.CodeAction

CodeAction calls testDocument/codeAction for the given path, and calls t.Fatal if there are errors.

func (*Env) CodeLens 

func (e *Env) CodeLens(path string) []protocol.CodeLens

CodeLens calls textDocument/codeLens for the given path, calling t.Fatal on any error.

func (*Env) Completion 

func (e *Env) Completion(loc protocol.Location) *protocol.CompletionList

Completion executes a completion request on the server.

func (*Env) CreateBuffer 

func (e *Env) CreateBuffer(name string, content string)

CreateBuffer creates a buffer in the editor, calling t.Fatal on any error.

func (*Env) DocumentHighlight 

func (e *Env) DocumentHighlight(loc protocol.Location) []protocol.DocumentHighlight
func (e *Env) DocumentLink(name string) []protocol.DocumentLink

func (*Env) DoneDiagnosingChanges added in v0.11.0 

func (e *Env) DoneDiagnosingChanges() Expectation

DoneDiagnosingChanges expects that diagnostics are complete from common change notifications: didOpen, didChange, didSave, didChangeWatchedFiles, and didClose.

This can be used when multiple notifications may have been sent, such as when a didChange is immediately followed by a didSave. It is insufficient to simply await NoOutstandingWork, because the LSP client has no control over when the server starts processing a notification. Therefore, we must keep track of

func (*Env) DoneWithChange 

func (e *Env) DoneWithChange() Expectation

DoneWithChange expects all didChange notifications currently sent by the editor to be completely processed.

func (*Env) DoneWithChangeWatchedFiles 

func (e *Env) DoneWithChangeWatchedFiles() Expectation

DoneWithChangeWatchedFiles expects all didChangeWatchedFiles notifications currently sent by the editor to be completely processed.

func (*Env) DoneWithClose 

func (e *Env) DoneWithClose() Expectation

DoneWithClose expects all didClose notifications currently sent by the editor to be completely processed.

func (*Env) DoneWithOpen 

func (e *Env) DoneWithOpen() Expectation

DoneWithOpen expects all didOpen notifications currently sent by the editor to be completely processed.

func (*Env) DoneWithSave 

func (e *Env) DoneWithSave() Expectation

DoneWithSave expects all didSave notifications currently sent by the editor to be completely processed.

func (*Env) DumpGoSum 

func (e *Env) DumpGoSum(dir string)

DumpGoSum prints the correct go.sum contents for dir in txtar format, for use in creating regtests.

func (*Env) EditBuffer 

func (e *Env) EditBuffer(name string, edits ...protocol.TextEdit)

EditBuffer applies edits to an editor buffer, calling t.Fatal on any error.

func (*Env) ExecuteCodeLensCommand 

func (e *Env) ExecuteCodeLensCommand(path string, cmd command.Command, result interface{})

ExecuteCodeLensCommand executes the command for the code lens matching the given command name.

func (*Env) ExecuteCommand 

func (e *Env) ExecuteCommand(params *protocol.ExecuteCommandParams, result interface{})

func (*Env) FormatBuffer 

func (e *Env) FormatBuffer(name string)

FormatBuffer formats the editor buffer, calling t.Fatal on any error.

func (*Env) GetQuickFixes 

func (e *Env) GetQuickFixes(path string, diagnostics []protocol.Diagnostic) []protocol.CodeAction

GetQuickFixes returns the available quick fix code actions.

func (*Env) GoToDefinition 

func (e *Env) GoToDefinition(loc protocol.Location) protocol.Location

GoToDefinition goes to definition in the editor, calling t.Fatal on any error. It returns the path and position of the resulting jump.

func (*Env) GoVersion 

func (e *Env) GoVersion() int

GoVersion checks the version of the go command. It returns the X in Go 1.X.

func (*Env) Hover 

func (e *Env) Hover(loc protocol.Location) (*protocol.MarkupContent, protocol.Location)

Hover in the editor, calling t.Fatal on any error.

func (*Env) Implementations 

func (e *Env) Implementations(loc protocol.Location) []protocol.Location

Implementations wraps Editor.Implementations, calling t.Fatal on any error.

func (*Env) InlayHints 

func (e *Env) InlayHints(path string) []protocol.InlayHint

InlayHints calls textDocument/inlayHints for the given path, calling t.Fatal on any error.

func (*Env) ListFiles 

func (e *Env) ListFiles(dir string) []string

ListFiles lists relative paths to files in the given directory. It calls t.Fatal on any error.

func (*Env) OnceMet added in v0.12.0 

func (e *Env) OnceMet(precondition Expectation, mustMeets ...Expectation)

OnceMet blocks until the precondition is met by the state or becomes unmeetable. If it was met, OnceMet checks that the state meets all expectations in mustMeets.

func (*Env) OpenFile 

func (e *Env) OpenFile(name string)

OpenFile opens a file in the editor, calling t.Fatal on any error.

func (*Env) OrganizeImports 

func (e *Env) OrganizeImports(name string)

OrganizeImports processes the source.organizeImports codeAction, calling t.Fatal on any error.

func (*Env) ReadWorkspaceFile 

func (e *Env) ReadWorkspaceFile(name string) string

ReadWorkspaceFile reads a file from the workspace, calling t.Fatal on any error.

func (*Env) References 

func (e *Env) References(loc protocol.Location) []protocol.Location

References wraps Editor.References, calling t.Fatal on any error.

func (*Env) RegexpReplace 

func (e *Env) RegexpReplace(name, regexpStr, replace string)

RegexpReplace replaces the first group in the first match of regexpStr with the replace text, calling t.Fatal on any error.

func (*Env) RegexpSearch 

func (e *Env) RegexpSearch(name, re string) protocol.Location

RegexpSearch returns the starting position of the first match for re in the buffer specified by name, calling t.Fatal on any error. It first searches for the position in open buffers, then in workspace files.

func (*Env) RemoveWorkspaceFile 

func (e *Env) RemoveWorkspaceFile(name string)

RemoveWorkspaceFile deletes a file on disk but does nothing in the editor. It calls t.Fatal on any error.

func (*Env) Rename 

func (e *Env) Rename(loc protocol.Location, newName string)

Rename wraps Editor.Rename, calling t.Fatal on any error.

func (*Env) RenameFile 

func (e *Env) RenameFile(oldPath, newPath string)

RenameFile wraps Editor.RenameFile, calling t.Fatal on any error.

func (*Env) RunGenerate 

func (e *Env) RunGenerate(dir string)

RunGenerate runs "go generate" in the given dir, calling t.Fatal on any error. It waits for the generate command to complete and checks for file changes before returning.

func (*Env) RunGoCommand 

func (e *Env) RunGoCommand(verb string, args ...string)

RunGoCommand runs the given command in the sandbox's default working directory.

func (*Env) RunGoCommandInDir 

func (e *Env) RunGoCommandInDir(dir, verb string, args ...string)

RunGoCommandInDir is like RunGoCommand, but executes in the given relative directory of the sandbox.

func (*Env) SaveBuffer 

func (e *Env) SaveBuffer(name string)

SaveBuffer saves an editor buffer, calling t.Fatal on any error.

func (*Env) SaveBufferWithoutActions 

func (e *Env) SaveBufferWithoutActions(name string)

func (*Env) SetBufferContent 

func (e *Env) SetBufferContent(name string, content string)

func (*Env) SignatureHelp added in v0.12.0 

func (e *Env) SignatureHelp(loc protocol.Location) *protocol.SignatureHelp

SignatureHelp wraps Editor.SignatureHelp, calling t.Fatal on error

func (*Env) StartedChange 

func (e *Env) StartedChange() Expectation

StartedChange expects that the server has at least started processing all didChange notifications sent from the client.

func (*Env) StartedChangeWatchedFiles added in v0.12.0 

func (e *Env) StartedChangeWatchedFiles() Expectation

StartedChangeWatchedFiles expects that the server has at least started processing all didChangeWatchedFiles notifications sent from the client.

func (*Env) Symbol 

func (e *Env) Symbol(query string) []protocol.SymbolInformation

Symbol calls workspace/symbol

func (*Env) WriteWorkspaceFile 

func (e *Env) WriteWorkspaceFile(name, content string)

WriteWorkspaceFile writes a file to disk but does nothing in the editor. It calls t.Fatal on any error.

func (*Env) WriteWorkspaceFiles 

func (e *Env) WriteWorkspaceFiles(files map[string]string)

WriteWorkspaceFiles deletes a file on disk but does nothing in the editor. It calls t.Fatal on any error.

type EnvVars 

type EnvVars map[string]string

EnvVars sets environment variables for the LSP session. When applying these variables to the session, the special string $SANDBOX_WORKDIR is replaced by the absolute path to the sandbox working directory.

type Expectation 

type Expectation struct {
	Check func(State) Verdict

	// Description holds a noun-phrase identifying what the expectation checks.
	//
	// TODO(rfindley): revisit existing descriptions to ensure they compose nicely.
	Description string
}

An Expectation is an expected property of the state of the LSP client. The Check function reports whether the property is met.

Expectations are combinators. By composing them, tests may express complex expectations in terms of simpler ones.

TODO(rfindley): as expectations are combined, it becomes harder to identify why they failed. A better signature for Check would be 

func(State) (Verdict, string)

returning a reason for the verdict that can be composed similarly to descriptions.

func AllOf added in v0.11.0 

func AllOf(allOf ...Expectation) Expectation

AllOf expects that all given expectations are met.

TODO(rfindley): the problem with these types of combinators (OnceMet, AnyOf and AllOf) is that we lose the information of *why* they failed: the Awaiter is not smart enough to look inside.

Refactor the API such that the Check function is responsible for explaining why an expectation failed. This should allow us to significantly improve test output: we won't need to summarize state at all, as the verdict explanation itself should describe clearly why the expectation not met.

func AnyOf 

func AnyOf(anyOf ...Expectation) Expectation

AnyOf returns an expectation that is satisfied when any of the given expectations is met.

func CompletedProgress added in v0.11.0 

func CompletedProgress(token protocol.ProgressToken, into *WorkStatus) Expectation

CompletedProgress expects that workDone progress is complete for the given progress token. When non-nil WorkStatus is provided, it will be filled when the expectation is met.

If the token is not a progress token that the client has seen, this expectation is Unmeetable.

func CompletedWork 

func CompletedWork(title string, count uint64, atLeast bool) Expectation

CompletedWork expects a work item to have been completed >= atLeast times.

Since the Progress API doesn't include any hidden metadata, we must use the progress notification title to identify the work we expect to be completed.

func Diagnostics added in v0.12.0 

func Diagnostics(filters ...DiagnosticFilter) Expectation

Diagnostics asserts that there is at least one diagnostic matching the given filters.

func FileWatchMatching 

func FileWatchMatching(re string) Expectation

FileWatchMatching expects that a file registration matches re.

func LogMatching 

func LogMatching(typ protocol.MessageType, re string, count int, atLeast bool) Expectation

LogMatching asserts that the client has received a log message of type typ matching the regexp re a certain number of times.

The count argument specifies the expected number of matching logs. If atLeast is set, this is a lower bound, otherwise there must be exactly count matching logs.

func NoDiagnostics 

func NoDiagnostics(filters ...DiagnosticFilter) Expectation

NoDiagnostics asserts that there are no diagnostics matching the given filters. Notably, if no filters are supplied this assertion checks that there are no diagnostics at all, for any file.

func NoErrorLogs 

func NoErrorLogs() Expectation

NoErrorLogs asserts that the client has not received any log messages of error severity.

func NoFileWatchMatching 

func NoFileWatchMatching(re string) Expectation

NoFileWatchMatching expects that no file registration matches re.

func NoLogMatching 

func NoLogMatching(typ protocol.MessageType, re string) Expectation

NoLogMatching asserts that the client has not received a log message of type typ matching the regexp re. If re is an empty string, any log message is considered a match.

func NoOutstandingWork 

func NoOutstandingWork() Expectation

NoOutstandingWork asserts that there is no work initiated using the LSP $/progress API that has not completed.

func NoShownMessage 

func NoShownMessage(subString string) Expectation

NoShownMessage asserts that the editor has not received a ShowMessage.

func OnceMet 

func OnceMet(precondition Expectation, mustMeets ...Expectation) Expectation

OnceMet returns an Expectation that, once the precondition is met, asserts that mustMeet is met.

func OutstandingWork 

func OutstandingWork(title, msg string) Expectation

OutstandingWork expects a work item to be outstanding. The given title must be an exact match, whereas the given msg must only be contained in the work item's message.

func ReadAllDiagnostics added in v0.12.0 

func ReadAllDiagnostics(into *map[string]*protocol.PublishDiagnosticsParams) Expectation

ReadAllDiagnostics is an expectation that stores all published diagnostics into the provided map, whenever it is evaluated.

It can be used in combination with OnceMet or AfterChange to capture the state of diagnostics when other expectations are satisfied.

func ReadDiagnostics 

func ReadDiagnostics(fileName string, into *protocol.PublishDiagnosticsParams) Expectation

ReadDiagnostics is an Expectation that stores the current diagnostics for fileName in into, whenever it is evaluated.

It can be used in combination with OnceMet or AfterChange to capture the state of diagnostics when other expectations are satisfied.

func ShowMessageRequest 

func ShowMessageRequest(title string) Expectation

ShowMessageRequest asserts that the editor has received a ShowMessageRequest with an action item that has the given title.

func ShownMessage 

func ShownMessage(containing string) Expectation

ShownMessage asserts that the editor has received a ShowMessageRequest containing the given substring.

func StartedWork 

func StartedWork(title string, atLeast uint64) Expectation

StartedWork expect a work item to have been started >= atLeast times.

See CompletedWork.

type Golden added in v0.12.0 

type Golden struct {
	// contains filtered or unexported fields
}

Golden holds extracted golden content for a single @<name> prefix.

When -update is set, golden captures the updated golden contents for later writing.

func (*Golden) Get added in v0.12.0 

func (g *Golden) Get(t testing.TB, name string, updated []byte) ([]byte, bool)

Get returns golden content for the given name, which corresponds to the relative path following the golden prefix @<name>/. For example, to access the content of @foo/path/to/result.json from the Golden associated with @foo, name should be "path/to/result.json".

If -update is set, the given update function will be called to get the updated golden content that should be written back to testdata.

Marker functions must use this method instead of accessing data entries directly otherwise the -update operation will delete those entries.

TODO(rfindley): rethink the logic here. We may want to separate Get and Set, and not delete golden content that isn't set.

type Mode 

type Mode int

Mode is a bitmask that defines for which execution modes a test should run.

Each mode controls several aspects of gopls' configuration:

  • Which server options to use for gopls sessions
  • Whether to use a shared cache
  • Whether to use a shared server
  • Whether to run the server in-process or in a separate process

The behavior of each mode with respect to these aspects is summarized below. TODO(rfindley, cleanup): rather than using arbitrary names for these modes, we can compose them explicitly out of the features described here, allowing individual tests more freedom in constructing problematic execution modes. For example, a test could assert on a certain behavior when running with experimental options on a separate process. Moreover, we could unify 'Modes' with 'Options', and use RunMultiple rather than a hard-coded loop through modes.

Mode | Options | Shared Cache? | Shared Server? | In-process? --------------------------------------------------------------------------- Default | Default | Y | N | Y Forwarded | Default | Y | Y | Y SeparateProcess | Default | Y | Y | N Experimental | Experimental | N | N | Y 

const (
	// Default mode runs gopls with the default options, communicating over pipes
	// to emulate the lsp sidecar execution mode, which communicates over
	// stdin/stdout.
	//
	// It uses separate servers for each test, but a shared cache, to avoid
	// duplicating work when processing GOROOT.
	Default Mode = 1 << iota

	// Forwarded uses the default options, but forwards connections to a shared
	// in-process gopls server.
	Forwarded

	// SeparateProcess uses the default options, but forwards connection to an
	// external gopls daemon.
	//
	// Only supported on GOOS=linux.
	SeparateProcess

	// Experimental enables all of the experimental configurations that are
	// being developed, and runs gopls in sidecar mode.
	//
	// It uses a separate cache for each test, to exercise races that may only
	// appear with cache misses.
	Experimental
)

func DefaultModes 

func DefaultModes() Mode

DefaultModes returns the default modes to run for each regression test (they may be reconfigured by the tests themselves).

func (Mode) String 

func (m Mode) String() string

type RunMultiple 

type RunMultiple []struct {
	Name   string
	Runner regtestRunner
}

func (RunMultiple) Run 

func (r RunMultiple) Run(t *testing.T, files string, f TestFunc)

type RunOption 

type RunOption interface {
	// contains filtered or unexported methods
}

A RunOption augments the behavior of the test runner.

func ClientName added in v0.12.0 

func ClientName(name string) RunOption

ClientName sets the LSP client name.

func InGOPATH 

func InGOPATH() RunOption

InGOPATH configures the workspace working directory to be GOPATH, rather than a separate working directory for use with modules.

func Modes 

func Modes(modes Mode) RunOption

Modes configures the execution modes that the test should run in.

By default, modes are configured by the test runner. If this option is set, it overrides the set of default modes and the test runs in exactly these modes.

func ProxyFiles 

func ProxyFiles(txt string) RunOption

ProxyFiles configures a file proxy using the given txtar-encoded string.

func WindowsLineEndings 

func WindowsLineEndings() RunOption

WindowsLineEndings configures the editor to use windows line endings.

func WorkspaceFolders 

func WorkspaceFolders(relFolders ...string) RunOption

WorkspaceFolders configures the workdir-relative workspace folders to send to the LSP server. By default the editor sends a single workspace folder corresponding to the workdir root. To explicitly configure no workspace folders, use WorkspaceFolders with no arguments.

type Runner 

type Runner struct {
	// Configuration
	DefaultModes             Mode                  // modes to run for each test
	Timeout                  time.Duration         // per-test timeout, if set
	PrintGoroutinesOnFailure bool                  // whether to dump goroutines on test failure
	SkipCleanup              bool                  // if set, don't delete test data directories when the test exits
	OptionsHook              func(*source.Options) // if set, use these options when creating gopls sessions
	// contains filtered or unexported fields
}

A Runner runs tests in gopls execution environments, as specified by its modes. For modes that share state (for example, a shared cache or common remote), any tests that execute on the same Runner will share the same state.

func (*Runner) Close 

func (r *Runner) Close() error

Close cleans up resource that have been allocated to this workspace.

func (*Runner) Run 

func (r *Runner) Run(t *testing.T, files string, test TestFunc, opts ...RunOption)

Run executes the test function in the default configured gopls execution modes. For each a test run, a new workspace is created containing the un-txtared files specified by filedata.

type Settings 

type Settings map[string]interface{}

Settings sets user-provided configuration for the LSP server.

As a special case, the env setting must not be provided via Settings: use EnvVars instead.

type State 

type State struct {
	// contains filtered or unexported fields
}

State encapsulates the server state TODO: explain more

func (State) String 

func (s State) String() string

This method, provided for debugging, accesses mutable fields without a lock, so it must not be called concurrent with any State mutation.

type TestFunc 

type TestFunc func(t *testing.T, env *Env)

type Verdict 

type Verdict int

A Verdict is the result of checking an expectation against the current editor state. 

const (
	// Met indicates that an expectation is satisfied by the current state.
	Met Verdict = iota
	// Unmet indicates that an expectation is not currently met, but could be met
	// in the future.
	Unmet
	// Unmeetable indicates that an expectation cannot be satisfied in the
	// future.
	Unmeetable
)

Order matters for the following constants: verdicts are sorted in order of decisiveness.

func (Verdict) String 

func (v Verdict) String() string

type WorkStatus added in v0.11.0 

type WorkStatus struct {
	// Last seen message from either `begin` or `report` progress.
	Msg string
	// Message sent with `end` progress message.
	EndMsg string
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.