fetch

Documentation

Overview

Package fetch provides a way to fetch modules from a proxy.

Copyright 2022 The Go Authors. All rights reserved. Use of this source code is governed by a BSD-style license that can be found in the LICENSE file.

Package fetch provides a way to fetch modules from a proxy.

Package fetch provides a way to fetch modules from a proxy.

Package fetch provides a way to fetch modules from a proxy.

Index

• Constants
• Variables
• func FSSignature(fsys fs.FS) (string, error)
• func GetInfo(ctx context.Context, modulePath, requestedVersion string, mg ModuleGetter) (_ *proxy.VersionInfo, err error)
• func LatestModuleVersions(ctx context.Context, modulePath string, prox *proxy.Client, ...) (info *internal.LatestModuleVersions, err error)
• func NewDirectoryModuleGetter(modulePath, dir string) (*directoryModuleGetter, error)
• func NewGoPackagesModuleGetter(ctx context.Context, dir string, patterns ...string) (*goPackagesModuleGetter, error)
• func NewGoPackagesStdlibModuleGetter(ctx context.Context, dir string) (*goPackagesModuleGetter, error)
• func NewModCacheGetter(dir string) (_ *modCacheModuleGetter, err error)
• func NewStdlibZipModuleGetter() *stdlibZipModuleGetter
• type BadPackageError
  • func (bpe *BadPackageError) Error() string
• type FetchResult
  • func FetchModule(ctx context.Context, modulePath, requestedVersion string, mg ModuleGetter) (fr *FetchResult)
• type LazyModule
  • func FetchLazyModule(ctx context.Context, modulePath, requestedVersion string, mg ModuleGetter) *LazyModule
  • func (lm *LazyModule) Unit(ctx context.Context, path string) (*internal.Unit, error)
• type ModuleGetter
  • func NewProxyModuleGetter(p *proxy.Client, s *source.Client) ModuleGetter
• type SearchableModuleGetter
• type VolatileModuleGetter

Constants

const (

	// MaxFileSize is the maximum filesize that is allowed for reading.
	// The fetch process should fail if it encounters a file exceeding
	// this limit.
	MaxFileSize = 30 * megabyte
)

Limits for discovery worker.

Variables

var (
	LocalVersion    = "v0.0.0"
	LocalCommitTime = time.Time{}
)

Version and commit time are pre specified when fetching a local module, as these fields are normally obtained from a proxy.

var ErrModuleContainsNoPackages = errors.New("module contains 0 packages")

var ZipSignatures = map[string][]internal.Modver{}/* 1895 elements not displayed */

Functions

func FSSignature

func FSSignature(fsys fs.FS) (string, error)

FSSignature calculates a signature that uniquely identifies a filesystem. It hashes every filename and its contents.

func GetInfo

func GetInfo(ctx context.Context, modulePath, requestedVersion string, mg ModuleGetter) (_ *proxy.VersionInfo, err error)

GetInfo returns the result of a request to the proxy .info endpoint. If the modulePath is "std", a request to @master will return an empty commit time.

func LatestModuleVersions

func LatestModuleVersions(ctx context.Context, modulePath string, prox *proxy.Client, hasGoMod func(v string) (bool, error)) (info *internal.LatestModuleVersions, err error)

LatestModuleVersions uses the proxy to get information about the latest versions of modulePath. It returns a LatestModuleVersions whose RawVersion and CookedVersion is obtained from the proxy @v/list and @latest endpoints. The cooked version is computed by choosing the latest version after removing versions that are retracted in the go.mod file of the raw version.

The GoodVersion of LatestModuleVersions is not set. It should be determined when inserting into a data source, since it depends on the contents of the data source.

The hasGoMod function that is passed in should check if version v of the module has a go.mod file, using a source other than the proxy (e.g. a database). If it doesn't have enough information to decide, it should return an error that wraps derrors.NotFound.

If a module has no tagged versions and hasn't been accessed at a pseudo-version in a while, then the proxy's list endpoint will serve nothing and its @latest endpoint will return a 404/410. (Example: cloud.google.com/go/compute/metadata, which has a v0.0.0-20181107005212-dafb9c8d8707 that @latest does not return.) That is not a failure, but a valid state in which there is no version information for a module, even though particular pseudo-versions of the module might exist. In this case, LatestModuleVersions returns (nil, nil).

As a special case, the "std" module's versions are fetched from the repo (by calling stdlib.Versions). We assume stdlib versions are never retracted, and that there are no incompatible versions.

func NewDirectoryModuleGetter

func NewDirectoryModuleGetter(modulePath, dir string) (*directoryModuleGetter, error)

NewDirectoryModuleGetter returns a ModuleGetter for reading a module from a directory.

func NewGoPackagesModuleGetter

func NewGoPackagesModuleGetter(ctx context.Context, dir string, patterns ...string) (*goPackagesModuleGetter, error)

NewGoPackagesModuleGetter returns a ModuleGetter that loads packages using go/packages.Load(pattern), from the requested directory.

func NewGoPackagesStdlibModuleGetter

func NewGoPackagesStdlibModuleGetter(ctx context.Context, dir string) (*goPackagesModuleGetter, error)

NewGoPackagesStdlibModuleGetter returns a ModuleGetter that loads stdlib packages using go/packages.Load, from the requested GOROOT.

func NewModCacheGetter

func NewModCacheGetter(dir string) (_ *modCacheModuleGetter, err error)

NewModCacheGetter returns a ModuleGetter that reads modules from a filesystem directory organized like the proxy. If allowed is non-empty, only module@versions in allowed are served; others result in NotFound errors.

func NewStdlibZipModuleGetter

func NewStdlibZipModuleGetter() *stdlibZipModuleGetter

NewStdlibZipModuleGetter returns a ModuleGetter that loads stdlib packages using stdlib zip files.

Types

type BadPackageError

type BadPackageError struct {
	Err error // Not nil.
}

BadPackageError represents an error loading a package because its contents do not make up a valid package.

This can happen, for example, if the .go files fail to parse or declare different package names.

func (*BadPackageError) Error

func (bpe *BadPackageError) Error() string

type FetchResult

type FetchResult struct {
	ModulePath       string
	RequestedVersion string
	ResolvedVersion  string
	// HasGoMod says whether the zip contain a go.mod file. If Module (below) is non-nil, then
	// Module.HasGoMod will be the same value. But HasGoMod will be populated even if Module is nil
	// because there were problems with it, as long as we can download and read the zip.
	HasGoMod             bool
	GoModPath            string
	Status               int
	Error                error
	Module               *internal.Module
	PackageVersionStates []*internal.PackageVersionState
}

func FetchModule

func FetchModule(ctx context.Context, modulePath, requestedVersion string, mg ModuleGetter) (fr *FetchResult)

FetchModule queries the proxy or the Go repo for the requested module version, downloads the module zip, and processes the contents to return an *internal.Module and related information.

Even if err is non-nil, the result may contain useful information, like the go.mod path.

type LazyModule

type LazyModule struct {
	internal.ModuleInfo
	UnitMetas []*internal.UnitMeta

	Error error
	// contains filtered or unexported fields
}

A LazyModule contains the information needed to compute a FetchResult, but has only done enough work to compute the UnitMetas in the module. It provides a Unit method to compute a single unit or a fetchResult method to compute the whole FetchResult.

func FetchLazyModule

func FetchLazyModule(ctx context.Context, modulePath, requestedVersion string, mg ModuleGetter) *LazyModule

FetchLazyModule queries the proxy or the Go repo for the requested module version, downloads the module zip, and does just enough processing to produce UnitMetas for all the modules. The full units are computed as needed.

func (*LazyModule) Unit

func (lm *LazyModule) Unit(ctx context.Context, path string) (*internal.Unit, error)

type ModuleGetter

type ModuleGetter interface {
	// Info returns basic information about the module.
	Info(ctx context.Context, path, version string) (*proxy.VersionInfo, error)

	// Mod returns the contents of the module's go.mod file.
	Mod(ctx context.Context, path, version string) ([]byte, error)

	// ContentDir returns an FS for the module's contents. The FS should match the
	// format of a module zip file's content directory. That is the
	// "<module>@<resolvedVersion>" directory that all module zips are expected
	// to have according to the zip archive layout specification at
	// https://golang.org/ref/mod#zip-files.
	ContentDir(ctx context.Context, path, version string) (fs.FS, error)

	// SourceInfo returns information about where to find a module's repo and
	// source files.
	SourceInfo(ctx context.Context, path, version string) (*source.Info, error)

	// SourceFS returns the path to serve the files of the modules loaded by
	// this ModuleGetter, and an FS that can be used to read the files. The
	// returned values are intended to be passed to
	// internal/frontend.Server.InstallFiles.
	SourceFS() (string, fs.FS)

	// String returns a representation of the getter for testing and debugging.
	String() string
}

ModuleGetter gets module data.

func NewProxyModuleGetter

func NewProxyModuleGetter(p *proxy.Client, s *source.Client) ModuleGetter

type SearchableModuleGetter

type SearchableModuleGetter interface {
	// Search searches for packages matching the given query, returning at most
	// limit results.
	Search(ctx context.Context, q string, limit int) ([]*internal.SearchResult, error)
}

SearchableModuleGetter is an additional interface that may be implemented by ModuleGetters to support search.

type VolatileModuleGetter

type VolatileModuleGetter interface {
	// HasChanged reports whether the referenced module has changed.
	HasChanged(context.Context, internal.ModuleInfo) (bool, error)
}

VolatileModuleGetter is an additional interface that may be implemented by ModuleGetters to support invalidating content.