datasource

package
v0.3.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 21, 2025 License: Apache-2.0 Imports: 40 Imported by: 2

Documentation

Overview

Package datasource provides clients to fetch data from different APIs.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func MakeMavenAuth 

func MakeMavenAuth(globalSettings, userSettings MavenSettingsXML) map[string]*HTTPAuthentication

MakeMavenAuth returns a map of Maven authentication information index by repository ID.

func NewMavenDecoder 

func NewMavenDecoder(reader io.Reader) *xml.Decoder

NewMavenDecoder returns an xml decoder with CharsetReader and Entity set.

Types

type CachedInsightsClient 

type CachedInsightsClient struct {
	pb.InsightsClient
	// contains filtered or unexported fields
}

CachedInsightsClient is a wrapper for InsightsClient that caches requests.

func NewCachedInsightsClient 

func NewCachedInsightsClient(addr string, userAgent string) (*CachedInsightsClient, error)

NewCachedInsightsClient creates a CachedInsightsClient.

func (*CachedInsightsClient) GetPackage 

func (c *CachedInsightsClient) GetPackage(ctx context.Context, in *pb.GetPackageRequest, opts ...grpc.CallOption) (*pb.Package, error)

GetPackage returns metadata about a package by querying deps.dev API.

func (*CachedInsightsClient) GetRequirements 

func (c *CachedInsightsClient) GetRequirements(ctx context.Context, in *pb.GetRequirementsRequest, opts ...grpc.CallOption) (*pb.Requirements, error)

GetRequirements returns requirements of the given version by querying deps.dev API.

func (*CachedInsightsClient) GetVersion 

func (c *CachedInsightsClient) GetVersion(ctx context.Context, in *pb.GetVersionRequest, opts ...grpc.CallOption) (*pb.Version, error)

GetVersion returns metadata about a version by querying deps.dev API.

func (*CachedInsightsClient) GobDecode 

func (c *CachedInsightsClient) GobDecode(b []byte) error

GobDecode decodes bytes to cache.

func (*CachedInsightsClient) GobEncode 

func (c *CachedInsightsClient) GobEncode() ([]byte, error)

GobEncode encodes cache to bytes.

type HTTPAuthMethod 

type HTTPAuthMethod int

HTTPAuthMethod definesthe type of HTTP authentication method. 

const (
	AuthBasic HTTPAuthMethod = iota
	AuthBearer
	AuthDigest
)

HTTP authentication method.

type HTTPAuthentication 

type HTTPAuthentication struct {
	SupportedMethods []HTTPAuthMethod // In order of preference, only one method will be attempted.

	// AlwaysAuth determines whether to always send auth headers.
	// If false, the server must respond with a WWW-Authenticate header which will be checked for supported methods.
	// Must be set to false to use Digest authentication.
	AlwaysAuth bool

	// Shared
	Username string // Basic & Digest, plain text.
	Password string // Basic & Digest, plain text.
	// Basic
	BasicAuth string // Base64-encoded username:password. Overrides Username & Password fields if set.
	// Bearer
	BearerToken string
	// Digest
	CnonceFunc func() string // Function used to generate cnonce string for Digest. OK to leave unassigned. Mostly for use in tests.
	// contains filtered or unexported fields
}

HTTPAuthentication holds the information needed for general HTTP Authentication support. Requests made through this will automatically populate the relevant info in the Authorization headers. This is a general implementation and should be suitable for use with any ecosystem.

func (*HTTPAuthentication) Get 

func (auth *HTTPAuthentication) Get(ctx context.Context, httpClient *http.Client, url string) (*http.Response, error)

Get makes an http GET request with the given http.Client. The Authorization Header will automatically be populated according from the fields in the HTTPAuthentication.

type MavenRegistry 

type MavenRegistry struct {
	URL    string
	Parsed *url.URL

	// Information from pom.xml
	ID               string
	ReleasesEnabled  bool
	SnapshotsEnabled bool
}

MavenRegistry defines a Maven registry.

type MavenRegistryAPIClient 

type MavenRegistryAPIClient struct {
	// contains filtered or unexported fields
}

MavenRegistryAPIClient defines a client to fetch metadata from a Maven registry.

func NewDefaultMavenRegistryAPIClient added in v0.3.5 

func NewDefaultMavenRegistryAPIClient(ctx context.Context, registry string) (*MavenRegistryAPIClient, error)

NewDefaultMavenRegistryAPIClient creates a new MavenRegistryAPIClient with default settings, using the provided registry URL.

func NewMavenRegistryAPIClient 

func NewMavenRegistryAPIClient(ctx context.Context, registry MavenRegistry, localRegistry string, disableGoogleClient bool) (*MavenRegistryAPIClient, error)

NewMavenRegistryAPIClient returns a new MavenRegistryAPIClient.

func (*MavenRegistryAPIClient) AddRegistry 

func (m *MavenRegistryAPIClient) AddRegistry(ctx context.Context, registry MavenRegistry) error

AddRegistry adds the given registry to the list of registries if it has not been added.

func (*MavenRegistryAPIClient) DisableGoogleAuth added in v0.3.6 

func (m *MavenRegistryAPIClient) DisableGoogleAuth()

DisableGoogleAuth prevents the creation of a Google client for authentication purpose.

func (*MavenRegistryAPIClient) GetProject 

func (m *MavenRegistryAPIClient) GetProject(ctx context.Context, groupID, artifactID, version string) (maven.Project, error)

GetProject fetches a pom.xml specified by groupID, artifactID and version and parses it to maven.Project. Each registry in the list is tried until we find the project. For a snapshot version, version level metadata is used to find the extact version string. More about Maven Repository Metadata Model: https://maven.apache.org/ref/3.9.9/maven-repository-metadata/ More about Maven Metadata: https://maven.apache.org/repositories/metadata.html

func (*MavenRegistryAPIClient) GetRegistries 

func (m *MavenRegistryAPIClient) GetRegistries() (registries []MavenRegistry)

GetRegistries returns the registries added to this client.

func (*MavenRegistryAPIClient) GetVersions 

func (m *MavenRegistryAPIClient) GetVersions(ctx context.Context, groupID, artifactID string) ([]maven.String, error)

GetVersions returns the list of available versions of a Maven package specified by groupID and artifactID. Versions found in all registries are unioned, then sorted by semver.

func (*MavenRegistryAPIClient) GobDecode 

func (m *MavenRegistryAPIClient) GobDecode(b []byte) error

GobDecode encodes bytes to cache.

func (*MavenRegistryAPIClient) GobEncode 

func (m *MavenRegistryAPIClient) GobEncode() ([]byte, error)

GobEncode encodes cache to bytes.

func (*MavenRegistryAPIClient) SetLocalRegistry added in v0.2.1 

func (m *MavenRegistryAPIClient) SetLocalRegistry(localRegistry string)

SetLocalRegistry sets the local directory that stores the downloaded Maven manifests.

func (*MavenRegistryAPIClient) WithoutRegistries 

func (m *MavenRegistryAPIClient) WithoutRegistries() *MavenRegistryAPIClient

WithoutRegistries makes MavenRegistryAPIClient including its cache but not registries.

type MavenSettingsXML 

type MavenSettingsXML struct {
	Servers []MavenSettingsXMLServer `xml:"servers>server"`
}

MavenSettingsXML defines Maven settings.xml.

func ParseMavenSettings 

func ParseMavenSettings(path string) MavenSettingsXML

ParseMavenSettings parses Maven settings at the given path.

type MavenSettingsXMLServer 

type MavenSettingsXMLServer struct {
	ID       string `xml:"id"`
	Username string `xml:"username"`
	Password string `xml:"password"`
}

MavenSettingsXMLServer defines a Maven server in settings.xml.

type NPMRegistryAPIClient added in v0.2.0 

type NPMRegistryAPIClient struct {
	// contains filtered or unexported fields
}

NPMRegistryAPIClient defines a client to fetch metadata from a NPM registry.

func NewNPMRegistryAPIClient added in v0.2.0 

func NewNPMRegistryAPIClient(projectDir string) (*NPMRegistryAPIClient, error)

NewNPMRegistryAPIClient returns a new NPMRegistryAPIClient. projectDir is the directory (on disk) to read the project-level .npmrc config file from (for registries).

func (*NPMRegistryAPIClient) Dependencies added in v0.2.0 

func (c *NPMRegistryAPIClient) Dependencies(ctx context.Context, pkg, version string) (NPMRegistryDependencies, error)

Dependencies returns all the defined dependencies of the given version of an npm package

func (*NPMRegistryAPIClient) FullJSON added in v0.2.0 

func (c *NPMRegistryAPIClient) FullJSON(ctx context.Context, pkg, version string) (gjson.Result, error)

FullJSON returns the entire npm registry JSON data for a given package version

func (*NPMRegistryAPIClient) GobDecode added in v0.2.0 

func (c *NPMRegistryAPIClient) GobDecode(b []byte) error

GobDecode decodes the cache from bytes.

func (*NPMRegistryAPIClient) GobEncode added in v0.2.0 

func (c *NPMRegistryAPIClient) GobEncode() ([]byte, error)

GobEncode encodes the cache to bytes.

func (*NPMRegistryAPIClient) Versions added in v0.2.0 

func (c *NPMRegistryAPIClient) Versions(ctx context.Context, pkg string) (NPMRegistryVersions, error)

Versions returns all the known versions and tags of a given npm package

type NPMRegistryAuths added in v0.2.0 

type NPMRegistryAuths map[string]*HTTPAuthentication

NPMRegistryAuths handles npm registry authentication in a manner similar to npm-registry-fetch https://github.com/npm/npm-registry-fetch/blob/237d33b45396caa00add61e0549cf09fbf9deb4f/lib/auth.js

func (NPMRegistryAuths) GetAuth added in v0.2.0 

func (auths NPMRegistryAuths) GetAuth(uri string) *HTTPAuthentication

GetAuth returns the HTTPAuthentication for the given URI. This is similar to npm-registry-fetch's getAuth function.

type NPMRegistryConfig added in v0.2.0 

type NPMRegistryConfig struct {
	ScopeURLs map[string]string // map of @scope to registry URL
	Auths     NPMRegistryAuths  // auth info per npm registry URI
}

NPMRegistryConfig holds the registry URL configuration from npm config.

func LoadNPMRegistryConfig added in v0.2.0 

func LoadNPMRegistryConfig(projectDir string) (NPMRegistryConfig, error)

LoadNPMRegistryConfig loads the npmrc file from the project directory and parses it.

func ParseNPMRegistryInfo added in v0.2.0 

func ParseNPMRegistryInfo(npmrc NpmrcConfig) NPMRegistryConfig

ParseNPMRegistryInfo parses the npmrc config into a NPMRegistryConfig.

func (NPMRegistryConfig) MakeRequest added in v0.2.0 

func (r NPMRegistryConfig) MakeRequest(ctx context.Context, httpClient *http.Client, urlComponents ...string) (*http.Response, error)

MakeRequest makes the http request to the corresponding npm registry api (with auth). urlComponents should be (package) or (package, version)

type NPMRegistryDependencies added in v0.2.0 

type NPMRegistryDependencies struct {
	Dependencies         map[string]string
	DevDependencies      map[string]string
	PeerDependencies     map[string]string
	OptionalDependencies map[string]string
	BundleDependencies   []string
}

NPMRegistryDependencies holds the dependencies of a package version, from the npm API.

type NPMRegistryVersions added in v0.2.0 

type NPMRegistryVersions struct {
	Versions []string
	Tags     map[string]string
}

NPMRegistryVersions holds the versions and tags of a package, from the npm API.

type NpmrcConfig added in v0.2.0 

type NpmrcConfig map[string]string

NpmrcConfig is the parsed npmrc config map.

type PyPIRegistryAPIClient added in v0.1.8 

type PyPIRegistryAPIClient struct {
	// contains filtered or unexported fields
}

PyPIRegistryAPIClient defines a client to fetch metadata from a PyPI registry. TODO(#541): support multiple registries and authentication

func NewPyPIRegistryAPIClient added in v0.1.8 

func NewPyPIRegistryAPIClient(registry string, localRegistry string) *PyPIRegistryAPIClient

NewPyPIRegistryAPIClient returns a new PyPIRegistryAPIClient.

func (*PyPIRegistryAPIClient) GetFile added in v0.3.1 

func (p *PyPIRegistryAPIClient) GetFile(ctx context.Context, url string) ([]byte, error)

GetFile retrieves the content of a file from the registry at the given URL.

func (*PyPIRegistryAPIClient) GetIndex added in v0.2.0 

func (p *PyPIRegistryAPIClient) GetIndex(ctx context.Context, project string) (pypi.IndexResponse, error)

GetIndex queries the simple API index for a given project.

func (*PyPIRegistryAPIClient) GobDecode added in v0.1.8 

func (p *PyPIRegistryAPIClient) GobDecode(b []byte) error

GobDecode encodes bytes to cache.

func (*PyPIRegistryAPIClient) GobEncode added in v0.1.8 

func (p *PyPIRegistryAPIClient) GobEncode() ([]byte, error)

GobEncode encodes cache to bytes.

func (*PyPIRegistryAPIClient) SetLocalRegistry added in v0.3.2 

func (p *PyPIRegistryAPIClient) SetLocalRegistry(localRegistry string)

SetLocalRegistry sets the local directory that stores the downloaded PyPI manifests.

type RequestCache 

type RequestCache[K comparable, V any] struct {
	// contains filtered or unexported fields
}

RequestCache is a map to cache the results of expensive functions that are called concurrently.

func NewRequestCache 

func NewRequestCache[K comparable, V any]() *RequestCache[K, V]

NewRequestCache creates a new RequestCache.

func (*RequestCache[K, V]) Get 

func (rq *RequestCache[K, V]) Get(key K, fn func() (V, error)) (V, error)

Get gets the value from the cache map if it's cached, otherwise it will call fn to get the value and cache it. fn will only ever be called once for a key, even if there are multiple simultaneous calls to Get before the first call is finished.

func (*RequestCache[K, V]) GetMap 

func (rq *RequestCache[K, V]) GetMap() map[K]V

GetMap gets a shallow clone of the stored cache map.

func (*RequestCache[K, V]) SetMap 

func (rq *RequestCache[K, V]) SetMap(m map[K]V)

SetMap loads (a shallow clone of) the provided map into the cache map.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.