exturl

package module
v0.4.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 28, 2024 License: Apache-2.0 Imports: 28 Imported by: 32

README

exturl

License Go Reference Go Report Card

Go beyond http:// with this advanced Go URL library.

Simply put, exturl allows you to get a Go Reader from a wide variety of URL types, including specific entries in archives using a URL structure inspired by Java's JarURLConnection.

Rationale

  1. Do you have a program or API server that needs to read from a file? If there is no strong reason for the file to be local, then exturl allows you to give the user the option of providing a URL to the file instead of a local path. Exturl will let you stream just the data you need, e.g. a single entry in a remote tarball, avoiding local caching whenever possible.

  2. Does that file reference other files in relation to its location? Such embedded relative paths are often challenging to resolve if the "current location" is remote or inside an archive (or inside a remote archive!). This complexit is one reason why some implementations insist on only supporting local filesystem. Exturl does efficient relative path resolution for all its supported URL schemes (even remote archives!), again making it easy for your program to accept URLs.

Features

Especially powerful is the ability to refer to entries in remote archives, e.g. a zip file over http. Where possible exturl will stream the data (e.g. remote tarballs), but if filesystem access is required (for remote zip, git repository clones, and Docker images) it will download them to a temporary local location. The use of a shared context allows for optimization, e.g. a remote zip file will not be downloaded again if it was already downloaded in the context. Examples:

tar:http://mysite.org/cloud.tar.gz!main.yaml
git:https://github.com/tliron/puccini.git!examples/openstack/hello-world.yaml

Another powerful feature is support for relative URL resolution using common filesystem-type paths, which includes usage of .. and .. All URL types support this: file URLs, local and remote zip URLs, etc. Use url.Relative().

You can also ensure that a URL is valid (e.g. remote location is available, tarball entry exists, etc.) before attempting to read from it (which may trigger a download) or passing it to other parts of your program. To do so, use NewValidURL() instead of NewURL(). NewValidURL() also supports relative URLs tested against a list of potential bases. Compare with how the PATH environment variable is used by the OS to find commands.

Also supported are URLs for in-memory data using a special internal: scheme. This allows you to have a unified API for accessing data, whether it's available externally or created internally by your program.

Finally, there are tools for mocking and testing, e.g. a MockURL type that can mimic any scheme with arbitrary data, and there is support for custom URL transformation functions within a context, including straightforward mapping of URLs to other URLs. For example, you can map a http: URL to a file: or internal: URL.

Example

import (
    "context"
    "github.com/tliron/exturl"
)

func ReadAll(url string) ([]byte, error) {
    urlContext := exturl.NewContext()
    defer urlContext.Release()

    url_ := urlContext.NewURL(url)
    if reader, err := url_.Open(context.TODO()); err == nil {
        defer reader.Close()
        return io.ReadAll(reader)
    } else {
        return nil, err
    }
}

Supported URL Types

http: and https:

Uses standard Go access libraries (net/http). Open() is an HTTP GET verb.

file:

An absolute path to the local filesystem.

The URL must begin with two slashes. If a hostname is present before the path it will be ignored by exturl, so this:

file://localhost/the/path

is equivalent to this:

file:///the/path

Because the path must be absolute, it always begins with a slash. The consequence is that most file: URLs begin with 3 slashes.

When compiled for Windows the URL path will be converted to a Windows path. The convention is that backslashes become slashes and a first slash is added to make it absolute. So this URL:

file:///C:/Windows/win.ini

is equivalent to this path:

C:\Windows\win.ini

Note that for security reasons relative file URLs are not tested against the current working directory (pwd) by default. This is unlike OS services, such as Go's os.Open(). If you do want to support the working directory then call NewWorkingDirFileURL() and add it explicitly to the bases of NewValidURL().

It is often desirable to accept input that is either a URL or a file path. For this use case NewAnyOrFileURL() and NewValidAnyOrFileURL() are provided. If the argument is not a parsable URL it will be treated as a file path and a *FileURL will be returned.

The functions' design may trip over a rare edge case for Windows. If there happens to be a drive that has the same name as a supported URL scheme, e.g. "http", then callers would have to provide a full file URL, otherwise it will be parsed as a URL of that scheme. E.g. instead of:

http:\Dir\file

you must use:

file:///http:/Dir/file
tar:

Entries in tarballs. .tar and .tar.gz (or .tgz) are supported. The archive URL can be any full exturl URL or a local filesystem path. Examples:

tar:http://mysite.org/cloud.tar.gz!path/to/main.yaml
tar:file:///local/path/cloud.tar.gz!path/to/main.yaml
tar:/local/path/cloud.tar.gz!path/to/main.yaml

Note that tarballs are serial containers optimized for streaming, meaning that, when read, unwanted entries will be skipped until our entry is found, and then subsequent entries will be ignored. This means that when accessing tarballs over the network the tarball does not have to be downloaded in its entirety, unlike with zip (see below).

Gzip decompression uses klauspost's pgzip library.

zip:

Entries in zip files. The archive URL can be any full exturl URL or a local filesystem path. Example:

zip:http://mysite.org/cloud.tar.gz!path/to/main.yaml

Note that zip files require random file access and thus must be on the local file system. Consequently for remote zips the entire archive must be downloaded in order to access one entry. Thus, if you have a choice of compression technologies and want good remote support, zip should be avoided. In any case, exturl will optimize by making sure to download the zip only once per context.

Uses klauspost's compress library.

git:

Files in git repositories. The repository URL is not an exturl URL, but rather must be URLs supported by go-git. Example:

git:https://github.com/tliron/puccini.git!examples/openstack/hello-world.yaml

You can specify a reference (tag, branch tip, or commit hash) in the URL fragment, e.g.:

git:https://github.com/tliron/puccini.git#main!examples/openstack/hello-world.yaml

Because we are only interested in reading files, not making commits, exturl will optimize by performing a shallow clone (depth=1) of only the requested reference.

docker:

Images on OCI/Docker registries. The URL structure is docker://HOSTNAME/[NAMESPACE/]REPOSITORY[:TAG]. The tag will default to "latest". Example:

docker://docker.io/tliron/prudence:latest

The url.Open() API will decode the first layer (a .tar.gz) it finds in the image. The intended use case is using OCI registries to store arbitrary data. In the future we may support more elaborate use cases.

Uses go-containerregistry.

internal:

Internal URLs can be stored globally so that all contexts are able to access them.

Supported APIs for global internal URLs are RegisterInternalURL() (which will fail if the URL has already been registered), DeregisterInternalURL(), UpdateInternalURL() (which will always succeed), ReadToInternalURL(), ReadToInternalURLFromStdin().

It is also possible to create ad-hoc internal URLs using NewInternalURL() and then url.SetContent(). These are not stored globally.

Content can be []byte or an implementation of the InternalURLProvider interface. Other types will be converted to strings and then to []byte.

Mock URLs

These are intended to be used for testing. They must be created explicitly via NewMockURL() and can use any scheme. They are not created by NewURL().

Documentation

Index

Constants

View Source 
const PathSeparator = string(filepath.Separator)

Variables

View Source 
var TARBALL_ARCHIVE_FORMATS = []string{"tar", "tar.gz"}

Functions

func DeleteTemporaryDir 

func DeleteTemporaryDir(path string) error

func DeleteTemporaryFile 

func DeleteTemporaryFile(path string) error

func DeregisterInternalURL 

func DeregisterInternalURL(path string)

Deletes registers content for InternalURL or does nothing if the content is not registered.

func Download 

func Download(context contextpkg.Context, url URL, temporaryPathPattern string) (*os.File, error)

func DownloadTo 

func DownloadTo(context contextpkg.Context, url URL, path string) error

func GetFormat 

func GetFormat(path string) string

func GetPath 

func GetPath(url URL) (string, error)

func GetTemporaryPathPattern added in v0.2.4 

func GetTemporaryPathPattern(key string) string

func IsNotFound 

func IsNotFound(err error) bool

func IsNotImplemented added in v0.2.8 

func IsNotImplemented(err error) bool

func IsValidTarballArchiveFormat 

func IsValidTarballArchiveFormat(archiveFormat string) bool

func ReadBytes 

func ReadBytes(context contextpkg.Context, url URL) ([]byte, error)

func ReadString 

func ReadString(context contextpkg.Context, url URL) (string, error)

func ReadToInternalURLsFromFS added in v0.2.7 

func ReadToInternalURLsFromFS(context contextpkg.Context, fs fspkg.FS, root string, translatePath func(path string) (string, bool)) error

Walks a io/fs.FS starting at "root" and registers its files' content for InternalURL. "root" can be an emptry string to read the entire FS.

The optional argument "translatePath" is a function that can translate the FS path to an internal URL path and can also filter out entries when it returns false. If the argument is not provided all files will be read with their paths as is.

If "fs" is an embed.FS, this function is optimized to handle it more efficiently via calls to embed.FS.ReadFile.

Will return an error if there is already content registered at an internal path.

func RegisterInternalURL 

func RegisterInternalURL(path string, content any) error

Registers content for InternalURL.

"content" can be []byte or an InternalURLProvider. Other types will be converted to string and then to []byte.

Will return an error if there is already content registered at the path. For a version that always succeeds, use UpdateInternalURL.

func Size 

func Size(context contextpkg.Context, url URL) (int64, error)

func ToNetURL 

func ToNetURL(url URL) (*neturlpkg.URL, error)

func URLPathToFilePath added in v0.2.1 

func URLPathToFilePath(path string) string

func UpdateInternalURL 

func UpdateInternalURL(path string, content any)

Updates registered content for InternalURL or registers it if not yet registered.

"content" can be []byte or an InternalURLProvider. Other types will be converted to string and then to []byte.

Types

type Context 

type Context struct {
	// contains filtered or unexported fields
}

func NewContext 

func NewContext() *Context

func (*Context) AddTransformer added in v0.4.0 

func (self *Context) AddTransformer(transformer URLTransformerFunc)

func (*Context) GetCredentials 

func (self *Context) GetCredentials(host string) *Credentials

Not thread-safe

func (*Context) GetHTTPRoundTripper 

func (self *Context) GetHTTPRoundTripper(host string) http.RoundTripper

Not thread-safe

func (*Context) GetLocalPath 

func (self *Context) GetLocalPath(context contextpkg.Context, url URL) (string, error)

Will download the file to the local temporary directory if not already locally available

func (*Context) GetMapping 

func (self *Context) GetMapping(fromUrl string) (string, bool)

URLTransformerFunc signature

func (*Context) Map 

func (self *Context) Map(fromUrl string, toUrl string)

Set toUrl to empty string to delete the mapping.

func (*Context) NewAnyOrFileURL added in v0.3.1 

func (self *Context) NewAnyOrFileURL(urlOrPath string) URL

Parses the argument as either an absolute URL or a file path.

In essence attempts to parse the URL via Context.NewURL and if that fails treats the URL as a file path and returns a *FileURL.

To support relative URLs, see Context.NewValidAnyOrFileURL.

On Windows note that if there happens to be a drive that has the same name as a supported URL scheme (e.g. "http") then callers would have to provide a full file URL, e.g. instead of "http:\Dir\file" provide "file:///http:/Dir/file", otherwise it will be parsed as a URL of that scheme.

func (*Context) NewDockerURL added in v0.2.0 

func (self *Context) NewDockerURL(neturl *neturlpkg.URL) *DockerURL

func (*Context) NewFileURL added in v0.2.0 

func (self *Context) NewFileURL(filePath string) *FileURL

Note that the argument is treated as an OS file path (using backslashes on Windows). The path must be absolute.

Directories must be suffixed with an OS path separator.

func (*Context) NewGitURL added in v0.2.0 

func (self *Context) NewGitURL(path string, repositoryUrl string) *GitURL

func (*Context) NewInternalURL added in v0.2.0 

func (self *Context) NewInternalURL(path string) *InternalURL

func (*Context) NewMockURL added in v0.2.9 

func (self *Context) NewMockURL(scheme string, path string, content any) *MockURL

"content" can be []byte or an InternalURLProvider. Other types will be converted to string and then to []byte.

func (*Context) NewNetworkURL added in v0.2.0 

func (self *Context) NewNetworkURL(neturl *neturlpkg.URL) *NetworkURL

func (*Context) NewURL added in v0.2.0 

func (self *Context) NewURL(url string) (URL, error)

Parses the argument as an absolute URL.

To support relative URLs, see Context.NewValidURL.

If you are expecting either a URL or a file path, consider Context.NewAnyOrFileURL.

func (*Context) NewValidAnyOrFileURL added in v0.3.1 

func (self *Context) NewValidAnyOrFileURL(context contextpkg.Context, urlOrPath string, bases []URL) (URL, error)

Parses the argument as an absolute URL or an absolute file path or a relative path. Relative paths support ".." and ".", with the returned URL path always being absolute.

The returned URL is "valid", meaning that during this call it was possible to call Open on it. Of course this can't guarantee that future calls to Open will succeed.

Relative URLs are tested against the "bases" argument in order. The first valid URL will be returned and the remaining bases will be ignored. Note that bases can be any of any URL type.

func (*Context) NewValidDockerURL added in v0.2.0 

func (self *Context) NewValidDockerURL(neturl *neturlpkg.URL) (*DockerURL, error)

func (*Context) NewValidFileURL added in v0.2.0 

func (self *Context) NewValidFileURL(filePath string) (*FileURL, error)

Note that the argument is treated as an OS file path (using backslashes on Windows). The path must be absolute.

If the path is a directory, it will automatically be suffixed with an OS path separator if it doesn't already have one.

func (*Context) NewValidGitURL added in v0.2.0 

func (self *Context) NewValidGitURL(path string, repositoryUrl string) (*GitURL, error)

func (*Context) NewValidInternalURL added in v0.2.0 

func (self *Context) NewValidInternalURL(path string) (*InternalURL, error)

func (*Context) NewValidNetworkURL added in v0.2.0 

func (self *Context) NewValidNetworkURL(neturl *neturlpkg.URL) (*NetworkURL, error)

func (*Context) NewValidURL added in v0.2.0 

func (self *Context) NewValidURL(context contextpkg.Context, urlOrPath string, bases []URL) (URL, error)

Parses the argument as either an absolute URL or a relative path. Relative paths support ".." and ".", with the returned URL path always being absolute.

The returned URL is "valid", meaning that during this call it was possible to call Open on it. Of course this can't guarantee that future calls to Open will succeed.

Relative URLs are tested against the "bases" argument in order. The first valid URL will be returned and the remaining bases will be ignored. Note that bases can be any of any URL type.

If you are expecting either a URL or a file path, consider Context.NewValidAnyOrFileURL.

func (*Context) NewWorkingDirFileURL added in v0.3.0 

func (self *Context) NewWorkingDirFileURL() (*FileURL, error)

A valid URL for the working directory.

func (*Context) OpenFile 

func (self *Context) OpenFile(context contextpkg.Context, url URL) (*os.File, error)

func (*Context) ParseGitURL added in v0.2.0 

func (self *Context) ParseGitURL(url string) (*GitURL, error)

func (*Context) ParseTarballURL added in v0.2.0 

func (self *Context) ParseTarballURL(url string) (*TarballURL, error)

func (*Context) ParseValidGitURL added in v0.2.0 

func (self *Context) ParseValidGitURL(url string) (*GitURL, error)

func (*Context) ParseValidTarballURL added in v0.2.0 

func (self *Context) ParseValidTarballURL(context contextpkg.Context, url string) (*TarballURL, error)

func (*Context) ParseValidZipURL added in v0.2.0 

func (self *Context) ParseValidZipURL(context contextpkg.Context, url string) (*ZipURL, error)

func (*Context) ParseZipURL added in v0.2.0 

func (self *Context) ParseZipURL(url string) (*ZipURL, error)

func (*Context) ReadToInternalURL added in v0.2.0 

func (self *Context) ReadToInternalURL(path string, reader io.Reader) (*InternalURL, error)

Registers content for InternalURL from an io.Reader.

Will automatically close "reader" if it supports io.Closer.

Will return an error if there is already content registered at the path.

func (*Context) ReadToInternalURLFromStdin added in v0.2.0 

func (self *Context) ReadToInternalURLFromStdin(context contextpkg.Context, format string) (*InternalURL, error)

Registers content for InternalURL from os.Stdin. "format" can be an empty string.

The URL will have a globally unique path in the form of "/stdin/GUID[.FORMAT]". The "format" extension is used to support URL.Format.

func (*Context) Release 

func (self *Context) Release() error

func (*Context) SetCredentials 

func (self *Context) SetCredentials(host string, username string, password string, token string)

Not thread-safe

func (*Context) SetHTTPRoundTripper 

func (self *Context) SetHTTPRoundTripper(host string, httpRoundTripper http.RoundTripper)

Not thread-safe

func (*Context) Transform added in v0.4.0 

func (self *Context) Transform(fromUrl string) (string, bool)

type Credentials 

type Credentials struct {
	Username string
	Password string
	Token    string
}

type DockerURL 

type DockerURL struct {
	URL *neturlpkg.URL
	// contains filtered or unexported fields
}

func (*DockerURL) Base added in v0.4.0 

func (self *DockerURL) Base() URL

(URL interface)

func (*DockerURL) Context 

func (self *DockerURL) Context() *Context

(URL interface)

func (*DockerURL) Format 

func (self *DockerURL) Format() string

(URL interface)

func (*DockerURL) Key 

func (self *DockerURL) Key() string

(URL interface)

func (*DockerURL) Open 

func (self *DockerURL) Open(context contextpkg.Context) (io.ReadCloser, error)

(URL interface)

func (*DockerURL) Relative 

func (self *DockerURL) Relative(path string) URL

(URL interface)

func (*DockerURL) RemoteOptions 

func (self *DockerURL) RemoteOptions(context contextpkg.Context) []remote.Option

func (*DockerURL) String 

func (self *DockerURL) String() string

(fmt.Stringer interface)

func (*DockerURL) ValidRelative added in v0.4.0 

func (self *DockerURL) ValidRelative(context contextpkg.Context, path string) (URL, error)

(URL interface)

func (*DockerURL) WriteFirstLayer added in v0.2.2 

func (self *DockerURL) WriteFirstLayer(context contextpkg.Context, writer io.Writer) error

func (*DockerURL) WriteTarball 

func (self *DockerURL) WriteTarball(context contextpkg.Context, writer io.Writer) error

type FileURL 

type FileURL struct {
	// This is an absolute OS file path.
	//
	// That means that when compiled on Windows it will expect and use
	// backslashes as path separators in addition to other Windows
	// filesystem convnentions.
	Path string
	// contains filtered or unexported fields
}

func (*FileURL) Base added in v0.4.0 

func (self *FileURL) Base() URL

(URL interface)

func (*FileURL) Context 

func (self *FileURL) Context() *Context

(URL interface)

func (*FileURL) Format 

func (self *FileURL) Format() string

(URL interface)

func (*FileURL) Key 

func (self *FileURL) Key() string

(URL interface)

func (*FileURL) Open 

func (self *FileURL) Open(context contextpkg.Context) (io.ReadCloser, error)

(URL interface)

func (*FileURL) Relative 

func (self *FileURL) Relative(path string) URL

Note that the argument can be a URL-type path or an OS file path (using backslashes on Windows).

Directories must be suffixed with an OS path separator.

(URL interface)

func (*FileURL) String 

func (self *FileURL) String() string

(fmt.Stringer interface)

func (*FileURL) ValidRelative added in v0.4.0 

func (self *FileURL) ValidRelative(context contextpkg.Context, filePath string) (URL, error)

Note that the argument can be a URL-type path or an OS file path (using backslashes on Windows).

If the path is a directory, it will automatically be suffixed with an OS path separator if it doesn't already have one.

(URL interface)

type GitURL 

type GitURL struct {
	Path          string
	RepositoryURL string
	Reference     string
	Username      string
	Password      string
	// contains filtered or unexported fields
}

func (*GitURL) Base added in v0.4.0 

func (self *GitURL) Base() URL

(URL interface)

func (*GitURL) Context 

func (self *GitURL) Context() *Context

(URL interface)

func (*GitURL) Format 

func (self *GitURL) Format() string

(URL interface)

func (*GitURL) Key 

func (self *GitURL) Key() string

(URL interface)

func (*GitURL) Open 

func (self *GitURL) Open(context contextpkg.Context) (io.ReadCloser, error)

(URL interface)

func (*GitURL) OpenRepository 

func (self *GitURL) OpenRepository() (*git.Repository, error)

func (*GitURL) Relative 

func (self *GitURL) Relative(path string) URL

(URL interface)

func (*GitURL) String 

func (self *GitURL) String() string

(fmt.Stringer interface)

func (*GitURL) ValidRelative added in v0.4.0 

func (self *GitURL) ValidRelative(context contextpkg.Context, path string) (URL, error)

(URL interface)

type InternalURL 

type InternalURL struct {
	Path string

	// If explicitly set to a non-nil value then it will override any globally
	// registered content when calling InternalURL.Open.
	//
	// []byte or InternalURLProvider.
	OverrideContent any
	// contains filtered or unexported fields
}

func (*InternalURL) Base added in v0.4.0 

func (self *InternalURL) Base() URL

(URL interface)

func (*InternalURL) Context 

func (self *InternalURL) Context() *Context

(URL interface)

func (*InternalURL) Format 

func (self *InternalURL) Format() string

(URL interface)

func (*InternalURL) Key 

func (self *InternalURL) Key() string

(URL interface)

func (*InternalURL) Open 

func (self *InternalURL) Open(context contextpkg.Context) (io.ReadCloser, error)

If InternalURL.Content was set to a non-nil value, then it will be used. Otherwise will attempt to retrieve the globally registered content.

(URL interface)

func (*InternalURL) Relative 

func (self *InternalURL) Relative(path string) URL

(URL interface)

func (*InternalURL) SetContent 

func (self *InternalURL) SetContent(content any)

Updates the contents of this instance only. To change the globally registered content use UpdateInternalURL.

"content" can be []byte or an InternalURLProvider. Other types will be converted to string and then to []byte.

func (*InternalURL) String 

func (self *InternalURL) String() string

(fmt.Stringer interface)

func (*InternalURL) ValidRelative added in v0.4.0 

func (self *InternalURL) ValidRelative(context contextpkg.Context, path string) (URL, error)

(URL interface)

type InternalURLProvider added in v0.2.4 

type InternalURLProvider interface {
	OpenPath(context contextpkg.Context, path string) (io.ReadCloser, error)
}

type MockURL added in v0.2.9 

type MockURL struct {
	Scheme  string
	Path    string
	Content any // []byte or InternalURLProvider
	// contains filtered or unexported fields
}

func (*MockURL) Base added in v0.4.0 

func (self *MockURL) Base() URL

(URL interface)

func (*MockURL) Context added in v0.2.9 

func (self *MockURL) Context() *Context

(URL interface)

func (*MockURL) Format added in v0.2.9 

func (self *MockURL) Format() string

(URL interface)

func (*MockURL) Key added in v0.2.9 

func (self *MockURL) Key() string

(URL interface)

func (*MockURL) Open added in v0.2.9 

func (self *MockURL) Open(context contextpkg.Context) (io.ReadCloser, error)

(URL interface)

func (*MockURL) Relative added in v0.2.9 

func (self *MockURL) Relative(path string) URL

(URL interface)

func (*MockURL) SetContent added in v0.4.0 

func (self *MockURL) SetContent(content any)

Updates the contents of this instance only. To change the globally registered content use UpdateInternalURL.

"content" can be []byte or an InternalURLProvider. Other types will be converted to string and then to []byte.

func (*MockURL) String added in v0.2.9 

func (self *MockURL) String() string

(fmt.Stringer interface)

func (*MockURL) ValidRelative added in v0.4.0 

func (self *MockURL) ValidRelative(context contextpkg.Context, path string) (URL, error)

(URL interface)

type NetworkURL 

type NetworkURL struct {
	URL *neturlpkg.URL
	// contains filtered or unexported fields
}

func (*NetworkURL) Base added in v0.4.0 

func (self *NetworkURL) Base() URL

(URL interface)

func (*NetworkURL) Context 

func (self *NetworkURL) Context() *Context

(URL interface)

func (*NetworkURL) Format 

func (self *NetworkURL) Format() string

(URL interface)

func (*NetworkURL) Key 

func (self *NetworkURL) Key() string

(URL interface)

func (*NetworkURL) Open 

func (self *NetworkURL) Open(context contextpkg.Context) (io.ReadCloser, error)

(URL interface)

func (*NetworkURL) Relative 

func (self *NetworkURL) Relative(path string) URL

(URL interface)

func (*NetworkURL) String 

func (self *NetworkURL) String() string

(fmt.Stringer interface)

func (*NetworkURL) ValidRelative added in v0.4.0 

func (self *NetworkURL) ValidRelative(context contextpkg.Context, path string) (URL, error)

(URL interface)

type NotFound 

type NotFound struct {
	Message string
}

func NewNotFound 

func NewNotFound(message string) *NotFound

func NewNotFoundf 

func NewNotFoundf(format string, arg ...any) *NotFound

func (*NotFound) Error 

func (self *NotFound) Error() string

(error interface)

type NotImplemented added in v0.2.8 

type NotImplemented struct {
	Message string
}

func NewNotImplemented added in v0.2.8 

func NewNotImplemented(message string) *NotImplemented

func NewNotImplementedf added in v0.2.8 

func NewNotImplementedf(format string, arg ...any) *NotImplemented

func (*NotImplemented) Error added in v0.2.8 

func (self *NotImplemented) Error() string

(error interface)

type TarballURL 

type TarballURL struct {
	Path          string
	ArchiveURL    URL
	ArchiveFormat string
}

func NewTarballURL 

func NewTarballURL(path string, archiveUrl URL, archiveFormat string) *TarballURL

func NewValidTarballURL 

func NewValidTarballURL(context contextpkg.Context, path string, archiveUrl URL, archiveFormat string) (*TarballURL, error)

func (*TarballURL) Base added in v0.4.0 

func (self *TarballURL) Base() URL

(URL interface)

func (*TarballURL) Context 

func (self *TarballURL) Context() *Context

(URL interface)

func (*TarballURL) Format 

func (self *TarballURL) Format() string

(URL interface)

func (*TarballURL) Key 

func (self *TarballURL) Key() string

(URL interface)

func (*TarballURL) Open 

func (self *TarballURL) Open(context contextpkg.Context) (io.ReadCloser, error)

(URL interface)

func (*TarballURL) OpenArchive 

func (self *TarballURL) OpenArchive(context contextpkg.Context) (*util.TarballReader, error)

func (*TarballURL) Relative 

func (self *TarballURL) Relative(path string) URL

(URL interface)

func (*TarballURL) String 

func (self *TarballURL) String() string

(fmt.Stringer interface)

func (*TarballURL) ValidRelative added in v0.4.0 

func (self *TarballURL) ValidRelative(context contextpkg.Context, path string) (URL, error)

(URL interface)

type URL 

type URL interface {
	fmt.Stringer

	// Format of the URL content's default representation.
	//
	// Can return "yaml", "json", "xml", etc., or an empty string if the format
	// is unknown.
	//
	// The format is often derived from a file extension if available, otherwise
	// it might be retrieved from metadata.
	//
	// An attempt is made to standardize the return values, e.g. a "yml" file
	// extension is always returned as "yaml", and a "tar.gz" file extension is
	// always returned as "tgz".
	Format() string

	// Returns a URL that is the equivalent of a "base directory" for this URL.
	//
	// Base URLs always often have a trailing slash to signify that they are
	// "directories" rather than "files". One notable exception is "file:" URLs
	// when compiled on Windows, in which case a trailing backslash is used
	// instead.
	//
	// The base is often used in two ways:
	//
	// 1. You can call URL.Relative on it to get a sibling URL to this one (relative
	//    to the same "base directory").
	// 2. You can use it in the "bases" list argument of Context.NewValidURL for the
	//    same purpose.
	//
	// Note that the base might not be a valid URL in itself, e.g. you might not
	// be able to call Open on it.
	Base() URL

	// Parses the argument as a path relative to the URL. That means that this
	// URL is treated as a "base directory" (see URL.Base). The argument supports
	// ".." and ".", with the returned URL path always being absolute.
	Relative(path string) URL

	// As URL.Relative but returns a valid URL.
	ValidRelative(context contextpkg.Context, path string) (URL, error)

	// Returns a string that uniquely identifies the URL.
	//
	// Useful for map and cache keys.
	Key() string

	// Opens the URL for reading.
	//
	// Note that for some URLs it can involve lengthy operations, e.g. cloning a
	// remote repository or downloading an archive. For this reason a cancellable
	// context can be provided as an argument.
	//
	// An effort is made to not repeat these lengthy operations by caching related
	// state in the URL's exturl Context (caching is deliberately not done globally).
	// For example, when accessing a "git:" URL on a remote git repository then that
	// repository will be cloned locally only if it's the first the repository has been
	// referred to for the exturl Context. Subsequent Open calls for URLs that refer
	// to the same git repository will reuse the existing clone.
	//
	// It is the caller's responsibility to call Close on the reader.
	Open(context contextpkg.Context) (io.ReadCloser, error)

	// The exturl context used to create this URL.
	Context() *Context
}

type URLTransformerFunc added in v0.4.0 

type URLTransformerFunc func(fromUrl string) (string, bool)

type ZipEntryReader 

type ZipEntryReader struct {
	EntryReader io.ReadCloser
	ZipReader   *ZipReader
}

func NewZipEntryReader 

func NewZipEntryReader(entryReader io.ReadCloser, zipReader *ZipReader) *ZipEntryReader

func (*ZipEntryReader) Close 

func (self *ZipEntryReader) Close() error

(io.Closer interface)

func (*ZipEntryReader) Read 

func (self *ZipEntryReader) Read(p []byte) (n int, err error)

(io.Reader interface)

type ZipReader 

type ZipReader struct {
	ZipReader *zip.Reader
	File      *os.File
}

func NewZipReader 

func NewZipReader(reader *zip.Reader, file *os.File) *ZipReader

func NewZipReaderForFile added in v0.4.0 

func NewZipReaderForFile(file *os.File) (*ZipReader, error)

func (*ZipReader) Close 

func (self *ZipReader) Close() error

(io.Closer interface)

func (*ZipReader) Has 

func (self *ZipReader) Has(path string) bool

func (*ZipReader) Iterate 

func (self *ZipReader) Iterate(f func(*zip.File) bool)

func (*ZipReader) Open 

func (self *ZipReader) Open(path string) (*ZipEntryReader, error)

type ZipURL 

type ZipURL struct {
	Path       string
	ArchiveURL URL
}

func NewValidZipURL 

func NewValidZipURL(context contextpkg.Context, path string, archiveUrl URL) (*ZipURL, error)

func NewZipURL 

func NewZipURL(path string, archiveUrl URL) *ZipURL

func (*ZipURL) Base added in v0.4.0 

func (self *ZipURL) Base() URL

(URL interface)

func (*ZipURL) Context 

func (self *ZipURL) Context() *Context

(URL interface)

func (*ZipURL) Format 

func (self *ZipURL) Format() string

(URL interface)

func (*ZipURL) Key 

func (self *ZipURL) Key() string

(URL interface)

func (*ZipURL) Open 

func (self *ZipURL) Open(context contextpkg.Context) (io.ReadCloser, error)

(URL interface)

func (*ZipURL) OpenArchive 

func (self *ZipURL) OpenArchive(context contextpkg.Context) (*ZipReader, error)

func (*ZipURL) Relative 

func (self *ZipURL) Relative(path string) URL

(URL interface)

func (*ZipURL) String 

func (self *ZipURL) String() string

(fmt.Stringer interface)

func (*ZipURL) ValidRelative added in v0.4.0 

func (self *ZipURL) ValidRelative(context contextpkg.Context, path string) (URL, error)

(URL interface)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.