README

This code provides helper functions for dealing with archive files.

Expand ▾ Collapse ▴

Documentation

Index

Constants

View Source
const (
	// ChangeModify represents the modify operation.
	ChangeModify = iota
	// ChangeAdd represents the add operation.
	ChangeAdd
	// ChangeDelete represents the delete operation.
	ChangeDelete
)
View Source
const WhiteoutLinkDir = WhiteoutMetaPrefix + "plnk"

    WhiteoutLinkDir is a directory AUFS uses for storing hardlink links to other layers. Normally these should not go into exported archives and all changed hardlinks should be copied to the top layer.

    View Source
    const WhiteoutMetaPrefix = WhiteoutPrefix + WhiteoutPrefix

      WhiteoutMetaPrefix prefix means whiteout has a special meaning and is not for removing an actual file. Normally these files are excluded from exported archives.

      View Source
      const WhiteoutOpaqueDir = WhiteoutMetaPrefix + ".opq"

        WhiteoutOpaqueDir file means directory has been made opaque - meaning readdir calls to this directory do not follow to lower layers.

        View Source
        const WhiteoutPrefix = ".wh."

          WhiteoutPrefix prefix means file is a whiteout. If this is followed by a filename this means that file has been removed from the base layer.

          Variables

          View Source
          var (
          	ErrNotDirectory      = errors.New("not a directory")
          	ErrDirNotExists      = errors.New("no such directory")
          	ErrCannotCopyDir     = errors.New("cannot copy directory")
          	ErrInvalidCopySource = errors.New("invalid copy source content")
          )

            Errors used or returned by this file.

            Functions

            func ApplyLayer

            func ApplyLayer(dest string, layer io.Reader) (int64, error)

              ApplyLayer parses a diff in the standard layer format from `layer`, and applies it to the directory `dest`. The stream `layer` can be compressed or uncompressed. Returns the size in bytes of the contents of the layer.

              func ApplyUncompressedLayer

              func ApplyUncompressedLayer(dest string, layer io.Reader, options *TarOptions) (int64, error)

                ApplyUncompressedLayer parses a diff in the standard layer format from `layer`, and applies it to the directory `dest`. The stream `layer` can only be uncompressed. Returns the size in bytes of the contents of the layer.

                func CanonicalTarNameForPath

                func CanonicalTarNameForPath(p string) string

                  CanonicalTarNameForPath returns platform-specific filepath to canonical posix-style path for tar archival. p is relative path.

                  func ChangesSize

                  func ChangesSize(newDir string, changes []Change) int64

                    ChangesSize calculates the size in bytes of the provided changes, based on newDir.

                    func CompressStream

                    func CompressStream(dest io.Writer, compression Compression) (io.WriteCloser, error)

                      CompressStream compresses the dest with specified compression algorithm.

                      func CopyResource

                      func CopyResource(srcPath, dstPath string, followLink bool) error

                        CopyResource performs an archive copy from the given source path to the given destination path. The source path MUST exist and the destination path's parent directory must exist.

                        func CopyTo

                        func CopyTo(content io.Reader, srcInfo CopyInfo, dstPath string) error

                          CopyTo handles extracting the given content whose entries should be sourced from srcInfo to dstPath.

                          func DecompressStream

                          func DecompressStream(archive io.Reader) (io.ReadCloser, error)

                            DecompressStream decompresses the archive and returns a ReaderCloser with the decompressed archive.

                            func ExportChanges

                            func ExportChanges(dir string, changes []Change, uidMaps, gidMaps []idtools.IDMap) (io.ReadCloser, error)

                              ExportChanges produces an Archive from the provided changes, relative to dir.

                              func FileInfoHeader

                              func FileInfoHeader(name string, fi os.FileInfo, link string) (*tar.Header, error)

                                FileInfoHeader creates a populated Header from fi. Compared to archive pkg this function fills in more information. Also, regardless of Go version, this function fills file type bits (e.g. hdr.Mode |= modeISDIR), which have been deleted since Go 1.9 archive/tar.

                                func Generate

                                func Generate(input ...string) (io.Reader, error)

                                  Generate generates a new archive from the content provided as input.

                                  `files` is a sequence of path/content pairs. A new file is added to the archive for each pair. If the last pair is incomplete, the file is created with an empty content. For example:

                                  Generate("foo.txt", "hello world", "emptyfile")

                                  The above call will return an archive with 2 files:

                                  * ./foo.txt with content "hello world"
                                  * ./empty with empty content
                                  

                                  FIXME: stream content instead of buffering FIXME: specify permissions and other archive metadata

                                  func GetRebaseName

                                  func GetRebaseName(path, resolvedPath string) (string, string)

                                    GetRebaseName normalizes and compares path and resolvedPath, return completed resolved path and rebased file name

                                    func IsArchivePath

                                    func IsArchivePath(path string) bool

                                      IsArchivePath checks if the (possibly compressed) file at the given path starts with a tar file header.

                                      func PrepareArchiveCopy

                                      func PrepareArchiveCopy(srcContent io.Reader, srcInfo, dstInfo CopyInfo) (dstDir string, content io.ReadCloser, err error)

                                        PrepareArchiveCopy prepares the given srcContent archive, which should contain the archived resource described by srcInfo, to the destination described by dstInfo. Returns the possibly modified content archive along with the path to the destination directory which it should be extracted to.

                                        func PreserveTrailingDotOrSeparator

                                        func PreserveTrailingDotOrSeparator(cleanedPath string, originalPath string, sep byte) string

                                          PreserveTrailingDotOrSeparator returns the given cleaned path (after processing using any utility functions from the path or filepath stdlib packages) and appends a trailing `/.` or `/` if its corresponding original path (from before being processed by utility functions from the path or filepath stdlib packages) ends with a trailing `/.` or `/`. If the cleaned path already ends in a `.` path segment, then another is not added. If the clean path already ends in the separator, then another is not added.

                                          func ReadSecurityXattrToTarHeader

                                          func ReadSecurityXattrToTarHeader(path string, hdr *tar.Header) error

                                            ReadSecurityXattrToTarHeader reads security.capability xattr from filesystem to a tar header

                                            func RebaseArchiveEntries

                                            func RebaseArchiveEntries(srcContent io.Reader, oldBase, newBase string) io.ReadCloser

                                              RebaseArchiveEntries rewrites the given srcContent archive replacing an occurrence of oldBase with newBase at the beginning of entry names.

                                              func ReplaceFileTarWrapper

                                              func ReplaceFileTarWrapper(inputTarStream io.ReadCloser, mods map[string]TarModifierFunc) io.ReadCloser

                                                ReplaceFileTarWrapper converts inputTarStream to a new tar stream. Files in the tar stream are modified if they match any of the keys in mods.

                                                func ResolveHostSourcePath

                                                func ResolveHostSourcePath(path string, followLink bool) (resolvedPath, rebaseName string, err error)

                                                  ResolveHostSourcePath decides real path need to be copied with parameters such as whether to follow symbol link or not, if followLink is true, resolvedPath will return link target of any symbol link file, else it will only resolve symlink of directory but return symbol link file itself without resolving.

                                                  func SplitPathDirEntry

                                                  func SplitPathDirEntry(path string) (dir, base string)

                                                    SplitPathDirEntry splits the given path between its directory name and its basename by first cleaning the path but preserves a trailing "." if the original path specified the current directory.

                                                    func Tar

                                                    func Tar(path string, compression Compression) (io.ReadCloser, error)

                                                      Tar creates an archive from the directory at `path`, and returns it as a stream of bytes.

                                                      func TarResource

                                                      func TarResource(sourceInfo CopyInfo) (content io.ReadCloser, err error)

                                                        TarResource archives the resource described by the given CopyInfo to a Tar archive. A non-nil error is returned if sourcePath does not exist or is asserted to be a directory but exists as another type of file.

                                                        This function acts as a convenient wrapper around TarWithOptions, which requires a directory as the source path. TarResource accepts either a directory or a file path and correctly sets the Tar options.

                                                        func TarResourceRebase

                                                        func TarResourceRebase(sourcePath, rebaseName string) (content io.ReadCloser, err error)

                                                          TarResourceRebase is like TarResource but renames the first path element of items in the resulting tar archive to match the given rebaseName if not "".

                                                          func TarWithOptions

                                                          func TarWithOptions(srcPath string, options *TarOptions) (io.ReadCloser, error)

                                                            TarWithOptions creates an archive from the directory at `path`, only including files whose relative paths are included in `options.IncludeFiles` (if non-nil) or not in `options.ExcludePatterns`.

                                                            func Unpack

                                                            func Unpack(decompressedArchive io.Reader, dest string, options *TarOptions) error

                                                              Unpack unpacks the decompressedArchive to dest with options.

                                                              func UnpackLayer

                                                              func UnpackLayer(dest string, layer io.Reader, options *TarOptions) (size int64, err error)

                                                                UnpackLayer unpack `layer` to a `dest`. The stream `layer` can be compressed or uncompressed. Returns the size in bytes of the contents of the layer.

                                                                func Untar

                                                                func Untar(tarArchive io.Reader, dest string, options *TarOptions) error

                                                                  Untar reads a stream of bytes from `archive`, parses it as a tar archive, and unpacks it into the directory at `dest`. The archive may be compressed with one of the following algorithms:

                                                                  identity (uncompressed), gzip, bzip2, xz.
                                                                  

                                                                  FIXME: specify behavior when target path exists vs. doesn't exist.

                                                                  func UntarUncompressed

                                                                  func UntarUncompressed(tarArchive io.Reader, dest string, options *TarOptions) error

                                                                    UntarUncompressed reads a stream of bytes from `archive`, parses it as a tar archive, and unpacks it into the directory at `dest`. The archive must be an uncompressed stream.

                                                                    Types

                                                                    type Archiver

                                                                    type Archiver struct {
                                                                    	Untar     func(io.Reader, string, *TarOptions) error
                                                                    	IDMapping *idtools.IdentityMapping
                                                                    }

                                                                      Archiver implements the Archiver interface and allows the reuse of most utility functions of this package with a pluggable Untar function. Also, to facilitate the passing of specific id mappings for untar, an Archiver can be created with maps which will then be passed to Untar operations.

                                                                      func NewDefaultArchiver

                                                                      func NewDefaultArchiver() *Archiver

                                                                        NewDefaultArchiver returns a new Archiver without any IdentityMapping

                                                                        func (*Archiver) CopyFileWithTar

                                                                        func (archiver *Archiver) CopyFileWithTar(src, dst string) (err error)

                                                                          CopyFileWithTar emulates the behavior of the 'cp' command-line for a single file. It copies a regular file from path `src` to path `dst`, and preserves all its metadata.

                                                                          func (*Archiver) CopyWithTar

                                                                          func (archiver *Archiver) CopyWithTar(src, dst string) error

                                                                            CopyWithTar creates a tar archive of filesystem path `src`, and unpacks it at filesystem path `dst`. The archive is streamed directly with fixed buffering and no intermediary disk IO.

                                                                            func (*Archiver) IdentityMapping

                                                                            func (archiver *Archiver) IdentityMapping() *idtools.IdentityMapping

                                                                              IdentityMapping returns the IdentityMapping of the archiver.

                                                                              func (*Archiver) TarUntar

                                                                              func (archiver *Archiver) TarUntar(src, dst string) error

                                                                                TarUntar is a convenience function which calls Tar and Untar, with the output of one piped into the other. If either Tar or Untar fails, TarUntar aborts and returns the error.

                                                                                func (*Archiver) UntarPath

                                                                                func (archiver *Archiver) UntarPath(src, dst string) error

                                                                                  UntarPath untar a file from path to a destination, src is the source tar file path.

                                                                                  type Change

                                                                                  type Change struct {
                                                                                  	Path string
                                                                                  	Kind ChangeType
                                                                                  }

                                                                                    Change represents a change, it wraps the change type and path. It describes changes of the files in the path respect to the parent layers. The change could be modify, add, delete. This is used for layer diff.

                                                                                    func Changes

                                                                                    func Changes(layers []string, rw string) ([]Change, error)

                                                                                      Changes walks the path rw and determines changes for the files in the path, with respect to the parent layers

                                                                                      func ChangesDirs

                                                                                      func ChangesDirs(newDir, oldDir string) ([]Change, error)

                                                                                        ChangesDirs compares two directories and generates an array of Change objects describing the changes. If oldDir is "", then all files in newDir will be Add-Changes.

                                                                                        func (*Change) String

                                                                                        func (change *Change) String() string

                                                                                        type ChangeType

                                                                                        type ChangeType int

                                                                                          ChangeType represents the change type.

                                                                                          func (ChangeType) String

                                                                                          func (c ChangeType) String() string

                                                                                          type Compression

                                                                                          type Compression int

                                                                                            Compression is the state represents if compressed or not.

                                                                                            const (
                                                                                            	// Uncompressed represents the uncompressed.
                                                                                            	Uncompressed Compression = iota
                                                                                            	// Bzip2 is bzip2 compression algorithm.
                                                                                            	Bzip2
                                                                                            	// Gzip is gzip compression algorithm.
                                                                                            	Gzip
                                                                                            	// Xz is xz compression algorithm.
                                                                                            	Xz
                                                                                            )

                                                                                            func DetectCompression

                                                                                            func DetectCompression(source []byte) Compression

                                                                                              DetectCompression detects the compression algorithm of the source.

                                                                                              func (*Compression) Extension

                                                                                              func (compression *Compression) Extension() string

                                                                                                Extension returns the extension of a file that uses the specified compression algorithm.

                                                                                                type CopyInfo

                                                                                                type CopyInfo struct {
                                                                                                	Path       string
                                                                                                	Exists     bool
                                                                                                	IsDir      bool
                                                                                                	RebaseName string
                                                                                                }

                                                                                                  CopyInfo holds basic info about the source or destination path of a copy operation.

                                                                                                  func CopyInfoDestinationPath

                                                                                                  func CopyInfoDestinationPath(path string) (info CopyInfo, err error)

                                                                                                    CopyInfoDestinationPath stats the given path to create a CopyInfo struct representing that resource for the destination of an archive copy operation. The given path should be an absolute local path.

                                                                                                    func CopyInfoSourcePath

                                                                                                    func CopyInfoSourcePath(path string, followLink bool) (CopyInfo, error)

                                                                                                      CopyInfoSourcePath stats the given path to create a CopyInfo struct representing that resource for the source of an archive copy operation. The given path should be an absolute local path. A source path has all symlinks evaluated that appear before the last path separator ("/" on Unix). As it is to be a copy source, the path must exist.

                                                                                                      type FileInfo

                                                                                                      type FileInfo struct {
                                                                                                      	// contains filtered or unexported fields
                                                                                                      }

                                                                                                        FileInfo describes the information of a file.

                                                                                                        func (*FileInfo) Changes

                                                                                                        func (info *FileInfo) Changes(oldInfo *FileInfo) []Change

                                                                                                          Changes add changes to file information.

                                                                                                          func (*FileInfo) LookUp

                                                                                                          func (info *FileInfo) LookUp(path string) *FileInfo

                                                                                                            LookUp looks up the file information of a file.

                                                                                                            type TarModifierFunc

                                                                                                            type TarModifierFunc func(path string, header *tar.Header, content io.Reader) (*tar.Header, []byte, error)

                                                                                                              TarModifierFunc is a function that can be passed to ReplaceFileTarWrapper to modify the contents or header of an entry in the archive. If the file already exists in the archive the TarModifierFunc will be called with the Header and a reader which will return the files content. If the file does not exist both header and content will be nil.

                                                                                                              type TarOptions

                                                                                                              type TarOptions struct {
                                                                                                              	IncludeFiles     []string
                                                                                                              	ExcludePatterns  []string
                                                                                                              	Compression      Compression
                                                                                                              	NoLchown         bool
                                                                                                              	UIDMaps          []idtools.IDMap
                                                                                                              	GIDMaps          []idtools.IDMap
                                                                                                              	ChownOpts        *idtools.Identity
                                                                                                              	IncludeSourceDir bool
                                                                                                              	// WhiteoutFormat is the expected on disk format for whiteout files.
                                                                                                              	// This format will be converted to the standard format on pack
                                                                                                              	// and from the standard format on unpack.
                                                                                                              	WhiteoutFormat WhiteoutFormat
                                                                                                              	// When unpacking, specifies whether overwriting a directory with a
                                                                                                              	// non-directory is allowed and vice versa.
                                                                                                              	NoOverwriteDirNonDir bool
                                                                                                              	// For each include when creating an archive, the included name will be
                                                                                                              	// replaced with the matching name from this map.
                                                                                                              	RebaseNames map[string]string
                                                                                                              	InUserNS    bool
                                                                                                              }

                                                                                                                TarOptions wraps the tar options.

                                                                                                                func TarResourceRebaseOpts

                                                                                                                func TarResourceRebaseOpts(sourceBase string, rebaseName string) *TarOptions

                                                                                                                  TarResourceRebaseOpts does not preform the Tar, but instead just creates the rebase parameters to be sent to TarWithOptions (the TarOptions struct)

                                                                                                                  type TempArchive

                                                                                                                  type TempArchive struct {
                                                                                                                  	*os.File
                                                                                                                  	Size int64 // Pre-computed from Stat().Size() as a convenience
                                                                                                                  	// contains filtered or unexported fields
                                                                                                                  }

                                                                                                                    TempArchive is a temporary archive. The archive can only be read once - as soon as reading completes, the file will be deleted.

                                                                                                                    func NewTempArchive

                                                                                                                    func NewTempArchive(src io.Reader, dir string) (*TempArchive, error)

                                                                                                                      NewTempArchive reads the content of src into a temporary file, and returns the contents of that file as an archive. The archive can only be read once - as soon as reading completes, the file will be deleted.

                                                                                                                      func (*TempArchive) Close

                                                                                                                      func (archive *TempArchive) Close() error

                                                                                                                        Close closes the underlying file if it's still open, or does a no-op to allow callers to try to close the TempArchive multiple times safely.

                                                                                                                        func (*TempArchive) Read

                                                                                                                        func (archive *TempArchive) Read(data []byte) (int, error)

                                                                                                                        type WhiteoutFormat

                                                                                                                        type WhiteoutFormat int

                                                                                                                          WhiteoutFormat is the format of whiteouts unpacked

                                                                                                                          const (
                                                                                                                          	// AUFSWhiteoutFormat is the default format for whiteouts
                                                                                                                          	AUFSWhiteoutFormat WhiteoutFormat = iota
                                                                                                                          	// OverlayWhiteoutFormat formats whiteout according to the overlay
                                                                                                                          	// standard.
                                                                                                                          	OverlayWhiteoutFormat
                                                                                                                          )