extract

README

go-extract

Secure extraction of zip/tar/tar.gz/gz archive type.

Code Example

Add to go.mod:

GOPRIVATE=github.com/hashicorp/go-extract go get github.com/hashicorp/go-extract

Usage in code:

import (
    ...
    "github.com/hashicorp/go-extract"
    "github.com/hashicorp/go-extract/config"
    "github.com/hashicorp/go-extract/target"
    ...
)

...

    ctx := context.Background()

    // open archive
    archive, _ := os.Open(...)

    // prepare config (these are the default values)
    config := config.NewConfig(
        config.WithAllowSymlinks(true),                      // allow symlink creation
        config.WithContinueOnError(false),                   // fail on error
        config.WithFollowSymlinks(false),                    // do not follow symlinks during creation
        config.WithLogger(*slog.Logger),                     // adjust logger (default: io.Discard)
        config.WithMaxExtractionSize(1 << (10 * 3)),         // limit to 1 Gb (disable check: -1)
        config.WithMaxFiles(1000),                           // only 1k files maximum (disable check: -1)
        config.WithMetricsHook(metricsHook(config.Metrics)), // define hook to receive metrics from extraction
        config.WithOverwrite(false),                         // don't replace existing files
    )

    // prepare context with timeout
    var cancel context.CancelFunc
    ctx, cancel = context.WithTimeout(context.Background(), (time.Second * time.Duration(MaxExtractionTime)))
    defer cancel()

    // extract archive
    if err := extract.Unpack(ctx, archive, destinationPath, target.NewOs(), config); err != nil {
      // handle error
    }

...

Cli Tool

The library can also be used directly on the cli extract.

Installation

GOPRIVATE=github.com/hashicorp/go-extract go install github.com/hashicorp/go-extract/cmd/extract@latest

Manual Build and Installation

git clone git@github.com:hashicorp/go-extract.git
cd go-extract
make
make test
make install

Usage

extract -h
Usage: extract <archive> [<destination>]

A secure extraction utility

Arguments:
  <archive>          Path to archive. ("-" for STDIN)
  [<destination>]    Output directory/file.

Flags:
  -h, --help                              Show context-sensitive help.
  -C, --continue-on-error                 Continue extraction on error.
  -D, --deny-symlinks                     Deny symlink extraction.
  -F, --follow-symlinks                   [Dangerous!] Follow symlinks to directories during extraction.
      --max-files=1000                    Maximum files that are extracted before stop. (disable check: -1)
      --max-extraction-size=1073741824    Maximum extraction size that allowed is (in bytes). (disable check: -1)
      --max-extraction-time=60            Maximum time that an extraction should take (in seconds). (disable check: -1)
  -M, --metrics                           Print metrics to log after extraction.
  -O, --overwrite                         Overwrite if exist.
  -v, --verbose                           Verbose logging.
  -V, --version                           Print release version information.

Feature collection

• Filetypes
  • zip (/jar)
  • tar
  • gzip
  • tar.gz
  • bzip2
  • 7zip
  • rar
  • deb
• extraction size check
• max num of extracted files
• extraction time exhaustion
• context based cancleation
• option pattern for configuration
• io.Reader as source
• symlink inside archive
• symlink to outside is detected
• symlink with absolute path is detected
• file with path traversal is detected
• file with absolute path is detected
• filetype detection based on magic bytes
• windows support
• tests for gzip
• function documentation
• check for windows
• Allow/deny symlinks in general
• Metrics call back function
• Handle passwords
• recursive extraction
• virtual fs as target

References

• SecureZip
• 42zip

Documentation

Index

• func Unpack(ctx context.Context, src io.Reader, dst string, t target.Target, ...) error
• type Extractor

Functions

func Unpack

func Unpack(ctx context.Context, src io.Reader, dst string, t target.Target, c *config.Config) error

Unpack reads data from src, identifies if its a known archive type. If so, dst is unpacked in dst. opts can be given to adjust the config and target.

Types

type Extractor

type Extractor interface {
	// Unpack is the main entrypoint to an extraction engine that takes the contents from src and extracts them to dst.
	Unpack(ctx context.Context, src io.Reader, dst string, target target.Target, config *config.Config) error
}

Extractor is an interface and defines all functions that needs to be implemented by an extraction engine.