stor-client

README

stor-client

what is stor?

stor is storage HTTP interface for sha256 files (objects)

features

• download retry
• concurent download (default 4)

cli

read (parse) SHA256 from STDIN and download it to destinationDir

echo EE2BF0BFD365EBF829F8D07B197B7A15F39760CD14C6D3BFDFBAD2B145CB72B8 | stor-client --storage http://stor.domain.tld .

help

usage: stor-cli [<flags>] <downloadDir>

Flags:
      --help           Show context-sensitive help (also try --help-long and --help-man).
  -u, --storage=http://stor.whale.int.avast.com storage url
      --max=4          max download process
      --devnull        download file to /dev/null
  -v, --verbose        more talkativ output
      --timeout=30s    connetion timeout
      --json           log in json format
      --delay=100ms    exponential retry - start delay time
      --attempts=10    count of attempts of retry
      --suffix=""      downloaded file suffix - like '.dat' => SHA.dat
      --upper          name of file will be upper case (not applied to suffix)
      --s3host=S3HOST  host to s3 endpoint with bucket e.g. https://bucket.s3.eu-central-1.amazonaws.com, if is s3url set, first will be use S3, then fallback to stor
      --s3template="{{.FirstShaByte}}/{{.SecondShaByte}}/{{.ThirdShaByte}}/{{.Sha}}" template to S3 path
      --version        Show application version.

Args:
  <downloadDir>  directory for downloaded files

golang client

golang stor-client library

async example

client := storclient.New(storageUrl, storclient.StorClientOpts{})

client.Start()

for _, sha := range shaList {
	client.Download(sha)
}

downloadStatus := client.Wait()

Documentation

Overview

stor-client is command line utility for downloading from stor

what is stor?

Stor (https://github.com/avast/stor) is storage HTTP interface for sha256 files (objects). Stor can help with step by step migration from posix storage to object storage (aka S3).

features

* download retry * concurent download (default `4`) * S3 download as primary place, stor as fallback

cli

read (parse) SHA256 from STDIN and download it to `destinationDir`

echo EE2BF0BFD365EBF829F8D07B197B7A15F39760CD14C6D3BFDFBAD2B145CB72B8 | stor-client --storage http://stor.domain.tld .

golang client

look to github.com/avast/stor-client/client