imaget

README

imaget

Imaget is a convenient image tool for finding images on any http(s) website and downloading them with optional flags to tweak tool behaviour and images output.

Imaget makes sure to only download the necessary bytes. Therefore, remembers image URLs and caches the downloads in the temporary directory of your operating system (/tmp/imaget/ for linux) to skip already downloaded images and also auto-resume incomplete downloads when canceled in the past.

The tool is user-friendly and can also be used in automation since the -y (skip user accept screen) or -t (timeout download) and other flags are provided for convenience of use.

Install

No worries, installation is straight forward!

Requirements:

• (Git & Go) OR (Docker) installed

Install with Go:

git clone https://github.com/robinbraemer/imaget.git
cd imaget
go install cmd/imaget.go

Test with Docker:

docker run -it --rm golang:1.15
git clone https://github.com/robinbraemer/imaget.git
cd imaget
go install cmd/imaget.go

Now try running imaget in your command line!

Showcase

While there are many more use cases for this tool this video shows 3 sample commands and how much faster it is due to the smart cache functionality used when running the last command twice.

1. Download Google's current image above the search box.
   • imaget -s -f -u google.com
2. Download all images found on amazon.com to new Zip archive.
   • imaget -y -f -u amazon.com -d amazon-images.zip
3. Download all images found on alibaba.com to hierarchical directories.
   • imaget -y -u alibaba.com -d alibaba-images
4. Re-run 3. to see how much faster the download is thanks to automatic cache use.
   • imaget -y -u alibaba.com -d alibaba-images

Usage

This is what's outputted when typing imaget in your command line.

usage: imaget -u URL [-d destination] [-t timeout] [-r regex] [-y] [-s] [-f]

Imaget is a convenient image tool for finding images on any http(s) website and
downloading them with optional flags to tweak tool behaviour and images output.

Flags
-----------------

-u (required): is the http(s) URL to find and images from to download.

-d (optional): is the destination to download the images to.
               It can either be the directory to save all images at or
               a path to create a .zip archive to save the images in.

-f (optional): saves the downloaded images as a flat hierarchie,
               instead of creating subdirectories as per the image download URLs.
               The name of the file is the base64 encoded download URL of the image.

-t (optional): is the timeout to wait before pausing the download
               and quitting the programm. Zero or below means no timeout.
               Example: 3m3s

-r (optional): is a regular expression to only download images from matching URLs.
               Examples: "(jpg|png)$", "^https?://"

-y (optional): starts the download directly without asking.

-s (optional): will make the console silent and produce no console output.
               If used the -y flag is used automatically.

Example commands
-----------------

Silently download Google's current image above the search box to the current directory.
> imaget -s -f -u google.com	

Download all images on amazon.com to new Zip archive in the current directory.
> imaget -y -f -u amazon.com -d amazon-images.zip

Download all images on alibaba.com to new directory 'alibaba-images' hierarchically sorted by image URL.
> imaget -y -u alibaba.com -d alibaba-images

Extend

The current features of the tool provide a sufficient foundation to other developers to extend functionality and support more advanced use cases such as...

• Log into Instagram/Pinterest/Facebook/... and download a complete image history
• Build out an image web crawler to automatically find baby groots in the wild ;)
  • allow many, many crawlers using a database for caching and distributed coordination

Which button was it again? (╯°□°）╯︵ ┻━┻

Documentation

Overview

Package imaget provides a convenient image tool for finding images on any http(s) website and downloading them with optional parameters to tweak behaviour and output.

Index

• Variables
• type Download
  • func (d *Download) Start(ctx context.Context) error
• type ProgressBar

Variables

var (
	Stdout io.Writer = os.Stdout
	Stderr io.Writer = os.Stderr
)

Std io

var Client = http.DefaultClient

Client is the http client to use.

Types

type Download

type Download struct {
	Src string // The source URL to find and download images from.
	Dst string // The destination to place the downloaded images at.

	Regex      *regexp.Regexp // A regex to filter images.
	SkipAccept bool           // Whether to skip the accept screen before downloading.
	// Whether to save the images flat, instead of creating
	// subdirectories as per the image download URLs.
	SaveFlat bool
	Bar      ProgressBar
}

Download holds all parameters to Start a download.

func (*Download) Start

func (d *Download) Start(ctx context.Context) error

Start searches for images on the specified website Src and downloads matching images to the desired destination Dst. Canceling the context will only pause the download and can be resumed to proceed downloading where paused at.

type ProgressBar

type ProgressBar interface {
	Start()           // Start showing the bar.
	Finish()          // Finish and hide the bar.
	SetTotal(int64)   // Set the maximum value.
	SetCurrent(int64) // Set the current value.
}

ProgressBar can show a progress bar for a download.