gorchive

README

Gorchive

Gorchive is a bulk archiving utility built in Go. It's currently a work in progress and undergoing a lot of changes.

Installation

Gorchive is simple!

Currently, there are no releases available as the tool should not be considered stable as of yet, but installation from code can be done with:

go get -u gitlab.com/draevin/gorchive

And, as long as your PATH is configured to contain your GOBIN directory, you should be all set.

Usage

Basic use can be as simple as running gorchive in a directory which will:

1. Create archives of all directories within the present working directory
2. Log this process to a yyyy-mm-dd-archive.log file within the present working directory

The archival can be customized with the following:

Option	Usage/Description
-source	Specify a source directory (all child directories will be archived)
-dest	Specify a destination directory for the archived files
-age	Specify the minimum age for the directories to be archived (since modified date if available, otherwise create date)
-log	Specify a directory or file to log to (Will log to standard yyyy-mm-dd-archive.log file if no file name is given)
-c	Don't log to a file; send log contents to standard output
-d	Delete source files after archiving (Use with caution!)
-v	Verbose output, increase log detail
-q	Quiet output, decrease log detail

Using these options, Gorchive is very flexible. For example:

gorchive -source ~/data/to/archive -dest /place/to/put/it -age 30 -log /place/to/log/file.txt -v -d

This will archive the files not modified in the past 30 days between the given directories, log the verbose output to the named file, and delete source directories following the archival. These options may change in the near future, and Gorchive should not be considered a stable product at this time.

Metrics

Say we need to archive everything in the directory data, where 1 also contains children B through E with identical internal structure to A, as well as 2 through 5 with their own identical children A through D:

data
├── 1
│   ├── A
│   │   ├── A1
│   │   │   ├── smltest1.txt ( 10 MiB)
│   │   │   ├── smltest2.txt ( 10 MiB)
│   │   │   └── smltest.txt  ( 10 MiB)
│   │   ├── A2
│   │   │   ├── smltest1.txt ( 10 MiB)
│   │   │   ├── smltest2.txt ( 10 MiB)
│   │   │   └── smltest.txt  ( 10 MiB)
│   │   ├── A3
│   │   │   ├── smltest1.txt ( 10 MiB)
│   │   │   ├── smltest2.txt ( 10 MiB)
│   │   │   └── smltest.txt  ( 10 MiB)
│   │   ├── A4
│   │   │   ├── smltest1.txt ( 10 MiB)
│   │   │   ├── smltest2.txt ( 10 MiB)
│   │   │   └── smltest.txt  ( 10 MiB)
│   │   ├── A5
│   │   │   ├── smltest1.txt ( 10 MiB)
│   │   │   ├── smltest2.txt ( 10 MiB)
│   │   │   └── smltest.txt  ( 10 MiB)
│   │   └── medtest.txt      (128 MiB)
│   ├── bigtest.txt          (512 MiB)
.   .
.   .
.   .

This stacks up at a total of 9.3GiB across 405 files in 150 directories.

An easy way to take care of this task in Bash could be the following:

for i in */; do zip -r "${i%/}.zip" "$i"; done

This clocks in at 1m4s total time.

Using Goroutines, Gorchive handles the exact same set of directories with default settings in only 22s.

Licensing

This project is licensed under the GNU GPLv3 License - see the LICENSE file for details

Acknowledgments

This tool uses logrus for logging and times for easy file times checks.