multiclean

Documentation

Overview

multiclean.go cleans up fragments from failed S3 multipart-uploads.

For normal cleanup of ingested tar files from the partner's receiving buckets, see cleanup.go.

Usage:

go run multiclean.go -config=dev
go run multiclean.go -config=dev -delete

The first example just lists all of the unrecoverable parts of failed multipart uploads and prints the total number of unreachable bytes we're paying for.

The second version actually deletes the unrecoverable bytes.

Be careful using -delete when the bag processor is running, because you may wind up deleting parts of a multipart upload that the bag processor is actively uploading. To be safe, use the -delete flag only when you know the bag processor is not running.