Documentation
¶
Overview ¶
Package extract unpacks downloaded archives (zip, rar incl. multi-volume, 7z incl. .001 volumes, tar, and the gzip/bzip2/xz/zstd single-stream formats whether or not they wrap a tar) into a folder next to the archive. Pure Go, no external binaries.
Which reader opens a file is decided by its magic bytes and only then by its name, and zip joins rar and 7z in taking a password: both WinZip AES and the legacy ZipCrypto. That half lives in format.go.
Index ¶
- Constants
- Variables
- func Collisions() []collide.Policy
- func Formats() []string
- func ParseCollision(s string) collide.Policy
- func SetKey(name string) (key string, ok bool)
- func SplitPart(name string) (stem string, part int, ok bool)
- func SplitStart(name string) bool
- func Startable(name string) bool
- func Supported(name string) bool
- func SweepTrash(root string, maxAge time.Duration) (int, error)
- func VolumeRank(name string) int
- type Disposal
- type Options
- type Outcome
- type Progress
- type Request
- type Result
Constants ¶
const DefaultCollision = collide.Overwrite
DefaultCollision is what an extraction does when its destination folder is already there, absent a setting.
Overwrite, deliberately, even though collide calls it the one policy that can lose data. It is what extraction has always done here - every file is opened O_TRUNC - so it is the only value that leaves an upgraded install behaving as it did yesterday. It is also the only one that survives the ordinary reason an extraction is run a second time, a password supplied on the second attempt, without leaving a second copy of a large folder behind.
const DefaultDepth = 4
DefaultDepth is how many archives deep a job follows when the caller does not say. Deep extraction is the norm in the wild - a release is a multi-volume rar holding a zip holding the files - but an archive that unpacks to a copy of itself is a disk filled in silence, so there is always a floor.
const DefaultDisposal = DisposalKeep
DefaultDisposal is what an unset setting means. Keeping is the only one of the three that cannot destroy something the user wanted, and the archive is the one copy of bytes they paid for in bandwidth.
const DefaultTrashDays = 14
DefaultTrashDays is how long a trashed archive survives. Two weeks is longer than it takes to notice that an extraction produced the wrong thing, and shorter than it takes for a disk to fill with archives nobody chose to keep.
const TrashName = ".knightloader-trash"
TrashName is the folder a trashed archive is moved into, directly under the download folder.
A hidden folder, and not a recycle bin, because there is no recycle bin to put anything in. A container has no desktop session, no trash daemon and no XDG user directories, and the host's own recycle bin sits on the far side of a bind mount where a rename cannot reach it. So "trash" here means exactly one thing, a rename into this folder plus the age-based sweep below, and the help text beside the setting has to say so in as many words. A recycle bin that is really a hidden folder is a promise broken quietly, which is the worst way to break one: nobody finds out until they go looking for the file.
Variables ¶
var ErrDestinationTaken = errors.New("extract: the destination already exists and the collision policy is to skip")
ErrDestinationTaken says an extraction did not run because its destination was already there and the policy is to skip. Nothing is wrong with the archive and nothing was written, so a caller reporting failures should put this one in different words from a broken file.
var ErrFormatRetired = errors.New("extract: retired archive format")
ErrFormatRetired says the archive is in a format KnightLoader will not read. It is a different thing from "unsupported archive", which means "I do not recognise this file": here the file was recognised perfectly well and turned down, and the user is owed the reason rather than a shrug.
var ErrPasswordRequired = errors.New("extract: the archive is encrypted and no password fits")
ErrPasswordRequired says the archive is encrypted and none of the supplied passwords opened it. It is a distinct error so the app can tell "I need a password" from "this archive is broken".
Functions ¶
func Collisions ¶
Collisions lists the policies an extraction can honour, in the order a chooser should offer them.
It is collide's own list filtered through ParseCollision, never a second list written out here. A policy that package gains appears in the menu only once this one has learnt to keep it, and a policy it drops leaves the menu by itself. That is the direction that fails safe: a hard-coded copy fails the other way round, by offering a word the extractor quietly does something else about.
func Formats ¶
func Formats() []string
Formats lists the archive extensions this build actually opens, one spelling per format, for the capability line the settings page shows.
Derived, never written down. A list typed into the interface drifts from the readers the moment one is added or retired, and the drift is invisible until somebody's download is refused by a page that promised to handle it. Every candidate here comes out of the extractor's own suffix table and is then put through Supported, the same gate that decides whether a finished download is offered to the unpacker at all, so an entry can only appear if a real file of that name would really be taken.
The table is longest-suffix-first, so that ".tar.gz" beats ".gz" when a name is being shortened. That is the wrong end for a capability line, where ".gz" is the spelling that names the format, so the walk runs backwards and takes the shortest spelling of each; the result is turned round again to read in table order.
func ParseCollision ¶
ParseCollision folds a stored policy onto one an extraction can honour.
collide.Ask is the one that has to go. It parks a task until a human answers, and an extraction has no way to raise the question and no state to sit in while it waits, so honouring it would mean an unpack that never finishes and never says why.
func SetKey ¶
SetKey identifies the volume set a file belongs to: every part of one multi-volume archive returns the same key. ok is false for names that cannot be part of a set (a .tar.gz, a plain file). A single-file archive is simply a set of one.
func SplitPart ¶
SplitPart reports the split set a numbered part belongs to, and which part it is. It is false for anything that is not a part, a split 7z included.
func SplitStart ¶
SplitStart reports whether name is the part a join starts from. Both spellings count: HJSplit numbers from 001 and `split -d` from 000.
func Startable ¶
Startable reports whether a job can be started on this file: an archive the readers can open, or the first part of a plain split file, which becomes one once it has been put back together.
func Supported ¶
Supported reports whether name looks like an archive this package can start extracting (first volume of a set, or a single archive).
func SweepTrash ¶
SweepTrash removes everything in root's trash folder that has been there longer than maxAge, and reports how many entries went.
A zero maxAge never sweeps, which is what "keep it until I say so" has to mean. A root with no trash folder is not an error either: there is nothing to sweep, and reporting that as a failure would put a red mark on every extraction of an install that has never trashed anything.
func VolumeRank ¶
VolumeRank orders the parts of one archive set the way the readers consume them, which is not the order their names sort in.
Two families disagree with each other, and both disagree with a plain sort: a spanned rar starts at "film.rar" and continues into "film.r00", while a spanned zip ends at "film.zip" and starts at "film.z01". Sorted by name the rar set puts its last part first and the zip set puts its last part in the middle, so a list numbering the volumes off the sort would tell the user part 1 of 5 is the part the archive finishes with.
Types ¶
type Disposal ¶
type Disposal string
Disposal is what happens to an archive whose extraction succeeded.
Three answers and not a boolean. "Keep" and "delete" were the only two on offer for a long time, and the gap between them is where everybody actually lives: the archive is forty gigabytes and the extraction looks right, but "looks right" is not "is right", and a delete cannot be taken back by changing the setting afterwards.
const ( // DisposalKeep leaves the archive where it is. DisposalKeep Disposal = "keep" // DisposalTrash moves it into the trash folder, from where the sweep takes // it once it is old enough. See TrashName for what that word can honestly // mean here. DisposalTrash Disposal = "trash" // DisposalDelete removes it at once and for good. DisposalDelete Disposal = "delete" )
func Disposals ¶
func Disposals() []Disposal
Disposals lists the values in the order a chooser should offer them, so the API, the settings page and ParseDisposal cannot drift apart.
func ParseDisposal ¶
ParseDisposal folds stored or user-supplied text onto a known value.
Anything unrecognised becomes DefaultDisposal rather than an error, and the case that matters is not a typo: it is a settings file written by a build older than this one, where this key was a boolean and here it is a word. settings.Load maps that file on the way in; this is the second net, and it errs towards keeping the archive.
type Options ¶
type Options struct {
// Passwords are tried in order when the archive turns out to be encrypted.
Passwords []string
// Dest collects extractions in one folder instead of leaving each beside
// its archive. Empty means beside the archive, which is what this package
// has always done. It is an absolute path with any template already
// expanded: this package never sees a task, so it could not expand one.
Dest string
// Package is the package the archive belongs to, for Subfolder.
Package string
// Subfolder puts each package in its own folder below Dest.
//
// It does nothing without Dest, and that is not an oversight: beside the
// archive, the download folder already is the package folder whenever
// subfolder-by-package is on for downloads, and a second one inside it
// would give "Films/Film/Film".
Subfolder bool
// Collision is what to do when the destination folder already exists. It is
// decided per folder and never per file inside the archive - see
// destination.
Collision collide.Policy
// Disposal is what happens to the archive volumes afterwards.
Disposal Disposal
// TrashRoot is the folder DisposalTrash creates its TrashName folder in,
// normally the download folder. Empty falls back to the archive's own
// folder, so trash degrades to a different folder and never to a delete.
TrashRoot string
// TrashMaxAge is how long a trashed file survives a sweep. Zero never
// sweeps, which is a real answer for somebody who tidies up by hand.
TrashMaxAge time.Duration
// InfoFiles sweeps the .nfo/.sfv/.diz/.url that came with the same package.
InfoFiles bool
}
Options is the archive half of the settings, plus the two facts about the download that only the caller knows.
func (Options) Dispose ¶
Dispose applies the disposal to files the caller has already decided may go.
A file that is no longer there is not an error: two volumes of one set can resolve to the same path, and a user removing the archive by hand between the extraction and this call is an ordinary thing to do.
func (Options) Extract ¶
Extract unpacks path under these options and reports what it wrote.
It disposes of nothing. Whether a volume is still somebody else's file is a question about the download list, which this package cannot see and must not guess at; the caller narrows the list and calls Dispose.
func (Options) InfoFilesIn ¶
InfoFilesIn narrows one package's own files down to the info files among them: the .nfo, the .sfv, the .diz, the .url advert.
The caller passes the files of ONE package, and that is the entire safety property of this sweep. The obvious implementation - list the folder and delete every .nfo in it - is a data-loss bug on the default layout, because subfolder-by-package is off out of the box and one folder therefore holds several releases. A neighbour's .nfo is not ours to remove, and nobody would ever connect its disappearance with having unpacked something else.
It also answers nothing at all while the disposal is Keep. The sweep has no disposal of its own: whether a swept file is trashed or deleted follows what was chosen for the archive, so "keep everything" cannot coherently mean "keep the archive and destroy the notes beside it".
type Outcome ¶
type Outcome struct {
// Dir is where the archive the job was started on unpacked to.
Dir string
// Dirs is every directory the job wrote into, the nested ones included.
Dirs []string
// Files and Bytes are the whole job's output.
Files int
Bytes int64
// Volumes is every archive file consumed at every depth, and the parts of
// every split file joined. It is what "delete the archive afterwards" acts
// on, so a nested archive left sitting inside the output is not something
// the caller has to go looking for.
Volumes []string
// Joined names the files put back together from a split set.
Joined []string
// Nested counts the archives found inside the output and unpacked in turn.
Nested int
}
Outcome is what a finished job produced.
func Run ¶
Run unpacks the archive at req.Path, follows the archives inside what it unpacked, and reports what it did.
A job that does not finish - a failure, or a caller cancelling the context - takes its own work back off the disk before it returns. That is the whole reason this exists as a job rather than a call: a half-written extraction folder is indistinguishable from a finished one. Nothing on disk says which, the person looking at it calls the download good, and the next deep pass walks into it.
type Progress ¶
type Progress struct {
// Archive is the base name of the file currently open, which for a deep
// extraction is not the one the job was started on.
Archive string `json:"archive"`
// Depth is how far inside the outermost archive that file was found; 0 is
// the archive the job started with.
Depth int `json:"depth"`
// Files and Bytes count everything this job has written, at every depth.
Files int `json:"files"`
Bytes int64 `json:"bytes"`
}
Progress is a snapshot of a job in flight, in the terms the list shows it in.
type Request ¶
type Request struct {
// Path is the archive to open: the first volume of a set, or the first part
// of a plain split file.
Path string
// Options is where it writes, what it does about a destination that is
// already there, and which passwords it tries.
//
// Dest and Subfolder apply to the archive the job was started on and to
// nothing found inside it. A collected extraction folder is a place to put
// downloads, not a place to flatten a release into: three nested archives
// all named "Setup.zip" would unpack over each other there, while beside
// their own parents they stay three separate folders.
Options Options
// Depth is how many archives deep to follow. Zero means DefaultDepth; one
// means "unpack what I named and nothing found inside it".
Depth int
// OnProgress is called on the goroutine running the job, no more often than
// reportEvery. It must not block: everything waiting on the extraction is
// waiting behind it.
OnProgress func(Progress)
}
Request is one unpacking as the worker takes it on.
type Result ¶
type Result struct {
Dir string // directory the archive was extracted into
Files int // number of files written
Volumes []string // all volume files consumed (for delete-after-extract)
}
Result describes a finished extraction.
func Extract ¶
Extract unpacks the archive at path into a sibling directory named after the archive. It refuses to write outside that directory (zip-slip safe).
func ExtractWith ¶
ExtractWith is Extract with passwords to try, in order, if the archive turns out to be encrypted. The unencrypted attempt always comes first, so a normal archive never pays for the list.
It is Options.Extract with every option left at its default: beside the archive, overwrite what is there, keep the archive afterwards. The walk lives with the options in options.go rather than being written out twice, because a second copy of "try each password in turn" is a second place to forget that a failed attempt has to leave the destination untouched.
Source Files