Documentation
¶
Overview ¶
Package magic identifies the type of a file or of bytes, as the file command does. It is a port of libmagic, the library of file, to pure Go (D22). It gives the output of libmagic 5.48 and the commits of file after it (D48), and needs no C compiler.
r, err := magic.Detect(ctx, buf)
if err != nil {
return err
}
fmt.Println(r.MIME) // image/gif
fmt.Println(r.Description) // GIF image data, version 89a, 1375 x 825
fmt.Println(r.Extensions) // [gif]
The rules of file are compiled into the package (D35). A caller cannot add rules. Every function is safe to call from many goroutines at once.
Index ¶
- Constants
- Variables
- type Check
- type Match
- type Option
- type Result
- func Detect[S Source](ctx context.Context, src S, opts ...Option) (Result, error)
- func DetectFile(ctx context.Context, name string, opts ...Option) (Result, error)
- func DetectReader(ctx context.Context, r io.Reader, opts ...Option) (Result, error)
- func DetectReaderAt(ctx context.Context, r io.ReaderAt, size int64, opts ...Option) (Result, error)
- type Source
Examples ¶
Constants ¶
const FileVersion = "5.48"
FileVersion is the version of file whose behavior the package copies. The package follows the commits of file after the release, to the commit that docs/upstream/upstreams.tsv names (D48), and file still calls that 5.48.
Variables ¶
var ErrStopped = errors.New("magic: the detection stopped")
ErrStopped is the error of a detection that stopped because its context is done. The error also wraps the error of the context.
Functions ¶
This section is empty.
Types ¶
type Match ¶ added in v0.5.0
type Match struct {
Description string
// MIME, Extensions and Apple are set for a match after the first only
// when libmagic gives them for each match, as it does when every match
// has one.
MIME string
Extensions []string
Apple string
}
Match is one match of the rules, or of a built-in test: one line that file prints with its option --keep-going.
type Option ¶ added in v0.5.0
type Option func(*options) error
Option changes how a call detects.
func WithAll ¶ added in v0.5.0
func WithAll() Option
WithAll keeps every match, as MAGIC_CONTINUE does. Description then holds each match on a line of its own, and Matches holds one Match for each.
func WithDecompress ¶ added in v0.5.0
func WithDecompress() Option
WithDecompress identifies what is inside compressed data, as MAGIC_COMPRESS does. It decodes the formats whose decoder a program imports from github.com/xo/magic/compress (D45). magic runs no outside program (D36).
func WithDevices ¶ added in v0.5.0
func WithDevices() Option
WithDevices makes DetectFile read a device, as MAGIC_DEVICES does.
func WithFollowSymlinks ¶ added in v0.5.0
func WithFollowSymlinks() Option
WithFollowSymlinks makes DetectFile identify the target of a link, as MAGIC_SYMLINK does. By default a link is named as a link.
func WithLimit ¶ added in v0.5.0
WithLimit sets how many bytes are read and examined. The default is 7 MiB, as in libmagic.
func WithRaw ¶ added in v0.5.0
func WithRaw() Option
WithRaw keeps characters that cannot be printed, as MAGIC_RAW does. By default they are written as octal escapes, as file writes them.
func WithoutCheck ¶ added in v0.5.0
WithoutCheck turns off one built-in test, as a MAGIC_NO_CHECK_* flag does.
type Result ¶ added in v0.5.0
type Result struct {
// Description is the text that file prints, such as
// "GIF image data, version 89a, 1375 x 825". With WithAll, it holds each
// match on a line of its own, after the first with the prefix "- ".
Description string
// MIME is the MIME type, such as "image/gif".
MIME string
// Charset is the MIME encoding, such as "binary", "us-ascii" or
// "utf-8".
Charset string
// Extensions are the usual extensions without the dot, such as
// ["jpeg" "jpg" "jpe" "jfif"], or nil when no rule gives one.
Extensions []string
// Apple is the creator and type code of classic Mac OS, such as
// "8BIMGIFf", or "" when no rule gives one.
Apple string
// Matches holds every match when WithAll is set, the first one included.
// Without WithAll it holds the one match that the fields above describe.
Matches []Match
}
Result is what libmagic says of the content, in every form at once.
func Detect ¶ added in v0.5.0
Detect identifies src. Bytes that match nothing are "data", and no bytes are "empty". It returns an error when ctx is done or an option is not valid, such as a negative limit. It does not keep src, and does not change it.
Example ¶
package main
import (
"context"
"fmt"
"os"
"github.com/xo/magic"
)
func main() {
buf, err := os.ReadFile("testdata/gif.gif")
if err != nil {
fmt.Println(err)
return
}
r, err := magic.Detect(context.Background(), buf)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(r.Description)
fmt.Println(r.ContentType())
fmt.Println(r.Extensions)
}
Output: GIF image data, version 89a, 1375 x 825 image/gif; charset=binary [gif]
func DetectFile ¶ added in v0.5.0
DetectFile identifies the file at name. A directory, a link, a pipe or a device is named by its kind, as file names it. A file that cannot be found or opened is an error, which wraps the error of the os package.
Example ¶
package main
import (
"context"
"fmt"
"github.com/xo/magic"
)
func main() {
r, err := magic.DetectFile(context.Background(), "testdata")
if err != nil {
fmt.Println(err)
return
}
fmt.Println(r.Description)
fmt.Println(r.MIME)
}
Output: directory inode/directory
func DetectReader ¶ added in v0.5.0
DetectReader reads from r up to the limit (WithLimit), and identifies what it read. A stream has no end that a rule can read relative to, so a rule that reads from the end of the file sees the end of what was read. Use DetectReaderAt when the size is known.
func DetectReaderAt ¶ added in v0.5.0
DetectReaderAt identifies the size bytes that r holds. It reads at the offsets that the rules ask for, including offsets from the end, and reads at most the limit from the start and the limit from the end. An *os.File, a *bytes.Reader and a *strings.Reader are each an io.ReaderAt.
func (Result) ContentType ¶ added in v0.5.0
ContentType returns MIME and Charset in the form of an HTTP header, such as "image/gif; charset=binary".
Directories
¶
| Path | Synopsis |
|---|---|
|
Package cfb reads Compound Files, the format of Microsoft that Office 97 to 2003, Outlook, Windows Installer and encrypted Office documents use (D62).
|
Package cfb reads Compound Files, the format of Microsoft that Office 97 to 2003, Outlook, Windows Installer and encrypted Office documents use (D62). |
|
cmd
|
|
|
file
command
Command file is a small example of the package magic: it prints the type of each file that it gets, as the file command does.
|
Command file is a small example of the package magic: it prints the type of each file that it gets, as the file command does. |
|
walk
command
Command walk is a small example of the package walk: it prints the files that are embedded in each file that it gets, as binwalk does.
|
Command walk is a small example of the package walk: it prints the files that are embedded in each file that it gets, as binwalk does. |
|
Package compress is the registry of the decoders that magic uses to look inside compressed data.
|
Package compress is the registry of the decoders that magic uses to look inside compressed data. |
|
adc
Package adc decodes ADC, the Apple Data Compression of older DMG files.
|
Package adc decodes ADC, the Apple Data Compression of older DMG files. |
|
bzip2
Package bzip2 registers the decoder of bzip2 data with github.com/xo/magic/compress, as the library bzlib does in libmagic.
|
Package bzip2 registers the decoder of bzip2 data with github.com/xo/magic/compress, as the library bzlib does in libmagic. |
|
huffcdic
Package huffcdic decodes the Huffman compression of Mobipocket, which MOBI books give as compression type 17480 in their header.
|
Package huffcdic decodes the Huffman compression of Mobipocket, which MOBI books give as compression type 17480 in their header. |
|
lzfse
Package lzfse decodes LZFSE, the compression of Apple.
|
Package lzfse decodes LZFSE, the compression of Apple. |
|
lzx
Package lzx decodes LZX, the compression of Microsoft that CAB archives and CHM help files use.
|
Package lzx decodes LZX, the compression of Microsoft that CAB archives and CHM help files use. |
|
palmdoc
Package palmdoc decodes the compression of PalmDOC, which the text records of PalmDOC and MOBI books use.
|
Package palmdoc decodes the compression of PalmDOC, which the text records of PalmDOC and MOBI books use. |
|
zlib
Package zlib registers the decoders of gzip and zlib data with github.com/xo/magic/compress, as the library zlib does in libmagic.
|
Package zlib registers the decoders of gzip and zlib data with github.com/xo/magic/compress, as the library zlib does in libmagic. |
|
xz
module
|
|
|
zstd
module
|
|
|
internal
|
|
|
engine
Package engine is the port of the engine of libmagic 5.48: it runs the compiled rules and the detectors on a buffer or a file, and gives the text that libmagic gives (D22, D34).
|
Package engine is the port of the engine of libmagic 5.48: it runs the compiled rules and the detectors on a buffer or a file, and gives the text that libmagic gives (D22, D34). |
|
rules
Package rules compiles the rules of file, the text of magic/Magdir, into the entries that the engine of the port matches (D35).
|
Package rules compiles the rules of file, the text of magic/Magdir, into the entries that the engine of the port matches (D35). |
|
table/gen
command
|
|
|
libmagic
|
|
|
darwin_amd64
module
|
|
|
darwin_arm64
module
|
|
|
freebsd_amd64
module
|
|
|
linux_amd64
module
|
|
|
linux_arm
module
|
|
|
linux_arm64
module
|
|
|
windows_amd64
module
|
|
|
windows_arm64
module
|
|
|
Package walk finds the files that are embedded in a binary file, such as a firmware image, as binwalk does (D31, D49, D50).
|
Package walk finds the files that are embedded in a binary file, such as a firmware image, as binwalk does (D31, D49, D50). |
|
format/all
Package all imports every format of the walker.
|
Package all imports every format of the walker. |
|
format/archive
Package archive registers the archives that binwalk-ng finds: ZIP, POSIX tar, CPIO, 7-Zip, ARJ, Microsoft Cabinet and RAR.
|
Package archive registers the archives that binwalk-ng finds: ZIP, POSIX tar, CPIO, 7-Zip, ARJ, Microsoft Cabinet and RAR. |
|
format/compressed
Package compressed registers the compressed data that binwalk-ng finds: gzip, zlib, bzip2, xz, lzma, zstd, the zlib data of a GPG signed file, LZ4, lzop, the LZW data of compress, and LZFSE.
|
Package compressed registers the compressed data that binwalk-ng finds: gzip, zlib, bzip2, xz, lzma, zstd, the zlib data of a GPG signed file, LZ4, lzop, the LZW data of compress, and LZFSE. |
|
format/crypto
Package crypto registers the keys, certificates and tables of cryptography that binwalk-ng finds: PEM certificates and keys, RSA session keys, PKCS DER hashes, DPAPI blobs, LUKS headers, the tables of AES, and the constants of CRC32, SHA256 and MD5.
|
Package crypto registers the keys, certificates and tables of cryptography that binwalk-ng finds: PEM certificates and keys, RSA session keys, PKCS DER hashes, DPAPI blobs, LUKS headers, the tables of AES, and the constants of CRC32, SHA256 and MD5. |
|
format/disk
Package disk registers the disk images and the file systems that binwalk-ng finds: MBR, EFI GPT, FAT, NTFS, EXT, BTRFS, APFS, LogFS, QCOW, DMG and Android sparse images.
|
Package disk registers the disk images and the file systems that binwalk-ng finds: MBR, EFI GPT, FAT, NTFS, EXT, BTRFS, APFS, LogFS, QCOW, DMG and Android sparse images. |
|
format/doc
Package doc registers the documents and the text that binwalk-ng finds: PDF, HP PJL, copyright text, Pcap-NG captures and Motorola S-records.
|
Package doc registers the documents and the text that binwalk-ng finds: PDF, HP PJL, copyright text, Pcap-NG captures and Motorola S-records. |
|
format/encrypted
Package encrypted decrypts the firmware images of D-Link that binwalk decrypts, with the keys that binwalk carries in the crate delink (D51).
|
Package encrypted decrypts the firmware images of D-Link that binwalk decrypts, with the keys that binwalk carries in the crate delink (D51). |
|
format/exec
Package exec registers the executables, kernels and boot images that binwalk-ng finds: ELF, PE, DirectX shader bytecode, Windows CE images, the VxWorks WIND kernel and symbol table, the eCos exception handler, the Linux kernel version string, the Linux boot images for x86, ARM and ARM64, Android boot images, the U-Boot version string and Fritz!Box EVA kernel images.
|
Package exec registers the executables, kernels and boot images that binwalk-ng finds: ELF, PE, DirectX shader bytecode, Windows CE images, the VxWorks WIND kernel and symbol table, the eCos exception handler, the Linux kernel version string, the Linux boot images for x86, ARM and ARM64, Android boot images, the U-Boot version string and Fritz!Box EVA kernel images. |
|
format/filesystem
Package filesystem registers the file systems that binwalk-ng finds and extracts: RomFS, which binwalk-ng extracts with its own code, SquashFS, which binwalk-ng extracts with sasquatch, JFFS2, which binwalk-ng extracts with jefferson, YAFFS2, which binwalk-ng extracts with unyaffs, CramFS and ISO 9660, which binwalk-ng extracts with 7z, and UBI and UBIFS, which binwalk-ng extracts with ubireader_extract_images and ubireader_extract_files.
|
Package filesystem registers the file systems that binwalk-ng finds and extracts: RomFS, which binwalk-ng extracts with its own code, SquashFS, which binwalk-ng extracts with sasquatch, JFFS2, which binwalk-ng extracts with jefferson, YAFFS2, which binwalk-ng extracts with unyaffs, CramFS and ISO 9660, which binwalk-ng extracts with 7z, and UBI and UBIFS, which binwalk-ng extracts with ubireader_extract_images and ubireader_extract_files. |
|
format/firmware
Package firmware registers the firmware headers that binwalk-ng finds: uImage, TRX, DLOB, PackImg, CHK, SEAMA, CFE, TP-Link, TP-Link RTOS, JBOOT ARM, JBOOT STAG, JBOOT SCH2, BIN header, RTK, PCH ROM, UEFI volumes and capsules, QNX IFS, Broadcom ProgramStore, and the device tree blob.
|
Package firmware registers the firmware headers that binwalk-ng finds: uImage, TRX, DLOB, PackImg, CHK, SEAMA, CFE, TP-Link, TP-Link RTOS, JBOOT ARM, JBOOT STAG, JBOOT SCH2, BIN header, RTK, PCH ROM, UEFI volumes and capsules, QNX IFS, Broadcom ProgramStore, and the device tree blob. |
|
format/image
Package image registers the images that binwalk finds: PNG, JPEG, GIF, BMP, RIFF and SVG.
|
Package image registers the images that binwalk finds: PNG, JPEG, GIF, BMP, RIFF and SVG. |
|
format/internal/jbootsig
Package jbootsig holds the parser of the JBOOT ARM header of binwalk, which the signatures jboot_arm and dlke share.
|
Package jbootsig holds the parser of the JBOOT ARM header of binwalk, which the signatures jboot_arm and dlke share. |
|
format/internal/zipsig
Package zipsig holds the parser of the ZIP signature of binwalk-ng, which the signatures zip and dahua_zip share.
|
Package zipsig holds the parser of the ZIP signature of binwalk-ng, which the signatures zip and dahua_zip share. |
|
format/vendor
Package vendor registers the firmware images of vendors, most of them encrypted or obfuscated, that binwalk finds: OpenSSL encryption, Arcadyan obfuscated LZMA, D-Link MH01, Autel, DMS, Matter OTA, Dahua ZIP, CSman DAT, known encrypted firmware, D-Link encrpted_img, D-Link TLV, D-Link firmware with a header of the model, DLKE, SHRS and DKBS.
|
Package vendor registers the firmware images of vendors, most of them encrypted or obfuscated, that binwalk finds: OpenSSL encryption, Arcadyan obfuscated LZMA, D-Link MH01, Autel, DMS, Matter OTA, Dahua ZIP, CSman DAT, known encrypted firmware, D-Link encrpted_img, D-Link TLV, D-Link firmware with a header of the model, DLKE, SHRS and DKBS. |
|
internal/binwalk
Package binwalk holds the helpers that the parsers of binwalk share: the reader of structures, the check of offsets, and the readers of strings.
|
Package binwalk holds the helpers that the parsers of binwalk share: the reader of structures, the check of offsets, and the readers of strings. |
|
internal/bzip2
Package bzip2 is the bzip2 decoder of the Go standard library, changed for the binary walker: it stops at the end of the first stream, as libbz2 stops, and Decode reports the bytes of the stream.
|
Package bzip2 is the bzip2 decoder of the Go standard library, changed for the binary walker: it stops at the end of the first stream, as libbz2 stops, and Decode reports the bytes of the stream. |
|
internal/flate
Package flate is the inflater of the Go standard library, changed for the binary walker to decode as miniz_oxide does, the inflater of binwalk.
|
Package flate is the inflater of the Go standard library, changed for the binary walker to decode as miniz_oxide does, the inflater of binwalk. |
|
internal/lz4
Package lz4 decodes blocks of LZ4.
|
Package lz4 decodes blocks of LZ4. |
|
internal/lzo
Package lzo decodes LZO1X, as lzo1x_decompress_safe of the LZO library.
|
Package lzo decodes LZO1X, as lzo1x_decompress_safe of the LZO library. |