gozim

module
v0.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 20, 2026 License: MIT

README

GoZIM

Minimal OpenZIM compliant implementation in Pure Go with zero dependencies and with Go stdlib's archive/* and compress/* interface compatibility.

Usage

# Inspect a ZIM file with structured debug output
gozim trace wikipedia.zim;

# Print file metadata: version, entries, checksum, namespace scheme
gozim info wikipedia.zim;

# List all entries by namespace+path order
gozim dump wikipedia.zim --list;

# Show entry details: path, title, MIME type, size
gozim dump wikipedia.zim;

# Filter by namespace (C=content, A=articles, I=images, M=metadata)
gozim dump wikipedia.zim --ns=C;

# Full-text search with BM25 ranking
gozim search wikipedia.zim "quantum physics" --limit 20;

# Extract archive contents to a directory
gozim unpack wikipedia.zim --output ./wiki/;

# Pack a directory into a ZIM archive
gozim pack ./html-content --output archive.zim --title "My Wiki" --language eng;

# Verify file integrity: checksum, broken links, redirect loops, redundancy
gozim check wikipedia.zim;

# Print version
gozim version;

GoZIM API

package main

import (
    "fmt"
    "github.com/cookiengineer/gozim/archive/zim"
)

func main() {

    // Read ZIM file
    archive, _ := zim.Open("wikipedia.zim")
    defer archive.Close()

    fmt.Println("entries:", archive.EntryCount())
    fmt.Println("uuid:", archive.Uuid())

    // Look up entry by path
    entry, _ := archive.EntryByPath("index.html")
    item, _ := entry.Item(false)
    data, _ := item.DataAll()
    fmt.Printf("content: %d bytes, MIME: %s\n", len(data), item.MimeType())

    // Resolve a redirect
    if entry.IsRedirect() {
        target, _ := entry.RedirectEntry()
        fmt.Println("redirects to:", target.Path())
    }

    // Iterate all entries
    for entry := range archive.IterateByPath() {
        fmt.Println(entry.Path(), entry.Title())
    }

    // Read metadata
    if title, ok := archive.Metadata("Title"); ok {
        fmt.Println("title:", title)
    }

    // Write ZIM file
    writer := zim.NewWriter().
        SetCompression(zim.CompressionZstd).
        SetClusterSize(2 * zim.Megabyte).
        SetIndexing(true, "eng").
        SetMainPath("index.html")

    writer.Create("output.zim")

    // Add content items
    item1 := zim.NewStringItem("page.html", "text/html", "My Page", "<html>...</html>")
    writer.AddItem(item1)

    // Add metadata
    writer.AddMetadata("Title", "My Wiki")
    writer.AddMetadata("Language", "eng")

    // Add redirect
    writer.AddRedirection("old.html", "Old Page", "new.html")

    // Add a favicon
    pngData, _ := os.ReadFile("favicon.png")
    writer.AddIllustration(48, pngData)

    writer.Finish()

    // Search entries
    searcher := zim.NewSearcher(map[string]*zim.Archive{"wikipedia.zim": archive})
    results, _ := searcher.Search("quantum physics", 0, 20)
    for _, r := range results.Results {
        fmt.Printf("%s (%s, score: %.2f)\n", r.Title, r.File, r.Score)
        fmt.Println(r.Snippet)
    }

    // Integrity checks
    report, _ := zim.CheckFile("wikipedia.zim", zim.CheckAll)
    fmt.Println("checksum OK:", report.ChecksumOK)
    fmt.Println("broken links:", len(report.BrokenLinks))
    fmt.Println("redirect loops:", len(report.RedirectLoops))

}

GoZIM's Compression Packages

import "github.com/cookiengineer/gozim/compress/lzma"
import "github.com/cookiengineer/gozim/compress/xz"
import "github.com/cookiengineer/gozim/compress/zstd"

// zstd
var buf bytes.Buffer
encoder, _ := zstd.NewWriter(&buf)
encoder.Write(data)
encoder.Close()
decoder, _ := zstd.NewReader(&buf)
decompressed, _ := io.ReadAll(decoder)
decoder.Close()

// xz
xzReader, _ := xz.NewReader(compressedReader)
data, _ := io.ReadAll(xzReader)

// lzma
lzmaWriter, _ := lzma.NewWriter(&buf)
lzmaWriter.Write(data)
lzmaWriter.Close()

Unit Tests

The library uses Go's standard test format for unit tests and integration tests.

cd /path/to/gozim;

# Run the test suite
go test -v ./...;

The current test coverage is:

Package Coverage Tests
archive/zim 52.3% 99
archive/zim/glass 67.6% 40
compress/lzma 67.9% 65
compress/lzma/hash 100.0% 25
compress/xz 70.8% 22
compress/zstd 43.6% 64
compress/zstd/huff0 51.5% 22
compress/zstd/huff0/fse 0.0% 0
compress/zstd/le 0.0% 0
compress/zstd/snapref 100.0% 5
compress/zstd/xxhash 100.0% 21
Total 47.0% 363

Specification Compliance Tests

Each package has a specification compliance test suite, and the specifications are included in the repository in the specifications folder for future reference.

License

This library is licensed under the X11/MIT License

Directories

Path Synopsis
archive
zim
Package zim implements reading and writing of ZIM files, the open file format used to store wiki content for offline viewing.
 Package zim implements reading and writing of ZIM files, the open file format used to store wiki content for offline viewing.
zim/bm25
Package bm25 implements the BM25 ranking function for full-text search within ZIM archives.
 Package bm25 implements the BM25 ranking function for full-text search within ZIM archives.
zim/glass
Package glass implements a pure Go reader and writer for the Xapian Glass database backend used by libzim for full-text indexing within ZIM files.
 Package glass implements a pure Go reader and writer for the Xapian Glass database backend used by libzim for full-text indexing within ZIM files.
cmds
gozim command
compress
lzma
Package lzma implements reading and writing of LZMA and LZMA2 compressed data.
 Package lzma implements reading and writing of LZMA and LZMA2 compressed data.
lzma/hash
Package hash provides rolling hash functions used by the LZMA compressor to maintain a sliding window of byte sequences.
 Package hash provides rolling hash functions used by the LZMA compressor to maintain a sliding window of byte sequences.
xz
Package xz implements reading and writing of XZ compressed data.
 Package xz implements reading and writing of XZ compressed data.
zstd
Package zstd provides Zstandard (zstd) compression and decompression.
 Package zstd provides Zstandard (zstd) compression and decompression.
zstd/huff0
Package huff0 provides fast Huffman encoding and decoding for the Zstandard literals section.
 Package huff0 provides fast Huffman encoding and decoding for the Zstandard literals section.
zstd/huff0/fse
Package fse implements Finite State Entropy encoding and decoding.
 Package fse implements Finite State Entropy encoding and decoding.
zstd/le
Package le provides functions to load 32-bit and 64-bit little-endian integers from byte slices at arbitrary (possibly unaligned) offsets.
 Package le provides functions to load 32-bit and 64-bit little-endian integers from byte slices at arbitrary (possibly unaligned) offsets.
zstd/snapref
Package snapref is a stub implementation of Snappy decompression for reading snappy-encoded blocks within Zstandard frames.
 Package snapref is a stub implementation of Snappy decompression for reading snappy-encoded blocks within Zstandard frames.
zstd/xxhash

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.