lzo1z

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 9, 2026 License: GPL-2.0 Imports: 1 Imported by: 0

README

lzo1z

CI Go Reference

LZO1Z decompression in pure Go. No cgo, no liblzo2 at runtime, nothing outside the standard library. Builds on Go 1.21+.

go get github.com/bandari-abhilash/LZO-GO

License: GPL-2.0-or-later. This is a derivative of Markus Oberhumer's GPL-licensed LZO library, so it cannot be anything more permissive. Go links statically, so importing this package makes your whole binary GPL when you distribute it. See Licensing before you depend on it.

LZO1Z is not LZO1X

This matters more than anything else in this README. LZO1Z shares LZO1X's block format but uses an M2 match offset of 0x0700 instead of 0x0800. The two are not interchangeable — decoding LZO1X data with this package gives you either an error or silently wrong bytes.

If your producer calls lzo1x_*, you want an LZO1X decoder. This package is for data produced by lzo1z_*, which shows up in a handful of exchange market data feeds and older embedded protocols.

Decompression only. There's no compressor here; use liblzo2 to produce data.

Usage

import lzo1z "github.com/bandari-abhilash/LZO-GO"

dst := make([]byte, maxDecompressedSize)

n, err := lzo1z.Decompress(src, dst)
if err != nil {
    return err
}
use(dst[:n])

The caller sizes dst. LZO carries no length header, so you must know the decompressed bound out of band — from your protocol's framing, or a conservative ceiling (64 KiB is typical for feed blocks).

Decompress never grows dst and never writes past it. An undersized buffer is a clean ErrOutputOverrun, not a corruption:

error meaning
ErrInputOverrun input ended mid-token — truncated or corrupt
ErrOutputOverrun dst too small; nothing was written past the end
ErrLookbehind match offset points before the start of output
ErrNotConsumed trailing bytes after the end-of-stream marker

Every malformed-input path is bounds-checked. TestDecompressTruncatedInput feeds every possible truncation of a real vector and requires an error from each one.

Performance

go test -bench . on an M-series Mac, single core:

BenchmarkDecompress/text-10        3954 ns/op    2020 MB/s    0 allocs/op
BenchmarkDecompress/records-10    14889 ns/op    1558 MB/s    0 allocs/op
BenchmarkDecompress/large-10      28471 ns/op    2107 MB/s    0 allocs/op

Zero allocations — the decoder writes only into the dst you hand it.

Test vectors

testdata/ holds .orig/.lzo1z pairs covering empty-ish inputs (1, 4 and 18 bytes), long-match text, a 16 KiB zero run, incompressible random data, fixed-width records, mixed-entropy runs, and a 60 KB buffer that exercises long match distances.

They are reproducible. tools/genvectors/genvectors.c regenerates the whole corpus with liblzo2's own lzo1z_999_compress:

cd tools/genvectors
cc -O2 -o genvectors genvectors.c -llzo2      # brew install lzo / apt install liblzo2-dev
./genvectors ../../testdata
go test ./...

Corpora come from a fixed LCG seed, so the .orig files are byte-identical on every machine; the .lzo1z files depend on your liblzo2 version, which is fine — the tests assert round-trip correctness, not specific compressed bytes.

Provenance

A direct port of lzo1z_decompress from LZO 2.10 (lzo1x_d.ch with the LZO1Z branches, M2_MAX_OFFSET = 0x0700), validated against vectors generated by liblzo2 2.10 itself.

Licensing

Because Go produces statically linked binaries, there is no LGPL-style "link dynamically and stay proprietary" option. If you import this package into a program and distribute that program, the whole program must be released under the GPL with source.

Two things that are not triggers: running it on your own servers without shipping binaries (GPL-2 has no network clause), and using it internally.

If you need LZO1Z in a proprietary product, the original author sells commercial LZO licenses — contact him rather than using this port.

Documentation

Overview

Package lzo1z implements LZO1Z decompression in pure Go, with no cgo and no dependencies outside the standard library.

LZO1Z is one variant of Markus Oberhumer's LZO family. It shares LZO1X's block format but uses a 0x0700 M2 match offset instead of 0x0800, so the two are NOT interchangeable: feeding LZO1X data to this decoder yields either an error or silent corruption. If your producer calls lzo1x_* you want an LZO1X decoder, not this one.

Decompression only. There is no compressor here; use liblzo2 if you need to produce LZO1Z data.

dst := make([]byte, maxDecompressedSize)
n, err := lzo1z.Decompress(src, dst)
if err != nil {
    return err
}
use(dst[:n])

The caller sizes dst. Decompress never grows it and returns ErrOutputOverrun rather than writing past the end, so an undersized buffer is a clean error and not a corruption. Every malformed-input path is checked: truncated input, output overrun, and a match offset pointing before the start of the output all return an error instead of reading or writing out of bounds.

Provenance

This is a direct port of lzo1z_decompress from LZO 2.10 (lzo1x_d.ch with the LZO1Z branches, M2_MAX_OFFSET = 0x0700). Because it is a derivative of GPL code it is distributed under the GNU General Public License v2 or later; see the LICENSE file. Commercial licensing of LZO is available separately from the original author.

Correctness is pinned against vectors produced by liblzo2's own lzo1z_999_compress; see tools/genvectors to regenerate them.

Index

Examples

Constants

This section is empty.

Variables

View Source
var (
	ErrInputOverrun  = errors.New("lzo1z: input overrun")
	ErrOutputOverrun = errors.New("lzo1z: output overrun")
	ErrLookbehind    = errors.New("lzo1z: lookbehind overrun")
	ErrNotConsumed   = errors.New("lzo1z: input not fully consumed")
)

Functions

func Decompress

func Decompress(src, dst []byte) (int, error)

Decompress decompresses one LZO1Z block from src into dst and returns the number of bytes written.

dst must be large enough to hold the entire decompressed block; LZO carries no length header, so the caller must know the bound out of band. Too small a dst returns ErrOutputOverrun with no partial success. Any trailing bytes in src after the end-of-stream marker return ErrNotConsumed.

Example
package main

import (
	"errors"
	"fmt"

	lzo1z "github.com/bandari-abhilash/LZO-GO"
)

func main() {
	// "ABCDEFGHIJKLMNOPQR" as compressed by liblzo2's lzo1z_999_compress.
	src := []byte{
		0x23, 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M',
		'N', 'O', 'P', 'Q', 'R', 0x11, 0x00, 0x00,
	}

	dst := make([]byte, 64*1024)
	n, err := lzo1z.Decompress(src, dst)
	if err != nil {
		fmt.Println(err)
		return
	}
	fmt.Printf("%s\n", dst[:n])

	// An undersized dst is a clean error, never an out-of-bounds write.
	_, err = lzo1z.Decompress(src, make([]byte, 4))
	fmt.Println(errors.Is(err, lzo1z.ErrOutputOverrun))
}
Output:
ABCDEFGHIJKLMNOPQR
true

Types

This section is empty.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL