Help us shape the future of Go! Take the 2026 Go Developer Survey

agc

package module
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 1, 2026 License: MIT Imports: 14 Imported by: 0

README

go-agc

A pure-Go reader for Assembled Genomes Compressor (AGC) v3 archives.

This repository was developed with substantial coding assistance from OpenAI Codex and Claude Code, which helped with implementation, refactoring, tests, documentation, and benchmarking under human direction and review.

The read-only library opens local files or arbitrary io.ReaderAt backends, validates file-format major version 3, lists samples and contigs, retrieves named contigs, and iterates one sample or the complete archive. Metadata and sequence streams are decoded lazily with bounded working memory. Other AGC file-format major versions are intentionally rejected.

archive, err := agc.Open("genomes.agc")
if err != nil {
	return err
}
defer archive.Close()

samples, err := archive.Samples()
reference, err := archive.ReferenceSample()
contigs, err := archive.Contigs(reference)

reader, err := archive.NewContigReader(reference)
for {
	contig, err := reader.Read()
	if errors.Is(err, io.EOF) {
		break
	}
	if err != nil {
		return err
	}
	consume(contig.Name, contig.Sequence)
}

err = archive.IterateSample(reference, func(contig agc.Contig) error {
	return consume(contig.Name, contig.Sequence)
})

Archives exposed by an HTTP server with byte-range support can be opened directly:

archive, err := agc.OpenURL(ctx, "https://example.org/genomes.agc", http.DefaultClient)
if err != nil {
	return err
}
defer archive.Close()

err = archive.IterateSample(agc.Sample{Name: "isolate-42"}, consumeContig)

Remote reads are pinned to a strong ETag when the server provides one. For sample iteration, required reference and delta parts are deduplicated, nearby file ranges are coalesced with bounded over-fetch, and remaining requests are issued concurrently. By default, objects up to 16 MiB are downloaded in one full-range request after the range-support probe. Larger objects remain range-backed; if a sample needs at least 80% of an object that fits the 64 MiB remote cache, the reader switches to one full request. The range defaults are a 64 KiB merge gap, an 8 MiB maximum request, and eight concurrent requests.

Use OpenURLWithOptions and HTTPOptions to tune those limits for a server or archive collection. A negative WholeObjectThreshold or WholeObjectFraction disables the corresponding full-download rule. Open and ordinary OpenReaderAt inputs retain their existing demand-driven local-file path and do not use remote prefetching.

The implementation is read-only and lazily reads indexed archive parts. It uses github.com/klauspost/compress/zstd and does not require cgo or the AGC executable. Tests include the upstream toy archive, generated v3 fixtures, corruption cases, optional reference-CLI cross-checks, and benchmarks.

Releases

This is a library-only Go module, so releases are semantic-version Git tags; there are no executable artifacts to build. See RELEASING.md for the tagging workflow and consumer installation command.

Documentation

Overview

Package agc reads Assembled Genomes Compressor (AGC) v3 archives.

Archives are decoded lazily from an io.ReaderAt. The package is read-only.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrClosed is returned when an operation uses a closed archive.
	ErrClosed = errors.New("agc: archive is closed")
	// ErrCorruptArchive marks malformed, truncated, or inconsistent input.
	ErrCorruptArchive = errors.New("agc: corrupt archive")
	// ErrUnsupportedVersion marks an AGC archive whose file-format major version is not 3.
	ErrUnsupportedVersion = errors.New("agc: unsupported file-format version")
	// ErrSampleNotFound marks a lookup for a sample not present in the archive.
	ErrSampleNotFound = errors.New("agc: sample not found")
	// ErrContigNotFound marks a lookup for a contig not present in the selected sample.
	ErrContigNotFound = errors.New("agc: contig not found")
)

Functions

This section is empty.

Types

type Archive 

type Archive struct {
	// contains filtered or unexported fields
}

Archive is an open, read-only AGC v3 archive.

func Open 

func Open(path string) (*Archive, error)

Open opens a local AGC v3 archive. Close releases the underlying file.

func OpenReaderAt 

func OpenReaderAt(r io.ReaderAt, size int64) (*Archive, error)

OpenReaderAt opens an AGC v3 archive from a ReaderAt and its exact size. The caller retains ownership of r; Archive.Close does not close it.

func OpenURL added in v0.4.0 

func OpenURL(ctx context.Context, rawURL string, client *http.Client) (*Archive, error)

OpenURL opens an AGC v3 archive through HTTP byte-range requests. The context remains in force for the lifetime of the archive. A nil client uses http.DefaultClient. The server must support single byte ranges.

func OpenURLWithOptions added in v0.4.0 

func OpenURLWithOptions(ctx context.Context, rawURL string, options HTTPOptions) (*Archive, error)

OpenURLWithOptions opens an HTTP byte-range-backed AGC v3 archive using the supplied retrieval policy.

func (*Archive) Close 

func (a *Archive) Close() error

Close closes the archive. It is safe to call more than once.

func (*Archive) Contig 

func (a *Archive) Contig(sample Sample, name string) (Contig, error)

Contig retrieves and decodes a named contig from a sample. Name matching follows AGC: the query and stored header are compared up to the first ASCII whitespace, while the returned Contig.Name retains the full stored header.

func (*Archive) Contigs 

func (a *Archive) Contigs(sample Sample) ([]ContigInfo, error)

Contigs returns a sample's contigs in archive order. Each result retains its sample identity separately from its stored contig name.

func (*Archive) IterateAll 

func (a *Archive) IterateAll(yield func(Contig) error) error

IterateAll decodes every contig in archive sample order and contig order, stopping on the first error returned by yield.

func (*Archive) IterateSample 

func (a *Archive) IterateSample(sample Sample, yield func(Contig) error) error

IterateSample decodes a sample's contigs in archive order, holding at most one yielded contig's sequence at a time. Iteration stops on the first error returned by yield.

func (*Archive) NewContigReader added in v0.2.0 

func (a *Archive) NewContigReader(sample Sample) (*ContigReader, error)

NewContigReader returns a pull-style reader for one sample. Read holds at most one returned contig's decoded sequence at a time.

func (*Archive) ReferenceSample 

func (a *Archive) ReferenceSample() (Sample, error)

ReferenceSample returns the first sample stored in the v3 collection.

func (*Archive) Samples 

func (a *Archive) Samples() ([]Sample, error)

Samples returns samples in archive order. The reference sample is first.

func (*Archive) Version 

func (a *Archive) Version() Version

Version returns the validated AGC file-format version.

type Contig 

type Contig struct {
	Sample   Sample
	Name     string
	Sequence []byte
}

Contig contains one decoded contig.

type ContigInfo 

type ContigInfo struct {
	Sample Sample
	Name   string
}

ContigInfo identifies a contig and its owning sample separately.

type ContigReader added in v0.2.0 

type ContigReader struct {
	// contains filtered or unexported fields
}

ContigReader reads one sample's contigs in archive order. A reader is not safe for concurrent use, although independent readers may share an Archive.

func (*ContigReader) Read added in v0.2.0 

func (r *ContigReader) Read() (Contig, error)

Read decodes and returns the next contig, or io.EOF after the sample's final contig.

type HTTPOptions added in v0.4.0 

type HTTPOptions struct {
	// Client performs HTTP requests. A nil client uses http.DefaultClient.
	Client *http.Client
	// ReadAheadBytes is fetched for small demand reads. The default is 64 KiB.
	ReadAheadBytes int64
	// MergeGapBytes is the largest gap included while joining nearby ranges.
	// The default is 64 KiB.
	MergeGapBytes int64
	// MaxMergedRangeBytes caps one joined range. The default is 8 MiB.
	MaxMergedRangeBytes int64
	// CacheBytes bounds compressed remote range data. The default is 64 MiB.
	CacheBytes int64
	// Concurrency caps simultaneous range requests. The default is 8.
	Concurrency int
	// WholeObjectThreshold downloads objects at or below this size in one
	// full-range request after the range-support probe. The default is 16 MiB;
	// a negative value disables it.
	WholeObjectThreshold int64
	// WholeObjectFraction downloads the whole object when uncached planned
	// ranges reach this fraction and the object fits CacheBytes. The default is
	// 0.80; a negative value disables it.
	WholeObjectFraction float64
}

HTTPOptions controls remote archive retrieval. Zero-valued fields select defaults. Set WholeObjectThreshold or WholeObjectFraction to a negative value to disable that whole-object optimization.

type Sample 

type Sample struct {
	Name string
}

Sample identifies a sample independently from any of its contigs.

type Version 

type Version struct {
	Major uint32
	Minor uint32
}

Version is the AGC file-format version, not the producer's software version.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.