gff3

README

gff3-go

Go library for parsing and writing GFF3 (Generic Feature Format Version 3) files.

Zero external dependencies. Standard library only.

Install

go get github.com/EndCredits/gff3-go

Quick start

package main

import (
    "fmt"
    "io"
    "os"

    "github.com/EndCredits/gff3-go"
)

func main() {
    f, _ := os.Open("annotations.gff3")
    defer f.Close()

    r := gff3.NewReader(f)
    for {
        rec, err := r.Read()
        if err == io.EOF {
            break
        }
        fmt.Printf("%s\t%s\t%d\t%d\n", rec.SeqID, rec.Type, rec.Start, rec.End)
    }

    // Directives collected during parsing
    for _, d := range r.Directives() {
        fmt.Printf("##%s %v\n", d.Kind, d.Args)
    }
}

Features

• Parse GFF3 files with a streaming Reader
• Write GFF3 files with a Writer (round-trip safe)
• Percent-Encoding for file-level and column-9 reserved characters
• Attribute parsing: tag=value pairs, multi-value splitting
• Sub-parsers: Target, Gap (CIGAR-style)
• Validation: Record.Validate(), DetectCycle()
• Binary index: mmap-based O(1) lookup, spatial queries, in-memory mode (gff3idx, Unix only, separate module)

Note: The binary index package (gff3idx) uses mmap and is Unix-only (Linux, macOS). The core parser and writer have no platform restrictions.

Performance

Benchmarks on Apple M1 (single core):

File	Size	Records	Time	Throughput
A. hypogaea genome	215 MB	983,853	0.95s	226 MB/s

Reproduce with:

go run ./cmd/gff3stat/ annotations.gff3 | jq .total_records

Micro-benchmarks:

go test -bench=. -benchmem ./internal/gff3/

Validating your GFF3 files

Quick statistics (CLI)

go run ./cmd/gff3stat/ your_annotations.gff3

Outputs JSON with record counts by type, source, strand, unique seqIDs, and any parse errors:

{
  "file": "annotations.gff3",
  "total_records": 983853,
  "type_counts": {"gene": 83107, "mRNA": 83107, "exon": 417771, "CDS": 399868},
  "source_counts": {"maker": 237762, "AUGUSTUS": 157012, ...},
  "strand_counts": {"+": 490545, "-": 493308},
  "unique_seqids": 140,
  "directives": [{"kind":"gff-version","args":["3"]}],
  "errors": 0
}

Validate programmatically

f, _ := os.Open("annotations.gff3")
defer f.Close()

r := gff3.NewReader(f)
var records []*gff3.Record
for {
    rec, err := r.Read()
    if err == io.EOF { break }
    if err != nil { log.Fatal(err) }
    records = append(records, rec)
}

// Check every record
for _, rec := range records {
    if err := rec.Validate(); err != nil {
        log.Printf("invalid record: %v", err)
    }
}

// Check for circular Parent/ID relationships
if err := gff3.DetectCycle(records); err != nil {
    log.Printf("parent cycle: %v", err)
}

// Check discontiguous features
groups := gff3.GroupByID(records)
for id, recs := range groups {
    if len(recs) > 1 && recs[0].Type == "CDS" {
        log.Printf("multi-segment CDS: %s (%d segments)", id, len(recs))
    }
}

Cross-validate with Python

# install dependencies (only needed once)
pip install bcbio-gff

# line-split validation (stdlib only)
python3 scripts/validate_gff3.py your_annotations.gff3

Compares feature counts, source distribution, and strand balance against our Go parser. Use --bcbio for a second independent parser:

python3 scripts/validate_gff3.py your_annotations.gff3 --bcbio

Round-trip integrity

go test -run TestRoundTripDeepFile -args -gff3 your_annotations.gff3

Parses the first 5000 records, writes them back, re-parses, and verifies all 9 columns plus every attribute value are identical.

Binary index verification

cd gff3idx && go run ./cmd/gff3verify/ your_annotations.gff3

Builds a binary index from the GFF3 file, then compares all entries, gene hierarchies, and spatial queries against the in-memory reference. Produces VERIFIED on success.

Full integration test (Python cross-validate + index + query)

GFF3_TEST_FILE=your_annotations.gff3 go test -run TestFullBuild -timeout 120s ./gff3idx/

Parses the file, cross-validates record and type counts against Python (line-split + BCBio-GFF), builds a binary index, then verifies ByID lookup, gene children, and spatial range queries against dynamically derived expectations. No hardcoded values.

Binary index (gff3idx, Unix only)

A separate module (github.com/EndCredits/gff3-go/gff3idx) providing O(1) feature lookup and spatial interval queries. Two backends, one interface:

go get github.com/EndCredits/gff3-go/gff3idx

import "github.com/EndCredits/gff3-go/gff3idx"

// In-memory: zero build cost
q := gff3idx.Wrap(records)
feat, _ := q.ByID("Ah01g000200")

// Binary index: persistent, mmap, ~50MB resident
gff3idx.Build(records, "genes.gff3idx")
idx, _ := gff3idx.Open("genes.gff3idx")
feat, _ := idx.ByID("Ah01g000200")

// Both implement Querier — swap backends without changing code
func search(q gff3idx.Querier) { ... }

# CLI build
cd gff3idx && go run ./cmd/gff3index/ annotations.gff3 annotations.gff3idx

Method	Complexity	Description
ByID(id)	O(1)	Lookup feature by ID
ChildrenOf(geneID)	O(1) + O(n) children	Gene hierarchy: transcripts, CDSs, exons
InRange(chr, start, end)	O(log n + m), m = features with Start ≤ end	All features overlapping a genomic interval

Run unit tests

go test -cover ./internal/gff3/     # core parser
cd gff3idx && go test -cover ./...  # binary index

Documentation

• Design overview
• API reference
• Package docs (pkg.go.dev)

License

MIT

Documentation

Overview

Package gff3 provides a parser, writer, and utilities for the Generic Feature Format Version 3 (GFF3) file format.

GFF3 is a nine-column, tab-delimited plain text format used in bioinformatics to represent genomic features such as genes, exons, and coding sequences.

Basic usage

Read a GFF3 file:

r := gff3.NewReader(file)
for {
    rec, err := r.Read()
    if err == io.EOF {
        break
    }
    fmt.Printf("%s\t%s\t%d-%d\t%s\n",
        rec.SeqID, rec.Type, rec.Start, rec.End, rec.Strand)
}
for _, d := range r.Directives() {
    fmt.Printf("##%s\n", d.Kind)
}

Write a GFF3 file:

w := gff3.NewWriter(file)
w.WriteDirective(gff3.Directive{Kind: gff3.DirGFFVersion, Args: []string{"3"}})
w.WriteRecord(&gff3.Record{
    SeqID:  "chr1",
    Source: ".",
    Type:   "gene",
    Start:  1000,
    End:    9000,
    Strand: gff3.StrandPlus,
    Attributes: gff3.Attributes{"ID": {"gene1"}, "Name": {"EDEN"}},
})

Validation and utilities

if err := rec.Validate(); err != nil {
    log.Fatal(err)
}
groups := gff3.GroupByID(records)
if err := gff3.DetectCycle(records); err != nil {
    log.Fatal(err)
}

FASTA section

After the Reader encounters ##FASTA, call ReadFASTA():

for {
    seq, err := r.ReadFASTA()
    if err == io.EOF {
        break
    }
    fmt.Printf(">%s\n%s\n", seq.ID, seq.SeqString())
}

Or parse standalone FASTA:

seqs, _ := gff3.ReadAllFASTA(reader)

Alignment sub-parsers

target, _ := gff3.ParseTarget(rec.Attributes.Get("Target"))
gap, _ := gff3.ParseGap(rec.Attributes.Get("Gap"))

The package has zero external dependencies.

Index

• Constants
• Variables
• type Attributes
• type CycleError
• type Directive
• type DirectiveKind
• type FastaRecord
• type GapOp
• type LineType
• type Reader
• type Record
• type SequenceRegion
• type Target
• type Writer

Constants

const (
	DirGFFVersion        = gff3.DirGFFVersion
	DirSequenceRegion    = gff3.DirSequenceRegion
	DirFeatureOntology   = gff3.DirFeatureOntology
	DirAttributeOntology = gff3.DirAttributeOntology
	DirSourceOntology    = gff3.DirSourceOntology
	DirSpecies           = gff3.DirSpecies
	DirGenomeBuild       = gff3.DirGenomeBuild
	DirTerminator        = gff3.DirTerminator
	DirFASTA             = gff3.DirFASTA
	DirUnknown           = gff3.DirUnknown
)

Directive kind constants.

const (
	LineDirective = gff3.LineDirective
	LineComment   = gff3.LineComment
	LineFeature   = gff3.LineFeature
	LineBlank     = gff3.LineBlank
	LineFASTA     = gff3.LineFASTA
)

Line type constants.

const (
	StrandPlus    = gff3.StrandPlus
	StrandMinus   = gff3.StrandMinus
	StrandNone    = gff3.StrandNone
	StrandUnknown = gff3.StrandUnknown
)

Strand constants.

const PhaseUndefined = gff3.PhaseUndefined

PhaseUndefined is the sentinel value for features without a phase (non-CDS).

Variables

var DetectCycle = gff3.DetectCycle

DetectCycle checks for circular Parent/ID relationships.

Returns CycleError if a cycle is found, nil otherwise.

var Escape = gff3.Escape

Escape encodes file-level reserved characters using Percent-Encoding.

Escapes: tab, newline, carriage return, %, and control characters.

var EscapeAttr = gff3.EscapeAttr

EscapeAttr encodes both file-level and column-9 reserved characters.

In addition to file-level escaping, also escapes ; = & , which have reserved meanings in GFF3 column 9. Use this when writing attribute values.

var GroupByID = gff3.GroupByID

GroupByID groups records by their ID attribute.

Records without an ID are excluded. Returns a map from ID to all records sharing that ID (discontiguous features).

var NewReader = gff3.NewReader

NewReader creates a GFF3 Reader from an io.Reader.

var NewWriter = gff3.NewWriter

NewWriter creates a GFF3 Writer writing to an io.Writer.

var ParseAttributes = gff3.ParseAttributes

ParseAttributes parses a column 9 attributes string into a tag→values map.

The format is tag=value pairs separated by semicolons. Multiple values for the same tag are separated by commas (only for Parent, Alias, Note, Dbxref, and Ontology_term). Percent-encoded characters are decoded after splitting on reserved delimiters.

var ParseDirective = gff3.ParseDirective

ParseDirective parses a ##-prefixed directive line.

var ParseGap = gff3.ParseGap

ParseGap parses a Gap attribute value into a slice of GapOp.

Format: space-separated (code,length) pairs, e.g. "M8 D3 M6".

var ParseSequenceRegion = gff3.ParseSequenceRegion

ParseSequenceRegion parses a ##sequence-region directive into a SequenceRegion.

var ParseTarget = gff3.ParseTarget

ParseTarget parses a Target attribute value.

Format: target_id start end [strand]. Strand defaults to "+" if omitted.

var ReadAllFASTA = gff3.ReadAllFASTA

ReadAllFASTA reads all FASTA sequences from an io.Reader.

The reader is assumed to contain only FASTA-formatted data (lines starting with > followed by sequence lines).

var Unescape = gff3.Unescape

Unescape decodes GFF3 Percent-Encoding according to RFC 3986.

Types

type Attributes

type Attributes = gff3.Attributes

Attributes holds the parsed tag=value pairs from column 9.

Reserved tags: ID, Name, Alias, Parent, Target, Gap, Derives_from, Note, Dbxref, Ontology_term, Is_circular.

Multi-value tags (Parent, Alias, Note, Dbxref, Ontology_term) store comma-separated values as individual strings in the slice.

type CycleError

type CycleError = gff3.CycleError

CycleError indicates a circular Parent/ID relationship among features.

type Directive

type Directive = gff3.Directive

Directive represents a ##-prefixed pragma line.

The Kind field identifies the directive type. Args contains the space-separated arguments following the keyword.

type DirectiveKind

type DirectiveKind = gff3.DirectiveKind

DirectiveKind identifies the type of a ## directive line.

type FastaRecord

type FastaRecord = gff3.FastaRecord

FastaRecord is a single FASTA sequence record.

Sequence bases are stored upper-cased with whitespace removed.

type GapOp

type GapOp = gff3.GapOp

GapOp is a single operation in a Gap (CIGAR-style) attribute.

M — match
I — insert gap into reference
D — delete from reference (gap in target)
F — frameshift forward
R — frameshift reverse

type LineType

type LineType = gff3.LineType

LineType distinguishes between kinds of GFF3 lines.

type Reader

type Reader = gff3.Reader

Reader reads GFF3 feature records from an io.Reader.

Reader skips blank lines and comments. Directives are collected internally and available via Directives(). When a ##FASTA directive or an implicit > line is encountered, Read() returns io.EOF and subsequent FASTA sequences can be read via ReadFASTA().

The first non-blank, non-comment line must be a ##gff-version directive. A ### (terminator) directive stops reading immediately.

type Record

type Record = gff3.Record

Record represents a single GFF3 feature line (9 columns).

Column 1: seqid     — landmark ID (chromosome, scaffold)
Column 2: source    — algorithm or database name
Column 3: type      — feature type (gene, mRNA, exon, CDS, ...)
Column 4: start     — 1-based start coordinate
Column 5: end       — 1-based end coordinate
Column 6: score     — floating point or "."
Column 7: strand    — +, -, ., or ?
Column 8: phase     — 0, 1, 2 (CDS), or PhaseUndefined
Column 9: attributes — tag=value pairs parsed into Attributes

type SequenceRegion

type SequenceRegion = gff3.SequenceRegion

SequenceRegion holds the parsed ##sequence-region directive.

Format: ##sequence-region seqid start end

type Target

type Target = gff3.Target

Target holds the parsed Target attribute for alignment features.

Format: target_id start end [strand]

type Writer

type Writer = gff3.Writer

Writer writes GFF3 formatted records and directives to an io.Writer.

Output is round-trip safe: records written and re-parsed produce identical values.