zerocsv

package module

v1.3.0 Latest Latest Go to latest Published: Aug 28, 2026 License: MIT Imports: 10 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/fikrimohammad/go-zerocsv

Links

Open Source Insights

README ¶

go-zerocsv

Zero-allocation, flat-memory CSV I/O for Go. go-zerocsv reads CSV faster than encoding/csv and writes with zero heap allocations per record. It reuses a single compacting buffer, so a 5M-row file maintains a constant ~5 KB memory footprint—compared to ~540 MB and 10 million heap allocations with standard encoding/csv (and 140 MB with ReuseRecord = true). Same parsing semantics, conformance-fuzzed against encoding/csv.

Features

Zero-allocation hot paths: Write() and lazy Read() parse without heap allocations via buffer reuse.
Flat-memory streaming: Reusable compacting buffer maintains a ~5 KB footprint regardless of input size.
Typed scanning: Record.Scan(&id, &name, &score) parses fields directly from byte slices without string allocations.
Custom types (Scanner & Valuer): Implement FieldScanner and FieldValuer for zero-allocation domain parsing/formatting.
Value-typed columns: Compact 48-byte Column struct passes fields by value without interface boxing.
Fuzz-tested parity: Conformance-fuzzed against encoding/csv (strict, lazy-quotes, and field counts). Zero dependencies.

Installation

Requires Go 1.18 or higher (for any and native fuzzing).

go get github.com/fikrimohammad/go-zerocsv

Quick Start

Writing

package main

import (
	"bytes"
	"fmt"

	zerocsv "github.com/fikrimohammad/go-zerocsv"
)

func main() {
	var buf bytes.Buffer
	w := zerocsv.NewWriter(&buf)

	w.Write(
		zerocsv.ColumnString("name"),
		zerocsv.ColumnInt(30),
		zerocsv.ColumnBool(true),
	)
	w.Flush()

	fmt.Print(buf.String())
	// name,30,true
}

Reading (Typed & Zero-Allocation)

package main

import (
	"fmt"
	"io"
	"log"
	"strings"

	zerocsv "github.com/fikrimohammad/go-zerocsv"
)

func main() {
	input := "id,name,score,active\n1,Alice,3.14,true\n2,Bob,2.50,false\n"
	r := zerocsv.NewReader(strings.NewReader(input))

	for {
		rec, err := r.Read()
		if err == io.EOF {
			break
		}
		if err != nil {
			log.Fatal(err)
		}
		if rec.IsFirst() {
			continue // skip header
		}

		var (
			id     int
			name   string
			score  float64
			active bool
		)
		if err := rec.Scan(&id, &name, &score, &active); err != nil {
			log.Fatal(err)
		}
		fmt.Printf("id=%d name=%s score=%.2f active=%t\n", id, name, score, active)
	}
}

Advanced Usage

Eager Reading with `ReadAll()`

To eagerly load all remaining CSV records into safe, owned Records:

records, err := r.ReadAll()
if err != nil {
	log.Fatal(err)
}
for _, rec := range records {
	fmt.Println(rec.Strings())
}

Custom options

Options are shared by the writer and reader; each applies only to the side it concerns.

// Tab-separated values.
w := zerocsv.NewWriter(&buf, zerocsv.WithDelimiter('\t'))

// CRLF line endings instead of LF.
w := zerocsv.NewWriter(&buf, zerocsv.WithCRLF())

// Tolerate malformed quoting instead of returning a parse error.
r := zerocsv.NewReader(f, zerocsv.WithLazyQuotes())

// Enforce 3 fields per record, or disable the auto-detected check:
r := zerocsv.NewReader(f, zerocsv.WithFieldsPerRecord(3))
r := zerocsv.NewReader(f, zerocsv.WithFieldsPerRecord(-1)) // variable widths

// Cap record size so an oversized record fails with ErrRecordTooLarge:
r := zerocsv.NewReader(f, zerocsv.WithMaxBufferSize(1 << 20)) // 1 MB cap
w := zerocsv.NewWriter(&buf, zerocsv.WithMaxBufferSize(1 << 20)) // 1 MB cap

// Pre-size the initial buffer for high-throughput batch ingestion or writing:
r := zerocsv.NewReader(f, zerocsv.WithBufferSize(64 << 10)) // 64 KB reader buffer
w := zerocsv.NewWriter(&buf, zerocsv.WithBufferSize(64 << 10)) // 64 KB writer buffer

// The auto-detected count is observable on both reader and writer:
w := zerocsv.NewWriter(&buf)
fmt.Println(w.FieldsPerRecord()) // 0 until the first record is written

Typed columns (Writer)

w.Write(
	zerocsv.ColumnInt64(42),
	zerocsv.ColumnFloat64(3.14),
	zerocsv.ColumnBool(true),
	zerocsv.ColumnString(time.Now().Format(time.RFC3339)),
	zerocsv.ColumnBytes([]byte("raw-bytes")),
)

Zero-allocation writing loop

Reuse one []Column slice across writes so nothing is allocated per record.

row := make([]zerocsv.Column, 3)
for _, v := range values {
	row[0] = zerocsv.ColumnString(v.Name)
	row[1] = zerocsv.ColumnInt(v.Age)
	row[2] = zerocsv.ColumnBool(v.Active)
	_ = w.Write(row...)
}

Custom Types (Scanner & Valuer)

Custom domain types can control their CSV parsing and formatting with zero allocations by implementing FieldScanner and FieldValuer:

type Date time.Time

// Reader: zero-allocation parsing from raw field bytes
func (d *Date) ScanCSV(field []byte) error {
	t, err := time.Parse("2006-01-02", string(field))
	if err != nil {
		return err
	}
	*d = Date(t)
	return nil
}

// Writer: zero-allocation formatting into writer scratch buffer
func (d Date) AppendCSV(dst []byte) ([]byte, error) {
	return time.Time(d).AppendFormat(dst, "2006-01-02"), nil
}

Usage:

// Scanning into custom type
var d Date
err := rec.Scan(&id, &name, &d)

// Writing custom type
err := w.Write(zerocsv.ColumnString("alice"), zerocsv.ColumnValuer(d))

Benchmarks

Measured on an AMD Ryzen 5 8400F (12 threads), linux/amd64, Go 1.26.5. Run them yourself with:

go test -bench=. -benchmem ./benchmark

B/op is the total bytes allocated per operation — the real memory cost. Allocation counts alone are misleading: zerocsv's few allocations are one reused buffer, while encoding/csv's many allocations are small objects that accumulate into hundreds of megabytes.

Reading — full pass over a whole file

Each iteration parses all n rows with a fresh reader; MB/s is throughput and B/op is the cumulative allocation for the whole pass.

Rows	zerocsv ns/op	zerocsv MB/s	zerocsv B/op	zerocsv allocs	stdlib (default) ns/op	stdlib B/op	stdlib allocs	stdlib (ReuseRecord) ns/op	stdlib (ReuseRecord) B/op	stdlib (ReuseRecord) allocs
100K	6.45ms	445.5	5.0 KB	12	11.33ms	10.8 MB	200,013	8.75ms	2.8 MB	100,014
500K	32.6ms	440.4	5.0 KB	12	54.54ms	54.0 MB	1,000,013	43.29ms	14.0 MB	500,014
1M	66.6ms	431.5	5.0 KB	12	109.2ms	108 MB	2,000,013	82.20ms	28.0 MB	1,000,014
5M	334.7ms	429.5	5.0 KB	12	530.4ms	540 MB	10,000,013	415.9ms	140 MB	5,000,014

zerocsv allocates a constant ~5.0 KB and 12 objects no matter how many rows are read: it reuses one small buffer that is compacted between records, so its B/op stays flat while encoding/csv's grows linearly to 540 MB at 5M rows (even with ReuseRecord = true, which reuses the outer slice header but still allocates 140 MB across 5 million field strings). A record larger than the buffer grows it on demand to fit that record, and the buffer is retained at its high-water mark so subsequent records are parsed with strictly zero allocations without memory churn. For hostile or untrusted input, WithMaxBufferSize caps the buffer so a single oversized record fails with ErrRecordTooLarge instead of growing without bound.

Writing — full pass over a whole file

Each iteration writes all n rows (6 fields each, including int, float64 and RFC3339 time) to io.Discard with a fresh writer. B/op is the cumulative allocation for the whole pass.

Rows	zerocsv ns/op	zerocsv B/op	zerocsv allocs	stdlib ns/op	stdlib B/op	stdlib allocs
100K	13.6ms	4.3 KB	6	20.1ms	4.7 MB	399,890
500K	67.3ms	4.3 KB	6	101.0ms	24.0 MB	1,999,892
1M	137.8ms	4.3 KB	6	204.4ms	47.9 MB	3,999,895
5M	708.3ms	4.3 KB	6	1,119ms	272 MB	19,999,917

zerocsv writes ~1.5x faster than encoding/csv and allocates a constant 4.3 KB (one 4 KB buffer plus a small numeric scratch) regardless of row count, whereas encoding/csv allocates ~4 objects per record for its strconv conversions — 272 MB of cumulative allocation for 5M rows.

Reading — per record

Benchmark	ns/op	B/op	allocs/op
zerocsv `Read`	50.9	19	0
`encoding/csv` `Read`	115.2	114	2
`encoding/csv` `Read` (`ReuseRecord = true`)	84.6	35	1

Writing — per record

Benchmark	ns/op	B/op	allocs/op
zerocsv `Write` (strings)	63.8	0	0
`encoding/csv` `Write` (strings)	58.2	0	0
zerocsv `Write` (mixed types)	95.7	0	0
`encoding/csv` `Write` (mixed types)	142.5	31	2
zerocsv `Write` (with time)	142.4	0	0
`encoding/csv` `Write` (with time)	200.5	54	3

For pre-formatted strings both writers are allocation-free and comparable. When values need formatting, zerocsv formats into its own scratch buffer without allocating, so the mixed and time cases stay at 0 B/op and 0 allocs/op while encoding/csv allocates for every strconv/Format call.

Variable row sizes & spike patterns

Real-world workloads (such as bank mutation feeds, transaction ledgers, and telemetry) frequently feature variable row lengths and occasional multi-kilobyte spikes.

1. Reading 1,000 Rows with Large Payload Spikes (512KB – 640KB)

Pattern Scenario	Rows Streamed	zerocsv Heap Alloc	zerocsv Allocs	stdlib Heap Alloc	stdlib Allocs	Hot-Path Allocs/Row
Case 1: Growing Rows (512KB → 524KB)	1,000	9.0 MB*	78*	545.0 MB	2,097	0.00 allocs/row
Case 2: Constant Large Rows (640KB)	1,000	2.7 MB*	19*	659.9 MB	2,089	0.00 allocs/row
Case 3: Spike Pattern (3KB, 3KB, 640KB)	1,000	2.7 MB*	20*	224.8 MB	2,047	0.00 allocs/row

*Note: The ~2.7 MB – 9.0 MB in zerocsv stems from the benchmark test generator's template strings before streaming begins. On the hot path, zerocsv retains its high-watermark buffer with strictly 0.00 allocs/record, whereas encoding/csv allocates 224 MB – 660 MB of heap strings.

2. Writing 1,000 Rows with Large Payload Spikes (512KB – 640KB)

Pattern Scenario	zerocsv Time	zerocsv Heap Alloc	zerocsv Allocs	stdlib Allocs	Hot-Path Allocs/Row
Case 1: Growing Rows (512KB → 524KB)	282.5ms	4.2 KB	5	901	0.00 allocs/row
Case 2: Constant Large Rows (640KB)	348.6ms	4.2 KB	5	901	0.00 allocs/row
Case 3: Spike Pattern (3KB, 3KB, 640KB)	119.9ms	4.2 KB	5	901	0.00 allocs/row

Real-World Impact: GC Pressure & Memory Limits (`GOMEMLIMIT`)

In containerized environments (Kubernetes, AWS ECS, Lambda), memory allocations trigger garbage collection (GC) cycles and CPU throttling. Below is the performance of streaming 3M rows under a 150 MiB memory ceiling (GOMEMLIMIT=150MiB):

1. Reading (Streaming Ingestion)

Reader	Ingestion Time	Cumulative Heap Alloc	GC Cycles Triggered	Behavior
`go-zerocsv`	267ms	~5 KB	0	Stable, flat memory
`encoding/csv (ReuseRecord=true)`	2,392ms	138 MB	14,594	GC Thrashing (9x slowdown)
`encoding/csv` (default)	1,941ms	366 MB	10,517	GC Thrashing (7x slowdown)

Because standard encoding/csv allocates millions of transient heap strings, the Go runtime repeatedly halts execution to collect dead strings to stay under the memory ceiling. go-zerocsv remains allocation-free and runs at full speed.

2. Writing (Batch Exporting with Mixed Types)

Writer	Export Time	Cumulative Heap Alloc	GC Cycles Triggered	Behavior
`go-zerocsv`	414ms	~4 KB	0	Direct buffer formatting
`encoding/csv` (with `strconv`/`Format`)	604ms	153 MB	43	GC overhead from conversions

go-zerocsv.Writer formats typed primitives (int, float, time.Time, bool) directly into its internal scratch buffer via value-typed Columns, avoiding the heap allocations required by manual strconv and time.Format calls.

Documentation

Full API documentation and runnable examples are available on pkg.go.dev.

Concurrency

Writer and Reader are stateful and not safe for concurrent use. Use a separate instance per goroutine.

Limitations & Memory Model

go-zerocsv intentionally makes trade-offs to achieve zero allocations:

Buffer aliasing on lazy Read(): Field byte slices in a lazy Record point directly into the reader's internal buffer. Data must be consumed (via Scan(), Bytes(), or String()) before the next call to Read(). Use ReadAll() if you need records that safely own their storage.
Single-byte delimiters: The delimiter is a single byte (e.g. ,, \t, |); multi-rune delimiters are not supported.
No comments or whitespace trimming: Comment lines (#) are not recognized, and leading whitespace is not automatically stripped.
Non-resumable parse errors: Once a fatal parse error occurs, Read and ReadAll continue returning it. (Field-count mismatches are non-fatal, matching encoding/csv).

Roadmap

Multi-byte & string delimiter support: Support for Unicode runes ('§', '·') and multi-character string delimiters ("||", "~|~") while retaining zero-allocation streaming.

Contributing

Fork the repository.
Create your feature branch (git checkout -b feature/amazing-feature).
Commit your changes (git commit -m 'Add some amazing feature').
Push to the branch (git push origin feature/amazing-feature).
Open a Pull Request.

Local Quality Checks

Make sure to run the linters and tests locally before submitting your code:

# Run golangci-lint locally
golangci-lint run ./...

# Run tests with the race detector enabled
go test -race -count=1 ./...

# Smoke-test the fuzzers (anchor the pattern; run in the package root)
go test -run='^$' -fuzz='^FuzzReaderConformance$' -fuzztime=30s .

License

Distributed under the MIT License. See LICENSE for more information.

Documentation ¶

Index ¶

Constants
Variables
type Column
- func (c Column) Kind() ColumnKind
type ColumnKind
type FieldScanner
type FieldValuer
type Option
type Reader
- func NewReader(r io.Reader, opts ...Option) *Reader
type Record
type Writer
- func NewWriter(w io.Writer, opts ...Option) *Writer

Constants ¶

View Source

const DefaultBufferSize = 4096

DefaultBufferSize is the initial capacity of the Reader's reusable buffer (4 KB).

View Source

const DefaultDelimiter = ','

DefaultDelimiter is the standard comma delimiter for CSV files.

Variables ¶

View Source

var (
	// ErrInvalidDelim is returned when a delimiter that would corrupt the CSV
	// structure is configured on a Writer or Reader.
	ErrInvalidDelim = errors.New("zerocsv: invalid field delimiter")

	// ErrEmptyRecord is returned when attempting to write an empty record or scan a record with no fields.
	ErrEmptyRecord = errors.New("zerocsv: empty record")

	// ErrRecordTooLarge is returned by Read or Write when a record is larger than the
	// maximum buffer size configured with WithMaxBufferSize and therefore cannot be
	// parsed or written.
	ErrRecordTooLarge = errors.New("zerocsv: record larger than the maximum buffer size")

	// ErrBareQuote is returned when a bare '"' appears in a non-quoted field.
	ErrBareQuote = errors.New("bare \" in non-quoted field")

	// ErrQuote is returned for an extraneous or missing '"' in a quoted field.
	ErrQuote = errors.New("extraneous or missing \" in quoted-field")

	// ErrFieldCount is returned by Read or Write when a record's field count does
	// not match the expected number of fields (see WithFieldsPerRecord). It is
	// non-fatal, like encoding/csv: the record is still returned or written and
	// reading or writing can continue.
	ErrFieldCount = errors.New("wrong number of fields")

	// ErrNilDestination is returned by Record.Scan when a nil destination argument is provided.
	ErrNilDestination = errors.New("cannot scan into nil destination")

	// ErrNilDestinationPointer is returned by Record.Scan when a typed pointer destination is nil.
	ErrNilDestinationPointer = errors.New("destination pointer is nil")
)

Sentinel errors returned by Reader, Writer, and Record operations.

Functions ¶

This section is empty.

Types ¶

type Column ¶

type Column struct {
	// contains filtered or unexported fields
}

Column is a tagged, value-typed CSV field. Pass it to Write by value; building and writing columns performs no heap allocation for all constructors.

func ColumnBool ¶

func ColumnBool(v bool) Column

ColumnBool returns a Column containing v, written as "true" or "false".

func ColumnBytes ¶

func ColumnBytes(b []byte) Column

ColumnBytes returns a Column containing b. The slice is written as-is, with no copy and no heap allocation.

func ColumnFloat32 ¶

func ColumnFloat32(v float32) Column

ColumnFloat32 returns a Column containing v.

func ColumnFloat64 ¶

func ColumnFloat64(v float64) Column

ColumnFloat64 returns a Column containing v.

func ColumnInt ¶

func ColumnInt(v int) Column

ColumnInt returns a Column containing v.

func ColumnInt8 ¶

func ColumnInt8(v int8) Column

ColumnInt8 returns a Column containing v.

func ColumnInt16 ¶

func ColumnInt16(v int16) Column

ColumnInt16 returns a Column containing v.

func ColumnInt32 ¶

func ColumnInt32(v int32) Column

ColumnInt32 returns a Column containing v.

func ColumnInt64 ¶

func ColumnInt64(v int64) Column

ColumnInt64 returns a Column containing v.

func ColumnString ¶

func ColumnString(s string) Column

ColumnString returns a Column containing s.

func ColumnUint ¶

func ColumnUint(v uint) Column

ColumnUint returns a Column containing v.

func ColumnUint8 ¶

func ColumnUint8(v uint8) Column

ColumnUint8 returns a Column containing v.

func ColumnUint16 ¶

func ColumnUint16(v uint16) Column

ColumnUint16 returns a Column containing v.

func ColumnUint32 ¶

func ColumnUint32(v uint32) Column

ColumnUint32 returns a Column containing v.

func ColumnUint64 ¶

func ColumnUint64(v uint64) Column

ColumnUint64 returns a Column containing v.

func ColumnUintptr ¶

func ColumnUintptr(v uintptr) Column

ColumnUintptr returns a Column containing v.

func ColumnValuer ¶ added in v1.2.0

func ColumnValuer(v FieldValuer) Column

ColumnValuer returns a Column containing v, which appends its CSV representation with zero heap allocations.

func (Column) Kind ¶ added in v1.2.0

func (c Column) Kind() ColumnKind

Kind returns the ColumnKind of c.

type ColumnKind ¶

type ColumnKind uint8

ColumnKind identifies the payload type stored in a Column.

const (
	ColumnKindString ColumnKind = iota
	ColumnKindBytes
	ColumnKindInt
	ColumnKindUint
	ColumnKindFloat
	ColumnKindFloat32
	ColumnKindBool
	ColumnKindValuer
)

type FieldScanner ¶ added in v1.2.0

type FieldScanner interface {
	ScanCSV(field []byte) error
}

FieldScanner is implemented by custom types that can scan their value directly from a raw CSV field byte slice.

type FieldValuer ¶ added in v1.2.0

type FieldValuer interface {
	AppendCSV(dst []byte) ([]byte, error)
}

FieldValuer is implemented by custom types that can append their CSV field representation directly into a scratch buffer with zero heap allocations.

type Option ¶

type Option func(*options)

Option configures a Writer or Reader at construction time.

func WithBufferSize ¶ added in v1.3.0

func WithBufferSize(n int) Option

WithBufferSize sets the initial size of the Writer or Reader's buffer in bytes. A non-positive n uses DefaultBufferSize (4096 bytes). For a Reader, if WithMaxBufferSize is also configured and n exceeds maxBuf, the initial buffer size is capped at maxBuf.

func WithCRLF ¶

func WithCRLF() Option

WithCRLF makes the Writer end each record with "\r\n" instead of "\n".

func WithDelimiter ¶

func WithDelimiter(d byte) Option

WithDelimiter sets the field delimiter, for example ',' for CSV, '\t' for TSV, or ';' for semicolon-separated values. Only single ASCII bytes are supported. The NUL byte, '"', '\r', '\n' and any byte above '\x7f' are rejected: an invalid delimiter marks a Writer or Reader as failed, and Read, ReadAll, Write, WriteAll, Flush and Error report the error.

func WithFieldsPerRecord ¶ added in v1.1.0

func WithFieldsPerRecord(n int) Option

WithFieldsPerRecord sets the expected number of fields per record, applying to both the Reader and the Writer.

If n is positive, Read, ReadAll and Write require every record to have exactly n fields and return ErrFieldCount otherwise. If n is 0, the count is taken from the first record and enforced on all subsequent ones, like encoding/csv's default. If n is negative, no check is made and records may have a variable number of fields. Blank lines read by the Reader never take part in the check.

Like encoding/csv, ErrFieldCount is non-fatal: the mismatched record is still returned (Reader) or written (Writer), and reading or writing may continue.

func WithLazyQuotes ¶

func WithLazyQuotes() Option

WithLazyQuotes makes the Reader tolerate malformed quoting: a bare '"' in an unquoted field, or a non-doubled '"' in a quoted field, is treated as a literal character instead of returning a parse error.

func WithMaxBufferSize ¶ added in v1.3.0

func WithMaxBufferSize(n int) Option

WithMaxBufferSize caps the Writer or Reader's internal buffer at n bytes. For a Reader, a record larger than n cannot be parsed in memory, so Read returns ErrRecordTooLarge rather than letting the buffer grow without bound. For a Writer, the internal buffer allocated for buffered I/O will not exceed n bytes. A non-positive n means no limit (the default).

type Reader ¶

type Reader struct {
	// contains filtered or unexported fields
}

Reader reads CSV records with zero allocations per record.

The input buffer and the field slice are allocated once in NewReader and reused for the lifetime of the Reader. Read returns records lazily one at a time with zero heap allocations, while ReadAll eagerly reads all remaining records into a slice of owned Records.

func NewReader ¶

func NewReader(r io.Reader, opts ...Option) *Reader

NewReader returns a Reader that parses CSV records from r, applying opts. An invalid delimiter marks the Reader as failed; Read, ReadAll and Error report the error.

func (*Reader) Error ¶ added in v0.2.0

func (r *Reader) Error() error

Error returns the first error encountered while reading, or nil if none has occurred. io.EOF is normal termination and is not treated as an error, so Error returns nil after a record stream has been read to completion. A Reader configured with an invalid delimiter is failed from the start.

func (*Reader) FieldsPerRecord ¶ added in v1.1.0

func (r *Reader) FieldsPerRecord() int

FieldsPerRecord returns the expected number of fields per record. It reflects the value configured with WithFieldsPerRecord: with auto-detection (the default) it is 0 until the first record is read, after which it is the field count learned from that record; a negative value means no check is in effect.

func (*Reader) Read ¶ added in v1.2.0

func (r *Reader) Read() (Record, error)

Read reads one record from r. The returned Record provides zero-allocation access to fields via Scan, or safe access via String, Bytes, and Strings.

If the record has an unexpected number of fields (see WithFieldsPerRecord), Read returns the Record along with ErrFieldCount. Like encoding/csv, this error is non-fatal: the record is usable and subsequent calls to Read continue reading the stream.

If no more records remain, Read returns a zero Record with io.EOF.

func (*Reader) ReadAll ¶ added in v1.2.0

func (r *Reader) ReadAll() ([]Record, error)

ReadAll reads all remaining records from r into a slice of Records. Each Record in the returned slice owns its field data and remains valid indefinitely. A successful call returns err == nil (io.EOF is treated as normal completion).

If an error (such as ErrFieldCount or a parse error) is encountered, ReadAll stops immediately and returns the records read so far along with the error, matching encoding/csv.ReadAll semantics.

type Record ¶

type Record struct {
	// contains filtered or unexported fields
}

Record is a single CSV record parsed by Reader.

A Record obtained from Read() provides safe, encapsulated access to the parsed fields. To achieve zero heap allocations on the hot path, field data is accessed via Scan (for typed values or reusable []byte buffers), String, Bytes, or Strings.

func (Record) Bytes ¶ added in v1.2.0

func (rec Record) Bytes(i int, dst []byte) []byte

Bytes copies field i into dst[:len(field)] and returns the subslice, growing dst only if cap(dst) is too small. If i is out of range it returns dst[:0]. When dst has sufficient capacity, Bytes performs zero heap allocations.

func (Record) Error ¶ added in v1.2.0

func (rec Record) Error() error

Error returns the record-level error (for example ErrFieldCount), or nil if the record was read without error.

func (Record) IsFirst ¶ added in v1.2.0

func (rec Record) IsFirst() bool

IsFirst reports whether this is the first data record produced by the Reader.

func (Record) Len ¶

func (rec Record) Len() int

Len returns the number of fields in the record.

func (Record) Scan ¶ added in v1.2.0

func (rec Record) Scan(dst ...any) error

Scan unpacks the record's fields into the destination pointers dst in order. The number of destinations must match Len() exactly.

Supported destination types:

*string: allocates a string (note: use *[]byte or primitives for 0 allocs)
*[]byte: copies bytes into existing slice capacity without allocation
*bool: parses bool ("true", "false", "1", "0", etc.) in-place (0 allocs)
*int, *int8, *int16, *int32, *int64: parses integer in-place (0 allocs)
*uint, *uint8, *uint16, *uint32, *uint64, *uintptr: parses unsigned integer in-place (0 allocs)
*float32, *float64: parses float in-place (0 allocs)
FieldScanner: delegates parsing to custom ScanCSV method (0 allocs)

Scan returns an error if the number of destinations does not match Len(), if any destination pointer is nil, or if parsing fails.

func (Record) String ¶ added in v1.2.0

func (rec Record) String(i int) string

String returns field i as a string. It returns "" if i is out of range. Note that constructing the return string allocates heap memory. For zero allocations on the hot path, use Scan with typed values or Bytes with a reusable slice.

func (Record) Strings ¶ added in v1.2.0

func (rec Record) Strings() []string

Strings returns all fields as a slice of newly allocated strings.

type Writer ¶

type Writer struct {
	// contains filtered or unexported fields
}

Writer writes CSV records with zero allocations per write.

The bufio.Writer and the numeric scratch buffer are allocated once in NewWriter and reused for the lifetime of the Writer. Write/WriteAll perform no heap allocations on the hot path as long as the caller reuses a []Column slice (e.g. Write(row...)) rather than passing freshly constructed variadic args.

func NewWriter ¶ added in v0.2.0

func NewWriter(w io.Writer, opts ...Option) *Writer

NewWriter returns a Writer that writes CSV records to w, applying opts. If w is already a *bufio.Writer with a buffer at least as large as the configured buffer size (default 4096 bytes), it is reused directly rather than being wrapped again.

func (*Writer) Error ¶

func (w *Writer) Error() error

Error returns the first error encountered during Write, WriteAll or Flush, or nil if none has occurred.

func (*Writer) FieldsPerRecord ¶ added in v1.1.0

func (w *Writer) FieldsPerRecord() int

FieldsPerRecord returns the expected number of fields per record, as configured with WithFieldsPerRecord. With auto-detection (the default) it is 0 until the first record is written, after which it is the field count learned from that record; a negative value means no check is in effect.

func (*Writer) Flush ¶

func (w *Writer) Flush() error

Flush writes any buffered data to the underlying writer and returns the first error encountered during Write, WriteAll or Flush, if any.

func (*Writer) Write ¶

func (w *Writer) Write(cols ...Column) error

Write writes cols as a single CSV record to the underlying writer. It returns ErrEmptyRecord if cols is empty, or the first error encountered while writing the record.

If a field count is in effect (see WithFieldsPerRecord) and cols has a different number of fields, Write returns ErrFieldCount. Like encoding/csv, the error is non-fatal: the record is still written and writing may continue.

func (*Writer) WriteAll ¶

func (w *Writer) WriteAll(rows [][]Column) error

WriteAll writes each row of rows as a CSV record to the underlying writer, flushes any buffered data, and returns the first error encountered, if any.

Source Files ¶

View all Source files

Directories ¶

Path	Synopsis
example

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL

README ¶

go-zerocsv

Features

Installation

Quick Start

Writing

Reading (Typed & Zero-Allocation)

Advanced Usage

Eager Reading with ReadAll()

Custom options

Typed columns (Writer)

Zero-allocation writing loop

Custom Types (Scanner & Valuer)

Benchmarks

Reading — full pass over a whole file

Writing — full pass over a whole file

Reading — per record

Writing — per record

Variable row sizes & spike patterns

1. Reading 1,000 Rows with Large Payload Spikes (512KB – 640KB)

2. Writing 1,000 Rows with Large Payload Spikes (512KB – 640KB)

Real-World Impact: GC Pressure & Memory Limits (GOMEMLIMIT)

1. Reading (Streaming Ingestion)

2. Writing (Batch Exporting with Mixed Types)

Documentation

Concurrency

Limitations & Memory Model

Roadmap

Contributing

Local Quality Checks

License

Documentation ¶

Index ¶

Constants ¶

Variables ¶

Functions ¶

Types ¶

type Column ¶

func ColumnBool ¶

func ColumnBytes ¶

func ColumnFloat32 ¶

func ColumnFloat64 ¶

func ColumnInt ¶

func ColumnInt8 ¶

func ColumnInt16 ¶

func ColumnInt32 ¶

func ColumnInt64 ¶

func ColumnString ¶

func ColumnUint ¶

func ColumnUint8 ¶

func ColumnUint16 ¶

func ColumnUint32 ¶

func ColumnUint64 ¶

func ColumnUintptr ¶

func ColumnValuer ¶ added in v1.2.0

func (Column) Kind ¶ added in v1.2.0

type ColumnKind ¶

type FieldScanner ¶ added in v1.2.0

type FieldValuer ¶ added in v1.2.0

type Option ¶

func WithBufferSize ¶ added in v1.3.0

func WithCRLF ¶

func WithDelimiter ¶

func WithFieldsPerRecord ¶ added in v1.1.0

func WithLazyQuotes ¶

func WithMaxBufferSize ¶ added in v1.3.0

type Reader ¶

func NewReader ¶

func (*Reader) Error ¶ added in v0.2.0

func (*Reader) FieldsPerRecord ¶ added in v1.1.0

func (*Reader) Read ¶ added in v1.2.0

func (*Reader) ReadAll ¶ added in v1.2.0

type Record ¶

func (Record) Bytes ¶ added in v1.2.0

func (Record) Error ¶ added in v1.2.0

func (Record) IsFirst ¶ added in v1.2.0

func (Record) Len ¶

func (Record) Scan ¶ added in v1.2.0

func (Record) String ¶ added in v1.2.0

func (Record) Strings ¶ added in v1.2.0

Eager Reading with `ReadAll()`

Real-World Impact: GC Pressure & Memory Limits (`GOMEMLIMIT`)