README
¶
go-zerocsv
go-zerocsv is a zero-allocation CSV reader and writer for Go. It reads and writes CSV records without heap-allocating per record, so throughput stays high and memory use stays flat no matter how large the file is.
Features
- Zero allocations on the hot path: each
Writeand eachNextparses without heap allocation; the buffer and field slices are allocated once and reused. - Zero dependencies: relies exclusively on the Go standard library.
- Fuzz-tested parity: reader output is conformance-fuzzed against
encoding/csvin both strict and lazy-quote modes. - Flat memory: the reader holds one reusable buffer regardless of input size, so a 5M-row file costs roughly as much memory as a 100-row one.
Installation
Requires Go 1.18 or higher (for any and native fuzzing).
go get github.com/fikrimohammad/go-zerocsv
Quick Start
Writing
package main
import (
"bytes"
"fmt"
zerocsv "github.com/fikrimohammad/go-zerocsv"
)
func main() {
var buf bytes.Buffer
w := zerocsv.NewWriter(&buf)
w.Write(
zerocsv.ColumnString("name"),
zerocsv.ColumnInt(30),
zerocsv.ColumnBool(true),
)
w.Flush()
fmt.Print(buf.String())
// name,30,true
}
Reading
package main
import (
"fmt"
"io"
"log"
"strings"
zerocsv "github.com/fikrimohammad/go-zerocsv"
)
func main() {
input := "name,age\nAlice,30\nBob,40\n"
r := zerocsv.NewReader(strings.NewReader(input))
for {
rec, err := r.Next()
if err == io.EOF {
break
}
if err != nil {
log.Fatal(err)
}
for i := 0; i < rec.Len(); i++ {
fmt.Printf("%d:%s ", i, rec.ValueAt(i))
}
fmt.Println()
}
}
Advanced Usage
Custom options
Options are shared by the writer and reader; each applies only to the side it concerns.
// Tab-separated values.
w := zerocsv.NewWriter(&buf, zerocsv.WithDelimiter('\t'))
// CRLF line endings instead of LF.
w := zerocsv.NewWriter(&buf, zerocsv.WithCRLF())
// Tolerate malformed quoting instead of returning a parse error.
r := zerocsv.NewReader(f, zerocsv.WithLazyQuotes())
Typed columns
w.Write(
zerocsv.ColumnInt64(42),
zerocsv.ColumnFloat64(3.14),
zerocsv.ColumnTime(time.Now(), time.RFC3339),
zerocsv.ColumnBytes([]byte("raw")),
)
Zero-allocation writing loop
Reuse one []Column slice across writes so nothing is allocated per record.
row := make([]zerocsv.Column, 3)
for _, v := range values {
row[0] = zerocsv.ColumnString(v.Name)
row[1] = zerocsv.ColumnInt(v.Age)
row[2] = zerocsv.ColumnBool(v.Active)
_ = w.Write(row...)
}
Zero-copy reading
ValueAt returns a zero-copy view into the reader's reused buffer. The value
is valid only until the next Next call. Convert it to a string to own the
data, since string([]byte) copies:
for {
rec, err := r.Next()
if err == io.EOF {
break
}
if err != nil {
log.Fatal(err)
}
name := string(rec.ValueAt(0)) // owned copy, safe to retain
_ = name
}
Benchmarks
Measured on an AMD Ryzen 5 8400F (12 threads), linux/amd64, Go 1.26.5. Run them yourself with:
go test -bench=. -benchmem ./benchmark
Reading — full pass over a whole file
Each iteration parses all n rows with a fresh reader; MB/s is throughput.
| Rows | zerocsv ns/op | stdlib ns/op | zerocsv MB/s | stdlib MB/s | zerocsv allocs | stdlib allocs |
|---|---|---|---|---|---|---|
| 100K | 9.00ms | 10.90ms | 319.6 | 263.8 | 8 | 200,013 |
| 500K | 45.4ms | 51.9ms | 316.7 | 277.2 | 8 | 1,000,013 |
| 1M | 91.6ms | 102.9ms | 313.8 | 279.5 | 8 | 2,000,013 |
| 5M | 446.6ms | 516.9ms | 321.9 | 278.1 | 8 | 10,000,013 |
The reader allocates a constant 8 objects (internal buffer growth) regardless
of how many rows are read, while encoding/csv allocates 2 objects per record
(10M allocations for 5M rows, ~540 MB of garbage). Memory use stays flat
because the reader keeps one reusable buffer; stdlib's footprint grows with
file size.
Writing — full pass over a whole file
Each iteration writes all n rows (6 fields each, including int, float64
and RFC3339 time) to io.Discard with a fresh writer.
| Rows | zerocsv ns/op | stdlib ns/op | zerocsv allocs | stdlib allocs |
|---|---|---|---|---|
| 100K | 14.1ms | 19.9ms | 5 | 399,890 |
| 500K | 71.5ms | 100.9ms | 5 | 1,999,890 |
| 1M | 143.2ms | 205.3ms | 5 | 3,999,890 |
| 5M | 735.0ms | 1,062ms | 5 | 19,999,891 |
zerocsv writes ~1.4x faster than encoding/csv and allocates a constant
5 objects (buffer and scratch growth) no matter how many rows are written,
whereas encoding/csv allocates ~4 objects per record for its strconv
conversions.
Reading — per record
| Benchmark | ns/op | B/op | allocs/op |
|---|---|---|---|
zerocsv Next |
64.0 | 18 | 0 |
encoding/csv Read |
115.1 | 114 | 2 |
Writing — per record
| Benchmark | ns/op | B/op | allocs/op |
|---|---|---|---|
zerocsv Write (strings) |
67.2 | 0 | 0 |
encoding/csv Write (strings) |
57.1 | 0 | 0 |
zerocsv Write (mixed types) |
99.3 | 0 | 0 |
encoding/csv Write (mixed types) |
141.7 | 31 | 2 |
zerocsv Write (with time) |
148.0 | 0 | 0 |
encoding/csv Write (with time) |
202.5 | 54 | 3 |
For pre-formatted strings both writers are allocation-free and comparable.
When values need formatting, zerocsv formats into its own scratch buffer
without allocating, so the mixed and time cases stay at 0 allocs/op while
encoding/csv allocates for every strconv/Format call.
Documentation
Full API documentation and runnable examples are available on pkg.go.dev.
Concurrency
Writer and Reader are stateful and not safe for concurrent use. Use a
separate instance per goroutine.
Contributing
- Fork the repository.
- Create your feature branch (
git checkout -b feature/amazing-feature). - Commit your changes (
git commit -m 'Add some amazing feature'). - Push to the branch (
git push origin feature/amazing-feature). - Open a Pull Request.
Local Quality Checks
Make sure to run the linters and tests locally before submitting your code:
# Run golangci-lint locally
golangci-lint run ./...
# Run tests with the race detector enabled
go test -race -count=1 ./...
# Smoke-test the fuzzers (anchor the pattern; run in the package root)
go test -run='^$' -fuzz='^FuzzReaderConformance$' -fuzztime=30s .
License
Distributed under the MIT License. See LICENSE for more information.
Documentation
¶
Index ¶
- Variables
- type Column
- func ColumnAny(v any) Column
- func ColumnBool(v bool) Column
- func ColumnBytes(b []byte) Column
- func ColumnFloat32(v float32) Column
- func ColumnFloat64(v float64) Column
- func ColumnInt(v int) Column
- func ColumnInt8(v int8) Column
- func ColumnInt16(v int16) Column
- func ColumnInt32(v int32) Column
- func ColumnInt64(v int64) Column
- func ColumnString(s string) Column
- func ColumnTime(t time.Time, layout string) Column
- func ColumnUint(v uint) Column
- func ColumnUint8(v uint8) Column
- func ColumnUint16(v uint16) Column
- func ColumnUint32(v uint32) Column
- func ColumnUint64(v uint64) Column
- func ColumnUintptr(v uintptr) Column
- type ColumnKind
- type Option
- type Reader
- type Record
- type Writer
Constants ¶
This section is empty.
Variables ¶
var ErrBareQuote = errors.New("bare \" in non-quoted field")
ErrBareQuote is returned when a bare '"' appears in a non-quoted field.
var ErrEmptyRecord = errors.New("zerocsv: empty record")
ErrEmptyRecord is returned by Write when no columns are provided.
var ErrInvalidDelim = errors.New("zerocsv: invalid field delimiter")
ErrInvalidDelim is returned when a delimiter that would corrupt the CSV structure is configured on a Writer or Reader.
var ErrQuote = errors.New("extraneous or missing \" in quoted-field")
ErrQuote is returned for an extraneous or missing '"' in a quoted field.
Functions ¶
This section is empty.
Types ¶
type Column ¶
type Column struct {
// contains filtered or unexported fields
}
Column is a tagged, value-typed CSV field. Pass it to Write by value; it holds no pointers that escape, so building and writing columns performs no heap allocation.
func ColumnAny ¶
ColumnAny returns a Column containing v, rendered with fmt.Sprint when the record is written. Unlike the typed constructors, this may allocate.
func ColumnBool ¶
ColumnBool returns a Column containing v, written as "true" or "false".
func ColumnBytes ¶
ColumnBytes returns a Column containing b. The slice is written as-is, with no copy.
func ColumnFloat32 ¶
ColumnFloat32 returns a Column containing v.
func ColumnFloat64 ¶
ColumnFloat64 returns a Column containing v.
func ColumnTime ¶
ColumnTime returns a Column containing t, formatted with layout when the record is written.
func ColumnUintptr ¶
ColumnUintptr returns a Column containing v.
type Option ¶
type Option func(*options)
Option configures a Writer or Reader at construction time.
func WithCRLF ¶
func WithCRLF() Option
WithCRLF makes the Writer end each record with "\r\n" instead of "\n".
func WithDelimiter ¶
WithDelimiter sets the field delimiter, for example ',' for CSV, '\t' for TSV, or ';' for semicolon-separated values. The NUL byte, '"', '\r' and '\n' are rejected: an invalid delimiter marks a Writer or Reader as failed, and Next, Write, WriteAll, Flush and Error report the error.
func WithLazyQuotes ¶
func WithLazyQuotes() Option
WithLazyQuotes makes the Reader tolerate malformed quoting: a bare '"' in an unquoted field, or a non-doubled '"' in a quoted field, is treated as a literal character instead of returning a parse error.
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Reader reads CSV records with zero allocations per record.
The input buffer and the field slice are allocated once in NewReader and reused for the lifetime of the Reader. The fields of a Record returned by Next are zero-copy views into that buffer, so they are valid only until the next call to Next.
func NewReader ¶
NewReader returns a Reader that parses CSV records from r, applying opts. An invalid delimiter marks the Reader as failed; Next and Error report the error.
func (*Reader) Error ¶ added in v0.2.0
Error returns the first error encountered while reading, or nil if none has occurred. io.EOF is normal termination and is not treated as an error, so Error returns nil after a record stream has been read to completion. A Reader configured with an invalid delimiter is failed from the start.
func (*Reader) Next ¶
Next parses and returns the next record, or io.EOF when no records remain. It returns ErrBareQuote or ErrQuote for malformed quoting, unless WithLazyQuotes is in effect, and propagates any error from the underlying reader.
The returned Record is reused by the Reader and overwritten by each call to Next. Its fields are zero-copy views into the Reader's internal buffer and are valid only until the next call to Next, like encoding/csv's ReuseRecord mode. Copy a field with Record.Copy, or convert it to a string, if it must outlive the read loop.
type Record ¶
type Record struct {
// contains filtered or unexported fields
}
Record is a parsed CSV record. The Reader reuses a single Record and overwrites it on every call to Next, so a Record is valid only until the Reader is advanced again. Its fields are zero-copy views into the Reader's internal buffer; use Copy to retain the values beyond the next call to Next.
func (*Record) Copy ¶ added in v0.2.0
Copy returns the record's fields as owned strings. The returned slice and its strings remain valid after subsequent calls to Next.
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Writer writes CSV records with zero allocations per write.
The bufio.Writer and the numeric scratch buffer are allocated once in NewWriter and reused for the lifetime of the Writer. Write/WriteAll perform no heap allocations on the hot path as long as the caller reuses a []Column slice (e.g. Write(row...)) rather than passing freshly constructed variadic args.
func NewWriter ¶ added in v0.2.0
NewWriter returns a Writer that writes CSV records to w, applying opts. If w is already a *bufio.Writer with a buffer at least as large as the default (4096 bytes), it is reused directly rather than being wrapped again.
func (*Writer) Error ¶
Error returns the first error encountered during Write, WriteAll or Flush, or nil if none has occurred.
func (*Writer) Flush ¶
Flush writes any buffered data to the underlying writer and returns the first error encountered during Write, WriteAll or Flush, if any.
Source Files
Directories