supercsv

README

SuperCSV - Generic CSV Iterator for Go

A flexible, type-safe CSV parser for Go that uses struct annotations to map CSV columns to struct fields.

Features

🎯 Generic Struct Parsing - Works with any struct type using Go generics 📝 CSV Annotations - Simple csv:"column_name" tag system 🔧 Multiple Sources - Read from files, URLs, or any io.Reader ✅ Required Fields - Mark fields as required with csv:"column,required" 🚀 Iterator Pattern - Memory efficient row-by-row processing 📦 Batch Operations - Convert to slice or use ForEach for bulk processing 🛡️ Type Safety - Full compile-time type checking 🔧 Custom Delimiters - Support for comma, semicolon, tab, and any custom delimiter

Quick Start

1. Define your struct with CSV annotations

type Person struct {
    Name   string   `csv:"name,required"`
    Age    int      `csv:"age"`
    Email  string   `csv:"email,required"`
    Salary *float64 `csv:"salary"`        // Pointer for optional fields
    Active bool     `csv:"active"`
}

2. Create an iterator

import "github.com/ivikasavnish/supercsv-go"

// From file (default comma delimiter)
iterator, err := supercsv.NewFromFile[Person]("data.csv")

// From file with custom delimiter (semicolon)
iterator, err := supercsv.NewFromFileWithDelimiter[Person]("data.csv", ';')

// From URL (default comma delimiter)
iterator, err := supercsv.NewFromURL[Person]("https://example.com/data.csv")

// From URL with custom delimiter (semicolon)
iterator, err := supercsv.NewFromURLWithDelimiter[Person]("https://example.com/data.csv", ';')

// From reader (default comma delimiter)
iterator, err := supercsv.NewFromReader[Person](reader)

// From reader with custom delimiter (semicolon) 
iterator, err := supercsv.NewFromReaderWithDelimiter[Person](reader, ';')

defer iterator.Close()

3. Process the data

// One by one
for {
    person, err := iterator.Next()
    if err == io.EOF {
        break
    }
    if err != nil {
        log.Fatal(err)
    }
    
    fmt.Printf("Name: %s, Age: %d\n", person.Name, person.Age)
}

// All at once
people, err := iterator.ToSlice()

// With callback
iterator.ForEach(func(person *Person, err error) bool {
    if err != nil {
        log.Printf("Error: %v", err)
        return false
    }
    
    processPerson(person)
    return true // Continue
})

CSV Annotation Rules

• Required: All struct fields must have csv:"column_name" annotation
• Column Mapping: csv:"column_name" maps to CSV header
• Required Fields: csv:"column_name,required" - fails if column missing
• Optional Fields: Use pointers for optional fields that can be nil

Supported Types

• string
• int, int8, int16, int32, int64
• uint, uint8, uint16, uint32, uint64
• float32, float64
• bool
• Pointers to any of the above (for optional fields)

Example CSV Data

name,age,email,salary,active
John Doe,30,john@example.com,50000.50,true
Jane Smith,25,jane@example.com,,false
Bob Johnson,35,bob@example.com,75000.00,true

Error Handling

The iterator provides detailed error messages for:

• Missing required CSV columns
• Missing struct annotations
• Type conversion errors
• Invalid CSV format

Memory Efficiency

The iterator processes CSV data row-by-row, making it suitable for large files without loading everything into memory at once.

Installation

go get github.com/ivikasavnish/supercsv-go

Testing

Run the tests:

go test -v

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Documentation

Overview

Package supercsv provides a flexible, type-safe CSV parser for Go using struct annotations.

This package allows you to parse CSV data into any struct type using Go generics. Simply define your struct with csv:"column_name" tags and the iterator will automatically map CSV columns to struct fields with full type safety.

Example usage:

type Person struct {
    Name   string   `csv:"name,required"`
    Age    int      `csv:"age"`
    Email  string   `csv:"email,required"`
    Salary *float64 `csv:"salary"`        // Optional field
    Active bool     `csv:"active"`
}

// From file
iterator, err := supercsv.NewFromFile[Person]("data.csv")
if err != nil {
    log.Fatal(err)
}
defer iterator.Close()

// Iterate through rows
for {
    person, err := iterator.Next()
    if err == io.EOF {
        break
    }
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("Name: %s, Age: %d\n", person.Name, person.Age)
}

Package supercsv provides a flexible, type-safe CSV parser for Go using struct annotations.

Overview

SuperCSV allows you to parse CSV data into any struct type using Go generics and struct tags. It supports reading from files, URLs, or any io.Reader with memory-efficient iteration.

Key Features

• Generic struct parsing using Go generics
• CSV column mapping via struct tags
• Support for required and optional fields
• Multiple data sources (files, URLs, readers)
• Memory-efficient row-by-row processing
• Type-safe parsing with comprehensive error handling
• Support for various Go types: string, int, uint, float, bool, time.Time, and pointers

Basic Usage

Define a struct with csv tags:

type Person struct {
    Name      string     `csv:"name,required"`
    Age       int        `csv:"age"`
    Email     string     `csv:"email,required"`
    Salary    *float64   `csv:"salary"`        // Optional field (pointer)
    Active    bool       `csv:"active"`
    BirthDate time.Time  `csv:"birth_date"`    // Date field
    CreatedAt *time.Time `csv:"created_at"`    // Optional timestamp
}

Create an iterator and process data:

// From file
iterator, err := supercsv.NewFromFile[Person]("data.csv")
if err != nil {
    log.Fatal(err)
}
defer iterator.Close()

// Process row by row
for {
    person, err := iterator.Next()
    if err == io.EOF {
        break
    }
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("Name: %s, Age: %d\n", person.Name, person.Age)
}

CSV Tag Format

The csv tag supports the following format:

`csv:"column_name"`          // Maps to CSV column, optional field
`csv:"column_name,required"` // Maps to CSV column, required field

All struct fields that should be parsed MUST have a csv tag. Fields without csv tags are ignored and will cause an error.

Supported Types

• string: Direct string values, no conversion needed
• Integers: int, int8, int16, int32, int64 (decimal format)
• Unsigned integers: uint, uint8, uint16, uint32, uint64 (decimal format)
• Floating point: float32, float64 (decimal format, scientific notation supported)
• Boolean: bool (accepts: true/false, 1/0, yes/no, on/off, case insensitive)
• Time: time.Time (multiple format auto-detection, see Time Parsing section)
• Pointers: *T where T is any supported type above (for optional/nullable fields)

Empty CSV values are handled as follows:

• Regular fields: Set to their zero value (0, "", false, time.Time{})
• Pointer fields: Set to nil
• Required fields: Generate an error if empty

Time Parsing

The time.Time type supports automatic format detection for common date/time formats:

• RFC3339: "2006-01-02T15:04:05Z07:00"
• RFC3339Nano: "2006-01-02T15:04:05.999999999Z07:00"
• SQL DateTime: "2006-01-02 15:04:05"
• Date only: "2006-01-02"
• Time only: "15:04:05"
• US date: "01/02/2006"
• US datetime: "01/02/2006 15:04:05"
• European date: "02/01/2006"
• European datetime: "02/01/2006 15:04:05"

For cross-platform consistency, dates without explicit timezone information are parsed as UTC. Only RFC3339 formats with explicit timezone are preserved.

Example with time fields:

type Event struct {
    Name      string    `csv:"event_name,required"`
    StartDate time.Time `csv:"start_date"`
    EndTime   *time.Time `csv:"end_time"` // Optional
}

// CSV data with various time formats:
// event_name,start_date,end_time
// "Conference",2024-03-15,2024-03-15T18:00:00Z
// "Workshop","03/20/2024 09:00:00",
// "Meeting",2024-03-22 14:30:00,2024-03-22T16:00:00Z
// "Webinar",01/15/2024,01/15/2024 16:30:00

Time Parsing Notes:

• Formats are tried in order until one succeeds
• All formats without explicit timezone are parsed as UTC for cross-platform consistency
• RFC3339 formats with timezone information preserve the original timezone
• Empty time fields result in time.Time{} (zero value) or nil for pointer fields
• Invalid time formats return descriptive error messages with supported formats

Time Format Troubleshooting

If you encounter time parsing errors, check the following:

• Ensure your date format matches one of the supported patterns
• For custom formats, consider preprocessing your CSV data
• Use RFC3339 format ("2006-01-02T15:04:05Z07:00") for maximum compatibility
• Check for extra whitespace around date values in your CSV
• Verify that date separators match expected patterns (/ vs - vs space)

Common time format examples:

• "2024-03-15" → March 15, 2024 (date only)
• "2024-03-15 14:30:00" → March 15, 2024 at 2:30 PM
• "2024-03-15T14:30:00Z" → March 15, 2024 at 2:30 PM UTC
• "03/15/2024" → March 15, 2024 (US format)
• "15/03/2024" → March 15, 2024 (European format)

Error Handling

The package provides detailed error messages for:

• Missing required CSV columns
• Missing struct annotations
• Type conversion errors (including time parsing failures)
• Invalid CSV format
• Network errors (for URL sources)
• Time format mismatches with helpful format suggestions

Data Sources

SuperCSV supports multiple data sources:

// From local file
iterator, err := supercsv.NewFromFile[Person]("data.csv")

// From URL
iterator, err := supercsv.NewFromURL[Person]("https://example.com/data.csv")

// From any io.Reader
iterator, err := supercsv.NewFromReader[Person](reader)

Batch Operations

For convenience, SuperCSV provides batch processing methods:

// Read all rows into a slice
people, err := iterator.ToSlice()

// Process with callback
iterator.ForEach(func(person *Person, err error) bool {
    if err != nil {
        log.Printf("Error: %v", err)
        return false // Stop processing
    }
    processPerson(person)
    return true // Continue
})

Advanced Example with Time Fields

Here's a comprehensive example showing time.Time usage with different formats:

type LogEntry struct {
    ID        int        `csv:"id,required"`
    Message   string     `csv:"message,required"`
    Timestamp time.Time  `csv:"timestamp"`        // Required time field
    ExpiresAt *time.Time `csv:"expires_at"`       // Optional time field
    Level     string     `csv:"level"`
}

// Sample CSV content:
// id,message,timestamp,expires_at,level
// 1,"System started","2024-03-15T10:30:00Z","2024-12-31T23:59:59Z","INFO"
// 2,"User login","2024-03-15 10:31:25",,DEBUG
// 3,"Error occurred","03/15/2024 10:32:00","12/31/2024","ERROR"

iterator, err := supercsv.NewFromFile[LogEntry]("logs.csv")
if err != nil {
    log.Fatal(err)
}
defer iterator.Close()

for {
    entry, err := iterator.Next()
    if err == io.EOF {
        break
    }
    if err != nil {
        log.Printf("Parse error: %v", err)
        continue
    }

    fmt.Printf("Log %d: %s at %s\n",
        entry.ID, entry.Message, entry.Timestamp.Format("2006-01-02 15:04:05"))

    if entry.ExpiresAt != nil {
        fmt.Printf("  Expires: %s\n", entry.ExpiresAt.Format("2006-01-02"))
    }
}

Memory Efficiency

The iterator processes CSV data row-by-row, making it suitable for large files without loading everything into memory at once. Only the current row and struct metadata are kept in memory during iteration.

Thread Safety

CSVIterator instances are not thread-safe. Each goroutine should use its own iterator instance.

Index

• type CSVIterator
  • func NewFromFile[T any](filepath string) (*CSVIterator[T], error)
  • func NewFromFileWithDelimiter[T any](filepath string, delimiter rune) (*CSVIterator[T], error)
  • func NewFromReader[T any](reader io.Reader) (*CSVIterator[T], error)
  • func NewFromReaderWithDelimiter[T any](reader io.Reader, delimiter rune) (*CSVIterator[T], error)
  • func NewFromURL[T any](url string) (*CSVIterator[T], error)
  • func NewFromURLWithDelimiter[T any](url string, delimiter rune) (*CSVIterator[T], error)
  • func (it *CSVIterator[T]) Close() error
  • func (it *CSVIterator[T]) ForEach(fn func(*T, error) bool)
  • func (it *CSVIterator[T]) Headers() []string
  • func (it *CSVIterator[T]) Next() (*T, error)
  • func (it *CSVIterator[T]) ToSlice() ([]*T, error)

Examples

• CSVIterator.ForEach
• NewFromFile
• NewFromReader (TimeSupport)
• NewFromURL

Types

type CSVIterator

type CSVIterator[T any] struct {
	// contains filtered or unexported fields
}

CSVIterator provides generic CSV parsing with struct annotations

func NewFromFile

func NewFromFile[T any](filepath string) (*CSVIterator[T], error)

NewFromFile creates a CSV iterator from a file path

Example

Example usage functions

// Create iterator from file
iterator, err := NewFromFile[Person]("/path/to/people.csv")
if err != nil {
	fmt.Printf("Error: %v\n", err)
	return
}
defer iterator.Close()

// Read all data
people, err := iterator.ToSlice()
if err != nil {
	fmt.Printf("Error reading data: %v\n", err)
	return
}

for _, person := range people {
	fmt.Printf("Name: %s, Age: %d, Email: %s\n",
		person.Name, person.Age, person.Email)
}

func NewFromFileWithDelimiter

func NewFromFileWithDelimiter[T any](filepath string, delimiter rune) (*CSVIterator[T], error)

NewFromFileWithDelimiter creates a CSV iterator from a file path with custom delimiter

func NewFromReader

func NewFromReader[T any](reader io.Reader) (*CSVIterator[T], error)

NewFromReader creates a CSV iterator from an io.Reader

Example (TimeSupport)

csvData := `event_name,start_date,end_time,duration_minutes
Conference,2024-03-15,2024-03-15T18:00:00Z,480
Workshop,03/20/2024 09:00:00,,240`

reader := strings.NewReader(csvData)
iterator, err := NewFromReader[Event](reader)
if err != nil {
	fmt.Printf("Error: %v\n", err)
	return
}
defer iterator.Close()

for {
	event, err := iterator.Next()
	if err == io.EOF {
		break
	}
	if err != nil {
		fmt.Printf("Error reading event: %v\n", err)
		break
	}

	fmt.Printf("Event: %s, Start: %s, Duration: %d mins\n",
		event.Name, event.StartDate.Format("2006-01-02 15:04"), event.Duration)

	if event.EndTime != nil {
		fmt.Printf("  End: %s\n", event.EndTime.Format("2006-01-02 15:04"))
	}
}

Output:

Event: Conference, Start: 2024-03-15 00:00, Duration: 480 mins
  End: 2024-03-15 18:00
Event: Workshop, Start: 2024-03-20 09:00, Duration: 240 mins

func NewFromReaderWithDelimiter

func NewFromReaderWithDelimiter[T any](reader io.Reader, delimiter rune) (*CSVIterator[T], error)

NewFromReaderWithDelimiter creates a CSV iterator from an io.Reader with custom delimiter

func NewFromURL

func NewFromURL[T any](url string) (*CSVIterator[T], error)

NewFromURL creates a CSV iterator from a URL

Example

// Create iterator from URL
iterator, err := NewFromURL[Product]("https://example.com/products.csv")
if err != nil {
	fmt.Printf("Error: %v\n", err)
	return
}
defer iterator.Close()

// Iterate one by one
for {
	product, err := iterator.Next()
	if err == io.EOF {
		break
	}
	if err != nil {
		fmt.Printf("Error reading product: %v\n", err)
		break
	}

	fmt.Printf("Product: %s, Price: $%.2f\n",
		product.Name, product.Price)
}

func NewFromURLWithDelimiter

func NewFromURLWithDelimiter[T any](url string, delimiter rune) (*CSVIterator[T], error)

NewFromURLWithDelimiter creates a CSV iterator from a URL with custom delimiter

func (*CSVIterator[T]) Close

func (it *CSVIterator[T]) Close() error

Close closes the underlying reader if it implements io.Closer

func (*CSVIterator[T]) ForEach

func (it *CSVIterator[T]) ForEach(fn func(*T, error) bool)

ForEach iterates through all CSV rows and calls the provided function

Example

csvData := `name,age,email,salary,active
John Doe,30,john@example.com,50000,true
Jane Smith,25,jane@example.com,60000,false`

reader := strings.NewReader(csvData)
iterator, err := NewFromReader[Person](reader)
if err != nil {
	fmt.Printf("Error: %v\n", err)
	return
}
defer iterator.Close()

// Use ForEach for processing
iterator.ForEach(func(person *Person, err error) bool {
	if err != nil {
		fmt.Printf("Error: %v\n", err)
		return false
	}

	fmt.Printf("Processing: %s (Age: %d)\n", person.Name, person.Age)
	return true // Continue iteration
})

func (*CSVIterator[T]) Headers

func (it *CSVIterator[T]) Headers() []string

Headers returns the CSV column headers

func (*CSVIterator[T]) Next

func (it *CSVIterator[T]) Next() (*T, error)

Next reads and parses the next CSV row into the struct type

func (*CSVIterator[T]) ToSlice

func (it *CSVIterator[T]) ToSlice() ([]*T, error)

ToSlice reads all remaining CSV rows into a slice