cursor

Documentation

Overview

Package cursor provides high-performance cursor-based (keyset) pagination.

Cursor pagination uses the values of sort columns to efficiently navigate large datasets without the performance degradation of offset pagination. It's ideal for infinite scroll, real-time feeds, and APIs with millions of records.

Key Features:

• O(1) complexity regardless of page depth
• Consistent performance with proper indexes
• Stable results during concurrent writes
• Opaque cursor encoding (Base64 JSON)
• Composite key support for deterministic ordering

Example usage:

encoder := cursor.NewCompositeCursorEncoder(func(u *models.User) map[string]any {
    return map[string]any{
        "created_at": u.CreatedAt,
        "id":         u.ID,
    }
})

fetcher := sqlboiler.NewFetcher(..., sqlboiler.CursorToQueryMods)

users, _ := fetcher.Fetch(ctx, paging.FetchParams{...})
paginator := cursor.New(pageArgs, encoder, users)
conn, _ := cursor.BuildConnection(paginator, users, encoder, toDomainUser)

Cursor Format:

Cursors are base64-encoded JSON objects containing column values:
{"created_at":"2024-01-01T00:00:00Z","id":"abc-123"}
→ eyJjcmVhdGVkX2F0IjoiMjAyNC0wMS0wMVQwMDowMDowMFoiLCJpZCI6ImFiYy0xMjMifQ==

Performance:

Requires a composite index on sort columns:
CREATE INDEX idx ON table(col1 DESC, col2 DESC);

With proper indexing, all pages have similar performance (~5ms per page).

Limitations:

• Forward pagination only (After + First). Backward pagination planned for Phase 2.5.
• Requires unique sort key (typically add ID as final column)
• PostgreSQL tuple comparison syntax (MySQL requires expanded form)
• Eventually consistent (cursors may become stale if data changes)

Index

• func BuildConnection[From any, To any](page *paging.Page[From], schema *Schema[From], args *paging.PageArgs, ...) (*paging.Connection[To], error)
• func New[T any](fetcher paging.Fetcher[T], schema *Schema[T]) paging.Paginator[T]
• func NewCompositeCursorEncoder[T any](extractor func(T) map[string]any) paging.CursorEncoder[T]
• type CompositeCursorEncoder
  • func (e *CompositeCursorEncoder[T]) Decode(cursor string) (*paging.CursorPosition, error)
  • func (e *CompositeCursorEncoder[T]) Encode(item T) (*string, error)
• type Direction
• type PageArgs
• type Paginator
  • func (p *Paginator[T]) Paginate(ctx context.Context, args *paging.PageArgs, opts ...paging.PaginateOption) (*paging.Page[T], error)
• type Schema
  • func NewSchema[T any]() *Schema[T]
  • func (s *Schema[T]) BuildOrderBy(userSorts []paging.Sort) []paging.Sort
  • func (s *Schema[T]) EncoderFor(pageArgs PageArgs) (paging.CursorEncoder[T], error)
  • func (s *Schema[T]) Field(name, cursorKey string, extractor func(T) any) *Schema[T]
  • func (s *Schema[T]) FixedField(name string, direction Direction, cursorKey string, extractor func(T) any) *Schema[T]
• type Spec
  • func (s *Spec[T]) Decode(cursor string) (*paging.CursorPosition, error)
  • func (s *Spec[T]) Encode(item T) (*string, error)

Functions

func BuildConnection

func BuildConnection[From any, To any](
	page *paging.Page[From],
	schema *Schema[From],
	args *paging.PageArgs,
	transform func(From) (To, error),
) (*paging.Connection[To], error)

BuildConnection transforms a Page[From] to a Connection[To] for GraphQL. Uses schema's encoder to generate composite key cursors.

Example:

result, _ := paginator.Paginate(ctx, args, paging.WithMaxSize(100))
conn, _ := cursor.BuildConnection(result, schema, toDomainUser)

func New

func New[T any](fetcher paging.Fetcher[T], schema *Schema[T]) paging.Paginator[T]

New creates a cursor paginator that implements paging.Paginator[T].

Example:

fetcher := sqlboiler.NewFetcher(queryFunc, countFunc, sqlboiler.CursorToQueryMods)
paginator := cursor.New(fetcher, schema)
result, err := paginator.Paginate(ctx, args, paging.WithMaxSize(100))

func NewCompositeCursorEncoder

func NewCompositeCursorEncoder[T any](extractor func(T) map[string]any) paging.CursorEncoder[T]

NewCompositeCursorEncoder creates a cursor encoder for composite key pagination.

The extractor function should return a map of column names to their values for the columns used in sorting. These values will be encoded into the cursor and used to resume pagination.

Example:

encoder := cursor.NewCompositeCursorEncoder(func(u *models.User) map[string]any {
    return map[string]any{
        "created_at": u.CreatedAt,
        "id":         u.ID,
    }
})

Types

type CompositeCursorEncoder

type CompositeCursorEncoder[T any] struct {
	// contains filtered or unexported fields
}

CompositeCursorEncoder encodes multiple column values into an opaque cursor string. It implements the paging.CursorEncoder interface for composite key pagination.

The encoder uses an extractor function to extract the relevant column values from each item, then encodes them as base64-encoded JSON.

Type parameter T is the item type (e.g., *models.User).

func (*CompositeCursorEncoder[T]) Decode

func (e *CompositeCursorEncoder[T]) Decode(cursor string) (*paging.CursorPosition, error)

Decode extracts cursor position from an opaque cursor string.

The cursor is expected to be a base64-encoded JSON object containing column name/value pairs.

Returns nil if the cursor is empty, invalid, or cannot be decoded. This graceful degradation ensures invalid cursors result in "start from beginning" behavior.

func (*CompositeCursorEncoder[T]) Encode

func (e *CompositeCursorEncoder[T]) Encode(item T) (*string, error)

Encode converts an item into an opaque cursor string. The cursor encodes the sort column values as base64-encoded JSON.

Returns nil if the item has no values or if encoding fails.

type Direction

type Direction bool

Direction represents the sort direction for a field.

const (
	ASC  Direction = false
	DESC Direction = true
)

type PageArgs

type PageArgs interface {
	GetFirst() *int
	GetAfter() *string
	GetSortBy() []paging.Sort
}

PageArgs represents pagination arguments.

type Paginator

type Paginator[T any] struct {
	// contains filtered or unexported fields
}

Paginator implements paging.Paginator[T] for cursor-based pagination.

func (*Paginator[T]) Paginate

func (p *Paginator[T]) Paginate(
	ctx context.Context,
	args *paging.PageArgs,
	opts ...paging.PaginateOption,
) (*paging.Page[T], error)

Paginate executes cursor-based pagination and returns a Page[T].

type Schema

type Schema[T any] struct {
	// contains filtered or unexported fields
}

Schema defines the sortable and fixed fields for cursor pagination. It enforces that cursor encoders and ORDER BY clauses match by providing a single source of truth for field configuration.

Schema solves several critical issues: 1. Information leakage: Uses short cursor keys instead of column names 2. Encoder/OrderBy mismatch: Enforces they match by design 3. Dynamic sorting: Validates user sort choices and provides correct encoder 4. Fixed fields: Automatically includes tenant_id, id in ORDER BY

Example:

var userSchema = cursor.NewSchema[*User]().
    FixedField("tenant_id", cursor.ASC, "t", func(u *User) any { return u.TenantID }).
    Field("name", "n", func(u *User) any { return u.Name }).
    Field("created_at", "c", func(u *User) any { return u.CreatedAt }).
    FixedField("id", cursor.DESC, "i", func(u *User) any { return u.ID })

func NewSchema

func NewSchema[T any]() *Schema[T]

NewSchema creates a new Schema for cursor pagination.

func (*Schema[T]) BuildOrderBy

func (s *Schema[T]) BuildOrderBy(userSorts []paging.Sort) []paging.Sort

BuildOrderBy constructs the complete ORDER BY clause including fixed fields. Fixed fields declared before user-sortable fields are prepended. Fixed fields declared after user-sortable fields are appended.

Example:

schema.FixedField("tenant_id", ASC, ...)  // Declared first
schema.Field("name", ...)                  // User-sortable
schema.FixedField("id", DESC, ...)         // Declared last

BuildOrderBy([{Column: "name", Desc: true}])
// Returns: [tenant_id ASC, name DESC, id DESC]

func (*Schema[T]) EncoderFor

func (s *Schema[T]) EncoderFor(pageArgs PageArgs) (paging.CursorEncoder[T], error)

EncoderFor validates the PageArgs and creates a Spec that implements CursorEncoder. It ensures that: 1. All sort fields in PageArgs.SortBy are valid (registered in schema) 2. The encoder extracts values for all sort fields + fixed fields 3. Short cursor keys are used (no information leakage)

Returns an error if any sort field is invalid.

func (*Schema[T]) Field

func (s *Schema[T]) Field(name, cursorKey string, extractor func(T) any) *Schema[T]

Field adds a user-sortable field to the schema. User-sortable fields can be specified in PageArgs.SortBy at runtime.

Parameters:

• name: SQL column name (can be qualified: "posts.created_at")
• cursorKey: Short key for cursor encoding (e.g., "c")
• extractor: Function to extract the value from an item

Example:

schema.Field("name", "n", func(u *User) any { return u.Name })

func (*Schema[T]) FixedField

func (s *Schema[T]) FixedField(name string, direction Direction, cursorKey string, extractor func(T) any) *Schema[T]

FixedField adds a fixed field to the schema. Fixed fields are always included in ORDER BY and cursors but cannot be chosen by users at runtime.

Parameters:

• name: SQL column name (can be qualified: "posts.id")
• direction: Sort direction (cursor.ASC or cursor.DESC)
• cursorKey: Short key for cursor encoding (e.g., "i")
• extractor: Function to extract the value from an item

Declaration order matters:

• FixedField before Field: Prepended to ORDER BY (e.g., tenant_id for partitioning)
• FixedField after Field: Appended to ORDER BY (e.g., id for uniqueness)

Example:

schema.FixedField("id", cursor.DESC, "i", func(u *User) any { return u.ID })

type Spec

type Spec[T any] struct {
	// contains filtered or unexported fields
}

Spec is the runtime configuration for cursor encoding/decoding. It implements CursorEncoder[T] and ensures encoder/OrderBy matching.

func (*Spec[T]) Decode

func (s *Spec[T]) Decode(cursor string) (*paging.CursorPosition, error)

Decode implements CursorEncoder.Decode. It decodes the cursor and maps short keys back to column names.

func (*Spec[T]) Encode

func (s *Spec[T]) Encode(item T) (*string, error)

Encode implements CursorEncoder.Encode. It encodes the item using short cursor keys from the schema.