backfill

Documentation

Index

• type Store
  • func Default() *Store
  • func NewStore(conn *gorm.DB) *Store
  • func (s *Store) AddProgress(id uuid.UUID, completedDelta, failedDelta int64) error
  • func (s *Store) Complete(id uuid.UUID, failed bool) error
  • func (s *Store) Create(b *models.Backfill) error
  • func (s *Store) Get(id uuid.UUID) (*models.Backfill, error)
  • func (s *Store) IsCancelRequested(id uuid.UUID) (bool, error)
  • func (s *Store) IsRunning(id uuid.UUID) (bool, error)
  • func (s *Store) LatestRunForLogicalDate(jobID uuid.UUID, logicalDate string) (string, error)
  • func (s *Store) List(jobID uuid.UUID) ([]*models.Backfill, error)
  • func (s *Store) MarkCancelled(id uuid.UUID) error
  • func (s *Store) RequestCancel(id uuid.UUID) error
  • func (s *Store) SetTotalRuns(id uuid.UUID, total int) error

Types

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store provides CRUD operations for Backfill records.

func Default

func Default() *Store

func NewStore

func NewStore(conn *gorm.DB) *Store

func (*Store) AddProgress

func (s *Store) AddProgress(id uuid.UUID, completedDelta, failedDelta int64) error

AddProgress applies accumulated run-completion counts to a backfill in a single UPDATE. The executor batches per-run increments and flushes them periodically: a large backfill otherwise issues one completed_runs/ failed_runs write per run, all on the same backfills row, which serialize through dqlite's single writer and starve concurrent control-plane writes (e.g. cancellation) past the busy-retry budget.

completedDelta and failedDelta are the counts to add and must be non-negative; the executor's source counters are increment-only.

func (*Store) Complete

func (s *Store) Complete(id uuid.UUID, failed bool) error

func (*Store) Create

func (s *Store) Create(b *models.Backfill) error

func (*Store) Get

func (s *Store) Get(id uuid.UUID) (*models.Backfill, error)

func (*Store) IsCancelRequested

func (s *Store) IsCancelRequested(id uuid.UUID) (bool, error)

IsCancelRequested returns true if a running backfill has a persisted cancel request.

func (*Store) IsRunning

func (s *Store) IsRunning(id uuid.UUID) (bool, error)

IsRunning returns true if a backfill with the given ID is in the running state.

func (*Store) LatestRunForLogicalDate

func (s *Store) LatestRunForLogicalDate(jobID uuid.UUID, logicalDate string) (string, error)

LatestRunForLogicalDate returns the status of the most recent run for a job whose params contain the given logical_date value, or "" if none exists. It uses Go-side filtering to remain portable across SQLite and Postgres.

func (*Store) List

func (s *Store) List(jobID uuid.UUID) ([]*models.Backfill, error)

func (*Store) MarkCancelled

func (s *Store) MarkCancelled(id uuid.UUID) error

MarkCancelled marks a running backfill as fully cancelled after in-flight work has drained.

func (*Store) RequestCancel

func (s *Store) RequestCancel(id uuid.UUID) error

RequestCancel persists a cancellation request for a running backfill.

func (*Store) SetTotalRuns

func (s *Store) SetTotalRuns(id uuid.UUID, total int) error

SetTotalRuns updates the total_runs counter on a backfill.