metrics

Documentation

Overview

Package metrics carries the lock-free counters for a run. Every field is an atomic so any worker can bump it without contention, and the periodic display reads a consistent-enough snapshot without stopping the world.

Index

• type Snapshot
• type Stats
  • func NewStats(start time.Time) *Stats
  • func (s *Stats) Done() int64
  • func (s *Stats) Snapshot() Snapshot

Types

type Snapshot

type Snapshot struct {
	Seeded, Attempted, OK, NotMod, Redirect            int64
	ClientErr, ServerErr, Failed, Skipped, Bytes, Done int64
	Elapsed                                            time.Duration
	RatePerSec                                         float64
	MiBPerSec                                          float64
}

Snapshot is an immutable copy of Stats for display.

type Stats

type Stats struct {
	Seeded    atomic.Int64 // URLs read from the seed
	Attempted atomic.Int64 // fetch attempts started
	OK        atomic.Int64 // 2xx responses stored
	NotMod    atomic.Int64 // 304 / unchanged revisits
	Redirect  atomic.Int64 // 3xx (other than 304)
	ClientErr atomic.Int64 // 4xx
	ServerErr atomic.Int64 // 5xx
	Failed    atomic.Int64 // transport/DNS/timeout failures
	Skipped   atomic.Int64 // dead host/domain or non-HTTP
	Bytes     atomic.Int64 // response body bytes stored
	// contains filtered or unexported fields
}

Stats holds the running totals for one crawl.

func NewStats

func NewStats(start time.Time) *Stats

NewStats returns a Stats stamped with a start time for rate computation.

func (*Stats) Done

func (s *Stats) Done() int64

Done reports the number of URLs that reached a terminal state.

func (*Stats) Snapshot

func (s *Stats) Snapshot() Snapshot

Snapshot reads the counters into a plain struct.