traffic

package
v0.0.0-...-31b1c06 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 26, 2026 License: MIT Imports: 10 Imported by: 0

Documentation

Overview

Package traffic provides fire-and-forget traffic recording and quota enforcement.

Package traffic — storage counter helpers.

These functions manage the storage_counters table which tracks bytes_used and file_count across four scopes: platform, org, user, and library.

The table uses ((scope, shard), day) as the PRIMARY KEY:

  • shard = CounterShard(orgID) for the global platform scope
  • shard = 0 for org/user/library scopes
  • day = storageTotalDay (1970-01-01) → running total for fast quota checks
  • day = <real date> → daily delta for time-series graphs

This mirrors how traffic uses traffic_counters (per-day) + traffic_monthly (aggregate), but in a single table.

Index

Constants

View Source 
const (
	SyncUpload   = "sync-file-upload"
	SyncDownload = "sync-file-download"
	WebUpload    = "web-file-upload"
	WebDownload  = "web-file-download"
	LinkUpload   = "link-file-upload"
	LinkDownload = "link-file-download"
)

Traffic type constants match Seafile's category names for API compatibility.

View Source 
const CounterShardCount = 32

CounterShardCount splits the two global hot counter aggregates into a modest number of deterministic shards. Reads only fan out on cold admin paths, so we can afford a wider spread here to reduce multiregion write concentration.

Variables

This section is empty.

Functions

func AddAggregateStorageReconciliationQueries 

func AddAggregateStorageReconciliationQueries(batch *gocql.Batch, orgID, ownerID string, requestedAt time.Time)

AddAggregateStorageReconciliationQueries records the aggregate scopes that must be recomputed after a library soft-delete or restore.

func AdjustAggregateStorageCounters 

func AdjustAggregateStorageCounters(db DBSession, orgID, ownerID, libraryID string, increment bool)

AdjustAggregateStorageCounters reads the lib-scope total and increments (increment=true) or decrements (increment=false) the org, user, and platform scopes by that amount. Runs synchronously because callers need the adjustment to be visible before returning (e.g. quota checks right after restore).

func AdjustStorageCountersByDelta 

func AdjustStorageCountersByDelta(db DBSession, orgID, userID, libraryID string, deltaBytes, deltaFiles int64)

AdjustStorageCountersByDelta applies an arbitrary signed delta to platform, org, user, and library storage counters. It is used when a commit publishes a new tree and the exact change is known only after comparing aggregate stats.

func AdjustStorageCountersByDeltaSync 

func AdjustStorageCountersByDeltaSync(db DBSession, orgID, userID, libraryID string, deltaBytes, deltaFiles int64) error

AdjustStorageCountersByDeltaSync applies an arbitrary signed delta and returns after all affected scope rows have been updated.

func CounterShard 

func CounterShard(orgID string) int

func CounterShardUUID 

func CounterShardUUID(orgID gocql.UUID) int

CounterShardUUID returns the deterministic shard for a canonical org UUID.

func CurrentMonth 

func CurrentMonth() string

CurrentMonth returns the current UTC month in yyyymm format.

func DecrementStorageCounters 

func DecrementStorageCounters(db DBSession, orgID, userID, libraryID string, deltaBytes int64, deltaFiles int64)

DecrementStorageCounters atomically decrements storage usage for org, user, and library. Reads the current total first and caps the delta to avoid negative values. Runs fire-and-forget — never blocks the caller.

func DecrementStorageCountersSync 

func DecrementStorageCountersSync(db DBSession, orgID, userID, libraryID string, deltaBytes int64, deltaFiles int64) error

DecrementStorageCountersSync decrements storage usage and caps negative deltas at the current running total for each scope.

func DeleteLibraryStorageCounter 

func DeleteLibraryStorageCounter(db DBSession, orgID, libraryID string) error

DeleteLibraryStorageCounter removes all rows for the lib-scope after permanent deletion. Aggregate scopes were already adjusted by a prior soft-delete.

func EffectivePeriodStart 

func EffectivePeriodStart(periodStart *time.Time, now time.Time) time.Time

EffectivePeriodStart returns the active quota period start for an org. If the period is missing, it falls back to the first instant of the current UTC calendar month for backward compatibility.

func EffectiveTrafficResetDate 

func EffectiveTrafficResetDate(periodStart *time.Time, periodEnd *time.Time, now time.Time) string

EffectiveTrafficResetDate returns the human-facing reset date used by Phase 2 payloads. When the org has an explicit period end, that is authoritative. Otherwise it derives the period end from the effective quota period start so traffic reset messaging matches monthly rollover semantics everywhere.

func ForEachCounterShard 

func ForEachCounterShard(fn func(int))

func IncrementStorageCounters 

func IncrementStorageCounters(db DBSession, orgID, userID, libraryID string, deltaBytes int64, deltaFiles int64)

IncrementStorageCounters atomically increments storage usage for org, user, and library. Updates both the running total and today's daily delta. Runs fire-and-forget — never blocks the caller.

func IncrementStorageCountersSync 

func IncrementStorageCountersSync(db DBSession, orgID, userID, libraryID string, deltaBytes int64, deltaFiles int64) error

IncrementStorageCountersSync increments storage usage and returns only after all scope rows have been updated.

func LibraryStorageScope 

func LibraryStorageScope(orgID, libraryID string) string

func OrganizationStorageScope 

func OrganizationStorageScope(orgID string) string

func PlatformStorageScope 

func PlatformStorageScope() string

func ReadMonthlyScopeTotals 

func ReadMonthlyScopeTotals(db DBSession, orgID, month string) map[string]int64

ReadMonthlyScopeTotals returns all traffic_monthly counters for one org+month partition in a single query.

func ReadPeriodScopeTotals 

func ReadPeriodScopeTotals(db DBSession, orgID string, periodStartedAt time.Time) map[string]int64

ReadPeriodScopeTotals returns all traffic_period_usage counters for one org+period partition in a single query.

func ReadStorageDailyDeltas 

func ReadStorageDailyDeltas(db DBSession, scope string, start, end time.Time) map[string]int64

ReadStorageDailyDeltas returns storage deltas for each day in [start, end]. The sentinel total row is never included because storageTotalDay < any real date.

func ReadStorageUsed 

func ReadStorageUsed(db DBSession, scope string) int64

ReadStorageUsed returns the live bytes_used from the running-total row for the given scope. Returns 0 if the row does not exist or on any error.

func ReconcileStorageScope 

func ReconcileStorageScope(db DBSession, scope string, expected StorageSnapshot) error

ReconcileStorageScope corrects a scope to the expected running total. The delta is derived from the current live total, so repeated runs converge.

func ReconcileStorageScopeSharded 

func ReconcileStorageScopeSharded(db DBSession, scope string, expectedByShard map[int]StorageSnapshot) error

ReconcileStorageScopeSharded corrects the platform scope shard-by-shard so retries converge even when the global aggregate is distributed.

func ReconstructStorageHistory 

func ReconstructStorageHistory(db DBSession, scope string, start, end time.Time) map[string]int64

ReconstructStorageHistory returns date → bytes_used for each day in [start, end], working backwards from the current running total using daily deltas.

func RecordCheckedTransfer 

func RecordCheckedTransfer(recorder TrafficPeriodRecorder, quotaStatus QuotaStatus, orgID, userID, trafficType string, bytes int64)

RecordCheckedTransfer records bytes using the period resolved during the earlier traffic quota pre-check. When quotaStatus.PeriodStartedAt is zero, the recorder falls back to its legacy DB lookup path.

func RolloverExpiredPeriods 

func RolloverExpiredPeriods(session *gocql.Session, now time.Time) (int, error)

RolloverExpiredPeriods scans all organizations and advances the traffic quota period for any whose current_period_ends_at <= now.

Traffic periods are ALWAYS monthly regardless of billing_cycle (annual billing still has monthly traffic limits and monthly overage charges).

Relationship with Accounts (external billing service):

  • Paid orgs: Accounts is the source of truth and may push updated period dates at any time. This rollover serves as a safety net in case Accounts is slow or temporarily unreachable — the next Accounts sync overwrites.
  • Free orgs: No external billing service resets them, so this cron is their ONLY mechanism for period advancement.

Idempotent and safe for concurrent execution: two instances will compute the same deterministic period values, so the last writer wins with the same result.

func SetChecker 

func SetChecker(c *Checker)

SetChecker installs the global Checker. Called once from server.go.

func SetRecorder 

func SetRecorder(r *Recorder)

SetRecorder installs the global Recorder. Called once from server.go.

func TrafficQuotaExceededResponse 

func TrafficQuotaExceededResponse(quotaStatus QuotaStatus, message string, includeReason bool) map[string]interface{}

TrafficQuotaExceededResponse builds a consistent JSON payload for blocked traffic requests while allowing each handler to choose its user-facing message and whether to expose the internal reason code.

func TrafficQuotaWarningHeader 

func TrafficQuotaWarningHeader(quotaStatus QuotaStatus) (string, bool)

TrafficQuotaWarningHeader returns the value that should be written to the X-Quota-Warning header when the status represents a soft warning.

func UserStorageScope 

func UserStorageScope(orgID, userID string) string

Types

type Checker 

type Checker struct {
	// contains filtered or unexported fields
}

Checker reads quota configuration and current usage from ScyllaDB to evaluate whether an operation should be allowed, warned, or blocked.

func GetChecker 

func GetChecker() *Checker

GetChecker returns the global Checker, or nil if not initialized.

func NewChecker 

func NewChecker(session *gocql.Session) *Checker

NewChecker creates a Checker backed by the given ScyllaDB session.

func (*Checker) CheckMaxUsers 

func (c *Checker) CheckMaxUsers(orgID string) (QuotaStatus, error)

CheckMaxUsers evaluates whether adding a new user to the org is allowed.

func (*Checker) CheckStorageQuota 

func (c *Checker) CheckStorageQuota(orgID, userID string, additionalBytes int64) (QuotaStatus, error)

CheckStorageQuota evaluates whether uploading additionalBytes would exceed the org or per-user storage quota. Both limits use the org's quota_policy; the most restrictive result wins.

func (*Checker) CheckTrafficQuota 

func (c *Checker) CheckTrafficQuota(orgID, userID, direction string, additionalBytes int64) (QuotaStatus, error)

CheckTrafficQuota evaluates upload or download traffic quotas. direction must be "upload" or "download". All three checks (combined, per-direction org, per-user) are evaluated; the most restrictive result is returned.

func (*Checker) CheckTrafficQuotaContext 

func (c *Checker) CheckTrafficQuotaContext(ctx context.Context, orgID, userID, direction string, additionalBytes int64) (QuotaStatus, error)

CheckTrafficQuotaContext is CheckTrafficQuota bound to ctx for request paths that have a finite preparation budget.

type DBSession 

type DBSession interface {
	Session() *gocql.Session
}

DBSession is the minimal interface needed to execute CQL queries. Both *db.DB and *gocql.Session satisfy this via their Session() method.

type MonthlyTransferUsage 

type MonthlyTransferUsage struct {
	Combined int64
	Upload   int64
	Download int64
}

MonthlyTransferUsage is the common org/user traffic summary shape used by admin, org-admin, and account/subscription endpoints.

func ReadOrgMonthlyUsage 

func ReadOrgMonthlyUsage(db DBSession, orgID, month string) MonthlyTransferUsage

ReadOrgMonthlyUsage returns the current org-level traffic usage for the month.

func ReadOrgPeriodUsage 

func ReadOrgPeriodUsage(db DBSession, orgID string, periodStartedAt time.Time) MonthlyTransferUsage

ReadOrgPeriodUsage returns the current org-level traffic usage for the active quota period.

func ReadUserMonthlyUsage 

func ReadUserMonthlyUsage(db DBSession, orgID, userID, month string) MonthlyTransferUsage

ReadUserMonthlyUsage returns the current per-user upload/download usage for the month.

func ReadUserPeriodUsage 

func ReadUserPeriodUsage(db DBSession, orgID, userID string, periodStartedAt time.Time) MonthlyTransferUsage

ReadUserPeriodUsage returns the current per-user upload/download usage for the active quota period.

type QuotaStatus 

type QuotaStatus struct {
	Allowed         bool      // may the operation proceed?
	Warning         bool      // >80% of the included limit (paid plans only)
	UsedBytes       int64     // current usage relevant to the check
	LimitBytes      int64     // limit that was evaluated; values <= 0 mean unlimited in SesameFS enforcement
	Reason          string    // "storage", "traffic-combined", "traffic-upload", "traffic-download", "max-users"
	Plan            string    // plan name from organizations table
	PeriodStartedAt time.Time // the quota period resolved during CheckTrafficQuota; zero for non-traffic checks
}

QuotaStatus describes the result of a quota check.

func CheckTrafficQuotaWithChecker 

func CheckTrafficQuotaWithChecker(checker TrafficQuotaPrechecker, orgID, userID, direction string, additionalBytes int64) (QuotaStatus, error)

CheckTrafficQuotaWithChecker evaluates traffic quota with the supplied checker. A nil checker means quota enforcement is disabled for this request path.

type Recorder 

type Recorder struct {
	// contains filtered or unexported fields
}

Recorder writes traffic counters asynchronously. All methods are safe for concurrent use. Record never blocks the caller.

func Get 

func Get() *Recorder

Get returns the global Recorder, or nil if SetRecorder has not been called. Callers must check for nil before using.

func NewRecorder 

func NewRecorder(session *gocql.Session) *Recorder

NewRecorder creates a Recorder backed by the given ScyllaDB session.

func (*Recorder) Record 

func (r *Recorder) Record(orgID, userID, trafficType string, bytes int64)

Record increments the traffic counters for a single transfer. It runs completely asynchronously - the caller is never blocked and errors are logged but not returned.

orgID and userID must be valid UUID strings. trafficType must be one of the package-level constants (SyncUpload, WebDownload, etc.).

Callers that have already run a CheckTrafficQuota pre-check should use RecordWithPeriod instead - it reuses the already-resolved period and saves an extra SELECT per event.

func (*Recorder) RecordWithPeriod 

func (r *Recorder) RecordWithPeriod(orgID, userID, trafficType string, bytes int64, periodStartedAt time.Time)

RecordWithPeriod is like Record but accepts the quota period start that was already resolved by a preceding CheckTrafficQuota call. This eliminates the SELECT on organizations that Record would otherwise perform per event, and guarantees that enforcement and recording use the exact same period.

periodStartedAt must be the PeriodStartedAt value from the QuotaStatus returned by CheckTrafficQuota. If zero, falls back to the DB lookup (same behavior as Record).

type StorageSnapshot 

type StorageSnapshot struct {
	BytesUsed int64
	FileCount int64
}

StorageSnapshot is the running-total storage state for a given scope.

func ReadStorageSnapshot 

func ReadStorageSnapshot(db DBSession, scope string) StorageSnapshot

ReadStorageSnapshot returns the running-total bytes_used and file_count for the given scope. Missing rows are treated as zero, and read errors are logged then treated as zero on these best-effort read paths.

type TrafficPeriodRecorder 

type TrafficPeriodRecorder interface {
	RecordWithPeriod(orgID, userID, trafficType string, bytes int64, periodStartedAt time.Time)
}

TrafficPeriodRecorder is the subset of Recorder used after quota pre-checks.

type TrafficQuotaPrechecker 

type TrafficQuotaPrechecker interface {
	CheckTrafficQuota(orgID, userID, direction string, additionalBytes int64) (QuotaStatus, error)
}

TrafficQuotaPrechecker is the subset of Checker used by request handlers.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.