Documentation
¶
Overview ¶
Package storage owns provider-independent placement, replicas and recovery. Callers supply current authorization; credentials are resolved outside this package.
Index ¶
- Variables
- type Admission
- type Authorizer
- type Config
- type ContentGroup
- type ContentHandle
- type ContributionCandidate
- type ContributionCandidates
- type ContributionControl
- type ContributionSnapshot
- type CreateRequest
- type CredentialRenewal
- type Driver
- type Event
- type EventData
- type Idempotency
- type Instance
- type InstanceCommand
- type InstanceEvidence
- type InstanceHealth
- type Job
- type JobFailure
- type JobTotals
- type Locator
- type MaintenanceResult
- type Object
- type ObjectHealth
- type Operations
- type Policy
- type Principal
- type Protection
- type Reference
- type Replica
- type Repository
- type Resolver
- type Service
- func (s *Service) ApplyContribution(ctx context.Context, p Principal, id string, c ContributionControl) (ContributionSnapshot, error)
- func (s *Service) ChangeProtection(ctx context.Context, p Principal, id, key string, revision int64, ...) (Status, error)
- func (s *Service) Close(ctx context.Context) error
- func (s *Service) CommandInstance(ctx context.Context, p Principal, id, key string, command InstanceCommand) (InstanceHealth, error)
- func (s *Service) Create(ctx context.Context, p Principal, key string, r CreateRequest) (Status, error)
- func (s *Service) Delete(ctx context.Context, p Principal, id, key string, revision int64) (Status, error)
- func (s *Service) Download(ctx context.Context, p Principal, id string, dst io.Writer) error
- func (s *Service) Ingest(ctx context.Context, p Principal, id, key string, src io.Reader) (Status, error)
- func (s *Service) InspectContribution(ctx context.Context, p Principal, id, owner string) (ContributionSnapshot, error)
- func (s *Service) Instances(ctx context.Context, p Principal, limit int, after string) ([]Instance, error)
- func (s *Service) ListContributionCandidates(ctx context.Context, p Principal, owner string, n int, after string) (ContributionCandidates, error)
- func (s *Service) Operations(ctx context.Context, p Principal, limit int, after, failuresAfter string) (Operations, error)
- func (s *Service) ReauthorizeInstance(ctx context.Context, p Principal, key string, q CredentialRenewal) (Instance, error)
- func (s *Service) Reconcile(ctx context.Context, p Principal, limit int, after string) (MaintenanceResult, error)
- func (s *Service) Reference(ctx context.Context, p Principal, id, key string, ref Reference, attach bool) (Status, error)
- func (s *Service) RefreshInstance(ctx context.Context, p Principal, id string) error
- func (s *Service) Register(ctx context.Context, p Principal, key string, v Instance) (Instance, error)
- func (s *Service) RetryJob(ctx context.Context, p Principal, objectID, jobID, key string, ...) (JobFailure, error)
- func (s *Service) ReviewDownload(ctx context.Context, p Principal, id string, dst io.Writer) error
- func (s *Service) RunOne(ctx context.Context, worker string) (bool, error)
- func (s *Service) Status(ctx context.Context, p Principal, id string) (Status, error)
- func (s *Service) UpdateInstance(ctx context.Context, p Principal, key string, v Instance) (Instance, error)
- type Status
- type Transaction
- type Verifier
Constants ¶
This section is empty.
Variables ¶
var ( ErrInput = errors.New("storage_invalid_input") ErrDenied = errors.New("storage_permission_denied") ErrNotFound = errors.New("storage_not_found") ErrConflict = errors.New("storage_conflict") ErrCapacity = errors.New("storage_capacity_unavailable") ErrIntegrity = errors.New("storage_integrity_failed") ErrStale = errors.New("storage_stale_operation") ErrUnknown = errors.New("storage_remote_result_unknown") )
Functions ¶
This section is empty.
Types ¶
type Authorizer ¶
type Authorizer interface {
Authorize(context.Context, Principal, string, string, Protection) error
}
Authorizer must check the current actor, action, object and protection scope. HTTP headers and previously issued publication events are not authorization facts.
type Config ¶
type Config struct {
Clock func() time.Time
MaxObjectSize int64
TemporaryDirectory string
ParallelTransfers int
Lease time.Duration
VerificationAge time.Duration
QuotaValidity time.Duration
MaxAttempts int
TemporaryBytes int64
MaxOutstandingJobs int
AccountTransfers int
BytesPerSecond int64
AccountBytesPerSecond int64
}
type ContentGroup ¶
type ContentGroup struct {
CleanupCheckedAt time.Time `json:"cleanup_checked_at"`
ID string `json:"id"`
Fingerprint string `json:"fingerprint"`
ObjectID string `json:"object_id"`
Owner Principal `json:"owner"`
Size int64 `json:"size"`
SHA256 string `json:"sha256"`
Protection Protection `json:"protection"`
Policy Policy `json:"policy"`
State string `json:"state"`
}
ContentGroup owns one immutable physical object. Handles retain independent authorization, references and publication state. Group identity includes the service principal and every placement requirement; a digest is not access proof.
type ContentHandle ¶
type ContentHandle struct {
CreatedAt time.Time `json:"created_at"`
ID string `json:"id"`
GroupID string `json:"group_id"`
Owner Principal `json:"owner"`
CreationHash string `json:"creation_hash"`
Revision int64 `json:"revision"`
State string `json:"state"`
Verified bool `json:"verified"`
Published bool `json:"published"`
References []Reference `json:"references"`
SourceGroup string `json:"source_group,omitempty"`
SourceObject string `json:"source_object,omitempty"`
SourceID string `json:"source_id,omitempty"`
PendingGroup string `json:"pending_group,omitempty"`
}
type ContributionCandidate ¶
type ContributionCandidates ¶
type ContributionCandidates struct {
Items []ContributionCandidate `json:"items"`
NextCursor string `json:"next_cursor"`
}
type ContributionControl ¶
type ContributionControl struct {
ID string `json:"id"`
OwnerDomain string `json:"owner_domain"`
Revision int64 `json:"revision"`
ExpectedRevision int64 `json:"expected_revision"`
State string `json:"state"`
LimitBytes int64 `json:"limit_bytes"`
PersonalReserve int64 `json:"personal_reserve"`
Until time.Time `json:"until"`
}
ContributionControl is a permanent, monotonically revised business binding. It contains no provider credential and cannot extend provider authorization.
type ContributionSnapshot ¶
type ContributionSnapshot struct {
InstanceID string `json:"instance_id"`
Adapter string `json:"adapter"`
Revision int64 `json:"revision"`
Control *ContributionControl `json:"control,omitempty"`
State string `json:"state"`
Authorized bool `json:"authorized"`
AuthorizationExpiresAt time.Time `json:"authorization_expires_at"`
QuotaExpiresAt time.Time `json:"quota_expires_at"`
UsedBytes int64 `json:"used_bytes"`
ReservedBytes int64 `json:"reserved_bytes"`
AvailableBytes int64 `json:"available_bytes"`
RemainingReplicas int64 `json:"remaining_replicas"`
ObservedAt time.Time `json:"observed_at"`
}
type CreateRequest ¶
type CreateRequest struct {
Deduplicate bool `json:"deduplicate,omitempty"`
ReuseFrom string `json:"reuse_from,omitempty"`
ContentGroup string `json:"-"`
ClientKey string `json:"client_key,omitempty"`
Size int64 `json:"size"`
SHA256 string `json:"sha256"`
Protection Protection `json:"protection"`
Policy Policy `json:"policy"`
}
type CredentialRenewal ¶
type Driver ¶
type Driver interface {
Find(context.Context, Instance, string, Locator) (Locator, error)
Write(context.Context, Instance, string, int64, io.Reader, func(Locator) error) (Locator, error)
Read(context.Context, Instance, Locator, io.Writer) (int64, error)
Remove(context.Context, Instance, Locator) error
}
Driver's intent is a unique, platform-owned name in an isolated root. Find reconciles a write whose result was lost. Checkpoint must run before data write. Errors must be sanitized; absence must be explicit ErrNotFound.
type Event ¶
type Event struct {
SpecVersion string `json:"specversion"`
Source string `json:"source"`
Subject string `json:"subject"`
DataContentType string `json:"datacontenttype"`
Data EventData `json:"data"`
ID string `json:"id"`
Type string `json:"type"`
ObjectID string `json:"-"`
Revision int64 `json:"-"`
Time time.Time `json:"time"`
Actor Principal `json:"-"`
JobID string `json:"-"`
}
type Idempotency ¶
type Idempotency struct {
Key, Digest, ObjectID string
Result json.RawMessage
ExpiresAt time.Time
}
type Instance ¶
type Instance struct {
ContributionControl *ContributionControl `json:"contribution_control,omitempty"`
ID string `json:"id"`
Adapter string `json:"adapter"`
AccountDomain string `json:"account_domain"`
OwnerDomain string `json:"owner_domain"`
Ownership string `json:"ownership"`
CredentialRef string `json:"credential_ref"`
CredentialRevision int64 `json:"credential_revision"`
Root string `json:"root"`
Revision int64 `json:"revision"`
State string `json:"state"`
Authorized bool `json:"authorized"`
Limit int64 `json:"limit"`
Used int64 `json:"used"`
Reserved int64 `json:"reserved"`
ProviderFree int64 `json:"provider_free"`
PersonalReserve int64 `json:"personal_reserve"`
SafetyMargin int64 `json:"safety_margin"`
QuotaRevision int64 `json:"quota_revision"`
QuotaExpiresAt time.Time `json:"quota_expires_at"`
ExpiresAt time.Time `json:"expires_at"`
Admissions []Admission `json:"admissions"`
}
type InstanceCommand ¶
type InstanceEvidence ¶
type InstanceHealth ¶
type InstanceHealth struct {
Revision int64 `json:"revision"`
LimitBytes int64 `json:"limit_bytes"`
UsedBytes int64 `json:"used_bytes"`
ReservedBytes int64 `json:"reserved_bytes"`
ID string `json:"id"`
State string `json:"state"`
Reason string `json:"reason"`
AvailableBytes int64 `json:"available_bytes"`
AuthorizationExpiresAt time.Time `json:"authorization_expires_at"`
QuotaExpiresAt time.Time `json:"quota_expires_at"`
}
type Job ¶
type Job struct {
Intent string `json:"intent"`
ObjectRevision int64 `json:"object_revision"`
InstanceRevision int64 `json:"instance_revision"`
CredentialRevision int64 `json:"credential_revision"`
ID string `json:"id"`
ObjectID string `json:"object_id"`
ReplicaID string `json:"replica_id"`
Kind string `json:"kind"`
State string `json:"state"`
Generation int64 `json:"generation"`
LeaseOwner string `json:"lease_owner"`
LeaseUntil time.Time `json:"lease_until"`
AvailableAt time.Time `json:"available_at"`
Attempts int `json:"attempts"`
ErrorCode string `json:"error_code"`
}
type JobFailure ¶
type MaintenanceResult ¶
type Object ¶
type Object struct {
ContentGroup string `json:"content_group,omitempty"`
CreationHash string `json:"creation_hash,omitempty"`
ID string `json:"id"`
Owner Principal `json:"owner"`
Size int64 `json:"size"`
SHA256 string `json:"sha256"`
Protection Protection `json:"protection"`
Policy Policy `json:"policy"`
Revision int64 `json:"revision"`
State string `json:"state"`
Published bool `json:"published"`
CreatedAt time.Time `json:"created_at"`
References []Reference `json:"references"`
}
type ObjectHealth ¶
type Operations ¶
type Operations struct {
GeneratedAt time.Time `json:"generated_at"`
Instances []InstanceHealth `json:"instances"`
Objects []ObjectHealth `json:"objects"`
Jobs JobTotals `json:"jobs"`
Failures []JobFailure `json:"failures"`
NextCursor string `json:"next_cursor"`
FailuresNextCursor string `json:"failures_next_cursor"`
}
type Policy ¶
func DefaultPolicy ¶
func DefaultPolicy() Policy
type Protection ¶
type Protection struct {
Class string `json:"class"`
Scope string `json:"scope"`
Public bool `json:"public"`
Encrypted bool `json:"encrypted"`
}
func (Protection) Valid ¶
func (p Protection) Valid() bool
type Replica ¶
type Replica struct {
ID string `json:"id"`
ObjectID string `json:"object_id"`
InstanceID string `json:"instance_id"`
Intent string `json:"intent"`
Remote Locator `json:"remote"`
State string `json:"state"`
InstanceRevision int64 `json:"instance_revision"`
CredentialRevision int64 `json:"credential_revision"`
ObjectRevision int64 `json:"object_revision"`
VerifiedAt time.Time `json:"verified_at"`
Size int64 `json:"size"`
Reserved bool `json:"reserved"`
Accounted bool `json:"accounted"`
IssuedWrite bool `json:"issued_write"`
}
type Repository ¶
type Service ¶
type Service struct {
// contains filtered or unexported fields
}
func New ¶
func New(repo Repository, auth Authorizer, resolver Resolver, verifier Verifier, cfg Config) (*Service, error)
func (*Service) ApplyContribution ¶
func (s *Service) ApplyContribution(ctx context.Context, p Principal, id string, c ContributionControl) (ContributionSnapshot, error)
func (*Service) ChangeProtection ¶
func (*Service) Close ¶
Close rejects new transfers, cancels active I/O and waits for cleanup. The directory stays locked until the last operation actually exits, even on timeout.
func (*Service) CommandInstance ¶
func (s *Service) CommandInstance(ctx context.Context, p Principal, id, key string, command InstanceCommand) (InstanceHealth, error)
CommandInstance deliberately cannot change identity, credentials, admissions or authorization deadlines. Restoring revoked custody requires fresh evidence.
func (*Service) Download ¶
Download buffers within the configured size/concurrency budget so corrupted or replaced remote content cannot partially escape before checksum validation.
func (*Service) Ingest ¶
func (s *Service) Ingest(ctx context.Context, p Principal, id, key string, src io.Reader) (Status, error)
Ingest hashes bounded local staging before any provider write. The staging file is removed on return; retry uses the immutable object's size and digest.
func (*Service) InspectContribution ¶
func (*Service) ListContributionCandidates ¶
func (*Service) Operations ¶
func (s *Service) Operations(ctx context.Context, p Principal, limit int, after, failuresAfter string) (Operations, error)
Operations is a global administration privilege, independent of content read access. It reads the ledger only and never extends supplier sessions.
func (*Service) ReauthorizeInstance ¶
func (s *Service) ReauthorizeInstance(ctx context.Context, p Principal, key string, q CredentialRenewal) (Instance, error)
ReauthorizeInstance changes only custody facts. Current contribution, quota, state, ownership and admissions are copied inside the committing transaction.
func (*Service) Reconcile ¶
func (s *Service) Reconcile(ctx context.Context, p Principal, limit int, after string) (MaintenanceResult, error)
Reconcile is bounded and resumable. Expired unreferenced uploads are tombstoned; no operation recursively deletes a provider directory or personal files.
func (*Service) RefreshInstance ¶
RefreshInstance samples capacity only without outstanding writes. This avoids counting the same in-flight write both in provider usage and local reservations.
func (*Service) Register ¶
func (s *Service) Register(ctx context.Context, p Principal, key string, v Instance) (Instance, error)
Register verifies account identity before accepting administrative placement claims. An alias or a second directory under the same account cannot add capacity.
func (*Service) RetryJob ¶
func (s *Service) RetryJob(ctx context.Context, p Principal, objectID, jobID, key string, generation int64) (JobFailure, error)
RetryJob does not reconstruct a write. It requeues the original intent only when the bindings still agree, leaving unknown-write reconciliation to workers.
func (*Service) ReviewDownload ¶
ReviewDownload uses a separate permission to inspect an unpublished verified copy. It never changes publication, references, replica policy or protection.
func (*Service) RunOne ¶
RunOne claims a durable lease and executes one bounded operation. No in-memory queue is required to resume after a worker restart.
type Status ¶
type Status struct {
ID string `json:"id"`
Size int64 `json:"size"`
SHA256 string `json:"sha256"`
Protection Protection `json:"protection"`
Policy Policy `json:"policy"`
Revision int64 `json:"revision"`
State string `json:"state"`
Published bool `json:"published"`
Healthy int `json:"healthy"`
Desired int `json:"desired"`
}
type Transaction ¶
type Transaction interface {
ContentGroup(string) (ContentGroup, error)
ActiveContentGroup(string) (ContentGroup, error)
SaveContentGroup(ContentGroup) error
ContentHandle(string) (ContentHandle, error)
SaveContentHandle(ContentHandle) error
ContentGroupHeld(string) (bool, error)
ContentCleanup(int, string) ([]ContentGroup, error)
ExpiredContentHandles(int, time.Time) ([]ContentHandle, error)
Lock(string) error
Instance(string) (Instance, error)
Instances(int, string) ([]Instance, error)
ContributionCandidates(string, int, string) ([]Instance, error)
SaveInstance(Instance) error
InstanceReplicas(string) (int64, error)
Object(string) (Object, error)
Objects(int, string) ([]Object, error)
SaveObject(Object) error
Replicas(string) ([]Replica, error)
SaveReplica(Replica) error
Job(string) (Job, error)
// JobSnapshot reads a binding without a row lock, for instance-before-job locking.
JobSnapshot(string) (Job, error)
SaveJob(Job) error
OutstandingJobs(string, int) (int, error)
ActiveWrites(string, time.Time) (bool, error)
JobTotals() (JobTotals, error)
FailedJobs(int, string) ([]Job, error)
Idempotency(string) (Idempotency, error)
SaveIdempotency(Idempotency) error
Event(Event) error
}
Locks are acquired object -> instances in sorted ID order -> job. Claim only locks a job and commits before execution, so it cannot invert that order.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package custody coordinates explicit provider custody separately from storage contribution.
|
Package custody coordinates explicit provider custody separately from storage contribution. |
|
drivers
|
|
|
bhpan
Package bhpan connects the independent BHpan protocol library to storage.
|
Package bhpan connects the independent BHpan protocol library to storage. |
|
Package executor runs bounded workers against the storage-owned durable queue.
|
Package executor runs bounded workers against the storage-owned durable queue. |
|
Package httpapi exposes the storage contract.
|
Package httpapi exposes the storage contract. |
|
Package postgres implements the storage-owned transactional ledger.
|
Package postgres implements the storage-owned transactional ledger. |
|
Package sdk provides primitives to interact with the openapi HTTP API.
|
Package sdk provides primitives to interact with the openapi HTTP API. |