xs

package
v1.4.7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 3, 2026 License: MIT Imports: 47 Imported by: 0

Documentation

Overview

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2021-2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2024-2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2018-2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2018-2025, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2018-2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2018-2025, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2018-2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2018-2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2018-2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2018-2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2018-2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2025-2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2018-2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2018-2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2022-2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2018-2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2024-2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2018-2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2025-2026, NVIDIA CORPORATION. All rights reserved.

Package xact provides core functionality for the AIStore eXtended Actions (xactions).

  • Copyright (c) 2021-2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2021-2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2018-2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2021-2026, NVIDIA CORPORATION. All rights reserved.

Package xs contains most of the supported eXtended actions (xactions) with some exceptions that include certain storage services (mirror, EC) and extensions (downloader, lru).

  • Copyright (c) 2022-2026, NVIDIA CORPORATION. All rights reserved.

Package xs contains most of the supported eXtended actions (xactions) with some exceptions that include certain storage services (mirror, EC) and extensions (downloader, lru).

  • Copyright (c) 2018-2026, NVIDIA CORPORATION. All rights reserved.

Package xs is a collection of eXtended actions (xactions), including multi-object operations, list-objects, (cluster) rebalance and (target) resilver, ETL, and more.

  • Copyright (c) 2025-2026, NVIDIA CORPORATION. All rights reserved.

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrGone = errors.New("gone")
)

Functions

func FreeCOI added in v1.3.26 

func FreeCOI(a *CoiParams)

func Pinit added in v1.4.3 

func Pinit()

for additional startup-time reg-s see lru, downloader, ec

func PrmObjName added in v1.3.22 

func PrmObjName(objfqn, dirfqn, prefix string) (_ string, err error)

func RenewBlobDl added in v1.3.22 

func RenewBlobDl(xid string, params *core.BlobParams, oa *cmn.ObjAttrs) xreg.RenewRes

Blob Download Flow =================================================================

Main goroutine coordinates reusable blobWI instances over workCh/doneCh. Workers perform backend range reads, write chunk files, optionally fill SGLs for streaming, and add completed chunks to the manifest.

Ordering is enforced by the coordinator:

  • completed chunks with roff > woff are kept in pending;
  • chunks at roff == woff advance the ordered stream;
  • SGLs are written to RespWriter sequentially, when streaming;
  • blobWI/SGL cleanup and reuse are coordinator-owned.

Abort/cancelation:

  • Abort() marks the xaction aborted and cancels the blob-download context;
  • range-read contexts derive from that context, so abort interrupts active GetObjReader/body-copy operations;
  • finalize() runs only after range-read I/O has stopped;
  • on success it completes the manifest; on error/abort it aborts it.

=====================================================================================

func Tinit added in v1.4.3 

func Tinit(coi COI)

func TuneBlobDlWorkers added in v1.4.5 

func TuneBlobDlWorkers(xname string, requestedWorkers, numMpaths int, fullSize int64) (int, error)

Effective workers = min(tuned, objCap, blobMaxWorkers), where

  • tuned = TuneNumWorkers(requested, mpaths) -- folds in user request, media, system load;
  • objCap = ceil(fullSize / blobMinBytesPerWorker) -- one worker per ~256MiB of data.

`requested == -1` (NwpNone) and extreme load short-circuit to -1 (serial). Examples (auto request, 4 mountpaths):

  • 64 MiB, SSD : tuned=16, objCap=1 -> 1 (object too small to fan out)
  • 2 GiB, NVMe : tuned=32, objCap=8 -> 8 (object-size cap binds)
  • 8 GiB, NVMe : tuned=32, objCap=32 -> 32 (hits hard cap)
  • 16 GiB, NVMe: tuned=32, objCap=64 -> 32 (clamped by blobMaxWorkers)

Types

type BckRename added in v1.4.0 

type BckRename struct {
	*XactTCB
	// contains filtered or unexported fields
}

func (*BckRename) CtlMsg added in v1.4.1 

func (r *BckRename) CtlMsg() string

func (*BckRename) FromTo added in v1.4.0 

func (r *BckRename) FromTo() (*meta.Bck, *meta.Bck)

func (*BckRename) Name added in v1.4.0 

func (r *BckRename) Name() string

func (*BckRename) Run added in v1.4.0 

func (r *BckRename) Run(wg *sync.WaitGroup)

BckRename xaction is a wrapper around XactTCB that adds the following: - if not aborted; call BMDVersionFixup to piggyback bucket renaming to remove bckFrom from BMD (see `whatRenamedLB` in proxy.go) -

func (*BckRename) Snap added in v1.4.0 

func (r *BckRename) Snap() (snap *core.Snap)

func (*BckRename) String added in v1.4.0 

func (r *BckRename) String() string

type COI added in v1.3.26 

type COI interface {
	CopyObject(lom *core.LOM, dm *bundle.DM, coi *CoiParams) CoiRes
}

type CoiParams added in v1.3.26 

type CoiParams struct {
	Xact   core.Xact
	OAH    cos.OAH // object attributes after applying core.GetROC
	Config *cmn.Config
	BckTo  *meta.Bck
	core.GetROC
	core.PutWOC
	ETLArgs         *core.ETLArgs
	ObjnameTo       string
	Buf             []byte
	OWT             cmn.OWT
	Finalize        bool // copies and EC (as in poi.finalize())
	DryRun          bool // no changes
	LatestVer       bool // can be used without changing bucket's 'versioning.validate_warm_get'; see also: QparamLatestVer
	Sync            bool // see core.GetROC at core/ldp.go
	ContinueOnError bool // when false, a failure to copy triggers abort
}

func AllocCOI added in v1.3.26 

func AllocCOI() *CoiParams

type CoiRes added in v1.3.28 

type CoiRes struct {
	Err   error
	Lsize int64
	Ecode int
	RGET  bool // when reading source via backend.GetObjReader
}

type Election 

type Election struct {
	xact.Base
}

func (*Election) CtlMsg added in v1.4.1 

func (*Election) CtlMsg() string

func (*Election) Run 

func (*Election) Run(*sync.WaitGroup)

func (*Election) Snap added in v1.3.16 

func (r *Election) Snap() *core.Snap

type LsoRsp 

type LsoRsp struct {
	Err    error
	Lst    *cmn.LsoRes
	Status int
}

`on-demand` per list-objects request

type LsoXact 

type LsoXact struct {
	// contains filtered or unexported fields
}

`on-demand` per list-objects request

func (*LsoXact) Abort 

func (r *LsoXact) Abort(err error) (ok bool)

skip on-demand idleness check

func (*LsoXact) CtlMsg added in v1.4.1 

func (r *LsoXact) CtlMsg() string

func (*LsoXact) Do 

func (r *LsoXact) Do(msg *apc.LsoMsg) *LsoRsp

func (*LsoXact) Run 

func (r *LsoXact) Run(wg *sync.WaitGroup)

func (*LsoXact) Snap 

func (r *LsoXact) Snap() *core.Snap

func (*LsoXact) String 

func (r *LsoXact) String() (s string)

func (*LsoXact) TxnAbort 

func (r *LsoXact) TxnAbort(err error)

limited pre-run abort

type Rebalance 

type Rebalance struct {
	Args *xreg.RebArgs
	xact.Base
	// contains filtered or unexported fields
}

func (*Rebalance) CtlMsg added in v1.4.1 

func (xreb *Rebalance) CtlMsg() string

implementation (via xreb.ctl.fn) - in reb/ctlmsg.go e.g.: "t[xyz]:<fin-streams> done trav:<1s post-trav:2s fin:46s fin-streams:12s errs:3"

func (*Rebalance) FinalCtlMsg added in v1.4.5 

func (xreb *Rebalance) FinalCtlMsg()

func (*Rebalance) RebID 

func (xreb *Rebalance) RebID() int64

func (*Rebalance) Run 

func (*Rebalance) Run(*sync.WaitGroup)

func (*Rebalance) Snap 

func (xreb *Rebalance) Snap() (snap *core.Snap)

type Resilver 

type Resilver struct {
	Args *xreg.ResArgs

	xact.Base
	Nbusy atomic.Int64
	// contains filtered or unexported fields
}

func (*Resilver) CtlMsg added in v1.4.1 

func (xres *Resilver) CtlMsg() string

func (*Resilver) Run 

func (*Resilver) Run(*sync.WaitGroup)

func (*Resilver) SetJgroup added in v1.4.2 

func (xres *Resilver) SetJgroup(jgroup *mpather.Jgroup)

func (*Resilver) Snap 

func (xres *Resilver) Snap() *core.Snap

func (*Resilver) String 

func (xres *Resilver) String() string

type TestBmvFactory 

type TestBmvFactory = bmvFactory

type XactArch 

type XactArch struct {
	// contains filtered or unexported fields
}

func (*XactArch) BeginMsg added in v1.3.28 

func (r *XactArch) BeginMsg(msg *cmn.ArchiveBckMsg, archlom *core.LOM) (err error)

func (*XactArch) CtlMsg added in v1.4.1 

func (r *XactArch) CtlMsg() string

func (*XactArch) DoMsg added in v1.3.28 

func (r *XactArch) DoMsg(msg *cmn.ArchiveBckMsg)

func (*XactArch) FromTo added in v1.3.16 

func (r *XactArch) FromTo() (src, dst *meta.Bck)

func (*XactArch) Name added in v1.3.16 

func (r *XactArch) Name() (s string)

func (*XactArch) Run 

func (r *XactArch) Run(wg *sync.WaitGroup)

func (*XactArch) Snap 

func (r *XactArch) Snap() (snap *core.Snap)

func (*XactArch) String 

func (r *XactArch) String() (s string)

func (*XactArch) TxnAbort 

func (r *XactArch) TxnAbort(err error)

limited pre-run abort

type XactBlobDl added in v1.3.22 

type XactBlobDl struct {
	xact.Base
	// contains filtered or unexported fields
}

func (*XactBlobDl) Abort added in v1.4.1 

func (r *XactBlobDl) Abort(err error) bool

func (*XactBlobDl) CtlMsg added in v1.4.1 

func (r *XactBlobDl) CtlMsg() string

func (*XactBlobDl) Name added in v1.3.22 

func (r *XactBlobDl) Name() string

func (*XactBlobDl) Run added in v1.3.22 

func (r *XactBlobDl) Run(wg *sync.WaitGroup)

func (*XactBlobDl) Size added in v1.3.26 

func (r *XactBlobDl) Size() int64

func (*XactBlobDl) Snap added in v1.3.22 

func (r *XactBlobDl) Snap() (snap *core.Snap)

func (*XactBlobDl) String added in v1.3.26 

func (r *XactBlobDl) String() string

TODO: remove; use CtlMsg() instead

type XactDirPromote 

type XactDirPromote struct {
	xact.BckJog
	// contains filtered or unexported fields
}

func (*XactDirPromote) CtlMsg added in v1.4.1 

func (r *XactDirPromote) CtlMsg() string

func (*XactDirPromote) Run 

func (r *XactDirPromote) Run(wg *sync.WaitGroup)

func (*XactDirPromote) SetFshare 

func (r *XactDirPromote) SetFshare(v bool)

func (*XactDirPromote) Snap added in v1.3.16 

func (r *XactDirPromote) Snap() *core.Snap

type XactMoss added in v1.3.30 

type XactMoss struct {
	xact.DemandBase
	// contains filtered or unexported fields
}

func (*XactMoss) Abort added in v1.3.30 

func (r *XactMoss) Abort(err error) bool

base abort + more (do NOT call from the asm() path - activeWG must be done)

func (*XactMoss) Assemble added in v1.3.30 

func (r *XactMoss) Assemble(req *apc.MossReq, w http.ResponseWriter, wid string) error

(phase 2) assemble requested data (local and remote); emit resulting archive a note on error handling: - buffered mode: full error propagation, real HTTP status, client sees the error - streaming: truncated TAR on the wire, user sees the error via CLI 'ais show job' ex post facto

func (*XactMoss) CtlMsg added in v1.4.1 

func (r *XactMoss) CtlMsg() string

func (*XactMoss) PrepRx added in v1.3.30 

func (r *XactMoss) PrepRx(req *apc.MossReq, config *cmn.Config, smap *meta.Smap, wid string, nat int, usingPrev bool) error

(phase 1)

func (*XactMoss) RecvCtrl added in v1.4.5 

func (r *XactMoss) RecvCtrl(wid, sid, opcode string, body []byte) error

func (*XactMoss) RecvObj added in v1.3.30 

func (r *XactMoss) RecvObj(hdr *transport.ObjHdr, reader io.Reader, err error) error

demux -> wi.recv() [convention:] received hdr.ObjName is `nameInArch` (ie., filename in resulting TAR)

func (*XactMoss) Run added in v1.3.30 

func (*XactMoss) Run(*sync.WaitGroup)

func (*XactMoss) Send added in v1.3.30 

func (r *XactMoss) Send(req *apc.MossReq, smap *meta.Smap, dt *meta.Snode, wid string) error

send all requested local data => DT (`tsi`) (phase 3)

func (*XactMoss) Snap added in v1.3.30 

func (r *XactMoss) Snap() *core.Snap

type XactNBI added in v1.4.3 

type XactNBI struct {
	xact.Base
	// contains filtered or unexported fields
}

x-nbi

func (*XactNBI) Abort added in v1.4.3 

func (r *XactNBI) Abort(err error) bool

func (*XactNBI) CtlMsg added in v1.4.3 

func (r *XactNBI) CtlMsg() string

func (*XactNBI) Run added in v1.4.3 

func (r *XactNBI) Run(wg *sync.WaitGroup)

main method

func (*XactNBI) Snap added in v1.4.3 

func (r *XactNBI) Snap() (snap *core.Snap)

type XactNsumm added in v1.3.21 

type XactNsumm struct {
	xact.BckJog
	// contains filtered or unexported fields
}

TODO: migrate to xact.BckJogRunner to reduce boilerplate and gain auto-tuned worker pool

func (*XactNsumm) CtlMsg added in v1.4.1 

func (r *XactNsumm) CtlMsg() string

func (*XactNsumm) Name added in v1.3.21 

func (r *XactNsumm) Name() string

func (*XactNsumm) Result added in v1.3.21 

func (r *XactNsumm) Result() (cmn.AllBsummResults, error)

func (*XactNsumm) Run added in v1.3.21 

func (r *XactNsumm) Run(started *sync.WaitGroup)

func (*XactNsumm) Snap added in v1.3.21 

func (r *XactNsumm) Snap() *core.Snap

func (*XactNsumm) String added in v1.3.21 

func (r *XactNsumm) String() string

type XactShardSumm added in v1.4.7 

type XactShardSumm struct {

	// TODO: include orphaned index count by sampling sysbucket indexes.
	xact.BckJogRunner
	// contains filtered or unexported fields
}

func (*XactShardSumm) CtlMsg added in v1.4.7 

func (r *XactShardSumm) CtlMsg() string

func (*XactShardSumm) Result added in v1.4.7 

func (r *XactShardSumm) Result() (*apc.ShardSummResult, error)

func (*XactShardSumm) Run added in v1.4.7 

func (r *XactShardSumm) Run(wg *sync.WaitGroup)

func (*XactShardSumm) Snap added in v1.4.7 

func (r *XactShardSumm) Snap() *core.Snap

type XactTCB added in v1.3.22 

type XactTCB struct {
	xact.BckJogRunner // mountpath joggers + managed worker pool
	// contains filtered or unexported fields
}

func (*XactTCB) Args added in v1.3.22 

func (r *XactTCB) Args() *xreg.TCBArgs

func (*XactTCB) CtlMsg added in v1.4.1 

func (r *XactTCB) CtlMsg() string

func (*XactTCB) FromTo added in v1.3.22 

func (r *XactTCB) FromTo() (*meta.Bck, *meta.Bck)

func (*XactTCB) Name added in v1.3.22 

func (r *XactTCB) Name() string

func (*XactTCB) Run added in v1.3.22 

func (r *XactTCB) Run(wg *sync.WaitGroup)

func (*XactTCB) Snap added in v1.3.22 

func (r *XactTCB) Snap() (snap *core.Snap)

func (*XactTCB) String added in v1.3.22 

func (r *XactTCB) String() string

func (*XactTCB) TxnAbort added in v1.3.22 

func (r *XactTCB) TxnAbort(err error)

limited pre-run abort

type XactTCO added in v1.3.28 

type XactTCO struct {
	// contains filtered or unexported fields
}

func (*XactTCO) BeginMsg added in v1.3.28 

func (r *XactTCO) BeginMsg(msg *cmn.TCOMsg)

func (*XactTCO) ContMsg added in v1.3.28 

func (r *XactTCO) ContMsg(msg *cmn.TCOMsg)

func (*XactTCO) CtlMsg added in v1.4.1 

func (r *XactTCO) CtlMsg() string

func (*XactTCO) FromTo added in v1.3.28 

func (r *XactTCO) FromTo() (*meta.Bck, *meta.Bck)

func (*XactTCO) Name added in v1.3.28 

func (r *XactTCO) Name() string

func (*XactTCO) Run added in v1.3.28 

func (r *XactTCO) Run(wg *sync.WaitGroup)

func (*XactTCO) Snap added in v1.3.28 

func (r *XactTCO) Snap() (snap *core.Snap)

func (*XactTCO) String added in v1.3.28 

func (r *XactTCO) String() string

func (*XactTCO) TxnAbort added in v1.3.28 

func (r *XactTCO) TxnAbort(err error)

limited pre-run abort

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.