s3scanner

Documentation

Index

• type BucketStatistics
  • func (stats *BucketStatistics) Cost() float32
• type NonVersionedBucketError
  • func (e *NonVersionedBucketError) Error() string
• type S3ClientAPI
• type S3Object
• type S3ObjectMetadata
  • func (o *S3ObjectMetadata) Serialize() []byte
• type Scanner
  • func NewScanner(s3Client S3ClientAPI, ctx context.Context, maxConcurrentScans int) (*Scanner, error)
  • func (s *Scanner) Scan(bucketName string, fn func(o *S3Object) error) (*BucketStatistics, error)
  • func (s *Scanner) SetLogger(logger *log.Logger)

Types

type BucketStatistics

type BucketStatistics struct {
	Pages   uint64
	Objects uint64
}

BucketStatistics contains information about the number of pages and objects processed during the scanning operation.

func (*BucketStatistics) Cost

func (stats *BucketStatistics) Cost() float32

Cost calculates the cost of the scanning operation based on the number of pages processed. It returns the cost as a float32.

type NonVersionedBucketError

type NonVersionedBucketError struct {
	BucketName string
}

NonVersionedBucketError is an error type representing an error encountered when attempting to scan a non-versioned S3 bucket. This error is returned by the Scan function if the specified bucket does not have versioning enabled.

func (*NonVersionedBucketError) Error

func (e *NonVersionedBucketError) Error() string

type S3ClientAPI

type S3ClientAPI interface {
	ListObjectVersions(ctx context.Context, params *s3.ListObjectVersionsInput, optFns ...func(*s3.Options)) (*s3.ListObjectVersionsOutput, error)
	GetBucketVersioning(ctx context.Context, params *s3.GetBucketVersioningInput, optFns ...func(*s3.Options)) (*s3.GetBucketVersioningOutput, error)
}

S3ClientAPI is an interface that defines a minimal set of methods required from the S3 client for the Scanner to function. This interface makes it easier to use a mock client for testing purposes without relying on an actual S3 service.

type S3Object

type S3Object struct {
	Key      *string
	Metadata S3ObjectMetadata
}

S3Object represents an object within an S3 bucket.

type S3ObjectMetadata

type S3ObjectMetadata struct {
	VersionId      *string
	LastModified   *time.Time
	IsDeleteMarker bool
	IsLatest       bool
}

S3ObjectMetadata holds metadata associated with an S3Object.

func (*S3ObjectMetadata) Serialize

func (o *S3ObjectMetadata) Serialize() []byte

Serialize returns the JSON byte representation of an S3ObjectMetadata instance, ignoring any marshaling errors.

type Scanner

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner is a concurrent S3 bucket scanner.

func NewScanner

func NewScanner(s3Client S3ClientAPI, ctx context.Context, maxConcurrentScans int) (*Scanner, error)

NewScanner creates a new Scanner instance with the specified context and maximum number of concurrent folder workers. It returns a pointer to the Scanner and an error if any occurred.

func (*Scanner) Scan

func (s *Scanner) Scan(bucketName string, fn func(o *S3Object) error) (*BucketStatistics, error)

Scan performs a concurrent scan of the specified S3 bucket, processing each object using the provided function. It returns a pointer to a BucketStatistics instance containing the number of pages and objects processed, and an error if any occurred.

func (*Scanner) SetLogger

func (s *Scanner) SetLogger(logger *log.Logger)

SetLogger allows users to set a custom logger for the Scanner instance. The provided logger will be used for logging messages during the scanning process.