testutil

package
v0.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 31, 2025 License: MIT Imports: 10 Imported by: 0

README

StorageX Testing Utilities

This package provides testing utilities for StorageX. For most S3 testing needs, we recommend using gofakes3 instead of the custom mock.

gofakes3 is a production-ready, in-memory S3 server that's fully compatible with AWS SDK v2:

import (
    "github.com/johannesboyne/gofakes3"
    "github.com/johannesboyne/gofakes3/backend/s3mem"
    "net/http/httptest"
)

func TestWithFakeS3(t *testing.T) {
    // Create in-memory S3 server
    backend := s3mem.New()
    faker := gofakes3.New(backend)
    ts := httptest.NewServer(faker.Server())
    defer ts.Close()
    
    // Create bucket
    backend.CreateBucket("test-bucket")
    
    // Configure storage to use fake server
    cfg := &storagex.Config{
        Bucket:       "test-bucket",
        Endpoint:     ts.URL,
        AccessKey:    "TEST",
        SecretKey:    "TEST",
        UsePathStyle: true,
    }
    
    storage, _ := s3.NewS3Storage(context.Background(), cfg)
    // Use storage like real S3!
}

Advantages of gofakes3:

  • ✅ Full S3 protocol implementation
  • ✅ Works with real AWS SDK
  • ✅ Battle-tested in production
  • ✅ Catches S3-specific edge cases
  • ✅ No mocking needed

See /Users/danecao/source/gostratum/storagex/test/integration_test.go for a complete example.

Alternative: Custom MockStorage

For simple interface-level testing without HTTP overhead:

Usage

import (
    "context"
    "testing"
    
    "github.com/gostratum/storagex/internal/testutil"
    "github.com/stretchr/testify/assert"
    "github.com/stretchr/testify/require"
)

func TestYourCode(t *testing.T) {
    // Create mock storage
    storage := testutil.NewMockStorage()
    
    // Use it like any other storagex.Storage
    ctx := context.Background()
    
    // Put an object
    stat, err := storage.PutBytes(ctx, "test/file.txt", []byte("hello world"), nil)
    require.NoError(t, err)
    assert.Equal(t, int64(11), stat.Size)
    
    // Get the object
    reader, stat, err := storage.Get(ctx, "test/file.txt")
    require.NoError(t, err)
    defer reader.Close()
    
    // Read content
    data, err := io.ReadAll(reader)
    require.NoError(t, err)
    assert.Equal(t, "hello world", string(data))
}

Features

The mock storage implements all storagex.Storage interface methods:

  • ✅ Put/PutBytes: Store objects in memory
  • ✅ Get: Retrieve objects with metadata
  • ✅ Head: Get object metadata without content
  • ✅ List: List objects with prefix/delimiter filtering and pagination
  • ✅ Delete/DeleteBatch: Delete objects (idempotent like S3)
  • ✅ MultipartUpload: Simplified multipart upload (stores as regular object)
  • ✅ PresignGet/PresignPut: Returns mock URLs
  • ✅ Thread-safe: All operations use read/write locks

Testing Against Real S3

The integration tests in storagex/test/integration_test.go use mock storage by default for fast, isolated unit tests. To run tests against real S3/MinIO:

# Set environment variable to use real S3
export STORAGEX_USE_REAL_S3=true

# Configure S3 credentials
export STRATUM_STORAGE_BUCKET=test-bucket
export STRATUM_STORAGE_REGION=us-east-1
export STRATUM_STORAGE_ACCESS_KEY=your-access-key
export STRATUM_STORAGE_SECRET_KEY=your-secret-key

# For MinIO
export STRATUM_STORAGE_ENDPOINT=http://localhost:9000
export STRATUM_STORAGE_USE_PATH_STYLE=true
export STRATUM_STORAGE_DISABLE_SSL=true

# Run tests
go test ./test/...

Design Philosophy

Following GoStratum's testing philosophy:

  1. Test behavior, not internals - Mock implements the public Storage interface
  2. Fakes at boundaries - Mock replaces network/S3 dependency
  3. No external dependencies - Tests run fast without Docker/MinIO
  4. Concurrency-safe - Uses proper locking for thread safety

Limitations

The mock storage is intentionally simplified:

  • PutFile: Not implemented (returns error)
  • ETag generation: Simple checksum-based, not MD5 like real S3
  • Multipart uploads: Simplified to regular Put operations
  • Presigned URLs: Returns mock URLs that can't actually be used
  • Storage in memory: All data is lost when test completes

For comprehensive S3-specific behavior testing (e.g., actual presigned URL uploads, large file multipart uploads), use the real S3/MinIO tests by setting STORAGEX_USE_REAL_S3=true.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var TestModule = fx.Module("storagex-test",
	fx.Provide(
		NewTestConfig,
		NewTestKeyBuilder,
	),
)

TestModule provides a module for testing with mock/test implementations. This module provides a test configuration and key builder suitable for unit tests without requiring external configuration.

Example usage: 

import "github.com/gostratum/storagex/internal/testutil"

func TestMyApp(t *testing.T) {
    app := fx.New(
        testutil.TestModule,
        fx.Invoke(func(cfg *storagex.Config) {
            // Use test config
        }),
    )
    // ...
}

Functions

func NewTestConfig added in v0.2.0 

func NewTestConfig() *storagex.Config

NewTestConfig creates a test configuration suitable for unit tests. The configuration points to a local MinIO instance with default credentials.

func NewTestKeyBuilder added in v0.2.0 

func NewTestKeyBuilder() storagex.KeyBuilder

NewTestKeyBuilder creates a test key builder with a "test" prefix.

Types

type MockStorage 

type MockStorage struct {
	// contains filtered or unexported fields
}

MockStorage is a thread-safe in-memory implementation of storagex.Storage for testing

func NewMockStorage 

func NewMockStorage() *MockStorage

NewMockStorage creates a new in-memory mock storage

func (*MockStorage) AbortMultipart 

func (m *MockStorage) AbortMultipart(ctx context.Context, key, uploadID string) error

AbortMultipart cancels a multipart upload and cleans up parts

func (*MockStorage) CompleteMultipart 

func (m *MockStorage) CompleteMultipart(ctx context.Context, key, uploadID string, etags []string) (storagex.Stat, error)

CompleteMultipart finalizes a multipart upload

func (*MockStorage) CreateMultipart 

func (m *MockStorage) CreateMultipart(ctx context.Context, key string, putOpts *storagex.PutOptions) (uploadID string, err error)

CreateMultipart initiates a multipart upload session

func (*MockStorage) Delete 

func (m *MockStorage) Delete(ctx context.Context, key string) error

Delete removes a single object

func (*MockStorage) DeleteBatch 

func (m *MockStorage) DeleteBatch(ctx context.Context, keys []string) ([]string, error)

DeleteBatch removes multiple objects, returns keys that failed to delete

func (*MockStorage) Get 

func (m *MockStorage) Get(ctx context.Context, key string) (storagex.ReaderAtCloser, storagex.Stat, error)

Get retrieves an object as a streaming reader with metadata

func (*MockStorage) Head 

func (m *MockStorage) Head(ctx context.Context, key string) (storagex.Stat, error)

Head retrieves object metadata without the payload

func (*MockStorage) List 

func (m *MockStorage) List(ctx context.Context, opts storagex.ListOptions) (storagex.ListPage, error)

List retrieves objects with optional filtering and pagination

func (*MockStorage) MultipartUpload 

func (m *MockStorage) MultipartUpload(ctx context.Context, key string, src io.Reader, cfg *storagex.MultipartConfig, putOpts *storagex.PutOptions) (storagex.Stat, error)

MultipartUpload uploads large objects using multipart upload

func (*MockStorage) PresignGet 

func (m *MockStorage) PresignGet(ctx context.Context, key string, opts *storagex.PresignOptions) (url string, err error)

PresignGet generates a presigned URL for downloading an object

func (*MockStorage) PresignPut 

func (m *MockStorage) PresignPut(ctx context.Context, key string, opts *storagex.PresignOptions) (url string, err error)

PresignPut generates a presigned URL for uploading an object

func (*MockStorage) Put 

func (m *MockStorage) Put(ctx context.Context, key string, r io.Reader, opts *storagex.PutOptions) (storagex.Stat, error)

Put stores an object from an io.Reader

func (*MockStorage) PutBytes 

func (m *MockStorage) PutBytes(ctx context.Context, key string, data []byte, opts *storagex.PutOptions) (storagex.Stat, error)

PutBytes stores an object from a byte slice

func (*MockStorage) PutFile 

func (m *MockStorage) PutFile(ctx context.Context, key string, path string, opts *storagex.PutOptions) (storagex.Stat, error)

PutFile stores an object from a local file path

func (*MockStorage) UploadPart 

func (m *MockStorage) UploadPart(ctx context.Context, key, uploadID string, partNumber int32, part io.Reader, size int64) (etag string, err error)

UploadPart uploads a single part in a multipart upload

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.