testing

README

Cloud Provider Testing Interface

Reference Idea

This package provides a cloud-agnostic testing interface for Kubernetes cloud providers, similar to how cloud.go provides a common interface for cloud provider implementations. The testing interface allows cloud provider developers to write tests that are independent of the specific cloud provider implementation while ensuring consistent behavior across all cloud providers.

Overview

The testing interface is designed to:

1. Abstract Cloud Provider Details: Provide a common interface that abstracts away the specific details of each cloud provider
2. Ensure Consistent Testing: Enable cloud providers to test the same functionality in a consistent way
3. Simplify Test Development: Reduce the complexity of writing tests for cloud providers
4. Support Multiple Test Types: Support unit tests, integration tests, and end-to-end tests
5. Provide Test Infrastructure: Offer common test utilities, mock implementations, and test runners

Architecture

Core Components

1. TestInterface

The main interface that cloud provider test implementations must satisfy:

type TestInterface interface {
    SetupTestEnvironment(config *TestConfig) error
    TeardownTestEnvironment() error
    GetCloudProvider() cloudprovider.Interface
    CreateTestNode(ctx context.Context, nodeConfig *TestNodeConfig) (*v1.Node, error)
    DeleteTestNode(ctx context.Context, nodeName string) error
    CreateTestService(ctx context.Context, serviceConfig *TestServiceConfig) (*v1.Service, error)
    DeleteTestService(ctx context.Context, serviceName string) error
    CreateTestRoute(ctx context.Context, routeConfig *TestRouteConfig) (*cloudprovider.Route, error)
    DeleteTestRoute(ctx context.Context, routeName string) error
    WaitForCondition(ctx context.Context, condition TestCondition) error
    GetTestResults() *TestResults
    ResetTestState() error
}

2. TestRunner

Responsible for executing test suites and managing test results:

type TestRunner struct {
    TestInterface TestInterface
    TestSuites    []TestSuite
    Results       []TestResult
}

3. TestSuite

Defines a collection of related tests:

type TestSuite struct {
    Name         string
    Description  string
    Tests        []Test
    Setup        func(TestInterface) error
    Teardown     func(TestInterface) error
    Dependencies []string
}

4. Test

Defines a single test case:

type Test struct {
    Name         string
    Description  string
    Run          func(TestInterface) error
    Skip         bool
    SkipReason   string
    Timeout      time.Duration
    Dependencies []string
    Cleanup      func(TestInterface) error
}

Logic and Design Principles

1. Cloud-Agnostic Design

The interface is designed to be cloud-agnostic, meaning:

• Tests can be written without knowledge of specific cloud provider APIs
• Test logic focuses on Kubernetes resource behavior rather than cloud-specific implementation details
• Cloud provider implementations can be swapped without changing test code

2. Resource Lifecycle Management

The interface provides methods for creating and managing test resources:

• Nodes: Test node registration, addressing, and metadata
• Services: Test load balancer creation and management
• Routes: Test route creation and management

3. Test State Management

The interface includes comprehensive test state management:

• Setup/Teardown: Proper initialization and cleanup of test environments
• Resource Tracking: Automatic tracking of created resources for cleanup
• Result Collection: Structured collection of test results, metrics, and logs
• State Reset: Ability to reset test state between test runs

4. Condition-Based Testing

The interface supports condition-based testing for asynchronous operations:

• WaitForCondition: Wait for specific conditions to be met
• Custom Check Functions: Define custom logic for condition checking
• Timeout Management: Configurable timeouts for condition checking

5. Mock and Fake Implementations

The package provides mock and fake implementations for testing:

• BaseTestImplementation: Base implementation that can be extended
• FakeTestImplementation: Implementation using the fake cloud provider
• MockClientBuilder: Mock implementation of ControllerClientBuilder

Usage Examples

Basic Usage

// Create a test implementation
fakeImpl := NewFakeTestImplementation()

// Create a test runner
runner := NewTestRunner(fakeImpl)

// Add test suites
runner.AddTestSuite(ExampleTestSuite())

// Run tests
ctx := context.Background()
err := runner.RunTests(ctx)
if err != nil {
    log.Fatalf("Test execution failed: %v", err)
}

// Get results
results := runner.GetResults()
summary := runner.GetSummary()

Creating a Test Suite

func MyTestSuite() TestSuite {
    return TestSuite{
        Name:        "My Cloud Provider Tests",
        Description: "Tests for my cloud provider implementation",
        Setup: func(ti TestInterface) error {
            config := &TestConfig{
                ProviderName:        "my-provider",
                ClusterName:         "test-cluster",
                Region:              "us-west-1",
                Zone:                "us-west-1a",
                TestTimeout:         5 * time.Minute,
                CleanupResources:    true,
                MockExternalServices: true,
            }
            return ti.SetupTestEnvironment(config)
        },
        Teardown: func(ti TestInterface) error {
            return ti.TeardownTestEnvironment()
        },
        Tests: []Test{
            {
                Name:        "Test Load Balancer",
                Description: "Tests load balancer functionality",
                Run:         testLoadBalancer,
                Timeout:     2 * time.Minute,
            },
        },
    }
}

Writing Individual Tests

func testLoadBalancer(ti TestInterface) error {
    ctx := context.Background()

    // Create test resources
    nodeConfig := &TestNodeConfig{
        Name:         "test-node",
        ProviderID:   "my-provider://test-node",
        InstanceType: "t3.medium",
        Zone:         "us-west-1a",
        Region:       "us-west-1",
    }

    node, err := ti.CreateTestNode(ctx, nodeConfig)
    if err != nil {
        return fmt.Errorf("failed to create test node: %w", err)
    }

    // Test cloud provider functionality
    cloud := ti.GetCloudProvider()
    loadBalancer, supported := cloud.LoadBalancer()
    if !supported {
        return fmt.Errorf("load balancer not supported")
    }

    // Perform tests...
    return nil
}

Testing Steps and Logic

Step 1: Environment Setup

1. Initialize Test Configuration: Set up provider-specific configuration
2. Create Test Environment: Initialize cloud provider and test infrastructure
3. Set Up Mock Services: Configure mock external services if needed
4. Initialize Cloud Provider: Call the cloud provider's Initialize method

Step 2: Test Execution

1. Create Test Resources: Use the interface to create nodes, services, and routes
2. Test Cloud Provider Methods: Call cloud provider methods and verify results
3. Verify Resource State: Check that resources are in the expected state
4. Test Error Conditions: Verify proper error handling

Step 3: Resource Cleanup

1. Track Created Resources: Automatically track all created resources
2. Clean Up Resources: Delete resources in the correct order
3. Verify Cleanup: Ensure all resources are properly cleaned up
4. Reset Test State: Reset the test environment for the next test

Step 4: Result Collection

1. Collect Test Results: Gather test results, metrics, and logs
2. Generate Test Summary: Create a summary of test execution
3. Report Failures: Provide detailed information about test failures
4. Store Test Artifacts: Save test artifacts for analysis

Integration with Existing Cloud Providers

For New Cloud Providers

1. Implement TestInterface: Create a test implementation for your cloud provider
2. Extend BaseTestImplementation: Use the base implementation as a starting point
3. Add Provider-Specific Tests: Create tests specific to your cloud provider's features
4. Register Test Suites: Register your test suites with the test runner

For Existing Cloud Providers

1. Create Test Adapter: Create an adapter that implements TestInterface
2. Map to Existing Tests: Map existing tests to the new interface
3. Gradually Migrate: Gradually migrate existing tests to use the new interface
4. Maintain Compatibility: Ensure existing test infrastructure continues to work

Best Practices

1. Test Organization

• Group Related Tests: Organize tests into logical test suites
• Use Descriptive Names: Use clear, descriptive names for tests and test suites
• Document Test Purpose: Provide clear descriptions of what each test validates

2. Resource Management

• Always Clean Up: Ensure all test resources are properly cleaned up
• Use Resource Tracking: Leverage the built-in resource tracking for cleanup
• Handle Cleanup Failures: Handle cleanup failures gracefully

3. Error Handling

• Test Error Conditions: Include tests for error conditions and edge cases
• Provide Clear Error Messages: Use descriptive error messages for test failures
• Handle Timeouts: Use appropriate timeouts for asynchronous operations

4. Test Isolation

• Independent Tests: Ensure tests are independent and can run in any order
• Reset State: Reset test state between tests
• Avoid Shared State: Avoid sharing state between tests

5. Performance Considerations

• Use Appropriate Timeouts: Set reasonable timeouts for test operations
• Limit Resource Creation: Avoid creating unnecessary resources
• Clean Up Promptly: Clean up resources as soon as they're no longer needed

Extending the Interface

Adding New Resource Types

1. Define Configuration: Create a configuration struct for the new resource type
2. Add Interface Methods: Add create and delete methods to TestInterface
3. Update Base Implementation: Update BaseTestImplementation with default behavior
4. Add Test Examples: Create example tests for the new resource type

Adding New Test Utilities

1. Create Utility Functions: Create utility functions for common test operations
2. Add to Interface: Add utility methods to TestInterface if needed
3. Document Usage: Document how to use the new utilities
4. Add Examples: Provide examples of using the new utilities

Integration Guide for Cloud Provider Repositories

This section provides step-by-step instructions for integrating the cloud provider testing interface into your cloud provider repository.

Step 1: Add the Dependency

Add the testing interface as a dependency to your cloud provider's go.mod file:

go get github.com/miyadav/cloud-provider-testing-interface

Or manually add to your go.mod:

require (
    github.com/miyadav/cloud-provider-testing-interface v0.1.0
    // ... other dependencies
)

Step 2: Create a Test Implementation

Create a test implementation that extends the base implementation for your specific cloud provider:

// pkg/testing/cloud_provider_test_impl.go
package testing

import (
    "context"
    "fmt"
    
    "k8s.io/cloudprovider"
    testing "github.com/miyadav/cloud-provider-testing-interface"
)

// CloudProviderTestImplementation implements the testing interface for your cloud provider
type CloudProviderTestImplementation struct {
    *testing.BaseTestImplementation
    CloudProvider *cloudprovider.CloudProvider
    TestConfig    *testing.TestConfig
}

// NewCloudProviderTestImplementation creates a new test implementation
func NewCloudProviderTestImplementation(cloudProvider *cloudprovider.CloudProvider) *CloudProviderTestImplementation {
    baseImpl := testing.NewBaseTestImplementation(cloudProvider)
    return &CloudProviderTestImplementation{
        BaseTestImplementation: baseImpl,
        CloudProvider:          cloudProvider,
    }
}

// SetupTestEnvironment sets up the test environment for your cloud provider
func (c *CloudProviderTestImplementation) SetupTestEnvironment(config *testing.TestConfig) error {
    // Call the base implementation first
    if err := c.BaseTestImplementation.SetupTestEnvironment(config); err != nil {
        return err
    }
    
    // Add cloud provider-specific setup
    c.TestConfig = config
    
    // Initialize your cloud provider with test configuration
    if err := c.CloudProvider.Initialize(config.ClientBuilder, make(chan struct{})); err != nil {
        return fmt.Errorf("failed to initialize cloud provider: %w", err)
    }
    
    // Set up any cloud provider-specific test resources
    if err := c.setupCloudProviderResources(); err != nil {
        return fmt.Errorf("failed to setup cloud provider resources: %w", err)
    }
    
    return nil
}

// TeardownTestEnvironment cleans up the test environment
func (c *CloudProviderTestImplementation) TeardownTestEnvironment() error {
    // Clean up cloud provider-specific resources
    if err := c.cleanupCloudProviderResources(); err != nil {
        return fmt.Errorf("failed to cleanup cloud provider resources: %w", err)
    }
    
    // Call the base implementation
    return c.BaseTestImplementation.TeardownTestEnvironment()
}

// setupCloudProviderResources sets up cloud provider-specific test resources
func (c *CloudProviderTestImplementation) setupCloudProviderResources() error {
    // Implement cloud provider-specific resource setup
    // For example, create test VPCs, subnets, security groups, etc.
    return nil
}

// cleanupCloudProviderResources cleans up cloud provider-specific test resources
func (c *CloudProviderTestImplementation) cleanupCloudProviderResources() error {
    // Implement cloud provider-specific resource cleanup
    return nil
}

Step 3: Create Test Suites

Create test suites that test your cloud provider's functionality:

// pkg/testing/test_suites.go
package testing

import (
    "context"
    "fmt"
    "time"
    
    v1 "k8s.io/api/core/v1"
    "k8s.io/apimachinery/pkg/util/intstr"
    testing "github.com/miyadav/cloud-provider-testing-interface"
)

// CreateLoadBalancerTestSuite creates a test suite for load balancer functionality
func CreateLoadBalancerTestSuite() testing.TestSuite {
    return testing.TestSuite{
        Name:        "Load Balancer Tests",
        Description: "Tests load balancer creation, update, and deletion",
        Setup: func(ti testing.TestInterface) error {
            config := &testing.TestConfig{
                ProviderName:         "your-cloud-provider",
                ClusterName:          "test-cluster",
                Region:               "us-west-1",
                Zone:                 "us-west-1a",
                TestTimeout:          5 * time.Minute,
                CleanupResources:     true,
                MockExternalServices: false, // Use real cloud services for integration tests
            }
            return ti.SetupTestEnvironment(config)
        },
        Teardown: func(ti testing.TestInterface) error {
            return ti.TeardownTestEnvironment()
        },
        Tests: []testing.Test{
            {
                Name:        "Test Load Balancer Creation",
                Description: "Tests that a load balancer can be created successfully",
                Run:         testLoadBalancerCreation,
                Timeout:     2 * time.Minute,
            },
            {
                Name:        "Test Load Balancer Update",
                Description: "Tests that a load balancer can be updated",
                Run:         testLoadBalancerUpdate,
                Timeout:     2 * time.Minute,
            },
            {
                Name:        "Test Load Balancer Deletion",
                Description: "Tests that a load balancer can be deleted",
                Run:         testLoadBalancerDeletion,
                Timeout:     2 * time.Minute,
            },
        },
    }
}

// testLoadBalancerCreation tests load balancer creation
func testLoadBalancerCreation(ti testing.TestInterface) error {
    ctx := context.Background()
    
    // Create a test node
    nodeConfig := &testing.TestNodeConfig{
        Name:         "test-node-1",
        ProviderID:   "your-provider://test-node-1",
        InstanceType: "t3.medium",
        Zone:         "us-west-1a",
        Region:       "us-west-1",
        Addresses: []v1.NodeAddress{
            {Type: v1.NodeInternalIP, Address: "10.0.0.1"},
            {Type: v1.NodeExternalIP, Address: "192.168.1.1"},
        },
    }
    
    node, err := ti.CreateTestNode(ctx, nodeConfig)
    if err != nil {
        return fmt.Errorf("failed to create test node: %w", err)
    }
    
    // Create a test service
    serviceConfig := &testing.TestServiceConfig{
        Name:      "test-service",
        Namespace: "default",
        Type:      v1.ServiceTypeLoadBalancer,
        Ports: []v1.ServicePort{
            {Port: 80, TargetPort: intstr.FromInt(8080), Protocol: v1.ProtocolTCP},
        },
        ExternalTrafficPolicy: v1.ServiceExternalTrafficPolicyCluster,
    }
    
    service, err := ti.CreateTestService(ctx, serviceConfig)
    if err != nil {
        return fmt.Errorf("failed to create test service: %w", err)
    }
    
    // Test load balancer functionality
    cloud := ti.GetCloudProvider()
    loadBalancer, supported := cloud.LoadBalancer()
    if !supported {
        return fmt.Errorf("load balancer not supported by cloud provider")
    }
    
    // Test EnsureLoadBalancer
    nodes := []*v1.Node{node}
    status, err := loadBalancer.EnsureLoadBalancer(ctx, "test-cluster", service, nodes)
    if err != nil {
        return fmt.Errorf("failed to ensure load balancer: %w", err)
    }
    
    // Verify load balancer status
    if status == nil || len(status.Ingress) == 0 {
        return fmt.Errorf("load balancer status is empty")
    }
    
    ti.GetTestResults().AddLog("Load balancer creation test completed successfully")
    return nil
}

// testLoadBalancerUpdate tests load balancer update
func testLoadBalancerUpdate(ti testing.TestInterface) error {
    // Implement load balancer update test
    return nil
}

// testLoadBalancerDeletion tests load balancer deletion
func testLoadBalancerDeletion(ti testing.TestInterface) error {
    // Implement load balancer deletion test
    return nil
}

Step 4: Create Integration Tests

Create integration tests that use your test implementation:

// pkg/testing/integration_test.go
package testing

import (
    "context"
    "testing"
    
    "your-cloud-provider/pkg/cloudprovider"
    testing "github.com/miyadav/cloud-provider-testing-interface"
)

// TestLoadBalancerIntegration tests load balancer integration
func TestLoadBalancerIntegration(t *testing.T) {
    // Create your cloud provider instance
    cloudProvider := &cloudprovider.CloudProvider{}
    
    // Create test implementation
    testImpl := NewCloudProviderTestImplementation(cloudProvider)
    
    // Create test runner
    runner := testing.NewTestRunner(testImpl)
    
    // Add test suites
    runner.AddTestSuite(CreateLoadBalancerTestSuite())
    
    // Run tests
    ctx := context.Background()
    err := runner.RunTests(ctx)
    if err != nil {
        t.Fatalf("Test execution failed: %v", err)
    }
    
    // Verify results
    summary := runner.GetSummary()
    
    if summary.TotalTests == 0 {
        t.Error("No tests were executed")
    }
    
    if summary.FailedTests > 0 {
        t.Errorf("Some tests failed: %d failed out of %d total", summary.FailedTests, summary.TotalTests)
    }
    
    // Print results for debugging
    t.Logf("Test Summary: %d total, %d passed, %d failed, %d skipped",
        summary.TotalTests, summary.PassedTests, summary.FailedTests, summary.SkippedTests)
}

Step 5: Create a Test Runner Script

Create a script to run your tests:

// cmd/test-runner/main.go
package main

import (
    "context"
    "flag"
    "fmt"
    "log"
    "os"
    
    "your-cloud-provider/pkg/cloudprovider"
    "your-cloud-provider/pkg/testing"
    testing "github.com/miyadav/cloud-provider-testing-interface"
)

func main() {
    var (
        testSuite = flag.String("suite", "all", "Test suite to run (all, loadbalancer, nodes, routes)")
        verbose   = flag.Bool("verbose", false, "Enable verbose output")
    )
    flag.Parse()
    
    // Create cloud provider instance
    cloudProvider := &cloudprovider.CloudProvider{}
    
    // Create test implementation
    testImpl := testing.NewCloudProviderTestImplementation(cloudProvider)
    
    // Create test runner
    runner := testing.NewTestRunner(testImpl)
    
    // Add test suites based on flag
    switch *testSuite {
    case "all":
        runner.AddTestSuite(testing.CreateLoadBalancerTestSuite())
        // Add other test suites
    case "loadbalancer":
        runner.AddTestSuite(testing.CreateLoadBalancerTestSuite())
    default:
        log.Fatalf("Unknown test suite: %s", *testSuite)
    }
    
    // Run tests
    ctx := context.Background()
    err := runner.RunTests(ctx)
    if err != nil {
        log.Fatalf("Test execution failed: %v", err)
    }
    
    // Get results
    results := runner.GetResults()
    summary := runner.GetSummary()
    
    // Print summary
    fmt.Printf("Test Summary:\n")
    fmt.Printf("  Total Tests: %d\n", summary.TotalTests)
    fmt.Printf("  Passed: %d\n", summary.PassedTests)
    fmt.Printf("  Failed: %d\n", summary.FailedTests)
    fmt.Printf("  Skipped: %d\n", summary.SkippedTests)
    fmt.Printf("  Duration: %v\n", summary.TotalDuration)
    
    // Print detailed results if verbose
    if *verbose {
        for _, result := range results {
            status := "PASSED"
            if !result.Success {
                status = "FAILED"
            }
            if result.Test.Skip {
                status = "SKIPPED"
            }
            fmt.Printf("  %s: %s (%v)\n", status, result.Test.Name, result.Duration)
        }
    }
    
    // Exit with error code if any tests failed
    if summary.FailedTests > 0 {
        os.Exit(1)
    }
}

Step 6: Add to Your Build Pipeline

Add the test runner to your CI/CD pipeline:

# .github/workflows/test.yml
name: Cloud Provider Tests

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  test:
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v3
    
    - name: Set up Go
      uses: actions/setup-go@v4
      with:
        go-version: '1.24'
    
    - name: Install dependencies
      run: go mod download
    
    - name: Run unit tests
      run: go test -v ./pkg/...
    
    - name: Run integration tests
      run: go test -v ./pkg/testing/...
      env:
        CLOUD_PROVIDER_CONFIG: ${{ secrets.CLOUD_PROVIDER_CONFIG }}
    
    - name: Run test runner
      run: go run cmd/test-runner/main.go -suite=all -verbose
      env:
        CLOUD_PROVIDER_CONFIG: ${{ secrets.CLOUD_PROVIDER_CONFIG }}

Step 7: Configuration Management

Create configuration files for different test environments:

# config/test-config.yaml
test:
  provider:
    name: "your-cloud-provider"
    region: "us-west-1"
    zone: "us-west-1a"
  
  cluster:
    name: "test-cluster"
    version: "1.24.0"
  
  resources:
    cleanup: true
    timeout: "5m"
  
  external_services:
    mock: false  # Use real cloud services for integration tests

Step 8: Documentation

Add documentation to your cloud provider repository:

# Testing

This cloud provider uses the [cloud-provider-testing-interface](https://github.com/miyadav/cloud-provider-testing-interface) for comprehensive testing.

## Running Tests

### Unit Tests
```bash
go test -v ./pkg/...

Integration Tests

go test -v ./pkg/testing/...

Full Test Suite

go run cmd/test-runner/main.go -suite=all -verbose

Test Configuration

Tests can be configured using environment variables or configuration files. See config/test-config.yaml for available options.

Adding New Tests

1. Create a new test function in pkg/testing/test_suites.go
2. Add the test to an appropriate test suite
3. Update the test runner to include the new suite
4. Add integration tests in pkg/testing/integration_test.go

## Best Practices for Cloud Provider Integration

### 1. Test Environment Isolation
- Use separate test accounts/projects for each test run
- Implement proper cleanup to avoid resource leaks
- Use unique resource names to avoid conflicts

### 2. Configuration Management
- Use environment variables for sensitive configuration
- Provide default configurations for local development
- Validate configuration before running tests

### 3. Error Handling
- Implement proper error handling and logging
- Provide clear error messages for debugging
- Handle cloud provider API errors gracefully

### 4. Resource Management
- Track all created resources for cleanup
- Implement timeouts for resource operations
- Handle resource creation failures

### 5. Test Organization
- Group related tests into logical test suites
- Use descriptive test names and descriptions
- Implement test dependencies when needed

## Conclusion

The cloud provider testing interface provides a comprehensive, cloud-agnostic way to test cloud provider implementations. By abstracting away cloud-specific details and providing common test infrastructure, it enables cloud provider developers to focus on testing the behavior and functionality that should be consistent across all cloud providers.

The interface is designed to be extensible and can be adapted to support new resource types, test utilities, and testing patterns as needed. By following the design principles and best practices outlined in this document, cloud provider developers can create robust, maintainable tests that ensure their implementations work correctly and consistently.

## Getting Help

If you encounter issues while integrating the testing interface into your cloud provider:

1. Check the [examples](examples_test.go) for usage patterns
2. Review the [test implementations](implementation_test.go) for best practices
3. Open an issue in the [repository](https://github.com/miyadav/cloud-provider-testing-interface/issues)
4. Join the Kubernetes cloud provider community discussions

Documentation

Index

• type BaseTestImplementation
  • func NewBaseTestImplementation(cloudProvider cloudprovider.Interface) *BaseTestImplementation
  • func (b *BaseTestImplementation) CreateTestNode(ctx context.Context, nodeConfig *TestNodeConfig) (*v1.Node, error)
  • func (b *BaseTestImplementation) CreateTestRoute(ctx context.Context, routeConfig *TestRouteConfig) (*cloudprovider.Route, error)
  • func (b *BaseTestImplementation) CreateTestService(ctx context.Context, serviceConfig *TestServiceConfig) (*v1.Service, error)
  • func (b *BaseTestImplementation) DeleteTestNode(ctx context.Context, nodeName string) error
  • func (b *BaseTestImplementation) DeleteTestRoute(ctx context.Context, routeName string) error
  • func (b *BaseTestImplementation) DeleteTestService(ctx context.Context, serviceName string) error
  • func (b *BaseTestImplementation) GetCloudProvider() cloudprovider.Interface
  • func (b *BaseTestImplementation) GetTestResults() *TestResults
  • func (b *BaseTestImplementation) ResetTestState() error
  • func (b *BaseTestImplementation) SetupTestEnvironment(config *TestConfig) error
  • func (b *BaseTestImplementation) TeardownTestEnvironment() error
  • func (b *BaseTestImplementation) WaitForCondition(ctx context.Context, condition TestCondition) error
• type FakeTestImplementation
  • func NewFakeTestImplementation() *FakeTestImplementation
  • func (f *FakeTestImplementation) GetFakeCloud() *fakecloud.Cloud
• type MockClientBuilder
  • func (m *MockClientBuilder) Client(name string) (clientset.Interface, error)
  • func (m *MockClientBuilder) ClientOrDie(name string) clientset.Interface
  • func (m *MockClientBuilder) Config(name string) (*rest.Config, error)
  • func (m *MockClientBuilder) ConfigOrDie(name string) *rest.Config
• type Test
• type TestCondition
• type TestConfig
• type TestInterface
• type TestNodeConfig
• type TestResult
• type TestResults
  • func (tr *TestResults) AddLog(log string)
  • func (tr *TestResults) IncrementResourceCount(resourceType string)
  • func (tr *TestResults) SetMetric(key string, value interface{})
• type TestRouteConfig
• type TestRunner
  • func NewTestRunner(testInterface TestInterface) *TestRunner
  • func (tr *TestRunner) AddTestSuite(suite TestSuite)
  • func (tr *TestRunner) GetResults() []TestResult
  • func (tr *TestRunner) GetSummary() TestSummary
  • func (tr *TestRunner) RunTests(ctx context.Context) error
• type TestServiceConfig
• type TestSuite
• type TestSummary

Examples

• TestRunner

Types

type BaseTestImplementation

type BaseTestImplementation struct {
	// CloudProvider is the cloud provider instance being tested.
	CloudProvider cloudprovider.Interface

	// ClientBuilder is the client builder for creating Kubernetes clients.
	ClientBuilder cloudprovider.ControllerClientBuilder

	// InformerFactory is the informer factory for creating informers.
	InformerFactory informers.SharedInformerFactory

	// TestConfig holds the current test configuration.
	TestConfig *TestConfig

	// TestResults holds the current test results.
	TestResults *TestResults

	// CreatedResources tracks resources created during tests for cleanup.
	CreatedResources map[string][]string
	// contains filtered or unexported fields
}

BaseTestImplementation provides a base implementation of the TestInterface that can be extended by specific cloud provider test implementations.

func NewBaseTestImplementation

func NewBaseTestImplementation(cloudProvider cloudprovider.Interface) *BaseTestImplementation

NewBaseTestImplementation creates a new base test implementation.

func (*BaseTestImplementation) CreateTestNode

func (b *BaseTestImplementation) CreateTestNode(ctx context.Context, nodeConfig *TestNodeConfig) (*v1.Node, error)

CreateTestNode creates a test node.

func (*BaseTestImplementation) CreateTestRoute

func (b *BaseTestImplementation) CreateTestRoute(ctx context.Context, routeConfig *TestRouteConfig) (*cloudprovider.Route, error)

CreateTestRoute creates a test route.

func (*BaseTestImplementation) CreateTestService

func (b *BaseTestImplementation) CreateTestService(ctx context.Context, serviceConfig *TestServiceConfig) (*v1.Service, error)

CreateTestService creates a test service.

func (*BaseTestImplementation) DeleteTestNode

func (b *BaseTestImplementation) DeleteTestNode(ctx context.Context, nodeName string) error

DeleteTestNode deletes a test node.

func (*BaseTestImplementation) DeleteTestRoute

func (b *BaseTestImplementation) DeleteTestRoute(ctx context.Context, routeName string) error

DeleteTestRoute deletes a test route.

func (*BaseTestImplementation) DeleteTestService

func (b *BaseTestImplementation) DeleteTestService(ctx context.Context, serviceName string) error

DeleteTestService deletes a test service.

func (*BaseTestImplementation) GetCloudProvider

func (b *BaseTestImplementation) GetCloudProvider() cloudprovider.Interface

GetCloudProvider returns the cloud provider instance.

func (*BaseTestImplementation) GetTestResults

func (b *BaseTestImplementation) GetTestResults() *TestResults

GetTestResults returns the test results.

func (*BaseTestImplementation) ResetTestState

func (b *BaseTestImplementation) ResetTestState() error

ResetTestState resets the test state.

func (*BaseTestImplementation) SetupTestEnvironment

func (b *BaseTestImplementation) SetupTestEnvironment(config *TestConfig) error

SetupTestEnvironment initializes the test environment.

func (*BaseTestImplementation) TeardownTestEnvironment

func (b *BaseTestImplementation) TeardownTestEnvironment() error

TeardownTestEnvironment cleans up the test environment.

func (*BaseTestImplementation) WaitForCondition

func (b *BaseTestImplementation) WaitForCondition(ctx context.Context, condition TestCondition) error

WaitForCondition waits for a specific condition to be met.

type FakeTestImplementation

type FakeTestImplementation struct {
	*BaseTestImplementation
	FakeCloud *fakecloud.Cloud
}

FakeTestImplementation provides a test implementation using the fake cloud provider. This is useful for testing the test framework itself or for cloud providers that want to use the fake provider for testing.

func NewFakeTestImplementation

func NewFakeTestImplementation() *FakeTestImplementation

NewFakeTestImplementation creates a new fake test implementation.

func (*FakeTestImplementation) GetFakeCloud

func (f *FakeTestImplementation) GetFakeCloud() *fakecloud.Cloud

GetFakeCloud returns the fake cloud provider instance.

type MockClientBuilder

type MockClientBuilder struct {
	ConfigFunc      func(name string) (*rest.Config, error)
	ClientFunc      func(name string) (clientset.Interface, error)
	ConfigOrDieFunc func(name string) *rest.Config
	ClientOrDieFunc func(name string) clientset.Interface
}

MockClientBuilder provides a mock implementation of ControllerClientBuilder for testing.

func (*MockClientBuilder) Client

func (m *MockClientBuilder) Client(name string) (clientset.Interface, error)

Client implements ControllerClientBuilder.Client.

func (*MockClientBuilder) ClientOrDie

func (m *MockClientBuilder) ClientOrDie(name string) clientset.Interface

ClientOrDie implements ControllerClientBuilder.ClientOrDie.

func (*MockClientBuilder) Config

func (m *MockClientBuilder) Config(name string) (*rest.Config, error)

Config implements ControllerClientBuilder.Config.

func (*MockClientBuilder) ConfigOrDie

func (m *MockClientBuilder) ConfigOrDie(name string) *rest.Config

ConfigOrDie implements ControllerClientBuilder.ConfigOrDie.

type Test

type Test struct {
	// Name is the name of the test.
	Name string

	// Description is the description of the test.
	Description string

	// Run is the function that runs the test.
	Run func(TestInterface) error

	// Skip determines whether to skip this test.
	Skip bool

	// SkipReason is the reason for skipping the test.
	SkipReason string

	// Timeout is the timeout for the test.
	Timeout time.Duration

	// Dependencies are the dependencies required for the test.
	Dependencies []string

	// Cleanup is the cleanup function for the test.
	Cleanup func(TestInterface) error
}

Test defines a single test that can be run against a cloud provider.

type TestCondition

type TestCondition struct {
	// Type is the type of the condition.
	Type string

	// Status is the status of the condition.
	Status string

	// Reason is the reason for the condition.
	Reason string

	// Message is the message of the condition.
	Message string

	// Timeout is the timeout for the condition.
	Timeout time.Duration

	// CheckFunction is a custom function to check the condition.
	CheckFunction func() (bool, error)
}

TestCondition represents a condition that should be met during testing.

type TestConfig

type TestConfig struct {
	// ProviderName is the name of the cloud provider being tested.
	ProviderName string

	// ClusterName is the name of the test cluster.
	ClusterName string

	// Region is the region where the test resources should be created.
	Region string

	// Zone is the zone where the test resources should be created.
	Zone string

	// ClientBuilder is the client builder for creating Kubernetes clients.
	ClientBuilder cloudprovider.ControllerClientBuilder

	// InformerFactory is the informer factory for creating informers.
	InformerFactory informers.SharedInformerFactory

	// TestTimeout is the timeout for test operations.
	TestTimeout time.Duration

	// CleanupResources determines whether to clean up resources after tests.
	CleanupResources bool

	// MockExternalServices determines whether to use mock external services.
	MockExternalServices bool

	// TestData contains additional test-specific configuration.
	TestData map[string]interface{}
}

TestConfig holds the configuration for a test environment.

type TestInterface

type TestInterface interface {
	// SetupTestEnvironment initializes the test environment with the given configuration.
	// This should create any necessary test resources, mock services, or test data.
	SetupTestEnvironment(config *TestConfig) error

	// TeardownTestEnvironment cleans up the test environment and removes any test resources.
	TeardownTestEnvironment() error

	// GetCloudProvider returns the cloud provider instance to be tested.
	// This allows the test framework to access the actual cloud provider implementation.
	GetCloudProvider() cloudprovider.Interface

	// CreateTestNode creates a test node with the specified configuration.
	// The node should be created in a way that simulates a real node in the cloud provider.
	CreateTestNode(ctx context.Context, nodeConfig *TestNodeConfig) (*v1.Node, error)

	// DeleteTestNode deletes a test node.
	DeleteTestNode(ctx context.Context, nodeName string) error

	// CreateTestService creates a test service with the specified configuration.
	// The service should be created in a way that simulates a real service in the cloud provider.
	CreateTestService(ctx context.Context, serviceConfig *TestServiceConfig) (*v1.Service, error)

	// DeleteTestService deletes a test service.
	DeleteTestService(ctx context.Context, serviceName string) error

	// CreateTestRoute creates a test route with the specified configuration.
	CreateTestRoute(ctx context.Context, routeConfig *TestRouteConfig) (*cloudprovider.Route, error)

	// DeleteTestRoute deletes a test route.
	DeleteTestRoute(ctx context.Context, routeName string) error

	WaitForCondition(ctx context.Context, condition TestCondition) error

	// GetTestResults returns the results of the test execution.
	GetTestResults() *TestResults

	// ResetTestState resets the test state to a clean state.
	ResetTestState() error
}

TestInterface is an abstract, pluggable interface for testing cloud providers. This interface provides a cloud-agnostic way to test cloud provider implementations by abstracting away the specific cloud provider details and focusing on the behavior and functionality that should be consistent across all cloud providers.

type TestNodeConfig

type TestNodeConfig struct {
	// Name is the name of the test node.
	Name string

	// ProviderID is the provider ID of the node.
	ProviderID string

	// InstanceType is the instance type of the node.
	InstanceType string

	// Zone is the zone where the node should be created.
	Zone string

	// Region is the region where the node should be created.
	Region string

	// Addresses are the network addresses of the node.
	Addresses []v1.NodeAddress

	// Labels are the labels to be applied to the node.
	Labels map[string]string

	// Annotations are the annotations to be applied to the node.
	Annotations map[string]string

	// Conditions are the conditions of the node.
	Conditions []v1.NodeCondition
}

TestNodeConfig holds the configuration for creating a test node.

type TestResult

type TestResult struct {
	// Test is the test that was run.
	Test Test

	// Success indicates whether the test was successful.
	Success bool

	// Error is the error that occurred during the test.
	Error error

	// Duration is the duration of the test.
	Duration time.Duration

	// StartTime is the start time of the test.
	StartTime time.Time

	// EndTime is the end time of the test.
	EndTime time.Time
}

TestResult holds the result of a single test.

type TestResults

type TestResults struct {
	// Success indicates whether the test was successful.
	Success bool

	// Error is the error that occurred during the test.
	Error error

	// Duration is the duration of the test.
	Duration time.Duration

	// ResourceCounts contains counts of resources created during the test.
	ResourceCounts map[string]int

	// Metrics contains test-specific metrics.
	Metrics map[string]interface{}

	// Logs contains test logs.
	Logs []string
	// contains filtered or unexported fields
}

TestResults holds the results of a test execution.

func (*TestResults) AddLog

func (tr *TestResults) AddLog(log string)

AddLog adds a log entry to the test results.

func (*TestResults) IncrementResourceCount

func (tr *TestResults) IncrementResourceCount(resourceType string)

IncrementResourceCount increments the count for a resource type.

func (*TestResults) SetMetric

func (tr *TestResults) SetMetric(key string, value interface{})

SetMetric sets a metric in the test results.

type TestRouteConfig

type TestRouteConfig struct {
	// Name is the name of the test route.
	Name string

	// ClusterName is the name of the cluster.
	ClusterName string

	// TargetNode is the target node for the route.
	TargetNode types.NodeName

	// DestinationCIDR is the destination CIDR for the route.
	DestinationCIDR string

	// Blackhole determines whether this is a blackhole route.
	Blackhole bool
}

TestRouteConfig holds the configuration for creating a test route.

type TestRunner

type TestRunner struct {
	// TestInterface is the test interface to use.
	TestInterface TestInterface

	// TestSuites are the test suites to run.
	TestSuites []TestSuite

	// Results are the results of the test execution.
	Results []TestResult
	// contains filtered or unexported fields
}

TestRunner is responsible for running tests against cloud providers.

Example

ExampleTestRunner demonstrates how to use the TestRunner to run tests.

// Create a fake test implementation
fakeImpl := NewFakeTestImplementation()

// Create a test runner
runner := NewTestRunner(fakeImpl)

// Add test suites
runner.AddTestSuite(CreateExampleTestSuite())

// Run tests
ctx := context.Background()
err := runner.RunTests(ctx)
if err != nil {
	fmt.Printf("Test execution failed: %v\n", err)
	return
}

// Get results
results := runner.GetResults()
summary := runner.GetSummary()

fmt.Printf("Test Summary:\n")
fmt.Printf("  Total Tests: %d\n", summary.TotalTests)
fmt.Printf("  Passed: %d\n", summary.PassedTests)
fmt.Printf("  Failed: %d\n", summary.FailedTests)
fmt.Printf("  Skipped: %d\n", summary.SkippedTests)
fmt.Printf("  Duration: %v\n", summary.TotalDuration)

// Print detailed results
for _, result := range results {
	status := "PASSED"
	if !result.Success {
		status = "FAILED"
	}
	if result.Test.Skip {
		status = "SKIPPED"
	}
	fmt.Printf("  %s: %s (%v)\n", status, result.Test.Name, result.Duration)
}

func NewTestRunner

func NewTestRunner(testInterface TestInterface) *TestRunner

NewTestRunner creates a new test runner.

func (*TestRunner) AddTestSuite

func (tr *TestRunner) AddTestSuite(suite TestSuite)

AddTestSuite adds a test suite to the test runner.

func (*TestRunner) GetResults

func (tr *TestRunner) GetResults() []TestResult

GetResults returns the results of the test execution.

func (*TestRunner) GetSummary

func (tr *TestRunner) GetSummary() TestSummary

GetSummary returns a summary of the test results.

func (*TestRunner) RunTests

func (tr *TestRunner) RunTests(ctx context.Context) error

RunTests runs all the tests in the test runner.

type TestServiceConfig

type TestServiceConfig struct {
	// Name is the name of the test service.
	Name string

	// Namespace is the namespace of the service.
	Namespace string

	// Type is the type of the service.
	Type v1.ServiceType

	// Ports are the ports of the service.
	Ports []v1.ServicePort

	// LoadBalancerIP is the IP address for the load balancer.
	LoadBalancerIP string

	// ExternalTrafficPolicy is the external traffic policy.
	ExternalTrafficPolicy v1.ServiceExternalTrafficPolicy

	// InternalTrafficPolicy is the internal traffic policy.
	InternalTrafficPolicy *v1.ServiceInternalTrafficPolicy

	// Labels are the labels to be applied to the service.
	Labels map[string]string

	// Annotations are the annotations to be applied to the service.
	Annotations map[string]string
}

TestServiceConfig holds the configuration for creating a test service.

type TestSuite

type TestSuite struct {
	// Name is the name of the test suite.
	Name string

	// Description is the description of the test suite.
	Description string

	// Tests is the list of tests in the suite.
	Tests []Test

	// Setup is the setup function for the test suite.
	Setup func(TestInterface) error

	// Teardown is the teardown function for the test suite.
	Teardown func(TestInterface) error

	// Dependencies are the dependencies required for the test suite.
	Dependencies []string
}

TestSuite defines a collection of tests that can be run against a cloud provider.

type TestSummary

type TestSummary struct {
	// TotalTests is the total number of tests run.
	TotalTests int

	// PassedTests is the number of tests that passed.
	PassedTests int

	// FailedTests is the number of tests that failed.
	FailedTests int

	// SkippedTests is the number of tests that were skipped.
	SkippedTests int

	// TotalDuration is the total duration of all tests.
	TotalDuration time.Duration
}

TestSummary holds a summary of test results.