gcsconntest

package module
v1.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 3, 2026 License: MIT Imports: 15 Imported by: 0

README

GCS Connection Tester (gcsconntest)

Application Overview and Objectives

gcsconntest is a reusable Go library package and CLI utility designed to test and verify connectivity, authentication, and object accessibility to Google Cloud Storage (GCS) buckets.

Key Objectives
  • Dual-Interface Flexibility: Serves as a zero-dependency importable Go library (package gcsconntest) for health-checkers and background services, as well as a standalone CLI executable (cmd/gcsconntest).
  • Diagnostic Automation: Returns granular exit codes (0 to 5) for process orchestrators, Kubernetes probes, and CI/CD pipelines.
  • Keyless Cloud Support: Full support for GCP Application Default Credentials (ADC) on GKE, Cloud Run, Cloud Functions, and GCE VMs, as well as static Service Account key files and raw JSON bytes.
  • High Performance: Restricts query payloads via ProjectionNoACL and attribute selection, reducing network bandwidth usage by up to 80%.

1. Security Assessment

gcsconntest follows strict cloud security principles and defense-in-depth practices:

  • Encryption in Transit: All network communication with Google Cloud Storage and OAuth2 token endpoints is strictly encrypted using TLS 1.3 / HTTPS and gRPC (storage.googleapis.com:443).
  • Secret Management & Zero Disk Footprint:
    • Credential file paths pass through filepath.Clean to protect against path traversal attacks.
    • In-memory credential JSON bytes (CredJSON) allow loading keys directly from Secret Managers (HashiCorp Vault, AWS Secrets Manager, GCP Secret Manager) without writing secrets to disk.
    • SecretProtector Obfuscation Support: Optional integration with criticalsys.net/secretprotector enables AES-256-GCM authenticated decryption of stored service account credentials at rest using master keys (CLI flag, environment variable, or secure key file).
    • Memory Zeroing: Decrypted credential byte buffers are immediately zeroed out in memory using libsecsecrets.ZeroBuffer after GCP client initialization.
    • Raw credentials and private keys are never logged, printed, or serialized into JSON result structures.
  • Authentication Configuration: Supports static Service Account JSON keys (CredFile / CredJSON), AES-256-GCM encrypted credentials via SecretProtector (MasterKey / MasterKeyEnv / MasterKeyFile), and keyless Application Default Credentials (ADC) (AllowADC: true / -adc).
  • Least-Privilege RBAC / IAM: Requires only read-only permissions (storage.buckets.get and storage.objects.list). Recommended IAM roles include roles/storage.objectViewer or equivalent custom roles. Never requests write or delete permissions.
  • Current & Non-Vulnerable Dependencies: Built with modern, actively maintained standard Google Cloud SDK dependencies (cloud.google.com/go/storage, google.golang.org/api, criticalsys.net/secretprotector, standard Go toolchain).
  • Unprivileged Execution Context: Designed to run cleanly in unprivileged, non-root environments (e.g. non-root Docker containers, restricted Kubernetes pods, standard OS user accounts) with zero system root requirements.

For complete security architecture diagrams, IAM models, and ADC vs. JSON key comparisons, see the Security Architecture section in ARCHITECTURE.md.

2. Code Quality Assessment & Best Practices

The codebase adheres to enterprise Go best practices:

  • Interface Decoupling (StorageClient): Abstracted GCS storage operations behind the StorageClient interface in client.go, enabling 100% offline unit testing without network dependencies.
  • Query Payload Optimization: Configured ProjectionNoACL and SetAttrSelection([]string{"name"}) to fetch only object names, minimizing memory overhead and network bandwidth.
  • Memory Pre-Allocation: Pre-allocates slice capacities (make([]string, 0, maxObjects)) to prevent slice growth re-allocations during object iteration.
  • Context & Deadline Controls: Every network call is bounded by a configurable context.WithTimeout (default 30s). Cross-platform OS signals (SIGINT/SIGTERM) trigger graceful context cancellation.
  • Execution Injection (runApp): The CLI engine in cmd/gcsconntest/main.go accepts custom argument slices and io.Writer buffers for fast in-memory CLI testing.

For comprehensive architecture diagrams, operational sequences, and code relationships, see ARCHITECTURE.md.

3. Command Line Arguments & Exit Codes

CLI Command Flags
Flag Type Description Required Default
-credentials string Path to the Google Cloud service account JSON credential key file (supports plaintext or SecretProtector encrypted files). No* ""
-adc bool Allow falling back to GCP Application Default Credentials if credentials file is omitted. No* false
-bucket string Name of the target GCS bucket to test. Yes ""
-project string Google Cloud Project ID. Yes ""
-prefix string Optional object name prefix filter for listing queries. No ""
-max int Maximum number of object names to retrieve. No 10
-timeout duration Operation timeout duration (e.g., 10s, 1m, 30s). No 30s
-json bool Output test results as formatted JSON for machine consumption. No false
-key string Direct SecretProtector master key (64-char hex or 32-byte raw) to decrypt -credentials content. No ""
-key-env string Environment variable name containing SecretProtector master key. No ""
-key-file string Path to restricted key file containing SecretProtector master key. No ""
-version bool Display application version string and exit. No false

*Either -credentials or -adc (or environment GOOGLE_APPLICATION_CREDENTIALS) is required.

Granular Diagnostic Exit Codes

gcsconntest classifies runtime outcomes into specific exit codes for container health probes and automated monitoring scripts:

Exit Code Classification Name Description & Cause
0 ExitSuccess Connectivity test succeeded, bucket access verified, objects listed.
1 ExitUsageError Missing required flags, invalid parameters, or bad flag syntax.
2 ExitAuthError Credential file unreadable, malformed key JSON, or GCP OAuth2 authentication failed.
3 ExitNetworkError Connection timeout, DNS resolution failure, socket error, or context deadline exceeded.
4 ExitPermissionError HTTP 403 Forbidden or HTTP 404 Bucket Not Found from GCP API.
5 ExitApiError General GCS API iterator error or server-side GCP fault.

4. Usage & Deployment Examples with Output Samples

4.1 Using as a Go Library Package

Import criticalsys.net/gcsconntest directly into your Go application:

package main

import (
	"context"
	"fmt"
	"log"
	"time"

	"criticalsys.net/gcsconntest"
)

func main() {
	ctx := context.Background()

	cfg := gcsconntest.Config{
		CredFile:     "path/to/service-account.json", // or CredJSON: []byte(...), or AllowADC: true
		BucketName:   "my-production-bucket",
		ProjectID:    "my-gcp-project",
		BucketPrefix: "logs/",
		MaxObjects:   3,
		Timeout:      15 * time.Second,
	}

	result, err := gcsconntest.TestConnection(ctx, cfg)
	if err != nil {
		exitCode := gcsconntest.ClassifyError(err)
		log.Fatalf("GCS Health Check Failed [Exit %d]: %v", exitCode, err)
	}

	fmt.Printf("Connected to bucket '%s' successfully!\n", result.BucketName)
	fmt.Printf("Total Listed: %d objects\n", result.TotalListed)
	for _, name := range result.ObjectNames {
		fmt.Println(" -", name)
	}
}
Output Sample (Library Console Output)
Connected to bucket 'my-production-bucket' successfully!
Total Listed: 2 objects
 - logs/2026-07-27-app.log
 - logs/2026-07-27-audit.log

For complete architectural details on integrating gcsconntest into external Go services (such as HTTP handlers or health-checker), see Section 5 of ARCHITECTURE.md.

4.2 Standalone CLI Deployment Examples
Building the Executable
go build -o gcsconntest ./cmd/gcsconntest
Example 1: Service Account Key File (Human Output)
./gcsconntest -credentials /etc/secrets/sa-key.json -bucket my-gcs-bucket -project my-gcp-project -max 3
Meaningful Output Sample (Human Output)
Listing up to 3 objects in bucket my-gcs-bucket:
 - data/backup-2026.tar.gz
 - data/index.csv
 - data/schema.json
GCS connectivity test successful.
Example 2: Keyless Application Default Credentials (ADC) with JSON Output

Ideal for Cloud Run, GKE Workload Identity, or Kubernetes probes:

./gcsconntest -adc -bucket my-gcs-bucket -project my-gcp-project -prefix logs/ -max 2 -json
Meaningful Output Sample (JSON Output)
{
  "bucket_name": "my-gcs-bucket",
  "bucket_attrs": {
    "Name": "my-gcs-bucket",
    "Location": "US",
    "LocationType": "multi-region",
    "StorageClass": "STANDARD",
    "Created": "2025-10-15T08:30:00Z"
  },
  "object_names": [
    "logs/app.log",
    "logs/system.log"
  ],
  "total_listed": 2
}

5. Architectural & Testing Documentation

For in-depth technical details, architectural blueprints, test inventories, and execution guides, refer to the authoritative documentation files:

  • ARCHITECTURE.md:
    • System Architecture & Component Blueprints
    • Operational Flow & Sequence Diagrams
    • Security Architecture & IAM Model
    • Service Account Keys vs. ADC Comparison
    • External Integration Guide for Go Microservices (e.g. health-checker)
  • TESTING.md:
    • Test Suite Architecture & Mocking Design
    • 34-Item Test Inventory Table
    • Statement Coverage Breakdown (87.5%+ Total Coverage)
    • Live GCS Integration Testing Guide across PowerShell, CMD, and Bash

Documentation

Overview

OBJECTIVES: - Define the StorageClient interface to decouple storage operations from concrete GCP client implementations. - Implement defaultStorageClient to execute optimized GCS bucket attribute fetching and object listing.

CORE COMPONENTS & DATA FLOW: - StorageClient Interface: Defines BucketAttrs and ListObjects contracts. - defaultStorageClient Struct: Wraps concrete *storage.Client. - BucketAttrs(): Calls storage.Client.Bucket(name).Attrs(ctx). - ListObjects(): Applies ProjectionNoACL and SetAttrSelection([]string{"name"}), pre-allocates object slice, iterates GCS objects up to maxObjects.

Package gcsconntest provides a production-grade, reusable Go engine and CLI to verify Google Cloud Storage (GCS) connectivity, authentication, and object accessibility.

OBJECTIVES: - Encapsulate configuration parameters for GCS connection testing. - Perform input sanitization (path cleaning) and parameter defaulting. - Enforce mandatory validation rules before initiating network connections.

CORE COMPONENTS & DATA FLOW: - Config struct: Holds credential paths, raw credential bytes, ADC flags, project/bucket metadata, limits, and timeouts. - Config.Clean(): Sanitizes input paths via filepath.Clean and populates default limits/timeouts. - Config.Validate(): Evaluates mandatory parameter presence, returning ErrInvalidConfig on failure.

OBJECTIVES: - Define domain error variables and granular exit codes (0 to 5) for process orchestration. - Implement error classification logic to map standard Go errors, net.Error, and googleapi.Error to diagnostic exit codes.

CORE COMPONENTS & DATA FLOW: - Exit code constants (ExitSuccess, ExitUsageError, ExitAuthError, ExitNetworkError, ExitPermissionError, ExitApiError). - Sentinel error definitions (ErrInvalidConfig, ErrAuthFailed, ErrBucketAccess, ErrNetworkFailed, ErrApiError). - ClassifyError(err error) int: Inspects error tree using errors.Is and errors.As to return precise exit code.

OBJECTIVES: - Encapsulate execution results from GCS connection testing. - Provide JSON serialization support for structured logging and machine consumption.

CORE COMPONENTS & DATA FLOW: - Result struct: Holds target bucket name, bucket attributes (*storage.BucketAttrs), object names slice, and object count. - Result.ToJSON(): Converts Result into a formatted JSON string using json.MarshalIndent.

OBJECTIVES: - Implement core GCS connection verification logic for library consumers and CLI entry points. - Manage client lifecycle, authentication resolution (File, JSON Bytes, ADC), context timeouts, and error handling. - Expose interface-based TestConnectionWithStorageClient to support mock-driven testing.

CORE COMPONENTS & DATA FLOW: - NewClient(ctx, cfg): Resolves authentication options -> creates *storage.Client. - TestConnection(ctx, cfg): Validates config -> cleans parameters -> applies timeout context -> instantiates client -> executes test -> closes client. - TestConnectionWithClient(ctx, client, cfg): Wraps concrete *storage.Client into StorageClient interface. - TestConnectionWithStorageClient(ctx, client, cfg): Executes BucketAttrs and ListObjects operations -> aggregates into *Result.

Index

Constants

View Source 
const (
	ExitSuccess         = 0 // Operational success
	ExitUsageError      = 1 // Missing or invalid configuration / parameters
	ExitAuthError       = 2 // Credentials file unreadable, malformed, or authentication failed
	ExitNetworkError    = 3 // Network failure, DNS issue, or request context timeout
	ExitPermissionError = 4 // GCS 403 Forbidden or 404 Bucket Not Found
	ExitApiError        = 5 // General GCS API or iterator failure
)

Exit codes for CLI diagnostics and process orchestration.

View Source 
const DefaultMaxObjects = 10

DefaultMaxObjects is the default limit for listed objects when not specified.

View Source 
const DefaultTimeout = 30 * time.Second

DefaultTimeout is the default duration limit for GCS operations.

Variables

View Source 
var (
	ErrInvalidConfig = errors.New("invalid configuration")
	ErrAuthFailed    = errors.New("authentication failed")
	ErrBucketAccess  = errors.New("bucket access denied or not found")
	ErrNetworkFailed = errors.New("network connectivity error")
	ErrApiError      = errors.New("API operation error")
)

Custom error types for programmatic classification.

Functions

func ClassifyError 

func ClassifyError(err error) int

ClassifyError inspects an error and returns the corresponding diagnostic exit code. DATA FLOW: Input error -> Context check -> Sentinel error check -> OS file error check -> net.Error check -> googleapi.Error status code check -> Exit Code Integer.

func NewClient 

func NewClient(ctx context.Context, cfg Config) (*storage.Client, error)

NewClient initializes a storage.Client using credentials specified in Config. DATA FLOW: Config -> SecretProtector Key Resolution (optional) -> Credential Mode Resolution (CredJSON / CredFile / AllowADC) -> Decryption (optional) -> Memory Zeroing -> storage.NewClient -> *storage.Client / Error.

Types

type Config 

type Config struct {
	// CredFile is the path to the Google Cloud service account JSON key file.
	CredFile string
	// CredJSON is raw service account JSON key content. If provided, CredFile is ignored.
	CredJSON []byte
	// AllowADC allows falling back to GCP Application Default Credentials if CredFile/CredJSON are omitted.
	AllowADC bool
	// BucketName is the target GCS bucket name (required).
	BucketName string
	// BucketPrefix is an optional prefix filter for object listing.
	BucketPrefix string
	// ProjectID is the Google Cloud Project ID (required).
	ProjectID string
	// MaxObjects limits the number of objects to list. Defaults to 10 if <= 0.
	MaxObjects int
	// Timeout specifies the maximum time allowed for the connection test. Defaults to 30s if <= 0.
	Timeout time.Duration
	// MasterKey is an optional raw/hex SecretProtector master key used to decrypt CredFile or CredJSON.
	MasterKey string
	// MasterKeyEnv is the name of an environment variable containing the SecretProtector master key.
	MasterKeyEnv string
	// MasterKeyFile is the file path containing the SecretProtector master key.
	MasterKeyFile string
}

Config holds the parameters for GCS connection testing.

func (Config) Clean 

func (c Config) Clean() Config

Clean returns a copy of Config with sanitized file paths and defaulted parameters. DATA FLOW: Input Config -> Path Cleaning (filepath.Clean) -> Default Fallbacks -> Cleaned Config Copy.

func (Config) Validate 

func (c Config) Validate() error

Validate checks whether mandatory parameters are supplied. DATA FLOW: Evaluates BucketName, ProjectID, and authentication options (CredFile, CredJSON, AllowADC). Returns a wrapped ErrInvalidConfig on validation error, or nil on success.

type Result 

type Result struct {
	// BucketName is the target bucket tested.
	BucketName string `json:"bucket_name"`
	// BucketAttrs contains metadata retrieved for the bucket.
	BucketAttrs *storage.BucketAttrs `json:"bucket_attrs,omitempty"`
	// ObjectNames contains names of objects found (up to MaxObjects).
	ObjectNames []string `json:"object_names"`
	// TotalListed is the count of objects listed in this execution.
	TotalListed int `json:"total_listed"`
}

Result contains details returned from a GCS connection test.

func TestConnection 

func TestConnection(ctx context.Context, cfg Config) (*Result, error)

TestConnection tests connectivity to GCS by validating configuration, applying timeouts, instantiating a client, fetching bucket attributes, and listing objects. DATA FLOW: Context + Config -> Validation & Cleaning -> Timeout Context Creation -> Client Initialization -> Test Execution -> Resource Cleanup -> *Result.

func TestConnectionWithClient 

func TestConnectionWithClient(ctx context.Context, client *storage.Client, cfg Config) (*Result, error)

TestConnectionWithClient runs the connectivity test using a caller-provided storage.Client. DATA FLOW: Context + *storage.Client + Config -> StorageClient Interface Wrapper -> TestConnectionWithStorageClient.

func TestConnectionWithStorageClient 

func TestConnectionWithStorageClient(ctx context.Context, client StorageClient, cfg Config) (*Result, error)

TestConnectionWithStorageClient runs the connectivity test using a StorageClient interface (enables full offline mocking). DATA FLOW: Context + StorageClient + Config -> BucketAttrs() -> ListObjects() -> Aggregated *Result / Error.

func (*Result) ToJSON 

func (r *Result) ToJSON() (string, error)

ToJSON formats the Result as a pretty-printed JSON string. DATA FLOW: Result Receiver -> json.MarshalIndent -> Formatted JSON String / Error.

type StorageClient 

type StorageClient interface {
	BucketAttrs(ctx context.Context, bucketName string) (*storage.BucketAttrs, error)
	ListObjects(ctx context.Context, bucketName, prefix string, maxObjects int) ([]string, error)
}

StorageClient abstracts GCS bucket attribute and object listing operations to enable unit testing and mocking without live network requests.

func NewStorageClient 

func NewStorageClient(client *storage.Client) StorageClient

NewStorageClient creates a StorageClient interface wrapper around *storage.Client. DATA FLOW: *storage.Client -> StorageClient Interface Wrapper.

Source Files

View all Source files

Directories

Path Synopsis
cmd
gcsconntest command
OBJECTIVES: - Standalone CLI application entry point for Google Cloud Storage connectivity testing.
 OBJECTIVES: - Standalone CLI application entry point for Google Cloud Storage connectivity testing.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.