fileutil

Documentation

Overview

Package fileutil provides secure file system utilities for Azure Developer CLI extensions.

This package offers atomic file operations, JSON handling, and directory management with built-in security considerations and retry logic. All operations are designed to be safe in concurrent environments and protect against common file system issues.

Key Features

• Atomic file writes with retry logic to prevent partial writes
• JSON read/write with graceful handling of missing files
• Directory creation with secure permissions (0750)
• File existence checks (single, any, all patterns)
• File extension detection
• Text containment checks with security validation

Security Considerations

The package integrates with the security package to validate paths before reading files, preventing path traversal attacks. Files are created with 0644 permissions and directories with 0750 permissions to prevent unauthorized access.

Atomic Write Operations

AtomicWriteJSON and AtomicWriteFile ensure that files are never left in a partial state by writing to a temporary file first, then atomically renaming it to the target path. This approach includes:

• Unique temporary file names to avoid concurrent writer collisions
• Explicit sync operations to ensure data is flushed to disk
• Retry logic (5 attempts with 20ms backoff) for rename operations
• Automatic cleanup of temporary files on failure

Example Usage

// Write configuration as JSON atomically
config := map[string]interface{}{
    "version": "1.0",
    "services": []string{"api", "web"},
}
if err := fileutil.AtomicWriteJSON("config.json", config); err != nil {
    log.Fatal(err)
}

// Read JSON with graceful handling of missing files
var data map[string]interface{}
if err := fileutil.ReadJSON("config.json", &data); err != nil {
    log.Fatal(err)
}

// Ensure directory exists with secure permissions
if err := fileutil.EnsureDir("./cache/data"); err != nil {
    log.Fatal(err)
}

// Check if project file exists
if fileutil.FileExists(".", "package.json") {
    fmt.Println("Node.js project detected")
}

// Check for any C# project file
if fileutil.HasFileWithExt(".", ".csproj") {
    fmt.Println(".NET project detected")
}

// Check if all required files exist
if fileutil.FilesExistAll(".", "package.json", "tsconfig.json") {
    fmt.Println("TypeScript project detected")
}

// Check for framework-specific configuration
if fileutil.ContainsTextInFile(".", "package.json", "\"next\"") {
    fmt.Println("Next.js project detected")
}

// Write raw file data atomically
data := []byte("Hello, World!")
if err := fileutil.AtomicWriteFile("output.txt", data, 0644); err != nil {
    log.Fatal(err)
}

File Permissions

The package uses secure default permissions:

• DirPermission (0750): rwxr-x--- - Owner can read/write/execute, group can read/execute
• FilePermission (0644): rw-r--r-- - Owner can read/write, others can read only

These defaults prevent unauthorized modification while allowing appropriate access.

Concurrency Safety

Atomic write operations are designed to be safe when called concurrently:

• Temporary files use unique names based on process ID and timestamp
• Rename operations are atomic on most file systems
• Retry logic handles transient failures from concurrent access

Error Handling

Functions return descriptive errors with context:

• ReadJSON returns nil (no error) for missing files
• ContainsText returns false for invalid paths or read errors
• Atomic writes clean up temporary files on any failure

All errors are wrapped with context using fmt.Errorf and %w for proper error chains.

Index

• Constants
• func AtomicWriteFile(path string, data []byte, perm os.FileMode) error
• func AtomicWriteJSON(path string, data interface{}) error
• func ClearCache(path string) error
• func ContainsText(filePath string, text string) bool
• func ContainsTextInFile(dir string, filename string, text string) bool
• func EnsureDir(path string) error
• func FileExists(dir string, filename string) bool
• func FileExistsAny(dir string, filenames ...string) bool
• func FilesExistAll(dir string, filenames ...string) bool
• func HasAnyFileWithExts(dir string, exts ...string) bool
• func HasFileWithExt(dir string, ext string) bool
• func LoadCacheJSON(path string, target interface{}, opts CacheOptions) (valid bool, err error)
• func ReadJSON(path string, target interface{}) error
• func SaveCacheJSON(path string, data interface{}, version string) error
• type CacheEntry
• type CacheMetadata
  • func (m CacheMetadata) IsCacheValid(opts CacheOptions) bool
• type CacheOptions

Constants

const (
	// DirPermission is the default permission for creating directories (rwxr-x---)
	DirPermission = 0750
	// FilePermission is the default permission for creating files (rw-r--r--)
	FilePermission = 0644
)

File permissions

Functions

func AtomicWriteFile

func AtomicWriteFile(path string, data []byte, perm os.FileMode) error

AtomicWriteFile writes raw bytes to a file atomically. It writes to a temporary file first, then renames it to the target path. This ensures the file is never left in a partial/corrupt state.

func AtomicWriteJSON

func AtomicWriteJSON(path string, data interface{}) error

AtomicWriteJSON writes data as JSON to a file atomically. It writes to a temporary file first, then renames it to the target path. This ensures the file is never left in a partial/corrupt state.

func ClearCache

func ClearCache(path string) error

ClearCache removes a cache file if it exists. Returns nil if the file doesn't exist.

func ContainsText

func ContainsText(filePath string, text string) bool

ContainsText checks if a file contains the specified text. Returns false if file doesn't exist, can't be read, or validation fails.

func ContainsTextInFile

func ContainsTextInFile(dir string, filename string, text string) bool

ContainsTextInFile checks if file contains text at the specified path. Convenience function combining filepath.Join and ContainsText.

func EnsureDir

func EnsureDir(path string) error

EnsureDir creates a directory if it doesn't exist.

func FileExists

func FileExists(dir string, filename string) bool

FileExists checks if a file exists in a directory. Returns true if the file exists, false otherwise.

func FileExistsAny

func FileExistsAny(dir string, filenames ...string) bool

FileExistsAny checks if any of the given filenames exist in the directory.

func FilesExistAll

func FilesExistAll(dir string, filenames ...string) bool

FilesExistAll checks if all of the given filenames exist in the directory.

func HasAnyFileWithExts

func HasAnyFileWithExts(dir string, exts ...string) bool

HasAnyFileWithExts checks if any file with any of the given extensions exists.

func HasFileWithExt

func HasFileWithExt(dir string, ext string) bool

HasFileWithExt checks if any file with the given extension exists in the directory. ext should include the dot (e.g., ".csproj")

func LoadCacheJSON

func LoadCacheJSON(path string, target interface{}, opts CacheOptions) (valid bool, err error)

LoadCacheJSON loads a JSON cache file if it exists and is valid. Returns:

• valid=true if cache was loaded and is valid
• valid=false if cache doesn't exist, is expired, or version mismatched
• error only for actual read/parse errors (not for missing files)

The target should be a pointer to a CacheEntry or a struct containing CacheMetadata.

Example:

type MyCache struct {
    Metadata fileutil.CacheMetadata `json:"_cache"`
    Items    []string               `json:"items"`
}
var cache MyCache
valid, err := fileutil.LoadCacheJSON(path, &cache, fileutil.CacheOptions{TTL: 24*time.Hour})
if err != nil {
    return err
}
if !valid {
    // Rebuild cache
}

func ReadJSON

func ReadJSON(path string, target interface{}) error

ReadJSON reads JSON from a file into the target interface. Returns nil error if file doesn't exist (target unchanged).

func SaveCacheJSON

func SaveCacheJSON(path string, data interface{}, version string) error

SaveCacheJSON saves data to a JSON cache file with metadata. It wraps the data in a CacheEntry with the current timestamp.

Example:

err := fileutil.SaveCacheJSON(path, myData, "1.0.0")

Types

type CacheEntry

type CacheEntry struct {
	Metadata CacheMetadata `json:"_cache"`
	Data     interface{}   `json:"data"`
}

CacheEntry wraps cached data with metadata. Use this structure when saving cache data:

entry := fileutil.CacheEntry{
    Metadata: fileutil.CacheMetadata{CachedAt: time.Now(), Version: "1.0.0"},
    Data:     myData,
}
fileutil.SaveCacheJSON(path, entry)

type CacheMetadata

type CacheMetadata struct {
	// CachedAt is when the cache was created.
	CachedAt time.Time `json:"cachedAt"`
	// Version is the version string when the cache was created.
	Version string `json:"version,omitempty"`
}

CacheMetadata is embedded in cached data to track validity.

func (CacheMetadata) IsCacheValid

func (m CacheMetadata) IsCacheValid(opts CacheOptions) bool

IsCacheValid checks if a cache entry is still valid according to the options.

type CacheOptions

type CacheOptions struct {
	// TTL is the time-to-live for the cache. If the cache is older than this, it's considered invalid.
	TTL time.Duration
	// Version is used to invalidate the cache when it changes (e.g., app version).
	Version string
}

CacheOptions configures cache behavior for LoadCacheJSON.