utils

package
v1.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 4, 2026 License: BSD-3-Clause Imports: 20 Imported by: 0

README

utils

Shared utilities for console output, file operations, downloads, and script parsing.

Architecture

console.go   Styled console output, logging, user prompts
download.go  HTTP downloads with progress bars
files.go     File and directory utilities
parser.go    Build script metadata parsing

Console Output

Print: PrintMessage, PrintSuccess, PrintWarning, PrintError, PrintHint, PrintNote, PrintDebug

Style: StyleName (yellow), StylePath, StyleAction, StyleCommand, StyleHint (cyan), StyleError (red), StyleSuccess (green), StyleWarning (yellow)

Modes: DebugMode, QuietMode, YesMode

utils.PrintMessage("Installing %s", utils.StyleName("package"))
utils.PrintSuccess("Build completed")
utils.ShouldAnswerYes()                      // true when YesMode or non-interactive
utils.ReadLineContext(ctx context.Context)    // read line with cancellation

File Operations

Checks: FileExists, DirExists, IsImg, IsSqf, IsSif, IsOverlay Operations: EnsureDir Permissions: PermFile (0664), PermDir (0775), PermExec (0775) Permission Fixers: FixPermissionsDefault(path)

Downloads

utils.DownloadFile(url, destPath)       // with progress bar
utils.DownloadExecutable(url, destPath) // sets exec permissions

Script Parsing

// parseModuleLoad: also extract "module load" / "ml" lines as deps
deps, err := utils.GetDependenciesFromScript(scriptPath, parseModuleLoad)

prompts, err := utils.GetInteractivePromptsFromScript(scriptPath)

// extract scheduler directives and apply defaults (scheduler package)
specs, err := scheduler.ReadScriptSpecsFromPath(scriptPath)
Template Helpers
// Extract {var} → value from a concrete name matched against a #TARGET: pattern.
// E.g. MatchTemplateTarget("salmon/{ver}", "salmon/1.10.2") → {"ver":"1.10.2"}, true
vars, ok := utils.MatchTemplateTarget(pattern, concrete)

// Interpolate {key} tokens in a string from a vars map.
result := utils.InterpolateVars(template, vars)

// Sort version strings descending (newest first). Returns a new slice.
sorted := utils.SortVersionsDescending(versions)

// Render a #TARGET: pattern with {placeholder} tokens in bold-yellow.
styled := utils.HighlightTemplatePlaceholders(pattern)
Version Constraint Helpers
// Split "samtools/1.22.1>=1.10" → ("samtools/1.22.1", ">=", "1.10")
nv, op, minVer := utils.SplitDepConstraint(raw)

// Compare partial version strings ("1.10" == "1.10.0"). Returns -1/0/1.
cmp := utils.CompareVersions(a, b)

// True if installedVersion satisfies op+minVersion and does not exceed preferredVersion.
// preferredVersion="" skips the upper bound check.
ok := utils.DepSatisfiedByVersion(installed, op, minVersion, preferredVersion)

#DEP:name/version>=min semantics: the preferred version is the implicit upper bound. An installed version is accepted if min <= installed <= preferred.

Documentation

Index

Constants

View Source 
const PermDir os.FileMode = 0775

Dir: u=rwx, g=rwx, o=rx (Requires +x to traverse)

View Source 
const PermExec os.FileMode = 0775

Exec: u=rwx, g=rwx, o=rx (Executable files with group write access)

View Source 
const PermFile os.FileMode = 0664

Standard default permissions File: u=rw, g=rw, o=r

Variables

View Source 
var DebugMode = false

DebugMode controls whether PrintDebug output is visible.

View Source 
var QuietMode = false

QuietMode controls whether verbose messages are suppressed (errors/warnings still shown)

View Source 
var YesMode = false

YesMode controls whether to automatically answer yes to all prompts

Functions

func CanWriteToDir added in v1.4.0 

func CanWriteToDir(dir string) bool

CanWriteToDir checks whether an existing directory is writable using a permission check syscall, without creating the directory or writing any files. Returns false if the directory does not exist. Use this for probe operations (display, bind decisions, search-path scanning). Use EnsureWritableDir when you intend to actually create the directory on first use.

func CanWriteToExistingAncestor added in v1.5.0 

func CanWriteToExistingAncestor(dir string) bool

CanWriteToExistingAncestor walks up the path until it finds an existing directory and checks whether it is writable. Use this when dir may not exist yet but will be created by MkdirAll — a non-existent dir is not read-only, it just needs a writable parent.

func CompareVersions added in v1.4.0 

func CompareVersions(a, b string) int

CompareVersions compares two partial version strings component by component. Missing trailing components are treated as 0 (so "1.16" == "1.16.0"). Returns -1 if a < b, 0 if equal, 1 if a > b.

func CreateFileWritable added in v1.3.0 

func CreateFileWritable(path string) (*os.File, error)

CreateFileWritable creates or truncates a file using standard writable file permissions.

func CurrentCondaPlatform added in v1.4.0 

func CurrentCondaPlatform() string

CurrentCondaPlatform returns the conda platform string for the current OS and architecture, e.g. "linux-64", "linux-aarch64", "osx-arm64". Note: Linux ARM64 is "linux-aarch64" while macOS ARM64 is "osx-arm64".

func DepSatisfiedByVersion added in v1.4.0 

func DepSatisfiedByVersion(installedVersion, op, minVersion, preferredVersion string) bool

DepSatisfiedByVersion returns true if installedVersion satisfies the constraint (op + minVersion) and does not exceed preferredVersion (upper bound). op must be ">=" or ">". preferredVersion may be empty to skip the upper bound check.

func DirExists 

func DirExists(path string) bool

DirExists checks if a path exists and is a directory.

func DownloadExecutable 

func DownloadExecutable(ctx context.Context, url, destPath string) error

DownloadExecutable downloads a file and sets it as executable (PermExec). The download is cancelled if ctx is cancelled.

func DownloadFile 

func DownloadFile(ctx context.Context, url, destPath string) error

DownloadFile downloads a file from url to destPath. The download is cancelled if ctx is cancelled.

func EnsureTmpSubdir added in v1.3.0 

func EnsureTmpSubdir(dir string) error

EnsureTmpSubdir creates the leaf directory dir with permissions based on the parent:

  • Parent world-writable (o+w, e.g. /tmp with mode 1777): use 0700 so other users on the same node cannot read or list the contents.
  • Parent private (scheduler job dir, user-controlled path): use PermDir (0775).

Only the leaf is created; the parent must already exist. An already-existing directory is accepted silently (safe for concurrent builds).

func EnsureWritableDir added in v1.4.0 

func EnsureWritableDir(dir string) bool

EnsureWritableDir creates dir if it does not exist, then checks it is writable. Permissions are set only when the directory is newly created. Returns true if the directory exists (or was created) and is writable.

func ExpandPlaceholders added in v1.4.0 

func ExpandPlaceholders(defs []PlaceholderDef) []map[string]string

ExpandPlaceholders returns the Cartesian product of all placeholder value combinations. "*" is never used as a concrete iteration value. Returns [{}] (a slice with one empty map) when defs is empty.

func ExtractImgPackageTokens added in v1.5.0 

func ExtractImgPackageTokens(imgPackages string) []string

ExtractImgPackageTokens returns the unique {KEY} token names from an #IMG_PACKAGES: template.

func FetchCondaSummary added in v1.4.0 

func FetchCondaSummary(packageName string, channels []string) string

FetchCondaSummary queries the anaconda.org API for a package summary. Channels are tried in the order provided; returns "" if not found or on error. The summary is truncated to at most 100 words.

func FileExists 

func FileExists(path string) bool

FileExists checks if a file exists and is not a directory.

func FindEnvOverlay added in v1.5.0 

func FindEnvOverlay(envImg, wd string) string

FindEnvOverlay returns the first env.img found under wd using the standard search order. At each location the user-specific variant (env-$USER.img) is checked before the generic one: 

{wd}/env-$USER.img → {wd}/env.img
{wd}/overlay/env-$USER.img → {wd}/overlay/env.img
{wd}/src/overlay/env-$USER.img → {wd}/src/overlay/env.img

envImg may be an explicit path (returned as-is) or "" / "env.img" (triggers search). Returns "" if nothing is found.

func FixPermissions 

func FixPermissions(path string, filePerm, dirPerm os.FileMode) error

FixPermissions automatically adjusts permissions for a path using the provided modes. If 'path' is a file, it sets it to 'filePerm'. If 'path' is a directory, it recursively sets files to 'filePerm' and subdirs to 'dirPerm'.

func FixPermissionsDefault 

func FixPermissionsDefault(path string) error

FixPermissionsDefault is a helper that applies standard permissions (0664/0775). It is a shorthand for FixPermissions(path, PermFile, PermDir).

func FormatBytes 

func FormatBytes(bytes int64) string

FormatBytes formats bytes into human-readable format (e.g., "1.50 GB").

func FormatChoicesInline added in v1.5.0 

func FormatChoicesInline(vlist []string) string

FormatChoicesInline formats a value list for single-line display. Version lists (entries containing a dot) are grouped by minor version with the patch shown in dim parentheses: "3.13(.2), 3.12(.10), 3.11(.12)". Option lists are joined with " | ": "github | microsoft".

func FormatDuration added in v1.3.0 

func FormatDuration(d time.Duration) string

FormatDuration formats a duration dropping zero trailing components. Examples: 2h0m0s → "2h", 48h → "2d", 25h30m → "1d1h30m", 1h30m15s → "1h30m15s".

func FormatMemoryMB added in v1.3.0 

func FormatMemoryMB(mb int64) string

FormatMemoryMB formats a MB value into a human-readable string. Uses "GB" when divisible by 1024, otherwise "MB". Examples: 8192 → "8GB", 1536 → "1536MB".

func GetDependenciesFromScript 

func GetDependenciesFromScript(scriptPath string, parseModuleLoad bool) ([]string, error)

GetDependenciesFromScript parses a build script and extracts dependencies from #DEP: comments and, optionally, module load / ml commands. Set parseModuleLoad=true to also parse "module load" and "ml" lines. Returns a list of normalized dependency names with duplicates removed.

func GetExternalBuildTypeFromScript added in v1.3.0 

func GetExternalBuildTypeFromScript(scriptPath string) (string, error)

GetExternalBuildTypeFromScript parses external build script TYPE metadata. Supported tag forms:

  • #TYPE:<value>
  • TYPE:<value>

If TYPE is not present, it defaults to "app". Supported values (case-insensitive):

  • app aliases: app, env, tool, conda, small
  • data aliases: data, ref, large

func GetInteractivePromptsFromScript 

func GetInteractivePromptsFromScript(scriptPath string) ([]string, error)

GetInteractivePromptsFromScript parses a build script and extracts interactive prompt lines beginning with "#INTERACTIVE:". Returns a list of prompt strings (without the prefix) or an error if the file cannot be read.

func GetTargetFromScript added in v1.4.0 

func GetTargetFromScript(scriptPath string) (string, error)

GetTargetFromScript reads a build script and returns the value of the first #TARGET: line (trimmed). Returns "" if no #TARGET: line is present.

func GetTmpDir added in v1.3.0 

func GetTmpDir() string

GetTmpDir returns the best available tmp directory for condatainer builds. Priority:

  1. CNT_TMPDIR (explicit override)
  2. Scheduler-specific local node storage: SLURM_TMPDIR, PBS_TMPDIR, LSF_TMPDIR, _CONDOR_SCRATCH_DIR
  3. TMPDIR (POSIX standard)
  4. /tmp (always available)

Always appends cnt-$USER to avoid collisions between users.

func GetWhatIsFromScript added in v1.2.0 

func GetWhatIsFromScript(scriptPath string) string

GetWhatIsFromScript reads a script and extracts the first #WHATIS: line. Returns the trimmed description string, or empty string if not found.

func HighlightTemplatePlaceholders added in v1.4.0 

func HighlightTemplatePlaceholders(pattern string) string

HighlightTemplatePlaceholders renders a #TARGET: pattern with {var} tokens in yellow-bold and the surrounding literal path segments in default color.

func InterpolateVars added in v1.4.0 

func InterpolateVars(s string, vars map[string]string) string

InterpolateVars replaces {varname} occurrences in s with the corresponding value from vars. Unknown keys are left unchanged.

func IsImg 

func IsImg(path string) bool

IsImg checks if the path has an ext3 overlay extension (.img). Note: In Apptainer context, .img usually implies a writable ext3 overlay.

func IsInteractiveShell 

func IsInteractiveShell() bool

IsInteractiveShell checks if stdout is connected to a TTY (interactive terminal). Returns true if the program is running in an interactive shell, false otherwise.

func IsOverlay 

func IsOverlay(path string) bool

IsOverlay checks if the path is an overlay file (.img, .sqf, .sqsh, .squashfs). This is used for CondaTainer overlay detection.

func IsSif 

func IsSif(path string) bool

IsSif checks if the path has a Singularity Image Format extension (.sif). This is the native format for Apptainer/Singularity.

func IsSqf 

func IsSqf(path string) bool

IsSqf checks if the path has a SquashFS extension (.sqf, .sqsh, .squashfs). These are read-only compressed images.

func IsStdinPiped added in v1.4.0 

func IsStdinPiped() bool

IsStdinPiped returns true if stdin is a pipe or redirected (not a TTY).

func IsYaml 

func IsYaml(path string) bool

IsYaml checks if the path has a YAML extension (.yaml, .yml). Useful for Conda environment definition files.

func LatestVersion added in v1.5.0 

func LatestVersion(versions []string) string

LatestVersion returns the first (newest) entry in a versions list, or "" if empty.

func MatchTemplateTarget added in v1.4.0 

func MatchTemplateTarget(pattern, concrete string) (map[string]string, bool)

MatchTemplateTarget extracts placeholder values from a concrete name by matching it against a template pattern (e.g. a #TARGET: or #DEP: line containing {var} tokens). Returns the vars map and true on a full match, or nil and false otherwise.

func MatchVersion added in v1.5.0 

func MatchVersion(input string, versions []string) string

MatchVersion does a partial version match against a list sorted newest-first. Input "3.12" matches the first entry beginning with "3.12." (latest patch). Exact match always takes priority. Returns input unchanged if nothing matches.

func NormalizeNameVersion 

func NormalizeNameVersion(nameVersion string) string

NormalizeNameVersion normalizes package spec formats so that "name/version", "name=version", "name@version" are treated the same. Converts = and @ to /, and -- to /, then strips whitespace. Any version constraint suffix (e.g. ">=1.10") is preserved as-is.

func ParseDHMSTime added in v1.2.0 

func ParseDHMSTime(timeStr string) (time.Duration, error)

ParseDHMSTime handles colon-separated and D-HH:MM:SS walltime formats.

func ParseHMSTime added in v1.2.0 

func ParseHMSTime(timeStr string) (time.Duration, error)

ParseHMSTime parses colon-separated walltime "HH:MM:SS", "HH:MM", or "MM" (bare minutes).

func ParseMemoryMB added in v1.2.0 

func ParseMemoryMB(memStr string) (int64, error)

ParseMemoryMB parses memory strings like "8G", "1024M", "512K", "1T" into MB (int64). Default unit is MB when no suffix is given.

func ParseMemoryMBWithDefault added in v1.3.0 

func ParseMemoryMBWithDefault(memStr, defaultUnit string) (int64, error)

ParseMemoryMBWithDefault parses a memory string into MB. When the string has an explicit unit suffix (G/GB/M/MB/K/KB/T/TB) it is used directly. For bare numbers the provided defaultUnit is applied (e.g. "KB", "MB", "GB").

func ParseSizeToMB 

func ParseSizeToMB(sizeStr string) (int, error)

ParseSizeToMB converts strings like "10G", "500M", "1024" into Megabytes (int). Default unit is MB if no suffix is provided. Delegates to ParseMemoryMB.

func ParseWalltime added in v1.2.0 

func ParseWalltime(timeStr string) (time.Duration, error)

ParseWalltime parses a walltime string into a duration. Supported formats:

Compound (Go-style, with int day): 4d12h, 2h30m, 3h, 90m, 1d2h30m45s, 1.5h

Colon-separated: D-HH:MM:SS, HH:MM:SS, HH:MM, MM

func PrintDebug 

func PrintDebug(format string, a ...any)

PrintDebug prints a debug message with a Gray prefix (only if DebugMode is true). → stderr Output: [CNT…] Executing: rm -rf /tmp/foo

func PrintError 

func PrintError(format string, a ...any)

PrintError prints an error message with a Red prefix. → stderr Output: [CNT✗] Something failed.

func PrintHint 

func PrintHint(format string, a ...any)

PrintHint prints a helpful hint with a Cyan prefix. → stderr Output: [CNT▶] Try running with --force.

func PrintMessage 

func PrintMessage(format string, a ...any)

PrintMessage prints a standard info message. → stderr Output: [CNT] Message...

func PrintNote 

func PrintNote(format string, a ...any)

PrintNote prints a note with a Magenta prefix. → stderr Output: [CNT◇] This might take a while.

func PrintSuccess 

func PrintSuccess(format string, a ...any)

PrintSuccess prints a success message with a Green prefix. → stderr Output: [CNT✓] Operation complete.

func PrintWarning 

func PrintWarning(format string, a ...any)

PrintWarning prints a warning with a Yellow prefix. → stderr Output: [CNT!] Disk is almost full.

func ReadGzipJSONFile added in v1.3.0 

func ReadGzipJSONFile(path string, out any) error

ReadGzipJSONFile opens a .json.gz file and decodes JSON into out.

func ReadLineContext added in v1.2.0 

func ReadLineContext(ctx context.Context) (string, error)

ReadLineContext reads a trimmed line from stdin, preserving case. Returns context.Canceled if ctx is done before input arrives.

func RemoveDirIfEmpty added in v1.3.0 

func RemoveDirIfEmpty(dir string)

RemoveDirIfEmpty removes dir if it exists and contains no files or subdirectories. Silently does nothing if dir is not empty or does not exist.

func ShouldAnswerYes 

func ShouldAnswerYes() bool

ShouldAnswerYes checks if we should automatically answer yes to prompts Returns true if --yes flag is set, false otherwise

func SortVersionsDescending added in v1.4.0 

func SortVersionsDescending(values []string) []string

SortVersionsDescending sorts version strings in descending natural order. Segments are split on ".", "-", or "_" and compared numerically when both segments are integers, otherwise lexicographically. The highest version comes first.

func SplitDepConstraint added in v1.4.0 

func SplitDepConstraint(raw string) (nameVersion, op, minVersion string)

SplitDepConstraint splits a raw dep string into its nameVersion and optional version constraint. Supports ">=" and ">" operators. Example: "samtools/1.21>=1.16" → ("samtools/1.21", ">=", "1.16") Example: "samtools/1.21" → ("samtools/1.21", "", "")

func StripInlineComment added in v1.2.0 

func StripInlineComment(s string) string

StripInlineComment removes everything after the first '#' character (inline comment). Returns the trimmed string without the comment.

func StyleAction 

func StyleAction(act string) string

StyleAction formats verbs or active operations (Yellow).

func StyleDebug 

func StyleDebug(msg string) string

StyleDebug formats low-level technical info (Gray).

func StyleError 

func StyleError(msg string) string

StyleError formats critical failure messages (Red).

func StyleHighlight 

func StyleHighlight(text string) string

StyleHighlight formats search matches or highlighted text (Yellow Bold).

func StyleHint 

func StyleHint(msg string) string

StyleHint formats helpful tips or suggestions (Cyan).

func StyleInfo 

func StyleInfo(msg string) string

StyleInfo formats status labels or properties (Magenta)

func StyleName 

func StyleName(name string) string

StyleName formats names, identifiers, or keys (Yellow).

func StyleNote 

func StyleNote(msg string) string

StyleNote formats neutral notes or annotations (Magenta).

func StyleNumber 

func StyleNumber(num any) string

StyleNumber formats counts, sizes, or IDs (Magenta).

func StylePath 

func StylePath(path string) string

StylePath formats file paths with context-aware coloring.

func StyleSuccess 

func StyleSuccess(msg string) string

StyleSuccess formats success messages (Green).

func StyleTitle 

func StyleTitle(title string) string

StyleTitle

func StyleWarning 

func StyleWarning(msg string) string

StyleWarning formats non-critical warnings (Yellow).

func TruncateWords added in v1.4.0 

func TruncateWords(s string, n int) string

TruncateWords returns s truncated to at most n words, appending "..." if truncated.

func URLExists 

func URLExists(ctx context.Context, url string) bool

URLExists checks if a URL exists (returns HTTP 200) using a HEAD request. Returns false if ctx is cancelled.

func VersionChoicesDisplay added in v1.5.0 

func VersionChoicesDisplay(versions []string) []string

VersionChoicesDisplay formats a newest-first full-patch list into grouped minor version summary strings: ["3.13.2","3.12.10","3.11.12"] → ["3.13(.2)", "3.12(.10)", "3.11(.12)"].

func WriteGzipJSONFileAtomic added in v1.3.0 

func WriteGzipJSONFileAtomic(path string, value any) error

WriteGzipJSONFileAtomic encodes value as JSON, writes it as .json.gz to a temp file, then atomically renames it to path.

Types

type CondaSearchResult added in v1.4.0 

type CondaSearchResult struct {
	Name      string   `json:"name"`
	Channel   string   `json:"owner"` // anaconda.org uses "owner" for the channel name
	Summary   string   `json:"summary"`
	Versions  []string `json:"versions"`
	Platforms []string `json:"conda_platforms"` // supported conda platforms, e.g. ["linux-64", "osx-arm64"]
}

CondaSearchResult holds a single result from the anaconda.org search API.

func SearchCondaPackages added in v1.4.0 

func SearchCondaPackages(name string, channels []string, fuzzy bool, limit int) ([]CondaSearchResult, bool, error)

SearchCondaPackages looks up conda packages on anaconda.org.

Exact match (default): queries /package/<channel>/<name> for each channel in order and returns the first hit — matching install behaviour.

Fuzzy match (fuzzy=true): issues two requests to /search?name=<query> — one for the current platform (e.g. linux-64) and one for noarch — then merges and deduplicates the results. Two requests are required because the API only accepts a single platform value and silently drops noarch-only packages when a platform filter is set. Returns capped=true if either request hit the limit (results may be incomplete).

Versions within each result are sorted descending.

type PlaceholderDef added in v1.4.0 

type PlaceholderDef struct {
	Name   string
	Values []string // sorted desc concrete values; last element is "*" if Open
	Open   bool     // true if * was present in the original list
}

PlaceholderDef holds a single #PL: declaration from a build script. Values are sorted descending (natural version sort); if Open is true, the last element of Values is "*" indicating any input is accepted.

func GetPlaceholdersFromScript added in v1.4.0 

func GetPlaceholdersFromScript(scriptPath string) ([]PlaceholderDef, error)

GetPlaceholdersFromScript parses a build script and returns all #PL: declarations in declaration order. Values are sorted descending; if the list contained "*" the last element is "*" and Open is true.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.