Documentation
¶
Overview ¶
Package tools provides utilities consumed by agent-facing tool implementations.
Index ¶
- Constants
- func ApplyEditsToNormalizedContent(content string, edits []Edit, path string) (base, updated string, err error)
- func DetectLineEnding(content string) string
- func DetectSupportedImageMimeType(data []byte) string
- func FormatSize(bytes int) string
- func FuzzyFindText(content, oldText string) (start, end int, found bool)
- func GenerateDiffString(oldContent, newContent string) (diff string, firstChangedLine int)
- func GenerateUnifiedPatch(path, oldContent, newContent string, contextLines int) string
- func NewBashTool(opts BashToolOptions) pi.RegisteredTool
- func NewEditTool(opts EditToolOptions) pi.RegisteredTool
- func NewReadTool(opts ReadToolOptions) pi.RegisteredTool
- func NewWriteTool(opts WriteToolOptions) pi.RegisteredTool
- func NormalizeForFuzzyMatch(text string) string
- func NormalizeToLF(text string) string
- func NormalizeToolPath(path string) string
- func ResolveReadToolPath(ctx context.Context, env ExecutionEnv, path string) (string, error)
- func ResolveToolPath(ctx context.Context, env ExecutionEnv, path string) (string, error)
- func RestoreLineEndings(text, ending string) string
- func StripBOM(content string) (bomOut, text string)
- type BashExecution
- type BashToolOptions
- type Edit
- type EditToolOptions
- type ExecResult
- type ExecSpec
- type ExecutionEnv
- type FileInfo
- type FileKind
- type OSEnv
- func (e *OSEnv) AbsolutePath(ctx context.Context, path string) (string, error)
- func (e *OSEnv) AppendFile(ctx context.Context, path string, data []byte) error
- func (e *OSEnv) CanonicalPath(ctx context.Context, path string) (string, error)
- func (e *OSEnv) CreateTemp(ctx context.Context, prefix, suffix string) (string, error)
- func (e *OSEnv) Cwd() string
- func (e *OSEnv) Exec(ctx context.Context, spec ExecSpec) (*ExecResult, error)
- func (e *OSEnv) Exists(ctx context.Context, path string) (bool, error)
- func (e *OSEnv) FileInfo(ctx context.Context, path string) (FileInfo, error)
- func (e *OSEnv) ReadFile(ctx context.Context, path string) ([]byte, error)
- func (e *OSEnv) WriteFile(ctx context.Context, path string, data []byte) error
- type ReadImageProcessor
- type ReadImageProcessorResult
- type ReadToolOptions
- type ShellCapture
- type ShellCaptureOptions
- type ShellProgress
- type TruncationOptions
- type TruncationResult
- type WriteToolOptions
Constants ¶
const DefaultMaxBytes = 50 * 1024
DefaultMaxBytes is the byte limit (50KB) applied when TruncationOptions.MaxBytes is zero.
const DefaultMaxLines = 2000
DefaultMaxLines is the line limit applied when TruncationOptions.MaxLines is zero.
Variables ¶
This section is empty.
Functions ¶
func ApplyEditsToNormalizedContent ¶
func ApplyEditsToNormalizedContent(content string, edits []Edit, path string) (base, updated string, err error)
ApplyEditsToNormalizedContent applies one or more exact-text replacements to LF-normalized content. All edits are matched against the same original content; replacements are then applied in reverse offset order so offsets stay stable. If any edit needs fuzzy matching, the operation runs in fuzzy-normalized content space and then overlays those line-level changes onto content so unchanged line blocks keep their original bytes. path is used only to name the file in error messages. Ported from upstream's applyEditsToNormalizedContent.
Deliberate addition beyond the upstream port: upstream's "edits must contain at least one replacement" check lives in edit.ts's validateEditInput (the Task 11 edit tool), not in this engine function - applyEditsToNormalizedContent given a zero-length edits array falls through to the generic "No changes made" error instead. Task 9's pinned test set requires this engine to reject a zero-length edits array with that exact tool-level message, so this guard reproduces it verbatim here too, making the engine self-defending independent of which caller invokes it.
func DetectLineEnding ¶
DetectLineEnding reports which line ending predominates at the START of content: it looks at whichever of the first "\r\n" or the first "\n" occurs earlier in content, and returns "\r\n" only if a CRLF occurs no later than the first bare LF. Ported from upstream's detectLineEnding.
func DetectSupportedImageMimeType ¶
DetectSupportedImageMimeType sniffs data's leading bytes and returns the MIME type of a supported image format ("image/jpeg", "image/png", "image/gif", "image/webp", or "image/bmp"), or "" if data is not a recognized image, or is a recognized-but-unsupported variant (an `ff d8 ff f7` JPEG-family marker, or an animated PNG). Ported from upstream's detectSupportedImageMimeType.
func FormatSize ¶
FormatSize formats bytes as a human-readable size, e.g. "512B", "50.0KB", "1.2MB".
func FuzzyFindText ¶
FuzzyFindText locates oldText within content and reports the byte range [start, end) of the match. It tries an exact substring match first; on exact match, start/end are byte offsets into content itself. If no exact match exists, it retries against NormalizeForFuzzyMatch(content) and NormalizeForFuzzyMatch(oldText) - on a fuzzy match, start/end are byte offsets into NormalizeForFuzzyMatch(content), NOT into content, since fuzzy normalization can change byte length (trimming trailing whitespace, collapsing multi-byte Unicode quotes/dashes/spaces to single-byte ASCII). found is false, with start=end=0, when neither match succeeds. Ported from upstream's fuzzyFindText.
func GenerateDiffString ¶
GenerateDiffString renders a display-oriented diff of oldContent vs. newContent: each changed line is prefixed with "+" or "-" and its line number (in the new or old file respectively); unchanged lines are shown as up to generateDiffStringContextLines of context around each change, with "..." marking elided runs. firstChangedLine is the 1-indexed line number (in newContent) of the first change, or 0 if oldContent and newContent produce no changes. Ported from upstream's generateDiffString.
func GenerateUnifiedPatch ¶
GenerateUnifiedPatch renders a standard unified diff of oldContent vs. newContent, with "--- <path>" / "+++ <path>" file headers (no index or underline lines) and up to contextLines of surrounding context per hunk. contextLines of 0 selects upstream's default of 4. Ported from upstream's generateUnifiedPatch, which always calls jsdiff's createTwoFilesPatch with headerOptions: Diff.FILE_HEADERS_ONLY and no oldHeader/newHeader.
func NewBashTool ¶
func NewBashTool(opts BashToolOptions) pi.RegisteredTool
NewBashTool creates the "bash" tool: run a shell command, streaming throttled partial output as it arrives and reporting truncation/exit/ timeout/abort status in the final result. Ported from upstream's createBashTool. Registered via ExecuteStream (not Execute): the root package's pi.NewTool detects this and builds a tool that also supports plain Execute (with emit silently discarded) for callers that don't care about partial updates.
func NewEditTool ¶
func NewEditTool(opts EditToolOptions) pi.RegisteredTool
NewEditTool creates the "edit" tool: apply one or more exact-text replacements to a file. Ported from upstream's createEditTool.
func NewReadTool ¶
func NewReadTool(opts ReadToolOptions) pi.RegisteredTool
NewReadTool creates the "read" tool: read a text or image file. Ported from upstream's createReadTool.
func NewWriteTool ¶
func NewWriteTool(opts WriteToolOptions) pi.RegisteredTool
NewWriteTool creates the "write" tool: write content to a file, creating the file (and its parent directories) if it doesn't exist and overwriting it if it does. Ported from upstream's createWriteTool.
func NormalizeForFuzzyMatch ¶
NormalizeForFuzzyMatch normalizes text for fuzzy matching by applying, in order: Unicode NFKC normalization, stripping trailing whitespace from each line, normalizing smart quotes to ASCII equivalents, normalizing Unicode dashes/hyphens to ASCII hyphen, and normalizing special Unicode spaces to a regular space. Ported from upstream's normalizeForFuzzyMatch.
func NormalizeToLF ¶
NormalizeToLF collapses every "\r\n" and lone "\r" in text down to "\n". Ported from upstream's normalizeToLF.
func NormalizeToolPath ¶
NormalizeToolPath collapses Unicode space variants (see unicodeSpaces) to a plain ASCII space, and strips a single leading "@" (a mention-style prefix some clients prepend to pasted file paths). Ported from upstream's normalizeToolPath.
func ResolveReadToolPath ¶
ResolveReadToolPath resolves path like ResolveToolPath, then - if that exact resolved path doesn't exist - tries a fixed, ordered set of Unicode "healing" variants and returns the first one that exists on disk:
- the resolved path itself
- narrowNoBreakSpace substituted for the ASCII space before "AM."/"PM."
- NFD (canonical decomposition) Unicode normalization
- curlyRightSingleQuote in place of every ASCII apostrophe
- both (3) and (4) combined
Duplicate variants (e.g. when none of the substitutions apply) are only checked once. If none of the variants exist, the plain resolved path is returned unchanged - ResolveReadToolPath never itself decides a path doesn't exist; that's left to the caller's subsequent read. Ported from upstream's resolveReadToolPath.
func ResolveToolPath ¶
ResolveToolPath normalizes path (via NormalizeToolPath) and resolves it to an absolute path via env.AbsolutePath. Ported from upstream's resolveToolPath.
func RestoreLineEndings ¶
RestoreLineEndings expands every "\n" in text back to ending ("\r\n" or "\n"). Ported from upstream's restoreLineEndings.
Types ¶
type BashExecution ¶
type BashExecution struct {
// Command is the shell command that will run, already including
// BashToolOptions.CommandPrefix if one was configured.
Command string
// Cwd is the working directory the command will run in. Defaults to
// the tool's ExecutionEnv.Cwd().
Cwd string
// Env holds extra environment variables for the command, layered on
// top of the inherited environment per InheritEnv. Starts empty.
Env map[string]string
// InheritEnv controls whether the command additionally inherits the
// calling process's environment. Defaults to true.
InheritEnv bool
}
BashExecution describes a single "bash" tool invocation about to run. It is handed to BashToolOptions.Prepare (by pointer) so callers can inspect or mutate it - e.g. adding environment variables, or rewriting the working directory - before the command actually executes.
type BashToolOptions ¶
type BashToolOptions struct {
// Env is the filesystem/shell seam the tool runs through.
Env ExecutionEnv
// CommandPrefix, if non-empty, is prepended to every command as
// CommandPrefix + "\n" + command - e.g. "set -e" to make the command
// fail fast on any non-zero-exiting step.
CommandPrefix string
// Prepare, if set, runs after the BashExecution is built (command,
// cwd, env, inheritEnv) and before it executes. It may mutate exec in
// place. Returning an error aborts the call without executing the
// command - the error's message becomes the tool's error result.
Prepare func(ctx context.Context, exec *BashExecution) error
}
BashToolOptions configures NewBashTool.
type Edit ¶
Edit is one targeted oldText -> newText replacement. Ported from upstream's Edit interface.
type EditToolOptions ¶
type EditToolOptions struct {
// Env is the filesystem seam the tool edits through.
Env ExecutionEnv
}
EditToolOptions configures NewEditTool.
type ExecResult ¶
type ExecResult struct {
// ExitCode is the process's exit code. When Cancelled or TimedOut is
// true, the process was killed rather than exiting normally, and
// ExitCode reflects whatever the Go runtime reports for a
// signal-terminated process (-1 on POSIX, via
// os.ProcessState.ExitCode()) - Cancelled/TimedOut, not ExitCode, are
// the authoritative signal that the process didn't complete on its
// own.
ExitCode int
// Cancelled reports whether the command was killed because ctx was
// cancelled.
Cancelled bool
// TimedOut reports whether the command was killed because Timeout
// elapsed.
TimedOut bool
}
ExecResult is the outcome of an ExecutionEnv.Exec call.
type ExecSpec ¶
type ExecSpec struct {
// Command is the shell command to run (via `bash -c`).
Command string
// Dir is the working directory for the command. Empty selects the
// environment's Cwd().
Dir string
// Env holds extra environment variables for the command. Their
// interaction with the inherited environment is controlled by
// InheritEnv.
Env map[string]string
// InheritEnv, when true, runs the command with the calling process's
// environment plus Env layered on top. When false, the command runs
// with EXACTLY Env as its environment (nothing inherited).
InheritEnv bool
// Timeout bounds how long the command may run. A value <= 0 means no
// timeout is applied at this layer (validating a user-supplied timeout,
// e.g. rejecting <= 0, is the caller's responsibility).
Timeout time.Duration
// OnChunk, if set, is called with each chunk of output as it arrives.
// stdout and stderr are merged into a single stream, delivered in
// arrival order. Calls are serialized (mutex-guarded on the OSEnv
// implementation): OnChunk is never invoked concurrently with itself,
// and must not retain the passed slice past the call. Because calls
// are serialized, a slow or blocking OnChunk stalls output pumping for
// both stdout and stderr - it holds the underlying writer's lock for
// the duration of the call.
OnChunk func(chunk []byte)
}
ExecSpec configures a call to ExecutionEnv.Exec.
type ExecutionEnv ¶
type ExecutionEnv interface {
// Cwd returns the environment's working directory, as an absolute path.
Cwd() string
// AbsolutePath resolves path to an absolute path, relative to Cwd() if
// path is not already absolute. It does not touch the filesystem.
AbsolutePath(ctx context.Context, path string) (string, error)
// CanonicalPath resolves path (via AbsolutePath) and then resolves any
// symlinks in it, returning the real underlying path. The path must
// exist.
CanonicalPath(ctx context.Context, path string) (string, error)
// Exists reports whether path exists (following symlinks is not
// required to succeed: a broken symlink still exists).
Exists(ctx context.Context, path string) (bool, error)
// FileInfo stats path (without following a trailing symlink) and
// returns its kind and size.
FileInfo(ctx context.Context, path string) (FileInfo, error)
// ReadFile reads the entire contents of path.
ReadFile(ctx context.Context, path string) ([]byte, error)
// WriteFile writes data to path, creating path's parent directories if
// they don't already exist, and truncating/overwriting any existing
// file at path.
WriteFile(ctx context.Context, path string, data []byte) error
// AppendFile appends data to path, creating the file (and its parent
// directories) if it doesn't already exist.
AppendFile(ctx context.Context, path string, data []byte) error
// CreateTemp creates a new, empty, uniquely-named file (not a
// directory) named prefix + <random> + suffix, and returns its path.
CreateTemp(ctx context.Context, prefix, suffix string) (string, error)
// Exec runs spec.Command as a shell command and returns its result.
Exec(ctx context.Context, spec ExecSpec) (*ExecResult, error)
}
ExecutionEnv is the filesystem/shell seam the built-in tools run over. Implementations MUST be pointer types: the mutation queue keys on instance identity. All methods are ctx-first so remote/sandbox adapters can honor cancellation.
type FileInfo ¶
type FileInfo struct {
// Kind is the entry's type.
Kind FileKind
// Size is the entry's size in bytes, as reported by lstat (i.e. the
// symlink's own size, not its target's, when Kind is FileKindSymlink).
Size int64
}
FileInfo describes a filesystem entry as reported by ExecutionEnv.FileInfo.
type FileKind ¶
type FileKind string
FileKind identifies what kind of filesystem entry FileInfo describes.
type OSEnv ¶
type OSEnv struct {
// contains filtered or unexported fields
}
OSEnv is the ExecutionEnv implementation backed by the local OS process: real files, real subprocesses. It is the default environment built-in tools run over outside of remote/sandbox adapters.
func NewOSEnv ¶
NewOSEnv creates an OSEnv rooted at cwd. cwd is resolved to an absolute path and must name an existing directory.
func (*OSEnv) AbsolutePath ¶
AbsolutePath resolves path against e.Cwd() (if path is relative) and cleans the result. It does not touch the filesystem, so it succeeds even for paths that don't exist.
func (*OSEnv) AppendFile ¶
AppendFile appends data to path, creating the file (and its parent directories) if it doesn't already exist.
func (*OSEnv) CanonicalPath ¶
CanonicalPath resolves path (via AbsolutePath) and then resolves any symlinks in it. path must exist.
func (*OSEnv) CreateTemp ¶
CreateTemp creates a new, empty, uniquely-named file named prefix + <random> + suffix in the system temp directory, and returns its path.
func (*OSEnv) Exec ¶
Exec runs spec.Command via `bash -c` in a new process group. Timeout and ctx cancellation both kill the process group and reap the process (via cmd.Wait, run on a background goroutine so the kill path never blocks on it) before returning, so callers never leak a goroutine or a zombie.
func (*OSEnv) Exists ¶
Exists reports whether path exists (via lstat, so a broken symlink still reports true).
func (*OSEnv) FileInfo ¶
FileInfo stats path (without following a trailing symlink) and returns its kind and size.
type ReadImageProcessor ¶
type ReadImageProcessor func(ctx context.Context, data []byte, mimeType string, autoResize bool) (ReadImageProcessorResult, error)
ReadImageProcessor is an optional hook the read tool (a later task) can be configured with to resize/re-encode image bytes before they're embedded in a tool result - e.g. downscaling oversized images or converting a supported-but-not-directly-embeddable format. This release defines the seam type only; no default implementation ships, and a read tool without one configured embeds sniffed bytes as-is.
type ReadImageProcessorResult ¶
type ReadImageProcessorResult struct {
// OK reports whether processing succeeded. When false, Data and
// MimeType are unset and Message explains what happened.
OK bool
// Data is the processed image bytes. Set only when OK is true.
Data []byte
// MimeType is the MIME type of Data. Set only when OK is true.
MimeType string
// Hints are optional human-readable notes about the processing (e.g.
// "resized from 4032x3024 to 1024x768") to surface alongside the image.
Hints []string
// Message explains why processing failed. Set only when OK is false.
Message string
}
ReadImageProcessorResult is the outcome of a ReadImageProcessor call: the (possibly resized/re-encoded) image bytes to embed on success, or a human-readable Message to surface in place of the image on failure.
type ReadToolOptions ¶
type ReadToolOptions struct {
// Env is the filesystem seam the tool reads through.
Env ExecutionEnv
// AutoResizeImages controls whether an injected ImageProcessor should
// resize images. nil selects true (upstream's `autoResizeImages ?? true`).
// Only consulted when ImageProcessor is non-nil.
AutoResizeImages *bool
// ImageProcessor optionally resizes/re-encodes sniffed image bytes
// before they're embedded in the result. Nil embeds sniffed bytes as-is,
// except BMP - which without a processor can't be embedded and is
// reported as omitted instead (see readImageToolResult).
ImageProcessor ReadImageProcessor
}
ReadToolOptions configures NewReadTool.
type ShellCapture ¶
type ShellCapture struct {
// Output, Truncation, FullOutputPath mirror ShellProgress's fields as
// of process exit, cancellation, or timeout.
Output string
Truncation TruncationResult
FullOutputPath string // "" unless the output overflowed and was spilled
// LastLineBytes is the byte length of the stream's final (possibly
// still-open, i.e. not newline-terminated) line. Unlike Output, it is
// tracked against the WHOLE stream and is never affected by tail-buffer
// trimming.
LastLineBytes int
// ExitCode is the command's exit code. When Cancelled or TimedOut is
// true, treat those flags - not ExitCode - as authoritative that the
// process didn't complete on its own; see ExecResult.ExitCode's doc
// comment.
ExitCode int
Cancelled bool
TimedOut bool
}
ShellCapture is the final result of a completed ExecuteShellWithCapture call.
func ExecuteShellWithCapture ¶
func ExecuteShellWithCapture(ctx context.Context, env ExecutionEnv, command string, opts ShellCaptureOptions) (*ShellCapture, error)
ExecuteShellWithCapture runs command via env.Exec, capturing its merged stdout+stderr into the tail-buffer/spill-file scheme documented in this file's package comment. It returns a non-nil error only when env.Exec itself fails to run the command (e.g. bash isn't available) or when writing to the spill file fails - never for a non-zero exit code, a timeout, or a cancellation, all of which are reported on the returned ShellCapture instead.
Deviation from upstream: after env.Exec returns, upstream re-checks `progress.truncation.truncated && !fullOutputRequested` and spills as a safety net before building the final result. That check is unreachable here and is deliberately omitted: env.Exec (env_unix.go's OSEnv.Exec) only returns once cmd.Wait has reaped the process, which - because stdout/stderr are plain io.Writers, not manually-drained pipes - only happens once every OnChunk call has already completed. observeChunk's overflow check runs on every chunk against the same monotonically non-decreasing totals the final snapshot would use, so if the stream ever overflows, spilling has already started by the time env.Exec returns.
ctx is threaded into every observeChunk call (including its env.CreateTemp/env.AppendFile spill I/O), per this task's brief. A consequence: if ctx is cancelled or times out while a spill write is in-flight, that write can fail with ctx's error, which - like any other spillErr - aborts the whole call with a non-nil error instead of returning a partial ShellCapture. In practice this only matters for commands whose output is large enough to have already overflowed (triggering a spill) at the moment of cancellation/timeout; the small partial output a cancelled/timed-out command typically produces stays well under the spill threshold and is unaffected.
type ShellCaptureOptions ¶
type ShellCaptureOptions struct {
// Dir, Env, InheritEnv, Timeout are forwarded to ExecutionEnv.Exec as
// the corresponding ExecSpec fields.
Dir string
Env map[string]string
InheritEnv bool
Timeout time.Duration
// OnChunk, if set, is called with each sanitized output chunk as it
// arrives (calls are serialized - see ExecSpec.OnChunk's doc comment,
// which this wraps) along with a progress func that snapshots the
// capture's current state. progress is safe to call synchronously from
// within OnChunk, or stashed and called later, including after
// ExecuteShellWithCapture has returned.
OnChunk func(chunk []byte, progress func() ShellProgress)
}
ShellCaptureOptions configures ExecuteShellWithCapture.
type ShellProgress ¶
type ShellProgress struct {
// Output is the best-effort output seen so far: the raw tail buffer,
// or - once the whole stream has exceeded DefaultMaxBytes or
// DefaultMaxLines - that buffer run through TruncateTail.
Output string
// Truncation describes Output's truncation status. Unlike a bare
// TruncateTail(Output) call, TotalLines/TotalBytes here reflect the
// WHOLE stream seen so far, not just the (possibly trimmed) tail
// buffer.
Truncation TruncationResult
// FullOutputPath is the spill file's path once the stream has
// overflowed and the file has been created, "" until then.
FullOutputPath string
}
ShellProgress is a point-in-time snapshot of an in-progress ExecuteShellWithCapture call, obtained via the progress func passed to ShellCaptureOptions.OnChunk. It remains safe to call after ExecuteShellWithCapture has returned - it reads from the same mutex-guarded state the final ShellCapture is built from, so a caller may stash the progress func and invoke it later.
type TruncationOptions ¶
type TruncationOptions struct {
// MaxLines is the maximum number of lines. Zero selects DefaultMaxLines.
MaxLines int
// MaxBytes is the maximum number of bytes. Zero selects DefaultMaxBytes.
MaxBytes int
}
TruncationOptions configures TruncateHead and TruncateTail. A zero value for either field selects the corresponding Default* constant.
type TruncationResult ¶
type TruncationResult struct {
// Content is the truncated content.
Content string
// Truncated reports whether truncation occurred.
Truncated bool
// TruncatedBy is which limit was hit: "lines", "bytes", or "" if Truncated is false.
TruncatedBy string
// TotalLines is the total number of lines in the original content.
TotalLines int
// TotalBytes is the total number of bytes in the original content.
TotalBytes int
// OutputLines is the number of complete lines in the truncated output.
OutputLines int
// OutputBytes is the number of bytes in the truncated output.
OutputBytes int
// LastLinePartial reports whether the last line was partially truncated.
// Only ever set by TruncateTail's single-oversized-line edge case.
LastLinePartial bool
// FirstLineExceedsLimit reports whether the first line alone exceeded MaxBytes.
// Only ever set by TruncateHead.
FirstLineExceedsLimit bool
// MaxLines is the max lines limit that was applied.
MaxLines int
// MaxBytes is the max bytes limit that was applied.
MaxBytes int
}
TruncationResult describes the outcome of a TruncateHead or TruncateTail call.
func TruncateHead ¶
func TruncateHead(content string, opts TruncationOptions) TruncationResult
TruncateHead truncates content from the head (keeps the first N lines/bytes). Suitable for file reads where you want to see the beginning.
Never returns partial lines. If the first line exceeds MaxBytes, returns empty content with FirstLineExceedsLimit set.
func TruncateTail ¶
func TruncateTail(content string, opts TruncationOptions) TruncationResult
TruncateTail truncates content from the tail (keeps the last N lines/bytes). Suitable for bash output where you want to see the end (errors, final results).
May return a partial last line if the last line of the original content exceeds the byte limit.
type WriteToolOptions ¶
type WriteToolOptions struct {
// Env is the filesystem seam the tool writes through.
Env ExecutionEnv
}
WriteToolOptions configures NewWriteTool.