Documentation
¶
Index ¶
- Variables
- func ExtractReadableMarkdownFromURL(ctx context.Context, rawURL string) (string, error)
- type Attachment
- type AttachmentContentBlockKind
- type AttachmentContentBlockMode
- type AttachmentKind
- type ContentBlock
- type ContentBlockOption
- type DirectoryAttachmentsResult
- type FileRef
- type GenericRef
- type ImageRef
- type URLRef
Constants ¶
This section is empty.
Variables ¶
var ( ErrNonTextContentBlock = errors.New("content block is not of kind - text") ErrUnreadableFile = errors.New("unreadable file") ErrExistingContentBlock = errors.New("content block already exists") ErrAttachmentModifiedSinceSnapshot = errors.New("attachment modified since snapshot") )
var ( // ErrResponseTooLarge is used internally by fetchURLBytes to signal // that the remote resource exceeded the configured maxBytes limit. ErrResponseTooLarge = errors.New("remote resource larger than the configured limit") // ErrPageTooLarge is returned by ExtractReadableMarkdownFromURL when // the HTML page exceeds maxPageContentBytes. ErrPageTooLarge = errors.New("page content too large") // ErrNoContentExtracted is returned when the extractor produces no // readable content at all (e.g., empty body, or extraction failure). ErrNoContentExtracted = errors.New("no readable content extracted") )
Functions ¶
func ExtractReadableMarkdownFromURL ¶
ExtractReadableMarkdownFromURL downloads a web page, runs it through go-trafilatura to obtain the main readable content, converts that to Markdown, and returns the Markdown text.
It enforces a hard size limit (maxPageContentBytes). If the remote server is larger than that, ErrPageTooLarge is returned. Only the HTML of the page itself is downloaded; sub-resources (images, scripts, etc.) are not fetched.
This function is conservative: it only attempts extraction on content types that look like HTML or generic text.
Types ¶
type Attachment ¶
type Attachment struct {
Kind AttachmentKind `json:"kind"`
Label string `json:"label"`
// Mode selected for this attachment for the current turn.
// Frontend picks smart defaults; backend can override missing values.
Mode AttachmentContentBlockMode `json:"mode,omitempty"`
// Optional: allowed modes for this attachment (primarily for UI).
AvailableContentBlockModes []AttachmentContentBlockMode `json:"availableContentBlockModes,omitempty"`
// Exactly one field below should be non-nil.
FileRef *FileRef `json:"fileRef,omitempty"`
ImageRef *ImageRef `json:"imageRef,omitempty"`
URLRef *URLRef `json:"urlRef,omitempty"`
GenericRef *GenericRef `json:"genericRef,omitempty"`
ContentBlock *ContentBlock `json:"contentBlock,omitempty"`
}
Attachment is a lightweight reference to external context (files, docs, images, etc.).
func BuildAttachmentForFile ¶
func BuildAttachmentForFile(pathInfo *fileutil.PathInfo) (*Attachment, error)
BuildAttachmentForFile builds an Attachment for a local filesystem path. It inspects the MIME type / extension and chooses an appropriate AttachmentKind, default Mode, and AvailableContentBlockModes. The returned attachment is fully populated via PopulateRef(). Note that this builds a fresh attachment, i.e both original ref and current are populated here.
func BuildAttachmentForURL ¶
func BuildAttachmentForURL(rawURL string) (*Attachment, error)
BuildAttachmentForURL builds an Attachment for a remote URL. It uses a timeout context to peek and infer type and then give proper options.
func BuildAttachmentForURLWithContext ¶
func BuildAttachmentForURLWithContext(ctx context.Context, rawURL string) (*Attachment, error)
BuildAttachmentForURLWithContext builds an Attachment for a remote URL.
Mode detection strategy:
- Best effort Content-Type detection via HEAD -> Range GET sniff.
- Fallback to extension-based detection.
- Ultimate fallback: LinkOnly (do not error out due to detection failures).
Note: It can still return an error for invalid/empty/non-absolute URLs because PopulateRef enforces validity.
func (*Attachment) BuildContentBlock ¶
func (att *Attachment) BuildContentBlock(ctx context.Context, opts ...ContentBlockOption, ) (*ContentBlock, error)
BuildContentBlock function builds and returns a content block for an attachment. It does NOT attach the content block to the attachment.
func (*Attachment) GetTextBlockWithDisplayNameOnly ¶
func (att *Attachment) GetTextBlockWithDisplayNameOnly(suffix string) (*ContentBlock, error)
func (*Attachment) PopulateRef ¶
func (att *Attachment) PopulateRef(replaceOrig bool) error
type AttachmentContentBlockKind ¶
type AttachmentContentBlockKind string
const ( ContentBlockText AttachmentContentBlockKind = "text" ContentBlockImage AttachmentContentBlockKind = "image" ContentBlockFile AttachmentContentBlockKind = "file" )
type AttachmentContentBlockMode ¶
type AttachmentContentBlockMode string
AttachmentContentBlockMode describes how a given attachment should be used for the current turn.
const ( AttachmentContentBlockModeText AttachmentContentBlockMode = "text" // "Text content" AttachmentContentBlockModeFile AttachmentContentBlockMode = "file" // "File (original format)" AttachmentContentBlockModeImage AttachmentContentBlockMode = "image" // Image rendering AttachmentContentBlockModePageContent AttachmentContentBlockMode = "page" // "Page content" for HTML/URLs AttachmentContentBlockModeTextLink AttachmentContentBlockMode = "textlink" // "Link as text block" – no fetch AttachmentContentBlockModeImageURL AttachmentContentBlockMode = "imageurl" AttachmentContentBlockModeFileURL AttachmentContentBlockMode = "fileurl" AttachmentContentBlockModeNotReadable AttachmentContentBlockMode = "not-readable" // Binary/unknown – cannot process AttachmentContentBlockModePRDiff AttachmentContentBlockMode = "pr-diff" AttachmentContentBlockModePRPage AttachmentContentBlockMode = "pr-page" AttachmentContentBlockModeCommitDiff AttachmentContentBlockMode = "commit-diff" AttachmentContentBlockModeCommitPage AttachmentContentBlockMode = "commit-page" )
type AttachmentKind ¶
type AttachmentKind string
AttachmentKind enumerates contextual attachment categories that can be associated with messages sent to the inference layer.
const ( AttachmentFile AttachmentKind = "file" AttachmentImage AttachmentKind = "image" AttachmentURL AttachmentKind = "url" AttachmentDocIndex AttachmentKind = "docIndex" AttachmentPR AttachmentKind = "pr" AttachmentCommit AttachmentKind = "commit" )
type ContentBlock ¶
type ContentBlock struct {
Kind AttachmentContentBlockKind `json:"kind"`
// For Kind == text: Text, MIMEType and FileName are populated.
Text *string `json:"text,omitempty"`
MIMEType *string `json:"mimeType,omitempty"`
FileName *string `json:"fileName,omitempty"`
// For Kind == image or file:
// - either Base64Data, MIMEType and FileName are populated (local/file-url fetched),
// - or URL (+optional MIMEType/FileName) is populated for URL-based attachments.
Base64Data *string `json:"base64Data,omitempty"`
URL *string `json:"url,omitempty"`
}
ContentBlock represents a provider-agnostic chunk of content derived from an Attachment. Providers (OpenAI, Anthropic, etc.) adapt this into their own message/part formats.
func BuildContentBlocks ¶
func BuildContentBlocks(ctx context.Context, atts []Attachment, opts ...ContentBlockOption) ([]ContentBlock, error)
BuildContentBlocks converts high-level attachments (file paths, URLs, etc.) into provider-agnostic content blocks that can then be adapted for each LLM.
type ContentBlockOption ¶
type ContentBlockOption func(*buildContentBlockOptions)
func WithForceFetchContentBlock ¶
func WithForceFetchContentBlock(forceFetch bool) ContentBlockOption
WithForceFetchContentBlock. Default False. False = Dont fetch the content block if already present in attachment. True = Override content block even if present.
func WithOnlyTextKindContentBlock ¶
func WithOnlyTextKindContentBlock(textOnly bool) ContentBlockOption
WithOnlyTextKindContentBlock. Default False. False = Build content block of any kind. True = Return content block only if it is of kind text, else error.
func WithOverrideOriginalContentBlock ¶
func WithOverrideOriginalContentBlock(override bool) ContentBlockOption
WithOverrideOriginalContentBlock. Default false. False = If this is an attachment from a previous turn whose underlying ref (file path, size, mod time, etc.) has changed, do not re-send the possibly mismatched content.
type DirectoryAttachmentsResult ¶
type DirectoryAttachmentsResult struct {
DirPath string `json:"dirPath"`
Attachments []Attachment `json:"attachments"` // included attachments (flattened)
OverflowDirs []fileutil.DirectoryOverflowInfo `json:"overflowDirs"` // directories not fully included
MaxFiles int `json:"maxFiles"` // max number of files returned (after clamping)
TotalSize int64 `json:"totalSize"` // sum of Files[i].Size
HasMore bool `json:"hasMore"` // true if not all content included
}
type FileRef ¶
type FileRef struct {
fileutil.PathInfo
// Snapshot of the original file state when it was first attached.
// This lets us detect if the underlying file changed between turns.
OrigPath string `json:"origPath"`
OrigSize int64 `json:"origSize"`
OrigModTime time.Time `json:"origModTime"`
}
FileRef carries metadata for file attachments.
func (*FileRef) BuildContentBlock ¶
func (ref *FileRef) BuildContentBlock( attachmentContentBlockMode AttachmentContentBlockMode, onlyIfTextKind bool, ) (*ContentBlock, error)
func (*FileRef) IsModified ¶
IsModified reports whether the current file state differs from the original snapshot captured when the attachment was first used.
func (*FileRef) PopulateRef ¶
type GenericRef ¶
GenericRef preserves legacy handle-style references (PRs, doc indexes, etc.).
func (*GenericRef) IsModified ¶
func (ref *GenericRef) IsModified() bool
func (*GenericRef) PopulateRef ¶
func (ref *GenericRef) PopulateRef(replaceOrig bool) error
type ImageRef ¶
type ImageRef struct {
fileutil.ImageInfo
// Original snapshot (for change detection across turns).
OrigPath string `json:"origPath"`
OrigSize int64 `json:"origSize"`
OrigModTime time.Time `json:"origModTime"`
}
ImageRef carries metadata for image attachments.
func (*ImageRef) BuildContentBlock ¶
func (ref *ImageRef) BuildContentBlock() (*ContentBlock, error)
func (*ImageRef) IsModified ¶
func (*ImageRef) PopulateRef ¶
type URLRef ¶
type URLRef struct {
URL string `json:"url"`
Normalized string `json:"normalized,omitempty"`
OrigNormalized string `json:"origNormalized"`
}
URLRef carries metadata for URL-based attachments.
URL: the raw user-provided URL (after trimming whitespace). Normalized: a canonicalized string representation used internally. OrigNormalized: snapshot of the original Normalized value so we can
detect whether a URL has been modified in-place.
func (*URLRef) BuildContentBlock ¶
func (ref *URLRef) BuildContentBlock( ctx context.Context, attachmentContentBlockMode AttachmentContentBlockMode, onlyIfTextKind bool, ) (*ContentBlock, error)
BuildContentBlock builds a ContentBlock representation for a URL-based attachment, depending on the desired AttachmentContentBlockMode.
The behaviour is:
- AttachmentContentBlockModeTextLink: always returns a simple text link.
- AttachmentContentBlockModeImage: tries to fetch the URL as an image and return a ContentBlockImage, else falls back to a simple link.
- AttachmentContentBlockModePageContent / AttachmentContentBlockModeText: runs the URL through the "page pipeline" (HTML/text/PDF/image handling) and returns a text or file/image block as appropriate.
- AttachmentContentBlockModeFile: fetches the raw bytes and returns a ContentBlockFile, else falls back to a link.
- Any other modes (PR diff/page, commit diff/page, not readable, etc.): safest fallback is a link-only block.
This function assumes that Attachment, AttachmentContentBlockMode, ContentBlock and ContentBlock* constants are defined elsewhere in this package.
func (*URLRef) IsModified ¶
IsModified reports whether the URL has been modified from its original normalized form. This is useful for detecting in-place edits.
func (*URLRef) PopulateRef ¶
PopulateRef validates and normalizes the URL stored in the URLRef. It must be called before the URLRef is used.
It ensures:
- The URL is non-empty.
- The URL parses successfully.
- The URL is absolute (has a scheme/host).
- Normalized and OrigNormalized are populated.