cgpt

README

cgpt

cgpt is a command-line tool for interacting with Large Language Models (LLMs) using various backends.

Features

• Supports multiple backends: Anthropic, OpenAI, Ollama, and Google AI
• Interactive mode for continuous conversations
• Streaming output
• History management
• Configurable via YAML file and environment variables
• Vim plugin for easy integration

Installation

Using Homebrew

brew tap tmc/tap
brew install cgpt

From Source

go install github.com/tmc/cgpt/cmd/cgpt@latest

Usage

cgpt [flags]

Flags

• -b, --backend string: The backend to use (default "anthropic")
• -m, --model string: The model to use (default "claude-3-5-sonnet-20240620")
• -i, --input string: Direct string input (overrides -f)
• -f, --file string: Input file path. Use '-' for stdin (default "-")
• -c, --continuous: Run in continuous mode (interactive)
• -s, --system-prompt string: System prompt to use
• -p, --prefill string: Prefill the assistant's response
• -I, --history-load string: File to read completion history from
• -O, --history-save string: File to store completion history in
• --config string: Path to the configuration file (default "config.yaml")
• -v, --verbose: Verbose output
• --debug: Debug output
• -n, --completions int: Number of completions (when running non-interactively with history)
• -t, --max-tokens int: Maximum tokens to generate (default 8000)
• --completion-timeout duration: Maximum time to wait for a response (default 2m0s)

Configuration

cgpt can be configured using a YAML file. By default, it looks for config.yaml in the current directory. You can specify a different configuration file using the --config flag.

Example config.yaml:

backend: "anthropic"
modelName: "claude-3-5-sonnet-20240620"
stream: true
maxTokens: 2048
systemPrompt: "You are a helpful assistant."

Environment Variables

• CGPT_OPENAI_API_KEY: OpenAI API key
• CGPT_ANTHROPIC_API_KEY: Anthropic API key
• CGPT_GOOGLE_API_KEY: Google AI API key

Vim Plugin

cgpt includes a Vim plugin for easy integration. To use it, copy the vim/plugin/cgpt.vim file to your Vim plugin directory.

Vim Plugin Usage

1. Visually select the text you want to process with cgpt.
2. Press cg or use the :CgptRun command to run cgpt on the selected text.
3. The output will be appended after the visual selection.

Vim Plugin Configuration

• g:cgpt_backend: Set the backend for cgpt (default: 'anthropic')
• g:cgpt_model: Set the model for cgpt (default: 'claude-3-5-sonnet-20240620')
• g:cgpt_system_prompt: Set the system prompt for cgpt
• g:cgpt_config_file: Set the path to the cgpt configuration file
• g:cgpt_include_filetype: Include the current filetype in the prompt (default: 0)

Examples

# Simple query
echo "Explain quantum computing" | cgpt

# Interactive mode
cgpt -c

# Use a specific backend and model
cgpt -b openai -m gpt-4 -i "Translate 'Hello, world!' to French"

# Load and save history
cgpt -I input_history.yaml -O output_history.yaml -i "Continue the conversation"

License

This project is licensed under the ISC License. See the LICENSE file for details.

This README provides an overview of the cgpt tool, including its features, installation instructions, usage examples, configuration options, and information about the Vim plugin. It also includes details about the supported backends and environment variables for API keys.

Happy hacking! 🚀

Documentation

Index

• type ChatCompletionPayload
• type CompletionService
  • func NewCompletionService(cfg *Config) (*CompletionService, error)
  • func (s *CompletionService) PerformCompletionStreaming(ctx context.Context, payload *ChatCompletionPayload, showSpinner bool) (<-chan string, error)
  • func (s *CompletionService) Run(ctx context.Context, runCfg RunConfig) error
  • func (s *CompletionService) SetNextCompletionPrefill(content string)
• type Config
  • func LoadConfig(path string, flagSet *pflag.FlagSet) (*Config, error)
  • func (cfg *Config) SetDefaults(defaultBackend string) *Config
• type RunConfig

Types

type ChatCompletionPayload

type ChatCompletionPayload struct {
	Model    string `json:"model"`
	Messages []llms.MessageContent
	Stream   bool `json:"stream,omitempty"`
}

type CompletionService

type CompletionService struct {
	// contains filtered or unexported fields
}

func NewCompletionService

func NewCompletionService(cfg *Config) (*CompletionService, error)

NewCompletionService creates a new CompletionService with the given configuration.

func (*CompletionService) PerformCompletionStreaming

func (s *CompletionService) PerformCompletionStreaming(ctx context.Context, payload *ChatCompletionPayload, showSpinner bool) (<-chan string, error)

func (*CompletionService) Run

func (s *CompletionService) Run(ctx context.Context, runCfg RunConfig) error

Run runs the completion service with the given configuration.

func (*CompletionService) SetNextCompletionPrefill

func (s *CompletionService) SetNextCompletionPrefill(content string)

type Config

type Config struct {
	Backend   string `yaml:"backend"`
	Model     string `yaml:"modelName"`
	Stream    bool   `yaml:"stream"`
	MaxTokens int    `yaml:"maxTokens"`

	SystemPrompt string             `yaml:"systemPrompt"`
	LogitBias    map[string]float64 `yaml:"logitBias"`

	CompletionTimeout time.Duration `yaml:"completionTimeout"`

	Debug bool `yaml:"debug"`

	// API keys
	OpenAIAPIKey    string `yaml:"openaiAPIKey"`
	AnthropicAPIKey string `yaml:"anthropicAPIKey"`
	GoogleAPIKey    string `yaml:"googleAPIKey"`
}

Config is the configuration for cgpt.

func LoadConfig

func LoadConfig(path string, flagSet *pflag.FlagSet) (*Config, error)

LoadConfig loads the config file from the given path. if the file is not found, it returns the default config.

func (*Config) SetDefaults

func (cfg *Config) SetDefaults(defaultBackend string) *Config

SetDefaults sets the default values for the config.

type RunConfig

type RunConfig struct {
	// InputString is the input text to complete. If present, this will be used instead of reading from a file.
	// This will only be used for the first completion when either running in continuous mode or when running multiple completions.
	InputString string

	// InputFile is the file to read input from. Use "-" for stdin.
	InputFile string

	// Continuous will run the completion API in a loop, using the previous output as the input for the next request.
	Continuous bool

	// Stream will stream results as they come in.
	Stream bool

	// Prefill is the message to prefill the assistant with.
	// This will only be used for the first completion if more than one completion is run.
	Prefill string

	// HistoryIn is the file to read cgpt history from.
	HistoryIn string
	// HistoryOut is the file to store cgpt history in.
	HistoryOut string

	// NCompletions is the number of completions to complete in a history-enabled context.
	NCompletions int

	// Verbose will enable verbose output.
	Verbose bool

	// DebugMode will enable debug output.
	DebugMode bool

	// ReadlineHistoryFile is the file to store readline history in.
	ReadlineHistoryFile string

	// MaximumTimeout is the maximum time to wait for a response.
	MaximumTimeout time.Duration
}

RunConfig is the configuration for the Run method.