api

Documentation

Index

• type ErrorResponse
• type ResponseJSON
• type Router
  • func NewRouter(cfg config.Config, logger l.Logger, registry providers.ProviderRegistry, ...) Router
• type RouterImpl
  • func (router *RouterImpl) ChatCompletionsHandler(c *gin.Context)
  • func (router *RouterImpl) HealthcheckHandler(c *gin.Context)
  • func (router *RouterImpl) ListAgentsHandler(c *gin.Context)
  • func (router *RouterImpl) ListModelsHandler(c *gin.Context)
  • func (router *RouterImpl) ListToolsHandler(c *gin.Context)
  • func (router *RouterImpl) NotFoundHandler(c *gin.Context)
  • func (router *RouterImpl) ProxyHandler(c *gin.Context)

Types

type ErrorResponse

type ErrorResponse struct {
	Error string `json:"error"`
}

type ResponseJSON

type ResponseJSON struct {
	Message string `json:"message"`
}

type Router

type Router interface {
	ListModelsHandler(c *gin.Context)
	ChatCompletionsHandler(c *gin.Context)
	ListToolsHandler(c *gin.Context)
	ListAgentsHandler(c *gin.Context)
	ProxyHandler(c *gin.Context)
	HealthcheckHandler(c *gin.Context)
	NotFoundHandler(c *gin.Context)
}

func NewRouter

func NewRouter(
	cfg config.Config,
	logger l.Logger,
	registry providers.ProviderRegistry,
	client providers.Client,
	mcpClient mcp.MCPClientInterface,
	a2aClient a2a.A2AClientInterface,
) Router

type RouterImpl

type RouterImpl struct {
	// contains filtered or unexported fields
}

func (*RouterImpl) ChatCompletionsHandler

func (router *RouterImpl) ChatCompletionsHandler(c *gin.Context)

ChatCompletionsHandler implements an OpenAI-compatible API endpoint that generates text completions in the standard OpenAI format.

Regular response format:

{
  "choices": [
    {
      "finish_reason": "stop",
      "message": {
        "content": "Hello, how can I help you today?",
        "role": "assistant"
      }
    }
  ],
  "created": 1742165657,
  "id": "chatcmpl-118",
  "model": "deepseek-r1:1.5b",
  "object": "chat.completion",
  "usage": {
    "completion_tokens": 139,
    "prompt_tokens": 10,
    "total_tokens": 149
  }
}

Streaming response format:

{
  "choices": [
    {
      "index": 0,
      "finish_reason": "stop",
      "delta": {
        "content": "Hello",
        "role": "assistant"
      }
    }
  ],
  "created": 1742165657,
  "id": "chatcmpl-118",
  "model": "deepseek-r1:1.5b",
  "object": "chat.completion.chunk",
  "usage": {
    "completion_tokens": 139,
    "prompt_tokens": 10,
    "total_tokens": 149
  }
}

It returns token completions as chat in the standard OpenAI format, allowing applications built for OpenAI's API to work seamlessly with the Inference Gateway's multi-provider architecture.

func (*RouterImpl) HealthcheckHandler

func (router *RouterImpl) HealthcheckHandler(c *gin.Context)

func (*RouterImpl) ListAgentsHandler

func (router *RouterImpl) ListAgentsHandler(c *gin.Context)

ListAgentsHandler implements an endpoint that returns available A2A agents when A2A_EXPOSE environment variable is enabled.

This handler supports the Agent-to-Agent (A2A) protocol by exposing a list of connected agents along with their metadata such as name, description, and URL.

The endpoint follows the OpenAI-compatible API format pattern used throughout the gateway for consistency.

Request:

• Method: GET
• Path: /a2a/agents
• Authentication: Required (Bearer token)
• Query Parameters: None

Response format when A2A is exposed and agents are available:

{
  "object": "list",
  "data": [
    {
      "id": "https://agent1.example.com",
      "name": "Calculator Agent",
      "description": "An agent that can perform mathematical calculations",
      "url": "https://agent1.example.com"
    },
    {
      "id": "https://agent2.example.com",
      "name": "Weather Agent",
      "description": "An agent that provides weather information",
      "url": "https://agent2.example.com"
    }
  ]
}

Response when A2A is not exposed:

{
  "error": "A2A agents endpoint is not exposed. Set A2A_EXPOSE=true to enable."
}

Response when no agents are available:

{
  "object": "list",
  "data": []
}

Error Handling:

• Returns 403 Forbidden if A2A_EXPOSE is not enabled
• Returns empty list if A2A client is not initialized
• Continues processing other agents if individual agent card retrieval fails
• Logs errors for failed agent card retrievals but doesn't fail the entire request

The handler gracefully handles various states:

• A2A client is nil (returns empty list)
• A2A client is not initialized (returns empty list)
• Individual agent card retrieval failures (skips failed agents, continues with others)
• No agents configured (returns empty list)

Security:

• Requires authentication via Bearer token
• Only exposes agents when explicitly configured via A2A_EXPOSE=true
• Does not expose internal errors to clients

func (*RouterImpl) ListModelsHandler

func (router *RouterImpl) ListModelsHandler(c *gin.Context)

ListModelsHandler implements an OpenAI-compatible API endpoint that returns model information in the standard OpenAI format.

This handler supports the OpenAI GET /v1/models endpoint specification: https://platform.openai.com/docs/api-reference/models/list

Parameters:

• provider (query): Optional. When specified, returns models from only that provider. If not specified, returns models from all configured providers.

Response format:

{
  "object": "list",
  "data": [
   {
      "id": "model-id",
      "object": "model",
      "created": 1686935002,
      "owned_by": "provider-name",
      "served_by": "provider-name"
   },
   ...
  ]
}

This endpoint allows applications built for OpenAI's API to work seamlessly with the Inference Gateway's multi-provider architecture.

func (*RouterImpl) ListToolsHandler

func (router *RouterImpl) ListToolsHandler(c *gin.Context)

ListToolsHandler implements an endpoint that returns available MCP tools when EXPOSE_MCP environment variable is enabled.

Response format when MCP is exposed:

{
  "object": "list",
  "data": [
    {
      "name": "read_file",
      "description": "Read the contents of a file",
      "server": "filesystem-server",
      "input_schema": {...}
    },
    ...
  ]
}

Response when MCP is not exposed:

{
  "error": "MCP tools endpoint is not exposed"
}

func (*RouterImpl) NotFoundHandler

func (router *RouterImpl) NotFoundHandler(c *gin.Context)

func (*RouterImpl) ProxyHandler

func (router *RouterImpl) ProxyHandler(c *gin.Context)