api

Documentation

Index

• type ErrorResponse
• type ResponseJSON
• type Router
  • func NewRouter(cfg config.Config, logger l.Logger, registry providers.ProviderRegistry, ...) Router
• type RouterImpl
  • func (router *RouterImpl) ChatCompletionsHandler(c *gin.Context)
  • func (router *RouterImpl) HealthcheckHandler(c *gin.Context)
  • func (router *RouterImpl) ListModelsHandler(c *gin.Context)
  • func (router *RouterImpl) ListToolsHandler(c *gin.Context)
  • func (router *RouterImpl) NotFoundHandler(c *gin.Context)
  • func (router *RouterImpl) ProxyHandler(c *gin.Context)

Types

type ErrorResponse

type ErrorResponse struct {
	Error string `json:"error"`
}

type ResponseJSON

type ResponseJSON struct {
	Message string `json:"message"`
}

type Router

type Router interface {
	ListModelsHandler(c *gin.Context)
	ChatCompletionsHandler(c *gin.Context)
	ListToolsHandler(c *gin.Context)
	ProxyHandler(c *gin.Context)
	HealthcheckHandler(c *gin.Context)
	NotFoundHandler(c *gin.Context)
}

func NewRouter

func NewRouter(
	cfg config.Config,
	logger l.Logger,
	registry providers.ProviderRegistry,
	client providers.Client,
	mcpClient mcp.MCPClientInterface,
) Router

type RouterImpl

type RouterImpl struct {
	// contains filtered or unexported fields
}

func (*RouterImpl) ChatCompletionsHandler

func (router *RouterImpl) ChatCompletionsHandler(c *gin.Context)

ChatCompletionsHandler implements an OpenAI-compatible API endpoint that generates text completions in the standard OpenAI format.

Regular response format:

{
  "choices": [
    {
      "finish_reason": "stop",
      "message": {
        "content": "Hello, how can I help you today?",
        "role": "assistant"
      }
    }
  ],
  "created": 1742165657,
  "id": "chatcmpl-118",
  "model": "deepseek-r1:1.5b",
  "object": "chat.completion",
  "usage": {
    "completion_tokens": 139,
    "prompt_tokens": 10,
    "total_tokens": 149
  }
}

Streaming response format:

{
  "choices": [
    {
      "index": 0,
      "finish_reason": "stop",
      "delta": {
        "content": "Hello",
        "role": "assistant"
      }
    }
  ],
  "created": 1742165657,
  "id": "chatcmpl-118",
  "model": "deepseek-r1:1.5b",
  "object": "chat.completion.chunk",
  "usage": {
    "completion_tokens": 139,
    "prompt_tokens": 10,
    "total_tokens": 149
  }
}

It returns token completions as chat in the standard OpenAI format, allowing applications built for OpenAI's API to work seamlessly with the Inference Gateway's multi-provider architecture.

func (*RouterImpl) HealthcheckHandler

func (router *RouterImpl) HealthcheckHandler(c *gin.Context)

func (*RouterImpl) ListModelsHandler

func (router *RouterImpl) ListModelsHandler(c *gin.Context)

ListModelsHandler implements an OpenAI-compatible API endpoint that returns model information in the standard OpenAI format.

This handler supports the OpenAI GET /v1/models endpoint specification: https://platform.openai.com/docs/api-reference/models/list

Parameters:

• provider (query): Optional. When specified, returns models from only that provider. If not specified, returns models from all configured providers.

Response format:

{
  "object": "list",
  "data": [
   {
      "id": "model-id",
      "object": "model",
      "created": 1686935002,
      "owned_by": "provider-name",
      "served_by": "provider-name"
   },
   ...
  ]
}

This endpoint allows applications built for OpenAI's API to work seamlessly with the Inference Gateway's multi-provider architecture.

func (*RouterImpl) ListToolsHandler

func (router *RouterImpl) ListToolsHandler(c *gin.Context)

ListToolsHandler implements an endpoint that returns available MCP tools when EXPOSE_MCP environment variable is enabled.

Response format when MCP is exposed:

{
  "object": "list",
  "data": [
    {
      "name": "read_file",
      "description": "Read the contents of a file",
      "server": "filesystem-server",
      "input_schema": {...}
    },
    ...
  ]
}

Response when MCP is not exposed:

{
  "error": "MCP tools endpoint is not exposed"
}

func (*RouterImpl) NotFoundHandler

func (router *RouterImpl) NotFoundHandler(c *gin.Context)

func (*RouterImpl) ProxyHandler

func (router *RouterImpl) ProxyHandler(c *gin.Context)