patch

README

Patch Plugin

The Patch Plugin provides powerful JSON request modification capabilities using sonic for high-performance JSON processing. It allows you to automatically modify API requests based on model types, field values, or custom conditions.

Features

• High Performance: Uses ByteDance's sonic library for fast JSON parsing and manipulation
• Predefined Patches: Built-in patches for common scenarios (DeepSeek max_tokens limits, GPT-5 compatibility, etc.)
• User-Defined Patches: Flexible configuration system for custom patches
• Conditional Logic: Apply patches based on model types, field values, or complex conditions
• Multiple Operations: Set, delete, add, and limit operations on JSON fields
• Nested Field Support: Use dot notation to modify nested JSON structures
• Placeholder Support: Dynamic value replacement using {{field_name}} syntax

Predefined Patches

The plugin comes with several built-in patches:

1. DeepSeek Max Tokens Limit

• Purpose: Limits max_tokens to 16000 for DeepSeek models
• Condition: Model name contains "deepseek"
• Operation: Limits max_tokens field to maximum 16000

2. GPT-5 Max Tokens Conversion

• Purpose: Converts max_tokens to max_completion_tokens for GPT-5 models
• Condition: Model name contains "gpt-5" and max_tokens field exists
• Operation:
  • Sets max_completion_tokens to the value of max_tokens
  • Removes the max_tokens field

3. O1 Models Max Tokens Conversion

• Purpose: Converts max_tokens to max_completion_tokens for o1 models
• Condition: Model matches o1, o1-preview, or o1-mini
• Operation: Same as GPT-5 conversion

4. Claude Max Tokens Limit

• Purpose: Limits max_tokens to 8192 for Claude models
• Condition: Model name contains "claude"
• Operation: Limits max_tokens field to maximum 8192

5. Remove Unsupported Stream Options

• Purpose: Removes stream_options for older GPT models that don't support it
• Condition: Model matches older GPT patterns and stream_options exists
• Operation: Removes stream_options field

Configuration

Basic Usage

import (
    "github.com/labring/aiproxy/core/relay/plugin/patch"
)

// Create plugin - configuration is loaded from model config
plugin := patch.New()

Configuration

The patch plugin loads configuration from the model's plugin configuration in the database. The configuration should be stored in the model config's plugin field under the key "patch".

Example model config plugin configuration:

{
  "patch": {
    "enable": true,
    "user_patches": [
      {
        "name": "custom_temperature_limit",
        "description": "Limit temperature for specific models",
        "conditions": [
          {
            "key": "model", 
            "operator": "contains",
            "value": "gpt-4"
          }
        ],
        "operations": [
          {
            "op": "limit",
            "key": "temperature", 
            "value": 1.0
          }
        ]
      }
    ]
  }
}

Predefined Patches

The plugin comes with built-in predefined patches that are always enabled:

• DeepSeek max_tokens limit: Automatically limits max_tokens to 16000 for DeepSeek models
• GPT-5 compatibility: Converts max_tokens to max_completion_tokens for GPT-5 models
• O1 models compatibility: Same conversion for o1, o1-preview, and o1-mini models
• Claude max_tokens limit: Limits max_tokens to 8192 for Claude models
• Stream options cleanup: Removes unsupported stream_options for older GPT models

These predefined patches run automatically and cannot be disabled.

Condition Operators

• equals: Exact string match
• not_equals: Not equal to string
• contains: String contains substring
• not_contains: String does not contain substring
• regex: Regular expression match
• exists: Field exists (non-nil)
• not_exists: Field does not exist (nil)

Operation Types

• set: Set field to a specific value
• delete: Remove field from JSON
• add: Add field only if it doesn't exist
• limit: Limit numeric field to maximum value

Special Keys

• model: References the actual model name from meta
• original_model: References the original model name from meta
• Any other key: References JSON field (supports dot notation)

Placeholder Syntax

Use {{field_name}} to reference values from the JSON data:

{
    Op:    patch.OpSet,
    Key:   "max_completion_tokens",
    Value: "{{max_tokens}}", // Will be replaced with actual max_tokens value
}

Nested Field Access

Use dot notation to access nested fields:

{
    Key: "parameters.max_tokens",  // Accesses parameters.max_tokens
    // ...
}

Integration with Plugin System

import (
    "github.com/labring/aiproxy/core/relay/plugin"
    "github.com/labring/aiproxy/core/relay/plugin/patch"
)

// Create patch plugin
patchPlugin := patch.New()

// Wrap adaptor with plugin
adaptor = plugin.WrapperAdaptor(adaptor, patchPlugin)

Performance Considerations

• Uses sonic library for high-performance JSON processing
• Efficient condition evaluation with early termination
• Minimal memory allocation for unchanged requests
• Lazy evaluation of patches (only applied when conditions match)

Error Handling

• Graceful degradation: if patching fails, original request is preserved
• Detailed logging of patch failures
• Type-safe operations with proper error checking

Examples

Example 1: Model-specific Max Tokens

{
    "name": "anthropic_max_tokens",
    "description": "Set appropriate max_tokens for Anthropic models",
    "conditions": [
        {
            "key": "model",
            "operator": "contains",
            "value": "claude"
        }
    ],
    "operations": [
        {
            "op": "limit",
            "key": "max_tokens",
            "value": 4096
        }
    ]
}

Example 2: Add Default Parameters

{
    "name": "add_default_temperature",
    "description": "Add default temperature if not specified",
    "conditions": [
        {
            "key": "temperature",
            "operator": "not_exists",
            "value": ""
        }
    ],
    "operations": [
        {
            "op": "add",
            "key": "temperature",
            "value": 0.7
        }
    ]
}

Example 3: Complex Conditional Logic

{
    "name": "streaming_optimization",
    "description": "Optimize streaming for specific models",
    "conditions": [
        {
            "key": "stream",
            "operator": "equals",
            "value": "true"
        },
        {
            "key": "model",
            "operator": "regex",
            "value": "^gpt-4"
        }
    ],
    "operations": [
        {
            "op": "set",
            "key": "stream_options.include_usage",
            "value": true
        }
    ]
}

Documentation

Overview

Package patch provides configuration types and structures for the patch plugin.

Package patch provides high-performance JSON request patching functionality using sonic. It allows automatic modification of API requests based on conditions and rules.

Index

• Constants
• Variables
• func AddLazyPatch(meta *meta.Meta, patch PatchOperation)
• func ToFloat64(v any) (float64, error)
• type ConditionOperator
• type Config
• type LazyPatchData
• type LogicOperator
• type OperationType
• type PatchCondition
• type PatchFunction
• type PatchOperation
  • func GetLazyPatches(meta *meta.Meta) []PatchOperation
• type PatchRule
• type Plugin
  • func NewPatchPlugin() *Plugin
  • func (p *Plugin) ApplyPatches(bodyBytes []byte, meta *meta.Meta, config *Config) ([]byte, bool, error)
  • func (p *Plugin) ConvertRequest(meta *meta.Meta, store adaptor.Store, req *http.Request, ...) (adaptor.ConvertResult, error)

Constants

const PluginName = "patch"

Variables

var DefaultPredefinedPatches = []PatchRule{
	{
		Name:           "deepseek_max_tokens_limit",
		Description:    "Limit max_tokens to 16000 for DeepSeek models",
		ConditionLogic: LogicOr,
		Conditions: []PatchCondition{
			{
				Key:      "model",
				Operator: OperatorContains,
				Value:    "deepseek-v3",
			},
		},
		Operations: []PatchOperation{
			{
				Op:    OpLimit,
				Key:   "max_tokens",
				Value: 16384,
			},
		},
	},
	{
		Name:           "deepseek_3.1_max_tokens_limit",
		Description:    "Limit max_tokens to 16000 for DeepSeek models",
		ConditionLogic: LogicOr,
		Conditions: []PatchCondition{
			{
				Key:      "model",
				Operator: OperatorContains,
				Value:    "deepseek-v3.1",
			},
			{
				Key:      "model",
				Operator: OperatorContains,
				Value:    "deepseek-chat",
			},
		},
		Operations: []PatchOperation{
			{
				Op:    OpLimit,
				Key:   "max_tokens",
				Value: 8192,
			},
		},
	},
	{
		Name:        "gpt5_max_tokens_to_max_completion_tokens",
		Description: "Convert max_tokens to max_completion_tokens for GPT-5 models",
		Conditions: []PatchCondition{
			{
				Key:      "model",
				Operator: OperatorContains,
				Value:    "gpt-5",
			},
			{
				Key:      "max_tokens",
				Operator: OperatorExists,
			},
		},
		Operations: []PatchOperation{
			{
				Op:    OpSet,
				Key:   "max_completion_tokens",
				Value: "{{max_tokens}}",
			},
			{
				Op:  OpDelete,
				Key: "max_tokens",
			},
		},
	},
	{
		Name:        "gpt5_remove_temperature",
		Description: "Remove temperature field for GPT-5 models",
		Conditions: []PatchCondition{
			{
				Key:      "model",
				Operator: OperatorContains,
				Value:    "gpt-5",
			},
			{
				Key:      "temperature",
				Operator: OperatorExists,
			},
		},
		Operations: []PatchOperation{
			{
				Op:  OpDelete,
				Key: "temperature",
			},
		},
	},
	{
		Name:        "gpt5_remove_top_p",
		Description: "Remove top_p field for GPT-5 models",
		Conditions: []PatchCondition{
			{
				Key:      "model",
				Operator: OperatorContains,
				Value:    "gpt-5",
			},
			{
				Key:      "top_p",
				Operator: OperatorExists,
			},
		},
		Operations: []PatchOperation{
			{
				Op:  OpDelete,
				Key: "top_p",
			},
		},
	},
	{
		Name:        "gemini_gpt5_remove_generation_config_top_p",
		Description: "Remove generationConfig.topP for GPT-5 models in Gemini format",
		Conditions: []PatchCondition{
			{
				Key:      "model",
				Operator: OperatorContains,
				Value:    "gpt-5",
			},
			{
				Key:      "generationConfig.topP",
				Operator: OperatorExists,
			},
		},
		Operations: []PatchOperation{
			{
				Op:  OpDelete,
				Key: "generationConfig.topP",
			},
		},
	},
	{
		Name:        "gemini3_remove_tool_choice_auto",
		Description: "Remove tool_choice when it is 'auto' for Gemini 3 models",
		Conditions: []PatchCondition{
			{
				Key:      "model",
				Operator: OperatorContains,
				Value:    "gemini-3",
			},
			{
				Key:      "tool_choice",
				Operator: OperatorEquals,
				Value:    "auto",
			},
		},
		Operations: []PatchOperation{
			{
				Op:  OpDelete,
				Key: "tool_choice",
			},
		},
	},
}

DefaultPredefinedPatches are built-in patches that are always available

Functions

func AddLazyPatch

func AddLazyPatch(meta *meta.Meta, patch PatchOperation)

AddLazyPatch adds data to the lazy patch queue in meta

func ToFloat64

func ToFloat64(v any) (float64, error)

Types

type ConditionOperator

type ConditionOperator string

ConditionOperator defines how to evaluate a condition

const (
	OperatorEquals      ConditionOperator = "equals"
	OperatorNotEquals   ConditionOperator = "not_equals"
	OperatorContains    ConditionOperator = "contains"
	OperatorNotContains ConditionOperator = "not_contains"
	OperatorRegex       ConditionOperator = "regex"
	OperatorExists      ConditionOperator = "exists"
	OperatorNotExists   ConditionOperator = "not_exists"
	OperatorHasPrefix   ConditionOperator = "has_prefix"
	OperatorHasSuffix   ConditionOperator = "has_suffix"
	OperatorGreaterThan ConditionOperator = "greater_than"
	OperatorLessThan    ConditionOperator = "less_than"
	OperatorGreaterEq   ConditionOperator = "greater_eq"
	OperatorLessEq      ConditionOperator = "less_eq"
	OperatorIn          ConditionOperator = "in"
	OperatorNotIn       ConditionOperator = "not_in"
)

type Config

type Config struct {

	// UserPatches are user-defined custom patches
	UserPatches []PatchRule `json:"user_patches,omitempty"`
}

Config holds the configuration for the patch plugin

type LazyPatchData

type LazyPatchData struct {
	Source string `json:"source"` // Source plugin name
	Data   any    `json:"data"`   // Data to be patched
}

LazyPatchData represents data to be applied by patch plugin later

type LogicOperator

type LogicOperator string

LogicOperator defines how multiple conditions are combined

const (
	// LogicAnd requires all conditions to be true (default)
	LogicAnd LogicOperator = "and"
	// LogicOr requires at least one condition to be true
	LogicOr LogicOperator = "or"
)

type OperationType

type OperationType string

OperationType defines the type of operation to perform

const (
	// OpSet sets a field to a specific value
	OpSet OperationType = "set"
	// OpDelete removes a field
	OpDelete OperationType = "delete"
	// OpAdd adds a field only if it doesn't exist
	OpAdd OperationType = "add"
	// OpLimit limits a numeric field to a maximum value
	OpLimit OperationType = "limit"
	// OpIncrement increments a numeric field by a value
	OpIncrement OperationType = "increment"
	// OpDecrement decrements a numeric field by a value
	OpDecrement OperationType = "decrement"
	// OpMultiply multiplies a numeric field by a value
	OpMultiply OperationType = "multiply"
	// OpDivide divides a numeric field by a value
	OpDivide OperationType = "divide"
	// OpAppend appends value to an array field
	OpAppend OperationType = "append"
	// OpPrepend prepends value to an array field
	OpPrepend OperationType = "prepend"
	// OpFunction executes an inline function on the field
	OpFunction OperationType = "function"
)

type PatchCondition

type PatchCondition struct {
	// Key is the field to check (supports dot notation for nested fields)
	// Special keys: "model", "original_model"
	Key string `json:"key"`
	// Operator defines how to compare the value
	Operator ConditionOperator `json:"operator"`
	// Value is the value to compare against
	Value string `json:"value"`
	// Values is an array of values for 'in' and 'not_in' operators
	Values []string `json:"values,omitempty"`
	// Negate inverts the result of this condition (for "not" logic)
	Negate bool `json:"negate,omitempty"`
}

PatchCondition defines when a patch should be applied

type PatchFunction

type PatchFunction func(root *ast.Node) (bool, error)

type PatchOperation

type PatchOperation struct {
	// Op is the operation type
	Op OperationType `json:"op"`
	// Key is the field to modify (supports dot notation for nested fields)
	Key string `json:"key"`
	// Value is the new value to set (not used for delete operations)
	Value any `json:"value,omitempty"`
	// Function is the inline Function code for OpFunction operations
	Function PatchFunction `json:"-"`
}

PatchOperation defines a modification to make to the JSON

func GetLazyPatches

func GetLazyPatches(meta *meta.Meta) []PatchOperation

GetLazyPatches retrieves all lazy patch data from meta

type PatchRule

type PatchRule struct {
	// Name is a descriptive name for this patch rule
	Name string `json:"name"`
	// Description explains what this patch does
	Description string `json:"description,omitempty"`
	// Conditions determine when this patch should be applied
	Conditions []PatchCondition `json:"conditions,omitempty"`
	// ConditionLogic defines how conditions are combined (default: "and")
	ConditionLogic LogicOperator `json:"condition_logic,omitempty"`
	// Operations define what modifications to make
	Operations []PatchOperation `json:"operations"`
}

PatchRule defines a complete patch rule with conditions and operations

type Plugin

type Plugin struct {
	noop.Noop
	// contains filtered or unexported fields
}

Plugin implements JSON request patching functionality

func NewPatchPlugin

func NewPatchPlugin() *Plugin

NewPatchPlugin creates a new patch plugin instance

func (*Plugin) ApplyPatches

func (p *Plugin) ApplyPatches(
	bodyBytes []byte,
	meta *meta.Meta,
	config *Config,
) ([]byte, bool, error)

ApplyPatches applies all applicable patches to the JSON body

func (*Plugin) ConvertRequest

func (p *Plugin) ConvertRequest(
	meta *meta.Meta,
	store adaptor.Store,
	req *http.Request,
	do adaptor.ConvertRequest,
) (adaptor.ConvertResult, error)

ConvertRequest applies JSON patches to the request body