httpclient

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 2, 2024 License: MIT Imports: 28 Imported by: 3

Documentation

Overview

http_client.go 

The `http_client` package provides a configurable HTTP client tailored for interacting with specific APIs.

It supports different authentication methods, including "bearer" and "oauth". The client is designed with a focus on concurrency management, structured error handling, and flexible configuration options. The package offers a default timeout, custom backoff strategies, dynamic rate limiting, and detailed logging capabilities. The main `Client` structure encapsulates all necessary components, like the baseURL, authentication details, and an embedded standard HTTP client.

http_concurrency_management.go package httpclient provides utilities to manage HTTP client interactions, including concurrency control. The Concurrency Manager ensures no more than a certain number of concurrent requests (e.g., 5 for Jamf Pro) are sent at the same time. This is managed using a semaphore

httpclient_cookies.go

httpclient_error_response.go This package provides utility functions and structures for handling and categorizing HTTP error responses.

http_helpers.go

httpclient_methods.go

httpclient_mocklogger.go

httpclient_ping.go

http_request.go

sdk_version.go

Index

Constants

View Source 
const (
	DefaultLogLevel                  = logger.LogLevelInfo
	DefaultMaxRetryAttempts          = 3
	DefaultEnableDynamicRateLimiting = true
	DefaultMaxConcurrentRequests     = 5
	DefaultTokenBufferPeriod         = 5 * time.Minute
	DefaultTotalRetryDuration        = 5 * time.Minute
	DefaultTimeout                   = 10 * time.Second
	FollowRedirects                  = true
	MaxRedirects                     = 10
)
View Source 
const (
	MaxConcurrency     = 10              // Maximum allowed concurrent requests
	MinConcurrency     = 1               // Minimum allowed concurrent requests
	EvaluationInterval = 1 * time.Minute // Time interval for evaluating metrics and adjusting concurrency
)
View Source 
const (
	SDKVersion    = "0.0.90"
	UserAgentBase = "go-api-http-client"
)

Variables

This section is empty.

Functions

func CookiesFromHeader added in v0.0.90 

func CookiesFromHeader(header http.Header) []*http.Cookie

Utility function to convert cookies from http.Header to []*http.Cookie. This can be useful if cookies are stored in http.Header (e.g., from a response).

func DetermineAuthMethod added in v0.0.9 

func DetermineAuthMethod(authConfig AuthConfig) (string, error)

DetermineAuthMethod determines the authentication method based on the provided credentials. It prefers strong authentication methods (e.g., OAuth) over weaker ones (e.g., bearer tokens). It logs an error and returns "unknown" if no valid credentials are provided.

func GetUserAgentHeader 

func GetUserAgentHeader() string

func IsIdempotentHTTPMethod added in v0.0.46 

func IsIdempotentHTTPMethod(method string) bool

IsIdempotentHTTPMethod checks if the given HTTP method is idempotent.

func IsNonIdempotentHTTPMethod added in v0.0.46 

func IsNonIdempotentHTTPMethod(method string) bool

IsNonIdempotentHTTPMethod checks if the given HTTP method is non-idempotent. PATCH can be idempotent but often isn't used as such.

func Min 

func Min(a, b int) int

Min returns the smaller of the two integers.

func ParseCookieHeader added in v0.0.90 

func ParseCookieHeader(header string) *http.Cookie

ParseCookieHeader parses a single Set-Cookie header and returns an *http.Cookie.

func ParseISO8601Date 

func ParseISO8601Date(dateStr string) (time.Time, error)

ParseISO8601Date attempts to parse a string date in ISO 8601 format.

func RedactSensitiveCookies added in v0.0.90 

func RedactSensitiveCookies(cookies []*http.Cookie) []*http.Cookie

RedactSensitiveCookies redacts sensitive information from cookies. It takes a slice of *http.Cookie and returns a redacted slice of *http.Cookie.

Types

type APIError added in v0.0.82 

type APIError struct {
	StatusCode int                    // HTTP status code
	Type       string                 // A brief identifier for the type of error
	Message    string                 // Human-readable message
	Detail     string                 // Detailed error message
	Errors     map[string]interface{} // A map to hold various error fields
	Raw        string                 // Raw response body for unstructured errors
}

APIError represents a more flexible structure for API error responses.

func (*APIError) Error added in v0.0.82 

func (e *APIError) Error() string

Error returns a JSON representation of the APIError.

type AuthConfig 

type AuthConfig struct {
	Username     string `json:"Username,omitempty"`
	Password     string `json:"Password,omitempty"`
	ClientID     string `json:"ClientID,omitempty"`
	ClientSecret string `json:"ClientSecret,omitempty"`
}

AuthConfig represents the structure to read authentication details from a JSON configuration file.

type Client 

type Client struct {
	InstanceName       string    // Website Instance name without the root domain
	AuthMethod         string    // Specifies the authentication method: "bearer" or "oauth"
	Token              string    // Authentication Token
	OverrideBaseDomain string    // Base domain override used when the default in the api handler isn't suitable
	Expiry             time.Time // Expiry time set for the auth token

	Logger           logger.Logger
	ConcurrencyMgr   *ConcurrencyManager
	PerfMetrics      PerformanceMetrics
	APIHandler       apihandler.APIHandler // APIHandler interface used to define which API handler to use
	AuthTokenHandler *authenticationhandler.AuthTokenHandler
	// contains filtered or unexported fields
}

Client represents an HTTP client to interact with a specific API.

func BuildClient 

func BuildClient(config ClientConfig) (*Client, error)

BuildClient creates a new HTTP client with the provided configuration.

func (*Client) AcquireConcurrencyToken added in v0.0.48 

func (c *Client) AcquireConcurrencyToken(ctx context.Context) (context.Context, error)

AcquireConcurrencyToken attempts to acquire a token from the ConcurrencyManager to manage the number of concurrent requests. This function is designed to ensure that the HTTP client adheres to predefined concurrency limits, preventing an excessive number of simultaneous requests. It creates a new context with a timeout to avoid indefinite blocking in case the concurrency limit is reached. Upon successfully acquiring a token, it records the time taken to acquire the token and updates performance metrics accordingly. The function then adds the acquired request ID to the context, which can be used for tracking and managing individual requests.

Parameters: - ctx: The parent context from which the new context with timeout will be derived. This allows for proper request cancellation and timeout handling.

Returns: - A new context containing the acquired request ID, which should be passed to subsequent operations requiring concurrency control.

- An error if the token could not be acquired within the timeout period or due to any other issues encountered by the ConcurrencyManager.

Usage: This function should be called before making an HTTP request that needs to be controlled for concurrency. The returned context should be used for the HTTP request to ensure it is associated with the acquired concurrency token.

func (*Client) AdjustConcurrencyBasedOnMetrics 

func (c *Client) AdjustConcurrencyBasedOnMetrics()

AdjustConcurrencyBasedOnMetrics evaluates the current metrics and adjusts the concurrency limit if required. It checks metrics like average token acquisition time and decides on a new concurrency limit. The method ensures that the new limit respects the minimum and maximum allowed concurrency bounds.

func (*Client) AverageAcquisitionTime 

func (c *Client) AverageAcquisitionTime() time.Duration

Returns the average Acquisition Time to get a token from the semaphore

func (*Client) DoMultipartRequest 

func (c *Client) DoMultipartRequest(method, endpoint string, fields map[string]string, files map[string]string, out interface{}) (*http.Response, error)

DoMultipartRequest creates and executes a multipart HTTP request. It is used for sending files and form fields in a single request. This method handles the construction of the multipart message body, setting the appropriate headers, and sending the request to the given endpoint.

Parameters: - method: The HTTP method to use (e.g., POST, PUT). - endpoint: The API endpoint to which the request will be sent. - fields: A map of form fields and their values to include in the multipart message. - files: A map of file field names to file paths that will be included as file attachments. - out: A pointer to a variable where the unmarshaled response will be stored.

Returns: - A pointer to the http.Response received from the server. - An error if the request could not be sent or the response could not be processed.

The function first validates the authentication token, then constructs the multipart request body based on the provided fields and files. It then constructs the full URL for the request, sets the required headers (including Authorization and Content-Type), and sends the request.

If debug mode is enabled, the function logs all the request headers before sending the request. After the request is sent, the function checks the response status code. If the response is not within the success range (200-299), it logs an error and returns the response and an error. If the response is successful, it attempts to unmarshal the response body into the 'out' parameter.

Note: The caller should handle closing the response body when successful.

func (*Client) DoPing added in v0.0.88 

func (c *Client) DoPing(method, endpoint string, body, out interface{}) (*http.Response, error)

Example: var result MyResponseType resp, err := client.DoPing("GET", "/api/health", nil, &result) 

if err != nil {
    // Handle error
}

// Process response

func (*Client) DoPingV2 added in v0.0.89 

func (c *Client) DoPingV2(host string, timeout time.Duration) error

func (*Client) DoRequest 

func (c *Client) DoRequest(method, endpoint string, body, out interface{}) (*http.Response, error)

func (*Client) EvaluateMetricsAndAdjustConcurrency 

func (c *Client) EvaluateMetricsAndAdjustConcurrency()

EvaluateMetricsAndAdjustConcurrency evaluates the performance metrics and makes necessary adjustments to the concurrency limit. The method assesses the average response time and adjusts the concurrency based on how it compares to the historical average acquisition time. If the average response time has significantly increased compared to the historical average, the concurrency limit is decreased, and vice versa. The method ensures that the concurrency limit remains within the bounds defined by the system's best practices.

func (*Client) GetPerformanceMetrics 

func (c *Client) GetPerformanceMetrics() *PerformanceMetrics

Returns performance metrics from the http client

func (*Client) HistoricalAverageAcquisitionTime 

func (c *Client) HistoricalAverageAcquisitionTime() time.Duration

func (*Client) StartConcurrencyAdjustment 

func (c *Client) StartConcurrencyAdjustment()

StartConcurrencyAdjustment launches a periodic checker that evaluates current metrics and adjusts concurrency limits if needed. It uses a ticker to periodically trigger the adjustment logic.

func (*Client) StartMetricEvaluation 

func (c *Client) StartMetricEvaluation()

StartMetricEvaluation continuously monitors the client's interactions with the API and adjusts the concurrency limits dynamically. The function evaluates metrics at regular intervals to detect burst activity patterns. If a burst activity is detected (e.g., many requests in a short period), the evaluation interval is reduced for more frequent checks. Otherwise, it reverts to a default interval for regular checks. After each evaluation, the function calls EvaluateMetricsAndAdjustConcurrency to potentially adjust the concurrency based on observed metrics.

The evaluation process works as follows: 1. Sleep for the defined evaluation interval. 2. Check if there's a burst in activity using the isBurstActivity method. 3. If a burst is detected, the evaluation interval is shortened to more frequently monitor and adjust the concurrency. 4. If no burst is detected, it maintains the default evaluation interval. 5. It then evaluates the metrics and adjusts the concurrency accordingly.

type ClientConfig added in v0.0.12 

type ClientConfig struct {
	Auth          AuthConfig        // User can either supply these values manually or pass from LoadAuthConfig/Env vars
	Environment   EnvironmentConfig // User can either supply these values manually or pass from LoadAuthConfig/Env vars
	ClientOptions ClientOptions     // Optional configuration options for the HTTP Client
}

Config holds configuration options for the HTTP Client.

func LoadConfigFromEnv added in v0.0.75 

func LoadConfigFromEnv(config *ClientConfig) (*ClientConfig, error)

LoadConfigFromEnv populates the ClientConfig structure with values from environment variables. It updates the configuration for authentication, environment specifics, and client options based on the presence of environment variables. For each configuration option, if an environment variable is set, its value is used; otherwise, the existing value in the ClientConfig structure is retained. It also sets default values if necessary and validates the final configuration, returning an error if the configuration is incomplete.

func LoadConfigFromFile added in v0.0.77 

func LoadConfigFromFile(filePath string) (*ClientConfig, error)

LoadConfigFromFile loads configuration values from a JSON file into the ClientConfig struct. This function opens the specified configuration file, reads its content, and unmarshals the JSON data into the ClientConfig struct. It's designed to initialize the client configuration with values from a file, complementing or overriding defaults and environment variable settings. LoadConfigFromFile loads configuration values from a JSON file into the ClientConfig struct.

type ClientOptions added in v0.0.12 

type ClientOptions struct {
	EnableCookieJar           bool   // Field to enable or disable cookie jar
	LogLevel                  string // Field for defining tiered logging level.
	LogOutputFormat           string // Field for defining the output format of the logs. Use "JSON" for JSON format, "console" for human-readable format
	LogConsoleSeparator       string // Field for defining the separator in console output format.
	HideSensitiveData         bool   // Field for defining whether sensitive fields should be hidden in logs.
	MaxRetryAttempts          int    // Config item defines the max number of retry request attempts for retryable HTTP methods.
	EnableDynamicRateLimiting bool   // Field for defining whether dynamic rate limiting should be enabled.
	MaxConcurrentRequests     int    // Field for defining the maximum number of concurrent requests allowed in the semaphore
	FollowRedirects           bool   // Flag to enable/disable following redirects
	MaxRedirects              int    // Maximum number of redirects to follow
	TokenRefreshBufferPeriod  time.Duration
	TotalRetryDuration        time.Duration
	CustomTimeout             time.Duration
}

ClientOptions holds optional configuration options for the HTTP Client.

type ConcurrencyManager 

type ConcurrencyManager struct {
	AcquisitionTimes []time.Duration
	// contains filtered or unexported fields
}

ConcurrencyManager controls the number of concurrent HTTP requests.

func NewConcurrencyManager 

func NewConcurrencyManager(limit int, logger logger.Logger, debugMode bool) *ConcurrencyManager

NewConcurrencyManager initializes a new ConcurrencyManager with the given concurrency limit, logger, and debug mode. The ConcurrencyManager ensures no more than a certain number of concurrent requests are made. It uses a semaphore to control concurrency.

func (*ConcurrencyManager) Acquire 

func (c *ConcurrencyManager) Acquire(ctx context.Context) (uuid.UUID, error)

Acquire attempts to get a token to allow an HTTP request to proceed. It blocks until a token is available or the context expires. Returns a unique request ID upon successful acquisition.

func (*ConcurrencyManager) AdjustConcurrencyLimit 

func (c *ConcurrencyManager) AdjustConcurrencyLimit(newLimit int)

AdjustConcurrencyLimit dynamically modifies the maximum concurrency limit based on the newLimit provided. This function helps in adjusting the concurrency limit in real-time based on observed system performance and other metrics. It transfers the tokens from the old semaphore to the new one, ensuring that there's no loss of tokens during the transition.

func (*ConcurrencyManager) AverageAcquisitionTime 

func (c *ConcurrencyManager) AverageAcquisitionTime() time.Duration

AverageAcquisitionTime computes the average time taken to acquire a token from the semaphore. It helps in understanding the contention for tokens and can be used to adjust concurrency limits.

func (*ConcurrencyManager) HistoricalAverageAcquisitionTime 

func (c *ConcurrencyManager) HistoricalAverageAcquisitionTime() time.Duration

HistoricalAverageAcquisitionTime computes the average time taken to acquire a token from the semaphore over a historical period (e.g., the last 5 minutes). It helps in understanding the historical contention for tokens and can be used to adjust concurrency limits.

func (*ConcurrencyManager) Release 

func (c *ConcurrencyManager) Release(requestID uuid.UUID)

Release returns a token back to the pool, allowing other requests to proceed. It uses the provided requestID for logging and debugging purposes.

type EnvironmentConfig added in v0.0.5 

type EnvironmentConfig struct {
	InstanceName       string `json:"InstanceName,omitempty"`
	OverrideBaseDomain string `json:"OverrideBaseDomain,omitempty"`
	APIType            string `json:"APIType,omitempty"`
}

EnvironmentConfig represents the structure to read authentication details from a JSON configuration file.

type MockLogger added in v0.0.65 

type MockLogger struct {
	mock.Mock
	*zap.Logger
}

MockLogger is a mock type for the logger.Logger interface used in the httpclient package.

func NewMockLogger added in v0.0.65 

func NewMockLogger() *MockLogger

NewMockLogger creates a new instance of MockLogger with an embedded no-op *zap.Logger.

func (*MockLogger) Debug added in v0.0.65 

func (m *MockLogger) Debug(msg string, fields ...zap.Field)

func (*MockLogger) Error added in v0.0.65 

func (m *MockLogger) Error(msg string, fields ...zap.Field) error

func (*MockLogger) Fatal added in v0.0.65 

func (m *MockLogger) Fatal(msg string, fields ...zap.Field)

func (*MockLogger) GetLogLevel added in v0.0.65 

func (m *MockLogger) GetLogLevel() logger.LogLevel

func (*MockLogger) Info added in v0.0.65 

func (m *MockLogger) Info(msg string, fields ...zap.Field)

func (*MockLogger) LogAuthTokenError added in v0.0.83 

func (m *MockLogger) LogAuthTokenError(event string, method string, url string, statusCode int, err error)

func (*MockLogger) LogCookies added in v0.0.90 

func (m *MockLogger) LogCookies(direction string, obj interface{}, method, url string)

func (*MockLogger) LogError added in v0.0.80 

func (m *MockLogger) LogError(event string, method string, url string, statusCode int, serverStatusMessage string, err error, rawResponse string)

func (*MockLogger) LogRateLimiting added in v0.0.80 

func (m *MockLogger) LogRateLimiting(event string, method string, url string, retryAfter string, waitDuration time.Duration)

func (*MockLogger) LogRequestEnd added in v0.0.80 

func (m *MockLogger) LogRequestEnd(event string, method string, url string, statusCode int, duration time.Duration)

func (*MockLogger) LogRequestStart added in v0.0.80 

func (m *MockLogger) LogRequestStart(event string, requestID string, userID string, method string, url string, headers map[string][]string)

func (*MockLogger) LogResponse added in v0.0.80 

func (m *MockLogger) LogResponse(event string, method string, url string, statusCode int, responseBody string, responseHeaders map[string][]string, duration time.Duration)

func (*MockLogger) LogRetryAttempt added in v0.0.80 

func (m *MockLogger) LogRetryAttempt(event string, method string, url string, attempt int, reason string, waitDuration time.Duration, err error)

func (*MockLogger) Panic added in v0.0.65 

func (m *MockLogger) Panic(msg string, fields ...zap.Field)

func (*MockLogger) SetLevel added in v0.0.65 

func (m *MockLogger) SetLevel(level logger.LogLevel)

Define all methods from the logger.Logger interface with mock implementations.

func (*MockLogger) Warn added in v0.0.65 

func (m *MockLogger) Warn(msg string, fields ...zap.Field)

func (*MockLogger) With added in v0.0.65 

func (m *MockLogger) With(fields ...zap.Field) logger.Logger

type PerformanceMetrics 

type PerformanceMetrics struct {
	TotalRequests        int64
	TotalRetries         int64
	TotalRateLimitErrors int64
	TotalResponseTime    time.Duration
	TokenWaitTime        time.Duration
	// contains filtered or unexported fields
}

ClientPerformanceMetrics captures various metrics related to the client's interactions with the API, providing insights into its performance and behavior.

type StructuredError added in v0.0.82 

type StructuredError struct {
	Error struct {
		Code    string `json:"code"`
		Message string `json:"message"`
	} `json:"error"`
}

StructuredError represents a structured error response from the API.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.