README
¶
============================================ ``utility`` -- Common Golang Utility Library ============================================ Overview -------- This package contains a collection of common utility functions collected and isolated from projects ``util`` (or "misc") packages. Key Features ------------ - HTTP Client pooling. - Reading and writing structured data to a file. - Convince functions for rounding time values to predictable intervals. - Slice manipulation and introspection functions for inspecting common string types. Use and Development ------------------- The utility package is available under the terms of the Apache License (v2). All dependencies are managed by go module. The main dependencies are grip, for logging and `rehttp <https://github.com/PuerkitoBio/rehttp>`_ which provides automatic retry logic for HTTP requests, though the rehttp dependency is isolated from callers. All documentation is available on the `godoc <https://godoc.org/github.com/deciduosity/utility>`_. Feel free to open issues or submit pull requests!
Documentation
¶
Index ¶
- Variables
- func BuildFileList(startPath string, expressions ...string) ([]string, error)
- func FileExists(path string) bool
- func FromNanoseconds(duration time.Duration) int64
- func FromPythonTime(pyTime float64) time.Time
- func GetCustomHTTPRetryableClient(retry HTTPRetryFunction, delay HTTPDelayFunction) *http.Client
- func GetDefaultHTTPRetryableClient() *http.Client
- func GetHTTPClient() *http.Client
- func GetHTTPRetryableClient(conf HTTPRetryConfiguration) *http.Client
- func GetOAuth2HTTPClient(oauthToken string) *http.Client
- func GetOauth2CustomHTTPRetryableClient(token string, retry HTTPRetryFunction, delay HTTPDelayFunction) *http.Client
- func GetOauth2DefaultHTTPRetryableClient(oauthToken string) *http.Client
- func GetOauth2HTTPRetryableClient(oauthToken string, conf HTTPRetryConfiguration) *http.Client
- func GetPublicIP() (string, error)
- func GetSetDifference(a, b []string) []string
- func GetUTCDay(date time.Time) time.Time
- func GetUTCHour(date time.Time) time.Time
- func IndexOf(a []string, toFind string) int
- func IsTemporaryError(err error) bool
- func IsZeroTime(t time.Time) bool
- func JitterInterval(interval time.Duration) time.Duration
- func MakeRandomString(size int) string
- func PrintJSON(data interface{}) error
- func PutHTTPClient(c *http.Client)
- func RandomString() string
- func ReadJSON(r io.ReadCloser, target interface{}) error
- func ReadJSONFile(path string, target interface{}) error
- func ReadYAML(r io.ReadCloser, target interface{}) error
- func ReadYAMLFile(path string, target interface{}) error
- func ReadYAMLStrict(r io.ReadCloser, target interface{}) error
- func RoundPartOfDay(n int) time.Time
- func RoundPartOfHour(n int) time.Time
- func RoundPartOfMinute(n int) time.Time
- func SplitCommas(originals []string) []string
- func StringSliceContains(slice []string, item string) bool
- func StringSliceIntersection(a, b []string) []string
- func StringSliceSymmetricDifference(a, b []string) ([]string, []string)
- func ToNanoseconds(duration time.Duration) time.Duration
- func ToPythonTime(t time.Time) float64
- func UniqueStrings(slice []string) []string
- func UnixMilli(t time.Time) int64
- func WriteFile(path string, data string) error
- func WriteJSONFile(fn string, data interface{}) error
- func WriteRawFile(path string, data []byte) error
- func WriteYAMLFile(fn string, data interface{}) error
- type HTTPDelayFunction
- type HTTPRetryConfiguration
- type HTTPRetryFunction
- type TemporaryError
Constants ¶
This section is empty.
Variables ¶
var MaxTime time.Time = time.Unix(1<<63-62135596801, 999999999)
MaxTime represents the latest useful golang date (219248499-12-06 15:30:07.999999999 +0000 UTC)
var ZeroTime time.Time = time.Unix(0, 0)
ZeroTime represents 0 in epoch time
Functions ¶
func BuildFileList ¶
BuildFileList returns a list of files that match the given list of expressions rooted at the given startPath. The expressions correspond to gitignore ignore expressions: anything that would be matched - and therefore ignored by git - is included in the returned list of file paths. BuildFileList does not follow symlinks as it uses filpath.Walk, which does not follow symlinks.
func FileExists ¶
FileExists provides a clearer interface for checking if a file exists.
func FromNanoseconds ¶
fromNanoSeconds returns milliseconds of a duration for queries in the database.
func FromPythonTime ¶
FromPythonTime returns a time.Time that corresponds to the float style python time which is <seconds>.<fractional_seconds> from unix epoch.
func GetCustomHTTPRetryableClient ¶
func GetCustomHTTPRetryableClient(retry HTTPRetryFunction, delay HTTPDelayFunction) *http.Client
GetCustomHTTPRetryableClient allows you to generate an HTTP client that automatically retries failed request based on the provided custom logic.
func GetDefaultHTTPRetryableClient ¶
GetDefaultHTTPRetryableClient provides a retryable client with the default settings. Couple calls to GetHTTPRetryableClient, with defered calls to PutHTTPClient.
func GetHTTPClient ¶
GetHTTPClient produces default HTTP client from the pool, constructing a new client if needed. Always pair calls to GetHTTPClient with defered calls to PutHTTPClient.
func GetHTTPRetryableClient ¶
func GetHTTPRetryableClient(conf HTTPRetryConfiguration) *http.Client
GetHTTPRetryableClient produces an HTTP client that automatically retries failed requests according to the configured parameters. Couple calls to GetHTTPRetryableClient, with defered calls to PutHTTPClient.
func GetOAuth2HTTPClient ¶
GetOAuth2HTTPClient produces an HTTP client that will supply OAuth2 credentials with all requests. There is no validation of the token, and you should always call PutHTTPClient to return the client to the pool when you're done with it.
func GetOauth2CustomHTTPRetryableClient ¶
func GetOauth2CustomHTTPRetryableClient(token string, retry HTTPRetryFunction, delay HTTPDelayFunction) *http.Client
GetOauth2HTTPRetryableClient constructs an HTTP client that supplies OAuth2 credentials with all requests, retrying failed requests automatically according to definitions of the provided functions. There is no validation of the token, and you should always call PutHTTPClient to return the client to the pool when you're done with it.
func GetOauth2DefaultHTTPRetryableClient ¶
GetOauth2DefaultHTTPRetryableClient constructs an HTTP client that supplies OAuth2 credentials with all requests, retrying failed requests automatically according to the default retryable options. There is no validation of the token, and you should always call PutHTTPClient to return the client to the pool when you're done with it.
func GetOauth2HTTPRetryableClient ¶
func GetOauth2HTTPRetryableClient(oauthToken string, conf HTTPRetryConfiguration) *http.Client
GetOauth2HTTPRetryableClient constructs an HTTP client that supplies OAuth2 credentials with all requests, retrying failed requests automatically according to the configuration provided. There is no validation of the token, and you should always call PutHTTPClient to return the client to the pool when you're done with it.
func GetPublicIP ¶
func GetSetDifference ¶
GetSetDifference returns the elements in A that are not in B
func GetUTCDay ¶
Creates and returns a time.Time corresponding to the start of the UTC day containing the given date.
func GetUTCHour ¶
Creates and returns a time.Time corresponding to the start of the UTC hour containing the given date.
func IsTemporaryError ¶
IsTemporaryError returns true if the error object is also a temporary error.
func IsZeroTime ¶
IsZeroTime checks that a time is either equal to golang ZeroTime or UTC ZeroTime.
func JitterInterval ¶
JitterInterval returns a duration that some value between the interval and 2x the interval.
func MakeRandomString ¶
MakeRandomString constructs a hex-encoded random string of a specific length. The size reflects the number of random bytes, not the length of the string.
func PrintJSON ¶
func PrintJSON(data interface{}) error
PrintJSON marshals the data to a pretty-printed (indented) string and then prints it to standard output.
func PutHTTPClient ¶
PutHTTPClient returns the client to the pool, automatically reconfiguring the transport.
func RandomString ¶
func RandomString() string
RandomString returns a hex-encoded cryptographically random string. This function always returns 16bytes of randomness encoded in a 32 character string.
func ReadJSON ¶
func ReadJSON(r io.ReadCloser, target interface{}) error
ReadJSON provides an alternate interface to json.Unmarshal that reads data from an io.ReadCloser.
func ReadJSONFile ¶
ReadJSONFile parses json into the target argument from the file located at the specifed path.
func ReadYAML ¶
func ReadYAML(r io.ReadCloser, target interface{}) error
ReadYAML provides an alternate interface to yaml.Unmarshal that reads data from an io.ReadCloser.
func ReadYAMLFile ¶
ReadYAMLFile parses yaml into the target argument from the file located at the specifed path.
func ReadYAMLStrict ¶
func ReadYAMLStrict(r io.ReadCloser, target interface{}) error
ReadYAMLStrict is the same as ReadYAML but uses strict unmarshalling.
func RoundPartOfDay ¶
RoundPartOfDay produces a time value with the hour value rounded down to the most recent interval.
func RoundPartOfHour ¶
RoundPartOfHour produces a time value with the minute value rounded down to the most recent interval.
func RoundPartOfMinute ¶
RoundPartOfMinute produces a time value with the second value rounded down to the most recent interval.
func SplitCommas ¶
SplitCommas returns the slice of strings after splitting each string by commas.
func StringSliceContains ¶
StringSliceContains determines if a string is in a slice
func StringSliceIntersection ¶
StringSliceIntersection returns the intersecting elements of slices a and b.
func StringSliceSymmetricDifference ¶
StringSliceSymmetricDifference returns only elements not in common between 2 slices (ie. inverse of the intersection)
func ToNanoseconds ¶
fromNanoSeconds returns milliseconds of a duration for queries in the database.
func ToPythonTime ¶
ToPythonTime returns a number in the format that python's time.time() returns as a float with <seconds>.<fractional_seconds>
func UniqueStrings ¶
UniqueStrings takes a slice of strings and returns a new slice with duplicates removed. Order is preserved.
func UnixMilli ¶
UnixMilli returns t as a Unix time, the number of nanoseconds elapsed since January 1, 1970 UTC. The result is undefined if the Unix time in nanoseconds in cannot be represented by an int64 (a date before the year 1678 or after 2262). Note that this means the result of calling UnixMilli on the zero Time on the zero Time is undefined. The result does not depend on the location associated with t.
func WriteJSONFile ¶
WriteJSONFile marshals the data into json and writes it into a file at the specified path.
func WriteRawFile ¶
WriteRawFile writes a sequence of byes to a new file created at the specified path.
func WriteYAMLFile ¶
WriteYAMLFile marshals the data into json and writes it into a file at the specified path.
Types ¶
type HTTPDelayFunction ¶
type HTTPDelayFunction func(index int, req *http.Request, resp *http.Response, err error) time.Duration
HTTPDelayFunction makes it possible to write customizable retry backoff logic, by allowing you to evaluate the previous request and response and return the duration to wait before the next request.
type HTTPRetryConfiguration ¶
type HTTPRetryConfiguration struct {
MaxDelay time.Duration
BaseDelay time.Duration
MaxRetries int
TemporaryErrors bool
Methods []string
Statuses []int
Errors []error
ErrorStrings []string
}
HTTPRetryConfiguration makes it possible to configure the retry semantics for retryable clients. In most cases, construct this object using the NewDefaultHttpRetryConf, which provides reasonable defaults.
func NewDefaultHTTPRetryConf ¶
func NewDefaultHTTPRetryConf() HTTPRetryConfiguration
NewDefaultHTTPRetryConf constructs a HTTPRetryConfiguration object with reasonable defaults.
type HTTPRetryFunction ¶
HTTPRetryFunction makes it possible to write customizable retry logic. Returning true if the request should be retried again and false otherwise.
type TemporaryError ¶
TemporayError defines an interface for use in retryable HTTP clients to identify certain errors as Temporary.
Source Files