util

package
v0.0.0-...-366f0b0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 2, 2019 License: Apache-2.0 Imports: 39 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	MarkerTaskID   = "EVR_TASK_ID"
	MarkerAgentPID = "EVR_AGENT_PID"
)
View Source 
const (
	DefaultSSHPort = "22"
)

Variables

View Source 
var ErrBufferFull = errors.New("buffer full")

ErrBufferFull indicates that a CappedWriter's bytes.Buffer has MaxBytes bytes.

View Source 
var MaxTime time.Time = time.Unix(1<<63-62135596801, 999999999)

MaxTime represents the latest useful golang date (219248499-12-06 15:30:07.999999999 +0000 UTC)

View Source 
var ZeroTime time.Time = time.Unix(0, 0)

ZeroTime represents 0 in epoch time

Functions

func BuildArchive 

func BuildArchive(ctx context.Context, tarWriter *tar.Writer, rootPath string, includes []string,
	excludes []string, logger grip.Journaler) (int, error)

BuildArchive reads the rootPath directory into the tar.Writer, taking included and excluded strings into account. Returns the number of files that were added to the archive

func BuildFileList 

func BuildFileList(startPath string, expressions ...string) ([]string, error)

BuildFileList returns a list of files that match the given list of expressions rooted at the given startPath. The expressions correspond to gitignore ignore expressions: anything that would be matched - and therefore ignored by git - is included in the returned list of file paths. BuildFileList does not follow symlinks as it uses filpath.Walk, which does not follow symlinks.

func CalculateHMACHash 

func CalculateHMACHash(secret []byte, body []byte) (string, error)

CalculateHMACHash calculates a sha256 HMAC has of the body with the given secret. The body must NOT be modified after calculating this hash. The string result will be prefixed with "sha256=", followed by the HMAC hash. When validating this hash, use a constant-time compare to avoid vulnerability to timing attacks.

func CleanForPath 

func CleanForPath(name string) string

func CleanName 

func CleanName(name string) string

CleanName returns a name with spaces and dashes replaced with safe underscores

func DeepCopy 

func DeepCopy(src, copy interface{}, registeredTypes []interface{}) error

DeepCopy makes a deep copy of the src value into the copy params The registeredTypes param can be optionally used to register additional types that need to be used by gob

func EscapeJQLReservedChars 

func EscapeJQLReservedChars(in string) string

see https://confluence.atlassian.com/jiracoreserver073/search-syntax-for-text-fields-861257223.html

func ExpandValues 

func ExpandValues(input interface{}, expansions *Expansions) error

Taking in the input and expansions map, apply the expansions to any appropriate fields in the input. The input must be a pointer to a struct so that the underlying struct can be modified.

func ExtractTarball 

func ExtractTarball(ctx context.Context, reader io.Reader, rootPath string, excludes []string) error

func FileExists 

func FileExists(path string) (bool, error)

FileExists returns true if 'path' exists.

func FromNanoseconds 

func FromNanoseconds(duration time.Duration) int64

fromNanoSeconds returns milliseconds of a duration for queries in the database.

func FromPythonTime 

func FromPythonTime(pyTime float64) time.Time

FromPythonTime returns a time.Time that corresponds to the float style python time which is <seconds>.<fractional_seconds> from unix epoch.

func GetBoolValue 

func GetBoolValue(r *http.Request, valueKey string, defaultValue bool) (bool, error)

GetBoolValue returns a form value as an integer

func GetHTTPClient 

func GetHTTPClient() *http.Client

func GetIntValue 

func GetIntValue(r *http.Request, valueKey string, defaultValue int) (int, error)

GetIntValue returns a form value as an integer

func GetOAuth2HTTPClient 

func GetOAuth2HTTPClient(oauthToken string) (*http.Client, error)

func GetRetryableOauth2HTTPClient 

func GetRetryableOauth2HTTPClient(oauthToken string, fRetry rehttp.RetryFn, fDelay rehttp.DelayFn) (*http.Client, error)

func GetSetDifference 

func GetSetDifference(a, b []string) []string

GetSetDifference returns the elements in A that are not in B

func GetStringArrayValue 

func GetStringArrayValue(r *http.Request, valueKey string, defaultValue []string) []string

GetStringArrayValue returns a form value as a string array

func GetUTCDay 

func GetUTCDay(date time.Time) time.Time

Creates and returns a time.Time corresponding to the start of the UTC day containing the given date.

func GetUTCHour 

func GetUTCHour(date time.Time) time.Time

Creates and returns a time.Time corresponding to the start of the UTC hour containing the given date.

func IndexWhiteSpace 

func IndexWhiteSpace(s string) int

IndexWhiteSpace returns the first index of white space in the given string. Returns -1 if no white space exists.

func IsExpandable 

func IsExpandable(param string) bool

IsExpandable returns true if the passed in string contains an expandable parameter

func IsFiniteNumericFloat 

func IsFiniteNumericFloat(f float64) bool

IsFiniteNumericFloat takes a float64 and checks that it is not +inf, -inf, or NaN

func IsZeroTime 

func IsZeroTime(t time.Time) bool

IsZeroTime checks that a time is either equal to golang ZeroTime or UTC ZeroTime.

func JitterInterval 

func JitterInterval(interval time.Duration) time.Duration

JitterInterval returns a duration that some value between the interval and 2x the interval.

func KillSpawnedProcs 

func KillSpawnedProcs(key string, logger grip.Journaler) error

KillSpawnedProcs cleans up any tasks that were spawned by the given task.

func MakeNotificationErrorHandler 

func MakeNotificationErrorHandler(name string) send.ErrorHandler

func Min 

func Min(a ...int) int

min function for ints

func NewEvergreenWebhookLogger 

func NewEvergreenWebhookLogger() (send.Sender, error)

func NewRequestReader 

func NewRequestReader(req *http.Request) io.ReadCloser

NewRequestReader returns an io.ReadCloser closer for the body of an *http.Request, using a limited reader internally to avoid unbounded reading from the request body. The reader is limited to 16 megabytes.

func NewRequestReaderWithSize 

func NewRequestReaderWithSize(req *http.Request, size int64) io.ReadCloser

NewRequestReaderWithSize returns an io.ReadCloser closer for the body of an *http.Request with a user-specified size.

func NewResponseReader 

func NewResponseReader(req *http.Response) io.ReadCloser

NewResponseReader returns an io.ReadCloser closer for the body of an *http.Response, using a limited reader internally to avoid unbounded reading from the request body. The reader is limited to 16 megabytes.

func NewResponseReaderWithSize 

func NewResponseReaderWithSize(req *http.Response, size int64) io.ReadCloser

NewResponseReaderWithSize returns an io.ReadCloser closer for the body of an *http.Response with a user-specified size.

func NewWebhookMessage 

func NewWebhookMessage(id string, url string, secret []byte, body []byte, headers map[string][]string) message.Composer

func NewWebhookMessageWithStruct 

func NewWebhookMessageWithStruct(raw EvergreenWebhook) message.Composer

func PutHTTPClient 

func PutHTTPClient(c *http.Client)

func RandomString 

func RandomString() string

RandomString returns a cryptographically random string.

func ReadFromYAMLFile 

func ReadFromYAMLFile(fn string, data interface{}) error

func ReadJSONInto 

func ReadJSONInto(r io.ReadCloser, data interface{}) error

ReadJSONInto reads JSON from an io.ReadCloser into the data pointer.

func ReadJSONIntoWithLength 

func ReadJSONIntoWithLength(r io.ReadCloser, data interface{}) (int, error)

ReadJSONInto reads JSON from an io.ReadCloser into the data pointer.

func ReadYAMLInto 

func ReadYAMLInto(r io.ReadCloser, data interface{}) error

ReadYAMLInto reads data for the given io.ReadCloser - until it hits an error or reaches EOF - and attempts to unmarshal the data read into the given interface.

func RehttpDelay 

func RehttpDelay(initialSleep time.Duration, numAttempts int) rehttp.DelayFn

func Retry 

func Retry(ctx context.Context, op RetriableFunc, attempts int, min time.Duration, max time.Duration) error

Retry provides a mechanism to retry an operation with exponential backoff with jitter. Specify minimum duration, maximum duration, and maximum number of retries.

It will set min to 100ms if not set. It will set max to (min * 2^attempts) if not set. It will set attempts to 1 if not set.

func RetryWithArgs 

func RetryWithArgs(ctx context.Context, op RetriableFunc, args RetryArgs) error

RetryWithArgs is a convenience wrapper around Retry that passes the parameters from RetryArgs to Retry.

func RoundPartOfDay 

func RoundPartOfDay(n int) time.Time

RoundPartOfDay produces a time value with the hour value rounded down to the most recent interval.

func RoundPartOfHour 

func RoundPartOfHour(n int) time.Time

RoundPartOfHour produces a time value with the minute value rounded down to the most recent interval.

func RoundPartOfMinute 

func RoundPartOfMinute(n int) time.Time

RoundPartOfMinute produces a time value with the second value rounded down to the most recent interval.

func StringSliceContains 

func StringSliceContains(slice []string, item string) bool

StringSliceContains determines if a string is in a slice

func StringSliceIntersection 

func StringSliceIntersection(a, b []string) []string

StringSliceIntersection returns the intersecting elements of slices a and b.

func TarGzReader 

func TarGzReader(path string) (f, gz io.ReadCloser, tarReader *tar.Reader, err error)

TarGzReader returns a file, gzip reader, and tar reader for the given path. The tar reader wraps the gzip reader, which wraps the file.

func TarGzWriter 

func TarGzWriter(path string) (f, gz io.WriteCloser, tarWriter *tar.Writer, err error)

TarGzWriter returns a file, gzip writer, and tarWriter for the path. The tar writer wraps the gzip writer, which wraps the file.

func ToNanoseconds 

func ToNanoseconds(duration time.Duration) time.Duration

fromNanoSeconds returns milliseconds of a duration for queries in the database.

func ToPythonTime 

func ToPythonTime(t time.Time) float64

ToPythonTime returns a number in the format that python's time.time() returns as a float with <seconds>.<fractional_seconds>

func TrackProcess 

func TrackProcess(key string, pid int, logger grip.Journaler)

func Truncate 

func Truncate(input string, outputLength int) string

Truncate returns a string of at most the given length.

func TryParseFloat 

func TryParseFloat(s string) (float64, error)

TryParseFloat takes an input string and validates that it is a valid finite floating point number. The number is returned if valid, NaN if not

func UniqueStrings 

func UniqueStrings(slice []string) []string

UniqueStrings takes a slice of strings and returns a new slice with duplicates removed. Order is preserved.

func WriteCSVResponse 

func WriteCSVResponse(w http.ResponseWriter, status int, data interface{})

WriteToCSVResponse takes in an interface that is a slice or an array and converts the struct to csv for fields that have the the csv struct tag.

func WriteJSONInto 

func WriteJSONInto(fn string, data interface{}) error

func WriteTempFile 

func WriteTempFile(prefix string, data []byte) (string, error)

WriteTempFile creates a temp file, writes the data to it, closes it and returns the file name.

func WriteToFile 

func WriteToFile(body io.ReadCloser, filepath string) error

WriteToFile takes in a body and filepath and writes out the data in the body

func WriteToTempFile 

func WriteToTempFile(data string) (string, error)

WriteToTempFile writes the given string to a temporary file and returns the path to the file.

Types

type ArchiveContentFile 

type ArchiveContentFile struct {
	Path string
	Info os.FileInfo
	// contains filtered or unexported fields
}

ArchiveContentFile represents a tar file on disk.

func FindContentsToArchive 

func FindContentsToArchive(ctx context.Context, rootPath string, includes, excludes []string) ([]ArchiveContentFile, error)

type CachedDurationValue 

type CachedDurationValue struct {
	Value       time.Duration `bson:"value"`
	TTL         time.Duration `bson:"ttl"`
	CollectedAt time.Time     `bson:"collected_at"`
	// contains filtered or unexported fields
}

CachedDurationValue represents a calculated int value saved in a database with a expiration time. When the data is not expired, the value is returned directly by the Get() method, otherwise, a refresh function is called, to update the value.

func NewCachedDurationValue 

func NewCachedDurationValue(start, ttl time.Duration, refresh CachedDurationValueRefresher) *CachedDurationValue

NewCachedDurationValue constructs a CachedDurationValue object.

func (*CachedDurationValue) Get 

func (v *CachedDurationValue) Get() (time.Duration, bool)

Get returns the value, refreshing it when its stale. The "ok" value tells the caller that the value needs to be persisted and may have changed since the last time Get was called.

func (*CachedDurationValue) SetRefresher 

func (v *CachedDurationValue) SetRefresher(r CachedDurationValueRefresher) error

SetRefresher sets CachedDurationValueRefresher for the object which is needed when reading CachedDurationValue objects out of the database.

It is not permissible to set the refresher to either nil or a value when it is *not* nil.

func (*CachedDurationValue) String 

func (v *CachedDurationValue) String() string

String implements fmt.Stringer reporting how stale the value is in the stale case.

type CachedDurationValueRefresher 

type CachedDurationValueRefresher func(time.Duration) (time.Duration, bool)

CachedDurationValueRefresher provides a mechanism for CachedDurationValues to update their values when the current cached value expires. Implementations are responsible for logging errors, as needed.

type CachedIntValue 

type CachedIntValue struct {
	Value       int           `bson:"value"`
	TTL         time.Duration `bson:"ttl"`
	CollectedAt time.Time     `bson:"collected_at"`
	// contains filtered or unexported fields
}

CachedIntValue represents a calculated int value saved in a database with a expiration time. When the data is not expired, the value is returned directly by the Get() method, otherwise, a refresh function is called, to update the value.

func NewCachedIntValue 

func NewCachedIntValue(start int, ttl time.Duration, refresh CachedIntValueRefresher) *CachedIntValue

NewCachedIntValue constructs a CachedIntValue object.

func (*CachedIntValue) Get 

func (v *CachedIntValue) Get() (int, bool)

Get returns the value, refreshing it when its stale. The "ok" value reports errors with the refresh process and alerts callers that the value might be stale.

func (*CachedIntValue) SetRefresher 

func (v *CachedIntValue) SetRefresher(r CachedIntValueRefresher) error

SetRefresher sets CachedIntValueRefresher for the object which is needed when reading CachedIntValue objects out of the database.

It is not permissible to set the refresher to either nil or a value when it is *not* nil.

func (*CachedIntValue) String 

func (v *CachedIntValue) String() string

String implements fmt.Stringer reporting how stale the value is in the stale case.

type CachedIntValueRefresher 

type CachedIntValueRefresher func(int) (int, bool)

CachedIntValueRefresher provides a mechanism for CachedIntValues to update their values when the current cached value expires. Implementations are responsible for logging errors, as needed.

type CappedWriter 

type CappedWriter struct {
	Buffer   *bytes.Buffer
	MaxBytes int
}

CappedWriter implements a buffer that stores up to MaxBytes bytes. Returns ErrBufferFull on overflowing writes

func (*CappedWriter) Close 

func (cw *CappedWriter) Close() error

Close is a noop method so that you can use CappedWriter as an io.WriteCloser.

func (*CappedWriter) IsFull 

func (cw *CappedWriter) IsFull() bool

IsFull indicates whether the buffer is full.

func (*CappedWriter) String 

func (cw *CappedWriter) String() string

String return the contents of the buffer as a string.

func (*CappedWriter) Write 

func (cw *CappedWriter) Write(in []byte) (int, error)

Write writes to the buffer. An error is returned if the buffer is full.

type EvergreenWebhook 

type EvergreenWebhook struct {
	NotificationID string      `bson:"notification_id"`
	URL            string      `bson:"url"`
	Secret         []byte      `bson:"secret"`
	Body           []byte      `bson:"body"`
	Headers        http.Header `bson:"headers"`
}

type Expansions 

type Expansions map[string]string

Wrapper for an expansions map, with some utility functions.

func NewExpansions 

func NewExpansions(initMap map[string]string) *Expansions

Return a new Expansions object with all of the specified expansions present.

func (*Expansions) Exists 

func (self *Expansions) Exists(expansion string) bool

Check if a value is present in the expansions.

func (*Expansions) ExpandString 

func (self *Expansions) ExpandString(toExpand string) (string, error)

Apply the expansions to a single string. Return the expanded string, or an error if the input string is malformed.

func (*Expansions) Get 

func (self *Expansions) Get(expansion string) string

Get a single value from the expansions. Return the value, or the empty string if the value is not present.

func (*Expansions) Map 

func (self *Expansions) Map() map[string]string

func (*Expansions) Put 

func (self *Expansions) Put(expansion string, value string)

Set a single value in the expansions.

func (*Expansions) Update 

func (self *Expansions) Update(newItems map[string]string)

Update all of the specified keys in the expansions to point to the specified values.

func (*Expansions) UpdateFromYaml 

func (self *Expansions) UpdateFromYaml(filename string) error

Read a map of keys/values from the given file, and update the expansions to include them (overwriting any duplicates with the new value).

type KeyValuePair 

type KeyValuePair struct {
	Key   string      `bson:"key" json:"key" yaml:"key"`
	Value interface{} `bson:"value" json:"value" yaml:"value"`
}

type KeyValuePairSlice 

type KeyValuePairSlice []KeyValuePair

func MakeKeyValuePair 

func MakeKeyValuePair(in map[string]string) KeyValuePairSlice

func MakeNestedKeyValuePair 

func MakeNestedKeyValuePair(in map[string]map[string]string) KeyValuePairSlice

func (KeyValuePairSlice) Map 

func (in KeyValuePairSlice) Map() (map[string]string, error)

func (KeyValuePairSlice) NestedMap 

func (in KeyValuePairSlice) NestedMap() (map[string]map[string]string, error)

type RetriableFunc 

type RetriableFunc func() (bool, error)

RetriableFunc is any function that takes no parameters and returns only an error interface. These functions can be used with util.Retry.

type RetryArgs 

type RetryArgs struct {
	MaxAttempts int
	MinDelay    time.Duration
	MaxDelay    time.Duration
}

RetryArgs defines the policy for retrying an operation.

type SafeCounter 

type SafeCounter struct {
	// contains filtered or unexported fields
}

func (*SafeCounter) Inc 

func (s *SafeCounter) Inc()

func (*SafeCounter) String 

func (s *SafeCounter) String() string

func (*SafeCounter) Value 

func (s *SafeCounter) Value() int

type StaticHostInfo 

type StaticHostInfo struct {
	User     string
	Hostname string
	Port     string
}

StaticHostInfo stores the connection parameters for connecting to an SSH host.

func ParseSSHInfo 

func ParseSSHInfo(fullHostname string) (*StaticHostInfo, error)

ParseSSHInfo reads in a hostname definition and reads the relevant SSH connection information from it. For example, 

"admin@myhostaddress:24"

will return 

StaticHostInfo{User: "admin", Hostname:"myhostaddress", Port: 24}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.