util

package
v0.4.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 9, 2019 License: Apache-2.0 Imports: 33 Imported by: 0

Documentation

Overview

Package util provides some utility tools for other components. Such as net-transporting, file-operating, rate-limiter.

Index

Constants

View Source 
const (
	// RequestTag is the tag name for parsing structure to query parameters.
	// see function ParseQuery.
	RequestTag = "request"

	// DefaultTimeout is the default timeout to check connect.
	DefaultTimeout = 500 * time.Millisecond
)
View Source 
const (
	ApplicationJSONUtf8Value = "application/json;charset=utf-8"
)

http content types

View Source 
const BufferSize = 8 * 1024 * 1024

BufferSize define the buffer size when reading and writing file

Variables

This section is empty.

Functions

func Atime 

func Atime(stat *syscall.Stat_t) time.Time

Atime returns the last access time in time.Time.

func AtimeSec 

func AtimeSec(stat *syscall.Stat_t) int64

AtimeSec returns the last access time in seconds.

func CheckConnect 

func CheckConnect(ip string, port int, timeout int) (localIP string, e error)

CheckConnect checks the network connectivity between local and remote. param timeout: its unit is milliseconds, reset to 500 ms if <= 0 returns localIP

func ConstructRangeStr 

func ConstructRangeStr(rangeStr string) string

ConstructRangeStr wrap the rangeStr as a HTTP Range header value.

func ConvertHeaders 

func ConvertHeaders(headers []string) map[string]string

ConvertHeaders converts headers from array type to map type for http request.

func ConvertTimeIntToString 

func ConvertTimeIntToString(timestamp int64) (string, error)

ConvertTimeIntToString converts a int64 timestamp to a string time.

func ConvertTimeStringToInt 

func ConvertTimeStringToInt(timeStr string) (int64, error)

ConvertTimeStringToInt converts a string time to a int64 timestamp.

func CopyFile 

func CopyFile(src string, dst string) (err error)

CopyFile copies the file src to dst.

func CreateDirectory 

func CreateDirectory(dirPath string) error

CreateDirectory creates directory recursively.

func Ctime 

func Ctime(stat *syscall.Stat_t) time.Time

Ctime returns the create time in time.Time.

func CtimeSec 

func CtimeSec(stat *syscall.Stat_t) int64

CtimeSec returns the create time in seconds.

func DeleteFile 

func DeleteFile(filePath string) error

DeleteFile deletes a file not a directory.

func DeleteFiles 

func DeleteFiles(filePaths ...string)

DeleteFiles deletes all the given files.

func Do 

func Do(url string, headers map[string]string, timeout time.Duration) (string, error)

Do performs the given http request and fills the given http response. When timeout <= 0, it will block until receiving response from server.

func ExtractHost 

func ExtractHost(hostAndPort string) string

ExtractHost extracts host ip from the giving string.

func FilterURLParam 

func FilterURLParam(url string, filters []string) string

FilterURLParam filters request queries in URL. Eg: If you pass parameters as follows: 

url: http://a.b.com/locate?key1=value1&key2=value2&key3=value3
filter: key2

and then you will get the following value as the return: 

http://a.b.com/locate?key1=value1&key3=value3

func Get 

func Get(url string, timeout time.Duration) (int, []byte, error)

Get sends a GET request to server. When timeout <= 0, it will block until receiving response from server.

func GetAllIPs 

func GetAllIPs() (ipList []string, err error)

GetAllIPs returns all non-loopback addresses.

func GetContentLength 

func GetContentLength(url string, headers map[string]string) (int64, int, error)

GetContentLength send a head request to get file length.

func GetIPAndPortFromNode 

func GetIPAndPortFromNode(node string, defaultPort int) (string, int)

GetIPAndPortFromNode return ip and port by parsing the node value. It will return defaultPort as the value of port when the node is a string without port or with an illegal port.

func GetSys 

func GetSys(info os.FileInfo) (*syscall.Stat_t, bool)

GetSys returns the underlying data source of the os.FileInfo.

func GetWithHeaders 

func GetWithHeaders(url string, headers map[string]string, timeout time.Duration) (code int, resBody []byte, err error)

GetWithHeaders sends a GET request to server. When timeout <= 0, it will block until receiving response from server.

func HTTPGet 

func HTTPGet(url string, headers map[string]string) (*http.Response, error)

HTTPGet send an HTTP GET request with headers.

func HTTPGetTimeout 

func HTTPGetTimeout(url string, headers map[string]string, timeout time.Duration) (*http.Response, error)

HTTPGetTimeout send an HTTP GET request with timeout.

func HTTPStatusOk 

func HTTPStatusOk(code int) bool

HTTPStatusOk reports whether the http response code is 200.

func HTTPWithHeaders 

func HTTPWithHeaders(method, url string, headers map[string]string, timeout time.Duration) (*http.Response, error)

HTTPWithHeaders send an HTTP request with headers and specified method.

func IsDir 

func IsDir(name string) bool

IsDir reports whether the path is a directory.

func IsEmptySlice 

func IsEmptySlice(values []string) bool

IsEmptySlice returns whether the slice values is empty.

func IsEmptyStr 

func IsEmptyStr(s string) bool

IsEmptyStr returns whether the string s is empty.

func IsExpired 

func IsExpired(url string, headers map[string]string, lastModified int64, eTag string) (bool, error)

IsExpired checks if a resource received or stored is the same.

func IsNatural 

func IsNatural(value string) bool

IsNatural returns whether the value>=0.

func IsNil 

func IsNil(value interface{}) (result bool)

IsNil returns whether the value is nil.

func IsNumeric 

func IsNumeric(value string) bool

IsNumeric returns whether the value is a numeric. If the bitSize of value below 0 or above 64 an error is returned.

func IsPositive 

func IsPositive(value int64) bool

IsPositive returns whether the value is a positive number.

func IsRegularFile 

func IsRegularFile(name string) bool

IsRegularFile reports whether the file is a regular file. If the given file is a symbol link, it will follow the link.

func IsSupportRange 

func IsSupportRange(url string, headers map[string]string) (bool, error)

IsSupportRange checks if the source url support partial requests.

func IsTrue 

func IsTrue(value bool) bool

IsTrue returns whether the value is true.

func IsValidIP 

func IsValidIP(ip string) bool

IsValidIP returns whether the string ip is a valid IP Address.

func IsValidURL 

func IsValidURL(url string) bool

IsValidURL returns whether the string url is a valid HTTP URL.

func JSONString 

func JSONString(v interface{}) string

JSONString returns json string of the v. 

func Link(src string, linkName string) error

Link creates a hard link pointing to src named linkName for a file.

func LoadYaml 

func LoadYaml(path string, out interface{}) error

LoadYaml load yaml config file.

func Max 

func Max(x, y int32) int32

Max returns the larger of x or y.

func Md5Sum 

func Md5Sum(name string) string

Md5Sum generate md5 for a given file

func Min 

func Min(x, y int32) int32

Min returns the smaller of x or y.

func MoveFile 

func MoveFile(src string, dst string) error

MoveFile moves the file src to dst.

func MoveFileAfterCheckMd5 

func MoveFileAfterCheckMd5(src string, dst string, md5 string) error

MoveFileAfterCheckMd5 will check whether the file's md5 is equals to the param md5 before move the file src to dst.

func NetLimit 

func NetLimit() string

NetLimit parse speed of interface that it has prefix of eth

func OpenFile 

func OpenFile(path string, flag int, perm os.FileMode) (*os.File, error)

OpenFile open a file. If the parent directory of the file isn't exist, it will create the directory.

func ParseQuery 

func ParseQuery(query interface{}) string

ParseQuery only parses the fields with tag 'request' of the query to parameters. query must be a pointer to a struct.

func PathExist 

func PathExist(name string) bool

PathExist reports whether the path is exist. Any error get from os.Stat, it will return false.

func PostJSON 

func PostJSON(url string, body interface{}, timeout time.Duration) (int, []byte, error)

PostJSON send a POST request whose content-type is 'application/json;charset=utf-8'.

func PostJSONWithHeaders 

func PostJSONWithHeaders(url string, headers map[string]string, body interface{}, timeout time.Duration) (int, []byte, error)

PostJSONWithHeaders send a POST request whose content-type is 'application/json;charset=utf-8'.

func Sha1 

func Sha1(contents []string) string

Sha1 returns the SHA-1 checksum of the contents.

func Sha256 

func Sha256(value string) string

Sha256 returns the SHA-256 checksum of the data.

func SubString 

func SubString(str string, start, end int) string

SubString returns the subString of {str} which begins at {start} and end at {end - 1}. 

func SymbolicLink(src string, target string) error

SymbolicLink creates target as a symbolic link to src.

func TransRate 

func TransRate(rate int) int32

TransRate trans the rate to multiples of 1000 For NewRateLimiter, the production of rate should be division by 1000.

Types

type AtomicInt 

type AtomicInt struct {
	// contains filtered or unexported fields
}

AtomicInt is a struct that can be added or subtracted atomically.

func NewAtomicInt 

func NewAtomicInt(value int32) *AtomicInt

NewAtomicInt returns a new AtomicInt.

func (*AtomicInt) Add 

func (ac *AtomicInt) Add(delta int32) int32

Add atomically adds delta to count and returns the new value.

func (*AtomicInt) Get 

func (ac *AtomicInt) Get() int32

Get the value atomically.

func (*AtomicInt) Set 

func (ac *AtomicInt) Set(value int32) int32

Set to value atomically and returns the previous value.

type LimitReader 

type LimitReader struct {
	Src     io.Reader
	Limiter *RateLimiter
	// contains filtered or unexported fields
}

LimitReader read stream with RateLimiter.

func NewLimitReader 

func NewLimitReader(src io.Reader, rate int, calculateMd5 bool) *LimitReader

NewLimitReader create LimitReader src: reader rate: bytes/second

func NewLimitReaderWithLimiter 

func NewLimitReaderWithLimiter(rateLimiter *RateLimiter, src io.Reader, calculateMd5 bool) *LimitReader

NewLimitReaderWithLimiter create LimitReader with a rateLimiter. src: reader rate: bytes/second

func NewLimitReaderWithLimiterAndMD5Sum 

func NewLimitReaderWithLimiterAndMD5Sum(src io.Reader, rateLimiter *RateLimiter, md5sum hash.Hash) *LimitReader

NewLimitReaderWithLimiterAndMD5Sum create LimitReader with rateLimiter and md5 sum. src: reader rate: bytes/second

func NewLimitReaderWithMD5Sum 

func NewLimitReaderWithMD5Sum(src io.Reader, rate int, md5sum hash.Hash) *LimitReader

NewLimitReaderWithMD5Sum create LimitReader with a md5 sum. src: reader rate: bytes/second

func (*LimitReader) Md5 

func (lr *LimitReader) Md5() string

Md5 calculate the md5 of all contents read

func (*LimitReader) Read 

func (lr *LimitReader) Read(p []byte) (n int, err error)

type MockHTTPClient 

type MockHTTPClient struct {
	PostJSONFunc            postJSONFunc
	GetFunc                 getFunc
	PostJSONWithHeadersFunc postJSONWithHeadersFunc
	GetWithHeadersFunc      getWithHeadersFunc
}

MockHTTPClient fakes a customized implementation of util.SimpleHTTPClient.

func NewMockHTTPClient 

func NewMockHTTPClient() *MockHTTPClient

NewMockHTTPClient returns a new MockHTTPClient instance.

func (*MockHTTPClient) CreateGetFunc 

func (m *MockHTTPClient) CreateGetFunc(code int, res []byte, e error) getFunc

CreateGetFunc returns a mock getFunc func which will always return the specific results.

func (*MockHTTPClient) CreateGetWithHeadersFunc 

func (m *MockHTTPClient) CreateGetWithHeadersFunc(code int, res []byte, e error) getWithHeadersFunc

CreateGetWithHeadersFunc returns a mock getWithHeadersFunc func which will always return the specific results.

func (*MockHTTPClient) CreatePostJSONFunc 

func (m *MockHTTPClient) CreatePostJSONFunc(code int, res []byte, e error) postJSONFunc

CreatePostJSONFunc returns a mock postJSONFunc func which will always return the specific results.

func (*MockHTTPClient) CreatePostJSONWithHeadersFunc 

func (m *MockHTTPClient) CreatePostJSONWithHeadersFunc(code int, res []byte, e error) postJSONWithHeadersFunc

CreatePostJSONWithHeadersFunc returns a mock postJSONWithHeadersFunc func / which will always return the specific results.

func (*MockHTTPClient) Get 

func (m *MockHTTPClient) Get(url string, timeout time.Duration) (int, []byte, error)

Get mocks base method.

func (*MockHTTPClient) GetWithHeaders 

func (m *MockHTTPClient) GetWithHeaders(url string, headers map[string]string, timeout time.Duration) (
	int, []byte, error)

GetWithHeaders mocks base method.

func (*MockHTTPClient) PostJSON 

func (m *MockHTTPClient) PostJSON(url string, body interface{}, timeout time.Duration) (
	int, []byte, error)

PostJSON mocks base method.

func (*MockHTTPClient) PostJSONWithHeaders 

func (m *MockHTTPClient) PostJSONWithHeaders(url string, headers map[string]string, body interface{}, timeout time.Duration) (
	int, []byte, error)

PostJSONWithHeaders mocks base method.

func (*MockHTTPClient) Reset 

func (m *MockHTTPClient) Reset()

Reset the MockHTTPClient.

type RangeStruct 

type RangeStruct struct {
	StartIndex int64
	EndIndex   int64
}

RangeStruct contains the start and end of a http header range.

func GetRangeSE 

func GetRangeSE(rangeHTTPHeader string, length int64) ([]*RangeStruct, error)

GetRangeSE parses the start and the end from range HTTP header and returns them.

type RateLimiter 

type RateLimiter struct {
	// contains filtered or unexported fields
}

RateLimiter is used for limiting the rate of transporting.

func NewRateLimiter 

func NewRateLimiter(rate int32, window int64) *RateLimiter

NewRateLimiter creates a RateLimiter instance. rate: how many tokens are generated per second. 0 represents that don't limit the rate. window: generating tokens interval (millisecond, [1,1000]). The production of rate and window should be division by 1000.

func (*RateLimiter) AcquireBlocking 

func (rl *RateLimiter) AcquireBlocking(token int32) int32

AcquireBlocking acquires tokens. It will be blocking unit the bucket has enough required number of tokens.

func (*RateLimiter) AcquireNonBlocking 

func (rl *RateLimiter) AcquireNonBlocking(token int32) int32

AcquireNonBlocking acquires tokens. It will return -1 immediately when there is no enough number of tokens.

func (*RateLimiter) SetRate 

func (rl *RateLimiter) SetRate(rate int32)

SetRate sets rate of RateLimiter.

type SimpleHTTPClient 

type SimpleHTTPClient interface {
	PostJSON(url string, body interface{}, timeout time.Duration) (code int, res []byte, e error)
	Get(url string, timeout time.Duration) (code int, res []byte, e error)
	PostJSONWithHeaders(url string, headers map[string]string, body interface{}, timeout time.Duration) (code int, resBody []byte, err error)
	GetWithHeaders(url string, headers map[string]string, timeout time.Duration) (code int, resBody []byte, err error)
}

SimpleHTTPClient defines some http functions used frequently. 

var DefaultHTTPClient SimpleHTTPClient = &defaultHTTPClient{}

DefaultHTTPClient is the default implementation of SimpleHTTPClient.

type SyncMap 

type SyncMap struct {
	*sync.Map
}

SyncMap is a thread-safe map.

func NewSyncMap 

func NewSyncMap() *SyncMap

NewSyncMap returns a new SyncMap.

func (*SyncMap) Add 

func (mmap *SyncMap) Add(key string, value interface{}) error

Add add a key-value pair into the *sync.Map. The ErrEmptyValue error will be returned if the key is empty.

func (*SyncMap) Get 

func (mmap *SyncMap) Get(key string) (interface{}, error)

Get returns result as interface{} according to the key. The ErrEmptyValue error will be returned if the key is empty. And the ErrDataNotFound error will be returned if the key cannot be found.

func (*SyncMap) GetAsAtomicInt 

func (mmap *SyncMap) GetAsAtomicInt(key string) (*AtomicInt, error)

GetAsAtomicInt returns result as *AtomicInt. The ErrConvertFailed error will be returned if the assertion fails.

func (*SyncMap) GetAsBitset 

func (mmap *SyncMap) GetAsBitset(key string) (*bitset.BitSet, error)

GetAsBitset returns result as *bitset.BitSet. The ErrConvertFailed error will be returned if the assertion fails.

func (*SyncMap) GetAsBool 

func (mmap *SyncMap) GetAsBool(key string) (bool, error)

GetAsBool returns result as bool. The ErrConvertFailed error will be returned if the assertion fails.

func (*SyncMap) GetAsInt 

func (mmap *SyncMap) GetAsInt(key string) (int, error)

GetAsInt returns result as int. The ErrConvertFailed error will be returned if the assertion fails.

func (*SyncMap) GetAsMap 

func (mmap *SyncMap) GetAsMap(key string) (*SyncMap, error)

GetAsMap returns result as SyncMap. The ErrConvertFailed error will be returned if the assertion fails.

func (*SyncMap) GetAsString 

func (mmap *SyncMap) GetAsString(key string) (string, error)

GetAsString returns result as string. The ErrConvertFailed error will be returned if the assertion fails.

func (*SyncMap) ListKeyAsIntSlice 

func (mmap *SyncMap) ListKeyAsIntSlice() (result []int)

ListKeyAsIntSlice returns the list of keys as a int slice.

func (*SyncMap) ListKeyAsStringSlice 

func (mmap *SyncMap) ListKeyAsStringSlice() (result []string)

ListKeyAsStringSlice returns the list of keys as a string slice.

func (*SyncMap) Remove 

func (mmap *SyncMap) Remove(key string) error

Remove deletes the key-value pair from the mmap. The ErrEmptyValue error will be returned if the key is empty. And the ErrDataNotFound error will be returned if the key cannot be found.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.