README
¶
tools-common
Welcome to tools-common, this package contains various utilities used by the Go code maintained by the tools team.
Dependencies
This package utilizes the go-sqlite3 package which requires GCC to be
installed, and CGO to be enabled (CGO_ENABLED=1). See the latest
README for the most up-to-date information.
Testing
Running the unit testing requires a number of dependencies:
- GCC (Latest version available via package manager)
- Go (1.18+)
- Make (Latest version available via package manager)
Once the required dependencies have been installed, testing can be done using the repository Makefile. Running make test will run all the available unit tests.
Generating a coverage report from the unit testing may be done by running make coverage, this report will be
automatically opened in the default browser.
When running tests, you may want to filter which tests are run; this may be at the package or test level. To run all the
tests in the netutil package, run make PACKAGE=netutil test. To run only tests which match a given regular
expression, run make TESTS=TestContains test.
Note that these variables may be used in conjunction and both also apply to the coverage target.
Contributing
The following sections cover some basics of how to contribute to tools-common whilst following some of our common
practices/conventions.
Gerrit
To contribute to this codebase you can upload patches through Gerrit. Make sure you have configured the git hooks as described in the Git Hooks section so that the code is linted and formatted before uploading the patch.
Once you are ready to make your first commit note that all commits must be linked to an MB. This is done by making
sure that the commit title has the following format MB-XXXXX Commit title where MB-XXXXX is a valid issue in
Jira.
Git Hooks
Before contributing any patches, the Git hooks should be configured to ensure code is correctly linted and formatted.
The Git hooks require the following dependencies:
- gofmt (Standard code formatting tool)
- gofumpt (A more opinionated code formatting tool)
- goimports (Automatic insertion/sorting of imported modules)
- golangci-lint (Bulk linting tool)
- sponge (Binary provided by
moreutilswhich "soaks" all input before writing output) - wget (Used to download the
commit-msghook from Gerrit)
Once installed, the Git hooks may be setup using the following command:
git config core.hooksPath .githooks
If the Git hooks have been setup correctly, the Gerrit commit-msg hook will be downloaded automatically when creating
your first commit. However, this can also be done manually by downloading the
commit-msg script, marking it as executable and placing it into
the .githooks directory.
Formatting
We automatically apply formatting on any staged Go files before committing. This may not be what you want if you ever have some hunks in a file staged but not others, as it will cause all of them to be committed. This behaviour can be configured using:
git config couchbase.tools.format.behaviour BEHAVIOUR
Where BEHAVIOUR is one of the following:
- error: echo what files have incorrect formatting and quit
- fix: echo what files have incorrect formatting, fix them and quit
- stage: echo what files have incorrect formatting, fix and stage them and quit
- commit/no config value/invalid config value: fix the files with incorrect formatting, stage them and allow the commit to proceed.
Coding style
In this section we will cover notes on the exact coding style to use for this codebase. Most of the style rules are enforced by the linters, so here we will only cover ones that are not.
Documenting
- All exported functions should have a matching docstring.
- Any non-trivial unexported function should also have a matching docstring. Note this is left up to the developer and reviewer consideration.
- Docstrings must end on a full stop (
.). - Comments must be wrapped at 120 characters.
- Notes on interesting/unexpected behavior should have a newline before them and use the
// NOTE:prefix.
Please note that not all the code in the repository follows these rules, however, newly added/updated code should generally adhere to them.
Related Projects
License
Copyright 2021 Couchbase Inc.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Directories
¶
| Path | Synopsis |
|---|---|
|
auth
module
|
|
|
provider
Package provider exposes types/interfaces which may be used to provide credentials.
|
Package provider exposes types/interfaces which may be used to provide credentials. |
|
cbbs
module
|
|
|
grpc/util
Package util provides gRPC utilties used by 'cbbs'.
|
Package util provides gRPC utilties used by 'cbbs'. |
|
rest/util
Package util provides REST utilties used by 'cbbs'.
|
Package util provides REST utilties used by 'cbbs'. |
|
cloud
module
|
|
|
objstore/objcli
Package objcli exposes a unified 'Client' interface for accessing/managing objects stored in the cloud.
|
Package objcli exposes a unified 'Client' interface for accessing/managing objects stored in the cloud. |
|
objstore/objcli/objaws
Package objaws provides an implementation of 'objstore.Client' for use with AWS S3.
|
Package objaws provides an implementation of 'objstore.Client' for use with AWS S3. |
|
objstore/objcli/objazure
Package objazure provides an implementation of 'objstore.Client' for use with Azure blob storage.
|
Package objazure provides an implementation of 'objstore.Client' for use with Azure blob storage. |
|
objstore/objcli/objgcp
Package objgcp provides an implementation of 'objstore.Client' for use with GCS.
|
Package objgcp provides an implementation of 'objstore.Client' for use with GCS. |
|
objstore/objerr
Package objerr provides error definitions used in 'objstore'.
|
Package objerr provides error definitions used in 'objstore'. |
|
objstore/objutil
Package objutil provide utility functions for object store clients which expose more complex/configurable behavior than using a base 'objcli.Client'.
|
Package objutil provide utility functions for object store clients which expose more complex/configurable behavior than using a base 'objcli.Client'. |
|
objstore/objval
Package objval provides types/definitions used by 'objstore'.
|
Package objval provides types/definitions used by 'objstore'. |
|
core
module
|
|
|
log
Package log provides an interface to setup logging when using 'tools-common'.
|
Package log provides an interface to setup logging when using 'tools-common'. |
|
couchbase
module
|
|
|
connstr
Package connstr provides utilties to parse and resolve Couchbase SDK style connection string.
|
Package connstr provides utilties to parse and resolve Couchbase SDK style connection string. |
|
evtlog
Package evtlog provides a client/service which allows asynchronously reporting events to the '/_event' endpoint provided by 'ns_server'.
|
Package evtlog provides a client/service which allows asynchronously reporting events to the '/_event' endpoint provided by 'ns_server'. |
|
keygen
Package keygen is used for generating in importing tools at Couchabse.
|
Package keygen is used for generating in importing tools at Couchabse. |
|
value
Package value exposes types related to Couchbase e.g.
|
Package value exposes types related to Couchbase e.g. |
|
databases
module
|
|
|
environment
module
|
|
|
variable
Package variable exposes utilities for getting configuration from the environment.
|
Package variable exposes utilities for getting configuration from the environment. |
|
errors
module
|
|
|
definitions
Package definitions provides useful error types such as 'MultiError'.
|
Package definitions provides useful error types such as 'MultiError'. |
|
fs
module
|
|
|
util
Package util provides utility functions for interacting with filesystems.
|
Package util provides utility functions for interacting with filesystems. |
|
functional
module
|
|
|
maps
Package maputil provides basic utility functions for generic maps.
|
Package maputil provides basic utility functions for generic maps. |
|
slices
Package slices provides generic slice utility functions.
|
Package slices provides generic slice utility functions. |
|
http
module
|
|
|
tls
Package tls provides utilities required for TLS such as created TLS configs allowing all the certificate format supported by Couchbase.
|
Package tls provides utilities required for TLS such as created TLS configs allowing all the certificate format supported by Couchbase. |
|
util
Package util provides network related utility functions.
|
Package util provides network related utility functions. |
|
strings
module
|
|
|
format
Package format provides the means to format types into human readable strings.
|
Package format provides the means to format types into human readable strings. |
|
parse
Package parse exposes functions to parse strings into Go types.
|
Package parse exposes functions to parse strings into Go types. |
|
util
Package util provides utility functions for interacting with strings.
|
Package util provides utility functions for interacting with strings. |
|
hofp
Package hofp exposes a generic higher order function pool which abstracts aways the logic/error handling required to perform tasks concurrently by wrapping complex tasks into a common 'func(context.Context) error' interface.
|
Package hofp exposes a generic higher order function pool which abstracts aways the logic/error handling required to perform tasks concurrently by wrapping complex tasks into a common 'func(context.Context) error' interface. |
|
testing
module
|
|
|
mock/matchers
Package matchers provides useful 'Matcher' implementations for 'testify/mock'.
|
Package matchers provides useful 'Matcher' implementations for 'testify/mock'. |
|
types
module
|
|
|
deque
Package deque provides a double ended queue implemented using a ring buffer.
|
Package deque provides a double ended queue implemented using a ring buffer. |
|
freelist
Package freelist implements a thread-safe pool of objects of a defined size.
|
Package freelist implements a thread-safe pool of objects of a defined size. |
|
iface
Package iface provides useful - generally composition - interface definitions such as 'WriteAtSeeker'.
|
Package iface provides useful - generally composition - interface definitions such as 'WriteAtSeeker'. |
|
lru
Package lru implements a constant time, generic LRU cache.
|
Package lru implements a constant time, generic LRU cache. |
|
pq
Package pq exposes a generic priority queue implemented using a heap.
|
Package pq exposes a generic priority queue implemented using a heap. |
|
ptr
Package ptr provides generic utility functions for operating on/creating pointers.
|
Package ptr provides generic utility functions for operating on/creating pointers. |
|
ringbuf
Package ringbuf provides a generic implementation of a circular ring buffer.
|
Package ringbuf provides a generic implementation of a circular ring buffer. |
|
utils
module
|
|
|
maths
Package maths provides utility functions for performing mathematical operations.
|
Package maths provides utility functions for performing mathematical operations. |
|
ratelimit
Package ratelimit exposes rate limited io implementations.
|
Package ratelimit exposes rate limited io implementations. |
|
retry
Package retry exposes a 'Retryer' allowing conditionally retrying (with back-off) of functions.
|
Package retry exposes a 'Retryer' allowing conditionally retrying (with back-off) of functions. |
|
system
Package system provides system utility functions initially required by `cbbackupmgr`.
|
Package system provides system utility functions initially required by `cbbackupmgr`. |