Documentation
¶
Overview ¶
Copyright (c) 2016-2019 Uber Technologies, Inc.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Copyright (c) 2016-2019 Uber Technologies, Inc.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Copyright (c) 2016-2019 Uber Technologies, Inc.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Copyright (c) 2016-2019 Uber Technologies, Inc.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Copyright (c) 2016-2019 Uber Technologies, Inc.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Copyright (c) 2016-2019 Uber Technologies, Inc.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Index ¶
Constants ¶
const NoopNamespace = "__noop__"
NoopNamespace is a special namespace which always returns a NoopClient.
Variables ¶
var (
ErrNamespaceNotFound = errors.New("no matches for namespace")
)
Manager errors.
Functions ¶
func Register ¶
func Register(name string, factory ClientFactory)
Register registers new Factory with corresponding backend client name.
Types ¶
type Auth ¶
type Auth map[string]AuthConfig
Auth defines auth credentials for corresponding namespaces. It has to be different due to langley secrets overlay structure.
type AuthConfig ¶
type AuthConfig map[string]interface{}
AuthConfig defines the union of authentication credentials for all type of remote backends.
type BandwidthWatcher ¶
type BandwidthWatcher struct {
// contains filtered or unexported fields
}
BandwidthWatcher is a hashring.Watcher which adjusts bandwidth on throttled backends when hashring membership changes.
func NewBandwidthWatcher ¶
func NewBandwidthWatcher(manager *Manager) *BandwidthWatcher
NewBandwidthWatcher creates a new BandwidthWatcher for manager.
func (*BandwidthWatcher) Notify ¶
func (w *BandwidthWatcher) Notify(latest stringset.Set)
Notify splits bandwidth across the size of latest.
type Client ¶
type Client interface {
// Stat returns blob info for name. All implementations should return
// backenderrors.ErrBlobNotFound when the blob was not found.
//
// Stat is useful when we need to quickly know if a blob exists (and maybe
// some basic information about it), without downloading the entire blob,
// which may be very large.
Stat(namespace, name string) (*core.BlobInfo, error)
// Upload uploads src into name.
Upload(namespace, name string, src io.Reader) error
// Download downloads name into dst. All implementations should return
// backenderrors.ErrBlobNotFound when the blob was not found.
Download(namespace, name string, dst io.Writer) error
// List lists entries whose names start with prefix.
List(prefix string) ([]string, error)
}
Client defines an interface for accessing blobs on a remote storage backend.
Implementations of Client must be thread-safe, since they are cached and used concurrently by Manager.
type ClientFactory ¶
ClientFactory creates backend client given name.
type Config ¶
type Config struct {
Namespace string `yaml:"namespace"`
Backend map[string]interface{} `yaml:"backend"`
// If enabled, throttles upload / download bandwidth.
Bandwidth bandwidth.Config `yaml:"bandwidth"`
}
Config defines the union of configuration for all backends. Only one is allowed for each config file.
type Manager ¶
type Manager struct {
// contains filtered or unexported fields
}
Manager manages backend clients for namespace regular expressions.
func ManagerFixture ¶
func ManagerFixture() *Manager
ManagerFixture returns a Manager with no clients for testing purposes.
func NewManager ¶
func NewManager(configs []Config, auth AuthConfig) (*Manager, error)
NewManager creates a new backend Manager.
func (*Manager) AdjustBandwidth ¶
AdjustBandwidth adjusts bandwidth limits across all throttled clients to the originally configured bandwidth divided by denominator.
func (*Manager) GetClient ¶
GetClient matches namespace to the configured Client. Returns ErrNamespaceNotFound if no clients match namespace.
type NoopClient ¶
type NoopClient struct{}
NoopClient is a special Client whose uploads always succeeds and whose blob lookups always 404. It is useful for users who want to operate on blobs that will be temporarily stored in the origin cluster and not backed up in remote storage.
func (NoopClient) Download ¶
func (c NoopClient) Download(namespace, name string, dst io.Writer) error
Download always returns ErrBlobNotFound.
func (NoopClient) List ¶
func (c NoopClient) List(prefix string) ([]string, error)
List always returns nil.
type ThrottledClient ¶
type ThrottledClient struct {
Client
// contains filtered or unexported fields
}
ThrottledClient is a backend client with speed limit.
func (*ThrottledClient) Download ¶
func (c *ThrottledClient) Download(namespace, name string, dst io.Writer) error
Download downloads name into dst.
func (*ThrottledClient) EgressLimit ¶
func (c *ThrottledClient) EgressLimit() int64
EgressLimit returns egress limit.
func (*ThrottledClient) IngressLimit ¶
func (c *ThrottledClient) IngressLimit() int64
IngressLimit returns ingress limit.
Source Files
Directories