dragonfly

module
v2.0.2-rc.8 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 17, 2022 License: Apache-2.0

README

Dragonfly

alt

GitHub release Artifact Hub CI Coverage Go Report Card Open Source Helpers TODOs Discussions Twitter GoDoc LICENSE

Provide efficient, stable, secure, low-cost file and image distribution services to be the best practice and standard solution in the related Cloud-Native area.

Introduction

Dragonfly is an open source intelligent P2P based image and file distribution system. Its goal is to tackle all distribution problems in cloud native scenarios. Currently Dragonfly focuses on being:

  • Simple: well-defined user-facing API (HTTP), non-invasive to all container engines;
  • Efficient: CDN support, P2P based file distribution to save enterprise bandwidth;
  • Intelligent: host-level speed limit, intelligent flow control due to host detection;
  • Secure: block transmission encryption, HTTPS connection support.

Dragonfly is now hosted by the Cloud Native Computing Foundation (CNCF) as an Incubating Level Project. Originally it was born to solve all kinds of distribution at very large scales, such as application distribution, cache distribution, log distribution, image distribution, and so on.

Features

  • Implement P2P files distribution with various storage types (HDFS, storage services from various cloud vendors, Maven, Yum, etc.) through a unified back-to-source adapter layer.
  • Support more distribution modes: active pull, active push, real-time synchronization, remote replication, automatic warm-up, cross-cloud transmission, etc.
  • Provide separation and decoupling between systems, scheduling and plug-in CDN. Support on-demand deployment with flexible types: light or heavy, inside or outside, to meet the actual needs of different scenarios.
  • Newly designed P2P protocol framework based on GRPC with improved efficiency and stability.
  • Perform encrypted transmission, account-based transmission authentication and rate limit, and multi-tenant isolation mechanism.
  • Bear more efficient IO methods: multithreaded IO, memory mapping, DMA, etc.
  • Advocate dynamic compression, in-memory file systems, and more efficient scheduling algorithms to improve distribution efficiency.
  • Client allows third-party software to natively integrate Dragonfly’s P2P capabilities through C/S mode.
  • Productivity: Support file uploading, task management of various distribution modes, data visualization, global control, etc.
  • Consistent internal and external versions, shared core features, and individual extensions of non-generic features.
  • Enhanced integration with ecology: Harbor, Nydus (on-demand image download), warehouse services for various cloud vendors, etc.

Architecture

alt

Manager: Used to manage the dynamic configuration that each module depends on, and provide keepalive and metrics functions.

Scheduler: The tracker and scheduler in the P2P network that chooses appropriate downloading net-path for each peer.

CDN: A CDN server that caches downloaded data from source to avoid downloading the same files repeatedly.

Daemon: It's a daemon of dfget client. It establishes a proxy between containerd/CRI-O and registry.

Dfget: The client of Dragonfly used for downloading files. It's similar to wget.

Documentation

You can find the full documentation on the repo.

Community

Welcome developers to actively participate in community discussions and contribute code to Dragonfly. We will remain concerned about the issues discussed in the community and respond quickly.

Contributing

You should check out our CONTRIBUTING and develop the project together.

Code of Conduct

Please refer to our Code of Conduct.

Directories

Path Synopsis
api
manager
Package manager GENERATED BY THE COMMAND ABOVE; DO NOT EDIT This file was generated by swaggo/swag
Package manager GENERATED BY THE COMMAND ABOVE; DO NOT EDIT This file was generated by swaggo/swag
cdn
gc
storedriver
Package storedriver is a generated GoMock package.
Package storedriver is a generated GoMock package.
supervisor/cdn/storage/mock
Package mock is a generated GoMock package.
Package mock is a generated GoMock package.
supervisor/mocks
Package progress is a generated GoMock package.
Package progress is a generated GoMock package.
supervisor/mocks/cdn
Package cdn is a generated GoMock package.
Package cdn is a generated GoMock package.
supervisor/mocks/progress
Package progress is a generated GoMock package.
Package progress is a generated GoMock package.
supervisor/mocks/task
Package task is a generated GoMock package.
Package task is a generated GoMock package.
client
config
Package config holds all DaemonConfig of dfget.
Package config holds all DaemonConfig of dfget.
config/mocks
Package mocks is a generated GoMock package.
Package mocks is a generated GoMock package.
daemon/test/mock/daemon
Package mock_server is a generated GoMock package.
Package mock_server is a generated GoMock package.
daemon/test/mock/peer
Package mock_peer is a generated GoMock package.
Package mock_peer is a generated GoMock package.
daemon/test/mock/scheduler
Package mock_client is a generated GoMock package.
Package mock_client is a generated GoMock package.
daemon/test/mock/service
Package mock_service is a generated GoMock package.
Package mock_service is a generated GoMock package.
daemon/test/mock/storage
Package mock_storage is a generated GoMock package.
Package mock_storage is a generated GoMock package.
cmd
cdn
internal
dynconfig/mocks
Package mock_manager_client is a generated GoMock package.
Package mock_manager_client is a generated GoMock package.
job
job
pkg
container/list/mocks
Package mocks is a generated GoMock package.
Package mocks is a generated GoMock package.
dfpath/mocks
Package mocks is a generated GoMock package.
Package mocks is a generated GoMock package.
gc
Package gc is a generated GoMock package.
Package gc is a generated GoMock package.
log
rpc
rpc/cdnsystem/mocks
Package mocks is a generated GoMock package.
Package mocks is a generated GoMock package.
rpc/cdnsystem/server/mocks
Package mocks is a generated GoMock package.
Package mocks is a generated GoMock package.
rpc/dfdaemon/cdnsystem/mocks
Package mocks is a generated GoMock package.
Package mocks is a generated GoMock package.
rpc/dfdaemon/client/mocks
Package mocks is a generated GoMock package.
Package mocks is a generated GoMock package.
rpc/dfdaemon/mocks
Package mocks is a generated GoMock package.
Package mocks is a generated GoMock package.
rpc/dfdaemon/server/mocks
Package mocks is a generated GoMock package.
Package mocks is a generated GoMock package.
rpc/manager/client/mocks
Package mocks is a generated GoMock package.
Package mocks is a generated GoMock package.
rpc/manager/mocks
Package mocks is a generated GoMock package.
Package mocks is a generated GoMock package.
rpc/manager/server/mocks
Package mocks is a generated GoMock package.
Package mocks is a generated GoMock package.
rpc/scheduler/client/mocks
Package mocks is a generated GoMock package.
Package mocks is a generated GoMock package.
rpc/scheduler/mocks
Package mocks is a generated GoMock package.
Package mocks is a generated GoMock package.
source/mock
Package mock is a generated GoMock package.
Package mock is a generated GoMock package.
util/mathutils
Package mathutils provides utilities supplementing the standard 'math' and 'math/rand' packages.
Package mathutils provides utilities supplementing the standard 'math' and 'math/rand' packages.
util/stringutils
Package stringutils provides utilities supplementing the standard 'strings' package.
Package stringutils provides utilities supplementing the standard 'strings' package.
util/structutils
Package structutils provides utilities supplementing the standard 'time' package.
Package structutils provides utilities supplementing the standard 'time' package.
util/timeutils
Package timeutils provides utilities supplementing the standard 'time' package.
Package timeutils provides utilities supplementing the standard 'time' package.
config/mocks
Package mocks is a generated GoMock package.
Package mocks is a generated GoMock package.
job
resource
Package resource is a generated GoMock package.
Package resource is a generated GoMock package.
scheduler/mocks
Package mocks is a generated GoMock package.
Package mocks is a generated GoMock package.
test
e2e

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL