database-lab

module
Version: v2.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 9, 2021 License: AGPL-3.0

README

Database Lab Engine - thin database clones for faster development

Database Lab Engine (DLE) is an open-source technology that allows blazing-fast cloning of Postgres databases of any size in seconds. This helps solve many problems such as:

  • help build dev/QA/staging environments involving full-size production-like databases,
  • provide temporary full-size database clones for SQL query analysis optimization (see Joe bot),
  • automatically verify database migrations (DB schema changes) and massive data operations to avoid downtime and performance bottlenecks.

As an example, cloning of 10 TiB PostgreSQL database takes less than 2 seconds.

Read more at Postgres.ai and in the Docs.

Please support the project giving a GitLab star! It's on the main page, at the upper right corner:

Add a star

Installation

See the Database Lab tutorials:

The latest stable Docker images are published in the Docker Hub repository.

Client CLI

Installation

Install Database Lab client CLI on your Linux machine (e.g., Ubuntu):

curl https://gitlab.com/postgres-ai/database-lab/-/raw/master/scripts/cli_install.sh | bash
Activate CLI autocompletion:
sudo curl https://gitlab.com/postgres-ai/database-lab/-/raw/master/scripts/bash_autocomplete --output /etc/bash_completion.d/dblab 
source /etc/bash_completion.d/dblab
How to use CLI

Docker Hub

References

Development

Open an Issue to discuss ideas, open a Merge Request to propose a change.

See our GitLab Container Registry to find the images built for development branches.

Development requirements
  1. Install golangci-lint: https://github.com/golangci/golangci-lint#install

Have a question?

  • Check our Q&A
  • or join our Community (links below)

Community

Directories

Path Synopsis
cmd
cli
cli/commands
Package commands provides general resources for CLI.
Package commands provides general resources for CLI.
cli/commands/clone
Package clone provides clones management commands.
Package clone provides clones management commands.
cli/commands/config
Package config provides commands for a CLI config management.
Package config provides commands for a CLI config management.
cli/commands/global
Package global provides general commands for CLI usage.
Package global provides general commands for CLI usage.
cli/commands/instance
Package instance provides instance management commands.
Package instance provides instance management commands.
cli/commands/snapshot
Package snapshot provides snapshot management commands.
Package snapshot provides snapshot management commands.
cli/templates
Package templates contains custom template output.
Package templates contains custom template output.
pkg
client/dblabapi
Package dblabapi provides a client for Database Lab HTTP API.
Package dblabapi provides a client for Database Lab HTTP API.
client/dblabapi/types
Package types provides request structures for Database Lab HTTP API.
Package types provides request structures for Database Lab HTTP API.
client/platform
Package platform provides the Platform API client.
Package platform provides the Platform API client.
config
Package config provides access to the Database Lab configuration.
Package config provides access to the Database Lab configuration.
config/global
Package global provides access to the global Database Lab Engine configuration.
Package global provides access to the global Database Lab Engine configuration.
estimator
Package estimator provides tools to estimate query timing for a production environment.
Package estimator provides tools to estimate query timing for a production environment.
log
models
Package models provides Database Lab struct.
Package models provides Database Lab struct.
observer
Package observer provides clone monitoring.
Package observer provides clone monitoring.
portfwd
Package portfwd provides an SSH port forwarder.
Package portfwd provides an SSH port forwarder.
retrieval
Package retrieval provides data retrieval pipeline.
Package retrieval provides data retrieval pipeline.
retrieval/components
Package components provides the key components of data retrieval.
Package components provides the key components of data retrieval.
retrieval/config
Package config contains configuration options of the data retrieval.
Package config contains configuration options of the data retrieval.
retrieval/dbmarker
Package dbmarker provides a tool for marking database data.
Package dbmarker provides a tool for marking database data.
retrieval/engine
Package engine provides different engines.
Package engine provides different engines.
retrieval/engine/postgres
Package postgres contains data retrieval jobs for a Postgres engine.
Package postgres contains data retrieval jobs for a Postgres engine.
retrieval/engine/postgres/logical
Package logical provides jobs for logical initial operations.
Package logical provides jobs for logical initial operations.
retrieval/engine/postgres/physical
Package physical provides jobs for physical initial operations.
Package physical provides jobs for physical initial operations.
retrieval/engine/postgres/snapshot
Package snapshot provides components for preparing initial snapshots.
Package snapshot provides components for preparing initial snapshots.
retrieval/engine/postgres/tools
Package tools provides helpers to initialize data.
Package tools provides helpers to initialize data.
retrieval/engine/postgres/tools/cont
Package cont provides tools to manage service containers started by Database Lab Engine.
Package cont provides tools to manage service containers started by Database Lab Engine.
retrieval/engine/postgres/tools/defaults
Package defaults contains default values.
Package defaults contains default values.
retrieval/engine/postgres/tools/fs
Package fs provides tools for working with the filesystem.
Package fs provides tools for working with the filesystem.
retrieval/engine/postgres/tools/health
Package health provides tools to set up container health check options.
Package health provides tools to set up container health check options.
retrieval/engine/postgres/tools/pgtool
Package pgtool provides tools to run PostgreSQL-specific commands.
Package pgtool provides tools to run PostgreSQL-specific commands.
retrieval/options
Package options provides helpers to process retrieval options.
Package options provides helpers to process retrieval options.
runci
Package runci provides a tools to run and check migrations in CI.
Package runci provides a tools to run and check migrations in CI.
runci/source
Package source provides a tools to use version control systems.
Package source provides a tools to use version control systems.
services/cloning
Package cloning provides a cloning service.
Package cloning provides a cloning service.
services/platform
Package platform provides a Platform service.
Package platform provides a Platform service.
services/provision
Package provision provides an interface to provision Database Lab clones.
Package provision provides an interface to provision Database Lab clones.
services/provision/databases/postgres
Package postgres provides an interface to work Postgres application.
Package postgres provides an interface to work Postgres application.
services/provision/databases/postgres/pgconfig
Package pgconfig provides tools for work with Postgres configuration.
Package pgconfig provides tools for work with Postgres configuration.
services/provision/docker
Package docker provides an interface to work with Docker containers.
Package docker provides an interface to work with Docker containers.
services/provision/pool
Package pool provides components to work with storage pools.
Package pool provides components to work with storage pools.
services/provision/resources
Package resources defines models used for provisioning.
Package resources defines models used for provisioning.
services/provision/runners
Package runners provides an interface to execute commands.
Package runners provides an interface to execute commands.
services/provision/thinclones
Package thinclones provides an interface to work different thin-clone managers.
Package thinclones provides an interface to work different thin-clone managers.
services/provision/thinclones/lvm
Package lvm provides an interface to work with LVM2.
Package lvm provides an interface to work with LVM2.
services/provision/thinclones/zfs
Package zfs provides an interface to work with ZFS.
Package zfs provides an interface to work with ZFS.
services/validator
Package validator provides a validation service.
Package validator provides a validation service.
srv
Package srv contains API routes and handlers.
Package srv contains API routes and handlers.
srv/api
Package api contains helpers to work with HTTP requests and responses.
Package api contains helpers to work with HTTP requests and responses.
srv/mw
Package mw contains middlewares.
Package mw contains middlewares.
util
Package util provides utility functions.
Package util provides utility functions.
util/networks
Package networks describes custom network elements.
Package networks describes custom network elements.
util/pglog
Package pglog provides helpers for a Postgres logs processing.
Package pglog provides helpers for a Postgres logs processing.
Package version provides the Database Lab version info.
Package version provides the Database Lab version info.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
t or T : Toggle theme light dark auto
y or Y : Canonical URL