chdb

README

chDB

中文

chDB is an embedded SQL OLAP Engine powered by ClickHouse

Features

• In-process SQL OLAP Engine, powered by ClickHouse
• No need to install ClickHouse
• Minimized data copy from C++ to Python with python memoryview
• Input&Output support Parquet, CSV, JSON, Arrow, ORC and 60+more formats, samples
• Support Python DB API 2.0, example

Arch

Installation

Currently, chDB only supports Python 3.8+ on macOS(x86_64 and ARM64) and Linux.

pip install chdb

Usage

Run in command line

python3 -m chdb SQL [OutputFormat]

python3 -m chdb "SELECT 1,'abc'" Pretty

Data Input

The following methods are available to access on-disk and in-memory data formats:

🗂️ Query On File

(Parquet, CSV, JSON, Arrow, ORC and 60+)

You can execute SQL and return desired format data.

import chdb
res = chdb.query('select version()', 'Pretty'); print(res.data())

Work with Parquet or CSV

# See more data type format in tests/format_output.py
res = chdb.query('select * from file("data.parquet", Parquet)', 'JSON'); print(res.data())
# With large data result, get_memview() is more quickly cos it does not do an extra data copy.
res = chdb.query('select * from file("data.csv", CSV)', 'CSV');  print(str(res.get_memview().tobytes()))

Pandas dataframe output

# See more in https://clickhouse.com/docs/en/interfaces/formats
chdb.query('select * from file("data.parquet", Parquet)', 'Dataframe')

🗂️ Query On Table

(Pandas DataFrame, Parquet file/bytes, Arrow bytes)

Query On Pandas DataFrame

import chdb.dataframe as cdf
import pandas as pd
tbl = cdf.Table(dataframe=pd.DataFrame({'a': [1, 2, 3], 'b': ['a', 'b', 'c']}))
ret_tbl = tbl.query('select * from __table__')
print(ret_tbl)
print(ret_tbl.query('select b, sum(a) from __table__ group by b'))

🗂️ Python DB-API 2.0

import chdb.dbapi as dbapi
print("chdb driver version: {0}".format(dbapi.get_client_info()))

conn1 = dbapi.connect()
cur1 = conn1.cursor()
cur1.execute('select version()')
print("description: ", cur1.description)
print("data: ", cur1.fetchone())
cur1.close()
conn1.close()

For more examples, see examples and tests.

Demos and Examples

• Examples
• Serverless Query Demo

Benchmark

• ClickBench of embedded engines

Documentation

• For SQL syntax, please refer to ClickHouse SQL Reference

Contributing

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated. There are something you can help:

• Help test and report bugs
• Help improve documentation
• Help improve code quality and performance

License

Apache 2.0, see LICENSE for more information.

Acknowledgments

chDB is mainly based on ClickHouse for trade mark and other reasons, I named it chDB.

Contact

• Discord: https://discord.gg/nqSkfJRR
• Email: auxtenwpc@gmail.com
• Twitter: @auxten