bonobo

README

Bonobo

Bonobo is a query engine that uses the Arrow type system and aims for maximum compatibility with the Substrait specification. It is capable of:

• Creating plans via SQL or a Dataframe API
• Serializing plans to and from Substrait
• Manipulating/Splitting/Validating query plans

Bonobo does not, by design, implement execution kernels for any plan relations or expressions. Instead, Bonobo aims to make it simple to delegate plans to any Substrait-compatible backends for distributed execution.

Install

go get github.com/joellubi/bonobo

Roadmap

Bonobo is still in early stages of development. Features are being added quickly and the API is subject to change.

For a feature to be considered implemented it should:

• Be supported using the Dataframe API
• Implement both serialization and deserialization to/from Substrait proto

Features:

• Logical Relations
  • ReadRel
    • NamedTable
    • VirtualTable
    • ExtensionTable
    • LocalFiles
    • Filter
    • Project
  • FilterRel
  • FetchRel
  • AggregateRel
  • SortRel
  • JoinRel
  • ProjectRel
  • SetRel
  • ExtensionSingleRel
  • ExtensionMultiRel
  • ExtensionLeafRel
  • CrossRel
  • ReferenceRel
  • WriteRel
  • DdlRel
• Physical Relations (TBD whether they will be supported)
• Expressions
  • Literal
  • FieldReference
  • ScalarFunction
  • WindowFunction
  • IfThen
  • SwitchExpression
  • SingularOrList
  • MultiOrList
  • Cast
  • Subquery
  • Nested
• Extensions
  • Simple Extensions
    • Type
    • Type Variation
    • Function
      • Scalar Function
      • Aggregate Function
      • Window Function
    • Type Syntax Parsing
  • Advanced Extensions
  • Capabilities
• SQL Support
  • Query Clauses
    • SELECT
    • FROM
    • WHERE
    • GROUP BY
    • ORDER BY
    • PARTITION BY
    • LIMIT
    • JOIN
    • OVER
    • UNION
  • Binary Operators
  • Parenthesis
  • Expression Aliases
  • Identifier Aliases
  • Table Subqueries
  • Scalar Subqueries
  • Functions

Documentation

Index

• Variables
• type Field
  • func (f Field) String() string
• type Schema
  • func NewSchema(fields []Field) *Schema
  • func NewSchemaFromProto(n *proto.NamedStruct) *Schema
  • func (s *Schema) Fields() []Field
  • func (s *Schema) Len() int
  • func (s *Schema) String() string
• type Type

Variables

var Types = struct {
	BooleanType func(nullable bool) Type
	Int8Type    func(nullable bool) Type
	Int16Type   func(nullable bool) Type
	Int32Type   func(nullable bool) Type
	Int64Type   func(nullable bool) Type
	FloatType   func(nullable bool) Type
	DoubleType  func(nullable bool) Type
	DateType    func(nullable bool) Type
	StringType  func(nullable bool) Type
	DecimalType func(p, s int32, nullable bool) Type
}{
	BooleanType: func(nullable bool) Type {
		return withNullability(&types.BooleanType{}, nullable)
	},
	Int8Type: func(nullable bool) Type {
		return withNullability(&types.Int8Type{}, nullable)
	},
	Int16Type: func(nullable bool) Type {
		return withNullability(&types.Int16Type{}, nullable)
	},
	Int32Type: func(nullable bool) Type {
		return withNullability(&types.Int32Type{}, nullable)
	},
	Int64Type: func(nullable bool) Type {
		return withNullability(&types.Int64Type{}, nullable)
	},
	FloatType: func(nullable bool) Type {
		return withNullability(&types.Float32Type{}, nullable)
	},
	DoubleType: func(nullable bool) Type {
		return withNullability(&types.Float64Type{}, nullable)
	},
	DateType: func(nullable bool) Type {
		return withNullability(&types.DateType{}, nullable)
	},
	StringType: func(nullable bool) Type {
		return withNullability(&types.StringType{}, nullable)
	},
	DecimalType: func(p, s int32, nullable bool) Type {
		return withNullability(&types.DecimalType{Precision: p, Scale: s}, nullable)
	},
}

Types

type Field

type Field struct {
	Name string
	Type Type
}

func (Field) String

func (f Field) String() string

type Schema

type Schema types.NamedStruct

func NewSchema

func NewSchema(fields []Field) *Schema

func NewSchemaFromProto

func NewSchemaFromProto(n *proto.NamedStruct) *Schema

func (*Schema) Fields

func (s *Schema) Fields() []Field

func (*Schema) Len

func (s *Schema) Len() int

func (*Schema) String

func (s *Schema) String() string

type Type

type Type types.Type