arrowpb

README

arrowpb

Convert Apache Arrow records to Protocol Buffers

Introduction

arrowpb is a library designed to facilitate the conversion of Apache Arrow records into Protocol Buffers format. This allows for efficient serialization and deserialization of data structures, making it easier to work with data across different systems.

Feature

• Convert Arrow records to Protocol Buffers and vice versa.
• High performance and low memory overhead.
• Easy integration with existing Arrow and Protocol Buffers projects.

Author

arrowpb is developed by @TFMV.

License

This project is licensed under the MIT License. See the LICENSE file for details.

Documentation

Index

• Variables
• func FormatArrowJSON(reader array.RecordReader, output io.Writer) error

Variables

var TypeMappings = map[arrow.DataType]descriptorpb.FieldDescriptorProto_Type{
	arrow.BinaryTypes.Binary:          descriptorpb.FieldDescriptorProto_TYPE_BYTES,
	arrow.FixedWidthTypes.Boolean:     descriptorpb.FieldDescriptorProto_TYPE_BOOL,
	arrow.PrimitiveTypes.Float64:      descriptorpb.FieldDescriptorProto_TYPE_DOUBLE,
	arrow.PrimitiveTypes.Int64:        descriptorpb.FieldDescriptorProto_TYPE_INT64,
	arrow.BinaryTypes.String:          descriptorpb.FieldDescriptorProto_TYPE_STRING,
	arrow.FixedWidthTypes.Date32:      descriptorpb.FieldDescriptorProto_TYPE_STRING,
	arrow.FixedWidthTypes.Timestamp_s: descriptorpb.FieldDescriptorProto_TYPE_STRING,
}

TypeMappings maps Apache Arrow types to Protocol Buffers field types.

Functions

func FormatArrowJSON

func FormatArrowJSON(reader array.RecordReader, output io.Writer) error