data

package
v0.1.0-dev.23 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 14, 2023 License: MIT Imports: 9 Imported by: 0

Documentation

Overview

Package data contains the all data related types and functionalities. such as data.Data, data.ValueType, data.Kind, etc.

Index

Constants

View Source 
const (
	DefaultMaxBufferSize = 1024 * 1024 * 256 // 256 MB
	// DefaultMaxBufferLength is the default maximum length of the buffer.
	DefaultMaxBufferLength = 500000
)

DefaultMaxBufferSize is the default maximum size of the buffer in bytes.

Variables

View Source 
var (
	ErrValueTypeParseFuncNotImplemented = errors.New("value type parse function not implemented")
	ErrValueTypeParentNotInitialized    = errors.New("value type parent not initialized")
	ErrDataTypeKindNotMatch             = errors.New("data type kind not match")
	ErrInvalidValue                     = errors.New("invalid passed value")
	ErrDestMustBePointer                = errors.New("destination must be a pointer")
	ErrDestNotAssignable                = errors.New("destination is not assignable")
)
View Source 
var (
	ErrBufferAlreadyIsClosed = errors.New("buffer already is closed")
	ErrBufferIsClosed        = errors.New("buffer is closed")
)

Functions

func GetBaseKindSize 

func GetBaseKindSize(k Kind) int

GetBaseKindSize returns the size of the base kinds such as int, float, bool, etc. in bytes. It returns 0 if the kind is not a base kind.

func IsCompatibleWithType 

func IsCompatibleWithType(t1, t2 ValueType) bool

Types

type BaseValueType 

type BaseValueType struct {
	// contains filtered or unexported fields
}

BaseValueType implements ValueType interface. It can be embedded in other types to implement ValueType interface quickly.

func (*BaseValueType) AssignTo 

func (b *BaseValueType) AssignTo(dest any) error

AssignTo assigns the value to the given destination. It returns ErrDestMustBePointer if the destination is not a pointer. It returns ErrDataTypeKindNotMatch if the destination type kind is not the same as the receiver type kind.

func (*BaseValueType) Clone 

func (b *BaseValueType) Clone() ValueType

Clone returns a copy of the receiver.

func (*BaseValueType) GetTypeKind 

func (*BaseValueType) GetTypeKind() Kind

GetTypeKind returns the kind of the type. if not implemented by the parent type, it returns KindUnknown.

func (*BaseValueType) GetTypeName 

func (b *BaseValueType) GetTypeName() string

GetTypeName returns the name of the type.

func (*BaseValueType) GetTypeSize 

func (b *BaseValueType) GetTypeSize() uint64

GetTypeSize returns the size of the type in bytes.

func (*BaseValueType) GetValue 

func (b *BaseValueType) GetValue() any

GetValue returns the value stored in the receiver.

func (*BaseValueType) GetValueSize 

func (b *BaseValueType) GetValueSize() uint64

GetValueSize returns the size of the value in bytes.

func (*BaseValueType) Init 

func (b *BaseValueType) Init(parent ValueType)

Init initializes the parent type.

func (*BaseValueType) Parse 

func (*BaseValueType) Parse(_ any) error

Parse parses the value and stores it in the receiver. It should be implemented by the parent type. Otherwise, it returns ErrValueTypeParseFuncNotImplemented.

func (*BaseValueType) To 

func (b *BaseValueType) To(t Type) (ValueType, error)

To convert the value to the given type.

type Batch 

type Batch []*Set

Batch is a collection of data sets.

func NewDataBatch 

func NewDataBatch() *Batch

NewDataBatch returns a new data batch.

func (*Batch) Add 

func (b *Batch) Add(set ...*Set)

Add adds the given data sets to the batch. It does not if the given set is nil. It does not check for duplicate sets.

func (*Batch) Clear 

func (b *Batch) Clear()

Clear removes all data sets from the batch.

func (*Batch) Clone 

func (b *Batch) Clone() *Batch

Clone returns a copy of the batch.

func (*Batch) Get 

func (b *Batch) Get(index int) *Set

Get returns the data set at the given index. It returns nil if the index is out of range.

func (*Batch) GetLength 

func (b *Batch) GetLength() uint64

GetLength returns the length of the batch.

func (*Batch) GetSize 

func (b *Batch) GetSize() uint64

GetSize returns approximate size of the batch in bytes. It returns 0 if the batch is nil.

func (*Batch) Pop 

func (b *Batch) Pop() *Set

Pop returns the first data set in the batch and removes it from the batch. It returns nil if the batch is empty.

func (*Batch) Remove 

func (b *Batch) Remove(index int)

Remove removes the data set by the given index. It does nothing if the index is out of range.

func (*Batch) ToCSV 

func (b *Batch) ToCSV() []byte

ToCSV converts the data batch to a RFC4180-compliant CSV string.

type Buffer 

type Buffer struct {
	// contains filtered or unexported fields
}

Buffer is a thread-safe data buffer. It is used to buffer data sets before they are written to the database.

func NewBuffer 

func NewBuffer(ctx context.Context, size ...uint64) *Buffer

NewBuffer creates a new buffer with the given maximum size in bytes. If no size is given, the DefaultMaxBufferLength is used. SizeChanged: the maximum size of the buffer in bytes. And cannot be 0.

func (*Buffer) Clear 

func (b *Buffer) Clear()

Clear clears the buffer.

func (*Buffer) Clone 

func (b *Buffer) Clone() *Buffer

Clone returns a copy of the buffer. The copy will not be affected by the original buffer. And Clone always returns a not closed buffer.

func (*Buffer) Close 

func (b *Buffer) Close() error

Close will close the buffer. Closed buffer will not accept new data sets. If the buffer is already closed, it will return an error.

func (*Buffer) GetLength 

func (b *Buffer) GetLength() uint64

GetLength returns the current length of the buffer.

func (*Buffer) GetSize 

func (b *Buffer) GetSize() uint64

GetSize returns the current size of the buffer in bytes.

func (*Buffer) IsClosed 

func (b *Buffer) IsClosed() bool

IsClosed returns true if the buffer is closed.

func (*Buffer) IsEmpty 

func (b *Buffer) IsEmpty() bool

IsEmpty returns true if the buffer is empty.

func (*Buffer) Length 

func (b *Buffer) Length(length uint64)

Length sets the maximum length of the buffer. If the buffer exceeds the maximum length, it will be blocked until the buffer conditions are met.

func (*Buffer) Read 

func (b *Buffer) Read() (*Set, error)

Read pops the first data set from the buffer. If the buffer is empty, it will be blocked until the next data set is written to the buffer. If the buffer is closed, and all data sets have been read, it will return ErrBufferIsClosed.

func (*Buffer) Size 

func (b *Buffer) Size(size uint64)

Size sets the maximum size of the buffer in bytes. If the buffer exceeds the maximum size, it will be blocked until the buffer conditions are met.

func (*Buffer) WithObserver 

func (b *Buffer) WithObserver(observer BufferObserver) *Buffer

func (*Buffer) Write 

func (b *Buffer) Write(data ...*Set) error

Write writes the given data sets to the buffer. If the buffer exceeds the maximum conditions, it will be blocked until the buffer conditions are met. If the buffer is closed, it will return an error.

type BufferObserver 

type BufferObserver interface {
	// SizeChanged is called when the buffer size changes.
	SizeChanged(s uint64)
	// LengthChanged is called when the buffer length changes.
	LengthChanged(l uint64)

	Write(n int)
	Read(n int)
}

BufferObserver is an interface that can be implemented to observe the buffer.

type Data 

type Data struct {
	Key   string
	Value ValueType
}

Data is a key-value pair. The key is a string and the value is a Type. The value can be any type that implements the ValueType interface. Data is the smallest unit of data in gloader: Data < Set < Batch Data: {key, value} Set: [{key, value}, {key, value}, {key, value}, ...] Batch: [ [{key, value}, {key, value}, {key, value}, ...], [{key, value}, {key, value}, {key, value}, ...], [{key, value}, {key, value}, {key, value}, ...], ...].

func NewData 

func NewData(key string, value ValueType) *Data

NewData creates a new data with the given key and value. The value must implement the ValueType interface.

func (*Data) Clone 

func (d *Data) Clone() *Data

Clone returns a copy of the data.

func (*Data) GetKey 

func (d *Data) GetKey() string

GetKey returns the key of the data.

func (*Data) GetSize 

func (d *Data) GetSize() uint64

GetSize returns approximate memory usage of the data in bytes.

func (*Data) GetValueType 

func (d *Data) GetValueType() ValueType

GetValueType returns the value of the data.

func (*Data) SetKey 

func (d *Data) SetKey(key string)

SetKey sets the key of the data.

func (*Data) SetValue 

func (d *Data) SetValue(value ValueType)

SetValue sets the value of the data.

type Kind 

type Kind uint8

Kind represents the kind of value. It is a subset of the reflect.Kind. The zero Kind is KindUnknown. 

const (
	KindUnknown Kind = iota
	KindString
	KindInt
	KindFloat
	KindBool
	KindTime
	KindTimestamp
	KindDuration
	KindBytes
	KindArray
	KindMap
	KindStruct
	KindInterface
	KindPointer
	KindFunc
	KindChan
	KindSlice
	KindUint
	KindUint8
	KindUint16
	KindUint32
	KindUint64
	KindInt8
	KindInt16
	KindInt32
	KindInt64
	KindFloat32
	KindFloat64
)

func GetKindFromName 

func GetKindFromName(name string) Kind

GetKindFromName returns the kind from the given name. It returns KindUnknown if the name is not found.

func (Kind) GetKindGroup 

func (k Kind) GetKindGroup() KindGroup

func (Kind) GetReflectKind 

func (k Kind) GetReflectKind() reflect.Kind

GetReflectKind returns the reflect.Kind of the kind.

func (Kind) IsCompatibleWith 

func (k Kind) IsCompatibleWith(other Kind) bool

func (Kind) String 

func (k Kind) String() string

String returns the name of the kind.

type KindGroup 

type KindGroup uint8

KindGroup represents the group of kind. The kinds in the same group are compatible. 

const (
	KindGroupUnknown KindGroup = iota
	KindGroupString
	KindGroupInt
	KindGroupFloat
)

type Map 

type Map struct {
	// contains filtered or unexported fields
}

Map is a map of data types. It is used to determine schema of dataCollection. The key is the name of the data type and the value is the data type.

func (*Map) Delete 

func (m *Map) Delete(key string)

Delete deletes the data type with the given name.

func (*Map) Get 

func (m *Map) Get(key string) Type

Get returns the data type with the given name. It returns nil if the data type does not exist.

func (*Map) GetIndex 

func (m *Map) GetIndex(index int) Type

GetIndex returns the data type with the given index.

func (*Map) GetNullableMap 

func (m *Map) GetNullableMap() map[string]bool

func (*Map) GetTypeMap 

func (m *Map) GetTypeMap() map[string]Type

func (*Map) Has 

func (m *Map) Has(key string) bool

Has returns true if the data type with the given name exists.

func (*Map) HasDefaultValue 

func (m *Map) HasDefaultValue(key string) bool

func (*Map) HasIndex 

func (m *Map) HasIndex(index int) bool

func (*Map) IsNullable 

func (m *Map) IsNullable(key string) bool

func (*Map) Keys 

func (m *Map) Keys() []string

Keys returns the keys of the map.

func (*Map) KeysExcept 

func (m *Map) KeysExcept(keys []string) []string

func (*Map) Len 

func (m *Map) Len() int

Len returns the number of data types in the map.

func (*Map) NotNullableKeys 

func (m *Map) NotNullableKeys() []string

func (*Map) Set 

func (m *Map) Set(key string, value Type, options ...bool)

Set sets the data type with the given name. Options[0] isNullable Options[1] hasDefaultValue.

func (*Map) Types 

func (m *Map) Types() []Type

Types returns the values of the map.

type Set 

type Set []*Data

Set is a collection of related data. It's used to collect and organize data. also in relational database, it's called a row.

func NewDataSet 

func NewDataSet() *Set

NewDataSet returns a new data set.

func (*Set) Add 

func (d *Set) Add(data *Data)

Add appends the given data to the set.

func (*Set) Clone 

func (d *Set) Clone() *Set

Clone returns a clone of the data set.

func (*Set) Get 

func (d *Set) Get(key string) *Data

Get returns the data with the given key. It returns nil if the data does not exist.

func (*Set) GetByIndex 

func (d *Set) GetByIndex(index int) *Data

GetByIndex returns the data at the given index. It returns nil if the index is out of range.

func (*Set) GetKeys 

func (d *Set) GetKeys() []string

GetKeys returns the keys of the data set.

func (*Set) GetLength 

func (d *Set) GetLength() int

GetLength returns the length of the data set.

func (*Set) GetSize 

func (d *Set) GetSize() uint64

GetSize returns approximate size of the data set in bytes.

func (*Set) GetStringValues 

func (d *Set) GetStringValues() []string

GetStringValues returns the string values of the data set.

func (*Set) GetValues 

func (d *Set) GetValues() []ValueType

GetValues returns the values of the data set.

func (*Set) Has 

func (d *Set) Has(key string) bool

func (*Set) Remove 

func (d *Set) Remove(key string)

Remove removes the data with the given key. if the key is not found, it does nothing.

func (*Set) RemoveByIndex 

func (d *Set) RemoveByIndex(index int)

RemoveByIndex removes the data at the given index. It does nothing if the index is out of range.

func (*Set) Set 

func (d *Set) Set(key string, value ValueType)

Set sets the value of the data with the given key. If the data does not exist, it creates new data with the given key and value.

func (*Set) SetByIndex 

func (d *Set) SetByIndex(index int, value ValueType)

SetByIndex sets the value of the data at the given index. It does nothing if the index is out of range.

func (*Set) String 

func (d *Set) String(delimiter string) string

String returns the string representation of the data set. It uses the given delimiter to separate the data. e.g. delimiter = ", " => "value1, value2".

func (*Set) Swap 

func (d *Set) Swap(i, j int)

Swap swaps the data at the given indexes. It does nothing if the indexes are out of range.

type Type 

type Type interface {
	// GetTypeName returns the name of the type.
	GetTypeName() string
	// GetTypeKind returns the kind of the type.
	GetTypeKind() Kind
	// GetTypeSize returns the size of 1 length value of the type in bytes.
	GetTypeSize() uint64
}

Type is the interface that has the basic data type methods. It is provided GetTypeName, GetTypeKind methods. All driver types must implement this interface. A Type represents a type in the database.

type ValueType 

type ValueType interface {
	Type // Type interface
	// Parse parses the value and stores it in the receiver.
	// It can parse nil and any type that is compatible with the type kind.
	Parse(v any) error
	// GetValueSize returns the size of the value in bytes.
	GetValueSize() uint64
	// GetValue returns the value stored in the receiver.
	// returned value kind is the same as the type kind which is accessible by GetTypeKind method.
	// note: the returned value could be nil.
	GetValue() any
	// To convert the value to the given type.
	To(t Type) (ValueType, error)
	// AssignTo assigns the value to the given pointer destination.
	// The t should be a pointer to a type that is same with the type kind.
	AssignTo(t any) error
	// Clone returns a copy of the receiver.
	Clone() ValueType
}

ValueType is a Type that holds a value with same kind. It is provided Parse, GetValueSize, GetValue, To, AssignTo, Clone methods. For handling values.

func GetNewValueTypeAs 

func GetNewValueTypeAs(t Type) ValueType

Source Files

View all Source files

Directories

Path Synopsis
Package types is a package that contains some generic data types.
 Package types is a package that contains some generic data types.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.