greptime

README

GreptimeDB Go Ingester

Provide API to insert data into GreptimeDB.

How To Use

Installation

go get -u github.com/GreptimeTeam/greptimedb-ingester-go

Import

import greptime "github.com/GreptimeTeam/greptimedb-ingester-go"

Config

Initiate a Config for Client

cfg := greptime.NewConfig("<host>").
    WithPort(4001).
    WithAuth("<username>", "<password>").
    WithDatabase("<database>")

Options

Secure

cfg.WithInsecure(false) // default insecure=true

keepalive

cfg.WithKeepalive(time.Second*30, time.Second*5) // keepalive isn't enabled by default

Client

cli, err := greptime.NewClient(cfg)

Insert & StreamInsert

• you can Insert data into GreptimeDB via different style:

  • Table style
  • ORM style

• streaming insert is to Send data into GreptimeDB without waiting for response.

Table style

you can define schema via Table and Column, and then AddRow to include the real data you want to write.

define table schema, and add rows

import(
    "github.com/GreptimeTeam/greptimedb-ingester-go/table"
    "github.com/GreptimeTeam/greptimedb-ingester-go/table/types"
)

tbl, err := table.New("<table_name>")

tbl.AddTagColumn("id", types.INT64)
tbl.AddFieldColumn("host", types.STRING)
tbl.AddTimestampColumn("ts", types.TIMESTAMP_MILLISECOND)

err := tbl.AddRow(1, "127.0.0.1", time.Now())
err := tbl.AddRow(2, "127.0.0.2", time.Now())
...

Write into GreptimeDB

resp, err := cli.Write(context.Background(), tbl)

Stream Write into GreptimeDB

err := cli.StreamWrite(context.Background(), tbl)
...
affected, err := cli.CloseStream(ctx)

ORM style

If you prefer ORM style, and define column-field relationship via struct field tag, you can try the following way.

Tag

• greptime is the struct tag key
• tag, field, timestamp is for SemanticType, and the value is ignored
• column is to define the column name
• type is to define the data type. if type is timestamp, precision is supported
• the metadata separator is ; and the key value separator is :

type supported is the same as described Datatypes supported, and case insensitive

define struct with tags

type Monitor struct {
    ID          int64     `greptime:"tag;column:id;type:int64"`
    Host        string    `greptime:"field;column:host;type:string"`
    Ts          time.Time `greptime:"timestamp;column:ts;type:timestamp;precision:millisecond"`
}

// TableName is to define the table name.
func (Monitor) TableName() string {
    return "<table_name>"
}

instance your struct

monitors := []Monitor{
    {
        ID:          randomId(),
        Host:        "127.0.0.1",
        Running:     true,
    },
    {
        ID:          randomId(),
        Host:        "127.0.0.2",
        Running:     true,
    },
}

WriteObject into GreptimeDB

resp, err := cli.WriteObject(context.Background(), monitors)

Stream WriteObject into GreptimeDB

err := cli.StreamWriteObject(context.Background(), monitors)
...
affected, err := cli.CloseStream(ctx)

Datatypes supported

The GreptimeDB column is for the datatypes supported in library, and the Go column is the matched Go type.

GreptimeDB	Go	Description
INT8	int8
INT16	int16
INT32	int32
INT64, INT	int64
UINT8	uint8
UINT16	uint16
UINT32	uint32
UINT64, UINT	uint64
FLOAT32	float32
FLOAT64, FLOAT	float64
BOOLEAN, BOOL	bool
STRING	string
BINARY, BYTES	[]byte
DATE	Int or time.Time	the day elapsed since 1970-1-1
DATETIME	Int or time.Time	the millisecond elapsed since 1970-1-1
TIMESTAMP_SECOND	Int or time.Time
TIMESTAMP_MILLISECOND, TIMESTAMP	Int or time.Time
TIMESTAMP_MICROSECOND	Int or time.Time
TIMESTAMP_NANOSECOND	Int or time.Time

NOTE: Int is for all of Integer and Unsigned Integer in Go

Query

You can use ORM library like gorm with MySQL or PostgreSQL driver to connect GreptimeDB and retrieve data from it.

type Monitor struct {
    ID          int64     `gorm:"primaryKey;column:id"`
    Host        string    `gorm:"column:host"`
    Ts          time.Time `gorm:"column:ts"`
}

// Get all monitors
var monitors []Monitor
result := db.Find(&monitors)

Documentation

Index

• type Client
  • func NewClient(cfg *Config) (*Client, error)
  • func (c *Client) CloseStream(ctx context.Context) (*gpb.AffectedRows, error)
  • func (c *Client) StreamWrite(ctx context.Context, tables ...*table.Table) error
  • func (c *Client) StreamWriteObject(ctx context.Context, body any) error
  • func (c *Client) Write(ctx context.Context, tables ...*table.Table) (*gpb.GreptimeResponse, error)
  • func (c *Client) WriteObject(ctx context.Context, obj any) (*gpb.GreptimeResponse, error)
• type Config
  • func NewConfig(host string) *Config
  • func (c *Config) WithAuth(username, password string) *Config
  • func (c *Config) WithDatabase(database string) *Config
  • func (c *Config) WithDialOption(opt grpc.DialOption) *Config
  • func (c *Config) WithInsecure(insecure bool) *Config
  • func (c *Config) WithKeepalive(time, timeout time.Duration) *Config
  • func (c *Config) WithPort(port int) *Config

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client helps to write data into GreptimeDB. A Client is safe for concurrent use by multiple goroutines,you can have one Client instance in your application.

func NewClient

func NewClient(cfg *Config) (*Client, error)

NewClient helps to create the greptimedb client, which will be responsible write data into GreptimeDB.

func (*Client) CloseStream

func (c *Client) CloseStream(ctx context.Context) (*gpb.AffectedRows, error)

CloseStream closes the stream. Once we’ve finished writing our client’s requests to the stream using client.StreamWrite or client.StreamWriteObject, we need to call client.CloseStream to let GreptimeDB know that we’ve finished writing and are expecting to receive a response.

func (*Client) StreamWrite

func (c *Client) StreamWrite(ctx context.Context, tables ...*table.Table) error

StreamWrite is to send the data into GreptimeDB via explicit schema.

tbl, err := table.New(<tableName>)

// add column at first. This is to define the schema of the table.
tbl.AddTagColumn("tag1", types.INT64)
tbl.AddFieldColumn("field1", types.STRING)
tbl.AddFieldColumn("field2", types.FLOAT64)
tbl.AddTimestampColumn("timestamp", types.TIMESTAMP_MILLISECOND)

// you can add multiple row(s). This is the real data.
tbl.AddRow(1, "hello", 1.1, time.Now())

// send data into GreptimeDB
resp, err := client.StreamWrite(context.Background(), tbl)

func (*Client) StreamWriteObject

func (c *Client) StreamWriteObject(ctx context.Context, body any) error

StreamWriteObject is like [StreamWrite] to send the data into GreptimeDB, but schema is defined in the struct tag.

type monitor struct {
  ID          int64     `greptime:"tag;column:id;type:int64"`
  Host        string    `greptime:"tag;column:host;type:string"`
  Memory      uint64    `greptime:"field;column:memory;type:uint64"`
  Cpu         float64   `greptime:"field;column:cpu;type:float64"`
  Temperature int64     `greptime:"field;column:temperature;type:int64"`
  Running     bool      `greptime:"field;column:running;type:boolean"`
  Ts          time.Time `greptime:"timestamp;column:ts;type:timestamp;precision:millisecond"`
}

func (monitor) TableName() string {
  return monitorTableName
}

monitors := []monitor{
	{
	    ID:          randomId(),
	    Host:        "127.0.0.1",
	    Memory:      1,
	    Cpu:         1.0,
	    Temperature: -1,
	    Ts:          time1,
	    Running:     true,
	},
	{
	    ID:          randomId(),
	    Host:        "127.0.0.2",
	    Memory:      2,
	    Cpu:         2.0,
	    Temperature: -2,
	    Ts:          time2,
	    Running:     true,
	},
}

resp, err := client.StreamWriteObject(context.Background(), monitors)

func (*Client) Write

func (c *Client) Write(ctx context.Context, tables ...*table.Table) (*gpb.GreptimeResponse, error)

Write is to write the data into GreptimeDB via explicit schema.

tbl, err := table.New(<tableName>)

// add column at first. This is to define the schema of the table.
tbl.AddTagColumn("tag1", types.INT64)
tbl.AddFieldColumn("field1", types.STRING)
tbl.AddFieldColumn("field2", types.FLOAT64)
tbl.AddTimestampColumn("timestamp", types.TIMESTAMP_MILLISECOND)

// you can add multiple row(s). This is the real data.
tbl.AddRow(1, "hello", 1.1, time.Now())

// write data into GreptimeDB
resp, err := client.Write(context.Background(), tbl)

func (*Client) WriteObject

func (c *Client) WriteObject(ctx context.Context, obj any) (*gpb.GreptimeResponse, error)

WriteObject is like [Write] to write the data into GreptimeDB, but schema is defined in the struct tag.

type Monitor struct {
  ID          int64     `greptime:"tag;column:id;type:int64"`
  Host        string    `greptime:"tag;column:host;type:string"`
  Memory      uint64    `greptime:"field;column:memory;type:uint64"`
  Cpu         float64   `greptime:"field;column:cpu;type:float64"`
  Temperature int64     `greptime:"field;column:temperature;type:int64"`
  Running     bool      `greptime:"field;column:running;type:boolean"`
  Ts          time.Time `greptime:"timestamp;column:ts;type:timestamp;precision:millisecond"`
}

func (Monitor) TableName() string {
  return monitorTableName
}

monitors := []Monitor{
	{
	    ID:          randomId(),
	    Host:        "127.0.0.1",
	    Memory:      1,
	    Cpu:         1.0,
	    Temperature: -1,
	    Ts:          time1,
	    Running:     true,
	},
	{
	    ID:          randomId(),
	    Host:        "127.0.0.2",
	    Memory:      2,
	    Cpu:         2.0,
	    Temperature: -2,
	    Ts:          time2,
	    Running:     true,
	},
}

resp, err := client.WriteObject(context.Background(), monitors)

type Config

type Config struct {
	Host     string // no scheme or port included. example: 127.0.0.1
	Port     int    // default: 4001
	Username string
	Password string
	Database string // the default database
	// contains filtered or unexported fields
}

Config is to define how the Client behaves.

• Host is 127.0.0.1 in local environment.
• Port default value is 4001.
• Username and Password can be left to empty in local environment. you can find them in GreptimeCloud service detail page.
• Database is the default database the client will operate on. But you can change the database in InsertRequest or QueryRequest.

func NewConfig

func NewConfig(host string) *Config

NewConfig helps to init Config with host only

func (*Config) WithAuth

func (c *Config) WithAuth(username, password string) *Config

WithAuth helps to specify the Basic Auth username and password. Leave them empty if you are in local environment.

func (*Config) WithDatabase

func (c *Config) WithDatabase(database string) *Config

WithDatabase helps to specify the default database the client operates on.

func (*Config) WithDialOption

func (c *Config) WithDialOption(opt grpc.DialOption) *Config

WithDialOption helps to specify the dial option which has not been supported by ingester sdk yet.

func (*Config) WithInsecure

func (c *Config) WithInsecure(insecure bool) *Config

TODO(yuanbohan): support more tls options

func (*Config) WithKeepalive

func (c *Config) WithKeepalive(time, timeout time.Duration) *Config

WithKeepalive helps to set the keepalive option.

• time. After a duration of this time if the client doesn't see any activity it pings the server to see if the transport is still alive. If set below 10s, a minimum value of 10s will be used instead.
• timeout. After having pinged for keepalive check, the client waits for a duration of Timeout and if no activity is seen even after that the connection is closed.

func (*Config) WithPort

func (c *Config) WithPort(port int) *Config

WithPort set the Port field. Do not change it if you have no idea what it is.