tbln

package module
v0.0.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 30, 2022 License: MIT Imports: 14 Imported by: 1

README

TBLN

PkgGoDev Actions Status Go Report Card

TBLN is a text format that represents the table.

This repository contains Go library for reading and writing files, and Go library for reading and writing RDBMS tables.

Here is a document about the library. See cmd/README.md for CLI Tool.

Features

  • TBLN can contain multiple columns like CSV.
  • Database tables and import/export are possible.
  • It can include the signature with the hash value in the file.
  • Merge, sync and diff is possible
    • TBLN file and TBLN file
    • TBLN file and DB Table
    • DB Table and DB Table

Please refer to TBLN for the specification of TBLN.

Install

$ go get github.com/noborus/tbln

Notes: Requires a version of Go that supports modules. e.g. Go 1.13+

Example

Write example

Example of creating TBLN. (Error check is omitted)

package main

import (
	"os"

	"github.com/noborus/tbln"
)

func main() {
	tb := tbln.NewTBLN()
	tb.SetTableName("sample")
	tb.SetNames([]string{"id", "name"})
	tb.SetTypes([]string{"int", "text"})
	tb.AddRows([]string{"1", "Bob"})
	tb.AddRows([]string{"2", "Alice"})
	tbln.WriteAll(os.Stdout, tb)
}

Execution result.

; TableName: sample
; created_at: 2019-04-06T01:05:17+09:00
; name: | id | name |
; type: | int | text |
| 1 | Bob |
| 1 | Alice |
Read example

All read to memory by tbln.ReadAll. Here, the table name is rewritten and output.

package main

import (
	"log"
	"os"

	"github.com/noborus/tbln"
)

func main() {
	if len(os.Args) <= 1 {
		log.Fatal("Requires tbln file")
	}

	file, err := os.Open(os.Args[1])
	if err != nil {
		log.Fatal(err)
	}
	at, err := tbln.ReadAll(file)
	if err != nil {
		log.Fatal(err)
	}
	at.SetTableName("newtable")
	err = tbln.WriteAll(os.Stdout, at)
	if err != nil {
		log.Fatal(err)
	}
}
TBLN files can be imported and exported into the database.

Example of importing into PostgreSQL.

# \d sample
               Table "public.sample"
 Column |  Type   | Collation | Nullable | Default
--------+---------+-----------+----------+---------
 id     | integer |           |          |
 name   | text    |           |          |

Example of exporting it to a TBLN file.

; TableName: sample
; character_octet_length: |  | 1073741824 |
; created_at: 2019-04-06T02:03:43+09:00
; is_nullable: | YES | YES |
; numeric_precision: | 32 |  |
; numeric_precision_radix: | 2 |  |
; numeric_scale: | 0 |  |
; postgres_type: | integer | text |
; Signature: | test | ED25519 | 6271909d82000c4f686785cf0f9080971470ad3247b091ca50f6ea12ccc96efde0e1ca77e16723ef0f9d781941dfb92bed094dbf2e4079dd25f5aa9f9f1aab01 |
; Hash: | sha256 | 65f7ce4e15ddc006153fe769b8f328c466cbd1dea4b15aa195ed63daf093668d |
; name: | id | name |
; type: | int | text |
| 1 | Bob |
| 1 | Alice |

See db/README.md for details.

You can also use tbln cli tool to quickly import and export to a database.

See cmd/README.md for details.

Documentation

Overview

Package tbln reads and writes tbln files.

tbln can contain multiple fields like csv. Also, it can include checksum and signature inside. TBLN contains three types of lines: data, comments, and extras. All rows end with a new line(LF). The number of fields in all rows must be the same.

Data begins with "| " and ends with " |". Multiple fields are separated by " | ". (Note that the delimiter contains spaces.) 

| fields1 | fields2 | fields3 |

White space is considered part of a field. If "|" is included in the field, "|" must be duplicated. 

| -> || , || -> |||

Comments begin with "# ". Comments are not interpreted. 

# Comments

Extras begin with ";". extras can be interpreted as a header. Extras is basically written in the item name: value. 

; ItemName: Value

Extras is optional. Extras has item names that are interpreted in some special ways.

Example 
package main

import (
	"log"
	"os"
	"strings"

	"github.com/noborus/tbln"
)

func main() {
	in := `; TableName: simple
; name: | id | name |
; type: | int | text |
| 1 | Bob |
| 2 | Alice |
`
	at, err := tbln.ReadAll(strings.NewReader(in))
	if err != nil {
		log.Fatal(err)
	}
	at.SetTableName("newtable")
	err = tbln.WriteAll(os.Stdout, at)
	if err != nil {
		log.Fatal(err)
	}
}

Output:

; TableName: newtable
; name: | id | name |
; type: | int | text |
| 1 | Bob |
| 2 | Alice |

Index

Examples

Constants

View Source 
const (
	SHA256 = "sha256" // import crypto/sha256
	SHA512 = "sha512" // import crypto/sha512
)

Types of supported hashes

View Source 
const (
	ED25519 = "ED25519"
)

Signature algorithm.

Variables

View Source 
var ESCAPE = regexp.MustCompile(`(\|+)`)

ESCAPE is escape | -> ||

View Source 
var UNESCAPE = regexp.MustCompile(`\|(\|+)`)

UNESCAPE is unescape || -> |

Functions

func ColumnPrimaryKey 

func ColumnPrimaryKey(pKeys []Pkey, row []string) []string

ColumnPrimaryKey return columns primary key.

func DiffAll 

func DiffAll(writer io.Writer, t1, t2 Reader, diffMode DiffMode) error

DiffAll Write diff to writer from two readers.

Example 
package main

import (
	"bytes"
	"log"
	"os"

	"github.com/noborus/tbln"
)

func main() {
	TestDiff1 := `; name: | id | name | age |
; type: | int | text | int |
; primarykey: | id |
; TableName: test1
| 1 | Bob | 19 |
`

	TestDiff2 := `; name: | id | name | age |
; type: | int | text | int |
; primarykey: | id |
; TableName: test1
| 1 | Bob | 19 |
| 2 | Alice | 14 |
`
	err := tbln.DiffAll(os.Stdout,
		tbln.NewReader(bytes.NewBufferString(TestDiff1)),
		tbln.NewReader(bytes.NewBufferString(TestDiff2)),
		tbln.AllDiff)
	if err != nil {
		log.Fatal(err)
	}
}

Output:

| 1 | Bob | 19 |
+| 2 | Alice | 14 |

func JoinRow 

func JoinRow(row []string) string

JoinRow makes a Row array a character string.

func SplitRow 

func SplitRow(str string) []string

SplitRow divides a character string into a row array.

func WriteAll 

func WriteAll(writer io.Writer, tbln *TBLN) error

WriteAll writes multiple tbln records to w using Write.

Types

type Compare 

type Compare struct {
	PK []Pkey
	// contains filtered or unexported fields
}

Compare represents a structure for comparing two TBLN.

func NewCompare 

func NewCompare(t1, t2 Reader) (*Compare, error)

NewCompare returns a Reader interface.

func (*Compare) ReadDiffRow 

func (cmp *Compare) ReadDiffRow() (*DiffRow, error)

ReadDiffRow compares two rows and returns the difference.

type Definition 

type Definition struct {
	Comments []string

	Extras map[string]Extra
	Hashes map[string][]byte
	Signs  Signatures
	// contains filtered or unexported fields
}

Definition is common table definition struct.

func MergeDefinition 

func MergeDefinition(t1d, t2d *Definition) (*Definition, error)

MergeDefinition merges two tbln Definitions.

func NewDefinition 

func NewDefinition() *Definition

NewDefinition is create Definition struct.

func (*Definition) AllTargetHash 

func (d *Definition) AllTargetHash(target bool)

AllTargetHash is set all target of hash.

func (*Definition) ColumnNum 

func (d *Definition) ColumnNum() int

ColumnNum returns the number of columns.

func (*Definition) ExtraValue 

func (d *Definition) ExtraValue(key string) interface{}

ExtraValue is return extra value.

func (*Definition) GetDefinition 

func (d *Definition) GetDefinition() *Definition

GetDefinition return Definition

func (*Definition) GetPKeyPos 

func (d *Definition) GetPKeyPos() ([]int, error)

GetPKeyPos return PrimaryKey position

func (*Definition) Names 

func (d *Definition) Names() []string

Names return column names.

func (*Definition) PrimaryKey 

func (d *Definition) PrimaryKey() []string

PrimaryKey return PrimaryKey

func (*Definition) SerializeHash 

func (d *Definition) SerializeHash() []byte

SerializeHash returns a []byte that serializes Hash's map.

func (*Definition) SetExtra 

func (d *Definition) SetExtra(keyName string, value string)

SetExtra is set Extra of the Definition.

func (*Definition) SetHashes 

func (d *Definition) SetHashes(hashes []string) error

SetHashes is set hashes.

func (*Definition) SetNames 

func (d *Definition) SetNames(names []string) error

SetNames is set Column Name to the Table.

func (*Definition) SetSignatures 

func (d *Definition) SetSignatures(sign []string) error

SetSignatures is set signatures.

func (*Definition) SetTableName 

func (d *Definition) SetTableName(name string)

SetTableName is set Table Name of the Definition..

func (*Definition) SetTimeStamp 

func (d *Definition) SetTimeStamp() error

SetTimeStamp is set time stamp. If there is no created_at, create it. Otherwise updated updated_at.

func (*Definition) SetTypes 

func (d *Definition) SetTypes(types []string) error

SetTypes is set Column Type to Table.

func (*Definition) TableName 

func (d *Definition) TableName() string

TableName returns the Table Name.

func (*Definition) ToTargetHash 

func (d *Definition) ToTargetHash(key string, target bool)

ToTargetHash is set as target of hash.

func (*Definition) Types 

func (d *Definition) Types() []string

Types return column types.

type DiffMode 

type DiffMode int

DiffMode represents the mode of diff. 

const (
	OnlyAdd DiffMode = iota
	OnlyDiff
	AllDiff
)

Represents the diff mode

func (DiffMode) String 

func (m DiffMode) String() string

type DiffRow 

type DiffRow struct {
	Les   int
	Self  []string
	Other []string
}

DiffRow represents the difference between two rows.

func (*DiffRow) Diff 

func (d *DiffRow) Diff(diffMode DiffMode) string

Diff returns diff format

func (*DiffRow) ExceptRow 

func (d *DiffRow) ExceptRow() []string

ExceptRow reads excepted rows from DiffRow

func (*DiffRow) MergeRow 

func (d *DiffRow) MergeRow(mode MergeMode) []string

MergeRow reads merged rows from DiffRow

type Extra 

type Extra struct {
	// contains filtered or unexported fields
}

Extra is table definition extra struct.

func NewExtra 

func NewExtra(value interface{}, hashTarget bool) Extra

NewExtra is return new extra struct.

func (*Extra) Value 

func (e *Extra) Value() interface{}

Value is return extra value.

type FileReader 

type FileReader struct {
	*Definition
	// contains filtered or unexported fields
}

FileReader reads records from a tbln file.

func NewReader 

func NewReader(r io.Reader) *FileReader

NewReader returns a new Reader that reads from r.

func (*FileReader) ReadRow 

func (tr *FileReader) ReadRow() ([]string, error)

ReadRow reads one record (a slice of fields) from tr.

type MergeMode 

type MergeMode int

MergeMode represents the mode of merge. 

const (
	MergeIgnore MergeMode = iota
	MergeUpdate
	MergeDelete
)

Represents the merge mode

type OwnReader 

type OwnReader struct {
	*TBLN
	// contains filtered or unexported fields
}

OwnReader reads records from TBLN.

func NewOwnReader 

func NewOwnReader(t *TBLN) *OwnReader

NewOwnReader return TBLNReader.

func (*OwnReader) ReadRow 

func (rr *OwnReader) ReadRow() ([]string, error)

ReadRow one record (a slice of fields) from TBLN.

type Pkey 

type Pkey struct {
	Pos  int
	Name string
	Typ  string
}

Pkey represents the primary key.

type Reader 

type Reader interface {
	ReadRow() ([]string, error)
	GetDefinition() *Definition
}

Reader is TBLN Reader interface.

type Signature 

type Signature struct {
	// contains filtered or unexported fields
}

Signature struct stores a signature, a name, and an algorithm.

type Signatures 

type Signatures map[string]Signature

Signatures is a map of signature name and signature.

type TBLN 

type TBLN struct {
	*Definition
	RowNum int
	Rows   [][]string
}

TBLN represents TBLN format data. It includes TBLN definition and TBLN rows.

Example 
package main

import (
	"log"
	"os"

	"github.com/noborus/tbln"
)

func main() {
	var err error
	tb := tbln.NewTBLN()
	tb.SetTableName("sample")
	// SetNames sets column names
	err = tb.SetNames([]string{"id", "name"})
	if err != nil {
		log.Fatal(err)
	}
	// SetTypes sets the column type
	err = tb.SetTypes([]string{"int", "text"})
	if err != nil {
		log.Fatal(err)
	}
	// Add a row.
	// The number of columns should be the same
	// number of columns in Names and Types.
	err = tb.AddRows([]string{"1", "Bob"})
	if err != nil {
		log.Fatal(err)
	}
	err = tb.AddRows([]string{"2", "Alice"})
	if err != nil {
		log.Fatal(err)
	}
	err = tbln.WriteAll(os.Stdout, tb)
	if err != nil {
		log.Fatal(err)
	}
}

Output:

; TableName: sample
; name: | id | name |
; type: | int | text |
| 1 | Bob |
| 2 | Alice |

func ExceptAll 

func ExceptAll(t1, t2 Reader) (*TBLN, error)

ExceptAll merges two tbln and returns one tbln.

func MergeAll 

func MergeAll(t1, t2 Reader, mode MergeMode) (*TBLN, error)

MergeAll merges two tbln and returns one tbln.

func NewTBLN 

func NewTBLN() *TBLN

NewTBLN is create tbln struct.

func ReadAll 

func ReadAll(r io.Reader) (*TBLN, error)

ReadAll reads all r and returns a tbln struct. ReadAll returns when the blank line is reached.

func (*TBLN) AddRows 

func (t *TBLN) AddRows(row []string) error

AddRows is Add row to Table.

func (*TBLN) Sign 

func (t *TBLN) Sign(name string, pkey []byte) (map[string]Signature, error)

Sign is returns signature for hash.

func (*TBLN) String added in v0.0.2 

func (t *TBLN) String() string

func (*TBLN) SumHash 

func (t *TBLN) SumHash(hashType string) error

SumHash calculated checksum.

func (*TBLN) Verify 

func (t *TBLN) Verify() bool

Verify returns the boolean value of the hash verification.

func (*TBLN) VerifySignature 

func (t *TBLN) VerifySignature(name string, pubkey []byte) bool

VerifySignature returns the boolean value of the signature verification and Verify().

type Writer 

type Writer struct {
	Writer io.Writer
}

Writer writes records to a TBLN encoded file.

func NewWriter 

func NewWriter(w io.Writer) *Writer

NewWriter returns a new Writer that writes to w.

func (*Writer) WriteDefinition 

func (w *Writer) WriteDefinition(d *Definition) error

WriteDefinition writes Definition (comment and extra) to w.

func (*Writer) WriteRow 

func (w *Writer) WriteRow(row []string) error

WriteRow writes a single tbln record to w along with any necessary escaping. A record is a slice of strings with each string being one field.

Source Files

View all Source files

Directories

Path Synopsis
_example
create
dbwrite
diff
genkey
merge
mergeTable
query
readfile
sign
string
table
verify
writeTable

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.