README
¶
TBLN
TBLN is a text format that represents the table.
This repository contains Go library for reading and writing files, and Go library for reading and writing RDBMS tables.
Here is a document about the library. See cmd/README.md for CLI Tool.
Features
- TBLN can contain multiple columns like CSV.
- Database tables and import/export are possible.
- It can include the signature with the hash value in the file.
- Merge, sync and diff is possible
- TBLN file and TBLN file
- TBLN file and DB Table
- DB Table and DB Table
Please refer to TBLN for the specification of TBLN.
Install
$ go get github.com/noborus/tbln
Notes: Requires a version of Go that supports modules. e.g. Go 1.13+
Example
Write example
Example of creating TBLN. (Error check is omitted)
package main
import (
"os"
"github.com/noborus/tbln"
)
func main() {
tb := tbln.NewTBLN()
tb.SetTableName("sample")
tb.SetNames([]string{"id", "name"})
tb.SetTypes([]string{"int", "text"})
tb.AddRows([]string{"1", "Bob"})
tb.AddRows([]string{"2", "Alice"})
tbln.WriteAll(os.Stdout, tb)
}
Execution result.
; TableName: sample
; created_at: 2019-04-06T01:05:17+09:00
; name: | id | name |
; type: | int | text |
| 1 | Bob |
| 1 | Alice |
Read example
All read to memory by tbln.ReadAll. Here, the table name is rewritten and output.
package main
import (
"log"
"os"
"github.com/noborus/tbln"
)
func main() {
if len(os.Args) <= 1 {
log.Fatal("Requires tbln file")
}
file, err := os.Open(os.Args[1])
if err != nil {
log.Fatal(err)
}
at, err := tbln.ReadAll(file)
if err != nil {
log.Fatal(err)
}
at.SetTableName("newtable")
err = tbln.WriteAll(os.Stdout, at)
if err != nil {
log.Fatal(err)
}
}
TBLN files can be imported and exported into the database.
Example of importing into PostgreSQL.
# \d sample
Table "public.sample"
Column | Type | Collation | Nullable | Default
--------+---------+-----------+----------+---------
id | integer | | |
name | text | | |
Example of exporting it to a TBLN file.
; TableName: sample
; character_octet_length: | | 1073741824 |
; created_at: 2019-04-06T02:03:43+09:00
; is_nullable: | YES | YES |
; numeric_precision: | 32 | |
; numeric_precision_radix: | 2 | |
; numeric_scale: | 0 | |
; postgres_type: | integer | text |
; Signature: | test | ED25519 | 6271909d82000c4f686785cf0f9080971470ad3247b091ca50f6ea12ccc96efde0e1ca77e16723ef0f9d781941dfb92bed094dbf2e4079dd25f5aa9f9f1aab01 |
; Hash: | sha256 | 65f7ce4e15ddc006153fe769b8f328c466cbd1dea4b15aa195ed63daf093668d |
; name: | id | name |
; type: | int | text |
| 1 | Bob |
| 1 | Alice |
See db/README.md for details.
You can also use tbln cli tool to quickly import and export to a database.
See cmd/README.md for details.
Documentation
¶
Overview ¶
Package tbln reads and writes tbln files.
tbln can contain multiple fields like csv. Also, it can include checksum and signature inside. TBLN contains three types of lines: data, comments, and extras. All rows end with a new line(LF). The number of fields in all rows must be the same.
Data begins with "| " and ends with " |". Multiple fields are separated by " | ". (Note that the delimiter contains spaces.)
| fields1 | fields2 | fields3 |
White space is considered part of a field. If "|" is included in the field, "|" must be duplicated.
| -> || , || -> |||
Comments begin with "# ". Comments are not interpreted.
# Comments
Extras begin with ";". extras can be interpreted as a header. Extras is basically written in the item name: value.
; ItemName: Value
Extras is optional. Extras has item names that are interpreted in some special ways.
Example ¶
Output: ; TableName: newtable ; name: | id | name | ; type: | int | text | | 1 | Bob | | 2 | Alice |
Index ¶
- Constants
- Variables
- func ColumnPrimaryKey(pKeys []Pkey, row []string) []string
- func DiffAll(writer io.Writer, t1, t2 Reader, diffMode DiffMode) error
- func JoinRow(row []string) string
- func SplitRow(str string) []string
- func WriteAll(writer io.Writer, tbln *TBLN) error
- type Compare
- type Definition
- func (d *Definition) AllTargetHash(target bool)
- func (d *Definition) ColumnNum() int
- func (d *Definition) ExtraValue(key string) interface{}
- func (d *Definition) GetDefinition() *Definition
- func (d *Definition) GetPKeyPos() ([]int, error)
- func (d *Definition) Names() []string
- func (d *Definition) PrimaryKey() []string
- func (d *Definition) SerializeHash() []byte
- func (d *Definition) SetExtra(keyName string, value string)
- func (d *Definition) SetHashes(hashes []string) error
- func (d *Definition) SetNames(names []string) error
- func (d *Definition) SetSignatures(sign []string) error
- func (d *Definition) SetTableName(name string)
- func (d *Definition) SetTimeStamp() error
- func (d *Definition) SetTypes(types []string) error
- func (d *Definition) TableName() string
- func (d *Definition) ToTargetHash(key string, target bool)
- func (d *Definition) Types() []string
- type DiffMode
- type DiffRow
- type Extra
- type FileReader
- type MergeMode
- type OwnReader
- type Pkey
- type Reader
- type Signature
- type Signatures
- type TBLN
- type Writer
Examples ¶
Constants ¶
const ( SHA256 = "sha256" // import crypto/sha256 SHA512 = "sha512" // import crypto/sha512 )
Types of supported hashes
const (
ED25519 = "ED25519"
)
Signature algorithm.
Variables ¶
var ESCAPE = regexp.MustCompile(`(\|+)`)
ESCAPE is escape | -> ||
var UNESCAPE = regexp.MustCompile(`\|(\|+)`)
UNESCAPE is unescape || -> |
Functions ¶
func ColumnPrimaryKey ¶
ColumnPrimaryKey return columns primary key.
Types ¶
type Compare ¶
type Compare struct { PK []Pkey // contains filtered or unexported fields }
Compare represents a structure for comparing two TBLN.
func NewCompare ¶
NewCompare returns a Reader interface.
func (*Compare) ReadDiffRow ¶
ReadDiffRow compares two rows and returns the difference.
type Definition ¶
type Definition struct { Comments []string Extras map[string]Extra Hashes map[string][]byte Signs Signatures // contains filtered or unexported fields }
Definition is common table definition struct.
func MergeDefinition ¶
func MergeDefinition(t1d, t2d *Definition) (*Definition, error)
MergeDefinition merges two tbln Definitions.
func (*Definition) AllTargetHash ¶
func (d *Definition) AllTargetHash(target bool)
AllTargetHash is set all target of hash.
func (*Definition) ColumnNum ¶
func (d *Definition) ColumnNum() int
ColumnNum returns the number of columns.
func (*Definition) ExtraValue ¶
func (d *Definition) ExtraValue(key string) interface{}
ExtraValue is return extra value.
func (*Definition) GetDefinition ¶
func (d *Definition) GetDefinition() *Definition
GetDefinition return Definition
func (*Definition) GetPKeyPos ¶
func (d *Definition) GetPKeyPos() ([]int, error)
GetPKeyPos return PrimaryKey position
func (*Definition) PrimaryKey ¶
func (d *Definition) PrimaryKey() []string
PrimaryKey return PrimaryKey
func (*Definition) SerializeHash ¶
func (d *Definition) SerializeHash() []byte
SerializeHash returns a []byte that serializes Hash's map.
func (*Definition) SetExtra ¶
func (d *Definition) SetExtra(keyName string, value string)
SetExtra is set Extra of the Definition.
func (*Definition) SetHashes ¶
func (d *Definition) SetHashes(hashes []string) error
SetHashes is set hashes.
func (*Definition) SetNames ¶
func (d *Definition) SetNames(names []string) error
SetNames is set Column Name to the Table.
func (*Definition) SetSignatures ¶
func (d *Definition) SetSignatures(sign []string) error
SetSignatures is set signatures.
func (*Definition) SetTableName ¶
func (d *Definition) SetTableName(name string)
SetTableName is set Table Name of the Definition..
func (*Definition) SetTimeStamp ¶
func (d *Definition) SetTimeStamp() error
SetTimeStamp is set time stamp. If there is no created_at, create it. Otherwise updated updated_at.
func (*Definition) SetTypes ¶
func (d *Definition) SetTypes(types []string) error
SetTypes is set Column Type to Table.
func (*Definition) TableName ¶
func (d *Definition) TableName() string
TableName returns the Table Name.
func (*Definition) ToTargetHash ¶
func (d *Definition) ToTargetHash(key string, target bool)
ToTargetHash is set as target of hash.
type DiffRow ¶
DiffRow represents the difference between two rows.
type Extra ¶
type Extra struct {
// contains filtered or unexported fields
}
Extra is table definition extra struct.
type FileReader ¶
type FileReader struct { *Definition // contains filtered or unexported fields }
FileReader reads records from a tbln file.
func NewReader ¶
func NewReader(r io.Reader) *FileReader
NewReader returns a new Reader that reads from r.
func (*FileReader) ReadRow ¶
func (tr *FileReader) ReadRow() ([]string, error)
ReadRow reads one record (a slice of fields) from tr.
type OwnReader ¶
type OwnReader struct { *TBLN // contains filtered or unexported fields }
OwnReader reads records from TBLN.
type Reader ¶
type Reader interface { ReadRow() ([]string, error) GetDefinition() *Definition }
Reader is TBLN Reader interface.
type Signature ¶
type Signature struct {
// contains filtered or unexported fields
}
Signature struct stores a signature, a name, and an algorithm.
type Signatures ¶
Signatures is a map of signature name and signature.
type TBLN ¶
type TBLN struct { *Definition RowNum int Rows [][]string }
TBLN represents TBLN format data. It includes TBLN definition and TBLN rows.
Example ¶
Output: ; TableName: sample ; name: | id | name | ; type: | int | text | | 1 | Bob | | 2 | Alice |
func ReadAll ¶
ReadAll reads all r and returns a tbln struct. ReadAll returns when the blank line is reached.
type Writer ¶
Writer writes records to a TBLN encoded file.
func (*Writer) WriteDefinition ¶
func (w *Writer) WriteDefinition(d *Definition) error
WriteDefinition writes Definition (comment and extra) to w.