river

Documentation

Index

• type Config
  • func NewConfig(data []byte) (*Config, error)
  • func NewConfigWithFile(name string) (*Config, error)
• type River
  • func NewRiver(c *Config) (*River, error)
  • func (r *River) Close()
  • func (r *River) Run() error
• type Rule
• type SourceConfig

Types

type Config

type Config struct {
	MyAddr     string
	MyUser     string
	MyPassword string

	ESAddr string

	StatAddr string

	ServerID uint32
	Flavor   string
	DataDir  string

	DumpExec string

	Sources []SourceConfig

	Rules []*Rule
}

func NewConfig

func NewConfig(data []byte) (*Config, error)

func NewConfigWithFile

func NewConfigWithFile(name string) (*Config, error)

type River

type River struct {
	// contains filtered or unexported fields
}

In Elasticsearch, river is a pluggable service within Elasticsearch pulling data then indexing it into Elasticsearch. We use this definition here too, although it may not run within Elasticsearch. Maybe later I can implement a acutal river in Elasticsearch, but I must learn java. :-)

func NewRiver

func NewRiver(c *Config) (*River, error)

func (*River) Close

func (r *River) Close()

func (*River) Run

func (r *River) Run() error

type Rule

type Rule struct {
	Schema string `toml:"schema"`
	Table  string `toml:"table"`
	Index  string `toml:"index"`
	Type   string `toml:"type"`

	// Default, a MySQL table field name is mapped to Elasticsearch field name.
	// Sometimes, you want to use different name, e.g, the MySQL file name is title,
	// but in Elasticsearch, you want to name it my_title.
	FieldMapping map[string]string `toml:"field"`

	// MySQL table information
	TableInfo *schema.Table
}

If you want to sync MySQL data into elasticsearch, you must set a rule to let use know how to do it. The mapping rule may thi: schema + table <-> index + document type. schema and table is for MySQL, index and document type is for Elasticsearch.

type SourceConfig

type SourceConfig struct {
	Schema string
	Tables []string
}