elasticbook

package module
v0.0.0-...-2ce39b0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 19, 2016 License: MIT Imports: 14 Imported by: 0

README

ElasticBook

Manage your Chrome bookmarks with Elasticsearch.

CLI Options

help
$ go run cmd/elasticbook/main.go -h

NAME:
   ElasticBook - Elasticsearch for your bookmarks

USAGE:
   main [global options] command [command options] [arguments...]

VERSION:
   0.0.1

COMMANDS:
   help, h  Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --command, -c  -c [alias|aliases|unalias|default|indices|index|count|health|parse|delete|web|persist]
   --search, -s   -s [term]
   --verbose, -V  I wanna read useless stuff
   --help, -h   show help
   --version, -v  print the version
count
$ go run cmd/elasticbook/main.go -c count
- Mobile Bookmarks (33)
- Bookmarks Bar (9)
- Other Bookmarks (10669)
Delete index

Sample usage (from go run code):

$ go run cmd/elasticbook/main.go -c delete
00] - elasticbook-20151226224925
01] - elasticbook-20151227201242
02] - foobar
[0-02]:  2
Want to delete the foobar index? [y/N]: y
&{Acknowledged:true}
Create index

Sample usage (from go run code):

$ go run cmd/elasticbook/main.go -c index
Node (10960/10960) 10m3s [====================================================================] 100%
Index created
- Mobile Bookmarks (33)
- Bookmarks Bar (9)
- Other Bookmarks (10918)
List indices

Sample usage (from go run code):

$ go run cmd/elasticbook/main.go -c indices
00] - elasticbook-20151227224924 (10956)
01] - elasticbook-20151228093443 (10960)
02] - elasticbook-20160103180734 (10979)
List aliases

Sample usage (from go run code):

$ go run cmd/elasticbook/main.go -c aliases
00] - elasticbook-20151227224924 (10956)    [old]
01] - elasticbook-20151228093443 (10960)    []
02] - elasticbook-20160103180734 (10979)    [elasticbookdefault]
Create alias

Sample usage (from go run code):

$ go run cmd/elasticbook/main.go -c alias
Index name: elasticbook-20151228073443
Alias name: default

00] - elasticbook-20151227224924:     []
01] - elasticbook-20151228073443:     [default]

The mapping

Here the mapping used. There is a "name_suggest" for the completion.

{
  "elasticbook-20160111213240" : {
    "mappings" : {
      "bookmark" : {
        "properties" : {
          "date_added" : {
            "type" : "date",
            "format" : "dateOptionalTime"
          },
          "id" : {
            "type" : "string"
          },
          "meta_info" : {
            "properties" : {
              "stars_id" : {
                "type" : "string"
              },
              "stars_imageData" : {
                "type" : "string"
              },
              "stars_isSynced" : {
                "type" : "string"
              },
              "stars_pageData" : {
                "type" : "string"
              },
              "stars_type" : {
                "type" : "string"
              }
            }
          },
          "name" : {
            "type" : "string"
          },
          "name_suggest" : {
            "type" : "completion",
            "analyzer" : "simple",
            "payloads" : false,
            "preserve_separators" : true,
            "preserve_position_increments" : true,
            "max_input_length" : 50
          },
          "sync_transaction_version" : {
            "type" : "string"
          },
          "type" : {
            "type" : "string"
          },
          "url" : {
            "type" : "string"
          }
        }
      }
    }
  }
}
Suggestion?

Yep, it's easy.

For instance: look for "ela" in this way:

POST /elasticbook-20160111213240/_suggest?pretty -d '{
    "yup" : {
        "text" : "ela",
        "completion" : {
            "field" : "name_suggest"
        }
    }
} '

And you'll receive this helpful suggestions:

{
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "yup" : [ {
    "text" : "ela",
    "offset" : 0,
    "length" : 3,
    "options" : [ {
      "text" : "ElasticSearch profiles index - Wiki",
      "score" : 14.0
    }, {
      "text" : "Blog - Bonsai - Hosted Elasticsearch",
      "score" : 14.0
    }, {
      "text" : "Elasticsearch - Installing Plugins",
      "score" : 14.0
    }, {
      "text" : "Bonsai - Hosted Elasticsearch",
      "score" : 14.0
    }, {
      "text" : "Discuss Elasticsearch, Logstash and Kibana | Elastic",
      "score" : 14.0
    } ]
  } ]
}

Web interface

$ go run cmd/cli/main.go --web
[martini] listening on :3000 (development)
[martini] Started POST /elasticbook/search for [::1]:51415
[martini] Found a total of 20 bookmarks
[martini] Completed 200 OK in 461.663807ms

There is a nice and handy autocomplete feature that uses the suggestion ES query while you type in the search bar:

autocomplete

Base search:

base search

Look for elastic and found these:

results

Elasticsearch

A POQ (plain old query)
$ go run cmd/elasticbook/main.go -c index

Node (9001/10870) 8m32s [=======================================================>------------]  83%

Once you indexed all your data, try this:

POST elasticbook/bookmark/_search

{
  "sort": [
    "_score",
    {
      "date_added": {
        "order": "asc"
      }
    }
  ],
  "query": {
    "filtered": {
      "query": {
        "bool": {
          "must": [
            {
              "match": {
                "name": "elastic"
              }
            }
          ]
        }
      }
    }
  }
}

Elastic

Elasticsearch client for Go.

Martini

Classy web framework for Go.

Utils

JQ

jq is a lightweight and flexible command-line JSON processor.

$ jq keys bookmarks_20151213.json
[
  "checksum",
  "roots",
  "version"
]

$ jq '.roots | keys' bookmarks_20151213.json
[
  "bookmark_bar",
  "other",
  "sync_transaction_version",
  "synced"
]

$ jq '.roots.bookmark_bar | keys' bookmarks_20151213.json
[
  "children",
  "date_added",
  "date_modified",
  "id",
  "name",
  "type"
]

$ jq '.roots.other | keys' bookmarks_20151213.json
[
  "children",
  "date_added",
  "date_modified",
  "id",
  "name",
  "type"
]

$ jq '.roots.other.children | length' bookmarks_20151213.json
10622

$ jq '.roots.other.children | .[42] | keys' bookmarks_20151213.json
[
  "date_added",
  "id",
  "meta_info",
  "name",
  "sync_transaction_version",
  "type",
  "url"
]

$ jq '.roots.other.children | .[] | .name ' bookmarks_20151213.json
...
...
...

$ jq '.roots.other.children | .[] | select(.url == "https://golang.org/")' bookmarks_20151215.json
{
  "date_added": "13094604045096757",
  "id": "37151",
  "name": "The Go Programming Language",
  "sync_transaction_version": "32917",
  "type": "url",
  "url": "https://golang.org/"
  }

Documentation

Index

Constants

View Source 
const (
	// DefaultRemote decides if the ES cluster is on Bonsai.io or it's local
	DefaultRemote = false

	// DefaultURL is the default ES local address
	DefaultURL = "http://127.0.0.1:9200"

	// DefaultVerbose decides if you wanna be bored by some noisy logs
	DefaultVerbose = false
)
View Source 
const DefaultAliasName = "elasticbookdefault"

DefaultAliasName is the Elasticsearch alias used in Searches

View Source 
const DefaultIndexName = "elasticbook"

DefaultIndexName is the Elasticsearch index

View Source 
const TypeName = "bookmark"

TypeName is the type used

Variables

View Source 
var DefaultFields = []string{"name", "url"}

DefaultFields is where to look when looking for bookmarks

Functions

This section is empty.

Types

type Base 

type Base struct {
	Children     []Bookmark `json:"children"`
	DateAdded    string     `json:"date_added"`
	DataModified string     `json:"date_modified"`
	ID           string     `json:"id"`
	Name         string     `json:"name"`
	NodeType     string     `json:"type"`
}

Base is a "folder-like" container of Bookmarks

func (*Base) String 

func (b *Base) String() string

type Bookmark 

type Bookmark struct {
	DateAdded              string `json:"date_added"`
	OriginalID             string `json:"id"`
	MetaInfo               Meta   `json:"meta_info,omitempty"`
	Name                   string `json:"name"`
	SyncTransactionVersion string `json:"sync_transaction_version"`
	Type                   string `json:"type"`
	URL                    string `json:"url"`
}

Bookmark is a bookmark entry

type BookmarkIndexable 

type BookmarkIndexable struct {
	DateAdded              time.Time     `json:"date_added"`
	OriginalID             string        `json:"id"`
	MetaInfo               MetaIndexable `json:"meta_info,omitempty"`
	Name                   string        `json:"name"`
	NameSuggest            NameSuggest   `json:"name_suggest"`
	SyncTransactionVersion string        `json:"sync_transaction_version"`
	Type                   string        `json:"type"`
	URL                    string        `json:"url"`
}

BookmarkIndexable is a bookmark entry with a sanitised MetaInfo

.Suggest: NewSuggestField(). 

Input("Cycling is fun.").
Output("Cycling is a fun sport.")

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is the ElasticBook wrapper to an Elastic Client The "elastic" package is really inspiring!

func ClientLocal 

func ClientLocal() (*Client, error)

ClientLocal connects to a local ES cluster

func ClientRemote 

func ClientRemote() (*Client, error)

ClientRemote connects to a remote ES cluster (Bonsai.io) Debug with this: 

/_nodes/http?pretty=1

https://github.com/olivere/elastic/wiki/Connection-Problems

func NewClient 

func NewClient(options ...ClientOptionFunc) (*Client, error)

NewClient Set up the default client

func (*Client) Alias 

func (c *Client) Alias(indexName string, aliasName string) (bool, error)

Alias creates an alias. It's enforced a constraint though: "No more than one index per alias" This means that, if the alias already exists, this method returns false.

func (*Client) AliasNames 

func (c *Client) AliasNames() ([]string, error)

AliasNames returns the list of existing aliases (just the names, sorted). Due to the constraint enforced by Client#Alias this slice should not contains dupes (^_^)

func (*Client) Aliases 

func (c *Client) Aliases() ([]string, error)

Aliases returns the list of existing aliases

func (*Client) Default 

func (c *Client) Default(indexName string) (bool, error)

Default switch the default alias to the given index name (if it exists). Returns true if the switch is successful.

func (*Client) Delete 

func (c *Client) Delete(indexName string)

Delete drops the index

func (*Client) Doctor 

func (c *Client) Doctor()

Doctor adds the Default alias to an index

func (*Client) Health 

func (c *Client) Health() (*elastic.ClusterHealthResponse, error)

Health check the status of the cluster

func (*Client) Index 

func (c *Client) Index(x *Root) (bool, error)

Index takes a parsed structure and index all the Bookmarks entries

func (*Client) IndexNames 

func (c *Client) IndexNames() ([]string, error)

IndexNames returns the list of existing indices (just the names)

func (*Client) Indices 

func (c *Client) Indices() ([]string, error)

Indices returns the list of existing indices

func (*Client) Mappings 

func (c *Client) Mappings() (map[string]interface{}, error)

Mappings returns the current index mapping

func (*Client) Parse 

func (c *Client) Parse(b []byte) (*Root, error)

Parse run the JSON parser

func (*Client) Search 

func (c *Client) Search(term string) (*elastic.SearchResult, error)

Search is the API for searching

func (*Client) Suggest 

func (c *Client) Suggest(term string) (elastic.SuggestResult, error)

Suggest performs a _suggest query

func (*Client) URL 

func (c *Client) URL() string

URL returns the current ES client/cluster URL

func (*Client) Unalias 

func (c *Client) Unalias(aliasName string) (bool, error)

Unalias deletes an alias

func (*Client) Version 

func (c *Client) Version() string

Version check the version of the cluster

type ClientOptionFunc 

type ClientOptionFunc func(*Client) error

ClientOptionFunc is a function that configures a Client. It is used in NewClient.

func SetElasticClient 

func SetElasticClient(elasticClient *elastic.Client) ClientOptionFunc

SetElasticClient decide which kind of elastic Client use (local or remote)

func SetURL 

func SetURL(u string) ClientOptionFunc

SetURL define the current URL used (just for convenience)

func SetVerbose 

func SetVerbose(vvv bool) ClientOptionFunc

SetVerbose define the verbose logging

type CountResult 

type CountResult struct {
	// contains filtered or unexported fields
}

CountResult contains the bookmarks counter

func (*CountResult) Add 

func (c *CountResult) Add(k string, v int)

Add a key value to the count container

func (*CountResult) String 

func (c *CountResult) String() string

func (*CountResult) Total 

func (c *CountResult) Total() int

Total return the grand total of Bookmark entries parsed/indexed

type Meta 

type Meta struct {
	StarsID        string `json:"stars.id"`
	StarsImageData string `json:"stars.imageData"`
	StarsIsSynced  string `json:"stars.isSynced"`
	StarsPageData  string `json:"stars.pageData"`
	StarsType      string `json:"stars.type"`
}

Meta contains the attached metadata to the Bookmark entry

type MetaIndexable 

type MetaIndexable struct {
	StarsID        string `json:"stars_id"`
	StarsImageData string `json:"stars_imageData"`
	StarsIsSynced  string `json:"stars_isSynced"`
	StarsPageData  string `json:"stars_pageData"`
	StarsType      string `json:"stars_type"`
}

MetaIndexable contains the attached metadata to the Bookmark entry w/o dots

type NameSuggest 

type NameSuggest struct {
	Input   []string `json:"input"`
	Oputput string   `json:"output"`
}

NameSuggest contains the input for the suggestion engine https://www.elastic.co/guide/en/elasticsearch/reference/current/search-suggesters-completion.html Example: 

POST /elasticbook-20160110174219/bookmark/_suggest?pretty

{
   "name_suggest" : {
       "text" : "Medium",
       "completion" : {
           "field" : "name_suggest"
       }
   }
	}

type Root 

type Root struct {
	Checksum string `json:"checksum"`
	Version  int    `json:"version"`
	Roots    Roots  `json:"roots"`
}

Root is the root of the Bookmarks tree

func (*Root) Count 

func (r *Root) Count() (c *CountResult)

Count returns a map with the RootFolder name and the count

type Roots 

type Roots struct {
	BookmarkBar            Base   `json:"bookmark_bar"`
	Other                  Base   `json:"other"`
	SyncTransactionVersion string `json:"sync_transaction_version"`
	Synced                 Base   `json:"synced"`
}

Roots is the container of the 4 main bookmark structure (high level)

Source Files

View all Source files

Directories

Path Synopsis
cmd
cli command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.