index

Documentation

Overview

Package index provides interfaces for indexing edocuments metadata and retrieving this metadata back from the index. Currently, there is only one implementation to those interfaces, using Bleve.

Index

• Constants
• func Create(path string) bleve.Index
• func CreateMapping() mapping.IndexMapping
• type BleveIndexer
  • func NewBleve(index bleve.Index, fs afero.Fs, libraryPath string, ...) *BleveIndexer
  • func (b *BleveIndexer) AddFile(file string) error
  • func (b *BleveIndexer) AddLibrary(batchSize int) error
  • func (b *BleveIndexer) Close() error
  • func (b *BleveIndexer) Count() (uint64, error)
  • func (b *BleveIndexer) Document(slug string) (Document, error)
  • func (b *BleveIndexer) Documents(IDs []string) (map[string]Document, error)
  • func (b *BleveIndexer) ID(meta DocumentWrite, file string) string
  • func (b *BleveIndexer) IndexingProgress() (Progress, error)
  • func (b *BleveIndexer) RemoveFile(file string) error
  • func (b *BleveIndexer) SameAuthors(slugID string, quantity int) ([]Document, error)
  • func (b *BleveIndexer) SameSeries(slugID string, quantity int) ([]Document, error)
  • func (b *BleveIndexer) SameSubjects(slugID string, quantity int) ([]Document, error)
  • func (b *BleveIndexer) Search(keywords string, page, resultsPerPage int) (result.Paginated[[]Document], error)
  • func (b *BleveIndexer) Slug(document DocumentWrite, batchSlugs map[string]struct{}) string
• type Document
• type DocumentWrite
  • func (d DocumentWrite) BleveType() string
• type Progress

Constants

const Version = "v2"

Version identifies the mapping used for indexing. Any changes in the mapping requires an increase of version, to signal that a new index needs to be created.

Functions

func Create

func Create(path string) bleve.Index

func CreateMapping

func CreateMapping() mapping.IndexMapping

Types

type BleveIndexer

type BleveIndexer struct {
	// contains filtered or unexported fields
}

func NewBleve

func NewBleve(index bleve.Index, fs afero.Fs, libraryPath string, read map[string]metadata.Reader) *BleveIndexer

NewBleve creates a new BleveIndexer instance using the passed parameters

func (*BleveIndexer) AddFile

func (b *BleveIndexer) AddFile(file string) error

AddFile adds a file to the index

func (*BleveIndexer) AddLibrary

func (b *BleveIndexer) AddLibrary(batchSize int) error

AddLibrary scans <libraryPath> for documents and adds them to the index in batches of <bathSize>

func (*BleveIndexer) Close

func (b *BleveIndexer) Close() error

Close closes the index

func (*BleveIndexer) Count

func (b *BleveIndexer) Count() (uint64, error)

Count returns the number of indexed documents

func (*BleveIndexer) Document

func (b *BleveIndexer) Document(slug string) (Document, error)

func (*BleveIndexer) Documents

func (b *BleveIndexer) Documents(IDs []string) (map[string]Document, error)

func (*BleveIndexer) ID

func (b *BleveIndexer) ID(meta DocumentWrite, file string) string

func (*BleveIndexer) IndexingProgress

func (b *BleveIndexer) IndexingProgress() (Progress, error)

func (*BleveIndexer) RemoveFile

func (b *BleveIndexer) RemoveFile(file string) error

RemoveFile removes a file from the index

func (*BleveIndexer) SameAuthors

func (b *BleveIndexer) SameAuthors(slugID string, quantity int) ([]Document, error)

SameAuthors returns an array of metadata of documents by the same authors which does not belong to the same collection

func (*BleveIndexer) SameSeries

func (b *BleveIndexer) SameSeries(slugID string, quantity int) ([]Document, error)

SameSeries returns an array of metadata of documents in the same series

func (*BleveIndexer) SameSubjects

func (b *BleveIndexer) SameSubjects(slugID string, quantity int) ([]Document, error)

SameSubjects returns an array of metadata of documents by other authors, different between each other, which have similar subjects as the passed one and does not belong to the same collection

func (*BleveIndexer) Search

func (b *BleveIndexer) Search(keywords string, page, resultsPerPage int) (result.Paginated[[]Document], error)

Search look for documents which match with the passed keywords. Returns a maximum <resultsPerPage> documents, offset by <page>

func (*BleveIndexer) Slug

func (b *BleveIndexer) Slug(document DocumentWrite, batchSlugs map[string]struct{}) string

As Bleve index is not updated until the batch is executed, we need to store the slugs processed in the current batch in memory to also compare the current doc slug against them.

type Document

type Document struct {
	metadata.Metadata
	ID          string
	Slug        string
	Highlighted bool
}

type DocumentWrite

type DocumentWrite struct {
	Document
	AuthorsEq  []string
	SeriesEq   string
	SubjectsEq []string
}

DocumentWrite is an extension to Document that is used only when writing to the index, as some of its fields are only used to perform searches and not returned

func (DocumentWrite) BleveType

func (d DocumentWrite) BleveType() string

BleveType is part of the bleve.Classifier interface and its purpose is to tell the indexer the type of the document, which will be used to decide which analyzer will parse it.

type Progress

type Progress struct {
	RemainingTime time.Duration
	Percentage    float64
}