tika

Documentation

Overview

Package tika provides a client and server for downloading, starting, and using Apache Tika's (http://tika.apache.org) Server.

Start with basic imports:

import "github.com/google/go-tika/tika"

You will need a running Server to make API calls to. So, if you don't have a server that is already running and you don't have the Server JAR already downloaded, you can download one. The caller is responsible for removing the file when no longer needed.

err := tika.DownloadServer(context.Background(), "1.16", "tika-server-1.16.jar")
if err != nil {
	log.Fatal(err)
}

If you don't have a running Tika Server, you can start one.

s, err := tika.NewServer("tika-server-1.16.jar")
if err != nil {
	log.Fatal(err)
}
cancel, err := s.Start(context.Background())
if err != nil {
	log.Fatal(err)
}
defer cancel()

Pass tika.Options to NewServer control the Server's behavior.

To parse the contents of a file (or any io.Reader), you will need to open the io.Reader, create a client, and call client.Parse.

// import "os"
f, err := os.Open("path/to/file")
if err != nil {
	log.Fatal(err)
}
defer f.Close()

client := tika.NewClient(nil, s.URL())
body, err := client.Parse(context.Background(), f)

If you pass an *http.Client to tika.NewClient, it will be used for all requests.

Some functions return a custom type, like Parsers(), Detectors(), and MIMETypes(). Use these to see what features are supported by the current Tika server.

Index

• Constants
• func DownloadServer(ctx context.Context, version Version, path string) error
• type Client
  • func NewClient(httpClient *http.Client, urlString string) *Client
  • func (c *Client) Detect(ctx context.Context, input io.Reader) (string, error)
  • func (c *Client) Detectors(ctx context.Context) (*Detector, error)
  • func (c *Client) Language(ctx context.Context, input io.Reader) (string, error)
  • func (c *Client) LanguageString(ctx context.Context, input string) (string, error)
  • func (c *Client) MIMETypes(ctx context.Context) (map[string]MIMEType, error)
  • func (c *Client) Meta(ctx context.Context, input io.Reader) (string, error)
  • func (c *Client) MetaField(ctx context.Context, input io.Reader, field string) (string, error)
  • func (c *Client) MetaRecursive(ctx context.Context, input io.Reader) ([]map[string][]string, error)
  • func (c *Client) Parse(ctx context.Context, input io.Reader) (string, error)
  • func (c *Client) ParseRecursive(ctx context.Context, input io.Reader) ([]string, error)
  • func (c *Client) Parsers(ctx context.Context) (*Parser, error)
  • func (c *Client) Translate(ctx context.Context, input io.Reader, t Translator, src, dst string) (string, error)
  • func (c *Client) Version(ctx context.Context) (string, error)
• type Detector
• type MIMEType
• type Option
  • func WithHostname(h string) Option
  • func WithPort(p string) Option
  • func WithStartupTimeout(d time.Duration) Option
• type Parser
• type Server
  • func NewServer(jar string, options ...Option) (*Server, error)
  • func (s *Server) Start(ctx context.Context) (cancel func(), err error)
  • func (s *Server) URL() string
• type Translator
• type Version

Constants

const XTIKAContent = "X-TIKA:content"

XTIKAContent is the metadata field of the content of a file after recursive parsing. See ParseRecursive and MetaRecursive.

Functions

func DownloadServer

func DownloadServer(ctx context.Context, version Version, path string) error

DownloadServer downloads and validates the given server version, saving it at path. DownloadServer returns an error if it could not be downloaded/validated. Valid values for the version are 1.14. It is the caller's responsibility to remove the file when no longer needed. If the file already exists and has the correct MD5, DownloadServer will do nothing.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client represents a connection to a Tika Server.

func NewClient

func NewClient(httpClient *http.Client, urlString string) *Client

NewClient creates a new Client. If httpClient is nil, the http.DefaultClient will be used.

func (*Client) Detect

func (c *Client) Detect(ctx context.Context, input io.Reader) (string, error)

Detect gets the mimetype of the given input, returning the mimetype and an error. If the error is not nil, the mimetype is undefined.

func (*Client) Detectors

func (c *Client) Detectors(ctx context.Context) (*Detector, error)

Detectors returns the list of available Detectors for this server. To get all available detectors, iterate through the Children of every Detector.

func (*Client) Language

func (c *Client) Language(ctx context.Context, input io.Reader) (string, error)

Language detects the language of the given input, returning the two letter language code and an error. If the error is not nil, the language is undefined.

func (*Client) LanguageString

func (c *Client) LanguageString(ctx context.Context, input string) (string, error)

LanguageString detects the language of the given string, returning the two letter language code and an error. If the error is not nil, the language is undefined.

func (*Client) MIMETypes

func (c *Client) MIMETypes(ctx context.Context) (map[string]MIMEType, error)

MIMETypes returns a map from MIME Type name to MIMEType, or properties about that specific MIMEType.

func (*Client) Meta

func (c *Client) Meta(ctx context.Context, input io.Reader) (string, error)

Meta parses the metadata from the given input, returning the metadata and an error. If the error is not nil, the metadata is undefined.

func (*Client) MetaField

func (c *Client) MetaField(ctx context.Context, input io.Reader, field string) (string, error)

MetaField parses the metadata from the given input and returns the given field. If the error is not nil, the result string is undefined.

func (*Client) MetaRecursive

func (c *Client) MetaRecursive(ctx context.Context, input io.Reader) ([]map[string][]string, error)

MetaRecursive parses the given input and all embedded documents. The result is a list of maps from metadata key to value for each document. The content of each document is in the XTIKAContent field. See ParseRecursive to just get the content of each document. If the error is not nil, the result list is undefined.

func (*Client) Parse

func (c *Client) Parse(ctx context.Context, input io.Reader) (string, error)

Parse parses the given input, returning the body of the input and an error. If the error is not nil, the body is undefined.

func (*Client) ParseRecursive

func (c *Client) ParseRecursive(ctx context.Context, input io.Reader) ([]string, error)

ParseRecursive parses the given input and all embedded documents, returning a list of the contents of the input with one element per document. See MetaRecursive for access to all metadata fields. If the error is not nil, the result is undefined.

func (*Client) Parsers

func (c *Client) Parsers(ctx context.Context) (*Parser, error)

Parsers returns the list of available parsers and an error. If the error is not nil, the list is undefined. To get all available parsers, iterate through the Children of every Parser.

func (*Client) Translate

func (c *Client) Translate(ctx context.Context, input io.Reader, t Translator, src, dst string) (string, error)

Translate returns an error and the translated input from src language to dst language using t. If the error is not nil, the translation is undefined.

func (*Client) Version

func (c *Client) Version(ctx context.Context) (string, error)

Version returns the default hello message from Tika server.

type Detector

type Detector struct {
	Name      string
	Composite bool
	Children  []Detector
}

A Detector represents a Tika Detector. Detectors are used to get the filetype of a file. To get a list of all Detectors, see Detectors().

type MIMEType

type MIMEType struct {
	Alias     []string
	SuperType string
}

MIMEType represents a Tika MIME Type. To get a list of all MIME Types, see MIMETypes.

type Option

type Option func(*Server)

An Option can be passed to NewServer to configure the Server.

func WithHostname

func WithHostname(h string) Option

WithHostname returns an Option to set the host of the Server (default localhost).

func WithPort

func WithPort(p string) Option

WithPort returns an Option to set the port of the Server (default 9998).

func WithStartupTimeout

func WithStartupTimeout(d time.Duration) Option

WithStartupTimeout returns an Option to set the timeout for how long to wait for the Server to start (default 10s).

type Parser

type Parser struct {
	Name           string
	Decorated      bool
	Composite      bool
	Children       []Parser
	SupportedTypes []string
}

A Parser represents a Tika Parser. To get a list of all Parsers, see Parsers().

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server represents a Tika server. Create a new Server with NewServer, start it with Start, and shut it down with the close function returned from Start. There is no need to create a Server for an already running Tika Server since you can pass its URL directly to a Client.

func NewServer

func NewServer(jar string, options ...Option) (*Server, error)

NewServer creates a new Server.

func (*Server) Start

func (s *Server) Start(ctx context.Context) (cancel func(), err error)

Start starts the given server. Start will start a new Java process. The caller must call cancel() to shut down the process when finished with the Server. The given Context is used for the Java process, not for cancellation of startup.

func (*Server) URL

func (s *Server) URL() string

URL returns the URL of this Server.

type Translator

type Translator string

Translator represents the Java package of a Tika Translator.

const (
	Lingo24Translator   Translator = "org.apache.tika.language.translate.Lingo24Translator"
	GoogleTranslator    Translator = "org.apache.tika.language.translate.GoogleTranslator"
	MosesTranslator     Translator = "org.apache.tika.language.translate.MosesTranslator"
	JoshuaTranslator    Translator = "org.apache.tika.language.translate.JoshuaTranslator"
	MicrosoftTranslator Translator = "org.apache.tika.language.translate.MicrosoftTranslator"
	YandexTranslator    Translator = "org.apache.tika.language.translate.YandexTranslator"
)

Translators available by defult in Tika. You must configure all required authentication details in Tika Server (for example, an API key).

type Version

type Version string

A Version represents a Tika Server version.

const (
	Version114 Version = "1.14"
	Version115 Version = "1.15"
	Version116 Version = "1.16"
)

Supported versions of Tika Server.