parse

Documentation

Overview

Package parse of the Dataflow kit is used by parse.d service which parses html content from web pages following the rules described in Payload JSON file.

Index

• func DecodeParseRequest(ctx context.Context, r *http.Request) (interface{}, error)
• func EncodeParseResponse(_ context.Context, w http.ResponseWriter, response interface{}) error
• func HealthCheckHandler(w http.ResponseWriter, r *http.Request)
• func MakeParseEndpoint(svc Service) endpoint.Endpoint
• func NewHttpHandler(ctx context.Context, endpoint Endpoints) http.Handler
• type Config
• type Endpoints
  • func (e Endpoints) Parse(p scrape.Payload) (io.ReadCloser, error)
• type HTMLServer
  • func Start(cfg Config) *HTMLServer
  • func (htmlServer *HTMLServer) Stop() error
• type ParseService
  • func (ps ParseService) Parse(p scrape.Payload) (io.ReadCloser, error)
• type Service
  • func NewHTTPClient(instance string) (Service, error)
• type ServiceMiddleware
  • func LoggingMiddleware(logger *zap.Logger) ServiceMiddleware
  • func Metrics(requestCount metrics.Counter, requestLatency metrics.Histogram) ServiceMiddleware

Functions

func DecodeParseRequest

func DecodeParseRequest(ctx context.Context, r *http.Request) (interface{}, error)

DecodeParseRequest decodes request sent to Parser if error occures, server returns 400 Bad Request

func EncodeParseResponse

func EncodeParseResponse(_ context.Context, w http.ResponseWriter, response interface{}) error

EncodeParseResponse encodes response returned by Parser

func HealthCheckHandler

func HealthCheckHandler(w http.ResponseWriter, r *http.Request)

HealthCheckHandler is used to check if Parse service is alive

func MakeParseEndpoint

func MakeParseEndpoint(svc Service) endpoint.Endpoint

MakeParseEndpoint creates Parse Endpoint

func NewHttpHandler

func NewHttpHandler(ctx context.Context, endpoint Endpoints) http.Handler

NewHttpHandler mounts all of the service endpoints into an http.Handler.

Types

type Config

type Config struct {
	Host    string
	Version string
}

Config provides basic configuration

type Endpoints

type Endpoints struct {
	ParseEndpoint endpoint.Endpoint
}

Endpoints wrapper

func (Endpoints) Parse

func (e Endpoints) Parse(p scrape.Payload) (io.ReadCloser, error)

Parse method is used for sending payload requests to parse service.

type HTMLServer

type HTMLServer struct {
	// contains filtered or unexported fields
}

HTMLServer represents the web service that serves up HTML

func Start

func Start(cfg Config) *HTMLServer

Start func launches Parsing service

func (*HTMLServer) Stop

func (htmlServer *HTMLServer) Stop() error

Stop turns off the HTML Server

type ParseService

type ParseService struct {
}

ParseService implements service with empty struct

func (ParseService) Parse

func (ps ParseService) Parse(p scrape.Payload) (io.ReadCloser, error)

Parse service processes fetched page following the rules from Payload.

type Service

type Service interface {
	Parse(scrape.Payload) (io.ReadCloser, error)
}

Service defines Parse service interface

func NewHTTPClient

func NewHTTPClient(instance string) (Service, error)

NewHTTPClient returns an Parse Service backed by an HTTP server living at the remote instance. We expect instance to come from a service discovery system, so likely of the form "host:port". We bake-in certain middlewares, implementing the client library pattern.

type ServiceMiddleware

type ServiceMiddleware func(Service) Service

ServiceMiddleware defines a middleware for a Parse service

func LoggingMiddleware

func LoggingMiddleware(logger *zap.Logger) ServiceMiddleware

LoggingMiddleware logs Parse Service endpoints

func Metrics

func Metrics(requestCount metrics.Counter,
	requestLatency metrics.Histogram) ServiceMiddleware

metrics function