weather

README

Conduit Connector for Weather

Conduit source connector for weather.

How to build?

Run make to build the connector.

Testing

Run make test to run all the unit tests.

Source

This source connector pulls data from OpenWeather and pushes the weather data to downstream resources via Conduit.

Configuration

name	description	required	default value
appid	your unique API key (you can always find it on your account page under https://home.openweathermap.org/api_keys)	true
city	city name to get the current weather for, ex: California, San Francisco, london. you can find the cities list {city.list.json.gz} on http://bulk.openweathermap.org/sample/	false	New York
units	units of measurement, for Fahrenheit use imperial, for Celsius use metric, for Kelvin use standard.	false	imperial
url	url that contains the weather data	false	https://api.openweathermap.org/data/2.5/weather
pollingPeriod	how often the connector will get data from the url, formatted as a time.Duration string	false	5m

Example

here's a pipeline configuration file sample:

   pipelines:
   weather-pipeline:
     status: running
     name: weather-pipeline
     description: get the current weather in California every 5 minutes
     connectors:
       con-weather:
         type: source
         plugin: standalone:weather
         name: weather-source
         settings:
           city: California
           appid: ${APPID}
           pollingPeriod: 2m
           units: metric
       con-file:
         type: destination
         plugin: builtin:file
         name: file-dest
         settings:
           path: ./weather.txt
           sdk.record.format: template
           sdk.record.format.options: '{{ toPrettyJson .Payload.After }}'

make sure to export your appid key to the env variable $APPID before running conduit. check Pipeline Configuration Files Docs to run this pipeline.

Results: the output file weather.txt will have a new weather reading every two minute, and would look something like:

{
  "base": "stations",
  "clouds": {
    "all": 100
  },
  "cod": 200,
  "coord": {
    "lat": 38.3004,
    "lon": -76.5074
  },
  "dt": 1675955436,
  "id": 4350049,
  "main": {
    "feels_like": 11.48,
    "humidity": 78,
    "pressure": 1022,
    "temp": 12.17,
    "temp_max": 13.94,
    "temp_min": 8.84
  },
  "name": "California",
  "sys": {
    "country": "US",
    "id": 2011802,
    "sunrise": 1675944220,
    "sunset": 1675982215,
    "type": 2
  },
  "timezone": -18000,
  "visibility": 10000,
  "weather": [
    {
      "description": "overcast clouds",
      "icon": "04d",
      "id": 804,
      "main": "Clouds"
    }
  ],
  "wind": {
    "deg": 150,
    "gust": 9.26,
    "speed": 6.17
  }
} 

Documentation

Index

• Variables
• func NewSource() sdk.Source
• func Specification() sdk.Specification
• type Source
  • func (s *Source) Ack(ctx context.Context, position sdk.Position) error
  • func (s *Source) Configure(ctx context.Context, cfg map[string]string) error
  • func (s *Source) CreateRequestURL() string
  • func (s *Source) Open(ctx context.Context, pos sdk.Position) error
  • func (s *Source) Parameters() map[string]sdk.Parameter
  • func (s *Source) Read(ctx context.Context) (sdk.Record, error)
  • func (s *Source) Teardown(ctx context.Context) error
• type SourceConfig
  • func (SourceConfig) Parameters() map[string]sdk.Parameter

Variables

var Connector = sdk.Connector{
	NewSpecification: Specification,
	NewSource:        NewSource,
	NewDestination:   nil,
}

Functions

func NewSource

func NewSource() sdk.Source

func Specification

func Specification() sdk.Specification

Specification returns the connector's specification.

Types

type Source

type Source struct {
	sdk.UnimplementedSource
	// contains filtered or unexported fields
}

func (*Source) Ack

func (s *Source) Ack(ctx context.Context, position sdk.Position) error

func (*Source) Configure

func (s *Source) Configure(ctx context.Context, cfg map[string]string) error

func (*Source) CreateRequestURL

func (s *Source) CreateRequestURL() string

func (*Source) Open

func (s *Source) Open(ctx context.Context, pos sdk.Position) error

func (*Source) Parameters

func (s *Source) Parameters() map[string]sdk.Parameter

func (*Source) Read

func (s *Source) Read(ctx context.Context) (sdk.Record, error)

func (*Source) Teardown

func (s *Source) Teardown(ctx context.Context) error

type SourceConfig

type SourceConfig struct {
	// url that contains the weather data
	URL string `json:"url" default:"https://api.openweathermap.org/data/2.5/weather"`
	// how often the connector will get data from the url
	PollingPeriod time.Duration `json:"pollingPeriod" default:"5m"`
	// city name to get the current weather for, ex: California, San Francisco, london. you can find the cities list
	// {city.list.json.gz} on http://bulk.openweathermap.org/sample/
	City string `json:"city" default:"new york"`
	// your unique API key (you can always find it on your account page under https://home.openweathermap.org/api_keys)
	APPID string `json:"appid" validate:"required"`
	// units of measurement, for Fahrenheit use imperial, for Celsius use metric, for Kelvin use standard.
	Units string `json:"units" default:"imperial" validate:"inclusion=imperial|standard|metric"`
}

func (SourceConfig) Parameters

func (SourceConfig) Parameters() map[string]sdk.Parameter