fiwarecrawlerlib

README

FIWARE Crawler Library

• FIWARE Crawler Library
  • Installation
  • Usage
  • Configuration
  • Contributing
  • License

The fiwarecrawlerlib is a Go package that provides utility functions for regularly crawling data and creating configuration groups and devices for a FIWARE IoT-Agent UL (Ultralight). It simplifies the integration of IoT devices with FIWARE IoT Agents using the Ultralight protocol for data communication.

Installation

To use this library in your Go project, you can install it with:

go get -u github.com/scilabs-bo/fiware-crawler-lib

Usage

package main

import (
    "github.com/scilabs-bo/fiware-crawler-lib"
)

func main() {
    // Create a new instance of the crawler
    crawler := fiwarecrawlerlib.New()

    // Example: Creating a new configuration group
    configGroup := crawler.NewConfigGroup()

    // Example: Creating a new device
    device := crawler.NewDevice()

    // Example: Upserting the configuration group and device
    crawler.UpsertConfigGroup(*configGroup)
    crawler.UpsertDevice(*device)
    // Customize the Device and ConfigGroup configuration if needed
    // ...

    // Start the crawler job
    crawler.StartJob(func() {
        // Define your crawling logic here
        // ...

        // Example: Crawling data and obtaining a map of attributes
		data := map[string]interface{}{
			"temperature": 25.5,
			"humidity":    60,
		}

		// Publish the data to an MQTT broker
		crawler.PublishMqtt(data)
    })
}

Configuration

The library relies on the following environment variables for configuration:

Name	Required	Default	Description
CRONTAB	true	-	Specifies the cron schedule for data crawling.
IOTA_HOST	true	-	FIWARE IoT-Agent UL host address.
IOTA_PORT	true	-	FIWARE IoT-Agent UL port number.
SERVICE	true	-	FIWARE service identifier.
SERVICE_PATH	true	-	FIWARE service path.
API_KEY	true	-	FIWARE API key for authentication.
RESOURCE	false	/iot/d	Resource path for data storage.
DEVICE_ID	true	-	Identifier for the IoT device.
ENTITY_TYPE	true	-	Type of the FIWARE entity associated with the device.
LOG_LEVEL	false	DEBUG	Log level for the library (TRACE, DEBUG, INFO, WARNING, ERROR, FATAL, PANIC).
MQTT_BROKER	false	mosquitto	MQTT broker address for data publishing.
MQTT_PORT	false	1883	MQTT broker port for data publishing.
CLIENT_ID	true	-	MQTT client identifier.
USERNAME	false	-	MQTT broker username (optional).
PASSWORD	false	-	MQTT broker password (optional).

Adjust the values according to your environment and requirements.

Contributing

Feel free to contribute to this library by opening issues or pull requests. Your feedback and contributions are highly appreciated.

License

This project is licensed under the MIT License.

Documentation

Overview

Package fiwarecrawlerlib provides utility functions for regularly crawling data and creating configuration groups and devices for a FIWARE IoT-Agent UL (Ultralight).

The main functionalities include configuring a crawler with scheduling, handling MQTT communication, and managing FIWARE service groups and devices. It is designed to simplify the integration of IoT devices with FIWARE IoT Agents using Ultralight protocol for data communication.

Index

• type Config
• type Crawler
  • func New() *Crawler
  • func (c *Crawler) NewConfigGroup() *i.ConfigGroup
  • func (c *Crawler) NewDevice() *i.Device
  • func (c *Crawler) PublishMqtt(data map[string]interface{})
  • func (c *Crawler) StartJob(jobFunc interface{})
  • func (c *Crawler) UpsertConfigGroup(cg i.ConfigGroup)
  • func (c *Crawler) UpsertDevice(d i.Device)

Types

type Config

type Config struct {
	Crontab string `env:"CRONTAB,required=true"`

	IotAHost string `env:"IOTA_HOST,required=true"`
	IotAPort int    `env:"IOTA_PORT,required=true"`

	Service     string `env:"SERVICE,required=true"`
	ServicePath string `env:"SERVICE_PATH,required=true"`

	ApiKey   i.Apikey   `env:"API_KEY,required=true"`
	Resource i.Resource `env:"RESOURCE,default=/iot/d"`

	DeviceId   i.DeciveId `env:"DEVICE_ID, required=true"`
	EntityType string     `env:"ENTITY_TYPE, required=true"`

	LogLevel string `env:"LOG_LEVEL,default=DEBUG"`

	MqttBroker string `env:"MQTT_BROKER,default=mosquitto"`
	MqttPort   int    `env:"MQTT_PORT,default=1883"`
	ClientId   string `env:"CLIENT_ID,required=true"`
	Username   string `env:"USERNAME,required=false"`
	Password   string `env:"PASSWORD,required=false"`
}

Config represents the configuration parameters for the fiwarecrawlerlib package. It is populated using environment variables.

type Crawler

type Crawler struct {
	Conf Config
	Iota i.IoTA
	Fs   i.FiwareService
	Cron *gocron.Scheduler
}

Crawler represents the main structure for the fiwarecrawlerlib package, encapsulating the configuration, FIWARE IoT-Agent, FiwareService, and the cron scheduler.

func New

func New() *Crawler

New creates a new Crawler instance and initializes it with the configuration parameters from environment variables. Return a pointer to the Crawler instance

func (*Crawler) NewConfigGroup

func (c *Crawler) NewConfigGroup() *i.ConfigGroup

NewConfigGroup creates a new ConfigGroup instance based on the configuration. It returns a pointer to the newly created ConfigGroup.

func (*Crawler) NewDevice

func (c *Crawler) NewDevice() *i.Device

NewDevice creates a new Device instance based on the configuration. It returns a pointer to the newly created Device.

func (*Crawler) PublishMqtt

func (c *Crawler) PublishMqtt(data map[string]interface{})

PublishMqtt publishes data to an MQTT broker based on the configuration parameters. data items will be joind to a ul payload, nested struct will be parsed to json.

func (*Crawler) StartJob

func (c *Crawler) StartJob(jobFunc interface{})

StartJob takes a function jobFunc, adds it to the cron scheduler and starts it blocking.

func (*Crawler) UpsertConfigGroup

func (c *Crawler) UpsertConfigGroup(cg i.ConfigGroup)

UpsertConfigGroup ensures the existence of a service group in the FIWARE IoT-Agent. It takes a ConfigGroup sg as input and returns no values.

func (*Crawler) UpsertDevice

func (c *Crawler) UpsertDevice(d i.Device)

UpsertDevice ensures the existence of a device in the FIWARE IoT-Agent. It takes a Device d as input and returns no values.