datawatch

README

DataWatch

DataWatch is a notification system for reporting data events. Users configure Listeners to indicate which paths they are interested in and what kind of notifications they would like to receive. The current implementation supports monitoring events from an iRODS data store via RabbitMQ and sending notifications via email, webdav, and webhook (HTTP requests). New data source types and notification types can be added in the future.

DataWatch is comprised of seven services:

• monitor-service: receives data store events, checks the database for listener path matches, uses the perm-service to verify permissions, and then passes notifications on to the notification services
• path-service: receives datastore entity IDs from monitor-service and returns the path for the corresponding data objects
• perm-service: receives paths from monitor-service to check iRODS permissions and returns whether or not the listening user has permission to view the paths
• api-service: RESTful service where users can configure their listeners
• email-service: notification service that bundles and then sends email notifications
• webdav-service: notification service that bundles and then PUTs notifications to a file via WebDAV
• webhook-service: notification service that bundles and then sends HTTP/S request notifications

DataWatch requires the following additional services:

• Go, all DataWatch services are written in Go
• Neo4j, graph database used to hold users and their listeners
• NATS Streaming, for communicating between EMail Service and DataWatch's monitor-service, and between the DataWatch services themselves
• an additional EMail service for doing the actual emailing

Architecture

Here is a diagram showing the different DataWatch services and their avenues of communication.

Installation

See README.md in the install directory