rdss-archivematica-channel-adapter

command module
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 17, 2017 License: AGPL-3.0 Imports: 1 Imported by: 0

README

Travis CI GoDoc Coverage Status Go Report Card Sourcegraph

RDSS Archivematica Channel Adapter

Introduction

THIS IS A PROTOTYPE!

This repository holds the source code of the channel adapter that connects Archivematica to RDSS's messaging API.

The adapter is written in Go as a standalone application that runs next to Archivematica. Its main role is to abstract the complexities and specifics of the underlying queuing system from its users.

Usage

We're producing a single binary executable file: rdss-archivematica-channel-adapter with two subcommands: consumer and publisher. You can build the Docker image locally running the following command:

$ docker build -t rdss-archivematica-channel-adapter .

The consumer is the component that brings Archivematica functionality to RDSS. It consumes the messages that come from the stream and convert them into Archivematica-specific calls. You can start it running:

$ docker run rdss-archivematica-channel-adapter consumer

On the other hand, the publisher is the component that brings RDSS functionality to Archivematica. It is implemented as a gRPC server which and it encapsulates the asynchronous nature of the messaging interaction, exposing regular synchronous methods to the application logic or the client. You can start it running:

$ docker run rdss-archivematica-channel-adapter publisher

Certain configuration parameter are required though. See the configuration section in this document for more details.

Configuration

The adapter is not configurable via command-line flags. You can choose between environment variables and the configuration file, having the former method precedence over the latter.

Environment variables

The following is a list of supported environment variables. They need to be prefixed with the string RDSS_ARCHIVEMATICA_ADAPTER_, e.g. RDSS_ARCHIVEMATICA_ADAPTER_LOGGING.LEVEL=INFO. Notice that the dot is used to separate nested attributes.

String Default
LOGGING.LEVEL INFO
AMCLIENT.URL ``
AMCLIENT.USER ``
AMCLIENT.KEY ``
PUBLISHER.LISTEN 0.0.0.0:8000
PUBLISHER.TLS false
PUBLISHER.TLS_CERT_FILE ``
PUBLISHER.TLS_KEY_FILE ``
BROKER.BACKEND kinesis
BROKER.QUEUES.MAIN main
BROKER.QUEUES.INVALID invalid
BROKER.QUEUES.ERROR error
BROKER.KINESIS.ENDPOINT ``
BROKER.KINESIS.DYNAMODB_ENDPOINT ``
BROKER.KINESIS.APP_NAME rdss_am
Configuration file

The adapter will try to read config from $HOME/.rdss-archivematica-channel-adapter.toml and /etc/archivematica/rdss-archivematica-channel-adapter.toml. Alternatively, you can pass a different location with the global --config string flag.

Notice how the environment variables in the previous section map to the nested configuration sections in the configuration file, e.g.:

# RDSS Archivematica Channel Adapter

[logging]
level = "INFO"

[amclient]
url = "https://archivematica.internal:9000"
user = "demo"
key = "eid3Aitheijoo1ohce2pho4eiDei0lah"

[publisher]
listen = "0.0.0.0:8000"
tls = false
tls_cert_file = "/foo.crt"
tls_key_file = "/foo.key"

[broker]
backend = "kinesis"

    [broker.kinesis]
    # This adapter uses aws-sdk-go. The credentials must be defined using the
    # canonical environment variables, read more at https://goo.gl/xsWyS9.

    # The name of the Amazon Kinesis stream.
    stream = "rdss-archivematica"

    # Kinesis endpoint. It can be used when you want the client to speak to a
    # server not run by AWS, e.g. a local kinesalite instance used during testing.
    #endpoint = "https://127.0.0.1:4567"

Code reusability

A few Go packages found in this repository are agnostic to Archivematica and could be used by other vendors:

  • github.com/JiscRDSS/rdss-archivematica-channel-adapter/amclient is a Archivematica HTTP API client.
  • github.com/JiscRDSS/rdss-archivematica-channel-adapter/broker is a RDSS client conforming to the RDSS messaging API. Both consumer and publisher packages in this repository are use cases. If you want to know more, there is a comprehensive example in broker_test.go.
  • github.com/JiscRDSS/rdss-archivematica-channel-adapter/s3 is a small S3 wrapper used to download files.

Diagram

This diagram is not up to date but it's close to the current design:

RDSS Archivematica Channel Adapter Diagram

Documentation

The Go Gopher

There is no documentation for this package.

Directories

Path Synopsis
Package broker implements a low-level interface for communicating with RDSS via message brokers such Amazon Kinesis Stream or RabbitMQ.
Package broker implements a low-level interface for communicating with RDSS via message brokers such Amazon Kinesis Stream or RabbitMQ.
cmd
Package publisher exposes RDSS functionality to Archivematica.
Package publisher exposes RDSS functionality to Archivematica.
pb
Package pb is a generated protocol buffer package.
Package pb is a generated protocol buffer package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL