instrumentation-api

README

Instrumentation API - Build API and Lambda Package

An Application Programming Interface (API) to manage instrumentation data, built with Golang and deployed on AWS Lambda.

Documentation

Documentation for the API is maintained in a Markdown file held at docs/APIDOC.md. A Postman documentation and testing environment is also maintained at tests/postman_environment.local.

How to Develop

Quickstart - Running the API Stack Locally with Docker Compose

1. Install, at a minimum, Docker Engine and Docker Compose. In place of these two, one can install Docker Desktop.

2. Copy the .env.example file to .env (e.g., cp .env.example .env). This provides configuration options to Docker Compose.

3. Run the ./startup.sh shell script to pull the most recent changes from the current upstream branch, build, and (re)start the Docker Compose services. You can now access the application locally through Docker.

Running a Database for Local Development

After starting up Docker Compose, you will find these two services (among others) on localhost

1. A Postgres database with postgis schema installed using the Docker image mdillon/postgis

2. pgAdmin4 using the Docker image dpage/pgadmin4

To modify the database using pgAdmin4, open a web browser and go to http://localhost:8081, or whichever port number is the value set to variable PGADMIN_PORT in .env.

Login with Email:postgres@postgres.com and Password:postgres respectively.

Create a database connection to the Postgres database by right-clicking servers --> register --> server in the left menu tree. Enter the following information and click save.

General Tab

Field	Value
Name	localhost (or other preferred name)

Connection Tab

Field	Value
Host name/address	postgres
Port	5432 (default)
Username	postgres
Password	postgres

Initialize the database and seed it with some data (docker-compose runs this for you)

Use the Query Tool in pgadmin4 and the .sql files in the database/ directory in this repository. You can find the query tool by expanding the left menu tree to Servers --> Databases --> postgres. Right click postgres --> Query Tool. From here, copy tables.sql into the Query Tool and run it by pressing f5. Note: to only run a portion of the SQL you've copied, you can highlight the section you want to run before hitting f5.

Running the GO API for Local Development

Either of these options starts the API at localhost:$API_PORT, where $API_PORT is the variable set in your project's .env file. The API uses JSON Web tokens (JWT) for authorization by default. To disable JWT for testing or development, you can set the environment variable JWT_DISABLED=TRUE.

With Visual Studio Code Debugger

You can use the launch.json file in this repository in lieu of go run main.go to run the API in the VSCode debugger. This takes care of the required environment variables to connect to the database.

Without Visual Studio Code Debugger

Set the following environment variables and type go run main.go from the top level of this repository.

* INSTRUMENTATION_DB_USER=postgres
* INSTRUMENTATION_DB_PASS=postgres
* INSTRUMENTATION_DB_NAME=postgres
* INSTRUMENTATION_DB_HOST=localhost
* INSTRUMENTATION_DB_SSLMODE=disable
* INSTRUMENTATION_LAMBDACONTEXT=FALSE
* JWT_DISABLED=FALSE

Note: When running the API locally, make sure environment variable INSTRUMENTATION_LAMBDACONTEXT is either not set or is set to INSTRUMENTATION_LAMBDACONTEXT=FALSE. _LAMBDA_SERVER_PORT and AWS_LAMBDA_RUNTIME_API should also be set if running under an AWS Lambda context.

Running Tests

Regression tests are maintained for the project in the aforementioned Postman environments. They are run automatically by GitHub Actions through the script test.sh.

In both cases, the Postman environment regression tests are run, then output. If the environment variable REPORT is set to true, then this output is sent to an HTML file. Otherwise, it is printed to the caller's stdout.

Running the Swagger UI to access API documentation locally

An API Document conforming to the OpenAPI 3.0.0 specification is generated from the most recent Postman Collection saved to tests/instrumentation-regression.postman_collection.json. When the collection file is modified and overwritten, an updated apidoc.json will be automatically created by a swagger_init docker service at docs/swagger/apidoc.json. To start the Swagger UI server and sync the apidoc.json with the Postman Collection, run docker compose -f docker-compose.swagger.yml up -d. This command is also executed in ./startup.sh.

Note:

• This service will need to be restarted if any changes are made to the Postman Collection file (i.e. when it is manually exported and overwritten).

• Unlike the postman collection, the .env.json file supplied to the migration script is NOT automatically generated. If you make any changes or additions to the Postman environment used tests/postman_environment.docker-compose.json, these changes must also be made to the configuration supplied to the apidoc generation script, docs/swagger/postman-compose.env.json.

• Swagger UI configuration can be adjusted with docs/swagger/swagger-config.json. See swagger-ui/docs/usage/configuration.md for the full list of configuration options.

How To Deploy

Postgres Database on AWS Relational Database Service (RDS)

Database should be initialized with the following SQL files in the order listed:

1. rds_init.sql (install PostGIS extensions)

2. tables.sql (create tables for application)

3. roles.sql (database roles, grants, etc.)

   Note: Change 'password' in roles.sql to a real password for the instrumentation_user account.

How to Update

Updating an instance of instrumentation-api is trivially completed by rebuilding the Docker container used by it, then restarting the service.

If a postgres database has already been created and is in use, updates are less trivial. Before rebuilding and restarting the aforementioned API instance, database migrations must be carried out manually. Snippets for doing so are supplied in database/snippets.