pAPI

README

pAPI (= payments API)

pAPI is a payments API written in Go, a fictional service that offers standard CRUD functionality on Payment resources.

The main objective of this repo is serving as an example of what I consider modern software engineering practices applied to the development of backend architectures based in microservices. It is aimed at covering not only the development of the software itself, but also the tools and strategies needed to walk the full way from writing the first line of code to getting the service in production, following the principles around agile development and DevOps.

Architecture overview

The payments API is implemented as a microservice that offers a REST API that allows clients to manage payment resources by offering standard CRUD functionality. API messages are written using the JSON format and are conformant with the json:api specification.

Swagger/OpenAPI is used to specify the API contract with clients. Swagger allows writing API specifications using the YAML format. Apart from serving as documentation of the API's exported functionality and exptected inputs and outputs, swagger files can be used to automatically generate model and boilerplate code which takes care of common tasks such as request handling and input validation.

Test-Driven Development/Behaviour-Driven Development

The Payments Service has been developed using TDD from start to finish. The development process starts by defining how the system as a whole should behave to satisfy business needs and requirements. These business requirements are usually expressed in the form of functionality the service is needed to offer to potential users and, thus, end to end or acceptance tests are the best kind of test to capture the essence of these requirements. Since these tests describe the system's capabilities from the viewpoint of an external user, they are usually part of the communication between technical and not technical stakeholders of the project. Because of this, Cucumber and the Gherkin syntax are great for acceptance tests, as they allow writing them in a form close to natural language, eliminating the need of having a technical background to read or write them.

This project's end to end tests are written in the form of feature files using Gherkin syntax. These files are then processed by godog, the semi-official implementation of Cucumber for Go. feature files, along with step implementations, can be found in the [e2e_test] folder.

The test strategy I used follows the well-known Test Pyramid approach, where e2e tests are at the top and unit tests form the base of the pyramid. Usually, e2e tests are more expensive in terms of time required to set up the environment and run them (as an example, e2e tests in this project are run against real infrastructure that gets deployed before the tests run and destroyed afterwards). Due to their higher cost, e2e tests are usually limited in number and only happy paths are checked as a way to guarantee that the system as a whole delivers the required functionality.

Deeper in the service logic, functionality is checked by unit tests. These tests are fast and are run several times during development. Writing the tests before any logic is implemented eases the process of defining what functionality is really needed, what architecture should be used and how the unit under test is expected to behave. As opposed to e2e tests, unit tests are developer tests, so the language used for development is also the most convenient to write unit tests in. Moreover, one of the great things about Go is its rich tooling, and testing is a first-class citizen in this tooling. Unit tests in this project use plain go test constructs. I don't even use an assertion library (with testify being one of the most prominent examples) in favor of standard mechanisms, such as reflect.DeepEqual. My opinion is that the little added value is not worth the time needed to learn and handle yet another API. Of course, this (as several other things) is debatable and I'm always open to being convinced of the opposite :).

Continous Integration/Continuous Deployment

A critical requirement of Continuous Integration workflows is that every commit to the master/trunk branch must build and pass tests. CI/CD platforms and tools are key to enable high-throughput teams that aim at release software at a fast pace.

This project uses Travis CI for CI/CD. Travis configuration is done using a single .travis.yml file that sets the environment up and then goes through each of the defined stages collecting their results. If any command returns an error, the build will be considered broken.

The configuration follows the usual lint-test-build cycle. A relevant point here is that the process is tailored to cover all tests, not only unit and integration ones but also end to end tests. These are not performed in a special, contrived environment. Instead, real infrastructure is created, the code just built is deployed in this infrastructure and end to end tests are done by using a test client to consume the API and check the results. Once the tests are done, the infrastructure is destroyed. Performing these tests in a test environment on real infrastructure, ideally as close as the production infrastructure the application will live in the future, builds a lot of confidence on the recently developed code.

In order for this approach to be practicable, infrastructure cannot be handled manually as this would be both time-consuming and error-prone. This is where Infrastructure as Code comes to the rescue.

Infrastructure as Code

Aside from allowing managing infrastructure in a quick and efficient manner, on the main benefits one gets from using IaC is reproducible deployments that are driven by configuration files which can be commited to a repository along with the rest of the source code.

Terraform is used in this project as IaC tool. As with other similar solutions, Terraform uses a proprietary DSL called HCL (Hashicorp Configuration Language) to declaratively describe infrastructure deployments. In this project I make use of Terraform backends to store remote state, so that infrastructure can be managed from both the Travis CI environment and my local development environment.

The cloud provider I used in this project is AWS, but all of the concepts and services can easily be translated to GCP, Azure or any other IaaS provider.

Amazon Web Services

Amazon offers a wide (wider, in fact) range of services and products as part of its cloud infrastructure offering. They provide different abstraction levels (PaaS, IaaS) and functionalities. One of the first and more popular services is EC2, which allows creating virtual machines with shared or dedicated resources that can be used to host one or more services. I will also make use of RDS to host the PostgreSQL database that is used for data persistence.

Infrastructure planning

The pAPI service is currently supported by two elements, an application server and a database. An EC2 t2.micro instance will be used as the application server and a RDS db.t2.micro instance will host the database. Both instances are quite small, but they are covered by AWS free tier :). When the service grows in complexity, I will explore the addition of new elements.

AWS setup

Setting everything up for automatic infrastructure configuration could be as easy as using root credentials to perform every needed change, but this is an obvious call for disaster. Best practices mandate users, groups and policies to be correctly configured to grant the minimum set of permissions needed to manage the resources described in Terraform configuration.

Within IAM (AWS identity and role management service), I created a travisci user to be used by the CI/CD pipeline and added it to a Terraformers group. I then created the needed policies and attached them to the group. When defining permissions policies, bear in mind that everything in AWS is a resource and that even the simplest configuration usually involves several types of them.

As an example, the basic configuration described in this project consists of an EC2 instance and a RDS instance, but it comprises the following resources:

• the EC2 and RDS instances themselves
• an AMI to launch the EC2 instance with
• an attached volume for storage
• security groups to define inbound and outbound traffic rules
• a key pair for SSH access

And one could also define a dedicated VPC, subnets and gateways instead of using the default ones.

The high level of granularity can sometimes make difficult to actually know what the minimum set of permissions looks like. A solution proposed by Amazon is using CloudTrail event history to know exactly what APIs are called when operating the infrastructure and use that information to narrow privileges down. For more information, see the discussion in this Terraform issue and have a look at this gist with example policies.

I also decided to use the same AWS account to give support to Terraform's backend for remote state. Doing so implies creating an S3 bucket to store the state file itself and a DynamoDB table to enable state locking. The instructions on the Terraform site are easy to follow. One gotcha I'd like to note here: the key in the DynamoDB table must be called LockID and the name is case-sensitive.

Resource state persistence

Choosing the right database is always an important decision in every application. It is a matter of selecting the best tool for the job, so the process boils down to analyzing carefully what data will be stored and how it will be used. In this case I had not much information about how clients will use the API or what the relations between payment resources and other types of resources are, so I made some assumptions and went for a low risk alternative.

PostgreSQL is one of the most popular relational databases out there. It's open source and has been developed for more than 20 years. Even though there are no indications about what a payment resource exactly represents, I assumed that strong consistency (and ACID-compliancy in general) is a must and thus opted for a RDBMS instead of a NoSQL or document-oriented alternative.

Besides, in the scenario depicted for the test there are no other resource types beyond payments, and would have been easy to implement persistence just by throwing JSON-marshalled payments into a resource collection in MongoDB. However, it is easy to imagine that a payment is not an isolated resource in a real-world use case. Instead, other resource types would exist which a payment would have relations with. The strict schema of a relational database looks like a better fit for this kind of situations. As an example, beneficiary and debtor account data is abstracted under a custom composite type, which in a real application would probably be on its own table in the schema (I used a composite type to keep the implementation simple).

I decided to go with raw queries through stdlib's database/sql with lib/pq as driver instead of using an ORM. The queries, even though they are big in terms of values and parameters, are not complex. Moreover, an ORM usually requires annotating model types, and those are automatically generated by go-swagger, so editing that code was not a good solution. An alternative would have been to tweak the templates used by go-swagger so that models are generated with the needed struct tags.

There are two main issues in the proposed solution that are still pending: correct handling of null values and using the QueryContext and ExecContext alternatives to implement timeouts and avoid unresponsiveness in the service in case of performance issues in the database. Unfortunately, lib/pq does not support context handling right now, although there are alternatives that do (such as jackc/pgx; I will give it a try the next time I work with a PostgreSQL database).