codepipeline-to-github

README

CodePipeline → Lambda → Github

Update a GitHub commit status via CodePipeline events

Table of Contents

• TL;DR
• Installation
• Deployment & Hosting
• Documentation
• Examples & Tests
• Code Standards
• Maintainers
• Contributing
• License

TL;DR

AWS CodePipeline lacks an easy way to update Github commit statuses (at this time). Launch this serverless application and immediately start updating commits as pipeline events occur. All you need is a Github personal access token and some AWS credentials.

Installation

Prerequisites

• An AWS account
  • Running functions locally requires permission to: CodePipeline and KMS
  • Deploying requires permission to: KMS, SSM, Secrets Manager and Cloud Formation
• AWS CLI (brew install awscli)
• Golang (brew install go)
• SAM CLI (brew tap aws/tap && brew install aws-sam-cli)
  • Running functions locally requires: Docker

Clone or go get the files locally

go get github.com/mrz1818/codepipeline-to-github
cd $GOPATH/src/github.com/mrz1818/codepipeline-to-github

Setup to run locally

1) Modify the event json to a recent pipeline execution and pipeline name

"detail": {
  "pipeline": "your-pipeline-name",
  "execution-id": "some-execution-id"
}

2) Modify the local-env.json file with your Github Personal Access Token

"StatusFunction": {
  "GITHUB_ACCESS_TOKEN": "your-token-goes-here"
}

3) Finally, run the handler which should produce null and the commit status should be updated

make run event="started"

Deployment & Hosting

This repository has CI integration using AWS CodePipeline.

Deploying to the master branch will automatically start the process of shipping the code to AWS Lambda.

Any changes to the environment via the AWS CloudFormation template will be applied. The actual build process can be found in the buildspec.yml file.

The application relies on AWS Secrets Manager and AWS SSM to store environment variables. Sensitive environment variables are encrypted using AWS KMS and then decrypted at runtime.

Deploy different environments by changing the <stage> to production or developement as example. The default stage is production if you do not specified.

Create Environment Encryption Key(s) (AWS)

Create a KMS Key per <stage> for your application(s) to encrypt environment variables

make create-env-key APPLICATION_STAGE_NAME="<stage>"

This will also store the kms_key_id in SSM located at: /<application>/<stage>/kms_key_id

Manage Environment Secrets (AWS)

• github_token is a personal token with access to make a webhook
• kms_key_id is from the previous step (Create Environment Encryption Keys)

Add or update your Github personal access token

make save-secrets \
      github_token="YOUR_GITHUB_TOKEN" \
      kms_key_id="YOUR_KMS_KEY_ID" \
      APPLICATION_STAGE_NAME="<stage>"

Create New CI & Hosting Environment (AWS)

This will create a new AWS CloudFormation stack with:

• (1) Lambda Function (Golang Runtime)
• (1) CloudWatch Event Rule to subscribe to Pipeline events
• (1) CloudWatch LogGroup for the Lambda function output
• (1) CodePipeline with multiple stages to deploy the application from Github
• (1) CodePipeline Webhook to receive Github notifications from a specific branch:name
• (1) CodeBuild Project to test, build and deploy the app
• (2) Service Roles for working with CodeBuild and CodePipeline

NOTE: Requires an existing S3 bucket for artifacts and sam-cli deployments (located in the Makefile)

One command will build, test, package and deploy the application to AWS. After initial deployment, updating the function is as simple as committing to Github.

make deploy

(Example) Customized deployment for another stage/branch

make deploy APPLICATION_STAGE_NAME="development" REPO_BRANCH="development"

If you make any adjustments to the command above, update the buildspec file accordingly.

Tear Down CI & Hosting Environment (AWS)

Remove the Stack(s)

make teardown

(Example) Teardown another stage

make teardown APPLICATION_STAGE_NAME="development"

Lambda Logging

View all the logs in AWS CloudWatch via Log Groups

/aws/lambda/<app_name>-<stage_name>

Documentation

The status handler does the following:

- Processes incoming CloudWatch events from CodePipeline
- Decrypts environment variables (Github Token)
- Gets the latest information from CodePipeline via an ExecutionID
- Determines the Github status based on the Execution status
- Initiates a http/post request to Github to update the commit status

Run the status function with different pipeline events

make run event="failed"

Release Deployment

goreleaser for easy binary or library deployment to Github and can be installed via: brew install goreleaser.

The .goreleaser.yml file is used to configure goreleaser.

Use make release-snap to create a snapshot version of the release, and finally make release to ship to production.

Makefile Commands

View all makefile commands

make help

List of all current commands:

all                        Run lint, test and vet
aws-param-certificate      Returns the ssm location for the domain ssl certificate id
aws-param-zone             Returns the ssm location for the host zone id
build                      Build the lambda function as a compiled application
clean                      Remove previous builds, test cache, and packaged releases
clean-mods                 Remove all the Go mod cache
coverage                   Shows the test coverage
create-env-key             Creates a new key in KMS for a new stage
create-secret              Creates an secret into AWS SecretsManager
decrypt                    Decrypts data using a KMY Key ID (awscli v2)
decrypt-deprecated         Decrypts data using a KMY Key ID (awscli v1)
deploy                     Build, prepare and deploy
encrypt                    Encrypts data using a KMY Key ID (awscli v2)
env-key-location           Returns the environment encryption key location
godocs                     Sync the latest tag with GoDocs
help                       Show this help message
invalidate-cache           Invalidates a cloudfront cache based on path
lambda                     Build a compiled version to deploy to Lambda
lint                       Run the Go lint application
package                    Process the CF template and prepare for deployment
release                    Full production release (creates release in Github)
release                    Runs common.release and then runs godocs
release-snap               Test the full release (build binaries)
release-test               Full production test release (everything except deploy)
replace-version            Replaces the version in HTML/JS (pre-deploy)
run                        Fires the lambda function (run event=started)
save-domain-info           Saves the zone id and the ssl id for use by CloudFormation
save-param                 Saves a plain-text string parameter in SSM
save-param-encrypted       Saves an encrypted string value as a parameter in SSM
save-secrets               Helper for saving Github token(s) to Secrets Manager (extendable for more secrets)
tag                        Generate a new tag and push (tag version=0.0.0)
tag-remove                 Remove a tag if found (tag-remove version=0.0.0)
tag-update                 Update an existing tag to current commit (tag-update version=0.0.0)
teardown                   Deletes the entire stack
test                       Runs vet, lint and ALL tests
test-short                 Runs vet, lint and tests (excludes integration tests)
test-travis                Runs tests via Travis (also exports coverage)
update-secret              Updates an existing secret in AWS SecretsManager
upload-files               Upload/puts files into S3 bucket
vet                        Run the Go vet application

Examples & Tests

All unit tests run via Travis CI and uses Go version 1.14.x. View the deployment configuration file.

Run all tests (including integration tests)

make test

Code Standards

Read more about this Go project's code standards.

Maintainers

MrZ

Contributing

View the contributing guidelines and please follow the code of conduct.

How can I help?

All kinds of contributions are welcome 🙌! The most basic way to show your support is to star 🌟 the project, or to raise issues 💬. You can also support this project by becoming a sponsor on GitHub 👏 or by making a bitcoin donation to ensure this journey continues indefinitely! 🚀

Credits

This application would not be possible without the work provided in these repositories:

• CPLiakas's SAM Golang Example
• InfoPark's Github Status
• Jenseickmeyer's Commit Status Bot
• Rowanu's SAM Golang Starter

License

Documentation

Overview

Package main is the CodePipeline status event receiver

More information: https://github.com/mrz1836/codepipeline-to-github