gateboard

README

gateboard

gateboard resolves AWS Private API Gateway ID.

Services

Service	Description
gateboard	Holds database of key value mappings: gateway_name => gateway_id. You can populate the database as you wish.
gateboard-discovery	Can be used to scan AWS API Gateway APIs and to save the name x ID mappings into gateboard.
gateboard-cache	Can be used as local fast cache to save resources on a centralized main gateboard service.

TODO

• SQS listener
• Client with async update
• Create mongodb index on startup
• Define TTL on server record (60s), restrict acceptable TTL range on client (60s..600s)
• Repository tests
• HTTP server tests
• SQS tests
• Docker image
• Client tests
• Refactor config
• Repository DynamoDB
• Optional authentication
• Generate token for optional authentication
• Gateway load sharing
• Repository redis
• Cache service
• Discovery service
• Zap logging
• Metrics
• Alarm for main server unreachable
• Tracing
• Benchmark
• User guide

Build

git clone https://github.com/udhos/gateboard
cd gateboard
CGO_ENABLED=0 go install ./...

Testing repository mongo

Start mongodb:

docker run --rm --name mongo-main -p 27017:27017 -d mongo

Run repository tests:

export TEST_REPO_MONGO=true ;# enable mongodb tests
go test -count=1 -run TestRepository ./cmd/gateboard

Testing repository dynamodb

Create a dynamodb table named gateboard_test with partition key gateway_name.

Make sure the table is empty before running the tests.

Run repository tests:

export TEST_REPO_DYNAMO=true ;# enable dynamodb tests
go test -count=1 -run TestRepository ./cmd/gateboard

Testing repository redis

Start redis:

docker run --rm --name redis-main -p 6379:6379 -d redis

Run repository tests:

export TEST_REPO_REDIS=true ;# enable redis tests
go test -count=1 -run TestRepository ./cmd/gateboard

Running both servers on same host

Main:

docker run --rm --name mongo-main -p 27017:27017 -d mongo
export QUEUE_URL=https://sqs.us-east-1.amazonaws.com/123456789012/gateboard
gateboard

Fallback:

docker run --rm --name mongo-fallback -p 27018:27017 -d mongo
export LISTEN_ADDR=:8181                   ;# main 8080
export HEALTH_ADDR=:9999                   ;# main 8888
export METRICS_ADDR=:3001                  ;# main 3000
export MONGO_URL=mongodb://localhost:27018 ;# main mongodb://localhost:27017
gateboard

Run interactive client:

gateboard-client-example

Optional Authentication

Enable WRITE_TOKEN=true in order to require token authentication for write requests.

export WRITE_TOKEN=true

Make sure the repository has the token token1 assigned to gateway gw2.

Example for mongodb:

db.gateboard.insertOne({"gateway_name":"gw2","token":"token1"})

Now requests to update gateway gw2 must include the token token1.

curl -X PUT -s -d '{"gateway_id":"id1","token":"token1"}' localhost:8080/gateway/gw2

{"gateway_name":"gw2","gateway_id":"id1"}

Otherwise the request will be denied.

curl -X PUT -v -d '{"gateway_id":"id2"}' localhost:8080/gateway/gw2
*   Trying ::1:8080...
* TCP_NODELAY set
* Connected to localhost (::1) port 8080 (#0)
> PUT /gateway/gw2 HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.68.0
> Accept: */*
> Content-Length: 20
> Content-Type: application/x-www-form-urlencoded
> 
* upload completely sent off: 20 out of 20 bytes
* Mark bundle as not supporting multiuse
< HTTP/1.1 401 Unauthorized
< Content-Type: application/json; charset=utf-8
< Date: Sat, 17 Dec 2022 00:59:22 GMT
< Content-Length: 65
< 
* Connection #0 to host localhost left intact
{"gateway_name":"gw2","gateway_id":"id2","error":"invalid token"}

Example

curl localhost:8080/gateway/gate1
{"gateway_name":"gate1","gateway_id":"","error":"gatewayGet: not found: repository gateway not found error"}

curl -X PUT -d '{"gateway_id":"id1"}' localhost:8080/gateway/gate1
{"gateway_name":"gate1","gateway_id":"id1"}

curl localhost:8080/gateway/gate1
{"gateway_name":"gate1","gateway_id":"id1"}

curl -X PUT -d '{"gateway_id":"id2"}' localhost:8080/gateway/gate1
{"gateway_name":"gate1","gateway_id":"id2"}

curl localhost:8080/gateway/gate1
{"gateway_name":"gate1","gateway_id":"id2"}

gateway-discovery

Save to server

Discovery writes directly to server.

Start server.

export REPO=mem
gateboard

Run discovery.

export SAVE=server
export DRY_RUN=false
gateboard-discovery

Dump database.

curl localhost:8080/dump | jq

Save to webhook

Discovery writes to webhook that forwards to SQS queue.

Start server.

export QUEUE_URL=https://sqs.us-east-1.amazonaws.com/123456789012/gateboard
export REPO=mem
gateboard

Run discovery.

export SAVE=webhook
# use lambda function url as webhook
export WEBHOOK_URL=https://xxxxxxxxxxxxx.lambda-url.us-east-1.on.aws
export DRY_RUN=false
gateboard-discovery

Dump database.

curl localhost:8080/dump | jq

Save to SQS

Discovery writes to SQS queue.

Start server.

export QUEUE_URL=https://sqs.us-east-1.amazonaws.com/123456789012/gateboard
export REPO=mem
gateboard

Run discovery.

export SAVE=sqs
export QUEUE_URL=https://sqs.us-east-1.amazonaws.com/123456789012/gateboard
export DRY_RUN=false
gateboard-discovery

Dump database.

curl localhost:8080/dump | jq

Save to SNS

Discovery writes to SNS topic that forwards to SQS queue.

Start server.

export QUEUE_URL=https://sqs.us-east-1.amazonaws.com/123456789012/gateboard
export REPO=mem
gateboard

Run discovery.

export SAVE=sns
export TOPIC_ARN=arn:aws:sns:us-east-1:123456789012:gateboard
export DRY_RUN=false
gateboard-discovery

Dump database.

curl localhost:8080/dump | jq

Save to lambda

Discovery writes to lambda function that forwards to SQS queue.

Start server.

export QUEUE_URL=https://sqs.us-east-1.amazonaws.com/123456789012/gateboard
export REPO=mem
gateboard

Run discovery.

export SAVE=lambda
export LAMBDA_ARN=arn:aws:lambda:us-east-1:123456789012:function:forward_to_sqs
export DRY_RUN=false
gateboard-discovery

Dump database.

curl localhost:8080/dump | jq

Docker

Docker hub:

https://hub.docker.com/r/udhos/gateboard

Pull from docker hub:

docker pull udhos/gateboard:0.0.0

Build recipe:

./docker/build.sh

Multiarch build recipe:

./docker/build-multiarch.sh

gateboard-discovery docker image

Docker hub:

https://hub.docker.com/r/udhos/gateboard-discovery

Pull from docker hub:

docker pull udhos/gateboard-discovery:0.0.0

Build recipe:

./docker/build-discovery.sh

Push:

docker push -a udhos/gateboard-discovery