lile

package module
v0.0.0-...-ed0f4a8 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 22, 2022 License: MIT Imports: 20 Imported by: 0

README

Actions Status

Lile is a application generator (think create-react-app, rails new or django startproject) for gRPC services in Go and a set of tools/libraries.

The primary focus of Lile is to remove the boilerplate when creating new services by creating a basic structure, test examples, Dockerfile, Makefile etc.

Lile comes with basic pre setup with pluggable options for things like...

Installation

Installing Lile is easy, using go get you can install the cmd line app to generate new services and the required libraries. First you'll need Google's Protocol Buffers installed.

$ brew install protobuf
$ go get -u github.com/lileio/lile/...

Guide

Creating a Service

Lile comes with a 'generator' to quickly generate new Lile services.

Lile follows Go's conventions around $GOPATH (see How to Write Go) and is smart enough to parse your new service's name to create the service in the right place.

If your Github username was tessthedog and you wanted to create a new service for posting to Slack you might use the following command.

lile new --name tessthedog/slack

Follow the command line instructions and this will create a new project folder for you with everything you need to continue.

Service Definitions

Lile creates gRPC and therefore uses protocol buffers as the language for describing the service methods, the requests and responses.

I highly recommend reading the Google API Design docs for good advice around general naming of RPC methods and messages and how they might translate to REST/JSON, via the gRPC gateway

An example of a service definition can be found in the Lile example project account_service

service AccountService {
  rpc List (ListAccountsRequest) returns (ListAccountsResponse) {}
  rpc GetById (GetByIdRequest) returns (Account) {}
  rpc GetByEmail (GetByEmailRequest) returns (Account) {}
  rpc AuthenticateByEmail (AuthenticateByEmailRequest) returns (Account) {}
  rpc GeneratePasswordToken (GeneratePasswordTokenRequest) returns (GeneratePasswordTokenResponse) {}
  rpc ResetPassword (ResetPasswordRequest) returns (Account) {}
  rpc ConfirmAccount (ConfirmAccountRequest) returns (Account) {}
  rpc Create (CreateAccountRequest) returns (Account) {}
  rpc Update (UpdateAccountRequest) returns (Account) {}
  rpc Delete (DeleteAccountRequest) returns (google.protobuf.Empty) {}
}

Generating RPC Methods

By default Lile will create a example RPC method and a simple message for request and response.

syntax = "proto3";
option go_package = "github.com/tessthedog/slack";
package slack;

message Request {
  string id = 1;
}

message Response {
  string id = 1;
}

service Slack {
  rpc Read (Request) returns (Response) {}
}

Let's modify this to be a real service and add our own method.

We're going to create an Announce method that will announce a message to a Slack room.

We're assuming that the Slack team and authentication is already handled by the services configuration, so a user of our service only needs to provide a room and their message. The service is going to send the special Empty response, since we only need to know if an error occurred and don't need to know anything else.

Our proto file now looks like this...

syntax = "proto3";
option go_package = "github.com/tessthedog/slack";
import "google/protobuf/empty.proto";
package slack;

message AnnounceRequest {
  string channel = 1;
  string msg = 2;
}

service Slack {
  rpc Announce (AnnounceRequest) returns (google.protobuf.Empty) {}
}

We now run the protoc tool with our file and the Lile method generator plugin.

protoc -I . slack.proto --lile-server_out=. --go_out=plugins=grpc:$GOPATH/src

Handily, Lile provides a Makefile with each project that has a proto build step already configured. So we can just run that.

make proto

We can see that Lile will create two files for us in the server directory.

$ make proto
protoc -I . slack.proto --lile-server_out=. --go_out=plugins=grpc:$GOPATH/src
2017/07/12 15:44:01 [Creating] server/announce.go
2017/07/12 15:44:01 [Creating test] server/announce_test.go

Let's take a look at the announce.go file that's created for us.

package server

import (
    "errors"

    "github.com/golang/protobuf/ptypes/empty"
    "github.com/tessthedog/slack"
    context "golang.org/x/net/context"
)

func (s SlackServer) Announce(ctx context.Context, r *slack.AnnounceRequest) (*empty.Empty, error) {
  return nil, errors.New("not yet implemented")
}

We can now fill in this generated method with the correct implementation. But let's start with a test!

Running & Writing Tests

When you generate an RPC method with Lile a counterpart test file is also created. For example, given our announce.go file, Lile will create announce_test.go in the same directory.

This should look something like the following..

package server

import (
	"testing"

	"github.com/tessthedog/slack"
	"github.com/stretchr/testify/assert"
	context "golang.org/x/net/context"
)

func TestAnnounce(t *testing.T) {
	ctx := context.Background()
	req := &slack.AnnounceRequest{}

	res, err := cli.Announce(ctx, req)
	assert.Nil(t, err)
	assert.NotNil(t, res)
}

You can now run the tests using the Makefile and running make test...

$ make test
=== RUN   TestAnnounce
--- FAIL: TestAnnounce (0.00s)
        Error Trace:    announce_test.go:16
        Error:          Expected nil, but got: &status.statusError{Code:2, Message:"not yet implemented", Details:[]*any.Any(nil)}
        Error Trace:    announce_test.go:17
        Error:          Expected value not to be nil.
FAIL
coverage: 100.0% of statements
FAIL    github.com/tessthedog/slack/server  0.011s
make: *** [test] Error 2

Our test failed because we haven't implemented our method, at the moment we're returning an error of "unimplemented" in our method.

Let's implement the Announce method in announce.go, here's an example using nlopes' slack library.

package server

import (
	"os"
	"google.golang.org/grpc/codes"
	"google.golang.org/grpc/status"

	"github.com/golang/protobuf/ptypes/empty"
	"github.com/tessthedog/slack"
	sl "github.com/nlopes/slack"
	context "golang.org/x/net/context"
)

var api = sl.New(os.Getenv("SLACK_TOKEN"))

func (s SlackServer) Announce(ctx context.Context, r *slack.AnnounceRequest) (*empty.Empty, error) {
	_, _, err := api.PostMessage(r.Channel, r.Msg, sl.PostMessageParameters{})
	if err != nil {
		return nil, status.Errorf(codes.Internal, err.Error())
	}

	return &empty.Empty{}, nil
}

Let's fill out our testing request and then run our tests again...

package server

import (
	"testing"

	"github.com/tessthedog/slack"
	"github.com/stretchr/testify/assert"
	context "golang.org/x/net/context"
)

func TestAnnounce(t *testing.T) {
	ctx := context.Background()
	req := &slack.AnnounceRequest{
		Channel: "@alex",
		Msg:     "hellooo",
	}

	res, err := cli.Announce(ctx, req)
	assert.Nil(t, err)
	assert.NotNil(t, res)
}

Now if I run the tests with my Slack token as an ENV variable, I should see a passing test!

$ alex@slack: SLACK_TOKEN=zbxkkausdkasugdk make test
go test -v ./... -cover
?       github.com/tessthedog/slack [no test files]
=== RUN   TestAnnounce
--- PASS: TestAnnounce (0.32s)
PASS
coverage: 75.0% of statements
ok      github.com/tessthedog/slack/server  0.331s  coverage: 75.0% of statements
?       github.com/tessthedog/slack/slack   [no test files]
?       github.com/tessthedog/slack/slack/cmd       [no test files]
?       github.com/tessthedog/slack/subscribers     [no test files]

Using the Generated cmds

Lile generates a cmd line application based on cobra when you generate your service. You can extend the app with your own cmds or use the built-in cmds to run the service.

Running the cmd line app without any arguments will print the generated help.

For example go run orders/main.go

up

Running up will run both the RPC server and the pubsub subscribers.

go run orders/main.go up

Adding your own cmds

To add your own cmd, you can use the built in generator from cobra which powers Lile's cmds

$ cd orders
$ cobra add import

You can now edit the file generated to create your cmd, cobra will automatically add the cmd's name to the help.

Documentation

Overview

Package lile provides helper methods to quickly create RPC based services that have metrics, tracing and pub/sub support

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AddStreamInterceptor 

func AddStreamInterceptor(sint grpc.StreamServerInterceptor)

AddStreamInterceptor adds a stream interceptor to the RPC server

func AddUnaryInterceptor 

func AddUnaryInterceptor(unint grpc.UnaryServerInterceptor)

AddUnaryInterceptor adds a unary interceptor to the RPC server

func BaseCommand 

func BaseCommand(serviceName, shortDescription string) *cobra.Command

BaseCommand provides the basic flags vars for running a service

func ContextClientInterceptor 

func ContextClientInterceptor() grpc.UnaryClientInterceptor

ContextClientInterceptor passes around headers for tracing and linkerd

func Name 

func Name(n string)

Name sets the name for the service

func NewTestServer 

func NewTestServer(s *grpc.Server) (string, func())

NewTestServer is a helper function to create a gRPC server on a non-network socket and it returns the socket location and a func to call which starts the server

func RegisterHealthProbe 

func RegisterHealthProbe(name string, ready func(ctx context.Context) error, alive func(ctx context.Context) error) (started func())

RegisterHealthProbe adds new probes to the health endpoints. Multiples can be added and the name is used to give some meaning to what's currently being probed. The ready func is use when the application gets in a state where it can't handle request but can recover. The alive func is used to determine service health, if this returns an error then it's saying that the application is in a state where it will have to be restarted. The started func is to be called when the service is has done its initial initialisation, in Kubernetes the ready and alive funcs won't be called until started as been executed.

func Run 

func Run() error

Run is a blocking cmd to run the gRPC and metrics server. You should listen to os signals and call Shutdown() if you want a graceful shutdown or want to handle other goroutines

func ServeGRPC 

func ServeGRPC(started func()) error

ServeGRPC creates and runs a blocking gRPC server

func Server 

func Server(r func(s *grpc.Server))

Server attaches the gRPC implementation to the service

func Shutdown 

func Shutdown()

Shutdown gracefully shuts down the gRPC and metrics servers

func TestConn 

func TestConn(addr string) *grpc.ClientConn

TestConn is a connection that connects to a socket based connection

func URLForService 

func URLForService(name string) string

URLForService returns a service URL via a registry or a simple DNS name if not available via the registry

Types

type RegisterImplementation 

type RegisterImplementation func(s *grpc.Server)

RegisterImplementation allows you to register your gRPC server

type Registry 

type Registry interface {
	// Register a service
	Register(s *Service) error
	// Deregister a service
	DeRegister(s *Service) error
	// Get a service by name
	Get(name string) (string, error)
}

Registry is the interface to implement for external registry providers

type ServerConfig 

type ServerConfig struct {
	Port int
	Host string
}

ServerConfig is a generic server configuration

func (*ServerConfig) Address 

func (c *ServerConfig) Address() string

Address Gets a logical addr for a ServerConfig

type Service 

type Service struct {
	ID   string
	Name string

	// Interceptors
	UnaryInts  []grpc.UnaryServerInterceptor
	StreamInts []grpc.StreamServerInterceptor

	// The RPC server implementation
	GRPCImplementation RegisterImplementation
	GRPCOptions        []grpc.ServerOption

	// gRPC and Prometheus endpoints
	Config           ServerConfig
	PrometheusConfig ServerConfig

	// Probe server for managing health endpoints and registering new probes
	ProbeServer probe.Server

	// Registry allows Lile to work with external registries like
	// consul, zookeeper or similar
	Registry Registry

	// Private utils. Exposed so they can be used if needed
	ServiceListener  net.Listener
	GRPCServer       *grpc.Server
	PrometheusServer *http.Server
	// contains filtered or unexported fields
}

Service is a gRPC based server with extra features

func GlobalService 

func GlobalService() *Service

GlobalService returns the global service

func NewService 

func NewService(n string) *Service

NewService creates a new service with a given name

Source Files

View all Source files

Directories

Path Synopsis
cmd
statik
Package statik contains static assets.
 Package statik contains static assets.
Package statik contains static assets.
 Package statik contains static assets.
Package test is a generated protocol buffer package.
 Package test is a generated protocol buffer package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.