Go developers incomplete guide to writing a typical backend service
Table of contents
Intro
What is this repository?
This repository works as a guide explaining how to write the most common type of backend service in go. The guide consists of 2 complementary parts:
- A set of documents explaining various aspects of typical go backend service,
- Fully working codebase, implementing these documents in practice.
The topic of the example project is a Bike rental service backend.
The purpose of this project is to:
- Show how to structure medium to big go projects.
- Explain some high-level concepts of go programming, such as organizing packages, error handling, passing context, etc.
- Explain how to embrace good design principles in a project, such as clean architecture and SOLID principles.
Why this guide might be helpful to you
Go is a great language. It's simple, easy to learn, and the code is straightforward. You can write a simple application in just main.go
. But when you want to write a bigger project, there isn't any single guide or framework that can tell you exactly how to organize it. All of the projects are different. Some of them are great but, usually, programmers struggle with this freedom. There are many examples of "transplanting" code pieces from other languages/frameworks into go projects (models
package!).
There are many great articles on how to write good go code, but there aren't that many sources explaining how to put all the good stuff together. One of the sources we recommend is Ben Johnson's blog: https://www.gobeyond.dev. He also uses a repository with an example code and has multiple posts that are worth reading. But this guide is going to be a little different. The other good source with series of blog posts is https://threedots.tech - check this as well.
Guide goal
The goal of this guide is to explain all the important parts of a typical Go project. And to show in that context how to design and write readable and maintainable code, also explaining some topics specific to Go.
Later we'll also explain some problems common to Go projects (logging, caching, metrics, terminating goroutines, etc.).
This guide will hopefully be useful for experienced programmers switching from other languages.
The structure of code presented in this repository is designed to be flexible to use in various projects. The idea is that you can copy it, replace some application logic, customize adapters (explained later), and then you have a new project with a familiar structure and (hopefully) good design.
But there's one caveat: remember that this example application is overengineered. It's done on purpose, to show some concepts. In real life, in a similar application, you can merge some packages for simplicity.
TODO: Point here to the chapter about simplification.
Will this example application always work for me?
This project structure is designed for medium to large-size applications. It's not a good idea to apply all the concepts and packages for:
- Libraries
Libraries are just different; We're not going to cover library design in this guide.
- Very small applications
If you just want to write
hello word
service, or you don't care about testing that much, or you want to write simple POC for some quick demo - don't copy this project. Later, in this guide, you'll find some tips for how to collapse some packages from this example to make things simpler.
- Very big projects
The author just lacks the experience to tell how does this guide relates to complex codebases.
How to read this repository
Start with this README file. Read it up to the chapter explaining example project design. After that point you can:
- browse and run the code,
- continue reading chapter by chapter,
- or pick any chapter you want - order is not relevant
You'll find multiple README.md
files in this repository. They contain explanations for some concepts in code. We recommend you to check them as well!
Repository structure
This repository's structure closely follows github.com/golang-standards/project-layout guide for organizing project in top-level directories. I strongly suggest using it in your project. It has few advanages:
- It's well known and broadly accepted standard in Go community.
- When you join a project following this guide, you can instantly feel familiar with the repository.
- On the other hand, when someone's joining your team, there's a high chance he knows this guide and will be more confident and productive faster.
Business requirements and initial design
Let's start with explanation of example project, that will be used to talk about other important stuff here. Have a look at the business requirements and initial design for our demo app.
Guide to Go application design
Guide to Go application design
Guide to writing Go packages hierarchy
Guide to Go packaging
Packages in example app
Testing
TODO: need help here, open for any discussion
Unit tests
TODO
- When
- How
Integration tests
TODO
- How our architecture helps with tests
- When
- How
Common functionalities in backend services
Logging
TODO
- What does "log" mean?
- Common misconception: this is not the same as output in your terminal
- Unless there is a special infrastructure to create structured logs, each log is just one line in the app's output stream
- These lines of text are usually collected by some aggregator from multiple running instances
- If one instance logs 3 lines, those lines will often be spread across other lines from other instances
- Conclusion: one log should contain all the information about an event
- Don't log messages like "function started" or "function ended". The result aggregated from all running instances will be rather useless.
- Standard error logging
- https://blog.golang.org/go1.13-errors
- Other logs
- What to log? (Actually, more importantly, what not to log)
- Incoming requests
- Outgoing requests
- System state changes
- How it relates to app layers
- Put log together into stories using trace id
- Later in microservice architecture - distributed transaction ids
Caching
TODO
- App or adapters? App, of course! Explain why.
- How adding cache affects application logic (hint: it doesn't!)
Instrumentation
TODO
- How it relates to app layers (similar to logging)
Other high-level concepts of go programming
TODO
Style and linters. Optimize for reading, not for writing
TODO
Error handling
TODO
https://blog.golang.org/go1.13-errors
Nice talk: https://www.youtube.com/watch?v=IKoSsJFdRtI
Context
TODO: Primarily for signaling end of execution to goroutines
Overusing language features
TODO
- Channels: use mutex whenever it makes things simple
- Named returns: exception, not a rule
Just kidding, don't do that. Optimize for reading; care more about your coworkers than CPU cycles.
Links to other guides
TODO: need more links
TODO: How to make this section short and to the point? We don't want 100+ links here.
High abstraction level
- https://www.gobeyond.dev/ - example repository and a series of blog posts.
- https://threedots.tech/ - example repository and a series of blog posts.
Medium abstraction level
- https://dave.cheney.net/practical-go/presentations/gophercon-singapore-2019.html
Low abstraction level
- https://github.com/golang/go/wiki/CodeReviewComments