todos

package module
v0.0.0-...-95e2c8e Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 4, 2020 License: MIT Imports: 26 Imported by: 0

README

TODOs

GoDoc Go Report Card Build Status codecov

Simple TODO API for personal task tracking (written in Go)

Building task/todo apps seems to be the "shooting hoops" of development -- good practice to test a lot of different components and technologies. In this case, I'm working with the following technologies:

  • Golang net/http API server (JSON with gin)
  • Vue.js frontend single page app
  • Docker and docker-compose for image management
  • CI/CD with Travis-CI

This development is meant to answer several questions which I will place in this README and answer throughout the course of the project.

Getting Started

This application consists of two parts, an API server and a command line application. The server can be used to create rich applications such as web or electron apps and the CLI app can be used on a day-to-day basis or for debugging purposes. You can install both of these tools using:

$ go get github.com/bbengfort/todos/...

Or by cloning the repository and building locally:

$ go install ./...

This should add the todos command to your $PATH, use todos --help to see what commands and options are available.

Server

The server is primarily configured through the environment (though some command line options can be specified using the todos serve command). There are several required configurations including:

  • $SECRET_KEY
  • $DATABASE_URL

If you're running the server in production, you'll also likely want to set $TODOS_MODE to "release" (by default it is set to "debug" but you can also specify "test"). For more settings please see the Settings object. Once the environment is configured, simply run todos serve.

Authentication

I've implemented password authentication using argon2 derived keys. Argon2 is a modern ASIC- and GPU- resistent secure key derivation function that stores passwords as a cryptographic hash in the database instead of plain text. The algorithm adds memory, time, and computational complexity to prevent rainbow and brute force attacks on a list of passwords stored this way. To compare passwords, you derive the key for the password and compare it to the derived key in the database without every saving it as plain text.

The database representation of derived keys is as follows:

$argon2id$v=19$m=65536,t=1,p=2$syEoYrFtsGBwudEnzzqvgw==$YPMFYzCdtdC1HEnQrxZlAj/Jl7HWLdqxcKqf7W4Om9w=

This standard format stores information needed to compute a per-user derived key with a $ delimiter. The first two fields are the algorithm (argon2i, argon2d, or argon2id) along with the version of the argon implementation. The third field contains parameters for the key derivation function. The fourth and fifth fields are the user-specific 16 byte salt and the 32 byte derived key, both base64 encoded.

JWT

Once the user logs in, they will be granted a JWT access token and JWT refresh token. I've done a lot of reading about API authorization schemes and honestly there is a lot of stuff out there. Frankly I'd prefer something like HAWK to JWT, but it seems like this scheme hasn't been updated since 2015. The way I intend to use JWT is similar to the database access/refresh method described here and here, though with Postgres instead of Redis as a backend (this is basically a single user app).

So here's the scheme:

  1. Login: grant an access token with a 4 hour expiration and a refresh token with a 12 hour expiration. These tokens are saved in the database with claims information. The login can optionally set a cookie.
  2. Authorization: check the Bearer header, cookie, and token request parameters for the access token, verify the key still exists in the database and that it hasn't expired. Load user information into the context of the request or return unauthorized/anonymous.
  3. Logout: fetch the access token in the same manner as Authorization, but then delete the token from the database, revoking access. The logout request can optionally take a "revoke all" parameter, which revokes all tokens for the user.
  4. Refresh: the refresh token cannot be used for authorization, but it does have a longer expiration than the access token, which means that it can be used to periodically refresh the access token with clients (particularly the CLI clients).

Other features/notes:

  • a new token is generated on every login, so the user can have different tokens on multiple devices.
  • a side go routine needs to run periodically to clean up expired tokens or an automatic mechanism needs to delete the token from the database when it's expired.

Documentation

Index

Constants

View Source
const (
	VersionMajor  = 1
	VersionMinor  = 2
	VersionPatch  = 0
	VersionSerial = 3
)

Version components for detailed version helpers

Variables

This section is empty.

Functions

func CreateDerivedKey

func CreateDerivedKey(password string) (_ string, err error)

CreateDerivedKey creates an encoded derived key with a random hash for the password.

func FindToken

func FindToken(c *gin.Context) (token string, err error)

FindToken uses the gin context to look up the access token in the Bearer header, in the cookies of the request, or as a url request parameter. It returns an error if it cannot find the token string.

func Migrate

func Migrate(db *gorm.DB) (err error)

Migrate the schema based on the models defined below.

func NotAllowed

func NotAllowed(c *gin.Context)

NotAllowed returns a JSON 405 response for the API.

func NotFound

func NotFound(c *gin.Context)

NotFound returns a JSON 404 response for the API.

func ParseDerivedKey

func ParseDerivedKey(encoded string) (dk, salt []byte, time, memory uint32, threads uint8, err error)

ParseDerivedKey returns the parts of the encoded derived key string.

func TokensCleanup

func TokensCleanup(db *gorm.DB) (rows int, err error)

TokensCleanup iterates through the database and finds any tokens that have expired, deleting them from the database. It returns the number of rows deleted and any errors that might have occurred during processing. Note that this function is run inside of a transaction in case it is long running.

func VerifyAuthToken

func VerifyAuthToken(tokenString string, access, refresh bool) (id uuid.UUID, err error)

VerifyAuthToken validates an access or refresh token string with its signature and claims fields and verifies the token is an access or refresh token if required by the input. If the token is valid, the database token id is returned without error, otherwise an error is returned to indicate that the token is no longer valid.

func VerifyDerivedKey

func VerifyDerivedKey(dk, password string) (_ bool, err error)

VerifyDerivedKey checks that the submitted password matches the derived key.

func Version

func Version() string

Version returns the human readable version of the package

func VersionURL

func VersionURL() string

VersionURL returns the URL prefix for the API at the current version

Types

type API

type API struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

API is the Todo server that wraps all context and variables for the handlers.

func New

func New(conf Settings) (api *API, err error)

New creates a Todos API server with the specified settings, fully initialized and ready to be run. Note that this function will attempt to connect to the database and migrate the latest schema to it.

func (*API) Administrative

func (s *API) Administrative() gin.HandlerFunc

Administrative is middleware that checks that the user is an admin user otherwise returns not authorized. This middleware must follow the Authenticate middleware or an internal error is returned.

func (*API) Authorize

func (s *API) Authorize() gin.HandlerFunc

Authorize is middleware that checks for an access token in the request and only allows processing to proceed if the user is valid and authorized. The middleware also loads the user information into the context so that it is available to downstream handlers.

TODO: this requires several database queries per request, can we simplify it?

func (*API) Available

func (s *API) Available() gin.HandlerFunc

Available is middleware that uses the healthy boolean to return a service unavailable http status code if the server is shutting down. It does this before all routes to ensure that complex handling doesn't bog down the server.

func (*API) CreateChecklist

func (s *API) CreateChecklist(c *gin.Context)

CreateChecklist creates a new grouping of tasks for the user.

func (*API) CreateTask

func (s *API) CreateTask(c *gin.Context)

CreateTask creates a new task assigned to the authenticated user in the database.

func (*API) DB

func (s *API) DB() *gorm.DB

DB returns the gorm database and is primarily exposed for testing purposes.

func (*API) DeleteChecklist

func (s *API) DeleteChecklist(c *gin.Context)

DeleteChecklist removes the checklist from the database and all associated tasks. TODO: ensure the list belongs to the user!

func (*API) DeleteTask

func (s *API) DeleteTask(c *gin.Context)

DeleteTask removes the task from the database. TODO: ensure that the task belongs to the user!

func (*API) DetailChecklist

func (s *API) DetailChecklist(c *gin.Context)

DetailChecklist gives as many details about the checklist as possible. TODO: ensure that the list belongs to the user!

func (*API) DetailTask

func (s *API) DetailTask(c *gin.Context)

DetailTask returns as much information about the task as possible. TODO: ensure that the task belongs to the user!

func (*API) ListChecklists

func (s *API) ListChecklists(c *gin.Context)

ListChecklists returns all checklists that belong to the authenticated user. TODO: add pagination

func (*API) ListTasks

func (s *API) ListTasks(c *gin.Context)

ListTasks returns all tasks for the authenticated user, sorted and filtered by the specified input parameters (e.g. by list or by most recent). TODO: add filtering by list TODO: add pagination

func (*API) Login

func (s *API) Login(c *gin.Context)

Login the user with the specified username and password. Login uses argon2 derived key comparisons to verify the user without storing the password in plain text. This handler binds to the loginUserForm and returns unauthorized if the password is not correct. On successful login, a JWT token is returned and a cookie set.

func (*API) Logout

func (s *API) Logout(c *gin.Context)

Logout expires the user's JWT token. Note that Logout does not have the authorization middleware so lookups up the access token in the same manner as that middleware. If no access token is provided, then a bad request is returned. Revoke all will delete all tokens for the user with the provided access token.

func (*API) Overview

func (s *API) Overview(c *gin.Context)

Overview returns statistics for the authenticated user, e.g. how many tasks and lists are currently open, completed, and archived. Although this is the root view of the API, this view requires an authenticated user in the context.

func (*API) RedirectVersion

func (s *API) RedirectVersion(c *gin.Context)

RedirectVersion sends the caller to the root of the current version

func (*API) Refresh

func (s *API) Refresh(c *gin.Context)

Refresh the access token with the refresh token if it's available and valid. The refresh token is essentially a time-limited one time key that will allow the user to reauthenticate without a username and password. If the user logs out, the refresh token will be revoked and no longer usable. Note that the server does not do any verification of the refresh token so it should be kept secret by the client in the same way a username and password should be kept secret. However, because the refresh token can be revoked and automatically expires, it is a slightly safer mechanism of reauthentication than resending a username and password combination.

func (*API) Register

func (s *API) Register(c *gin.Context)

Register a new user with the specified username and password. Register is POST only and binds the registerUserForm to get the data. Returns an error if the username or email is not unique.

func (*API) Routes

func (s *API) Routes() http.Handler

Routes returns the API router and is primarily exposed for testing purposes.

func (*API) Serve

func (s *API) Serve() (err error)

Serve the Todos API with the internal http server and specified routes.

func (*API) SetHealth

func (s *API) SetHealth(health bool)

SetHealth sets the health status on the API server, putting it into maintenance mode if health is false, and removing maintenance mode if health is true. Here primarily for testing purposes since it is unlikely an outside caller can access this.

func (*API) Shutdown

func (s *API) Shutdown() (err error)

Shutdown the API server gracefully

func (*API) Status

func (s *API) Status(c *gin.Context)

Status is an unauthenticated endpoint that returns the status of the api server and can be used for heartbeats and liveness checks.

func (*API) TokensCleanupService

func (s *API) TokensCleanupService()

TokensCleanupService is a go routine that runs the TokenCleanup function every hour, logging the results to disk. It is run when it is first called, then every hour.

func (*API) UpdateChecklist

func (s *API) UpdateChecklist(c *gin.Context)

UpdateChecklist modifies the database checklist. TODO: ensure that the list belongs to the user!

func (*API) UpdateTask

func (s *API) UpdateTask(c *gin.Context)

UpdateTask allows the user to modify a task. TODO: ensure that the task belongs to the user!

type Checklist

type Checklist struct {
	ID        uint       `gorm:"primary_key" json:"id,omitempty"`
	UserID    uint       `json:"-"`
	User      User       `json:"-"`
	Username  string     `gorm:"-" json:"user,omitempty"`
	Title     string     `gorm:"not null;size:255" json:"title,omitempty"`
	Details   string     `gorm:"not null;size:4095" json:"details,omitempty"`
	Completed uint       `gorm:"-" json:"completed,omitempty"`
	Archived  uint       `gorm:"-" json:"archived,omitempty"`
	Size      uint       `gorm:"-" json:"size"`
	Deadline  *time.Time `json:"deadline,omitempty"`
	CreatedAt time.Time  `json:"created_at"`
	UpdatedAt time.Time  `json:"updated_at"`
	Tasks     []Task     `json:"tasks,omitempty"`
}

Checklist groups related tasks so that they can be managed together. A task does not have to belong to a checklist, though it is recommended that all tasks are assigned to a list to prevent them from being stranded. Checklists are owned by individual users, which manage their lists. Similar to tasks, checklists are described by a title and optional details. However, checklists can only be "completed" if all of its tasks are either completed or archived, and this is not directly stored in the database, but is rather computed on demand. Checklists can also have a deadline, which is used for reminders and checklist ordering.

type CreateChecklistResponse

type CreateChecklistResponse struct {
	Success     bool   `json:"success"`
	Error       string `json:"error,omitempty" yaml:"error,omitempty"`
	ChecklistID uint   `json:"checklist,omitempty"`
}

CreateChecklistResponse returns the information about the created checklist. Currently the CreateChecklistRequest is simply the checklist object itself.

type CreateTaskResponse

type CreateTaskResponse struct {
	Success bool   `json:"success"`
	Error   string `json:"error,omitempty" yaml:"error,omitempty"`
	TaskID  uint   `json:"task,omitempty"`
}

CreateTaskResponse returns the information about the created task. Currently the CreateTaskRequest is simply the task object itself.

type DeleteChecklistResponse

type DeleteChecklistResponse struct {
	Success bool   `json:"success"`
	Error   string `json:"error,omitempty" yaml:"error,omitempty"`
}

DeleteChecklistResponse returns information about the delete call. Currently there is no DeleteChecklistRequest, because the request is in the URL.

type DeleteTaskResponse

type DeleteTaskResponse struct {
	Success bool   `json:"success"`
	Error   string `json:"error,omitempty" yaml:"error,omitempty"`
}

DeleteTaskResponse returns information about the delete call. Currently there is no DeleteTaskRequest, because the request is in the URL.

type DetailChecklistResponse

type DetailChecklistResponse struct {
	Success   bool      `json:"success"`
	Error     string    `json:"error,omitempty" yaml:"error,omitempty"`
	Checklist Checklist `json:"checklist"`
}

DetailChecklistResponse returns the detailed information about the checklist. Currently there is no DetailChecklistRequest, the request is in the URL.

type DetailTaskResponse

type DetailTaskResponse struct {
	Success bool   `json:"success"`
	Error   string `json:"error,omitempty" yaml:"error,omitempty"`
	Task    Task   `json:"task"`
}

DetailTaskResponse returns the detailed information about the task. Currently there is no DetailTaskRequest, the request is in the URL.

type ListChecklistsRequest

type ListChecklistsRequest struct {
	Page    int `json:"page,omitempty"`
	PerPage int `json:"per_page,omitempty"`
}

ListChecklistsRequest fetches checklists with specific filters.

type ListChecklistsResponse

type ListChecklistsResponse struct {
	Success    bool        `json:"success"`
	Error      string      `json:"error,omitempty" yaml:"error,omitempty"`
	Checklists []Checklist `json:"checklists,omitempty"`
	Page       int         `json:"page,omitempty"`
	NumPages   int         `json:"num_pages,omitempty"`
}

ListChecklistsResponse returns the checklists, and response info such as pagination.

type ListTasksRequest

type ListTasksRequest struct {
	Checklist uint `json:"checklist,omitempty"`
	Page      int  `json:"page,omitempty"`
	PerPage   int  `json:"per_page,omitempty"`
}

ListTasksRequest fetches tasks with specific filters.

type ListTasksResponse

type ListTasksResponse struct {
	Success  bool   `json:"success"`
	Error    string `json:"error,omitempty" yaml:"error,omitempty"`
	Tasks    []Task `json:"tasks,omitempty"`
	Page     int    `json:"page,omitempty"`
	NumPages int    `json:"num_pages,omitempty"`
}

ListTasksResponse returns the tasks, and response info such as pagination.

type LoginRequest

type LoginRequest struct {
	Username string `json:"username" binding:"required"`
	Password string `json:"password" binding:"required"`
	NoCookie bool   `json:"no_cookie"`
}

LoginRequest to authenticate a user with the service and return tokens

type LoginResponse

type LoginResponse struct {
	Success      bool   `json:"success"`
	Error        string `json:"error,omitempty" yaml:"error,omitempty"`
	AccessToken  string `json:"access_token,omitempty"`
	RefreshToken string `json:"refresh_token,omitempty"`
}

LoginResponse is returned on a successful login

type LogoutRequest

type LogoutRequest struct {
	RevokeAll bool `json:"revoke_all"`
}

LogoutRequest to logout the current user and optionally revoke all logins. Must be authenticated to log out a user.

type OverviewResponse

type OverviewResponse struct {
	Success    bool   `json:"success"`
	Error      string `json:"error,omitempty" yaml:"error,omitempty"`
	User       string `json:"user"`
	Tasks      int    `json:"tasks"`
	Checklists int    `json:"checklists"`
}

OverviewResponse is returned on an overview request.

type RefreshRequest

type RefreshRequest struct {
	RefreshToken string `json:"refresh_token" binding:"required"`
	NoCookie     bool   `json:"no_cookie"`
}

RefreshRequest is a reauthorization with a request token rather than username/password

type RegisterRequest

type RegisterRequest struct {
	Username string `json:"username" binding:"required"`
	Email    string `json:"email" binding:"required"`
	Password string `json:"password" binding:"required"`
	IsAdmin  bool   `json:"is_admin"`
}

RegisterRequest allows a administrative users to create new users.

type RegisterResponse

type RegisterResponse struct {
	Success  bool   `json:"success"`
	Error    string `json:"error,omitempty" yaml:"error,omitempty"`
	Username string `json:"username"`
}

RegisterResponse returns the status of a a Register request.

type Response

type Response struct {
	Success bool   `json:"success"`
	Error   string `json:"error,omitempty" yaml:"error,omitempty"`
}

Response contains standard fields that are embedded in most API responses

func ErrorResponse

func ErrorResponse(err error) Response

ErrorResponse constructs an new response from the error or returns a success: false.

type Settings

type Settings struct {
	Mode         string `default:"debug"`
	UseTLS       bool   `default:"false"`
	Bind         string `default:"127.0.0.1"`
	Port         int    `envconfig:"PORT" default:"8080" required:"true"`
	Domain       string `default:"localhost"`
	SecretKey    string `envconfig:"SECRET_KEY" required:"true"`
	DatabaseURL  string `envconfig:"DATABASE_URL" required:"true"`
	SentryDSN    string `envconfig:"SENTRY_DSN"`
	TokenCleanup bool   `default:"true" split_words:"true"`
}

Settings of the Todo API server. This is a fairly simple data structure that allows loading the configuration from the environment. See the Config() function for more. The settings also allow the server to create a mock database, which isn't something that I'm particularly fond of, but it's late and I'm not sure how to mock the internal database without a big mess of spaghetti.

func Config

func Config() (conf Settings, err error)

Config creates a new Settings object, loading it from the environment, processing default values and validating the configuration. If the Settings cannot be loaded, or validated then an error is returned.

func (Settings) Addr

func (s Settings) Addr() string

Addr returns the IPADDR:PORT to listen on

func (Settings) DBDialect

func (s Settings) DBDialect() (string, error)

DBDialect infers the dialect from the DatabaseURL

func (Settings) Endpoint

func (s Settings) Endpoint() string

Endpoint returns the URL to access the API on.

func (Settings) Environment

func (s Settings) Environment() string

Environment returns "production" if gin mode is release, otherwise develop or testing environments respectively. In the future we can configure this directly from the settings if we want "staging" or other environments.

type StatusResponse

type StatusResponse struct {
	Status    string    `json:"status"`
	Timestamp time.Time `json:"timestamp,omitempty"`
	Version   string    `json:"version,omitempty"`
	Error     string    `json:"error,omitempty" yaml:"error,omitempty"`
}

StatusResponse is returned on status requests. Note that no request is needed.

type Task

type Task struct {
	ID          uint       `gorm:"primary_key" json:"id,omitempty"`
	UserID      uint       `json:"-"`
	User        User       `json:"-"`
	Username    string     `gorm:"-" json:"user,omitempty"`
	Title       string     `gorm:"not null;size:255" json:"title,omitempty" binding:"required"`
	Details     string     `gorm:"not null;size:4095" json:"details,omitempty"`
	Completed   bool       `json:"completed"`
	Archived    bool       `json:"archived"`
	ChecklistID *uint      `json:"checklist,omitempty"`
	Checklist   *Checklist `json:"-"`
	Deadline    *time.Time `json:"deadline,omitempty"`
	CreatedAt   time.Time  `json:"created_at"`
	UpdatedAt   time.Time  `json:"updated_at"`
}

Task is the primary database structure for the todos application and represents a single unit of work that must be completed. Tasks are primarily described by their title, but can also have arbitrary text details stored alongside it. Optionally, each task can have a deadline, which is used for reminders and ordering. Each task is assigned to a user, generally the user that created the task and the task can optionally be assigned to a checklist. The primary modification of a task is to complete it (which marks it as done) or to archive it (deleting it without removal).

type Token

type Token struct {
	ID        uuid.UUID `gorm:"type:uuid;primary_key" json:"id"`
	UserID    uint      `gorm:"not null" json:"user_id"`
	User      User      `json:"-"`
	IssuedAt  time.Time `json:"issued_at"`
	ExpiresAt time.Time `json:"expires_at"`
	RefreshBy time.Time `json:"refresh_by"`
	// contains filtered or unexported fields
}

Token holds an access and refresh tokens, which are granted after authentication and used to authorize further requests using a Bearer header. The refresh token is used to update authentication without having to submit a login and password again.

func CreateAuthToken

func CreateAuthToken(db *gorm.DB, user uint) (token Token, err error)

CreateAuthToken generates acccess and refresh tokens for API authorization using a cookie or Bearer header and stores them in the database. A single user can create multiple auth tokens and each of them are assigned a unique uuid for lookup.

func (Token) AccessClaims

func (t Token) AccessClaims() jwt.Claims

AccessClaims returns the jwt.StandardClaims for the access token.

func (Token) AccessToken

func (t Token) AccessToken() (token string, err error)

AccessToken returns the cached access token or generates it from the claims.

func (Token) RefreshClaims

func (t Token) RefreshClaims() jwt.Claims

RefreshClaims returns the jwt.StandardClaims for the refresh token. Note that a refresh token cannot be used until one minute within the access token expiration.

func (Token) RefreshToken

func (t Token) RefreshToken() (token string, err error)

RefreshToken returns the cached refresh token or generates it from the claims.

type UpdateChecklistResponse

type UpdateChecklistResponse struct {
	Success bool   `json:"success"`
	Error   string `json:"error,omitempty" yaml:"error,omitempty"`
}

UpdateChecklistResponse returns information about the update call. Currently there is no UpdateChecklistRequest, because it is simply the checklist object itself.

type UpdateTaskResponse

type UpdateTaskResponse struct {
	Success bool   `json:"success"`
	Error   string `json:"error,omitempty" yaml:"error,omitempty"`
}

UpdateTaskResponse returns information about the update call. Currently there is no UpdateTaskRequest, because it is simply the task object itself.

type User

type User struct {
	ID            uint        `gorm:"primary_key" json:"id"`
	Username      string      `gorm:"unique;not null;size:255" json:"username"`
	Email         string      `gorm:"unique;not null;size:255" json:"email"`
	Password      string      `gorm:"not null;size:255" json:"-"`
	IsAdmin       bool        `json:"is_admin"`
	DefaultListID *uint       `json:"default_checklist,omitempty"`
	DefaultList   *Checklist  `json:"-"`
	LastSeen      *time.Time  `json:"last_seen"`
	CreatedAt     time.Time   `json:"created_at"`
	UpdatedAt     time.Time   `json:"updated_at"`
	Tasks         []Task      `json:"-"`
	Lists         []Checklist `json:"-"`
	Tokens        []Token     `json:"-"`
}

User is primarily used for authentication and storing json web tokens. Each user in the system manages their own tasks and checklists through the API. This is the primary partitioning mechanism between tasks.

func (User) SetPassword

func (u User) SetPassword(password string) (_ string, err error)

SetPassword uses the Argon2 derived key algorithm to store the user password along with a user-specific random salt into the database. Argon2 is a modern ASIC- and GPU- resistant secure key derivation function that prevents password cracking. The password is stored with the algorithm settings + salt + hash together in the database in a common format to ensure cross process compatibility. Each component is separated by a $ and hashes are base64 encoded.

func (User) VerifyPassword

func (u User) VerifyPassword(password string) (_ bool, err error)

VerifyPassword by comparing the original derived key with derived key from the submitted password. This function uses the parameters from the stored password to compute the dervied key and compare it.

Directories

Path Synopsis
cmd

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL