gunfish

README

Gunfish

APNs and FCM provider server on HTTP/2.

• Gunfish provides the interface as the APNs / FCM provider server.

Overview

Gunfish slides

Gunfish slides (jp)

Install

Binary

Download the latest binary from releases

Docker images

DockerHub

GitHub Packages

Homebrew

$ brew tap kayac/tap
$ brew install gunfish

Quick Started

$ gunfish -c ./config/gunfish.toml -E production

Commandline Options

option	required	description
-port	Optional	Port number of Gunfish provider server. Default is 8003.
-environment, -E	Optional	Default value is production.
-conf, -c	Optional	Please specify this option if you want to change toml config file path. (default: /etc/gunfish/config.toml.)
-log-level	Optional	Set the log level as 'warn', 'info', or 'debug'.
-log-format	Optional	Supports json or ltsv log formats.
-enable-pprof	Optional	You can set the flag of pprof debug port open.
-output-hook-stdout	Optional	Merge stdout of hook command to gunfish's stdout.
-output-hook-stderr	Optional	Merge stderr of hook command to gunfish's stderr.

API

POST /push/apns

To delivery remote notifications via APNS to user's devices.

param	description
Array	Array of JSON dictionary includes 'token' and 'payload' properties

payload param	description
token	Published token from APNS to user's remote device
payload	APNS notification payload

Post JSON example:

[
  {
    "payload": {
      "aps": {
        "alert": "test notification",
        "sound": "default"
      },
      "option1": "foo",
      "option2": "bar"
    },
    "token": "apns device token",
    "header": {
      "apns-id": "your apns id",
      "apns-topic": "your app bundle id",
      "apns-push-type": "alert"
    }
  }
]

Response example:

{"result": "ok"}

POST /push/fcm Deprecated

This API has been deleted at v0.6.0. Use /push/fcm/v1 instead.

See also https://firebase.google.com/docs/cloud-messaging/migrate-v1 .

POST /push/fcm/v1

To delivery remote notifications via FCM v1 API to user's devices.

Post body format is equal to it for FCM v1 origin server.

example:

{
  "message": {
    "notification": {
      "title": "message_title",
      "body": "message_body",
      "image": "https://example.com/notification.png"
    },
    "data": {
      "sample_key": "sample key",
      "message": "sample message"
    },
    "token": "InstanceIDTokenForDevice"
  }
}

Response example:

{"result": "ok"}

FCM v1 endpoint allows multiple payloads in a single request body. You can build request body simply concat multiple JSON payloads. Gunfish sends for each that payloads to FCM server. Limitation: Max count of payloads in a request body is 500.

GET /stats/app

{
  "pid": 57843,
  "debug_port": 0,
  "uptime": 384,
  "start_at": 1492476864,
  "su_at": 0,
  "period": 309,
  "retry_after": 10,
  "workers": 8,
  "queue_size": 0,
  "retry_queue_size": 0,
  "workers_queue_size": 0,
  "cmdq_queue_size": 0,
  "retry_count": 0,
  "req_count": 0,
  "sent_count": 0,
  "err_count": 0,
  "certificate_not_after": "2027-04-16T00:53:53Z",
  "certificate_expire_until": 315359584
}

To get the status of APNS proveder server.

stats type	description
pid	PID
debug_port	pprof port number
uptime	uptime
workers	number of workers
start_at	The time of started
queue_size	queue size of requests
retry_queue_size	queue size for resending notification
workers_queue_size	summary of worker's queue size
command_queue_size	error hook command queue size
retry_count	summary of retry count
request_count	request count to gunfish
err_count	count of recieving error response
sent_count	count of sending notification
certificate_not_after	certificates minimum expiration date for APNs
certificate_expire_until	certificates minimum expiration untile (sec)

GET /stats/profile

To get the status of go application.

See detail properties that url: (https://github.com/fukata/golang-stats-api-handler).

Configuration

The Gunfish configuration file is a TOML file that Gunfish server uses to configure itself. That configuration file should be located at /etc/gunfish.toml, and is required to start. Here is an example configuration:

[provider]
port = 8003
worker_num = 8
queue_size = 2000
max_request_size = 1000
max_connections = 2000
error_hook = "echo -e 'Hello Gunfish at error hook!'"

[apns]
key_file = "/path/to/server.key"
cert_file = "/path/to/server.crt"
kid = "kid"
team_id = "team_id"

[fcm_v1]
google_application_credentials = "/path/to/credentials.json"

[provider] section

This section is for Gunfish server configuration.

Parameter	Requirement	Description
port	optional	Listen port number.
worker_num	optional	Number of Gunfish owns http clients.
queue_size	optional	Limit number of posted JSON from the developer application.
max_request_size	optional	Limit size of Posted JSON array.
max_connections	optional	Max connections
error_hook	optional	Error hook command. This command runs when Gunfish catches an error response.

[apns] section

This section is for APNs provider configuration. If you don't need to APNs provider, you can skip this section.

Parameter	Requirement	Description
key_file	required	The key file path.
cert_file	optional	The cert file path.
kid	optional	kid for APNs provider authentication token.
team_id	optional	team id for APNs provider authentication token.

[fcm_v1] section

This section is for FCM v1 provider configuration. If you don't need to FCM v1 provider, you can skip this section.

Parameter	Requirement	Description
google_application_credentials	required	The path to the Google Cloud Platform service account key file.

Error Hook

Error hook command can get an each error response with JSON format by STDIN.

for example JSON structure: (>= v0.2.x)

// APNs
{
  "provider": "apns",
  "apns-id": "123e4567-e89b-12d3-a456-42665544000",
  "status": 400,
  "token": "9fe817acbcef8173fb134d8a80123cba243c8376af83db8caf310daab1f23003",
  "reason": "MissingTopic"
}

// FCM v1
{
  "provider": "fcmv1",
  "status": 400,
  "token": "testToken",
  "error": {
    "status": "INVALID_ARGUMENT",
    "message": "The registration token is not a valid FCM registration token"
  }
}

Graceful Restart

Gunfish supports graceful restarting based on Start Server. So, you should start on start_server command if you want graceful to restart.

### install start_server
$ go get github.com/lestrrat/go-server-starter/cmd/start_server

### Starts Gunfish with start_server
$ start_server --port 38003 --pid-file gunfish.pid -- ./gunfish -c conf/gunfish.toml

Test

$ make test

The following tools are useful to send requests to gunfish for test the following.

• gunfish-cli (send push notification to Gunfish for test)
• apnsmock (APNs mock server)

$ make tools/gunfish-cli
$ make tools/apnsmock

• send a request example with gunfish-cli

$ ./gunfish-cli -type apns -count 1 -json-file some.json -verbose
$ ./gunfish-cli -type apns -count 1 -token <device token> -apns-topic <your topic> -options key1=val1,key2=val2 -verbose

• start apnsmock server

$ ./apnsmock -cert-file ./test/server.crt -key-file ./test/server.key -verbose

Benchmark

Gunfish repository includes Lua script for the benchmark. You can use wrk command with err_and_success.lua script.

$ make tools/apnsmock
$ ./apnsmock -cert-file ./test/server.crt -key-file ./test/server.key -verbosea &
$ ./gunfish -c test/gunfish_test.toml -E test
$ wrk2 -t2 -c20 -s bench/scripts/err_and_success.lua -L -R100 http://localhost:38103

Documentation

Index

• Constants
• Variables
• func InitErrorResponseHandler(erh ResponseHandler) error
• func InitSuccessResponseHandler(sh ResponseHandler) error
• func InvokePipe(hook string, src io.Reader) ([]byte, error)
• func LogWithFields(fields map[string]interface{}) *logrus.Entry
• func StartServer(conf config.Config, env Environment)
• type Client
• type Command
• type DefaultResponseHandler
  • func (rh DefaultResponseHandler) HookCmd() string
  • func (rh DefaultResponseHandler) OnResponse(result Result)
• type Environment
  • func (i Environment) String() string
• type LtsvFormatter
  • func (f *LtsvFormatter) Format(entry *logrus.Entry) ([]byte, error)
• type Notification
• type PostedData
• type Provider
  • func (prov *Provider) PushAPNsHandler() http.HandlerFunc
  • func (prov *Provider) PushFCMHandler() http.HandlerFunc
  • func (prov *Provider) StatsHandler() http.HandlerFunc
• type Request
• type ResponseHandler
• type Result
• type SenderResponse
• type Stats
  • func NewStats(conf config.Config) Stats
  • func (st *Stats) GetStats() *Stats
• type Supervisor
  • func StartSupervisor(conf *config.Config) (Supervisor, error)
  • func (s *Supervisor) EnqueueClientRequest(reqs *[]Request) error
  • func (s *Supervisor) Shutdown()
• type Worker

Constants

const (
	// SendRetryCount is the threashold which is resend count.
	SendRetryCount = 10
	// RetryWaitTime is periodical time to retrieve notifications from retry queue to resend
	RetryWaitTime = time.Millisecond * 500
	// RetryOnceCount is the number of sending notification at once.
	RetryOnceCount = 1000
	// multiplicity of sending notifications.
	SenderNum     = 20
	RequestPerSec = 2000

	// About the average time of response from apns. That value is not accurate
	// because that is defined heuristically in Japan.
	AverageResponseTime = time.Millisecond * 150
	// Minimum RetryAfter time (seconds).
	RetryAfterSecond = time.Second * 10
	// Gunfish returns RetryAfter header based on 'Exponential Backoff'. Therefore,
	// that defines the wait time threshold so as not to wait too long.
	ResetRetryAfterSecond = time.Second * 60
	// FlowRateInterval is the designed value to enable to delivery notifications
	// for that value seconds. Gunfish is designed as to ensure to delivery
	// notifications for 10 seconds.
	FlowRateInterval = time.Second * 10
	// Default flow rate as notification requests per sec (req/sec).
	DefaultFlowRatePerSec = 2000
	// Wait millisecond interval when to shutdown.
	ShutdownWaitTime = time.Millisecond * 10
	// That is the count while request counter is 0 in the 'ShutdownWaitTime' period.
	RestartWaitCount = 50
)

Default values

const (
	DevServer  = "https://api.development.push.apple.com"
	ProdServer = "https://api.push.apple.com"
	MockServer = "https://localhost:2195"
)

Apns endpoints

const (
	ApplicationJSON              = "application/json"
	ApplicationXW3FormURLEncoded = "application/x-www-form-urlencoded"
)

Supports Content-Type

Variables

var (
	OutputHookStdout bool
	OutputHookStderr bool
)

var (
	AlertKeyToField = map[string]string{
		"title":          "Title",
		"body":           "Body",
		"title-loc-key":  "TitleLocKey",
		"title-loc-args": "TitleLocArgs",
		"action-loc-key": "ActionLocKey",
		"loc-key":        "LocKey",
		"loc-args":       "LocArgs",
		"launch-image":   "LaunchImage",
	}
)

Alert fields mapping

var RetryBackoff = true

Functions

func InitErrorResponseHandler

func InitErrorResponseHandler(erh ResponseHandler) error

InitErrorResponseHandler initialize error response handler.

func InitSuccessResponseHandler

func InitSuccessResponseHandler(sh ResponseHandler) error

InitSuccessResponseHandler initialize success response handler.

func InvokePipe

func InvokePipe(hook string, src io.Reader) ([]byte, error)

func LogWithFields

func LogWithFields(fields map[string]interface{}) *logrus.Entry

LogWithFields wraps logrus's WithFields

func StartServer

func StartServer(conf config.Config, env Environment)

StartServer starts an apns provider server on http.

Types

type Client

type Client interface {
	Send(Notification) ([]Result, error)
}

Client interface for fcm and apns client

type Command

type Command struct {
	// contains filtered or unexported fields
}

Command has execute command and input stream.

type DefaultResponseHandler

type DefaultResponseHandler struct {
	Hook string
}

DefaultResponseHandler is the default ResponseHandler if not specified.

func (DefaultResponseHandler) HookCmd

func (rh DefaultResponseHandler) HookCmd() string

HookCmd returns hook command to execute after getting response from APNS only when to get error response.

func (DefaultResponseHandler) OnResponse

func (rh DefaultResponseHandler) OnResponse(result Result)

OnResponse is performed when to receive result from APNs or FCM.

type Environment

type Environment int

Environment struct

const (
	Production Environment = iota
	Development
	Test
	Disable
)

Executed environment

func (Environment) String

func (i Environment) String() string

type LtsvFormatter

type LtsvFormatter struct {
	DisableTimestamp bool
	TimestampFormat  string
	DisableSorting   bool
}

LtsvFormatter is ltsv format for logrus

func (*LtsvFormatter) Format

func (f *LtsvFormatter) Format(entry *logrus.Entry) ([]byte, error)

Format entry

type Notification

type Notification interface{}

type PostedData

type PostedData struct {
	Header  apns.Header  `json:"header,omitempty"`
	Token   string       `json:"token"`
	Payload apns.Payload `json:"payload"`
}

PostedData is posted data to this provider server /push/apns.

type Provider

type Provider struct {
	Sup Supervisor
}

Provider defines Gunfish httpHandler and has a state of queue which is shared by the supervisor.

func (*Provider) PushAPNsHandler

func (prov *Provider) PushAPNsHandler() http.HandlerFunc

func (*Provider) PushFCMHandler

func (prov *Provider) PushFCMHandler() http.HandlerFunc

func (*Provider) StatsHandler

func (prov *Provider) StatsHandler() http.HandlerFunc

type Request

type Request struct {
	Notification Notification
	Tries        int
}

type ResponseHandler

type ResponseHandler interface {
	OnResponse(Result)
	HookCmd() string
}

ResponseHandler provides you to implement handling on success or on error response from apns. Therefore, you can specifies hook command which is set at toml file.

type Result

type Result interface {
	Err() error
	Status() int
	Provider() string
	RecipientIdentifier() string
	ExtraKeys() []string
	ExtraValue(string) string
	json.Marshaler
}

type SenderResponse

type SenderResponse struct {
	Results  []Result `json:"response"`
	RespTime float64  `json:"response_time"`
	Req      Request  `json:"request"`
	Err      error    `json:"error_msg"`
	UID      string   `json:"resp_uid"`
}

SenderResponse is responses to worker from sender.

type Stats

type Stats struct {
	Pid                    int       `json:"pid"`
	DebugPort              int       `json:"debug_port"`
	Uptime                 int64     `json:"uptime"`
	StartAt                int64     `json:"start_at"`
	ServiceUnavailableAt   int64     `json:"su_at"`
	Period                 int64     `json:"period"`
	RetryAfter             int64     `json:"retry_after"`
	Workers                int64     `json:"workers"`
	QueueSize              int64     `json:"queue_size"`
	RetryQueueSize         int64     `json:"retry_queue_size"`
	WorkersQueueSize       int64     `json:"workers_queue_size"`
	CommandQueueSize       int64     `json:"cmdq_queue_size"`
	RetryCount             int64     `json:"retry_count"`
	RequestCount           int64     `json:"req_count"`
	SentCount              int64     `json:"sent_count"`
	ErrCount               int64     `json:"err_count"`
	CertificateNotAfter    time.Time `json:"certificate_not_after"`
	CertificateExpireUntil int64     `json:"certificate_expire_until"`
}

Stats stores metrics

func NewStats

func NewStats(conf config.Config) Stats

NewStats initialize Stats

func (*Stats) GetStats

func (st *Stats) GetStats() *Stats

GetStats returns MemdStats of app

type Supervisor

type Supervisor struct {
	// contains filtered or unexported fields
}

Supervisor monitor mutiple http2 clients.

func StartSupervisor

func StartSupervisor(conf *config.Config) (Supervisor, error)

StartSupervisor starts supervisor

func (*Supervisor) EnqueueClientRequest

func (s *Supervisor) EnqueueClientRequest(reqs *[]Request) error

EnqueueClientRequest enqueues request to supervisor's queue from external application service

func (*Supervisor) Shutdown

func (s *Supervisor) Shutdown()

Shutdown supervisor

type Worker

type Worker struct {
	// contains filtered or unexported fields
}

Worker sends notification to apns.