gunfish

README

Gunfish

APNs and FCM provider server on HTTP/2.

• Gunfish provides the nterface as the APNs / FCM provider server.

Overview

Gunfish slides

Gunfish slides (jp)

Quick Started

$ go get github.com/kayac/Gunfish/cmd/gunfish
$ gunfish -c ./config/gunfish.toml -E production

Commandline Options

option	required	description
-port	Optional	Port number of Gunfish provider server. Default is 8003.
-environment, -E	Optional	Default value is production.
-conf, -c	Optional	Please specify this option if you want to change toml config file path. (default: /etc/gunfish/config.toml.)
-log-level	Optional	Set the log level as 'warn', 'info', or 'debug'.
-log-format	Optional	Supports json or ltsv log formats.
-enable-pprof	Optional	You can set the flag of pprof debug port open.

API

POST /push/apns

To delivery remote notifications via APNS to user's devices.

param	description
Array	Array of JSON dictionary includes 'token' and 'payload' properties

payload param	description
token	Published token from APNS to user's remote device
payload	APNS notification payload

Post JSON example:

[
  {
    "payload": {
      "aps": {
        "alert": "test notification",
        "sound": "default"
      },
      "option1": "foo",
      "option2": "bar"
    },
    "token": "apns device token",
    "header": {
      "apns-id": "your apns id",
      "apns-topic": "your app bundle id"
    }
  }
]

Response example:

{"result": "ok"}

POST /push/fcm

To delivery remote notifications via FCM to user's devices.

Post body format is equal to it for FCM original server.

example:

{
  "registration_ids": [
    "token1",
    "token2"
  ],
  "data": {
    "id": "2",
    "message": "Test2"
  },
  "notification": {
    "title": "message_title",
    "body": "message_body"
  }
}

Response example:

{"result": "ok"}

GET /stats/app

{
  "pid": 57843,
  "debug_port": 0,
  "uptime": 384,
  "start_at": 1492476864,
  "su_at": 0,
  "period": 309,
  "retry_after": 10,
  "workers": 8,
  "queue_size": 0,
  "retry_queue_size": 0,
  "workers_queue_size": 0,
  "cmdq_queue_size": 0,
  "retry_count": 0,
  "req_count": 0,
  "sent_count": 0,
  "err_count": 0,
  "certificate_not_after": "2027-04-16T00:53:53Z",
  "certificate_expire_until": 315359584
}

To get the status of APNS proveder server.

stats type	description
pid	PID
debug_port	pprof port number
uptime	uptime
workers	number of workers
start_at	The time of started
queue_size	queue size of requests
retry_queue_size	queue size for resending notification
workers_queue_size	summary of worker's queue size
command_queue_size	error hook command queue size
retry_count	summary of retry count
request_count	request count to gunfish
err_count	count of recieving error response
sent_count	count of sending notification
certificate_not_after	certificates minimum expiration date for APNs
certificate_expire_until	certificates minimum expiration untile (sec)

GET /stats/profile

To get the status of go application.

See detail properties that url: (https://github.com/fukata/golang-stats-api-handler).

Configuration

The Gunfish configuration file is a TOML file that Gunfish server uses to configure itself. That configuration file should be located /etc/gunfish.toml, and is required to start. Here is an example configuration:

[provider]
port = 8203
worker_num = 8
queue_size = 2000
max_request_size = 1000
max_connections = 2000
error_hook = "echo -e 'Hello Gunfish at error hook!'"

[apns]
skip_insecure = true
key_file = "/path/to/server.key"
cert_file = "/path/to/server.crt"

[fcm]
api_key = "API key for FCM"

param	status	description
port	optional	Listen port number.
worker_num	optional	Number of Gunfish owns http clients.
queue_size	optional	Limit number of posted JSON from the developer application.
max_request_size	optional	Limit size of Posted JSON array.
max_connections	optional	Max connections
skip_insecure	optional	Controls whether a client verifies the server's certificate chain and host name.
key_file	required	The key file path.
cert_file	required	The cert file path.
error_hook	optional	Error hook command. This command runs when Gunfish catches an error response.

Error Hook

Error hook command can get an each error response with JSON format by STDIN.

for example JSON structure: (>= v0.2.x)

// APNs
{
  "provider": "apns",
  "apns-id": "123e4567-e89b-12d3-a456-42665544000",
  "status": 400,
  "token": "9fe817acbcef8173fb134d8a80123cba243c8376af83db8caf310daab1f23003",
  "reason": "MissingTopic"
}

// FCM
{
  "provider": "fcm",
  "status": 200,
  "registration_id": "8kMSTcfqrca:APA91bEfS-uC1WV374Mg83Lkn43..",
  // or "to": "8kMSTcfqrca:APA91bEfS-uC1WV374Mg83Lkn43..",
  "error": "InvalidRegistration"
}

(~ v0.1.x)

{
  "response": {
    "apns-id": "",
    "status": 400
  },
  "response_time": 0.633673848,
  "request": {
    "header": {},
    "token": "9fe817acbcef8173fb134d8a80123cba243c8376af83db8caf310daab1f23003",
    "payload": {
      "aps": {
        "alert": "error alert test",
        "badge": 1,
        "sound": "default"
      },
      "Optional": {
        "option1": "hoge",
        "option2": "hoge"
      }
    },
    "tries": 0
  },
  "error_msg": {
    "reason": "MissingTopic",
    "timestamp": 0
  }
}

Graceful Restart

Gunfish supports graceful restarting based on Start Server. So, you should start on start_server command if you want graceful to restart.

### install start_server
$ go get github.com/lestrrat/go-server-starter/cmd/start_server

### Starts Gunfish with start_server
$ start_server --port 38003 --pid-file gunfish.pid -- ./gunfish -c conf/gunfish.toml

Customize

How to Implement Response Handlers

If you have to handle something on error or on success, you should implement error or success handlers. For example handlers you should implement is given below:

type CustomYourErrorHandler struct {
    hookCmd string
}

func (ch CustomYourErrorHandler) OnResponse(result Result){
    // ...
}

func (ch CustomYourErrorHandler) HookCmd( ) string {
    return ch.hookCmd
}

Then you can use these handlers to set before to start gunfish server ( gunfish.StartServer( Config, Environment ) ).

InitErrorResponseHandler(CustomYourErrorHandler{hookCmd: "echo 'on error!'"})

You can implement a success custom handler in the same way but a hook command is not executed in the success handler in order not to make cpu resource too tight.

Test

To do test for Gunfish, you have to install h2o. h2o is used as APNS mock server. So, if you want to test or optimize parameters for your application, you need to prepare the envronment that h2o APNs Mock server works.

Moreover, you have to build h2o with mruby-sleep mrbgem.

$ make test

Benchmark

Gunfish repository includes Lua script for the benchmark. You can use wrk command with err_and_success.lua script.

$ h2o -c conf/h2o/h2o.conf &
$ ./gunfish -c test/gunfish_test.toml -E test
$ wrk2 -t2 -c20 -s bench/scripts/err_and_success.lua -L -R100 http://localhost:38103

Documentation

Index

• Constants
• Variables
• func InitErrorResponseHandler(erh ResponseHandler) error
• func InitSuccessResponseHandler(sh ResponseHandler) error
• func LogWithFields(fields map[string]interface{}) *logrus.Entry
• func StartServer(conf Config, env Environment)
• type Client
• type Command
• type Config
  • func DefaultLoadConfig() (Config, error)
  • func LoadConfig(fn string) (Config, error)
• type DefaultResponseHandler
  • func (rh DefaultResponseHandler) HookCmd() string
  • func (rh DefaultResponseHandler) OnResponse(result Result)
• type Environment
  • func (i Environment) String() string
• type LtsvFormatter
  • func (f *LtsvFormatter) Format(entry *logrus.Entry) ([]byte, error)
• type Notification
• type PostedData
• type Provider
• type Request
• type ResponseHandler
• type Result
• type SectionApns
• type SectionFCM
• type SectionProvider
• type SenderResponse
• type Stats
  • func NewStats(conf Config) Stats
  • func (st *Stats) GetStats() *Stats
• type Supervisor
  • func StartSupervisor(conf *Config) (Supervisor, error)
  • func (s *Supervisor) EnqueueClientRequest(reqs *[]Request) error
  • func (s *Supervisor) Shutdown()
• type Worker

Constants

const (
	MaxWorkerNum           = 119   // Maximum of worker number
	MinWorkerNum           = 1     // Minimum of worker number
	MaxQueueSize           = 40960 // Maximum queue size.
	MinQueueSize           = 128   // Minimum Queue size.
	MaxRequestSize         = 5000  // Maximum of requset count.
	MinRequestSize         = 1     // Minimum of request size.
	LimitApnsTokenByteSize = 100   // Payload byte size.
)

Limit values

const (
	// SendRetryCount is the threashold which is resend count.
	SendRetryCount = 10
	// RetryWaitTime is periodical time to retrieve notifications from retry queue to resend
	RetryWaitTime = time.Millisecond * 500
	// RetryOnceCount is the number of sending notification at once.
	RetryOnceCount = 1000
	// multiplicity of sending notifications.
	SenderNum     = 20
	RequestPerSec = 2000

	// Default array size of posted data. If not configures at file, this value is set.
	DefaultRequestQueueSize = 2000
	// Default port number of provider server
	DefaultPort = 8003
	// Default supervisor's queue size. If not configures at file, this value is set.
	DefaultQueueSize = 1000
	// About the average time of response from apns. That value is not accurate
	// because that is defined heuristically in Japan.
	AverageResponseTime = time.Millisecond * 150
	// Minimum RetryAfter time (seconds).
	RetryAfterSecond = time.Second * 10
	// Gunfish returns RetryAfter header based on 'Exponential Backoff'. Therefore,
	// that defines the wait time threshold so as not to wait too long.
	ResetRetryAfterSecond = time.Second * 60
	// FlowRateInterval is the designed value to enable to delivery notifications
	// for that value seconds. Gunfish is designed as to ensure to delivery
	// notifications for 10 seconds.
	FlowRateInterval = time.Second * 10
	// Default flow rate as notification requests per sec (req/sec).
	DefaultFlowRatePerSec = 2000
	// Wait millisecond interval when to shutdown.
	ShutdownWaitTime = time.Millisecond * 10
	// That is the count while request counter is 0 in the 'ShutdownWaitTime' period.
	RestartWaitCount = 50
)

Default values

const (
	DevServer  = "https://api.development.push.apple.com"
	ProdServer = "https://api.push.apple.com"
	MockServer = "https://localhost:2195"
)

Apns endpoints

const (
	ApplicationJSON              = "application/json"
	ApplicationXW3FormURLEncoded = "application/x-www-form-urlencoded"
)

Supports Content-Type

Variables

var (
	AlertKeyToField = map[string]string{
		"title":          "Title",
		"body":           "Body",
		"title-loc-key":  "TitleLocKey",
		"title-loc-args": "TitleLocArgs",
		"action-loc-key": "ActionLocKey",
		"loc-key":        "LocKey",
		"loc-args":       "LocArgs",
		"launch-image":   "LaunchImage",
	}
)

Alert fields mapping

Functions

func InitErrorResponseHandler

func InitErrorResponseHandler(erh ResponseHandler) error

InitErrorResponseHandler initialize error response handler.

func InitSuccessResponseHandler

func InitSuccessResponseHandler(sh ResponseHandler) error

InitSuccessResponseHandler initialize success response handler.

func LogWithFields

func LogWithFields(fields map[string]interface{}) *logrus.Entry

LogWithFields wraps logrus's WithFields

func StartServer

func StartServer(conf Config, env Environment)

StartServer starts an apns provider server on http.

Types

type Client

type Client interface {
	Send(Notification) ([]Result, error)
}

Client interface for fcm and apns client

type Command

type Command struct {
	// contains filtered or unexported fields
}

Command has execute command and input stream.

type Config

type Config struct {
	Apns     SectionApns     `toml:"apns"`
	Provider SectionProvider `toml:"provider"`
	FCM      SectionFCM      `toml:"fcm"`
}

Config is the configure of an APNS provider server

func DefaultLoadConfig

func DefaultLoadConfig() (Config, error)

DefaultLoadConfig loads default /etc/gunfish.toml

func LoadConfig

func LoadConfig(fn string) (Config, error)

LoadConfig reads gunfish.toml and loads on ApnsConfig struct

type DefaultResponseHandler

type DefaultResponseHandler struct {
	// contains filtered or unexported fields
}

DefaultResponseHandler is the default ResponseHandler if not specified.

func (DefaultResponseHandler) HookCmd

func (rh DefaultResponseHandler) HookCmd() string

HookCmd returns hook command to execute after getting response from APNS only when to get error response.

func (DefaultResponseHandler) OnResponse

func (rh DefaultResponseHandler) OnResponse(result Result)

OnResponse is performed when to receive result from APNs or FCM.

type Environment

type Environment int

Environment struct

const (
	Production Environment = iota
	Development
	Test
	Disable
)

Executed environment

func (Environment) String

func (i Environment) String() string

type LtsvFormatter

type LtsvFormatter struct {
	DisableTimestamp bool
	TimestampFormat  string
	DisableSorting   bool
}

LtsvFormatter is ltsv format for logrus

func (*LtsvFormatter) Format

func (f *LtsvFormatter) Format(entry *logrus.Entry) ([]byte, error)

Format entry

type Notification

type Notification interface{}

type PostedData

type PostedData struct {
	Header  apns.Header  `json:"header,omitempty"`
	Token   string       `json:"token"`
	Payload apns.Payload `json:"payload"`
}

PostedData is posted data to this provider server /push/apns.

type Provider

type Provider struct {
	// contains filtered or unexported fields
}

Provider defines Gunfish httpHandler and has a state of queue which is shared by the supervisor.

type Request

type Request struct {
	Notification Notification
	Tries        int
}

type ResponseHandler

type ResponseHandler interface {
	OnResponse(Result)
	HookCmd() string
}

ResponseHandler provides you to implement handling on success or on error response from apns. Therefore, you can specifies hook command which is set at toml file.

type Result

type Result interface {
	Err() error
	Status() int
	Provider() string
	RecipientIdentifier() string
	ExtraKeys() []string
	ExtraValue(string) string
	json.Marshaler
}

type SectionApns

type SectionApns struct {
	Host                string
	SkipInsecure        bool   `toml:"skip_insecure"`
	CertFile            string `toml:"cert_file"`
	KeyFile             string `toml:"key_file"`
	CertificateNotAfter time.Time
	// contains filtered or unexported fields
}

SectionApns is the configure which is loaded from gunfish.toml

type SectionFCM

type SectionFCM struct {
	APIKey string `toml:"api_key"`
	// contains filtered or unexported fields
}

SectionFCM is the configuration of fcm

type SectionProvider

type SectionProvider struct {
	WorkerNum        int `toml:"worker_num"`
	QueueSize        int `toml:"queue_size"`
	RequestQueueSize int `toml:"max_request_size"`
	Port             int `toml:"port"`
	DebugPort        int
	MaxConnections   int    `toml:"max_connections"`
	ErrorHook        string `toml:"error_hook"`
}

SectionProvider is Gunfish provider configuration

type SenderResponse

type SenderResponse struct {
	Results  []Result `json:"response"`
	RespTime float64  `json:"response_time"`
	Req      Request  `json:"request"`
	Err      error    `json:"error_msg"`
	UID      string   `json:"resp_uid"`
}

SenderResponse is responses to worker from sender.

type Stats

type Stats struct {
	Pid                    int       `json:"pid"`
	DebugPort              int       `json:"debug_port"`
	Uptime                 int64     `json:"uptime"`
	StartAt                int64     `json:"start_at"`
	ServiceUnavailableAt   int64     `json:"su_at"`
	Period                 int64     `json:"period"`
	RetryAfter             int64     `json:"retry_after"`
	Workers                int64     `json:"workers"`
	QueueSize              int64     `json:"queue_size"`
	RetryQueueSize         int64     `json:"retry_queue_size"`
	WorkersQueueSize       int64     `json:"workers_queue_size"`
	CommandQueueSize       int64     `json:"cmdq_queue_size"`
	RetryCount             int64     `json:"retry_count"`
	RequestCount           int64     `json:"req_count"`
	SentCount              int64     `json:"sent_count"`
	ErrCount               int64     `json:"err_count"`
	CertificateNotAfter    time.Time `json:"certificate_not_after"`
	CertificateExpireUntil int64     `json:"certificate_expire_until"`
}

Stats stores metrics

func NewStats

func NewStats(conf Config) Stats

NewStats initialize Stats

func (*Stats) GetStats

func (st *Stats) GetStats() *Stats

GetStats returns MemdStats of app

type Supervisor

type Supervisor struct {
	// contains filtered or unexported fields
}

Supervisor monitor mutiple http2 clients.

func StartSupervisor

func StartSupervisor(conf *Config) (Supervisor, error)

StartSupervisor starts supervisor

func (*Supervisor) EnqueueClientRequest

func (s *Supervisor) EnqueueClientRequest(reqs *[]Request) error

EnqueueClientRequest enqueues request to supervisor's queue from external application service

func (*Supervisor) Shutdown

func (s *Supervisor) Shutdown()

Shutdown supervisor

type Worker

type Worker struct {
	// contains filtered or unexported fields
}

Worker sends notification to apns.