profiler

README

Web-Based Memory Profiler for Go Services

Profiler helps you track your service's memory usage and custom key/value diagnostic info.

Enabling Memory Profiling

The simplest way to use the profiler is to add its endpoints to your HTTP listener. See the extra_service_info example for how to serve the profiler's endpoints on its own IP/port.

import (
	"net/http"
	"github.com/wblakecaldwell/profiler"
)
func main() {
	// add the profiler handler endpoints
	profiler.AddMemoryProfilingHandlers()

    // add realtime extra key/value diagnostic info (optional)
	profiler.RegisterExtraServiceInfoRetriever(extraServiceInfo)

	// start the profiler on service start (optional) 
	profiler.StartProfiling()

	// listen on port 6060 (pick a port)
	http.ListenAndServe(":6060", nil)
}

// extraServiceInfo returns key/value diagnostic info
func extraServiceInfo() map[string]interface{} {
    extraInfo := make(map[string]interface{})
    extraInfo["uptime"] = fetchUptime()
	extraInfo["successful connection count"] = fetchSuccessfulConnectionCount()
	extraInfo["failure connection count"] = fetchFailureConnectionCount()
    return extraInfo
}

Using Memory Profiling

Enabling Memory Profiling exposes the following endpoints:

• http://localhost:6060/profiler/stop : Stop recording memory statistics
• http://localhost:6060/profiler/start : Start recording memory statistics
• http://localhost:6060/profiler/info.html : Main page you should visit
• http://localhost:6060/profiler/info : JSON data that feeds profiler/info.html

Examples

View and/or run the three working examples in the examples folder:

1. Simple: Serving the profiler's endpoints to a service's single HTTP IP:port
2. Separate Port: Serving the profiler's endpoints on its own IP:port
3. Extra Service Info: Same as the previous example, with the profiler also reporting diagnostic key/value pairs

Working With the Template Files

Template files are bundled in the Go binary with the 'go-bindata' tool. Everything in github.com/wblakecaldwell/profiler/profiler-web is bundled up into github.com/wblakecaldwell/profiler/profiler-web.go with the command, assuming your repository is in $GOPATH/src.

Production Code Generation (Check this in):

go get github.com/jteeuwen/go-bindata/...
go install github.com/jteeuwen/go-bindata/go-bindata

go-bindata -prefix "$GOPATH/src/github.com/wblakecaldwell/profiler/profiler-web/" -pkg "profiler" -nocompress -o "$GOPATH/src/github.com/wblakecaldwell/profiler/profiler-web.go" "$GOPATH/src/github.com/wblakecaldwell/profiler/profiler-web"

If you'd like to make changes to the templates, then use 'go-bindata' in debug mode. Instead of compiling the contents of the template files into profiler-web.go, it generates code to read the content of the template files as they exist at that moment. This way, you can start your service, view the page, make changes, then refresh the browser to see them:

Development Code Generation:

go-bindata -debug -prefix "$GOPATH/src/github.com/wblakecaldwell/profiler/profiler-web/" -pkg "profiler" -nocompress -o "$GOPATH/src/github.com/wblakecaldwell/profiler/profiler-web.go" "$GOPATH/src/github.com/wblakecaldwell/profiler/profiler-web"

When you've wrapped up development, make sure to rebuild profiler-web.go to contain the contents of the file with the first non-debug command.

Documentation

Overview

Package profiler contains tools to help profile a running Go service.

Index

• func Asset(name string) ([]byte, error)
• func AssetDir(name string) ([]string, error)
• func AssetInfo(name string) (os.FileInfo, error)
• func AssetNames() []string
• func MemStatsHTMLHandler(w http.ResponseWriter, r *http.Request)
• func MustAsset(name string) []byte
• func ProfilingInfoJSONHandler(w http.ResponseWriter, r *http.Request)
• func RegisterExtraServiceInfoRetriever(infoRetriever ExtraServiceInfoRetriever)
• func RestoreAsset(dir, name string) error
• func RestoreAssets(dir, name string) error
• func StartProfiling()
• func StartProfilingHandler(w http.ResponseWriter, r *http.Request)
• func StopProfiling()
• func StopProfilingHandler(w http.ResponseWriter, r *http.Request)
• func TrackMemoryStatistics(bufferSize int, sampleIntervalMs int64, ...)
• type ExtraServiceInfoRetriever
• type HeapMemStat
• type TimedMemStats

Functions

func Asset

func Asset(name string) ([]byte, error)

Asset loads and returns the asset for the given name. It returns an error if the asset could not be found or could not be loaded.

func AssetDir

func AssetDir(name string) ([]string, error)

AssetDir returns the file names below a certain directory embedded in the file by go-bindata. For example if you run go-bindata on data/... and data contains the following hierarchy:

data/
  foo.txt
  img/
    a.png
    b.png

then AssetDir("data") would return []string{"foo.txt", "img"} AssetDir("data/img") would return []string{"a.png", "b.png"} AssetDir("foo.txt") and AssetDir("notexist") would return an error AssetDir("") will return []string{"data"}.

func AssetInfo

func AssetInfo(name string) (os.FileInfo, error)

AssetInfo loads and returns the asset info for the given name. It returns an error if the asset could not be found or could not be loaded.

func AssetNames

func AssetNames() []string

AssetNames returns the names of the assets.

func MemStatsHTMLHandler

func MemStatsHTMLHandler(w http.ResponseWriter, r *http.Request)

MemStatsHTMLHandler is a HTTP Handler to fetch memstats.html or memstats-off.html content

func MustAsset

func MustAsset(name string) []byte

MustAsset is like Asset but panics when Asset would return an error. It simplifies safe initialization of global variables.

func ProfilingInfoJSONHandler

func ProfilingInfoJSONHandler(w http.ResponseWriter, r *http.Request)

ProfilingInfoJSONHandler is a HTTP Handler to return JSON of the Heap memory statistics and any extra info the server wants to tell us about

func RegisterExtraServiceInfoRetriever

func RegisterExtraServiceInfoRetriever(infoRetriever ExtraServiceInfoRetriever)

RegisterExtraServiceInfoRetriever sets the function that will provide us with extra service info when requested

func RestoreAsset

func RestoreAsset(dir, name string) error

RestoreAsset restores an asset under the given directory

func RestoreAssets

func RestoreAssets(dir, name string) error

RestoreAssets restores an asset under the given directory recursively

func StartProfiling

func StartProfiling()

StartProfiling is a function to start profiling automatically without web button

func StartProfilingHandler

func StartProfilingHandler(w http.ResponseWriter, r *http.Request)

StartProfilingHandler is a HTTP Handler to start memory profiling, if we're not already

func StopProfiling

func StopProfiling()

StopProfiling is a function to stop profiling automatically without web button

func StopProfilingHandler

func StopProfilingHandler(w http.ResponseWriter, r *http.Request)

StopProfilingHandler is a HTTP Handler to stop memory profiling, if we're profiling

func TrackMemoryStatistics

func TrackMemoryStatistics(bufferSize int, sampleIntervalMs int64, memStatsRequestChannel <-chan chan []TimedMemStats, quitChannel chan bool)

TrackMemoryStatistics keeps track of runtime.MemStats. Sample memory statistics every [sampleIntervalMs] milliseconds, keeping the last [bufferSize] samples. Returns a request/response channel to get recent memory statistics and a quit channel to stop polling. Parameters:

bufferSize: 				number of samples to keep
sampleIntervalMs: 			number of milliseconds to wait between polling
memStatsRequestChannel:		request/reply channel to report back the most recent N samples
quitChannel: 				once closed, all polling stops

Types

type ExtraServiceInfoRetriever

type ExtraServiceInfoRetriever func() map[string]interface{}

ExtraServiceInfoRetriever functions return a map with info about the running service.

type HeapMemStat

type HeapMemStat struct {
	TimeMsAgo      int64 // ms ago (negative)
	SysKb          uint64
	HeapSysKb      uint64
	HeapAllocKb    uint64
	HeapIdleKb     uint64
	HeapReleasedKb uint64
}

HeapMemStat represents a snapshot of the heap memory at a given time.

type TimedMemStats

type TimedMemStats struct {
	// the unix time (seconds) the stats were polled
	TimeEpochMs int64

	// memory statistics
	MemStats runtime.MemStats
}

TimedMemStats represents a MemStats reading with the time it was taken