livepprof

README

Live pprof

Live pprof is a Golang library to generate and use live profiles.

Status

Under heavy development, unstable, work in progress, use at your own risk.

Documentation

There are several ways to use the library, one is to directly call the low-level API:

import "github.com/ufoot/livepprof/collector/cpu"

// ...

collector := cpu.New("mypackage", 10*time.Second)
data, err := collector.Collect(nil)
// Now data contains a profile, do whatever you want with it,
// in itself it's not very different from a raw Go profile,
// the main difference is that names should be *resolved*,
// and also data is aggregated under "mypackage" entries.

This "mypackage" string is here to help you have data aggregated on functions which belong to your code. This assumes your files are in something that looks like "github.com/me/mypackage/something". Most of the time you're interested by profiling your code, if a dependency is slow you'll still know it but aggregated from a point in code that belongs to you.

Another way is to use a higher level profile interface which heartbeats with profiles on a regular basis. It can then be graphed, logged, I personally recommend using Datadog to do this, but you could technically use anything.

import (
    "log"
    "github.com/ufoot/livepprof"
)

// ...

p, err := livepprof.New(livepprof.WithFilter("mypackage"))
if err!=nil {
    // Handle error.
}
go func() { // This goroutine reports data in the background.
    for cpu := range p.CPU() {
        // This is going to be called every minute by default.
        for i, entry := range cpu.Entries {
            // Do whatever you want with entry.
            log.Printf("livepprof entry %d, function=%s, file=%s, stack=%s, value=%0.3f",
                i,
                entry.Key.Function,
                entry.Key.File,
                entry.Key.Stack,
                entry.Value, // This is the actual CPU usage of that entry.
            )
        }
    }
}()

// Your code that does things, here.

p.Stop() // Stop the goroutine reporting data.

Godoc links:

• livepprof
• livepprof/objfile
• livepprof/collector
• livepprof/collector/cpu
• livepprof/collector/heap

Bugs

Again, super experimental, among other things:

• it requires to have GNU binutils installed, which is akward as Go as builtin support to analyze binaries.
• the heap profiles look wrong, generally speaking if you want outliers, you're going to get the right ones, but data is skewed, need to figure out where the problem is exactly

Authors

• Christian Mauduit ufoot@ufoot.org : main developper, project maintainer.

License

Copyright (C) 2018 Christian Mauduit ufoot@ufoot.org

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

• Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
• Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
• Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Live pprof homepage: https://github.com/ufoot/livepprof

Contact author: ufoot@ufoot.org

Documentation

Index

• type Data
• type Entry
• type LP
  • func New(options ...Option) (*LP, error)
  • func (lp *LP) CPU() <-chan Data
  • func (lp *LP) Close()
  • func (lp *LP) Heap() <-chan Data
  • func (lp *LP) Start()
  • func (lp *LP) Stop()
• type Option
  • func WithDelay(delay time.Duration) Option
  • func WithEnabled(enabled bool) Option
  • func WithEnabledFunc(enabledFunc func() bool) Option
  • func WithErrorHandler(errHandler func(err error)) Option
  • func WithFilter(filter string) Option
  • func WithJitter(jitter float64) Option
  • func WithLimit(limit int) Option
• type Profiler

Types

type Data

type Data struct {
	// Timestamp when the data was generated.
	Timestamp time.Time
	// Entries, sorted by order of importance, greater numbers at the beginning.
	Entries []Entry
}

Data passed in channels.

type Entry

type Entry struct {
	// Key is the aggregation key for data, basically a location in the code.
	Key objfile.Location
	// Value is the measured value (bytes, CPU cycles...)
	Value float64
}

Entry is used to store data results.

type LP

type LP struct {
	// contains filtered or unexported fields
}

LP is an implementation of a live profiler.

func New

func New(options ...Option) (*LP, error)

New live profiler. The contains parameter is used to choose the leaf on which to aggregate data. Just choose something that is in your source files path, typically a top-level package name, namespace, whatever identifies your code.

func (*LP) CPU

func (lp *LP) CPU() <-chan Data

CPU channel on which cpu data is sent.

func (*LP) Close

func (lp *LP) Close()

Close the profiler. It can't be started again.

func (*LP) Heap

func (lp *LP) Heap() <-chan Data

Heap channel on which heap data is sent.

func (*LP) Start

func (lp *LP) Start()

Start the profiler.

func (*LP) Stop

func (lp *LP) Stop()

Stop the profiler.

type Option

type Option func(o *opts) error

Option passed when creating the live profiler.

func WithDelay

func WithDelay(delay time.Duration) Option

WithDelay allows a custom delay to be used. Default is one minute.

func WithEnabled

func WithEnabled(enabled bool) Option

WithEnabled allows you to enable/disable the profiler. If enabled is false, no profiling fill be done, even if the profiler is started.

func WithEnabledFunc

func WithEnabledFunc(enabledFunc func() bool) Option

WithEnabledFunc allows you to enable/disable the profiler with a callback. This is useful if you want to enable/disable it on-the-fly without explicitly stopping or starting it. One use-case is if you have a dynamic configuration. This will actually poll the dynamic configuration, and enable/disable it. Typically interesting if you are concerned with the CPU the profiling is consuming, and/or if you want to get rid of any side effect.

func WithErrorHandler

func WithErrorHandler(errHandler func(err error)) Option

WithErrorHandler allows custom handling of errors. This is useful as live profiler does thing in the background, the instanciation and start can not return all possible errors, so they need to be handled later, in a separate goroutine, as they happen.

func WithFilter

func WithFilter(filter string) Option

WithFilter helps you spot where time is spent within your code by reporting functions which are in packages containing a given filter in their name. Typically if you are coding something in github.com/thisisme/supergolib/package1 you might want to give this "supergolib/package1" as you want functions in that part of the code to be reported. Not doing this, you might not have insightful reports as you may get a lot of entries about small "leaf" part of the code. If you do this the live profiler will aggregate all data until it finds a common parent in "supergolib/package1".

func WithJitter

func WithJitter(jitter float64) Option

WithJitter allows a custom jitter to be used on top of delay. Default is 0.1. Which means value can be 5% lower or 5% higher than specified. Jitter needs to be within 0 (no jitter at all) and 1 (from 50% to 150% of the value).

func WithLimit

func WithLimit(limit int) Option

WithLimit allows a custom limit of displayed funcs to be used. Default is 20.

type Profiler

type Profiler interface {
	// CPU channel on which cpu data is sent.
	CPU() <-chan Data
	// Heap channel on which heap data is sent.
	Heap() <-chan Data
	// Close the profiler.
	Close()
}

Profiler can profile code dynamically.