httpsnoop

README

httpsnoop

Package httpsnoop provides an easy way to capture http related metrics (i.e. response time, bytes written, and http status code) from your application's http.Handlers.

Doing this requires non-trivial wrapping of the http.ResponseWriter interface, which is also exposed for users interested in a more low-level API.

Usage Example

// myH is your app's http handler, perhaps a http.ServeMux or similar.
var myH http.Handler
// wrappedH wraps myH in order to log every request.
wrappedH := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
	m := httpsnoop.CaptureMetrics(myH, w, r)
	log.Printf(
		"%s %s (code=%d dt=%s written=%d)",
		r.Method,
		r.URL,
		m.Code,
		m.Duration,
		m.Written,
	)
})
http.ListenAndServe(":8080", wrappedH)

Why this package exists

Instrumenting an application's http.Handler is surprisingly difficult.

However if you google for e.g. "capture ResponseWriter status code" you'll find lots of advise and code examples that suggest it to be a fairly trivial undertaking. Unfortunately everything I've seen so far has a high chance of breaking your application.

The main problem is that a http.ResponseWriter often implements additional interfaces such as http.Flusher, http.CloseNotifier, http.Hijacker, http.Pusher, and io.ReaderFrom. So the naive approach of just wrapping http.ResponseWriter in your own struct that also implements the http.ResponseWriter interface will hide the additional interfaces mentioned above. This has a high change of introducing subtle bugs into any non-trivial application.

Another approach I've seen people take is to return a struct that implements all of the interfaces above. However, that's also problematic, because it's difficult to fake some of these interfaces behaviors when the underlying http.ResponseWriter doesn't have an implementation. It's also dangerous, because an application may choose to operate differently, merely because it detects the presence of these additional interfaces.

This package solves this problem by checking which additional interfaces a http.ResponseWriter implements, returning a wrapped version implementing the exact same set of interfaces.

Additionally this package properly handles edge cases such as WriteHeader not being called, or called more than once, as well as concurrent calls to http.ResponseWriter methods, and even calls happening after the wrapped ServeHTTP has already returned.

Unfortunately this package is not perfect either. It's possible that it is still missing some interfaces provided by the go core (let me know if you find one), and it won't work for applications adding their own interfaces into the mix.

However, hopefully the explanation above has sufficiently scared you of rolling your own solution to this problem. httpsnoop may still break your application, but at least it tries to avoid it as much as possible.

Anyway, the real problem here is that smuggling additional interfaces inside http.ResponseWriter is a problematic design choice, but it probably goes as deep as the Go language specification itself. But that's okay, I still prefer Go over the alternatives ;).

Performance

BenchmarkBaseline-8      	   20000	     94912 ns/op
BenchmarkCaptureMetrics-8	   20000	     95461 ns/op

As you can see, using CaptureMetrics on a vanilla http.Handler introduces an overhead of ~500 ns per http request on my machine. However, the margin of error appears to be larger than that, therefor it should be reasonable to assume that the overhead introduced by CaptureMetrics is absolutely negligible.

License

MIT

Documentation

Overview

Package httpsnoop provides an easy way to capture http related metrics (i.e. response time, bytes written, and http status code) from your application's http.Handlers.

Doing this requires non-trivial wrapping of the http.ResponseWriter interface, which is also exposed for users interested in a more low-level API.

Index

• func Wrap(w http.ResponseWriter, hooks Hooks) http.ResponseWriter
• type CloseNotifyFunc
• type FlushFunc
• type HeaderFunc
• type HijackFunc
• type Hooks
• type Metrics
  • func CaptureMetrics(hnd http.Handler, w http.ResponseWriter, r *http.Request) Metrics
  • func CaptureMetricsFn(w http.ResponseWriter, fn func(http.ResponseWriter)) Metrics
• type PushFunc
• type ReadFromFunc
• type WriteFunc
• type WriteHeaderFunc

Functions

func Wrap

func Wrap(w http.ResponseWriter, hooks Hooks) http.ResponseWriter

Wrap returns a wrapped version of w that provides the exact same interface as w. Specifically if w implements any combination of:

- http.Flusher - http.CloseNotifier - http.Hijacker - io.ReaderFrom - http.Pusher

The wrapped version will implement the exact same combination. If no hooks are set, the wrapped version also behaves exactly as w. Hooks targeting methods not supported by w are ignored. Any other hooks will intercept the method they target and may modify the call's arguments and/or return values. The CaptureMetrics implementation serves as a working example for how the hooks can be used.

Types

type CloseNotifyFunc

type CloseNotifyFunc func() <-chan bool

CloseNotifyFunc is part of the http.CloseNotifier interface.

type FlushFunc

type FlushFunc func()

FlushFunc is part of the http.Flusher interface.

type HeaderFunc

type HeaderFunc func() http.Header

HeaderFunc is part of the http.ResponseWriter interface.

type HijackFunc

type HijackFunc func() (net.Conn, *bufio.ReadWriter, error)

HijackFunc is part of the http.Hijacker interface.

type Hooks

type Hooks struct {
	Header      func(HeaderFunc) HeaderFunc
	WriteHeader func(WriteHeaderFunc) WriteHeaderFunc
	Write       func(WriteFunc) WriteFunc
	Flush       func(FlushFunc) FlushFunc
	CloseNotify func(CloseNotifyFunc) CloseNotifyFunc
	Hijack      func(HijackFunc) HijackFunc
	ReadFrom    func(ReadFromFunc) ReadFromFunc
	Push        func(PushFunc) PushFunc
}

Hooks defines a set of method interceptors for methods included in http.ResponseWriter as well as some others. You can think of them as middleware for the function calls they target. See Wrap for more details.

type Metrics

type Metrics struct {
	// Code is the first http response code passed to the WriteHeader func of
	// the ResponseWriter. If no such call is made, a default code of 200 is
	// assumed instead.
	Code int
	// Duration is the time it took to execute the handler.
	Duration time.Duration
	// Written is the number of bytes successfully written by the Write or
	// ReadFrom function of the ResponseWriter. ResponseWriters may also write
	// data to their underlaying connection directly (e.g. headers), but those
	// are not tracked. Therefor the number of Written bytes will usually match
	// the size of the response body.
	Written int64
}

Metrics holds metrics captured from CaptureMetrics.

func CaptureMetrics

func CaptureMetrics(hnd http.Handler, w http.ResponseWriter, r *http.Request) Metrics

CaptureMetrics wraps the given hnd, executes it with the given w and r, and returns the metrics it captured from it.

func CaptureMetricsFn

func CaptureMetricsFn(w http.ResponseWriter, fn func(http.ResponseWriter)) Metrics

CaptureMetricsFn wraps w and calls fn with the wrapped w and returns the resulting metrics. This is very similar to CaptureMetrics (which is just sugar on top of this func), but is a more usable interface if your application doesn't use the Go http.Handler interface.

type PushFunc

type PushFunc func(target string, opts *http.PushOptions) error

PushFunc is part of the http.Pusher interface.

type ReadFromFunc

type ReadFromFunc func(src io.Reader) (int64, error)

ReadFromFunc is part of the io.ReaderFrom interface.

type WriteFunc

type WriteFunc func(b []byte) (int, error)

WriteFunc is part of the http.ResponseWriter interface.

type WriteHeaderFunc

type WriteHeaderFunc func(code int)

WriteHeaderFunc is part of the http.ResponseWriter interface.