README
¶
gopcapreader
Presents realtime wire-captured packets using the standard Go io.Reader interface.
See doc.go for a much more thorough description.
Prerequisites
First, of all, you need to install prerequisites:
sudo apt-get install libpcap-dev
Documentation
¶
Overview ¶
Package gopcapreader provides pcap TCP data through a simple io.Reader interface, for easy analysis of real-time TCP streams.
Library users initially set up a gopcap packet capture object to read packets off the wire, then pass that object into a gopcapreader.Multiplexer. The Multiplexer breaks up packets into unidirectional Stream objects, keyed by [srcip,srcport,dstip,dstport], then reorders/reassembles the payloads of those packets into an ordered stream of bytes. Those bytes can then be read in with the io.Reader interface to any code that wants to use it.
Example (see actual working implementation in tools/simple/main.go):
import (
"gopcapreader"
"net/http"
"pcap"
...
)
// HTTP Request handler, reads HTTP requests in real-time off the wire and
// prints them out to STDOUT.
type HttpRequests interface{}
func (h *HttpRequests) HandleStream(stream *gopcapreader.Stream) {
fmt.Println("GOT NEW HTTP STREAM", stream.Key)
eof := false
for !eof {
if req, err := http.ReadRequest(stream.BufferedReader()); req != nil {
bodyBytes := gopcapreader.DiscardBytes(req.Body)
req.Body.Close()
fmt.Println("HTTP REQUEST:", req)
fmt.Println("Read", bodyBytes, "bytes from request body")
} else if err == io.EOF || err == io.ErrUnexpectedEOF {
eof = true
} else {
fmt.Println("UNEXPECTED ERROR:", err)
}
}
}
func main() {
pcap := // set up pcap with filter 'tcp and dst port 80'
multiplexer := gopcapreader.NewMultiplexer(&HttpRequests{})
multiplexer.MultiplexPcap(h, 0) // 0 == never stop processing
}
This example uses the built-in net/http package's HTTP parsing to parse HTTP requests out of all live packet streams with destination port 80.
Index ¶
- func DiscardBytes(r io.Reader) (total int)
- type MaybeStreamHandler
- type MaybeStreamHandlerFunction
- type Multiplexer
- type OneTimeStreamHandler
- func (set *OneTimeStreamHandler) MaybeHandleStream(stream *Stream) bool
- func (set *OneTimeStreamHandler) SetUpOneTimeHalfHandlers(halfKey TcpHalfKey, src, dst StreamHandler, timeout time.Duration)
- func (set *OneTimeStreamHandler) SetUpOneTimeHandler(key TcpKey, handler StreamHandler, timeout time.Duration)
- func (set *OneTimeStreamHandler) WaitForOneTimeHandler(stream *Stream, timeout time.Duration) bool
- func (set *OneTimeStreamHandler) WaitingStreamHandler(timeout time.Duration) MaybeStreamHandler
- type Stream
- type StreamHandler
- type StreamHandlerFunction
- type StreamHandlerSlice
- type TcpHalfKey
- type TcpKey
- type TcpPacket
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DiscardBytes ¶
DiscardBytes reads in all bytes from the given io.Reader, throwing them away immediately. Returns when it it hits EOF, returning the total number of bytes discarded.
Types ¶
type MaybeStreamHandler ¶
type MaybeStreamHandler interface {
// MaybeHandleStream either handles a stream and returns true, or ignores the
// stream and returns false.
// Should return true only if this function has already completely
// handled the passed-in stream.
// Should return false only if this function has NOT destructively
// read from this stream. Calls to stream.BufferedReader().Peek()
// are acceptable, but calls to stream.Read() are destructive and
// should not be done by a function that's going to return false.
MaybeHandleStream(stream *Stream) bool
}
MaybeHandleStream provides an interface for an object that conditionally handles certain streams.
type MaybeStreamHandlerFunction ¶
MaybeStreamHandlerFunction is a convenience function type that implements the MaybeStreamHandler interface
func (MaybeStreamHandlerFunction) MaybeHandleStream ¶
func (m MaybeStreamHandlerFunction) MaybeHandleStream(stream *Stream) bool
MaybeHandleStream implements the MaybeStreamHandler interface
type Multiplexer ¶
type Multiplexer struct {
// Defaults to 1000, reset before calling MultiplexPcap to change. Users
// should reset this before calling MultiplexPcap if they want to override the
// default.
GarbageCollectMaxStreams int
// Statistics. Shouldn't be set by user, but can be read.
BytesProcessed int64
PacketsProcessed int64
PacketDecodeFailures int64
PacketsDroppedPrimaryBufferFull int64
StreamsGarbageCollected int64
// contains filtered or unexported fields
}
Multiplexer handles multiplexing incoming data into a set of streams. New streams are passed to the StreamHandler's HandleStream function.
func NewMultiplexer ¶
func NewMultiplexer(creator StreamHandler) *Multiplexer
NewMultiplexer creates a new Multiplexer object. The passed-in StreamHandler will be used to handle all new streams created by the Multiplexer.
func (*Multiplexer) Close ¶
func (m *Multiplexer) Close()
Close flushes all current streams and stops the multiplexer from accepting any new packets. Note that a close called while another goroutine is adding packets into the multiplexer is not recommended, and could result in a runtime panic. Close() will block until all streams have been flushed and all handlers have returned.
func (*Multiplexer) MultiplexPcap ¶
func (m *Multiplexer) MultiplexPcap(pcap_handle *pcap.Pcap, packets int64)
Read data in from the given pcap.Pcap handle, multiplexing all packets into streams and processing those streams with its stream handler. If 'packets' is given and > 0, will return after that many packets have been processed. Otherwise runs forever.
type OneTimeStreamHandler ¶
type OneTimeStreamHandler struct {
// contains filtered or unexported fields
}
OneTimeStreamHandler implements MaybeHandleStream and allows users to set up one-time handlers for TCP sessions using either TcpKey or TcpHalfKey keys. To use, the user calls one of the SetUp*Handler functions to set up the handler on the given key. The next time a stream matching that key is seen, it will be handled with the passed-in handler. Given the asynchronous nature of the Multiplexer, sometimes a race condition occurs between the call to SetUp*Handler and the new stream's creation. To handle this, we can place unprocessed streams in a 'waiting' state. We hold onto streams we don't know what to do with for some period of time, and if, during that time, a SetUp*Handler call occurs that matches that stream, we pull the stream out of the waiting state and call the handler on it. The WaitForOneTimeHandler call allows us to place a stream into this waiting state for a duration of time.
An example of where this could be useful: Some protocols use commands in one TCP stream to set up other TCP streams. The canonical example is FTP, where a single command channel sets up additional channels over which to send files. This API allows the user to see the command creating a channel on one stream, and define a handler to handle that stream when it's created.
func NewOneTimeStreamHandler ¶
func NewOneTimeStreamHandler() *OneTimeStreamHandler
func (*OneTimeStreamHandler) MaybeHandleStream ¶
func (set *OneTimeStreamHandler) MaybeHandleStream(stream *Stream) bool
MaybeHandleStream implements MaybeStreamHandler. It checks if it has a one-time handler for the given stream. If it does, it handles it and return true. If not, it immediately returns false.
func (*OneTimeStreamHandler) SetUpOneTimeHalfHandlers ¶
func (set *OneTimeStreamHandler) SetUpOneTimeHalfHandlers(halfKey TcpHalfKey, src, dst StreamHandler, timeout time.Duration)
SetUpOneTimeHalfHandlers sets up a set of handlers for both directions of a TcpHalfKey. src and dst can be nil, in which case only the other direction is set up for handling. After timeout has passed, both handlers are discarded.
func (*OneTimeStreamHandler) SetUpOneTimeHandler ¶
func (set *OneTimeStreamHandler) SetUpOneTimeHandler(key TcpKey, handler StreamHandler, timeout time.Duration)
SetUpOneTimeHandler sets up a one-time handler for a given TcpKey. After timeout has passed, the handler is discarded.
func (*OneTimeStreamHandler) WaitForOneTimeHandler ¶
func (set *OneTimeStreamHandler) WaitForOneTimeHandler(stream *Stream, timeout time.Duration) bool
WaitForOneTimeHandler places a stream in a waiting state. If, during the given duration, a call to SetUp*Handler matches the given stream, that stream will be pulled out of waiting and handled. After the given time, the stream is discarded.
func (*OneTimeStreamHandler) WaitingStreamHandler ¶
func (set *OneTimeStreamHandler) WaitingStreamHandler(timeout time.Duration) MaybeStreamHandler
WaitForOneTimeHandler returns a call to WaitForOneTimeHandler as a MaybeStreamHandler, so it can be passed into a StreamHandlerSet. The returned handler should should be the last handler placed into the set, since it always returns true.
type Stream ¶
type Stream struct {
Key TcpKey
// Number of packets dropped because input buffer was full
DroppedFullBuffer int
// Channel into which the reverse stream will be placed should it be found.
// NOTE: We do these channel writes in a non-blocking fashion. There is
// the potential for us to write into stream.ReverseStream two or more
// times. Consider the case where A is the reverse of B. A gets garbage
// collected, then immediately gets some new data. B will receive 2
// reverse streams, and without this non-blocking write, the second call
// could block forever.
// To protect against this, a client that carse should make sure that the
// ReverseStream channel remains empty by reading from it regularly.
// This channel has a capacity of one.
ReverseStream chan *Stream
// contains filtered or unexported fields
}
Stream contains all information about a single unidirectional TCP stream. It implements the io.Reader interface, so processors can read in the data flowing over that stream (in realtime) and process it. Users receive streams by creating a StreamHandler and passing it to a Multiplexer.
func (*Stream) BufferedReader ¶
BufferedReader eturns a bufio.Reader for this stream. Multiple calls to BufferedReader will return the same object. This enables multiple stream handlers to Peek() in order to attempt to discover the underlying protocol.
type StreamHandler ¶
type StreamHandler interface {
// HandleStream handles each new Stream a Multiplexer sees.
// Whenever the Multiplexer sees a new stream, it will call HandleStream on
// that new stream inside a GoRoutine. The HandleStream call can do all the
// (blocking) processing it wants on the given Stream object. When a call to
// HandleStream returns, all data still in the stream will be discarded until
// the stream closes.
HandleStream(*Stream)
}
StreamHandler tells the Multiplexer how to process a new stream.
type StreamHandlerFunction ¶
type StreamHandlerFunction func(stream *Stream)
StreamHandlerFunction allows a function to implement StreamHandler.
func (StreamHandlerFunction) HandleStream ¶
func (shf StreamHandlerFunction) HandleStream(stream *Stream)
HandleStream implementation of the StreamHandler interface, calls self.
type StreamHandlerSlice ¶
type StreamHandlerSlice []MaybeStreamHandler
StreamHandlerSet acts as a switch{} statement for a set of MaybeStreamHandler objects, calling them in sequence until one of them returns true.
func (StreamHandlerSlice) HandleStream ¶
func (set StreamHandlerSlice) HandleStream(stream *Stream)
HandleStream implements the StreamHandler interface.
type TcpHalfKey ¶
Half of a TcpKey. Used for specifying just the source or destination IP/port pair of a stream.
func (TcpHalfKey) NetIP ¶
func (p TcpHalfKey) NetIP() net.IP
Returns the IP of the TcpHalfKey as a net.IP object
func (TcpHalfKey) String ¶
func (p TcpHalfKey) String() string
Prints out the TcpHalfKey as a human-readable string.
type TcpKey ¶
A key for mapping a packet to a TCP stream, used by the Multiplexer.
func (TcpKey) HalfKeys ¶
func (p TcpKey) HalfKeys() (src, dst TcpHalfKey)
Returns the two half-keys of a given TcpKey
type TcpPacket ¶
type TcpPacket struct {
// contains filtered or unexported fields
}
A single TCP packet
func NewTcpPacket ¶
Decodes a pcap.Packet and wraps it in a TcpPacket object.
Source Files
Directories