README
¶
go-utp
The Micro Transport Protocol (µTP) for Go: one interface over two implementations, with the choice made when you build.
µTP is a reliable, ordered, stream-oriented transport that runs over UDP and backs off in the presence of competing traffic, so bulk transfers don't saturate the link they share with interactive traffic. Its congestion controller, LEDBAT, measures the one way queuing delay towards the peer and grows the window towards a fixed delay target rather than until something is dropped.
The two implementations are:
- libutp, BitTorrent's C++ reference implementation reached through cgo. It's what the rest of the µTP network is running.
purego, a port of those same sources to Go. No cgo, no C++ compiler, and nothing outside the standard library.
Install
go get github.com/anacrolix/go-utp
The import path is github.com/anacrolix/go-utp; the package name is utp.
Usage
Socket implements both net.Listener and net.PacketConn, and the connections it hands out
implement net.Conn, so µTP mostly drops into code written against TCP.
Dialling:
import "github.com/anacrolix/go-utp"
s, err := utp.NewSocket("udp", ":0")
if err != nil {
return err
}
defer s.Close()
c, err := s.DialContext(ctx, "", "example.com:4242")
if err != nil {
return err
}
defer c.Close()
_, err = io.WriteString(c, "hello")
Accepting:
s, err := utp.NewSocket("udp", ":4242")
if err != nil {
return err
}
defer s.Close()
for {
c, err := s.Accept()
if err != nil {
return err
}
go handle(c)
}
Socket.Dial and Socket.DialTimeout are also available for the simpler cases, and
NewSocketFromPacketConn runs µTP over a net.PacketConn you already have.
Picking an implementation
utp.Default is libutp. Either of two build tags, or cgo being off, makes it utp.PureGo
instead, and the C++ sources are then not compiled at all:
go build -tags purego ./... # the package this tag selects shares its name
go build -tags purego_utp ./... # the same choice, for this module alone
CGO_ENABLED=0 go build ./...
purego is the tag a build sets to say it wants no cgo anywhere, and plenty of modules answer to
it. purego_utp says it about µTP and nothing else — for a build that wants the pure
implementation here without making that choice for everything it depends on, or that wants to keep
cgo out of µTP while leaving it on elsewhere.
utp.NewSocket and utp.NewSocketFromPacketConn use Default and have the same signatures as the
constructors of those names in both underlying packages, so moving a caller onto this package is an
import change.
utp.Socket is an interface, so code can be handed one without caring which is underneath, and so
is utp.Implementation — utp.PureGo and utp.LibUtp are values of it, and utp.Default is
whichever the build picked. utp.PureGo is always available; utp.LibUtp only exists where libutp
is being compiled. That's how the interop tests run each implementation against the other, and each
against itself.
An Implementation has one method: it makes a Socket over a net.PacketConn. Opening a port
isn't its job, so utp.Listen does that for it, and code holding a PacketConn already can hand it
straight over:
s, err := utp.Listen(utp.PureGo, "udp", ":4242")
s, err := utp.PureGo.NewSocket(pc)
Sharing a port with non-µTP traffic
Packets that aren't µTP are not dropped: Socket implements net.PacketConn, and its ReadFrom
and WriteTo carry exactly those packets. That's how a single UDP port can serve µTP connections
and something else — a DHT, say — at the same time. Packets that aren't read promptly are dropped.
Options
The tunables both implementations share are options on the constructors: utp.WithLogger,
utp.WithBufferSizes and utp.WithTargetDelay, plus Socket.SetLogging for the protocol's own
log categories and Socket.SetFirewallCallback for rejecting incoming connections before they're
acknowledged. Anything an implementation offers beyond that stays in its own package, and both
sockets can be asserted for.
Two differences to know about. utp.Listen listens with net.ListenPacket, so it takes UDP
networks only — the cgo package's own NewSocket also understands the in-process test network, and
through this package you'd reach that by opening it yourself and calling
Implementation.NewSocket. And deadlines on the Socket itself, which only affect ReadFrom and
WriteTo, aren't supported by libutp: it returns an error wrapping errors.ErrUnsupported, where
the pure implementation honours them. Deadlines on connections work in both.
The pure Go implementation
purego can be used on its own, without this package, if the choice at build time isn't
wanted:
import "github.com/anacrolix/go-utp/purego"
s, err := purego.NewSocket("udp", ":4242")
The state machine, the LEDBAT congestion controller, selective acknowledgements and fast resend,
the retransmission timers and the MTU search all follow the reference implementation, down to the
constants they're tuned with. It passes golang.org/x/net/nettest's TestConn conformance suite,
and the interop package tests it against libutp itself: in both directions, in both
roles, and over a link that drops, delays and duplicates packets.
Two of libutp's inputs aren't available portably from Go. ICMP fragmentation-needed reports aren't fed back in, and the don't-fragment bit isn't set on MTU probes, so path MTU is inferred from timeouts and duplicate acks rather than reported outright. Neither affects correctness; the search just tends to settle high. The cgo package doesn't act on ICMP either.
Development
The justfile mirrors the CI jobs, so what you run locally is what CI runs:
just test # go test -race -count 2 ./...
just bench # build/smoke the benchmarks
just test-purego # the pure Go implementation selected via the build tag
just test-purego-utp # the same, via the µTP-only tag
just test-nocgo # everything with cgo off entirely
Release history
See CHANGELOG.md.
License
MIT, inherited from libutp. See LICENSE.
Documentation
¶
Overview ¶
Package utp provides one interface over both µTP implementations for Go, and picks one of them at build time.
By default it uses libutp, the C++ reference implementation, through cgo and github.com/anacrolix/go-libutp. Either of two build tags, or cgo being disabled, uses the pure Go implementation in github.com/anacrolix/go-utp/purego instead:
go build -tags purego ./... // the package this tag selects shares its name go build -tags purego_utp ./... // the same choice, for this module alone CGO_ENABLED=0 go build ./...
purego is the tag a build sets to say it wants no cgo anywhere, and plenty of modules answer to it. purego_utp says it about µTP and nothing else, for a build that wants the pure implementation here without making that choice for everything it depends on, or that wants to keep cgo out of µTP while leaving it on elsewhere.
Default is whichever one the build selected. NewSocket and NewSocketFromPacketConn use it and have the same signatures as the constructors of those names in both underlying packages, so switching a caller over to this package is an import change. Code that wants a particular implementation rather than the default can name PureGo or LibUtp directly; LibUtp only exists where libutp is being compiled, which is to say under neither tag and not with cgo disabled.
Implementations are values, so code that has to work with either can take an Implementation and be handed one, rather than choosing at its own build time. An Implementation makes a Socket over a PacketConn; Listen opens one for it.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var Default = LibUtp
Default is the implementation this binary was built with: libutp here, because the build set neither the purego nor the purego_utp tag, and didn't disable cgo. A program that wants to override the build's choice can assign to it before making any sockets.
Functions ¶
This section is empty.
Types ¶
type FirewallCallback ¶
A FirewallCallback reports whether an incoming connection should be ignored. Rejecting one this way is better than accepting and immediately closing it, because the peer sees no response at all rather than an acknowledgement followed by a reset.
It is called while the Socket holds its own lock, so it must not block or call back into the Socket.
It's an alias rather than a defined type, so that an implementation's own callback type is the same type and no conversion is needed either way.
type Implementation ¶
type Implementation interface {
// NewSocket runs µTP over a PacketConn. The Socket takes ownership of it: closing the Socket
// closes the PacketConn.
NewSocket(pc net.PacketConn, opts ...Option) (Socket, error)
}
An Implementation makes Sockets over a PacketConn. The two in this module are PureGo and LibUtp, and Default is whichever the build selected.
Owning a port is not part of it: use Listen to have one opened for an implementation. Both values here also print as their name, so fmt.Sprint(utp.Default) is "libutp" or "purego".
var LibUtp Implementation = &implementation{"libutp", newLibutpSocket}
LibUtp is the C++ reference implementation, github.com/anacrolix/go-libutp, reached through cgo. It only exists where libutp is being compiled: not under the purego or purego_utp tags, and not with cgo disabled.
var PureGo Implementation = &implementation{"purego", newPuregoSocket}
PureGo is the pure Go implementation, github.com/anacrolix/go-utp/purego. It is available whatever the build selected as Default.
type Option ¶
type Option func(*options)
An Option configures a Socket as it's created.
func WithBufferSizes ¶
WithBufferSizes sets the send and receive buffers new connections are given, in bytes. Zero leaves either at the default, which is a megabyte.
func WithLogger ¶
WithLogger gives a Socket its own logger, instead of the implementation's package level one: github.com/anacrolix/go-libutp.Logger or github.com/anacrolix/go-utp/purego.Logger, either of which is slog's default until it's set. This is the logger the categories in Socket.SetLogging write to, along with anything either implementation has to report about the socket itself.
func WithTargetDelay ¶
WithTargetDelay sets the one way queuing delay the congestion controller aims for. Lower means yielding sooner to whatever else is sharing the link. Zero leaves it at the default of 100ms, which is what the rest of the µTP network uses.
type Socket ¶
type Socket interface {
net.Listener
net.PacketConn
// Dial connects to addr on the Socket's own network.
Dial(addr string) (net.Conn, error)
// DialTimeout connects to addr, giving up after timeout. A zero timeout means no limit.
DialTimeout(addr string, timeout time.Duration) (net.Conn, error)
// DialContext connects to addr. An empty network means the Socket's own.
DialContext(ctx context.Context, network, addr string) (net.Conn, error)
// SetFirewallCallback sets the function consulted before each incoming connection is
// accepted. Pass nil to accept everything, which is the default.
SetFirewallCallback(FirewallCallback)
// ReadBufferLen and WriteBufferLen report the buffer sizes new connections are given. The
// write buffer also caps the congestion window; the read buffer is what the receive window
// advertised to peers is computed from. Connections already open keep the sizes they were
// created with.
ReadBufferLen() int
WriteBufferLen() int
SetReadBufferLen(int)
SetWriteBufferLen(int)
// SetLogging turns the protocol's own log categories on and off. They go to the Socket's
// logger at debug level, so it has to be passing debug records for them to appear. All are
// off by default, and they're noisy: normal covers connection lifecycle and packet loss, mtu
// the path MTU search, and debug every packet. libutp compiles its debug logging out, so
// debug does nothing there.
SetLogging(normal, mtu, debug bool)
}
Socket is a µTP endpoint: it owns a packet conn and multiplexes connections over it.
It is a net.Listener, so Accept hands out incoming connections, and a net.PacketConn, whose ReadFrom and WriteTo carry the datagrams on the port that aren't µTP. That's what lets one port serve µTP and another protocol, a DHT say, at the same time. Datagrams that aren't read promptly are dropped.
The connections Accept and Dial return are ordinary [net.Conn]s, deadlines included. Deadlines on the Socket itself, which only affect ReadFrom and WriteTo, are not supported by libutp: it returns an error wrapping errors.ErrUnsupported from the three deadline methods.
func Listen ¶
func Listen(impl Implementation, network, addr string, opts ...Option) (Socket, error)
Listen opens a port for impl and returns a Socket over it. Only UDP networks are supported; for anything else, listen yourself and pass the PacketConn to Implementation.NewSocket.
The Socket owns the port: closing the Socket closes it.
func NewSocket ¶
NewSocket listens on the given network and address using Default. It is Listen with the implementation the build selected, and has the same signature as the constructor of the same name in both underlying packages.
func NewSocketFromPacketConn ¶
func NewSocketFromPacketConn(pc net.PacketConn, opts ...Option) (Socket, error)
NewSocketFromPacketConn runs µTP over a PacketConn you already have, using Default. The Socket takes ownership of it: closing the Socket closes the PacketConn.
Source Files
Directories