tunnel

README

Go HTTP tunnel

Go HTTP tunnel is a reverse tunnel based on HTTP/2. It enables you to share your localhost when you don't have a public IP.

Features:

• HTTP proxy with basic authentication
• TCP proxy
• SNI vhost proxy
• Client auto reconnect
• Client management and eviction
• Easy to use CLI

Common use cases:

• Hosting a game server from home
• Developing webhook integrations
• Managing IoT devices

Project Status

IF YOU WOULD LIKE TO SEE THIS PROJECT MODERNIZED PLEASE UPVOTE THE ISSUE.

Installation

Build the latest version.

$ go get -u github.com/mmatczuk/go-http-tunnel/cmd/...

Alternatively download the latest release.

Running

There are two executables:

• tunneld - the tunnel server, to be run on publicly available host like AWS or GCE
• tunnel - the tunnel client, to be run on your local machine or in your private network

To get help on the command parameters run tunneld -h or tunnel -h.

Tunnel requires TLS certificates for both client and server.

$ openssl req -x509 -nodes -newkey rsa:2048 -sha256 -keyout client.key -out client.crt
$ openssl req -x509 -nodes -newkey rsa:2048 -sha256 -keyout server.key -out server.crt

Run client:

• Install tunnel binary
• Make .tunnel directory in your project directory
• Copy client.key, client.crt to .tunnel
• Create configuration file tunnel.yml in .tunnel
• Start all tunnels

$ tunnel -config ./tunnel/tunnel.yml start-all

Run server:

• Install tunneld binary
• Make .tunneld directory
• Copy server.key, server.crt to .tunneld
• Start tunnel server

$ tunneld -tlsCrt .tunneld/server.crt -tlsKey .tunneld/server.key

This will run HTTP server on port 80 and HTTPS (HTTP/2) server on port 443. If you want to use HTTPS it's recommended to get a properly signed certificate to avoid security warnings.

Run Server as a Service on Ubuntu using Systemd:

• After completing the steps above successfully, create a new file for your service (you can name it whatever you want, just replace the name below with your chosen name).

$ vim tunneld.service

• Add the following configuration to the file

[Unit]
Description=Go-Http-Tunnel Service
After=network.target
After=network-online.target

[Service]
ExecStart=/path/to/your/tunneld -tlsCrt /path/to/your/folder/.tunneld/server.crt -tlsKey /path/to/your/folder/.tunneld/server.key
TimeoutSec=30
Restart=on-failure
RestartSec=30

[Install]
WantedBy=multi-user.target

• Save and exit this file.
• Move this new file to /etc/systemd/system/

$ sudo mv tunneld.service /etc/systemd/system/

• Change the file permission to allow it to run.

$ sudo chmod u+x /etc/systemd/system/tunneld.service

• Start the new service and make sure you don't get any errors, and that your client is able to connect.

$ sudo systemctl start tunneld.service

• You can stop the service with:

$ sudo systemctl stop tunneld.service

• Finally, if you want the service to start automatically when the server is rebooted, you need to enable it.

$ sudo systemctl enable tunneld.service

There are many more options for systemd services, and this is by not means an exhaustive configuration file.

Configuration

The tunnel client tunnel requires configuration file, by default it will try reading tunnel.yml in your current working directory. If you want to specify other file use -config flag.

Sample configuration that exposes:

• localhost:8080 as webui.my-tunnel-host.com
• host in private network for ssh connections

looks like this

    server_addr: SERVER_IP:5223
    tunnels:
      webui:
        proto: http
        addr: localhost:8080
        auth: user:password
        host: webui.my-tunnel-host.com
      ssh:
        proto: tcp
        addr: 192.168.0.5:22
        remote_addr: 0.0.0.0:22
      tls:
  	    proto: sni
  	    addr: localhost:443
  	    host: tls.my-tunnel-host.com

Configuration options:

• server_addr: server TCP address, i.e. 54.12.12.45:5223
• tls_crt: path to client TLS certificate, default: client.crt in the config file directory
• tls_key: path to client TLS certificate key, default: client.key in the config file directory
• root_ca: path to trusted root certificate authority pool file, if empty any server certificate is accepted
• tunnels / [name]
  • proto: tunnel protocol, http, tcp or sni
  • addr: forward traffic to this local port number or network address, for proto=http this can be full URL i.e. https://machine/sub/path/?plus=params, supports URL schemes http and https
  • auth: (proto=http) (optional) basic authentication credentials to enforce on tunneled requests, format user:password
  • host: (proto=http, proto=sni) hostname to request (requires reserved name and DNS CNAME)
  • remote_addr: (proto=tcp) bind the remote TCP address
• backoff
  • interval: how long client would wait before redialing the server if connection was lost, exponential backoff initial interval, default: 500ms
  • multiplier: interval multiplier if reconnect failed, default: 1.5
  • max_interval: maximal time client would wait before redialing the server, default: 1m
  • max_time: maximal time client would try to reconnect to the server if connection was lost, set 0 to never stop trying, default: 15m

How it works

A client opens TLS connection to a server. The server accepts connections from known clients only. The client is recognized by its TLS certificate ID. The server is publicly available and proxies incoming connections to the client. Then the connection is further proxied in the client's network.

The tunnel is based HTTP/2 for speed and security. There is a single TCP connection between client and server and all the proxied connections are multiplexed using HTTP/2.

Donation

If this project help you reduce time to develop, you can give me a cup of coffee.

A GitHub star is always appreciated!

License

Copyright (C) 2017 Michał Matczuk

This project is distributed under the AGPL-3 license. See the LICENSE file for details. If you need an enterprise license contact me directly.

Documentation

Overview

Package tunnel is fast and secure client/server package that enables proxying public connections to your local machine over a tunnel connection from the local machine to the public server.

Index

• Variables
• type Auth
  • func NewAuth(auth string) *Auth
• type Backoff
• type Client
  • func NewClient(config *ClientConfig) (*Client, error)
  • func (c *Client) Start() error
  • func (c *Client) Stop()
• type ClientConfig
• type HTTPProxy
  • func NewHTTPProxy(localURL *url.URL, logger log.Logger) *HTTPProxy
  • func NewMultiHTTPProxy(localURLMap map[string]*url.URL, logger log.Logger) *HTTPProxy
  • func (p *HTTPProxy) Director(req *http.Request)
  • func (p *HTTPProxy) Proxy(w io.Writer, r io.ReadCloser, msg *proto.ControlMessage)
• type HostAuth
• type ProxyFunc
  • func Proxy(p ProxyFuncs) ProxyFunc
• type ProxyFuncs
• type RegistryItem
• type Server
  • func NewServer(config *ServerConfig) (*Server, error)
  • func (s *Server) Addr() string
  • func (r Server) IsSubscribed(identifier id.ID) bool
  • func (s *Server) Ping(identifier id.ID) (time.Duration, error)
  • func (s *Server) RoundTrip(r *http.Request) (*http.Response, error)
  • func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request)
  • func (s *Server) Start()
  • func (s *Server) Stop()
  • func (r Server) Subscribe(identifier id.ID)
  • func (r Server) Subscriber(hostPort string) (id.ID, *Auth, bool)
  • func (s *Server) Unsubscribe(identifier id.ID) *RegistryItem
  • func (s *Server) Upgrade(identifier id.ID, conn net.Conn, requestBytes []byte) error
• type ServerConfig
• type SubscriptionListener
• type TCPProxy
  • func NewMultiTCPProxy(localAddrMap map[string]string, logger log.Logger) *TCPProxy
  • func NewTCPProxy(localAddr string, logger log.Logger) *TCPProxy
  • func (p *TCPProxy) Proxy(w io.Writer, r io.ReadCloser, msg *proto.ControlMessage)

Variables

var (
	// DefaultKeepAliveIdleTime specifies how long connection can be idle
	// before sending keepalive message.
	DefaultKeepAliveIdleTime = 15 * time.Minute
	// DefaultKeepAliveCount specifies maximal number of keepalive messages
	// sent before marking connection as dead.
	DefaultKeepAliveCount = 8
	// DefaultKeepAliveInterval specifies how often retry sending keepalive
	// messages when no response is received.
	DefaultKeepAliveInterval = 5 * time.Second
)

var (
	// DefaultTimeout specifies a general purpose timeout.
	DefaultTimeout = 10 * time.Second
	// DefaultPingTimeout specifies a ping timeout.
	DefaultPingTimeout = 500 * time.Millisecond
)

Types

type Auth

type Auth struct {
	User     string
	Password string
}

Auth holds user and password.

func NewAuth

func NewAuth(auth string) *Auth

NewAuth creates new auth from string representation "user:password".

type Backoff

type Backoff interface {
	// Next returns the duration to sleep before retrying to reconnect.
	// If the returned value is negative, the retry is aborted.
	NextBackOff() time.Duration

	// Reset is used to signal a reconnection was successful and next
	// call to Next should return desired time duration for 1st reconnection
	// attempt.
	Reset()
}

Backoff defines behavior of staggering reconnection retries.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is responsible for creating connection to the server, handling control messages. It uses ProxyFunc for transferring data between server and local services.

func NewClient

func NewClient(config *ClientConfig) (*Client, error)

NewClient creates a new unconnected Client based on configuration. Caller must invoke Start() on returned instance in order to connect server.

func (*Client) Start

func (c *Client) Start() error

Start connects client to the server, it returns error if there is a connection error, or server cannot open requested tunnels. On connection error a backoff policy is used to reestablish the connection. When connected HTTP/2 server is started to handle ControlMessages.

func (*Client) Stop

func (c *Client) Stop()

Stop disconnects client from server.

type ClientConfig

type ClientConfig struct {
	// ServerAddr specifies TCP address of the tunnel server.
	ServerAddr string
	// TLSClientConfig specifies the tls configuration to use with
	// tls.Client.
	TLSClientConfig *tls.Config
	// DialTLS specifies an optional dial function that creates a tls
	// connection to the server. If DialTLS is nil, tls.Dial is used.
	DialTLS func(network, addr string, config *tls.Config) (net.Conn, error)
	// Backoff specifies backoff policy on server connection retry. If nil
	// when dial fails it will not be retried.
	Backoff Backoff
	// Tunnels specifies the tunnels client requests to be opened on server.
	Tunnels map[string]*proto.Tunnel
	// Proxy is ProxyFunc responsible for transferring data between server
	// and local services.
	Proxy ProxyFunc
	// Logger is optional logger. If nil logging is disabled.
	Logger log.Logger
}

ClientConfig is configuration of the Client.

type HTTPProxy

type HTTPProxy struct {
	httputil.ReverseProxy
	// contains filtered or unexported fields
}

HTTPProxy forwards HTTP traffic.

func NewHTTPProxy

func NewHTTPProxy(localURL *url.URL, logger log.Logger) *HTTPProxy

NewHTTPProxy creates a new direct HTTPProxy, everything will be proxied to localURL.

func NewMultiHTTPProxy

func NewMultiHTTPProxy(localURLMap map[string]*url.URL, logger log.Logger) *HTTPProxy

NewMultiHTTPProxy creates a new dispatching HTTPProxy, requests may go to different backends based on localURLMap.

func (*HTTPProxy) Director

func (p *HTTPProxy) Director(req *http.Request)

Director is ReverseProxy Director it changes request URL so that the request is correctly routed based on localURL and localURLMap. If no URL can be found the request is canceled.

func (*HTTPProxy) Proxy

func (p *HTTPProxy) Proxy(w io.Writer, r io.ReadCloser, msg *proto.ControlMessage)

Proxy is a ProxyFunc.

type HostAuth

type HostAuth struct {
	Host string
	Auth *Auth
}

HostAuth holds host and authentication info.

type ProxyFunc

type ProxyFunc func(w io.Writer, r io.ReadCloser, msg *proto.ControlMessage)

ProxyFunc is responsible for forwarding a remote connection to local server and writing the response.

func Proxy

func Proxy(p ProxyFuncs) ProxyFunc

Proxy returns a ProxyFunc that uses custom function if provided.

type ProxyFuncs

type ProxyFuncs struct {
	// HTTP is custom implementation of HTTP proxing.
	HTTP ProxyFunc
	// TCP is custom implementation of TCP proxing.
	TCP ProxyFunc
}

ProxyFuncs is a collection of ProxyFunc.

type RegistryItem

type RegistryItem struct {
	Hosts     []*HostAuth
	Listeners []net.Listener
}

RegistryItem holds information about hosts and listeners associated with a client.

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server is responsible for proxying public connections to the client over a tunnel connection.

func NewServer

func NewServer(config *ServerConfig) (*Server, error)

NewServer creates a new Server.

func (*Server) Addr

func (s *Server) Addr() string

Addr returns network address clients connect to.

func (Server) IsSubscribed

func (r Server) IsSubscribed(identifier id.ID) bool

IsSubscribed returns true if client is subscribed.

func (*Server) Ping

func (s *Server) Ping(identifier id.ID) (time.Duration, error)

Ping measures the RTT response time.

func (*Server) RoundTrip

func (s *Server) RoundTrip(r *http.Request) (*http.Response, error)

RoundTrip is http.RoundTriper implementation.

func (*Server) ServeHTTP

func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP proxies http connection to the client.

func (*Server) Start

func (s *Server) Start()

Start starts accepting connections form clients. For accepting http traffic from end users server must be run as handler on http server.

func (*Server) Stop

func (s *Server) Stop()

Stop closes the server.

func (Server) Subscribe

func (r Server) Subscribe(identifier id.ID)

Subscribe allows to connect client with a given identifier.

func (Server) Subscriber

func (r Server) Subscriber(hostPort string) (id.ID, *Auth, bool)

Subscriber returns client identifier assigned to given host.

func (*Server) Unsubscribe

func (s *Server) Unsubscribe(identifier id.ID) *RegistryItem

Unsubscribe removes client from registry, disconnects client if already connected and returns it's RegistryItem.

func (*Server) Upgrade

func (s *Server) Upgrade(identifier id.ID, conn net.Conn, requestBytes []byte) error

type ServerConfig

type ServerConfig struct {
	// Addr is TCP address to listen for client connections. If empty ":0"
	// is used.
	Addr string
	// AutoSubscribe if enabled will automatically subscribe new clients on
	// first call.
	AutoSubscribe bool
	// TLSConfig specifies the tls configuration to use with tls.Listener.
	TLSConfig *tls.Config
	// Listener specifies optional listener for client connections. If nil
	// tls.Listen("tcp", Addr, TLSConfig) is used.
	Listener net.Listener
	// Logger is optional logger. If nil logging is disabled.
	Logger log.Logger
	// Addr is TCP address to listen for TLS SNI connections
	SNIAddr string
	// Optional listener to manage subscribers
	SubscriptionListener SubscriptionListener
}

ServerConfig defines configuration for the Server.

type SubscriptionListener

type SubscriptionListener interface {
	// Invoked if AutoSubscribe is false and must return true if the client is allowed to subscribe or not.
	// If the tlsConfig is configured to require client certificate validation, chain will contain the first
	// verified chain, else the presented peer certificate.
	CanSubscribe(id id.ID, chain []*x509.Certificate) bool
	// Invoked when the client has been subscribed.
	// If the tlsConfig is configured to require client certificate validation, chain will contain the first
	// verified chain, else the presented peer certificate.
	Subscribed(id id.ID, tlsConn *tls.Conn, chain []*x509.Certificate)
	// Invoked before the client is unsubscribed.
	Unsubscribed(id id.ID)
}

A set of listeners to manage subscribers

type TCPProxy

type TCPProxy struct {
	// contains filtered or unexported fields
}

TCPProxy forwards TCP streams.

func NewMultiTCPProxy

func NewMultiTCPProxy(localAddrMap map[string]string, logger log.Logger) *TCPProxy

NewMultiTCPProxy creates a new dispatching TCPProxy, connections may go to different backends based on localAddrMap.

func NewTCPProxy

func NewTCPProxy(localAddr string, logger log.Logger) *TCPProxy

NewTCPProxy creates new direct TCPProxy, everything will be proxied to localAddr.

func (*TCPProxy) Proxy

func (p *TCPProxy) Proxy(w io.Writer, r io.ReadCloser, msg *proto.ControlMessage)

Proxy is a ProxyFunc.