remotehttp

README

remotehttp

This repository contains a trivial helper for making secure HTTP-requests with golang.

The Problem

Imagine you have a service to which users to submit tasks containing references to remote objects (HTTP-URLs).

• For example you might allow users to enter the location of a HTML document.
• Your service fetches that remote resource, then converts it to PDF, or similar.
• The results are then shown to the user.

Now imagine what happens if the user supplies URLs such as these, as input to your service:

• http://localhost/server-status
• http://169.254.169.254/latest/meta-data/

This package allows you to prevent these inputs from being processed, easily.

Using It

Downsides?

The only obvious downside I can see is that we have to actually establish a connection to the remote host, before we know where we went.

Connecting, then checking, is not ideal but there is no sane alternative.

(Rsolving the hostname of a user-supplied URL, then testing that against a whitelist/blacklist, is not sufficient because the DNS address might change after it has been tested, but before it has been used - i.e. race-condition.)

Steve

Documentation

Overview

Package remotehttp is a minor wrapper around a http.Transport which will refuse to fetch local resources.

This package is specifically designed to avoid security attacks which might result from making HTTP-requests with user-supplied URLs.

A prime example of this happening would be a web-service which is designed to fetch a document then convert it to PDF. If the user requests a URL such as `http://localhost/server-status` they would receive a pretty PDF version of private information to which they should not be able to access.

Of course you must make sure that users don't request `file://`, `ftp://` and other resources, but this wrapper will allow you to easily ensure that people cannot access your AWS-metadata store, or any other "internal" resources.

Index

• Variables

Variables

var (
	// Transport is our exported http.Transport object.
	//
	// This is the sole interface to this library, and it is
	// designed to automatically deny connections which have
	// been established to "local" resources.
	//
	// You may modify the transport as you wish, once you've received
	// it.  However note that the DialContext and DialTLS fields should
	// not be modified, or our protection is removed.
	Transport = &http.Transport{
		Dial: (&net.Dialer{
			Timeout:   30 * time.Second,
			KeepAlive: 30 * time.Second,
		}).Dial,
		DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {

			c, err := net.Dial(network, addr)
			if err != nil {
				return nil, err
			}

			ip, _, _ := net.SplitHostPort(c.RemoteAddr().String())

			err = _isLocalIP(ip)
			if err != nil {
				return c, err
			}

			return c, err
		},
		DialTLS: func(network, addr string) (net.Conn, error) {

			c, err := tls.Dial(network, addr, &tls.Config{})
			if err != nil {
				return nil, err
			}

			ip, _, _ := net.SplitHostPort(c.RemoteAddr().String())

			err = _isLocalIP(ip)
			if err != nil {
				return c, err
			}

			err = c.Handshake()
			if err != nil {
				return c, err
			}

			return c, c.Handshake()
		},
		TLSHandshakeTimeout:   5 * time.Second,
		ResponseHeaderTimeout: 5 * time.Second,
	}
)