remotehttp

README

remotehttp

This repository contains a trivial helper for making secure HTTP-requests with golang.

The Problem

Imagine you have a service to which users to submit tasks containing references to remote objects (HTTP-URLs).

• For example you might allow users to enter the location of a HTML document.
• Your service fetches that remote resource, then converts it to PDF, or similar.
• The results are then shown to the user.

Now imagine what happens if the user supplies URLs such as these, as input to your service:

• http://localhost/server-status
• http://169.254.169.254/latest/meta-data/

This package allows you to prevent these inputs from being processed, easily.

Using It

Sample usage can be found in remotehttp_example_test.go.

Other considerations

This wrapper-library only considers the case of http and https schemas; if you're accepting URIs of your own you should absolutely sanity-check you've not been given something with a file://, or ftp:// prefix (and more!)

Other things you'll want to consider:

• Resource limits such as timeout-handling.
• Resource limits such as whether to follow redirections, and if so how many.

Steve

Documentation

Overview

Package remotehttp is a minor wrapper around a http.Transport which will refuse to fetch local resources.

This package is specifically designed to avoid security attacks which might result from making HTTP-requests with user-supplied URLs.

A prime example of this happening would be a web-service which is designed to fetch a document then convert it to PDF. If the user requests a URL such as `http://localhost/server-status` they would receive a PDF file of private information which they should not have been able to access.

Of course you must make sure that users don't request `file://`, `ftp://` and other resources, but this wrapper will allow you to easily ensure that people cannot access your AWS-metadata store, or any other "internal" resources.

Example

Example shows how access to `http://localhost/server-status` is easily denied.

// The URL we're fetching
url := "http://localhost/server-status"

// Make a HTTP-client with our transport.
var netClient = &http.Client{
	Transport: Transport(),
	Timeout:   5 * time.Second,
}

// Create a request
req, err := http.NewRequest("GET", url, nil)
if err != nil {
	fmt.Printf("error preparing HTTP-request %s %s", url, err.Error())
	return
}

// Make the (GET) request
_, err = netClient.Do(req)
if err != nil {

	//
	// Remove "::1" and "127.0.0.1" in our error-message.
	//
	// Because we could get two different errors:
	//
	//    ip address ::1 is denied as local
	//    ip address 127.0.0.1 is denied as local
	//
	// We want to be stable, and work regardless of what the
	// local testing-system returns.
	//
	out := err.Error()
	out = strings.ReplaceAll(out, "127.0.0.1 ", "")
	out = strings.ReplaceAll(out, "::1 ", "")

	fmt.Printf("ERROR:%s\n", out)
}

Output:

ERROR:Get "http://localhost/server-status": ip address is denied as local

Index

• func Transport() *http.Transport

Examples

• Package

Functions

func Transport

func Transport() *http.Transport

Transport returns our wrapped http.Transport object.

This function is the sole interface to this library, which is designed to automatically deny connections to "local" resources.

You may modify the transport as you wish, once you've received it. However note that the `DialContext` function should not be changed, or our protection is removed.