gosh

README

gosh! Go Share

gosh is a simple HTTP file server on which users can upload their files without login or authentication. All files have a maximum lifetime and are then deleted.

Features

• Standalone HTTP web server, no additional server needed
• Store with both files and some metadata
• Only safe uploader's IP address for legal reasons, anonymous download
• File and all metadata are automatically deleted after expiration
• Configurable maximum lifetime and file size for uploads
• Replace or drop configured MIME types
• Simple upload via curl, wget or the like
• User manual available from the / page
• Uploads can specify their own shorter lifetime
• Burn after Reading: uploads can be deleted after the first download

Installation

Generic Installation

The system needs to have Go installed in version 1.11 or later.

git clone https://github.com/geistesk/gosh.git
cd gosh

go build ./cmd/goshd
go build ./cmd/gosh-query

NixOS Module

On a NixOS system one can import this repository and configure gosh.

# Example configuration to proxy gosh with nginx with a valid HTTPS certificate.

{ config, pkgs, ... }:
{
  imports = [ /path/to/gosh/repo ];

  services = {
    gosh = {
      enable = true;
      contactMail = "abuse@example.com";
      listenAddress = "127.0.0.1:30100";

      maxFilesize = "64MiB";
      maxLifetime = "1w";

      mimeMap = [
        { from = "text/html"; to = "text/plain"; }
      ];
    };

    nginx = {
      enable = true;

      recommendedTlsSettings = true;
      # This one is important.
      recommendedProxySettings = true;

      virtualHosts."gosh.example.com" = {
        enableACME = true;
        forceSSL = true;

        locations."/".proxyPass = "http://127.0.0.1:30100/";
      };
    };
  };
}

Commands

goshd

goshd is the web server, as described above.

Usage of ./goshd:
  -contact string
        Contact E-Mail for abuses
  -listen string
        Listen address for the HTTP server (default ":8080")
  -max-filesize string
        Maximum file size in bytes (default "10MiB")
  -max-lifetime string
        Maximum lifetime (default "24h")
  -mimemap string
        MimeMap to substitute/drop MIMEs
  -store string
        Path to the store
  -verbose
        Verbose logging

An example usage could look like this.

./goshd \
  -contact my@email.address \
  -max-filesize 64MiB \
  -max-lifetime 2w \
  -mimemap Mimemap \
  -store /path/to/my/store/dir

The MimeMap file contains both substitutions or drops in each line and could look as follows.

# Replace text/html with text/plain
text/html text/plain

# Drop PNGs, because reasons.
image/png DROP

gosh-query

The store can also be queried offline to get information or delete items. This is gosh-query's job.

Usage of ./gosh-query:
  -delete
        Delete selection
  -id string
        Query for an ID
  -ip-addr string
        Query for an IP address
  -store string
        Path to the store, env variable GOSHSTORE can also be used
  -verbose
        Verbose logging

# Find all uploads from the localhost:
./gosh-query -store /path/to/store -ip-addr ::1

# Show information for upload with ID abcdef
./gosh-query -store /path/to/store -id abcdef

# And delete this one
./gosh-query -store /path/to/store -delete -id abcdef

Posting

Files can be submitted via HTTP POST with common tools, e.g., with curl.

# Upload foo.png
curl -F 'file=@foo.png' http://our-server.example/

# Burn after reading:
curl -F 'file=@foo.png' -F 'burn=1' http://our-server.example/

# Set a custom expiry date, e.g., one day:
curl -F 'file=@foo.png' -F 'time=1d' http://our-server.example/

# Or all together:
curl -F 'file=@foo.png' -F 'time=1d' -F 'burn=1' http://our-server.example/

Related Work

Of course, there are already similar projects, for example:

• 0x0
• Pomf

Documentation

Index

• Constants
• Variables
• func NewOwnerTypes(r *http.Request) (owners map[OwnerType]net.IP, err error)
• func ParseBytesize(s string) (size int64, err error)
• func ParseDuration(s string) (d time.Duration, err error)
• func PrettyBytesize(bs int64) string
• func PrettyDuration(d time.Duration) string
• func WebProtocol(r *http.Request) string
• type Item
  • func NewItem(r *http.Request, maxSize int64, maxLifetime time.Duration) (item Item, file io.ReadCloser, err error)
  • func (i Item) DeleteFile(directory string) error
  • func (i Item) ReadFile(directory string) (io.ReadCloser, error)
  • func (i Item) WriteFile(file io.ReadCloser, directory string) error
• type MimeMap
  • func NewMimeMap(file io.Reader) (mm MimeMap, err error)
  • func (mm MimeMap) MustDrop(mime string) bool
  • func (mm MimeMap) Substitute(mime string) (mimeOut string, err error)
• type OwnerType
• type Server
  • func NewServer(storeDirectory string, maxSize int64, maxLifetime time.Duration, ...) (s *Server, err error)
  • func (serv *Server) Close() error
  • func (serv *Server) ServeHTTP(w http.ResponseWriter, r *http.Request)
• type Store
  • func NewStore(baseDir string, backgroundCleanup bool) (s *Store, err error)
  • func (s *Store) BadgerHold() *badgerhold.Store
  • func (s *Store) Close() error
  • func (s *Store) Delete(i Item) (err error)
  • func (s *Store) DeleteExpired() error
  • func (s *Store) Get(id string, delExpired bool) (i Item, err error)
  • func (s *Store) GetFile(i Item) (io.ReadCloser, error)
  • func (s *Store) Put(i Item, file io.ReadCloser) (id string, err error)

Constants

const (
	DirDatabase = "db"
	DirStorage  = "data"
)

const MimeDrop = "DROP"

Variables

var (
	ErrLifetimeToLong = errors.New("Lifetime is greater maximum lifetime")

	ErrFileToBig = errors.New("File size is greater maxium filesize")
)

var ErrMimeDrop = errors.New("MIME must be dropped")

var (
	ErrNoMatch = errors.New("Input does not match pattern")
)

var ErrNotFound = errors.New("No Item found for this ID")

ErrNotFound is returned by the `Store.Get` method if there is no Item for the requested ID.

Functions

func NewOwnerTypes

func NewOwnerTypes(r *http.Request) (owners map[OwnerType]net.IP, err error)

NewOwnerTypes creates a map of OwnerTypes to IP addresses based on a Request.

func ParseBytesize

func ParseBytesize(s string) (size int64, err error)

ParseBytesize parses a positive, human readable and whole byte amount in the binary prefix notation. Legit values might be "1B", "23KiB"/"23KB" etc.

func ParseDuration

func ParseDuration(s string) (d time.Duration, err error)

ParseDuration parses a (positive) duration string, similar to the `time.ParseDuration` method. A duration string is sequence of decimal numbers and a unit suffix. Valid time units are "s", "m", "h", "d", "w", "mo", "y".

func PrettyBytesize

func PrettyBytesize(bs int64) string

PrettyBytesize returns a human readable representation of a byte size.

func PrettyDuration

func PrettyDuration(d time.Duration) string

PrettyDuration returns a human readable representation of a time.Duration.

func WebProtocol

func WebProtocol(r *http.Request) string

WebProtocol returns "http" or "https", based on the X-Forwarded-Proto header.

Types

type Item

type Item struct {
	ID string `badgerhold:"key"`

	BurnAfterReading bool

	Filename    string
	ContentType string

	Created time.Time
	Expires time.Time `badgerholdIndex:"Expires"`

	Owner map[OwnerType]net.IP
}

Item describes an uploaded file.

func NewItem

func NewItem(r *http.Request, maxSize int64, maxLifetime time.Duration) (item Item, file io.ReadCloser, err error)

NewItem creates a new Item based on a Request. The ID will be left empty. Furthermore, if no error has occurred, a file is returned from which the file content should be read. This file must be closed afterwards.

func (Item) DeleteFile

func (i Item) DeleteFile(directory string) error

DeleteFile removes the file of an Item from the given directory.

func (Item) ReadFile

func (i Item) ReadFile(directory string) (io.ReadCloser, error)

ReadFile deserializes the file of an Item from the given directory into a ReadCloser.

func (Item) WriteFile

func (i Item) WriteFile(file io.ReadCloser, directory string) error

WriteFile serializes the file of an Item in the given directory. The file name will be the ID of the Item.

type MimeMap

type MimeMap map[string]string

MimeMap replaces predefined MIME types with others or requires them to be dropped.

# An example MimeMap could look like this, comment included:
text/html        text/plain
text/javascript  text/plain
text/mp4         DROP

func NewMimeMap

func NewMimeMap(file io.Reader) (mm MimeMap, err error)

NewMimeMap creates a new MimeMap based on the Reader's data.

func (MimeMap) MustDrop

func (mm MimeMap) MustDrop(mime string) bool

MustDrop indicates if a MIME type must be dropped.

func (MimeMap) Substitute

func (mm MimeMap) Substitute(mime string) (mimeOut string, err error)

Substitute returns the replaced MIME type and indicates with an error, if the input MIME type must be dropped.

type OwnerType

type OwnerType string

OwnerType describes a possible type of an owner, as an IP address. This can be the remote address as well as some header field.

const (
	RemoteAddr    OwnerType = "RemoteAddr"
	Forwarded     OwnerType = "Forwarded"
	XForwardedFor OwnerType = "X-Forwarded-For"
)

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server implements an http.Handler for up- and download.

func NewServer

func NewServer(storeDirectory string, maxSize int64, maxLifetime time.Duration,
	contactMail string, mimeMap MimeMap) (s *Server, err error)

NewServer creates a new Server with a given database directory, and configuration values. The Server must be started as an http.Handler.

func (*Server) Close

func (serv *Server) Close() error

Close the Server and its components.

func (*Server) ServeHTTP

func (serv *Server) ServeHTTP(w http.ResponseWriter, r *http.Request)

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store stores an index of all Items as well as the pure files.

func NewStore

func NewStore(baseDir string, backgroundCleanup bool) (s *Store, err error)

NewStore opens or initializes a Store in the given directory. A background task for continuous cleaning can be activated.

func (*Store) BadgerHold

func (s *Store) BadgerHold() *badgerhold.Store

BadgerHold returns a reference to the underlying BadgerHold instance.

func (*Store) Close

func (s *Store) Close() error

Close the Store and its database.

func (*Store) Delete

func (s *Store) Delete(i Item) (err error)

Delte an Item. Both the database entry and the file will be removed.

func (*Store) DeleteExpired

func (s *Store) DeleteExpired() error

DeleteExpired checks the Store for expired Items and deletes them.

func (*Store) Get

func (s *Store) Get(id string, delExpired bool) (i Item, err error)

Get an Item by its ID. The Item's file can be accessed with GetFile.

func (*Store) GetFile

func (s *Store) GetFile(i Item) (io.ReadCloser, error)

GetFile creates a ReadCloser to the Item's file.

func (*Store) Put

func (s *Store) Put(i Item, file io.ReadCloser) (id string, err error)

Put a new Item inside the Store. Both a database entry and a file will be created.