httpfstream

README

httpfstream

httpfstream provides HTTP handlers for simultaneous streaming uploads and downloads of files, as well as persistence and a standalone server.

It allows a writer to APPEND data to a resource via a WebSocket and multiple readers to FOLLOW updates to the resource using WebSockets.

Only one simultaneous appender is allowed for each resource. If there are no appenders at an existing resource, the server returns the full data in an HTTP 200 (bypassing WebSockets) to a follower. If the resource has never been written to, the server responds to a follower with HTTP 404.

• Documentation on Sourcegraph

Installation

go get github.com/sourcegraph/httpfstream

Usage

httpfstream supports 2 modes of usage: as a standalone server or as a Go library.

As a standalone server

The command httpfstream-server launches a server that allows clients to APPEND and FOLLOW arbitrary file paths. Run with -h for more information.

For example, first install the commands:

$ go get github.com/sourcegraph/httpfstream/cmd/...

Then run the server with:

$ httpfstream-server -root=/tmp/httpfstream -http=:8080

Then launch a follower on /foo.txt:

$ httpfstream-follow -v http://localhost:8080/foo.txt
# keep this terminal window open

And start appending to /foo.txt in a separate terminal:

$ httpfstream-append -v http://localhost:8080/foo.txt
# start typing:
foo
bar
baz
# now exit: ctrl-C

Notice that the httpfstream-follow window echoes what you type into the appender window. Once you close the appender, the follower quits as well.

As a Go library

Server

The function httpfstream.New(root string) takes the root file storage path as a parameter and returns an http.Handler that lets clients APPEND and FOLLOW to paths it handles.

The file cmd/httpfstream-server/server.go contains a full example, summarized here:

package main

import (
	"github.com/sourcegraph/httpfstream"
	"log"
	"net/http"
	"os"
)

func main() {
	h := httpfstream.New("/tmp/httpfstream")
	h.Log = log.New(os.Stderr, "", 0)
	http.Handle("/", h)

	err := http.ListenAndServe(":8080", nil)
	if err != nil {
		log.Fatalf("ListenAndServe: %s", err)
	}
}

Appender

Clients can append data to a resource using either httpfstream.Append(u *url.URL, r io.Reader) error (if they already have an io.Reader) or httpfstream.OpenAppend(u *url.URL) (io.WriteCloser, error).

Click on the function names (linked above) to see full docs and usage examples on Sourcegraph.

Follower

Clients can follow a resource's data using httpfstream.Follow(u *url.URL) (io.ReadCloser, error).

Click on the function names (linked above) to see full docs and usage examples on Sourcegraph.

Contributing

Patches and bug reports welcomed! Report issues and submit pull requests using GitHub.

Documentation

Overview

Package httpfstream provides HTTP handlers for simultaneous streaming uploads and downloads of files

Index

• Variables
• func Append(u *url.URL, r io.Reader) error
• func Follow(u *url.URL) (io.ReadCloser, error)
• func OpenAppend(u *url.URL) (io.WriteCloser, error)
• type Handler
  • func New(root string) Handler
  • func (h Handler) Append(w http.ResponseWriter, r *http.Request)
  • func (h Handler) Follow(w http.ResponseWriter, r *http.Request)
  • func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request)

Variables

var ErrWriterConflict = errors.New("path already has an active writer")

ErrWriterConflict indicates that the requested path is currently being written by another writer. A path may have at most one active writer.

Functions

func Append

func Append(u *url.URL, r io.Reader) error

Append appends data from r to the file at the given URL.

func Follow

func Follow(u *url.URL) (io.ReadCloser, error)

Follow opens a WebSocket to the file at the given URL (which must be handled by httpfstream's HTTP handler) and returns the file's contents. The io.ReadCloser continues to return data (blocking as needed) if, and as long as, there is an active writer to the file.

func OpenAppend

func OpenAppend(u *url.URL) (io.WriteCloser, error)

OpenAppend opens a WebSocket to the file at the given URL (which must point be handled by httpfstream's HTTP handler) and returns an io.WriteCloser that writes (via the WebSocket) to that file.

Types

type Handler

type Handler struct {
	Root string
	Log  *log.Logger
	// contains filtered or unexported fields
}

func New

func New(root string) Handler

New returns a new http.Handler for httpfstream.

func (Handler) Append

func (h Handler) Append(w http.ResponseWriter, r *http.Request)

Append handles APPEND requests and appends data to a file.

func (Handler) Follow

func (h Handler) Follow(w http.ResponseWriter, r *http.Request)

Follow handles FOLLOW requests to retrieve the contents of a file and a real-time stream of data that is appended to the file.

func (Handler) ServeHTTP

func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP implements net/http.Handler.