urldecode

README

About

urldecode is a small special-purpose streaming tokenizer for application/x-www-form-urlencoded data. Its main use case is to parse very long values (tens of megabytes) without allocating memory linear in the length of the values.

Usage

d := urldecode.NewDecoder(req.Body()) // any io.Reader is fine
key, value, err := d.NextPair() // err is io.EOF after last pair
if err != nil {
	panic(err)
}

fmt.Printf("value of key %s: ", key)
io.Copy(os.Stdout, value) // value is an io.Reader

FAQ

Why would anyone want to use this?
Unfortunately there are services out there which will happily POST 80 megabytes of data as application/x-www-form-urlencoded (hello Mandrill). This is a problem because most decoders (including the Go standard library) for this kind of data make the (sane) assumption that these data are relatively short and fit easily into memory. With this library, you can get away with a constant amount of memory and stream the data as needed.

Limitations

Pull requests welcome. Make sure to maintain 100% test coverage.

• Keys (not values) are currently not being decoded, i.e., foo%20 will not decode as foo but foo%20.
• Keys longer than 1023 bytes cannot be parsed.

Documentation

Index

• type Decoder
  • func NewDecoder(r io.Reader) *Decoder
  • func (d *Decoder) NextPair() (string, io.Reader, error)

Types

type Decoder

type Decoder struct {
	// contains filtered or unexported fields
}

Decoder holds the decoder's internal state

func NewDecoder

func NewDecoder(r io.Reader) *Decoder

NewDecoder returns a new decoder for the given source reader

func (*Decoder) NextPair

func (d *Decoder) NextPair() (string, io.Reader, error)

NextPair returns the next key/value pair in the stream or EOF if there are no more pairs