httpseek

README

httpseek

httpseek is a go package that implements io.Seeker and io.ReaderAt interface on HTTP Response bodies. This allows the client to read parts of a file without downloading it entirely.

To achieve this the HTTP Range request is used. If the HTTP server does not support this, the request will fail.

To prevent lots of tiny http request a buffer is implemented.

Install library

go get github.com/hnz/httpseek

License

MIT Licensed. See LICENSE.md.

Documentation

Overview

httpseek is a go package that implements io.Seeker and io.ReaderAt interface on HTTP Response bodies. This allows the client to read parts of a file without downloading it entirely.

To achieve this the HTTP Range request is used. If the HTTP server does not support this, the request will fail.

To prevent lots of tiny http request a buffer is implemented.

Example

package main

import (
	"fmt"
	"io"

	"github.com/hnz/httpseek"
)

func main() {
	client := new(httpseek.Client)
	client.Blocksize = 64 * 1024
	resp, err := client.Get("http://textfiles.com/100/phrack.01.phk")
	if err != nil {
		panic(err)
	}

	buf := make([]byte, 33)
	x, err := resp.Body.Seek(555, io.SeekStart)
	y, err := resp.Body.Read(buf)
	if err != nil {
		panic(err)
	}

	fmt.Printf("Read %d bytes starting from position %d. Output: %s", y, x, buf)
}

Output:

Read 33 bytes starting from position 555. Output: Welcome to the Phrack Inc. Philes

Index

• Variables
• type Client
  • func (c *Client) Do(req *http.Request) (*Response, error)
  • func (c *Client) Get(url string) (resp *Response, err error)
• type RangeNotSupported
  • func (e *RangeNotSupported) Error() string
• type ReaderAtBuffer
  • func (o *ReaderAtBuffer) ReadAt(p []byte, off int64) (n int, err error)
• type Response
• type ResponseBody
  • func (o *ResponseBody) Close() error
  • func (o *ResponseBody) Read(p []byte) (n int, err error)
  • func (o *ResponseBody) ReadAt(p []byte, off int64) (n int, err error)
  • func (o *ResponseBody) Seek(offset int64, whence int) (int64, error)

Examples

• Package
• Client

Variables

var Logger *log.Logger

Logger logs debug messages if set

Types

type Client

type Client struct {
	*http.Client // If empty use http.DefaultClient

	Blocksize int64
}

Client is an implementation of http.Client that allows for the response body to be seeked

Example

package main

import (
	"archive/zip"
	"fmt"

	"github.com/hnz/httpseek"
)

func main() {

	client := new(httpseek.Client)
	client.Blocksize = 64 * 1024
	resp, err := client.Get("https://climate.onebuilding.org/WMO_Region_6_Europe/LUX_Luxembourg/LUX_LU_Luxembourg.AP.065900_TMYx.2004-2018.zip")
	if err != nil {
		panic(err)
	}

	archive, err := zip.NewReader(&resp.Body, resp.ContentLength)
	if err != nil {
		panic(err)
	}

	for _, f := range archive.File {
		fmt.Println("Found file", f.Name)
	}
}

Output:

Found file LUX_LU_Luxembourg.AP.065900_TMYx.2004-2018.clm
Found file LUX_LU_Luxembourg.AP.065900_TMYx.2004-2018.ddy
Found file LUX_LU_Luxembourg.AP.065900_TMYx.2004-2018.epw
Found file LUX_LU_Luxembourg.AP.065900_TMYx.2004-2018.pvsyst
Found file LUX_LU_Luxembourg.AP.065900_TMYx.2004-2018.rain
Found file LUX_LU_Luxembourg.AP.065900_TMYx.2004-2018.stat
Found file LUX_LU_Luxembourg.AP.065900_TMYx.2004-2018.wea

func (*Client) Do

func (c *Client) Do(req *http.Request) (*Response, error)

See https://pkg.go.dev/net/http#Client.Do

func (*Client) Get

func (c *Client) Get(url string) (resp *Response, err error)

See https://pkg.go.dev/net/http#Client.Get

type RangeNotSupported

type RangeNotSupported struct {
	Url string
}

RangeNotSupported is returned when the remote http server does not support the http Range header

func (*RangeNotSupported) Error

func (e *RangeNotSupported) Error() string

type ReaderAtBuffer

type ReaderAtBuffer struct {
	Blocksize     int64
	ContentLength int64
	ReaderAt      io.ReaderAt
	// contains filtered or unexported fields
}

func (*ReaderAtBuffer) ReadAt

func (o *ReaderAtBuffer) ReadAt(p []byte, off int64) (n int, err error)

type Response

type Response struct {
	*http.Response

	Body ResponseBody
}

See https://pkg.go.dev/net/http#Response

type ResponseBody

type ResponseBody struct {
	ReaderAtBuffer

	Body io.ReadCloser
	// contains filtered or unexported fields
}

ResponseBody implents the io.ReaderAt and io.Seeker interfaces

func (*ResponseBody) Close

func (o *ResponseBody) Close() error

See https://pkg.go.dev/io#Closer

func (*ResponseBody) Read

func (o *ResponseBody) Read(p []byte) (n int, err error)

See https://pkg.go.dev/io#Reader

func (*ResponseBody) ReadAt

func (o *ResponseBody) ReadAt(p []byte, off int64) (n int, err error)

See https://pkg.go.dev/io#ReaderAt

func (*ResponseBody) Seek

func (o *ResponseBody) Seek(offset int64, whence int) (int64, error)

See https://pkg.go.dev/io#Seeker