hasher

package module
v1.0.8 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 21, 2023 License: MIT Imports: 19 Imported by: 1

README

hasher

hasher is a Golang package provides functions to compute hash checksums. It's based on iocopy.

Features

  • Start a new goroutine to compute hash checksums
  • Compute multiple hash checksums at one time
  • Provide caller an event channel to receive events
  • The channel is closed automatically when the goroutine exits(an error occurs, user cancels, computing hash checksums is done)

Supported Hash Functions

  • MD5
  • SHA-1
  • SHA-256
  • SHA-512
  • CRC-32

Docs

Usage

Documentation

Index

Examples

Constants

This section is empty.

Variables

View Source 
var (

	// ErrUnSupportedHashAlg indicates that the hash algorithm is not supported.
	ErrUnSupportedHashAlg = errors.New("unsupported hash algorithm")

	// No states specified
	ErrNoStates = errors.New("no states")

	// Not encoding.BinaryMarshaler
	ErrNotBinaryMarshaler = errors.New("not binary marshaler")

	// Not encoding.BinaryUnmarshaler
	ErrNotBinaryUnmarshaler = errors.New("not binary unmarshaler")

	// No checksums computed.
	ErrNoChecksums = errors.New("no checksums computed")

	// Incorrect computed size.
	ErrIncorrectComputedSize = errors.New("incorrect computed size")

	// Range header is not supported by the server.
	ErrRangeNotSupported = errors.New("range header is not supported")

	// Status code is not 200.
	ErrStatusCodeIsNot200 = errors.New("status code is not 200")

	// Status code is not 206.
	ErrStatusCodeIsNot206 = errors.New("status code is not 206")
)

Functions

func SupportedHashAlgs added in v1.0.1 

func SupportedHashAlgs() []string

SupportedHashAlgs returns supported hash algorithms of this package.

Example 
package main

import (
	"fmt"

	"github.com/northbright/hasher"
)

func main() {
	algs := hasher.SupportedHashAlgs()
	l := len(algs)

	for i, alg := range algs {
		fmt.Printf("%v: %v", i, alg)
		if i != l-1 {
			fmt.Printf("\n")
		}
	}

}

Output:

0: CRC-32
1: MD5
2: SHA-1
3: SHA-256
4: SHA-512

Types

type Hasher 

type Hasher struct {
	// contains filtered or unexported fields
}

Hasher is used to compute the hash algorithm checksums.

func FromFile added in v1.0.1 

func FromFile(
	file string,
	hashAlgs []string) (h *Hasher, total int64, err error)

FromFile creates a new Hasher to compute the hashes for the file. file: file path to compute the hashes. hashAlgs: hash algorithms.

Example 
package main

import (
	"context"
	"fmt"
	"io"
	"log"
	"net/http"
	"os"
	"path/filepath"
	"time"

	"github.com/northbright/hasher"
	"github.com/northbright/httputil"
	"github.com/northbright/iocopy"
)

// eventHandler reads the events from channel and block caller's go-routine.
// It updates and reports the progress of computing hashes.
// It returns the number of the computed bytes and the saved states after
// the channel is closed.
// The event channel will be closed after:
// (1). iocopy.EventError received.
// (2). iocopy.EventStop received.
// (3). iocopy.EventOK received.
func eventHandler(
	h *hasher.Hasher,
	total int64,
	previousComputed int64,
	expectedSHA256 string,
	ch <-chan iocopy.Event) (computed int64, states map[string][]byte) {
	states = make(map[string][]byte)

	for event := range ch {
		switch ev := event.(type) {
		case *iocopy.EventWritten:

			computed = previousComputed + ev.Written()
			percent := float32(float64(computed) / (float64(total) / float64(100)))
			log.Printf("on EventWritten: %v/%v bytes written(%.2f%%)", computed, total, percent)

		case *iocopy.EventStop:

			computed = previousComputed + ev.Written()
			states, _ = h.States()

			log.Printf("on EventStop: %v, computed: %v", ev.Err(), computed)

		case *iocopy.EventError:

			log.Printf("on EventError: %v", ev.Err())

		case *iocopy.EventOK:

			computed = previousComputed + ev.Written()
			percent := float32(float64(computed) / (float64(total) / float64(100)))
			log.Printf("on EventOK: %v/%v bytes written(%.2f%%)", computed, total, percent)

			states, _ = h.States()

			checksums := h.Checksums()
			fmt.Printf("SHA-256:\n%x\n", checksums["SHA-256"])

			matched, alg := h.Match(expectedSHA256)
			fmt.Printf("matched: %v, matched hash algorithm: %v", matched, alg)
		}
	}

	return computed, states
}

func download(downloadURL, file string) (int64, error) {
	f, err := os.Create(file)
	if err != nil {
		log.Printf("os.Create() error: %v", err)
		return 0, err
	}

	defer func() {
		f.Close()
	}()

	total, _, err := httputil.ContentLength(downloadURL)
	if err != nil {
		log.Printf("httputil.ContentLength() error: %v", err)
		return 0, err
	}

	resp, err := http.Get(downloadURL)
	if err != nil {
		log.Printf("http.Get() error: %v", err)
		return 0, err
	}
	defer resp.Body.Close()

	copied, err := io.Copy(f, resp.Body)
	if err != nil {
		log.Printf("io.Copy() error: %v", err)
		return 0, err
	}

	if copied != total {
		return 0, fmt.Errorf("downloaded size != total size")
	}

	return total, nil
}

func main() {
	// URL of the remote file.
	downloadURL := "https://golang.google.cn/dl/go1.20.1.darwin-amd64.pkg"
	expectedSHA256 := "9e2f2a4031b215922aa21a3695e30bbfa1f7707597834287415dbc862c6a3251"

	// Get temp file name.
	file := filepath.Join(os.TempDir(), filepath.Base(downloadURL))
	log.Printf("file: %v", file)
	defer os.Remove(file)

	// Download the file.
	total, err := download(downloadURL, file)
	if err != nil {
		log.Printf("download() error: %v", err)
		return
	}

	log.Printf("download file successfully, total: %v bytes", total)

	// Specify hash algorithms.
	// Currently, it supports: "MD5", "SHA-1", "SHA-256", "SHA-512", "CRC-32".
	// Call SupportedHashAlgs to get all available hash algorithms.
	algs := []string{"MD5", "SHA-256"}

	// Create a hasher from the file.
	h, total, err := hasher.FromFile(file, algs)
	if err != nil {
		log.Printf("FromFile() error: %v", err)
		return
	}

	// Close the hasher after use.
	defer h.Close()

	// Start a worker goroutine to compute the hashes of the URL.
	// It will return a channel used to read the events(iocopy.Event).
	ch := h.Start(
		// Context
		context.Background(),
		// Buffer size
		32*1024*1024,
		// Interval to report written bytes
		500*time.Millisecond)

	// Read the events from the events and block current go-routine.
	eventHandler(
		// Hasher
		h,
		// Number of total bytes to compute
		total,
		// Number of previous computed bytes
		0,
		// Expected SHA-256 Checksum.
		expectedSHA256,
		// Event Channel
		ch)

	log.Printf("h.Start() gouroutine exited and the event channel is closed")

}

Output:

SHA-256:
9e2f2a4031b215922aa21a3695e30bbfa1f7707597834287415dbc862c6a3251
matched: true, matched hash algorithm: SHA-256

func FromFileWithStates added in v1.0.1 

func FromFileWithStates(
	file string,
	computed int64,
	states map[string][]byte) (h *Hasher, total int64, err error)

FromFileWithStates creates a new Hasher to contiune to compute the hashes for the file. file: file path to compute the hashes. computed: number of computed(hashed) bytes. It should match the saved states. states: a map stores the saved states. The key is the hash algorithm and the value is the state in byte slice.

Example 
package main

import (
	"context"
	"fmt"
	"io"
	"log"
	"net/http"
	"os"
	"path/filepath"
	"time"

	"github.com/northbright/hasher"
	"github.com/northbright/httputil"
	"github.com/northbright/iocopy"
)

// eventHandler reads the events from channel and block caller's go-routine.
// It updates and reports the progress of computing hashes.
// It returns the number of the computed bytes and the saved states after
// the channel is closed.
// The event channel will be closed after:
// (1). iocopy.EventError received.
// (2). iocopy.EventStop received.
// (3). iocopy.EventOK received.
func eventHandler(
	h *hasher.Hasher,
	total int64,
	previousComputed int64,
	expectedSHA256 string,
	ch <-chan iocopy.Event) (computed int64, states map[string][]byte) {
	states = make(map[string][]byte)

	for event := range ch {
		switch ev := event.(type) {
		case *iocopy.EventWritten:

			computed = previousComputed + ev.Written()
			percent := float32(float64(computed) / (float64(total) / float64(100)))
			log.Printf("on EventWritten: %v/%v bytes written(%.2f%%)", computed, total, percent)

		case *iocopy.EventStop:

			computed = previousComputed + ev.Written()
			states, _ = h.States()

			log.Printf("on EventStop: %v, computed: %v", ev.Err(), computed)

		case *iocopy.EventError:

			log.Printf("on EventError: %v", ev.Err())

		case *iocopy.EventOK:

			computed = previousComputed + ev.Written()
			percent := float32(float64(computed) / (float64(total) / float64(100)))
			log.Printf("on EventOK: %v/%v bytes written(%.2f%%)", computed, total, percent)

			states, _ = h.States()

			checksums := h.Checksums()
			fmt.Printf("SHA-256:\n%x\n", checksums["SHA-256"])

			matched, alg := h.Match(expectedSHA256)
			fmt.Printf("matched: %v, matched hash algorithm: %v", matched, alg)
		}
	}

	return computed, states
}

func download(downloadURL, file string) (int64, error) {
	f, err := os.Create(file)
	if err != nil {
		log.Printf("os.Create() error: %v", err)
		return 0, err
	}

	defer func() {
		f.Close()
	}()

	total, _, err := httputil.ContentLength(downloadURL)
	if err != nil {
		log.Printf("httputil.ContentLength() error: %v", err)
		return 0, err
	}

	resp, err := http.Get(downloadURL)
	if err != nil {
		log.Printf("http.Get() error: %v", err)
		return 0, err
	}
	defer resp.Body.Close()

	copied, err := io.Copy(f, resp.Body)
	if err != nil {
		log.Printf("io.Copy() error: %v", err)
		return 0, err
	}

	if copied != total {
		return 0, fmt.Errorf("downloaded size != total size")
	}

	return total, nil
}

func main() {
	// URL of the remote file.
	downloadURL := "https://golang.google.cn/dl/go1.20.1.darwin-amd64.pkg"
	expectedSHA256 := "9e2f2a4031b215922aa21a3695e30bbfa1f7707597834287415dbc862c6a3251"

	// Get temp file name.
	file := filepath.Join(os.TempDir(), filepath.Base(downloadURL))
	log.Printf("file: %v", file)
	defer os.Remove(file)

	// Download the file.
	total, err := download(downloadURL, file)
	if err != nil {
		log.Printf("download() error: %v", err)
		return
	}

	log.Printf("download file successfully, total: %v bytes", total)

	// Specify hash algorithms.
	// Currently, it supports: "MD5", "SHA-1", "SHA-256", "SHA-512", "CRC-32".
	// Call SupportedHashAlgs to get all available hash algorithms.
	algs := []string{"MD5", "SHA-256"}

	// Stage 1.
	// Create a hasher from the file.
	h1, total, err := hasher.FromFile(file, algs)
	if err != nil {
		log.Printf("FromFile() error: %v", err)
		return
	}

	// Close the hasher after use.
	defer h1.Close()

	// create a context.
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()

	// Start a worker goroutine to compute the hashes of the file.
	// It will return a channel used to read the events(iocopy.Event).
	ch := h1.Start(
		// Context
		ctx,
		// Buffer size
		32*1024*1024,
		// Interval to report written bytes
		200*time.Millisecond)

	// Emulate: user cancelation.
	// Set the timeout.
	go func() {
		<-time.After(100 * time.Millisecond)
		cancel()
	}()

	// Read the events from the events and block current go-routine.
	computed, states := eventHandler(
		// Hasher
		h1,
		// Number of total bytes to compute
		total,
		// Number of previous computed bytes
		0,
		// Expected SHA-256 Checksum.
		expectedSHA256,
		// Event Channel
		ch)

	log.Printf("Stage 1: h1.Start() gouroutine exited and the event channel is closed. Computed: %v", computed)

	// Check if it's all done at stage 1.
	// No need to go to next stages.
	if computed == total {
		return
	}

	// Stage 2.
	// Emulate the user case: pause / resume the computing without
	// exiting the program.
	// The hasher(h1) is still in memory.
	// Re-use the hasher(h1) without loading the states.

	// create a context.
	ctx, cancel = context.WithCancel(context.Background())

	// Re-use previous hasher and continue computing.
	ch = h1.Start(
		// Context
		ctx,
		// Buffer size
		32*1024*1024,
		// Interval to report written bytes
		50*time.Millisecond)

	// Emulate: user cancelation.
	// Set the timeout.
	go func() {
		<-time.After(80 * time.Millisecond)
		cancel()
	}()

	// Read the events from the events and block current go-routine.
	computed, states = eventHandler(
		// Hasher
		h1,
		// Number of total bytes to compute
		total,
		// Number of previous computed bytes
		computed,
		// Expected SHA-256 Checksum.
		expectedSHA256,
		// Event Channel
		ch)

	log.Printf("Stage 2: h1.Start() gouroutine exited and the event channel is closed. Computed: %v", computed)

	// Check if it's all done at stage 2.
	// No need to go to next stages.
	if computed == total {
		return
	}

	// Stage 3.
	// Emulate the user case: exit the program and restart the program.
	// The hasher's memory is freed.
	// Use saved states to continue computing.

	// Create a new hasher from the file with saved states.
	h2, total, err := hasher.FromFileWithStates(
		// File
		file,
		// Number of computed bytes
		computed,
		// States of hashes
		states)

	if err != nil {
		log.Printf("FromUrlWithStates() error: %v", err)
		return
	}

	// Close the hasher after use.
	defer h2.Close()

	// Start a worker goroutine to compute the hashes of the file.
	// It will return a channel used to read the events(iocopy.Event).
	ch = h2.Start(
		// Context
		context.Background(),
		// Buffer size
		32*1024*1024,
		// Interval to report written bytes
		500*time.Millisecond)

	// Read the events from the events and block current go-routine.
	computed, states = eventHandler(
		// Hasher
		h2,
		// Number of total bytes to compute
		total,
		// Number of previous computed bytes
		computed,
		// Expected SHA-256 Checksum.
		expectedSHA256,
		// Event Channel
		ch)

	log.Printf("Stage 3: h2.Start() gouroutine exited and the event channel is closed. Computed: %v", computed)

}

Output:

SHA-256:
9e2f2a4031b215922aa21a3695e30bbfa1f7707597834287415dbc862c6a3251
matched: true, matched hash algorithm: SHA-256

func FromString added in v1.0.1 

func FromString(
	str string,
	hashAlgs []string) (h *Hasher, total int64, err error)

FromString creates a new Hasher to compute the hashes for the string. str: string to compute hashes. hashAlgs: hash algorithms.

func FromStrings added in v1.0.1 

func FromStrings(
	strs []string,
	hashAlgs []string) (h *Hasher, total int64, err error)

FromStrings creates a new Hasher to compute the hashes for the strings. strs: string slice to compute hashes. hashAlgs: hash algorithms.

Example 
package main

import (
	"context"
	"fmt"
	"log"

	"github.com/northbright/hasher"
)

func main() {
	// Example of computing strings hash.
	// Compute the SHA-256 hash of the strings in offical example:
	// https://pkg.go.dev/hash#example-package-BinaryMarshaler
	const (
		input1 = "The tunneling gopher digs downwards, "
		input2 = "unaware of what he will find."
	)

	// Specify hash algorithms.
	// Currently, it supports: "MD5", "SHA-1", "SHA-256", "SHA-512", "CRC-32".
	// Call SupportedHashAlgs to get all available hash algorithms.
	algs := []string{"MD5", "SHA-256", "CRC-32"}

	// Create a hasher with given hash algorithms.
	h, total, _ := hasher.FromStrings(
		// String slice
		[]string{input1, input2},
		// Hash algorithms
		algs)

	// Close the hasher after use.
	defer h.Close()

	// Compute the hashes of the strings.
	checksums, n, _ := h.Compute(context.Background())

	// Show the checksums and count of written bytes.
	for alg, checksum := range checksums {
		log.Printf("%s: %x", alg, checksum)
	}
	log.Printf("%d bytes written(total: %v)", n, total)

	// Output SHA-256 checksum.
	fmt.Printf("%x", checksums["SHA-256"])

}

Output:

57d51a066f3a39942649cd9a76c77e97ceab246756ff3888659e6aa5a07f4a52

func FromUrl added in v1.0.1 

func FromUrl(
	url string,
	hashAlgs []string) (h *Hasher, total int64, err error)

FromUrl creates a new Hasher to compute the hashes for the URL. url: URL to compute hashes. hashAlgs: hash algorithms.

Example 
package main

import (
	"context"
	"fmt"
	"log"
	"time"

	"github.com/northbright/hasher"
	"github.com/northbright/iocopy"
)

// eventHandler reads the events from channel and block caller's go-routine.
// It updates and reports the progress of computing hashes.
// It returns the number of the computed bytes and the saved states after
// the channel is closed.
// The event channel will be closed after:
// (1). iocopy.EventError received.
// (2). iocopy.EventStop received.
// (3). iocopy.EventOK received.
func eventHandler(
	h *hasher.Hasher,
	total int64,
	previousComputed int64,
	expectedSHA256 string,
	ch <-chan iocopy.Event) (computed int64, states map[string][]byte) {
	states = make(map[string][]byte)

	for event := range ch {
		switch ev := event.(type) {
		case *iocopy.EventWritten:

			computed = previousComputed + ev.Written()
			percent := float32(float64(computed) / (float64(total) / float64(100)))
			log.Printf("on EventWritten: %v/%v bytes written(%.2f%%)", computed, total, percent)

		case *iocopy.EventStop:

			computed = previousComputed + ev.Written()
			states, _ = h.States()

			log.Printf("on EventStop: %v, computed: %v", ev.Err(), computed)

		case *iocopy.EventError:

			log.Printf("on EventError: %v", ev.Err())

		case *iocopy.EventOK:

			computed = previousComputed + ev.Written()
			percent := float32(float64(computed) / (float64(total) / float64(100)))
			log.Printf("on EventOK: %v/%v bytes written(%.2f%%)", computed, total, percent)

			states, _ = h.States()

			checksums := h.Checksums()
			fmt.Printf("SHA-256:\n%x\n", checksums["SHA-256"])

			matched, alg := h.Match(expectedSHA256)
			fmt.Printf("matched: %v, matched hash algorithm: %v", matched, alg)
		}
	}

	return computed, states
}

func main() {
	// URL of the remote file.
	downloadURL := "https://golang.google.cn/dl/go1.20.1.darwin-amd64.pkg"
	expectedSHA256 := "9e2f2a4031b215922aa21a3695e30bbfa1f7707597834287415dbc862c6a3251"

	// Specify hash algorithms.
	// Currently, it supports: "MD5", "SHA-1", "SHA-256", "SHA-512", "CRC-32".
	// Call SupportedHashAlgs to get all available hash algorithms.
	algs := []string{"MD5", "SHA-256"}

	// Create a hasher from the URL.
	// The total content length of the URL will be returned if possible.
	h, total, err := hasher.FromUrl(downloadURL, algs)
	if err != nil {
		log.Printf("FromUrl() error: %v", err)
		return
	}

	// Close the hasher after use.
	defer h.Close()

	// Start a worker goroutine to compute the hashes of content of the URL.
	// It will return a channel used to read the events(iocopy.Event).
	ch := h.Start(
		// Context
		context.Background(),
		// Buffer size
		32*1024*1024,
		// Interval to report written bytes
		500*time.Millisecond)

	// Read the events from the events and block current go-routine.
	eventHandler(
		// Hasher
		h,
		// Number of total bytes to compute
		total,
		// Number of previous computed bytes
		0,
		// Expected SHA-256 Checksum.
		expectedSHA256,
		// Event Channel
		ch)

	log.Printf("h.Start() gouroutine exited and the event channel is closed")

}

Output:

SHA-256:
9e2f2a4031b215922aa21a3695e30bbfa1f7707597834287415dbc862c6a3251
matched: true, matched hash algorithm: SHA-256

func FromUrlWithStates added in v1.0.1 

func FromUrlWithStates(
	url string,
	computed int64,
	states map[string][]byte) (h *Hasher, total int64, err error)

FromUrlWithStates creates a new Hasher to contiune to compute the hashes for the URL. url: URL to compute the hashes. computed: number of computed(hashed) bytes. It should match the saved states. states: a map stores the saved states. The key is the hash algorithm and the value is the state in byte slice.

Example 
package main

import (
	"context"
	"fmt"
	"log"
	"time"

	"github.com/northbright/hasher"
	"github.com/northbright/iocopy"
)

// eventHandler reads the events from channel and block caller's go-routine.
// It updates and reports the progress of computing hashes.
// It returns the number of the computed bytes and the saved states after
// the channel is closed.
// The event channel will be closed after:
// (1). iocopy.EventError received.
// (2). iocopy.EventStop received.
// (3). iocopy.EventOK received.
func eventHandler(
	h *hasher.Hasher,
	total int64,
	previousComputed int64,
	expectedSHA256 string,
	ch <-chan iocopy.Event) (computed int64, states map[string][]byte) {
	states = make(map[string][]byte)

	for event := range ch {
		switch ev := event.(type) {
		case *iocopy.EventWritten:

			computed = previousComputed + ev.Written()
			percent := float32(float64(computed) / (float64(total) / float64(100)))
			log.Printf("on EventWritten: %v/%v bytes written(%.2f%%)", computed, total, percent)

		case *iocopy.EventStop:

			computed = previousComputed + ev.Written()
			states, _ = h.States()

			log.Printf("on EventStop: %v, computed: %v", ev.Err(), computed)

		case *iocopy.EventError:

			log.Printf("on EventError: %v", ev.Err())

		case *iocopy.EventOK:

			computed = previousComputed + ev.Written()
			percent := float32(float64(computed) / (float64(total) / float64(100)))
			log.Printf("on EventOK: %v/%v bytes written(%.2f%%)", computed, total, percent)

			states, _ = h.States()

			checksums := h.Checksums()
			fmt.Printf("SHA-256:\n%x\n", checksums["SHA-256"])

			matched, alg := h.Match(expectedSHA256)
			fmt.Printf("matched: %v, matched hash algorithm: %v", matched, alg)
		}
	}

	return computed, states
}

func main() {
	// URL of the remote file.
	downloadURL := "https://golang.google.cn/dl/go1.20.1.darwin-amd64.pkg"
	expectedSHA256 := "9e2f2a4031b215922aa21a3695e30bbfa1f7707597834287415dbc862c6a3251"

	// Specify hash algorithms.
	// Currently, it supports: "MD5", "SHA-1", "SHA-256", "SHA-512", "CRC-32".
	// Call SupportedHashAlgs to get all available hash algorithms.
	algs := []string{"MD5", "SHA-256"}

	// Stage 1.
	// Create a hasher from the URL.
	// The total content length of the URL will be returned if possible.
	h1, total, err := hasher.FromUrl(downloadURL, algs)
	if err != nil {
		log.Printf("FromUrl() error: %v", err)
		return
	}

	// Close the hasher after use.
	defer h1.Close()

	// create a context.
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()

	// Start a worker goroutine to compute the hashes of content of the URL.
	// It will return a channel used to read the events(iocopy.Event).
	ch := h1.Start(
		// Context
		ctx,
		// Buffer size
		32*1024*1024,
		// Interval to report written bytes
		200*time.Millisecond)

	// Emulate: user cancelation.
	// Set the timeout.
	go func() {
		<-time.After(500 * time.Millisecond)
		cancel()
	}()

	// Read the events from the events and block current go-routine.
	computed, states := eventHandler(
		// Hasher
		h1,
		// Number of total bytes to compute
		total,
		// Number of previous computed bytes
		0,
		// Expected SHA-256 Checksum.
		expectedSHA256,
		// Event Channel
		ch)

	log.Printf("Stage 1: h1.Start() gouroutine exited and the event channel is closed. Computed: %v", computed)

	// Check if it's all done at stage 1.
	// No need to go to next stages.
	if computed == total {
		return
	}

	// Stage 2.
	// Emulate the user case: pause / resume the computing without
	// exiting the program.
	// The hasher(h1) is still in memory.
	// Re-use the hasher(h1) without loading the states.

	// create a context.
	ctx, cancel = context.WithCancel(context.Background())

	// Re-use previous hasher and continue computing.
	ch = h1.Start(
		// Context
		ctx,
		// Buffer size
		32*1024*1024,
		// Interval to report written bytes
		200*time.Millisecond)

	// Emulate: user cancelation.
	// Set the timeout.
	go func() {
		<-time.After(500 * time.Millisecond)
		cancel()
	}()

	// Read the events from the events and block current go-routine.
	computed, states = eventHandler(
		// Hasher
		h1,
		// Number of total bytes to compute
		total,
		// Number of previous computed bytes
		computed,
		// Expected SHA-256 Checksum.
		expectedSHA256,
		// Event Channel
		ch)

	log.Printf("Stage 2: h1.Start() gouroutine exited and the event channel is closed. Computed: %v", computed)

	// Check if it's all done at stage 2.
	// No need to go to next stages.
	if computed == total {
		return
	}

	// Stage 3.
	// Emulate the user case: exit and restart the program.
	// The hasher's memory is freed.
	// Use saved states to continue the computing.

	// Create a new hasher from the URL with saved states.
	h2, total, err := hasher.FromUrlWithStates(
		// URL
		downloadURL,
		// Number of computed bytes
		computed,
		// States of hashes
		states)

	if err != nil {
		log.Printf("FromUrlWithStates() error: %v", err)
		return
	}

	// Close the hasher after use.
	defer h2.Close()

	// Start a worker goroutine to compute the hashes of the URL.
	// It will return a channel used to read the events(iocopy.Event).
	ch = h2.Start(
		// Context
		context.Background(),
		// Buffer size
		32*1024*1024,
		// Interval to report written bytes
		500*time.Millisecond)

	// Read the events from the events and block current go-routine.
	computed, states = eventHandler(
		// Hasher
		h2,
		// Number of total bytes to compute
		total,
		// Number of previous computed bytes
		computed,
		// Expected SHA-256 Checksum.
		expectedSHA256,
		// Event Channel
		ch)

	log.Printf("Stage 3: h2.Start() gouroutine exited and the event channel is closed. Computed: %v", computed)

}

Output:

SHA-256:
9e2f2a4031b215922aa21a3695e30bbfa1f7707597834287415dbc862c6a3251
matched: true, matched hash algorithm: SHA-256

func New 

func New(r io.Reader, hashAlgs []string) (*Hasher, error)

New creates a new Hasher. r: io.Reader to read data from. hashAlgs: hash algorithms to compute the checksums.

func NewWithStates added in v1.0.1 

func NewWithStates(r io.Reader, states map[string][]byte) (*Hasher, error)

NewWithStates creates a new Hasher with saved states. r: io.Reader to read data from. The offset to read at should match the saved states. states: a map stores the saved states. The key is the hash algorithm and the value is the state in byte slice.

func (*Hasher) ChecksumStrings added in v1.0.1 

func (h *Hasher) ChecksumStrings() map[string]string

ChecksumStrings returns the checksum strings.

func (*Hasher) Checksums added in v1.0.1 

func (h *Hasher) Checksums() map[string][]byte

Checksums returns the checksums of the hash algorithms. The checksums are stored in a map(key: algorithm, value: checksum in a byte slice). It's not goroutine-safe and should be called only if iocopy.EventOK received. Usage: 1. Call Hasher.Start with a context and read events from the channel. 2. Receive iocopy.EventOK event from the channel and then call Checksums.

func (*Hasher) Close added in v1.0.1 

func (h *Hasher) Close()

Close closes the internal reader if it implements io.ReadCloser. It's not goroutine-safe and should be called only if computing is done / stopped.

func (*Hasher) Compute added in v1.0.1 

func (h *Hasher) Compute(
	ctx context.Context) (checksums map[string][]byte, written int64, err error)

Compute returns the checksums and number of written(hashed) bytes. It blocks the caller's goroutine until the computing is done.

func (*Hasher) Match added in v1.0.1 

func (h *Hasher) Match(checksum string) (matched bool, matchedHashAlg string)

Match checks if the given checksum string matches any hash alogrithm's checksum. When the checksum string matches, it return true and the matched algorithm.

func (*Hasher) Start 

func (h *Hasher) Start(
	ctx context.Context,
	bufSize int64,
	interval time.Duration) <-chan iocopy.Event

Start starts a worker goroutine to read data and compute the hashes. It wraps the basic iocopy.Start fucntion. See https://pkg.go.dev/github.com/northbright/iocopy#Start for more information.

func (*Hasher) States added in v1.0.1 

func (h *Hasher) States() (map[string][]byte, error)

States returns a map stores the latest states of the hashes. The key is the hash algorithm and the value is the state in a byte slice. The states are used to continue to compute the hashes later. It's not goroutine-safe and should be called only if iocopy.EventStop received. Usage: 1. Call Hasher.Start with a context and read events from the channel. 2. Cancel the context or the deadline expires. 3. Receive iocopy.EventStop event from the channel and then call States.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.