processing

Documentation

Overview

Copyright 2022 Google LLC

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Package processing implements the hashing step for each provide input.

Index

• type BucketNode
• type FileResult
• type Hash
• type Stage
  • func (s *Stage) Run(ctx context.Context) error
• type Storer

Types

type BucketNode

type BucketNode struct {
	NodeHash        Hash `datastore:"node_hash"`
	FilesContained  int  `datastore:"files_contained,noindex"`
	DocumentVersion int  `datastore:"document_version,noindex"`
}

FileResult holds the per file hash and path information.

type FileResult

type FileResult struct {
	Path string `datastore:"path,noindex"`
	Hash Hash   `datastore:"hash"`
}

FileResult holds the per file hash and path information.

type Hash

type Hash = []byte

type Stage

type Stage struct {
	Storer                    Storer
	RepoHdl                   *storage.BucketHandle
	Input                     *pubsub.Subscription
	PubSubOutstandingMessages int
}

Stage holds the data structures necessary to perform the processing.

func (*Stage) Run

func (s *Stage) Run(ctx context.Context) error

Run runs the stages and hashes all files for each incoming request.

type Storer

type Storer interface {
	Store(ctx context.Context, repoInfo *preparation.Result, hashType string, bucketNodes []*BucketNode) error
	Clean(ctx context.Context, repoInfo *preparation.Result, hashType string) error
}

Storer is used to permanently store the results.