pps

Documentation

Index

• func DescribeSyntaxError(originalErr error, parsedBuffer bytes.Buffer) error
• func GetExpectedNumWorkers(kubeClient *kube.Clientset, spec *ppsclient.ParallelismSpec) (int, error)
• func JobRepo(job *ppsclient.Job) *pfs.Repo
• func PipelineRcName(name string, version uint64) string
• func PipelineRepo(pipeline *ppsclient.Pipeline) *pfs.Repo
• func PipelineReqFromInfo(pipelineInfo *ppsclient.PipelineInfo) *ppsclient.CreatePipelineRequest
• type Hasher
  • func NewHasher(jobModulus uint64, pipelineModulus uint64) *Hasher
  • func (s *Hasher) HashJob(jobID string) uint64
  • func (s *Hasher) HashPipeline(pipelineName string) uint64
• type PipelineManifestReader
  • func NewPipelineManifestReader(path string) (result *PipelineManifestReader, retErr error)
  • func (r *PipelineManifestReader) NextCreatePipelineRequest() (*ppsclient.CreatePipelineRequest, error)

Functions

func DescribeSyntaxError

func DescribeSyntaxError(originalErr error, parsedBuffer bytes.Buffer) error

DescribeSyntaxError describes a syntax error encountered parsing json.

func GetExpectedNumWorkers

func GetExpectedNumWorkers(kubeClient *kube.Clientset, spec *ppsclient.ParallelismSpec) (int, error)

GetExpectedNumWorkers computes the expected number of workers that pachyderm will start given the ParallelismSpec 'spec'.

This is only exported for testing

func JobRepo

func JobRepo(job *ppsclient.Job) *pfs.Repo

JobRepo creates a pfs repo for a given job.

func PipelineRcName

func PipelineRcName(name string, version uint64) string

PipelineRcName generates the name of the k8s replication controller that manages a pipeline's workers

func PipelineRepo

func PipelineRepo(pipeline *ppsclient.Pipeline) *pfs.Repo

PipelineRepo creates a pfs repo for a given pipeline.

func PipelineReqFromInfo

func PipelineReqFromInfo(pipelineInfo *ppsclient.PipelineInfo) *ppsclient.CreatePipelineRequest

PipelineReqFromInfo converts a PipelineInfo into a CreatePipelineRequest.

Types

type Hasher

type Hasher struct {
	JobModulus      uint64
	PipelineModulus uint64
}

A Hasher represents a job/pipeline hasher.

func NewHasher

func NewHasher(jobModulus uint64, pipelineModulus uint64) *Hasher

NewHasher creates a hasher.

func (*Hasher) HashJob

func (s *Hasher) HashJob(jobID string) uint64

HashJob computes and returns the hash of a job.

func (*Hasher) HashPipeline

func (s *Hasher) HashPipeline(pipelineName string) uint64

HashPipeline computes and returns the hash of a pipeline.

type PipelineManifestReader

type PipelineManifestReader struct {
	// contains filtered or unexported fields
}

PipelineManifestReader helps with unmarshalling pipeline configs from JSON. It's used by create-pipeline and update-pipeline

func NewPipelineManifestReader

func NewPipelineManifestReader(path string) (result *PipelineManifestReader, retErr error)

NewPipelineManifestReader creates a new manifest reader from a path.

func (*PipelineManifestReader) NextCreatePipelineRequest

func (r *PipelineManifestReader) NextCreatePipelineRequest() (*ppsclient.CreatePipelineRequest, error)

NextCreatePipelineRequest gets the next request from the manifest reader.