serviceenv

package
v1.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 12, 2019 License: Apache-2.0 Imports: 14 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Configuration

Configuration is the generic configuration structure used to access configuration fields.

func NewConfiguration

func NewConfiguration(config interface{}) *Configuration

NewConfiguration creates a generic configuration from a specific type of configuration.

type FeatureFlags

type FeatureFlags struct {
	NewHashTree bool `env:"NEW_HASH_TREE,default=false"`
}

FeatureFlags contains the configuration for feature flags.

type GlobalConfiguration

type GlobalConfiguration struct {
	FeatureFlags
	EtcdHost      string `env:"ETCD_SERVICE_HOST,required"`
	EtcdPort      string `env:"ETCD_SERVICE_PORT,required"`
	PPSWorkerPort uint16 `env:"PPS_WORKER_GRPC_PORT,default=80"`
	Port          uint16 `env:"PORT,default=650"`
	PProfPort     uint16 `env:"PPROF_PORT,default=651"`
	HTTPPort      uint16 `env:"HTTP_PORT,default=652"`
	PeerPort      uint16 `env:"PEER_PORT,default=653"`
	PPSEtcdPrefix string `env:"PPS_ETCD_PREFIX,default=pachyderm_pps"`
	Namespace     string `env:"NAMESPACE,default=default"`
	StorageRoot   string `env:"PACH_ROOT,default=/pach"`
}

GlobalConfiguration contains the global configuration.

type PachdFullConfiguration

type PachdFullConfiguration struct {
	GlobalConfiguration
	PachdSpecificConfiguration
}

PachdFullConfiguration contains the full pachd configuration.

type PachdSpecificConfiguration

type PachdSpecificConfiguration struct {
	NumShards             uint64 `env:"NUM_SHARDS,default=32"`
	StorageBackend        string `env:"STORAGE_BACKEND,default="`
	StorageHostPath       string `env:"STORAGE_HOST_PATH,default="`
	EtcdPrefix            string `env:"ETCD_PREFIX,default="`
	PFSEtcdPrefix         string `env:"PFS_ETCD_PREFIX,default=pachyderm_pfs"`
	AuthEtcdPrefix        string `env:"PACHYDERM_AUTH_ETCD_PREFIX,default=pachyderm_auth"`
	EnterpriseEtcdPrefix  string `env:"PACHYDERM_ENTERPRISE_ETCD_PREFIX,default=pachyderm_enterprise"`
	KubeAddress           string `env:"KUBERNETES_PORT_443_TCP_ADDR,required"`
	Metrics               bool   `env:"METRICS,default=true"`
	Init                  bool   `env:"INIT,default=false"`
	BlockCacheBytes       string `env:"BLOCK_CACHE_BYTES,default=1G"`
	PFSCacheSize          string `env:"PFS_CACHE_SIZE,default=0"`
	WorkerImage           string `env:"WORKER_IMAGE,default="`
	WorkerSidecarImage    string `env:"WORKER_SIDECAR_IMAGE,default="`
	WorkerImagePullPolicy string `env:"WORKER_IMAGE_PULL_POLICY,default="`
	LogLevel              string `env:"LOG_LEVEL,default=info"`
	IAMRole               string `env:"IAM_ROLE,default="`
	ImagePullSecret       string `env:"IMAGE_PULL_SECRET,default="`
	NoExposeDockerSocket  bool   `env:"NO_EXPOSE_DOCKER_SOCKET,default=false"`
	ExposeObjectAPI       bool   `env:"EXPOSE_OBJECT_API,default=false"`
	MemoryRequest         string `env:"PACHD_MEMORY_REQUEST,default=1T"`
	WorkerUsesRoot        bool   `env:"WORKER_USES_ROOT,default=true"`
	S3GatewayPort         uint16 `env:"S3GATEWAY_PORT,default=600"`
}

PachdSpecificConfiguration contains the pachd specific configuration.

type ServiceEnv

type ServiceEnv struct {
	*Configuration
	// contains filtered or unexported fields
}

ServiceEnv is a struct containing connections to other services in the cluster. In pachd, there is only one instance of this struct, but tests may create more, if they want to create multiple pachyderm "clusters" served in separate goroutines.

func InitPachOnlyEnv

func InitPachOnlyEnv(config *Configuration) *ServiceEnv

InitPachOnlyEnv initializes this service environment. This dials a GRPC connection to pachd only (in a background goroutine), and creates the template pachClient used by future calls to GetPachClient.

This call returns immediately, but GetPachClient will block until the client is ready.

func InitServiceEnv

func InitServiceEnv(config *Configuration) *ServiceEnv

InitServiceEnv initializes this service environment. This dials a GRPC connection to pachd and etcd (in a background goroutine), and creates the template pachClient used by future calls to GetPachClient.

This call returns immediately, but GetPachClient and GetEtcdClient block until their respective clients are ready.

func InitWithKube

func InitWithKube(config *Configuration) *ServiceEnv

InitWithKube is like InitServiceEnv, but also assumes that it's run inside a kubernetes cluster and tries to connect to the kubernetes API server.

func (*ServiceEnv) GetEtcdClient

func (env *ServiceEnv) GetEtcdClient() *etcd.Client

GetEtcdClient returns the already connected etcd client without modification.

func (*ServiceEnv) GetKubeClient

func (env *ServiceEnv) GetKubeClient() *kube.Clientset

GetKubeClient returns the already connected Kubernetes API client without modification.

func (*ServiceEnv) GetPachClient

func (env *ServiceEnv) GetPachClient(ctx context.Context) *client.APIClient

GetPachClient returns a pachd client with the same authentication credentials and cancellation as 'ctx' (ensuring that auth credentials are propagated through downstream RPCs).

Functions that receive RPCs should call this to convert their RPC context to a Pachyderm client, and internal Pachyderm calls should accept clients returned by this call.

(Warning) Do not call this function during server setup unless it is in a goroutine. A Pachyderm client is not available until the server has been setup.

type WorkerFullConfiguration

type WorkerFullConfiguration struct {
	GlobalConfiguration
	WorkerSpecificConfiguration
}

WorkerFullConfiguration contains the full worker configuration.

type WorkerSpecificConfiguration

type WorkerSpecificConfiguration struct {
	// Worker gets its own IP here, via the k8s downward API. It then writes that
	// IP back to etcd so that pachd can discover it
	PPSWorkerIP string `env:"PPS_WORKER_IP,required"`
	// The name of the pipeline that this worker belongs to
	PPSPipelineName string `env:"PPS_PIPELINE_NAME,required"`
	// The ID of the commit that contains the pipeline spec.
	PPSSpecCommitID string `env:"PPS_SPEC_COMMIT,required"`
	// The name of this pod
	PodName string `env:"PPS_POD_NAME,required"`
}

WorkerSpecificConfiguration contains the worker specific configuration.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL