Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Configuration ¶
type Configuration struct {
*GlobalConfiguration
*PachdSpecificConfiguration
*WorkerSpecificConfiguration
}
Configuration is the generic configuration structure used to access configuration fields.
func NewConfiguration ¶
func NewConfiguration(config interface{}) *Configuration
NewConfiguration creates a generic configuration from a specific type of configuration.
type FeatureFlags ¶
type FeatureFlags struct {
NewHashTree bool `env:"NEW_HASH_TREE,default=false"`
}
FeatureFlags contains the configuration for feature flags.
type GlobalConfiguration ¶
type GlobalConfiguration struct {
FeatureFlags
EtcdHost string `env:"ETCD_SERVICE_HOST,required"`
EtcdPort string `env:"ETCD_SERVICE_PORT,required"`
PPSWorkerPort uint16 `env:"PPS_WORKER_GRPC_PORT,default=80"`
Port uint16 `env:"PORT,default=650"`
PProfPort uint16 `env:"PPROF_PORT,default=651"`
HTTPPort uint16 `env:"HTTP_PORT,default=652"`
PeerPort uint16 `env:"PEER_PORT,default=653"`
PPSEtcdPrefix string `env:"PPS_ETCD_PREFIX,default=pachyderm_pps"`
Namespace string `env:"NAMESPACE,default=default"`
StorageRoot string `env:"PACH_ROOT,default=/pach"`
}
GlobalConfiguration contains the global configuration.
type PachdFullConfiguration ¶
type PachdFullConfiguration struct {
GlobalConfiguration
PachdSpecificConfiguration
}
PachdFullConfiguration contains the full pachd configuration.
type PachdSpecificConfiguration ¶
type PachdSpecificConfiguration struct {
NumShards uint64 `env:"NUM_SHARDS,default=32"`
StorageBackend string `env:"STORAGE_BACKEND,default="`
StorageHostPath string `env:"STORAGE_HOST_PATH,default="`
EtcdPrefix string `env:"ETCD_PREFIX,default="`
PFSEtcdPrefix string `env:"PFS_ETCD_PREFIX,default=pachyderm_pfs"`
AuthEtcdPrefix string `env:"PACHYDERM_AUTH_ETCD_PREFIX,default=pachyderm_auth"`
EnterpriseEtcdPrefix string `env:"PACHYDERM_ENTERPRISE_ETCD_PREFIX,default=pachyderm_enterprise"`
KubeAddress string `env:"KUBERNETES_PORT_443_TCP_ADDR,required"`
Metrics bool `env:"METRICS,default=true"`
Init bool `env:"INIT,default=false"`
BlockCacheBytes string `env:"BLOCK_CACHE_BYTES,default=1G"`
PFSCacheSize string `env:"PFS_CACHE_SIZE,default=0"`
WorkerImage string `env:"WORKER_IMAGE,default="`
WorkerSidecarImage string `env:"WORKER_SIDECAR_IMAGE,default="`
WorkerImagePullPolicy string `env:"WORKER_IMAGE_PULL_POLICY,default="`
LogLevel string `env:"LOG_LEVEL,default=info"`
IAMRole string `env:"IAM_ROLE,default="`
ImagePullSecret string `env:"IMAGE_PULL_SECRET,default="`
NoExposeDockerSocket bool `env:"NO_EXPOSE_DOCKER_SOCKET,default=false"`
ExposeObjectAPI bool `env:"EXPOSE_OBJECT_API,default=false"`
MemoryRequest string `env:"PACHD_MEMORY_REQUEST,default=1T"`
WorkerUsesRoot bool `env:"WORKER_USES_ROOT,default=true"`
S3GatewayPort uint16 `env:"S3GATEWAY_PORT,default=600"`
}
PachdSpecificConfiguration contains the pachd specific configuration.
type ServiceEnv ¶
type ServiceEnv struct {
*Configuration
// contains filtered or unexported fields
}
ServiceEnv is a struct containing connections to other services in the cluster. In pachd, there is only one instance of this struct, but tests may create more, if they want to create multiple pachyderm "clusters" served in separate goroutines.
func InitPachOnlyEnv ¶
func InitPachOnlyEnv(config *Configuration) *ServiceEnv
InitPachOnlyEnv initializes this service environment. This dials a GRPC connection to pachd only (in a background goroutine), and creates the template pachClient used by future calls to GetPachClient.
This call returns immediately, but GetPachClient will block until the client is ready.
func InitServiceEnv ¶
func InitServiceEnv(config *Configuration) *ServiceEnv
InitServiceEnv initializes this service environment. This dials a GRPC connection to pachd and etcd (in a background goroutine), and creates the template pachClient used by future calls to GetPachClient.
This call returns immediately, but GetPachClient and GetEtcdClient block until their respective clients are ready.
func InitWithKube ¶
func InitWithKube(config *Configuration) *ServiceEnv
InitWithKube is like InitServiceEnv, but also assumes that it's run inside a kubernetes cluster and tries to connect to the kubernetes API server.
func (*ServiceEnv) GetEtcdClient ¶
func (env *ServiceEnv) GetEtcdClient() *etcd.Client
GetEtcdClient returns the already connected etcd client without modification.
func (*ServiceEnv) GetKubeClient ¶
func (env *ServiceEnv) GetKubeClient() *kube.Clientset
GetKubeClient returns the already connected Kubernetes API client without modification.
func (*ServiceEnv) GetPachClient ¶
func (env *ServiceEnv) GetPachClient(ctx context.Context) *client.APIClient
GetPachClient returns a pachd client with the same authentication credentials and cancellation as 'ctx' (ensuring that auth credentials are propagated through downstream RPCs).
Functions that receive RPCs should call this to convert their RPC context to a Pachyderm client, and internal Pachyderm calls should accept clients returned by this call.
(Warning) Do not call this function during server setup unless it is in a goroutine. A Pachyderm client is not available until the server has been setup.
type WorkerFullConfiguration ¶
type WorkerFullConfiguration struct {
GlobalConfiguration
WorkerSpecificConfiguration
}
WorkerFullConfiguration contains the full worker configuration.
type WorkerSpecificConfiguration ¶
type WorkerSpecificConfiguration struct {
// Worker gets its own IP here, via the k8s downward API. It then writes that
// IP back to etcd so that pachd can discover it
PPSWorkerIP string `env:"PPS_WORKER_IP,required"`
// The name of the pipeline that this worker belongs to
PPSPipelineName string `env:"PPS_PIPELINE_NAME,required"`
// The ID of the commit that contains the pipeline spec.
PPSSpecCommitID string `env:"PPS_SPEC_COMMIT,required"`
// The name of this pod
PodName string `env:"PPS_POD_NAME,required"`
}
WorkerSpecificConfiguration contains the worker specific configuration.
Source Files