app

package
v1.12.0-RC2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 8, 2020 License: Apache-2.0 Imports: 32 Imported by: 1

Documentation

Overview

Package app contains OpenEBS Dynamic Local PV provisioner

Provisioner is created using the external storage provisioner library: https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner

Local PVs are an extension to hostpath volumes, but are more secure. https://kubernetes.io/docs/concepts/policy/pod-security-policy/#volumes-and-file-systems

Local PVs are great in cases like:

  • The Stateful Workload can take care of replicating the data across nodes to handle cases like a complete node (and/or its storage) failure.
  • For long running Stateful Workloads, the Backup/Recovery is provided by Operators/tools that can make use the Workload mounts and do not require the capabilities to be available in the underlying storage. Or if the hostpaths are created on external storage like EBS/GPD, administrator have tools that can periodically take snapshots/backups.

OpenEBS Local PVs extends the capabilities provided by the Kubernetes Local PV by making use of the OpenEBS Node Storage Device Manager (NDM), the significant differences include:

  • Users need not pre-format and mount the devices in the node.
  • Supports Dynamic Local PVs - where the devices can be used by CAS solutions and also by applications. CAS solutions typically directly access a device. OpenEBS Local PV ease the management of storage devices to be used between CAS solutions (direct access) and applications (via PV), by making use of BlockDeviceClaims supported by OpenEBS NDM.
  • Supports using hostpath as well for provisioning a Local PV. In fact in some cases, the Kubernetes nodes may have limited number of storage devices attached to the node and hostpath based Local PVs offer efficient management of the storage available on the node.

Inspiration: ------------ OpenEBS Local PV has been inspired by the prior work done by the following the Kubernetes projects: - https://github.com/kubernetes-sigs/sig-storage-lib-external-provisioner/tree/master/examples/hostpath-provisioner - https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner - https://github.com/rancher/local-path-provisioner

How it works: ------------- Step 1: Multiple Storage Classes can be created by the Kubernetes Administrator, to specify the required type of OpenEBS Local PV to be used by an application. A simple StorageClass looks like:

--- apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: 

name: hostpath
annotations:
  #Define a new OpenEBS CAS Type called `local`
  #which indicates that Data is stored
  #directly onto hostpath. The hostpath can be:
  #- device (as block or mounted path)
  #- hostpath (sub directory on OS or mounted path)
  openebs.io/cas-type: local
  cas.openebs.io/config: |
    #- name: StorageType
    #  value: "device"
    # (Default)
    - name: StorageType
      value: "hostpath"
    # If the StorageType is hostpath, then BasePath
    # specifies the location where the volume sub-directory
    # should be created.
    # (Default)
    - name: BasePath
      value: "/var/openebs/local"

provisioner: openebs.io/local volumeBindingMode: WaitForFirstConsumer reclaimPolicy: Delete ---

Step 2: The application developers will request for storage via PVC as follows: --- apiVersion: v1 kind: PersistentVolumeClaim metadata: 

name: pvc-hp

spec: 

accessModes:
  - ReadWriteOnce
storageClassName: hostpath
resources:
  requests:
    storage: 2Gi

---

Step 3: A Local PV (type=hostpath) provisioned via the OpenEBS Dynamic Local PV Provisioner looks like this: --- apiVersion: v1 kind: PersistentVolume metadata: 

annotations:
  pv.kubernetes.io/provisioned-by: openebs.io/local
creationTimestamp: 2019-05-02T15:44:35Z
finalizers:
- kubernetes.io/pv-protection
name: pvc-2fe08284-6cf1-11e9-be8b-42010a800155
resourceVersion: "2062"
selfLink: /api/v1/persistentvolumes/pvc-2fe08284-6cf1-11e9-be8b-42010a800155
uid: 2fedaff8-6cf1-11e9-be8b-42010a800155

spec: 

accessModes:
- ReadWriteOnce
capacity:
  storage: 2Gi
claimRef:
  apiVersion: v1
  kind: PersistentVolumeClaim
  name: pvc-hp
  namespace: default
  resourceVersion: "2060"
  uid: 2fe08284-6cf1-11e9-be8b-42010a800155
local:
  path: /var/openebs/local/pvc-2fe08284-6cf1-11e9-be8b-42010a800155
  fsType: ""
nodeAffinity:
  required:
    nodeSelectorTerms:
    - matchExpressions:
      - key: kubernetes.io/hostname
        operator: In
        values:
        - gke-kmova-helm-default-pool-6c1271a5-n8b0
persistentVolumeReclaimPolicy: Delete
storageClassName: hostpath

status: 

phase: Bound

---

Note that the location of the hostpaths on the node are abstracted from the application developers and are under the Administrators control.

Implementation Details: ----------------------- (a) The configuration of whether to select a complete storage device 

of a hostpath is determined by the StorageClass annotations, inline
with other configuration options provided by OpenEBS.

(b) When using StorageType as device, the Local Provisioner will 

interact with the OpenEBS Node Storage Device Manager (NDM) to
identify the device to be used. Each PVC of type StorageType=device
will create a BDC and wait for the NDM to provide an appropriate BD.
From the BD, the provisioner will extract the path details and create
a Local PV. When using unformatted block devices, the administrator
can specific the type of FS to be put on block devices using the
FSType CAS Policy in StorageClass.

(c) The StorageClass can work either with `waitForConsumer`, in which 

case, the PV is created on the node where the Pod is scheduled or
vice versa. (Note: In the initial version, only `waitForConsumer` is
supported.)

(d) When using the hostpath, the administrator can select the location using:

  • BasePath: By default, the hostpath volumes will be created under `/var/openebs/local`. This default path can be changed by passing the "OPENEBS_IO_BASE_PATH" ENV variable to the Hostpath Provisioner Pod. It is also possible to specify a different location using the CAS Policy `BasePath` in the StorageClass.

    The hostpath used in the above configuration can be:

  • OS Disk - possibly a folder dedicated to saving data on each node.

  • Additional Disks - mounted as ext4 or any other filesystem

  • External Storage - mounted as ext4 or any other filesystem

(e) The backup and restore via Velero Plugin has been verified to work for 

OpenEBS Local PV. Supported from OpenEBS 1.0 and higher.

Future Improvements and Limitations: ------------------------------------

  • Ability to enforce capacity limits. The application can exceed it usage of capacity beyond what it requested.
  • Ability to enforce provisioning limits based on the capacity available on a given node or the number of PVs already provisioned.
  • Ability to use hostpaths and devices that can potentially support snapshots. Example: a hostpath backed by github, or by LVM or ZFS where capacity also can be enforced.
  • Extend the capabilities of the Local PV provisioner to handle cases where underlying devices are moved to new node and needs changes to the node affinity.
  • Move towards using a CSI based Hostpath provisioner.

Index

Constants

View Source 
const (
	//KeyPVStorageType defines if the PV should be backed
	// a hostpath ( sub directory or a storage device)
	KeyPVStorageType = "StorageType"

	//KeyPVBasePath defines base directory for hostpath volumes
	// can be configured via the StorageClass annotations.
	KeyPVBasePath = "BasePath"

	//KeyPVFSType defines filesystem type to be used with devices
	// and can be configured via the StorageClass annotations.
	KeyPVFSType = "FSType"

	//KeyBDTag defines the value for the Block Device Tag
	//label selector configured via the StorageClass annotations.
	//User can group block devices across nodes by setting the
	//label on block devices as:
	//  openebs.io/block-device-tag=<tag-value>
	//
	//The <tag-value> used above can be passsed to the
	//Local PV device provisioner via the StorageClass
	//CAS annotations, to specify that Local PV (device)
	//should only make use of those block devices that
	//tagged with the given <tag-value>.
	//
	//Example: Local PV device StorageClass for picking devices
	//labeled as: openebs.io/block-device-tag=tag-x
	//will be as follows
	//
	// kind: StorageClass
	// metadata:
	//   name: openebs-device-tag-x
	//   annotations:
	//     openebs.io/cas-type: local
	//     cas.openebs.io/config: |
	//       - name: StorageType
	//         value: "device"
	//       - name: BlockDeviceTag
	//         value: "tag-x"
	// provisioner: openebs.io/local
	// volumeBindingMode: WaitForFirstConsumer
	// reclaimPolicy: Delete
	//
	KeyBDTag = "BlockDeviceTag"
)
View Source 
const (
	// ProvisionerHelperImage is the environment variable that provides the
	// container image to be used to launch the help pods managing the
	// host path
	ProvisionerHelperImage menv.ENVKey = "OPENEBS_IO_HELPER_IMAGE"

	// ProvisionerBasePath is the environment variable that provides the
	// default base path on the node where host-path PVs will be provisioned.
	ProvisionerBasePath menv.ENVKey = "OPENEBS_IO_BASE_PATH"
)
View Source 
const (

	// LocalPVFinalizer represents finalizer string used by LocalPV
	LocalPVFinalizer = "local.openebs.io/finalizer"
)

Variables

View Source 
var (
	//CmdTimeoutCounts specifies the duration to wait for cleanup pod
	//to be launched.
	CmdTimeoutCounts = 120
)
View Source 
var (
	//WaitForBDTimeoutCounts specifies the duration to wait for BDC to be associated with a BD
	//The duration is the value specified here multiplied by 5
	WaitForBDTimeoutCounts = 12
)

TODO

Functions

func GetLocalPVType 

func GetLocalPVType(pv *v1.PersistentVolume) string

GetLocalPVType extracts the Local PV Type from PV

func GetNodeHostname 

func GetNodeHostname(n *v1.Node) string

GetNodeHostname extracts the Hostname from the labels on the Node If hostname label `kubernetes.io/hostname` is not present an empty string is returned.

func GetStorageClassName 

func GetStorageClassName(pvc *v1.PersistentVolumeClaim) *string

GetStorageClassName extracts the StorageClass name from PVC

func GetTaints 

func GetTaints(n *v1.Node) []v1.Taint

GetTaints extracts the Taints from the Spec on the node If Taints are empty, it just returns empty structure of corev1.Taints

func RegisterShutdownChannel 

func RegisterShutdownChannel(done chan struct{})

RegisterShutdownChannel closes the channel when signaled for termination

func Start 

func Start(cmd *cobra.Command) error

Start will initialize and run the dynamic provisioner daemon

func StartProvisioner 

func StartProvisioner() (*cobra.Command, error)

StartProvisioner will start a new dynamic Host Path PV provisioner

Types

type GetVolumeConfigFn 

type GetVolumeConfigFn func(pvName string, pvc *v1.PersistentVolumeClaim) (*VolumeConfig, error)

GetVolumeConfigFn allows to plugin a custom function 

and makes it easy to unit test provisioner

type HelperBlockDeviceOptions 

type HelperBlockDeviceOptions struct {
	// contains filtered or unexported fields
}

HelperBlockDeviceOptions contains the options that will launch a BDC on a specific node (nodeHostname)

type HelperPodOptions 

type HelperPodOptions struct {
	// contains filtered or unexported fields
}

HelperPodOptions contains the options that will launch a Pod on a specific node (nodeHostname) to execute a command (cmdsForPath) on a given volume path (path)

type Provisioner 

type Provisioner struct {
	// contains filtered or unexported fields
}

Provisioner struct has the configuration and utilities required across the different work-flows.

func NewProvisioner 

func NewProvisioner(stopCh chan struct{}, kubeClient *clientset.Clientset) (*Provisioner, error)

NewProvisioner will create a new Provisioner object and initialize 

it with global information used across PV create and delete operations.

func (*Provisioner) Delete 

func (p *Provisioner) Delete(pv *v1.PersistentVolume) (err error)

Delete is invoked by the PVC controller to perform clean-up 

activities before deleteing the PV object. If reclaim policy is
set to not-retain, then this function will create a helper pod
to delete the host path from the node.

func (*Provisioner) DeleteBlockDevice 

func (p *Provisioner) DeleteBlockDevice(pv *v1.PersistentVolume) (err error)

DeleteBlockDevice is invoked by the PVC controller to perform clean-up 

activities before deleteing the PV object. If reclaim policy is
set to not-retain, then this function will delete the associated BDC

func (*Provisioner) DeleteHostPath 

func (p *Provisioner) DeleteHostPath(pv *v1.PersistentVolume) (err error)

DeleteHostPath is invoked by the PVC controller to perform clean-up 

activities before deleteing the PV object. If reclaim policy is
set to not-retain, then this function will create a helper pod
to delete the host path from the node.

func (*Provisioner) GetNodeObjectFromHostName 

func (p *Provisioner) GetNodeObjectFromHostName(hostName string) (*v1.Node, error)

GetNodeObjectFromHostName returns the Node Object with matching NodeHostName.

func (*Provisioner) GetVolumeConfig 

func (p *Provisioner) GetVolumeConfig(pvName string, pvc *v1.PersistentVolumeClaim) (*VolumeConfig, error)

GetVolumeConfig creates a new VolumeConfig struct by parsing and merging the configuration provided in the PVC annotation - cas.openebs.io/config with the default configuration of the provisioner.

func (*Provisioner) Provision 

func (p *Provisioner) Provision(opts pvController.ProvisionOptions) (*v1.PersistentVolume, error)

Provision is invoked by the PVC controller which expect the PV 

to be provisioned and a valid PV spec returned.

func (*Provisioner) ProvisionBlockDevice 

func (p *Provisioner) ProvisionBlockDevice(opts pvController.ProvisionOptions, volumeConfig *VolumeConfig) (*v1.PersistentVolume, error)

ProvisionBlockDevice is invoked by the Provisioner to create a Local PV 

with a Block Device

func (*Provisioner) ProvisionHostPath 

func (p *Provisioner) ProvisionHostPath(opts pvController.ProvisionOptions, volumeConfig *VolumeConfig) (*v1.PersistentVolume, error)

ProvisionHostPath is invoked by the Provisioner which expect HostPath PV 

to be provisioned and a valid PV spec returned.

func (*Provisioner) SupportsBlock 

func (p *Provisioner) SupportsBlock() bool

SupportsBlock will be used by controller to determine if block mode is 

supported by the host path provisioner.

type VolumeConfig 

type VolumeConfig struct {
	// contains filtered or unexported fields
}

VolumeConfig struct contains the merged configuration of the PVC and the associated SC. The configuration is derived from the annotation `cas.openebs.io/config`. The configuration will be in the following json format: 

{
  Key1:{
	enabled: true
	value: "string value"
  },
  Key2:{
	enabled: true
	value: "string value"
  },
}

func (*VolumeConfig) GetBDTagValue 

func (c *VolumeConfig) GetBDTagValue() string

GetBDTagValue returns the block device tag value configured in StorageClass.

Default is "", no device tag will be set and any available block device (without labelled with tag) can be used for creating Local PV(device).

func (*VolumeConfig) GetFSType 

func (c *VolumeConfig) GetFSType() string

GetFSType returns the FSType value configured in StorageClass. Default is "", auto-determined by Local PV

func (*VolumeConfig) GetPath 

func (c *VolumeConfig) GetPath() (string, error)

GetPath returns a valid PV path based on the configuration or an error. The Path is constructed using the following rules: If AbsolutePath is specified return it. (Future) If PVPath is specified, suffix it with BasePath and return it. (Future) If neither of above are specified, suffix the PVName to BasePath 

and return it

Also before returning the path, validate that path is safe 

and matches the filters specified in StorageClass.

func (*VolumeConfig) GetStorageType 

func (c *VolumeConfig) GetStorageType() string

GetStorageType returns the StorageType value configured in StorageClass. Default is hostpath

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.