model

package
v0.0.0-...-386b7fa Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 14, 2019 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// OOMBumpUpRatio specifies how much memory will be added after observing OOM.
	OOMBumpUpRatio float64 = 1.2
	// OOMMinBumpUp specifies minimal increase of memory after observing OOM.
	OOMMinBumpUp float64 = 100 * 1024 * 1024 // 100MB
)
View Source 
const (
	// RecommendationMissingMaxDuration is maximum time that we accept the recommendation can be missing.
	RecommendationMissingMaxDuration = 30 * time.Minute
)
View Source 
const (
	// SupportedCheckpointVersion is the tag of the supported version of serialized checkpoints.
	// Version id should be incremented on every non incompatible change, i.e. if the new
	// version of the recommender binary can't initialize from the old checkpoint format or the
	// previous version of the recommender binary can't initialize from the new checkpoint format.
	SupportedCheckpointVersion = "v3"
)

Variables

View Source 
var (
	// MemoryAggregationWindowLength is the length of the memory usage history
	// aggregated by VPA, which is 8 days.
	MemoryAggregationWindowLength = time.Hour * 8 * 24
	// MemoryAggregationInterval is the length of a single interval, for
	// which the peak memory usage is computed.
	// Memory usage peaks are aggregated in daily intervals. In other words
	// there is one memory usage sample per day (the maximum usage over that
	// day).
	// Note: AggregationWindowLength must be integrally divisible by this value.
	MemoryAggregationInterval = time.Hour * 24
	// CPUHistogramOptions are options to be used by histograms that store
	// CPU measures expressed in cores.
	CPUHistogramOptions = cpuHistogramOptions()
	// MemoryHistogramOptions are options to be used by histograms that
	// store memory measures expressed in bytes.
	MemoryHistogramOptions = memoryHistogramOptions()
	// HistogramBucketSizeGrowth defines the growth rate of the histogram buckets.
	// Each bucket is wider than the previous one by this fraction.
	HistogramBucketSizeGrowth = 0.05 // Make each bucket 5% larger than the previous one.
	// MemoryHistogramDecayHalfLife is the amount of time it takes a historical
	// memory usage sample to lose half of its weight. In other words, a fresh
	// usage sample is twice as 'important' as one with age equal to the half
	// life period.
	MemoryHistogramDecayHalfLife = time.Hour * 24
	// CPUHistogramDecayHalfLife is the amount of time it takes a historical
	// CPU usage sample to lose half of its weight.
	CPUHistogramDecayHalfLife = time.Hour * 24
)

Functions

func BytesFromMemoryAmount 

func BytesFromMemoryAmount(memoryAmount ResourceAmount) float64

BytesFromMemoryAmount converts ResourceAmount to number of bytes expressed as float64.

func CoresFromCPUAmount 

func CoresFromCPUAmount(cpuAmount ResourceAmount) float64

CoresFromCPUAmount converts ResourceAmount to number of cores expressed as float64.

func QuantityFromCPUAmount 

func QuantityFromCPUAmount(cpuAmount ResourceAmount) resource.Quantity

QuantityFromCPUAmount converts CPU ResourceAmount to a resource.Quantity.

func QuantityFromMemoryAmount 

func QuantityFromMemoryAmount(memoryAmount ResourceAmount) resource.Quantity

QuantityFromMemoryAmount converts memory ResourceAmount to a resource.Quantity.

func ResourcesAsResourceList 

func ResourcesAsResourceList(resources Resources) apiv1.ResourceList

ResourcesAsResourceList converts internal Resources representation to ResourcesList.

Types

type AggregateContainerState 

type AggregateContainerState struct {
	// AggregateCPUUsage is a distribution of all CPU samples.
	AggregateCPUUsage util.Histogram
	// AggregateMemoryPeaks is a distribution of memory peaks from all containers:
	// each container should add one peak per memory aggregation interval (e.g. once every 24h).
	AggregateMemoryPeaks util.Histogram
	// Note: first/last sample timestamps as well as the sample count are based only on CPU samples.
	FirstSampleStart  time.Time
	LastSampleStart   time.Time
	TotalSamplesCount int
	CreationTime      time.Time
}

AggregateContainerState holds input signals aggregated from a set of containers. It can be used as an input to compute the recommendation. The CPU and memory distributions use decaying histograms by default (see NewAggregateContainerState()). Implements ContainerStateAggregator interface.

func NewAggregateContainerState 

func NewAggregateContainerState() *AggregateContainerState

NewAggregateContainerState returns a new, empty AggregateContainerState.

func (*AggregateContainerState) AddSample 

func (a *AggregateContainerState) AddSample(sample *ContainerUsageSample)

AddSample aggregates a single usage sample.

func (*AggregateContainerState) LoadFromCheckpoint 

func (a *AggregateContainerState) LoadFromCheckpoint(checkpoint *vpa_types.VerticalPodAutoscalerCheckpointStatus) error

LoadFromCheckpoint deserializes data from VerticalPodAutoscalerCheckpointStatus into the AggregateContainerState.

func (*AggregateContainerState) MergeContainerState 

func (a *AggregateContainerState) MergeContainerState(other *AggregateContainerState)

MergeContainerState merges two AggregateContainerStates.

func (*AggregateContainerState) SaveToCheckpoint 

func (a *AggregateContainerState) SaveToCheckpoint() (*vpa_types.VerticalPodAutoscalerCheckpointStatus, error)

SaveToCheckpoint serializes AggregateContainerState as VerticalPodAutoscalerCheckpointStatus. The serialization may result in loss of precission of the histograms.

func (*AggregateContainerState) SubtractSample 

func (a *AggregateContainerState) SubtractSample(sample *ContainerUsageSample)

SubtractSample removes a single usage sample from an aggregation. The subtracted sample should be equal to some sample that was aggregated with AddSample() in the past. Only memory samples can be subtracted at the moment. Support for CPU could be added if necessary.

type AggregateStateKey 

type AggregateStateKey interface {
	Namespace() string
	ContainerName() string
	Labels() labels.Labels
}

AggregateStateKey determines the set of containers for which the usage samples are kept aggregated in the model.

type ClusterState 

type ClusterState struct {
	// Pods in the cluster.
	Pods map[PodID]*PodState
	// VPA objects in the cluster.
	Vpas map[VpaID]*Vpa
	// VPA objects in the cluster that have no recommendation mapped to the first
	// time we've noticed the recommendation missing or last time we logged
	// a warning about it.
	EmptyVPAs map[VpaID]time.Time
	// VpasWithMatchingPods contains information if there exist live pods that
	// this VPAs selector matches.
	VpasWithMatchingPods map[VpaID]bool
	// Observed VPAs. Used to check if there are updates needed.
	ObservedVpas []*vpa_types.VerticalPodAutoscaler
	// contains filtered or unexported fields
}

ClusterState holds all runtime information about the cluster required for the VPA operations, i.e. configuration of resources (pods, containers, VPA objects), aggregated utilization of compute resources (CPU, memory) and events (container OOMs). All input to the VPA Recommender algorithm lives in this structure.

func NewClusterState 

func NewClusterState() *ClusterState

NewClusterState returns a new ClusterState with no pods.

func (*ClusterState) AddOrUpdateContainer 

func (cluster *ClusterState) AddOrUpdateContainer(containerID ContainerID, request Resources) error

AddOrUpdateContainer creates a new container with the given ContainerID and adds it to the parent pod in the ClusterState object, if not yet present. Requires the pod to be added to the ClusterState first. Otherwise an error is returned.

func (*ClusterState) AddOrUpdatePod 

func (cluster *ClusterState) AddOrUpdatePod(podID PodID, newLabels labels.Set, phase apiv1.PodPhase)

AddOrUpdatePod updates the state of the pod with a given PodID, if it is present in the cluster object. Otherwise a new pod is created and added to the Cluster object. If the labels of the pod have changed, it updates the links between the containers and the aggregations.

func (*ClusterState) AddOrUpdateVpa 

func (cluster *ClusterState) AddOrUpdateVpa(apiObject *vpa_types.VerticalPodAutoscaler, selector labels.Selector) error

AddOrUpdateVpa adds a new VPA with a given ID to the ClusterState if it didn't yet exist. If the VPA already existed but had a different pod selector, the pod selector is updated. Updates the links between the VPA and all aggregations it matches.

func (*ClusterState) AddSample 

func (cluster *ClusterState) AddSample(sample *ContainerUsageSampleWithKey) error

AddSample adds a new usage sample to the proper container in the ClusterState object. Requires the container as well as the parent pod to be added to the ClusterState first. Otherwise an error is returned.

func (*ClusterState) DeletePod 

func (cluster *ClusterState) DeletePod(podID PodID)

DeletePod removes an existing pod from the cluster.

func (*ClusterState) DeleteVpa 

func (cluster *ClusterState) DeleteVpa(vpaID VpaID) error

DeleteVpa removes a VPA with the given ID from the ClusterState.

func (*ClusterState) GarbageCollectAggregateCollectionStates 

func (cluster *ClusterState) GarbageCollectAggregateCollectionStates(now time.Time)

GarbageCollectAggregateCollectionStates removes obsolete AggregateCollectionStates from the ClusterState. AggregateCollectionState is obsolete in following situations: 1) It has no samples and there are no more active pods that can contribute, 2) The last sample is too old to give meaningful recommendation (>8 days), 3) There are no samples and the aggregate state was created >8 days ago.

func (*ClusterState) GetContainer 

func (cluster *ClusterState) GetContainer(containerID ContainerID) *ContainerState

GetContainer returns the ContainerState object for a given ContainerID or null if it's not present in the model.

func (*ClusterState) GetMatchingPods 

func (cluster *ClusterState) GetMatchingPods(vpa *Vpa) []PodID

GetMatchingPods returns a list of currently active pods that match the given VPA. Traverses through all pods in the cluster - use sparingly.

func (*ClusterState) MakeAggregateStateKey 

func (cluster *ClusterState) MakeAggregateStateKey(pod *PodState, containerName string) AggregateStateKey

MakeAggregateStateKey returns the AggregateStateKey that should be used to aggregate usage samples from a container with the given name in a given pod.

func (*ClusterState) RecordOOM 

func (cluster *ClusterState) RecordOOM(containerID ContainerID, timestamp time.Time, requestedMemory ResourceAmount) error

RecordOOM adds info regarding OOM event in the model as an artificial memory sample.

func (*ClusterState) RecordRecommendation 

func (cluster *ClusterState) RecordRecommendation(vpa *Vpa, now time.Time) error

RecordRecommendation marks the state of recommendation in the cluster. We keep track of empty recommendations and log information about them periodically.

func (*ClusterState) StateMapSize 

func (cluster *ClusterState) StateMapSize() int

StateMapSize is the number of pods being tracked by the VPA

type ContainerID 

type ContainerID struct {
	PodID
	// ContainerName is the name of the container, unique within a pod.
	ContainerName string
}

ContainerID contains information needed to identify a Container within a cluster.

type ContainerNameToAggregateStateMap 

type ContainerNameToAggregateStateMap map[string]*AggregateContainerState

ContainerNameToAggregateStateMap maps a container name to AggregateContainerState that aggregates state of containers with that name.

func AggregateStateByContainerName 

func AggregateStateByContainerName(aggregateContainerStateMap aggregateContainerStatesMap) ContainerNameToAggregateStateMap

AggregateStateByContainerName takes a set of AggregateContainerStates and merge them grouping by the container name. The result is a map from the container name to the aggregation from all input containers with the given name.

type ContainerState 

type ContainerState struct {
	// Current request.
	Request Resources
	// Start of the latest CPU usage sample that was aggregated.
	LastCPUSampleStart time.Time

	// End time of the current memory aggregation interval (not inclusive).
	WindowEnd time.Time
	// contains filtered or unexported fields
}

ContainerState stores information about a single container instance. Each ContainerState has a pointer to the aggregation that is used for aggregating its usage samples. It holds the recent history of CPU and memory utilization. 

Note: samples are added to intervals based on their start timestamps.

func NewContainerState 

func NewContainerState(request Resources, aggregator ContainerStateAggregator) *ContainerState

NewContainerState returns a new ContainerState.

func (*ContainerState) AddSample 

func (container *ContainerState) AddSample(sample *ContainerUsageSample) bool

AddSample adds a usage sample to the given ContainerState. Requires samples for a single resource to be passed in chronological order (i.e. in order of growing MeasureStart). Invalid samples (out of order or measure out of legal range) are discarded. Returns true if the sample was aggregated, false if it was discarded. Note: usage samples don't hold their end timestamp / duration. They are implicitly assumed to be disjoint when aggregating.

func (*ContainerState) GetMaxMemoryPeak 

func (container *ContainerState) GetMaxMemoryPeak() ResourceAmount

GetMaxMemoryPeak returns maximum memory usage in the sample, possibly estimated from OOM

func (*ContainerState) RecordOOM 

func (container *ContainerState) RecordOOM(timestamp time.Time, requestedMemory ResourceAmount) error

RecordOOM adds info regarding OOM event in the model as an artificial memory sample.

type ContainerStateAggregator 

type ContainerStateAggregator interface {
	// AddSample aggregates a single usage sample.
	AddSample(sample *ContainerUsageSample)
	// SubtractSample removes a single usage sample. The subtracted sample
	// should be equal to some sample that was aggregated with AddSample()
	// in the past.
	SubtractSample(sample *ContainerUsageSample)
}

ContainerStateAggregator is an interface for objects that consume and aggregate container usage samples.

func NewContainerStateAggregatorProxy 

func NewContainerStateAggregatorProxy(cluster *ClusterState, containerID ContainerID) ContainerStateAggregator

NewContainerStateAggregatorProxy creates a ContainerStateAggregatorProxy pointing to the cluster state.

type ContainerStateAggregatorProxy 

type ContainerStateAggregatorProxy struct {
	// contains filtered or unexported fields
}

ContainerStateAggregatorProxy is a wrapper for ContainerStateAggregator that creates CnontainerStateAgregator for container if it is no longer present in the cluster state.

func (*ContainerStateAggregatorProxy) AddSample 

func (p *ContainerStateAggregatorProxy) AddSample(sample *ContainerUsageSample)

AddSample adds a container sample to the aggregator.

func (*ContainerStateAggregatorProxy) SubtractSample 

func (p *ContainerStateAggregatorProxy) SubtractSample(sample *ContainerUsageSample)

SubtractSample subtracts a container sample from the aggregator.

type ContainerUsageSample 

type ContainerUsageSample struct {
	// Start of the measurement interval.
	MeasureStart time.Time
	// Average CPU usage in cores or memory usage in bytes.
	Usage ResourceAmount
	// CPU or memory request at the time of measurment.
	Request ResourceAmount
	// Which resource is this sample for.
	Resource ResourceName
}

ContainerUsageSample is a measure of resource usage of a container over some interval.

type ContainerUsageSampleWithKey 

type ContainerUsageSampleWithKey struct {
	ContainerUsageSample
	Container ContainerID
}

ContainerUsageSampleWithKey holds a ContainerUsageSample together with the ID of the container it belongs to.

type KeyError 

type KeyError struct {
	// contains filtered or unexported fields
}

KeyError is returned when the mapping key was not found.

func NewKeyError 

func NewKeyError(key interface{}) KeyError

NewKeyError returns a new KeyError.

func (KeyError) Error 

func (e KeyError) Error() string

type PodID 

type PodID struct {
	// Namespaces where the Pod is defined.
	Namespace string
	// PodName is the name of the pod unique within a namespace.
	PodName string
}

PodID contains information needed to identify a Pod within a cluster.

type PodState 

type PodState struct {
	// Unique id of the Pod.
	ID PodID

	// Containers that belong to the Pod, keyed by the container name.
	Containers map[string]*ContainerState
	// PodPhase describing current life cycle phase of the Pod.
	Phase apiv1.PodPhase
	// contains filtered or unexported fields
}

PodState holds runtime information about a single Pod.

type ResourceAmount 

type ResourceAmount int64

ResourceAmount represents quantity of a certain resource within a container. Note this keeps CPU in millicores (which is not a standard unit in APIs) and memory in bytes. Allowed values are in the range from 0 to MaxResourceAmount.

func CPUAmountFromCores 

func CPUAmountFromCores(cores float64) ResourceAmount

CPUAmountFromCores converts CPU cores to a ResourceAmount.

func MemoryAmountFromBytes 

func MemoryAmountFromBytes(bytes float64) ResourceAmount

MemoryAmountFromBytes converts memory bytes to a ResourceAmount.

func ResourceAmountMax 

func ResourceAmountMax(amount1, amount2 ResourceAmount) ResourceAmount

ResourceAmountMax returns the larger of two resource amounts.

func RoundResourceAmount 

func RoundResourceAmount(amount, unit ResourceAmount) ResourceAmount

RoundResourceAmount returns the given resource amount rounded down to the whole multiple of another resource amount (unit).

func ScaleResource 

func ScaleResource(amount ResourceAmount, factor float64) ResourceAmount

ScaleResource returns the resource amount multiplied by a given factor.

type ResourceName 

type ResourceName string

ResourceName represents the name of the resource monitored by recommender. 

const (
	// ResourceCPU represents CPU in millicores (1core = 1000millicores).
	ResourceCPU ResourceName = "cpu"
	// ResourceMemory represents memory, in bytes. (500Gi = 500GiB = 500 * 1024 * 1024 * 1024).
	ResourceMemory ResourceName = "memory"
	// MaxResourceAmount is the maximum allowed value of resource amount.
	MaxResourceAmount = ResourceAmount(1e14)
)

type Resources 

type Resources map[ResourceName]ResourceAmount

Resources is a map from resource name to the corresponding ResourceAmount.

type Vpa 

type Vpa struct {
	ID VpaID
	// Labels selector that determines which Pods are controlled by this VPA
	// object. Can be nil, in which case no Pod is matched.
	PodSelector labels.Selector
	// Map of the status conditions (keys are condition types).
	Conditions vpaConditionsMap
	// Most recently computed recommendation. Can be nil.
	Recommendation *vpa_types.RecommendedPodResources

	// Pod Resource Policy provided in the VPA API object. Can be nil.
	ResourcePolicy *vpa_types.PodResourcePolicy
	// Initial checkpoints of AggregateContainerStates for containers.
	// The key is container name.
	ContainersInitialAggregateState ContainerNameToAggregateStateMap
	// UpdateMode describes how recommendations will be applied to pods
	UpdateMode *vpa_types.UpdateMode
	// Created denotes timestamp of the original VPA object creation
	Created time.Time
	// CheckpointWritten indicates when last checkpoint for the VPA object was stored.
	CheckpointWritten time.Time
	// IsV1Beta1API is set to true if VPA object has labelSelector defined as in v1beta1 api.
	IsV1Beta1API bool
	// TargetRef points to the controller managing the set of pods.
	TargetRef *autoscaling.CrossVersionObjectReference
	// contains filtered or unexported fields
}

Vpa (Vertical Pod Autoscaler) object is responsible for vertical scaling of Pods matching a given label selector.

func NewVpa 

func NewVpa(id VpaID, selector labels.Selector, created time.Time) *Vpa

NewVpa returns a new Vpa with a given ID and pod selector. Doesn't set the links to the matched aggregations.

func (*Vpa) AggregateStateByContainerName 

func (vpa *Vpa) AggregateStateByContainerName() ContainerNameToAggregateStateMap

AggregateStateByContainerName returns a map from container name to the aggregated state of all containers with that name, belonging to pods matched by the VPA.

func (*Vpa) AsStatus 

func (vpa *Vpa) AsStatus() *vpa_types.VerticalPodAutoscalerStatus

AsStatus returns this objects equivalent of VPA Status. UpdateConditions should be called first.

func (*Vpa) DeleteAggregation 

func (vpa *Vpa) DeleteAggregation(aggregationKey AggregateStateKey)

DeleteAggregation deletes aggregation used by this container

func (*Vpa) HasMatchedPods 

func (vpa *Vpa) HasMatchedPods() bool

HasMatchedPods returns true if there are are currently active pods in the cluster matching this VPA, based on conditions. UpdateConditions should be called first.

func (*Vpa) HasRecommendation 

func (vpa *Vpa) HasRecommendation() bool

HasRecommendation returns if the VPA object contains any recommendation

func (*Vpa) MergeCheckpointedState 

func (vpa *Vpa) MergeCheckpointedState(aggregateContainerStateMap ContainerNameToAggregateStateMap)

MergeCheckpointedState adds checkpointed VPA aggregations to the given aggregateStateMap.

func (*Vpa) UpdateConditions 

func (vpa *Vpa) UpdateConditions(podsMatched bool)

UpdateConditions updates the conditions of VPA objects based on it's state. PodsMatched is passed to indicate if there are currently active pods in the cluster matching this VPA.

func (*Vpa) UseAggregationIfMatching 

func (vpa *Vpa) UseAggregationIfMatching(aggregationKey AggregateStateKey, aggregation *AggregateContainerState) bool

UseAggregationIfMatching checks if the given aggregation matches (contributes to) this VPA and adds it to the set of VPA's aggregations if that is the case. Returns true if the aggregation matches VPA.

func (*Vpa) UsesAggregation 

func (vpa *Vpa) UsesAggregation(aggregationKey AggregateStateKey) bool

UsesAggregation returns true iff an aggregation with the given key contributes to the VPA.

type VpaID 

type VpaID struct {
	Namespace string
	VpaName   string
}

VpaID contains information needed to identify a VPA API object within a cluster.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.