framework

package
v1.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 14, 2023 License: Apache-2.0 Imports: 13 Imported by: 0

README

NodeResource Framework

Overview

The node resource controller is responsible for calculating the over-commit resources and updating the result on the node. The node resource framework defines some extension points for resource calculation and updating strategy. Each node resource plugin can implement one or more stages to reconcile the node resources.

The current extension stages provided by the node resource framework are as following:

  • Setup: It setups the plugin with options like controller client, scheme and event recorder.
type Plugin interface {
	Name() string
}

type SetupPlugin interface {
	Plugin
	Setup(opt *Option) error
}
  • Calculate: It calculates the node resources according to the Node and NodeMetric and generate a list of calculated node resource items. All node resource items will be merged into a NodeResource as the intermediate result of the Calculate stage. In case of the NodeMetric is abnormal, a plugin can implement the degraded calculation inside this stage. The Calculate plugin is also responsible for the reset of corresponding node resources when the colocation is configured as disabled.
type ResourceCalculatePlugin interface {
	Plugin
	Reset(node *Node, message string) []ResourceItem
	Calculate(strategy *ColocationStrategy, node *Node, podList *PodList, metrics *ResourceMetrics) ([]ResourceItem, error)
}

type ResourceItem struct {
	Name        ResourceName
	Quantity    *Quantity
	Labels      map[string]string
	Annotations map[string]string
	Message     string
	Reset       bool
}
  • Prepare: It prepares the Node object with the calculated result NodeResource. Before the Preparing, it is invoked after the Calculate so to allow the plugin to retry when the client updates conflicts.
type NodePreparePlugin interface {
	Plugin
	Execute(strategy *ColocationStrategy, node *Node, nr *NodeResource) error
}

type NodeResource struct {
	Resources   map[ResourceName]*Quantity
	Labels      map[string]string
	Annotations map[string]string
	Messages    map[ResourceName]string
	Resets      map[ResourceName]bool
}
  • NeedSync: It checks if the newly-prepared Node object should be synchronized to the kube-apiserver. To be more specific, there are two types of NeedSync plugins for different client update methods, where one can determine whether the node status should be updated and another determines whether node metadata should be updated.
type NodeSyncPlugin interface {
	Plugin
	NeedSync(strategy *ColocationStrategy, oldNode, newNode *Node) (bool, string)
}

type NodeMetaSyncPlugin interface {
	Plugin
	NeedSyncMeta(strategy *ColocationStrategy, oldNode, newNode *Node) (bool, string)
}

There is the workflow about how the node resource controller handles a dequeued Node object with plugins:

framework-img

Example: Batch Resource Plugin

The default BatchResource plugin is responsible for calculating and updating the Batch-tier resources. It implements the stages Calculate, Reset, Prepare and NeedSync:

Calculate:

For each node, the plugin summarizes the resource allocated and the usage of high-priority (HP, priority classes higher than Batch) pods, then derives the allocatable resources of the Batch-tier with the formula:

batchAllocatable := nodeAllocatable * thresholdPercent - podUsage(HP) - systemUsage

Besides, the plugin implements the Reset method to clean up the Batch resources when the node colocation is disabled.

Prepare:

The plugin sets the extended resources kubernetes.io/batch-cpu, kubernetes.io/batch-memory in the node.status.allocatable according to the calculated results from the Calculate or Reset stage.

NeedSync:

The plugin checks the extended resources kubernetes.io/batch-cpu, kubernetes.io/batch-memory of the prepared node and the old node. If the node's Batch resources have not been updated for too long or the calculated results changes too much, it will update the prepared Node object to the kube-apiserver.

What's More

The node resource framework is in Alpha. The defined stages may not be enough for some new scenarios. Please feel free to send Issues and PRs for improving the framework.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var AllPass = func(string) bool {
	return true
}

Functions

func RegisterNodeMetaSyncExtender added in v1.3.0 

func RegisterNodeMetaSyncExtender(filter FilterFn, plugins ...NodeMetaSyncPlugin)

func RegisterNodePrepareExtender 

func RegisterNodePrepareExtender(filter FilterFn, plugins ...NodePreparePlugin)

func RegisterNodeSyncExtender 

func RegisterNodeSyncExtender(filter FilterFn, plugins ...NodeSyncPlugin)

func RegisterResourceCalculateExtender 

func RegisterResourceCalculateExtender(filter FilterFn, plugins ...ResourceCalculatePlugin)

func RegisterSetupExtender added in v1.3.0 

func RegisterSetupExtender(filter FilterFn, plugins ...SetupPlugin)

func RunNodeMetaSyncExtenders added in v1.3.0 

func RunNodeMetaSyncExtenders(strategy *configuration.ColocationStrategy, oldNode, newNode *corev1.Node) bool

func RunNodePrepareExtenders 

func RunNodePrepareExtenders(strategy *configuration.ColocationStrategy, node *corev1.Node, nr *NodeResource)

func RunNodeSyncExtenders 

func RunNodeSyncExtenders(strategy *configuration.ColocationStrategy, oldNode, newNode *corev1.Node) bool

func RunResourceCalculateExtenders 

func RunResourceCalculateExtenders(nr *NodeResource, strategy *configuration.ColocationStrategy, node *corev1.Node,
	podList *corev1.PodList, resourceMetrics *ResourceMetrics)

func RunResourceResetExtenders 

func RunResourceResetExtenders(nr *NodeResource, node *corev1.Node, message string)

func RunSetupExtenders added in v1.3.0 

func RunSetupExtenders(opt *Option)

func UnregisterNodeMetaSyncExtender added in v1.3.0 

func UnregisterNodeMetaSyncExtender(name string)

func UnregisterNodePrepareExtender 

func UnregisterNodePrepareExtender(name string)

func UnregisterNodeSyncExtender 

func UnregisterNodeSyncExtender(name string)

func UnregisterResourceCalculateExtender 

func UnregisterResourceCalculateExtender(name string)

func UnregisterSetupExtender added in v1.3.0 

func UnregisterSetupExtender(name string)

Types

type FilterFn added in v1.3.0 

type FilterFn func(string) bool

type NodeMetaSyncPlugin added in v1.3.0 

type NodeMetaSyncPlugin interface {
	Plugin
	NeedSyncMeta(strategy *configuration.ColocationStrategy, oldNode, newNode *corev1.Node) (bool, string)
}

type NodePreparePlugin 

type NodePreparePlugin interface {
	Plugin
	Execute(strategy *configuration.ColocationStrategy, node *corev1.Node, nr *NodeResource) error
}

NodePreparePlugin implements node resource preparing for the calculated results. For example, assign extended resources in the node allocatable. It is invoked each time the controller tries updating the latest NodeResource object with calculated results.

type NodeResource 

type NodeResource struct {
	Resources   map[corev1.ResourceName]*resource.Quantity `json:"resources,omitempty"`
	Labels      map[string]string                          `json:"labels,omitempty"`
	Annotations map[string]string                          `json:"annotations,omitempty"`
	Messages    map[corev1.ResourceName]string             `json:"messages,omitempty"`
	Resets      map[corev1.ResourceName]bool               `json:"resets,omitempty"`
}

func NewNodeResource 

func NewNodeResource(items ...ResourceItem) *NodeResource

func (*NodeResource) Delete 

func (nr *NodeResource) Delete(items ...ResourceItem)

func (*NodeResource) Get 

func (nr *NodeResource) Get(name corev1.ResourceName) *resource.Quantity

func (*NodeResource) Set 

func (nr *NodeResource) Set(items ...ResourceItem)

func (*NodeResource) SetResourceList 

func (nr *NodeResource) SetResourceList(rl corev1.ResourceList, message string)

type NodeSyncPlugin 

type NodeSyncPlugin interface {
	Plugin
	NeedSync(strategy *configuration.ColocationStrategy, oldNode, newNode *corev1.Node) (bool, string)
}

NodeSyncPlugin implements the check of resource updating. For example, trigger an update if the values of the current is more than 10% different with the former.

type Option added in v1.3.0 

type Option struct {
	Client   client.Client
	Recorder record.EventRecorder
	Scheme   *runtime.Scheme
}

func NewOption added in v1.3.0 

func NewOption() *Option

func (*Option) WithClient added in v1.3.0 

func (o *Option) WithClient(c client.Client) *Option

func (*Option) WithManager added in v1.3.0 

func (o *Option) WithManager(mgr ctrl.Manager) *Option

func (*Option) WithRecorder added in v1.3.0 

func (o *Option) WithRecorder(r record.EventRecorder) *Option

func (*Option) WithScheme added in v1.3.0 

func (o *Option) WithScheme(s *runtime.Scheme) *Option

type Plugin 

type Plugin interface {
	Name() string
}

Plugin has its name. Plugins in a registry are executed in order of the registration.

type PluginConfig 

type PluginConfig struct {
	Plugin Plugin
}

type PluginRegistry 

type PluginRegistry struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

func NewRegistry 

func NewRegistry(registryName string, plugins ...Plugin) *PluginRegistry

func (*PluginRegistry) GetAll 

func (r *PluginRegistry) GetAll() []Plugin

func (*PluginRegistry) MustRegister 

func (r *PluginRegistry) MustRegister(plugins ...Plugin)

func (*PluginRegistry) Size 

func (r *PluginRegistry) Size() int

func (*PluginRegistry) Unregister 

func (r *PluginRegistry) Unregister(name string)

type ResourceCalculatePlugin 

type ResourceCalculatePlugin interface {
	ResourceResetPlugin
	Calculate(strategy *configuration.ColocationStrategy, node *corev1.Node, podList *corev1.PodList, metrics *ResourceMetrics) ([]ResourceItem, error)
}

ResourceCalculatePlugin implements resource counting and overcommitment algorithms. It implements Reset which can be invoked when the calculated resources need a reset. A ResourceCalculatePlugin can handle the case when the metrics are abnormal by implementing degraded calculation.

type ResourceItem 

type ResourceItem struct {
	Name        corev1.ResourceName `json:"name,omitempty"`
	Quantity    *resource.Quantity  `json:"quantity,omitempty"`
	Labels      map[string]string   `json:"labels,omitempty"`
	Annotations map[string]string   `json:"annotations,omitempty"`
	Message     string              `json:"message,omitempty"` // the message about the resource calculation
	Reset       bool                `json:"reset,omitempty"`   // whether to reset the resource or not
}

type ResourceMetrics 

type ResourceMetrics struct {
	NodeMetric *slov1alpha1.NodeMetric `json:"nodeMetric,omitempty"`
	// extended metrics
	Extensions *slov1alpha1.ExtensionsMap `json:"extensions,omitempty"`
}

type ResourceResetPlugin 

type ResourceResetPlugin interface {
	Plugin
	Reset(node *corev1.Node, message string) []ResourceItem
}

type SetupPlugin added in v1.3.0 

type SetupPlugin interface {
	Plugin
	Setup(opt *Option) error
}

SetupPlugin implements setup for the plugin. The Setup of each plugin will be called before other extension stages and invoked only once.

type SyncContext 

type SyncContext struct {
	// contains filtered or unexported fields
}

func NewSyncContext 

func NewSyncContext() *SyncContext

func (*SyncContext) Delete 

func (s *SyncContext) Delete(key string)

func (*SyncContext) Load 

func (s *SyncContext) Load(key string) (time.Time, bool)

func (*SyncContext) Store 

func (s *SyncContext) Store(key string, value time.Time)

func (*SyncContext) WithContext 

func (s *SyncContext) WithContext(m map[string]time.Time) *SyncContext

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.