v1.3.0 Latest Latest

This package is not in the latest version of its module.

Go to latest
Published: Aug 14, 2023 License: Apache-2.0 Imports: 13 Imported by: 0


NodeResource Framework


The node resource controller is responsible for calculating the over-commit resources and updating the result on the node. The node resource framework defines some extension points for resource calculation and updating strategy. Each node resource plugin can implement one or more stages to reconcile the node resources.

The current extension stages provided by the node resource framework are as following:

  • Setup: It setups the plugin with options like controller client, scheme and event recorder.
type Plugin interface {
	Name() string

type SetupPlugin interface {
	Setup(opt *Option) error
  • Calculate: It calculates the node resources according to the Node and NodeMetric and generate a list of calculated node resource items. All node resource items will be merged into a NodeResource as the intermediate result of the Calculate stage. In case of the NodeMetric is abnormal, a plugin can implement the degraded calculation inside this stage. The Calculate plugin is also responsible for the reset of corresponding node resources when the colocation is configured as disabled.
type ResourceCalculatePlugin interface {
	Reset(node *Node, message string) []ResourceItem
	Calculate(strategy *ColocationStrategy, node *Node, podList *PodList, metrics *ResourceMetrics) ([]ResourceItem, error)

type ResourceItem struct {
	Name        ResourceName
	Quantity    *Quantity
	Labels      map[string]string
	Annotations map[string]string
	Message     string
	Reset       bool
  • Prepare: It prepares the Node object with the calculated result NodeResource. Before the Preparing, it is invoked after the Calculate so to allow the plugin to retry when the client updates conflicts.
type NodePreparePlugin interface {
	Execute(strategy *ColocationStrategy, node *Node, nr *NodeResource) error

type NodeResource struct {
	Resources   map[ResourceName]*Quantity
	Labels      map[string]string
	Annotations map[string]string
	Messages    map[ResourceName]string
	Resets      map[ResourceName]bool
  • NeedSync: It checks if the newly-prepared Node object should be synchronized to the kube-apiserver. To be more specific, there are two types of NeedSync plugins for different client update methods, where one can determine whether the node status should be updated and another determines whether node metadata should be updated.
type NodeSyncPlugin interface {
	NeedSync(strategy *ColocationStrategy, oldNode, newNode *Node) (bool, string)

type NodeMetaSyncPlugin interface {
	NeedSyncMeta(strategy *ColocationStrategy, oldNode, newNode *Node) (bool, string)

There is the workflow about how the node resource controller handles a dequeued Node object with plugins:


Example: Batch Resource Plugin

The default BatchResource plugin is responsible for calculating and updating the Batch-tier resources. It implements the stages Calculate, Reset, Prepare and NeedSync:


For each node, the plugin summarizes the resource allocated and the usage of high-priority (HP, priority classes higher than Batch) pods, then derives the allocatable resources of the Batch-tier with the formula:

batchAllocatable := nodeAllocatable * thresholdPercent - podUsage(HP) - systemUsage

Besides, the plugin implements the Reset method to clean up the Batch resources when the node colocation is disabled.


The plugin sets the extended resources, in the node.status.allocatable according to the calculated results from the Calculate or Reset stage.


The plugin checks the extended resources, of the prepared node and the old node. If the node's Batch resources have not been updated for too long or the calculated results changes too much, it will update the prepared Node object to the kube-apiserver.

What's More

The node resource framework is in Alpha. The defined stages may not be enough for some new scenarios. Please feel free to send Issues and PRs for improving the framework.




This section is empty.


View Source
var AllPass = func(string) bool {
	return true


func RegisterNodeMetaSyncExtender added in v1.3.0

func RegisterNodeMetaSyncExtender(filter FilterFn, plugins ...NodeMetaSyncPlugin)

func RegisterNodePrepareExtender

func RegisterNodePrepareExtender(filter FilterFn, plugins ...NodePreparePlugin)

func RegisterNodeSyncExtender

func RegisterNodeSyncExtender(filter FilterFn, plugins ...NodeSyncPlugin)

func RegisterResourceCalculateExtender

func RegisterResourceCalculateExtender(filter FilterFn, plugins ...ResourceCalculatePlugin)

func RegisterSetupExtender added in v1.3.0

func RegisterSetupExtender(filter FilterFn, plugins ...SetupPlugin)

func RunNodeMetaSyncExtenders added in v1.3.0

func RunNodeMetaSyncExtenders(strategy *configuration.ColocationStrategy, oldNode, newNode *corev1.Node) bool

func RunNodePrepareExtenders

func RunNodePrepareExtenders(strategy *configuration.ColocationStrategy, node *corev1.Node, nr *NodeResource)

func RunNodeSyncExtenders

func RunNodeSyncExtenders(strategy *configuration.ColocationStrategy, oldNode, newNode *corev1.Node) bool

func RunResourceCalculateExtenders

func RunResourceCalculateExtenders(nr *NodeResource, strategy *configuration.ColocationStrategy, node *corev1.Node,
	podList *corev1.PodList, resourceMetrics *ResourceMetrics)

func RunResourceResetExtenders

func RunResourceResetExtenders(nr *NodeResource, node *corev1.Node, message string)

func RunSetupExtenders added in v1.3.0

func RunSetupExtenders(opt *Option)

func UnregisterNodeMetaSyncExtender added in v1.3.0

func UnregisterNodeMetaSyncExtender(name string)

func UnregisterNodePrepareExtender

func UnregisterNodePrepareExtender(name string)

func UnregisterNodeSyncExtender

func UnregisterNodeSyncExtender(name string)

func UnregisterResourceCalculateExtender

func UnregisterResourceCalculateExtender(name string)

func UnregisterSetupExtender added in v1.3.0

func UnregisterSetupExtender(name string)


type FilterFn added in v1.3.0

type FilterFn func(string) bool

type NodeMetaSyncPlugin added in v1.3.0

type NodeMetaSyncPlugin interface {
	NeedSyncMeta(strategy *configuration.ColocationStrategy, oldNode, newNode *corev1.Node) (bool, string)

type NodePreparePlugin

type NodePreparePlugin interface {
	Execute(strategy *configuration.ColocationStrategy, node *corev1.Node, nr *NodeResource) error

NodePreparePlugin implements node resource preparing for the calculated results. For example, assign extended resources in the node allocatable. It is invoked each time the controller tries updating the latest NodeResource object with calculated results.

type NodeResource

type NodeResource struct {
	Resources   map[corev1.ResourceName]*resource.Quantity `json:"resources,omitempty"`
	Labels      map[string]string                          `json:"labels,omitempty"`
	Annotations map[string]string                          `json:"annotations,omitempty"`
	Messages    map[corev1.ResourceName]string             `json:"messages,omitempty"`
	Resets      map[corev1.ResourceName]bool               `json:"resets,omitempty"`

func NewNodeResource

func NewNodeResource(items ...ResourceItem) *NodeResource

func (*NodeResource) Delete

func (nr *NodeResource) Delete(items ...ResourceItem)

func (*NodeResource) Get

func (*NodeResource) Set

func (nr *NodeResource) Set(items ...ResourceItem)

func (*NodeResource) SetResourceList

func (nr *NodeResource) SetResourceList(rl corev1.ResourceList, message string)

type NodeSyncPlugin

type NodeSyncPlugin interface {
	NeedSync(strategy *configuration.ColocationStrategy, oldNode, newNode *corev1.Node) (bool, string)

NodeSyncPlugin implements the check of resource updating. For example, trigger an update if the values of the current is more than 10% different with the former.

type Option added in v1.3.0

type Option struct {
	Client   client.Client
	Recorder record.EventRecorder
	Scheme   *runtime.Scheme

func NewOption added in v1.3.0

func NewOption() *Option

func (*Option) WithClient added in v1.3.0

func (o *Option) WithClient(c client.Client) *Option

func (*Option) WithManager added in v1.3.0

func (o *Option) WithManager(mgr ctrl.Manager) *Option

func (*Option) WithRecorder added in v1.3.0

func (o *Option) WithRecorder(r record.EventRecorder) *Option

func (*Option) WithScheme added in v1.3.0

func (o *Option) WithScheme(s *runtime.Scheme) *Option

type Plugin

type Plugin interface {
	Name() string

Plugin has its name. Plugins in a registry are executed in order of the registration.

type PluginConfig

type PluginConfig struct {
	Plugin Plugin

type PluginRegistry

type PluginRegistry struct {
	// contains filtered or unexported fields

func NewRegistry

func NewRegistry(registryName string, plugins ...Plugin) *PluginRegistry

func (*PluginRegistry) GetAll

func (r *PluginRegistry) GetAll() []Plugin

func (*PluginRegistry) MustRegister

func (r *PluginRegistry) MustRegister(plugins ...Plugin)

func (*PluginRegistry) Size

func (r *PluginRegistry) Size() int

func (*PluginRegistry) Unregister

func (r *PluginRegistry) Unregister(name string)

type ResourceCalculatePlugin

type ResourceCalculatePlugin interface {
	Calculate(strategy *configuration.ColocationStrategy, node *corev1.Node, podList *corev1.PodList, metrics *ResourceMetrics) ([]ResourceItem, error)

ResourceCalculatePlugin implements resource counting and overcommitment algorithms. It implements Reset which can be invoked when the calculated resources need a reset. A ResourceCalculatePlugin can handle the case when the metrics are abnormal by implementing degraded calculation.

type ResourceItem

type ResourceItem struct {
	Name        corev1.ResourceName `json:"name,omitempty"`
	Quantity    *resource.Quantity  `json:"quantity,omitempty"`
	Labels      map[string]string   `json:"labels,omitempty"`
	Annotations map[string]string   `json:"annotations,omitempty"`
	Message     string              `json:"message,omitempty"` // the message about the resource calculation
	Reset       bool                `json:"reset,omitempty"`   // whether to reset the resource or not

type ResourceMetrics

type ResourceMetrics struct {
	NodeMetric *slov1alpha1.NodeMetric `json:"nodeMetric,omitempty"`
	// extended metrics
	Extensions *slov1alpha1.ExtensionsMap `json:"extensions,omitempty"`

type ResourceResetPlugin

type ResourceResetPlugin interface {
	Reset(node *corev1.Node, message string) []ResourceItem

type SetupPlugin added in v1.3.0

type SetupPlugin interface {
	Setup(opt *Option) error

SetupPlugin implements setup for the plugin. The Setup of each plugin will be called before other extension stages and invoked only once.

type SyncContext

type SyncContext struct {
	// contains filtered or unexported fields

func NewSyncContext

func NewSyncContext() *SyncContext

func (*SyncContext) Delete

func (s *SyncContext) Delete(key string)

func (*SyncContext) Load

func (s *SyncContext) Load(key string) (time.Time, bool)

func (*SyncContext) Store

func (s *SyncContext) Store(key string, value time.Time)

func (*SyncContext) WithContext

func (s *SyncContext) WithContext(m map[string]time.Time) *SyncContext

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL