Documentation
¶
Overview ¶
Package vnode implements the Virtual Kubelet integration: one static virtual Node per provider, whose PodLifecycleHandler provisions/terminates external instances through the provider seam. See docs/architecture.md §3.
Ownership model ("VK owns provisioning"): the pod controller's CreatePod calls provider.Provision and DeletePod calls provider.Terminate directly. The Pod is the single source of truth for the workload; the only provisioning input that is not on the Pod — the optimizer's capacity tier — rides on the CapacityTypeAnnotation, written by the placement controller when it ungates the Pod. Instance identity is derived deterministically from the Pod (ClaimName), so a provider whose List reports the claim tag can recover and reclaim an instance across a controller restart without a durable ledger.
Index ¶
- Constants
- func ConnectSecretName(podName string) string
- func NodeName(providerName string) string
- func PodIPFromEnv() string
- type Blocklister
- type Handler
- func (h *Handler) AttachToContainer(context.Context, string, string, string, vkapi.AttachIO) error
- func (h *Handler) CreatePod(ctx context.Context, pod *corev1.Pod) error
- func (h *Handler) DeletePod(ctx context.Context, pod *corev1.Pod) error
- func (h *Handler) GetContainerLogs(ctx context.Context, namespace, podName, containerName string, ...) (io.ReadCloser, error)
- func (h *Handler) GetMetricsResource(context.Context) ([]*dto.MetricFamily, error)
- func (h *Handler) GetPod(ctx context.Context, namespace, name string) (*corev1.Pod, error)
- func (h *Handler) GetPodStatus(_ context.Context, namespace, name string) (*corev1.PodStatus, error)
- func (h *Handler) GetPods(_ context.Context) ([]*corev1.Pod, error)
- func (h *Handler) GetStatsSummary(context.Context) (*statsv1alpha1.Summary, error)
- func (h *Handler) NotifyPods(ctx context.Context, cb func(*corev1.Pod))
- func (h *Handler) PortForward(context.Context, string, string, int32, io.ReadWriteCloser) error
- func (h *Handler) RunInContainer(ctx context.Context, namespace, podName, containerName string, cmd []string, ...) error
- func (h *Handler) Tracks(namespace, podName string) bool
- func (h *Handler) UpdatePod(_ context.Context, pod *corev1.Pod) error
- type KubeletServer
- type Runner
Constants ¶
const DefaultKubeletAddr = ":10250"
DefaultKubeletAddr is where the kubelet API listens. Nothing requires 10250 (the API server dials whatever the Node advertises), but matching a real kubelet keeps existing NetworkPolicies and operator intuition intact.
Variables ¶
This section is empty.
Functions ¶
func ConnectSecretName ¶
ConnectSecretName is the Secret holding a Pod's connect credential.
func NodeName ¶
NodeName returns the virtual node name for a provider: "nebula-<provider>". One static node per provider (see docs/architecture.md §3); the scheduler routes an ungated Pod to it via the ProviderLabel nodeSelector.
func PodIPFromEnv ¶
func PodIPFromEnv() string
PodIPFromEnv reads the manager's own Pod IP, projected by config/manager via a fieldRef. Empty means no endpoint can be advertised, so the caller degrades to logs-unsupported rather than publishing an unreachable address.
Types ¶
type Blocklister ¶
type Blocklister interface {
Record(prov string, scope provider.BlockScope, ttl time.Duration)
}
Blocklister records a failed placement so the placement controller fails over to the next candidate instead of hot-looping on a provider that just said no. The write half of pkg/failover.Blocklist; a nil blocklist is a no-op.
type Handler ¶
type Handler struct {
// contains filtered or unexported fields
}
Handler bridges one provider into the virtual kubelet: CreatePod provisions an external instance, DeletePod terminates it. This is the "VK owns provisioning" model — no separate controller issues Provision/Terminate.
Leak-safety: CreatePod records the instance id before returning success, so a paid instance is always reachable for teardown, and Provision is idempotent on ClaimName, so a retry after a crash adopts the existing instance instead of creating a second.
func NewHandler ¶
func NewHandler(prov provider.Provider, client kubernetes.Interface, blocklist Blocklister) *Handler
NewHandler builds a Handler for one provider backend. The poll cadence comes from Capabilities.PollInterval, falling back to defaultPollInterval. blocklist (failover recording) and client (the endpoint patch) may both be nil.
func (*Handler) AttachToContainer ¶
func (*Handler) CreatePod ¶
CreatePod provisions an external instance for the Pod through the provider. The Pod carries the whole workload shape; the only out-of-band input, the optimizer's capacity tier, rides on CapacityTypeAnnotation.
func (*Handler) DeletePod ¶
DeletePod terminates the external instance and drops the pod from tracking. Terminate is idempotent, so a repeated DeletePod (VK may call it more than once) is safe.
func (*Handler) GetContainerLogs ¶
func (h *Handler) GetContainerLogs( ctx context.Context, namespace, podName, containerName string, opts vkapi.ContainerLogOpts, ) (io.ReadCloser, error)
GetContainerLogs serves `kubectl logs` for a Pod on this virtual node (kubelet.go carries the endpoint). Three things must line up, each failing as NotFound: the Pod is TRACKED here, it has an instance id (one still inside Provision has nothing to read), and the provider implements provider.LogStreamer (logs are optional).
containerName is IGNORED: a Nebula Pod is one external instance with one console, so there is no per-container stream to select.
func (*Handler) GetMetricsResource ¶
func (*Handler) GetPod ¶
GetPod returns the tracked pod, or a NotFound error the pod controller understands.
Tracking is in-memory, so a restart (redeploy, crash, leader handoff) starts with an empty map while the instances are still running. VK's createOrUpdatePod treats a nil result as "create", which would re-drive provisioning. So on a cold map we RE-ADOPT: ask the provider whether an instance with this claim is live, and rebuild the entry from it. VK then takes the adopt branch and the next poll tick advances the pod.
The three outcomes stay DISTINCT, because "no such instance" and "could not ask" have opposite consequences. Reporting the latter as NotFound is what let one failed List destroy a healthy workload: VK discards this error and branches on nil-ness alone, so a nil pod re-issues CreatePod against a running instance — and with the provider still unreachable that Provision fails too, marking the Pod Failed for a reap while the real instance keeps billing behind a zero id. So an unreachable provider returns a non-nil pod WITH an error: the pod suppresses the create, and the non-NotFound error makes the delete path requeue instead of terminating. Both wait for the next sync, the only correct move while ownership is unknown.
func (*Handler) GetPodStatus ¶
func (h *Handler) GetPodStatus(_ context.Context, namespace, name string) (*corev1.PodStatus, error)
GetPodStatus returns the tracked pod's status.
func (*Handler) GetStatsSummary ¶
func (*Handler) NotifyPods ¶
NotifyPods registers the async status callback and starts the poll loop. VK calls it once at startup; the loop runs until ctx is cancelled.
We WRAP VK's callback: its status path writes only the /status subresource and silently drops metadata changes on the same object, but the endpoint has to live on metadata (PodIP cannot hold a DNS name — see applyState) and so does the instance id. So the wrapper writes that metadata first, then hands the same Pod to VK. Every status push therefore also reconciles how the workload is reached and which instance backs it.
func (*Handler) PortForward ¶
func (*Handler) RunInContainer ¶
func (h *Handler) RunInContainer( ctx context.Context, namespace, podName, containerName string, cmd []string, attach vkapi.AttachIO, ) error
RunInContainer serves `kubectl exec` for a Pod on this virtual node: it starts cmd in the external instance and hands the streams to runExec, which owns the pumping.
Same three preconditions as logs, each a NotFound: the Pod is tracked here, it has an instance, and the provider implements provider.Executor (exec is optional — a backend with no way into the box simply cannot serve it).
containerName is IGNORED, as for logs: a Nebula Pod is one external instance, so there is no second container to pick.
type KubeletServer ¶
type KubeletServer struct {
// contains filtered or unexported fields
}
KubeletServer serves the kubelet API routes Nebula implements — container logs and exec — for every provider's virtual node.
Why it exists: `kubectl logs` and `kubectl exec` are not control-plane reads. The API server proxies them to the kubelet of the Pod's node, at that Node's addresses and daemonEndpoints. A virtual node has no kubelet, so without a listener both fail whatever the provider can serve.
One listener for all nodes: the routes carry only namespace/pod/container, so a request is resolved by asking each registered Handler whether it tracks that Pod — at most one can. Cheaper than a port per provider, and than reading the Pod to learn its node.
TLS uses a self-signed in-memory cert, which is what the API server expects: it does not verify a kubelet's serving cert unless --kubelet-certificate-authority is set. The webhook cert rotator cannot help, since it mints for a Service DNS name and this endpoint is dialed by Pod IP.
Client certs are verified only when ClientCAPath is set. Off by default because which CA signs the API server's kubelet client cert is not portable (kubeadm uses the cluster CA, EKS/GKE their own), so requiring it would break `kubectl logs` on managed control planes. The cost is now larger than logs: anything that can reach this port can also RUN COMMANDS in these Pods' instances, without passing through the API server's RBAC. Set the CA if you can name it, else close the port with a NetworkPolicy.
func NewKubeletServer ¶
func NewKubeletServer(nodeIP, addr, clientCAPath string) (*KubeletServer, error)
NewKubeletServer builds the shared kubelet endpoint. nodeIP is the manager's Pod IP, addr defaults to DefaultKubeletAddr, clientCAPath may be empty.
A bad address or IP is an error here, so a misconfiguration fails at startup instead of producing nodes that advertise an endpoint nothing serves.
func (*KubeletServer) Register ¶
func (s *KubeletServer) Register(nodeName string, h *Handler)
Register wires one provider's Handler in. Called by Runner before it starts; the node name is only a key, since lookup is by Pod.
type Runner ¶
type Runner struct {
// contains filtered or unexported fields
}
Runner is a manager.Runnable owning one provider's virtual node. It starts the VK node/pod controllers and blocks until the manager's context is cancelled, so it follows the manager's lifecycle and leader election like any other runnable.
It wires virtual-kubelet's lower-level `node` package directly rather than the `nodeutil` wrapper, which pulls in a kubelet HTTP/auth stack whose apiserver dependency does not compile against the k8s 0.33 line we pin. The one kubelet route we serve (logs) is hosted by KubeletServer, which needs none of it.
func NewRunner ¶
func NewRunner( prov provider.Provider, client kubernetes.Interface, blocklist Blocklister, kubelet *KubeletServer, ) *Runner
NewRunner builds the virtual-node runner for one provider. blocklist (Provision failures) and kubelet (the log endpoint) are both shared, and both may be nil.
Source Files