kubernetes

README

kubernetes

Name

kubernetes - enables reading zone data from a Kubernetes cluster.

Description

This plugin implements the Kubernetes DNS-Based Service Discovery Specification.

CoreDNS running the kubernetes plugin can be used as a replacement for kube-dns in a kubernetes cluster. See the deployment repository for details on how to deploy CoreDNS in Kubernetes.

stubDomains and upstreamNameservers are implemented via the forward plugin. See the examples below.

This plugin can only be used once per Server Block.

Syntax

kubernetes [ZONES...]

With only the plugin specified, the kubernetes plugin will default to the zone specified in the server's block. It will handle all queries in that zone and connect to Kubernetes in-cluster. It will not provide PTR records for services or A records for pods. If ZONES is used it specifies all the zones the plugin should be authoritative for.

kubernetes [ZONES...] {
    endpoint URL
    tls CERT KEY CACERT
    kubeconfig KUBECONFIG CONTEXT
    namespaces NAMESPACE...
    labels EXPRESSION
    pods POD-MODE
    endpoint_pod_names
    ttl TTL
    noendpoints
    transfer to ADDRESS...
    fallthrough [ZONES...]
    ignore empty_service
}

• endpoint specifies the URL for a remote k8s API endpoint. If omitted, it will connect to k8s in-cluster using the cluster service account.

• tls CERT KEY CACERT are the TLS cert, key and the CA cert file names for remote k8s connection. This option is ignored if connecting in-cluster (i.e. endpoint is not specified).

• kubeconfig KUBECONFIG CONTEXT authenticates the connection to a remote k8s cluster using a kubeconfig file. It supports TLS, username and password, or token-based authentication. This option is ignored if connecting in-cluster (i.e., the endpoint is not specified).

• namespaces NAMESPACE [NAMESPACE...] only exposes the k8s namespaces listed. If this option is omitted all namespaces are exposed

• namespace_labels EXPRESSION only expose the records for Kubernetes namespaces that match this label selector. The label selector syntax is described in the Kubernetes User Guide - Labels. An example that only exposes namespaces labeled as "istio-injection=enabled", would use: labels istio-injection=enabled.

• labels EXPRESSION only exposes the records for Kubernetes objects that match this label selector. The label selector syntax is described in the Kubernetes User Guide - Labels. An example that only exposes objects labeled as "application=nginx" in the "staging" or "qa" environments, would use: labels environment in (staging, qa),application=nginx.

• pods POD-MODE sets the mode for handling IP-based pod A records, e.g. 1-2-3-4.ns.pod.cluster.local. in A 1.2.3.4. This option is provided to facilitate use of SSL certs when connecting directly to pods. Valid values for POD-MODE:

  • disabled: Default. Do not process pod requests, always returning NXDOMAIN
  • insecure: Always return an A record with IP from request (without checking k8s). This option is vulnerable to abuse if used maliciously in conjunction with wildcard SSL certs. This option is provided for backward compatibility with kube-dns.
  • verified: Return an A record if there exists a pod in same namespace with matching IP. This option requires substantially more memory than in insecure mode, since it will maintain a watch on all pods.

• endpoint_pod_names uses the pod name of the pod targeted by the endpoint as the endpoint name in A records, e.g., endpoint-name.my-service.namespace.svc.cluster.local. in A 1.2.3.4 By default, the endpoint-name name selection is as follows: Use the hostname of the endpoint, or if hostname is not set, use the dashed form of the endpoint IP address (e.g., 1-2-3-4.my-service.namespace.svc.cluster.local.) If this directive is included, then name selection for endpoints changes as follows: Use the hostname of the endpoint, or if hostname is not set, use the pod name of the pod targeted by the endpoint. If there is no pod targeted by the endpoint, use the dashed IP address form.

• ttl allows you to set a custom TTL for responses. The default is 5 seconds. The minimum TTL allowed is 0 seconds, and the maximum is capped at 3600 seconds. Setting TTL to 0 will prevent records from being cached.

• noendpoints will turn off the serving of endpoint records by disabling the watch on endpoints. All endpoint queries and headless service queries will result in an NXDOMAIN.

• transfer enables zone transfers. It may be specified multiples times. To signals the direction (only to is allowed). ADDRESS must be denoted in CIDR notation (127.0.0.1/32 etc.) or just as plain addresses. The special wildcard * means: the entire internet. Sending DNS notifies is not supported. Deprecated pod records in the subdomain pod.cluster.local are not transferred.

• fallthrough [ZONES...] If a query for a record in the zones for which the plugin is authoritative results in NXDOMAIN, normally that is what the response will be. However, if you specify this option, the query will instead be passed on down the plugin chain, which can include another plugin to handle the query. If [ZONES...] is omitted, then fallthrough happens for all zones for which the plugin is authoritative. If specific zones are listed (for example in-addr.arpa and ip6.arpa), then only queries for those zones will be subject to fallthrough.

• ignore empty_service returns NXDOMAIN for services without any ready endpoint addresses (e.g., ready pods). This allows the querying pod to continue searching for the service in the search path. The search path could, for example, include another Kubernetes cluster.

Ready

This plugin reports readiness to the ready plugin. This will happen after it has synced to the Kubernetes API.

Examples

Handle all queries in the cluster.local zone. Connect to Kubernetes in-cluster. Also handle all in-addr.arpa PTR requests for 10.0.0.0/17 . Verify the existence of pods when answering pod requests.

10.0.0.0/17 cluster.local {
    kubernetes {
        pods verified
    }
}

Or you can selectively expose some namespaces:

kubernetes cluster.local {
    namespaces test staging
}

Connect to Kubernetes with CoreDNS running outside the cluster:

kubernetes cluster.local {
    endpoint https://k8s-endpoint:8443
    tls cert key cacert
}

stubDomains and upstreamNameservers

Here we use the forward plugin to implement a stubDomain that forwards example.local to the nameserver 10.100.0.10:53. Also configured is an upstreamNameserver 8.8.8.8:53 that will be used for resolving names that do not fall in cluster.local or example.local.

cluster.local:53 {
    kubernetes cluster.local
}
example.local {
    forward . 10.100.0.10:53
}

. {
    forward . 8.8.8.8:53
}

The configuration above represents the following Kube-DNS stubDomains and upstreamNameservers configuration.

stubDomains: |
   {“example.local”: [“10.100.0.10:53”]}
upstreamNameservers: |
   [“8.8.8.8:53”]

AutoPath

The kubernetes plugin can be used in conjunction with the autopath plugin. Using this feature enables server-side domain search path completion in Kubernetes clusters. Note: pods must be set to verified for this to function properly. Furthermore, the remote IP address in the DNS packet received by CoreDNS must be the IP address of the Pod that sent the request.

cluster.local {
    autopath @kubernetes
    kubernetes {
        pods verified
    }
}

Wildcards

Some query labels accept a wildcard value to match any value. If a label is a valid wildcard (*, or the word "any"), then that label will match all values. The labels that accept wildcards are:

• endpoint in an A record request: endpoint.service.namespace.svc.zone, e.g., *.nginx.ns.svc.cluster.local
• service in an A record request: service.namespace.svc.zone, e.g., *.ns.svc.cluster.local
• namespace in an A record request: service.namespace.svc.zone, e.g., nginx.*.svc.cluster.local
• port and/or protocol in an SRV request: _port._protocol.service.namespace.svc.zone., e.g., _http.*.service.ns.svc.cluster.local
• multiple wildcards are allowed in a single query, e.g., A Request *.*.svc.zone. or SRV request *.*.*.*.svc.zone.

For example, wildcards can be used to resolve all Endpoints for a Service as A records. e.g.: *.service.ns.svc.myzone.local will return the Endpoint IPs in the Service service in namespace default:

*.service.default.svc.cluster.local. 5	IN A	192.168.10.10
*.service.default.svc.cluster.local. 5	IN A	192.168.25.15

Metadata

The kubernetes plugin will publish the following metadata, if the metadata plugin is also enabled:

• kubernetes/endpoint: the endpoint name in the query
• kubernetes/kind: the resource kind (pod or svc) in the query
• kubernetes/namespace: the namespace in the query
• kubernetes/port-name: the port name in an SRV query
• kubernetes/protocol: the protocol in an SRV query
• kubernetes/service: the service name in the query
• kubernetes/client-namespace: the client pod's namespace (see requirements below)
• kubernetes/client-pod-name: the client pod's name (see requirements below)

The kubernetes/client-namespace and kubernetes/client-pod-name metadata work by reconciling the client IP address in the DNS request packet to a known pod IP address. Therefore the following is required:

• pods verified mode must be enabled
• the remote IP address in the DNS packet received by CoreDNS must be the IP address of the Pod that sent the request.

Metrics

If monitoring is enabled (via the prometheus plugin) then the following metrics are exported:

• coredns_kubernetes_dns_programming_duration_seconds{service_kind} - Exports the DNS programming latency SLI. The metrics has the service_kind label that identifies the kind of the kubernetes service. It may take one of the three values:
  • cluster_ip
  • headless_with_selector
  • headless_without_selector

Bugs

The duration metric only supports the "headless_with_selector" service currently.

Documentation

Overview

Package kubernetes provides the kubernetes backend.

Index

• Constants
• Variables
• type Kubernetes
  • func New(zones []string) *Kubernetes
  • func ParseStanza(c *caddy.Controller) (*Kubernetes, error)
  • func (k *Kubernetes) AutoPath(state request.Request) []string
  • func (k *Kubernetes) External(state request.Request) ([]msg.Service, int)
  • func (k *Kubernetes) ExternalAddress(state request.Request) []dns.RR
  • func (k *Kubernetes) InitKubeCache(ctx context.Context) (err error)
  • func (k *Kubernetes) IsNameError(err error) bool
  • func (k *Kubernetes) Lookup(ctx context.Context, state request.Request, name string, typ uint16) (*dns.Msg, error)
  • func (k *Kubernetes) Metadata(ctx context.Context, state request.Request) context.Context
  • func (k *Kubernetes) MinTTL(state request.Request) uint32
  • func (k Kubernetes) Name() string
  • func (k *Kubernetes) Ready() bool
  • func (k *Kubernetes) Records(ctx context.Context, state request.Request, exact bool) ([]msg.Service, error)
  • func (k *Kubernetes) RegisterKubeCache(c *caddy.Controller)
  • func (k *Kubernetes) Reverse(ctx context.Context, state request.Request, exact bool, opt plugin.Options) ([]msg.Service, error)
  • func (k *Kubernetes) Serial(state request.Request) uint32
  • func (k Kubernetes) ServeDNS(ctx context.Context, w dns.ResponseWriter, r *dns.Msg) (int, error)
  • func (k *Kubernetes) Services(ctx context.Context, state request.Request, exact bool, opt plugin.Options) (svcs []msg.Service, err error)
  • func (k *Kubernetes) Transfer(ctx context.Context, state request.Request) (int, error)

Constants

const (

	// DNSSchemaVersion is the schema version: https://github.com/kubernetes/dns/blob/master/docs/specification.md
	DNSSchemaVersion = "1.1.0"
	// Svc is the DNS schema for kubernetes services
	Svc = "svc"
	// Pod is the DNS schema for kubernetes pods
	Pod = "pod"
)

Variables

var (
	// DnsProgrammingLatency is defined as the time it took to program a DNS instance - from the time
	// a service or pod has changed to the time the change was propagated and was available to be
	// served by a DNS server.
	// The definition of this SLI can be found at https://github.com/kubernetes/community/blob/master/sig-scalability/slos/dns_programming_latency.md
	// Note that the metrics is partially based on the time exported by the endpoints controller on
	// the master machine. The measurement may be inaccurate if there is a clock drift between the
	// node and master machine.
	// The service_kind label can be one of:
	//   * cluster_ip
	//   * headless_with_selector
	//   * headless_without_selector
	DnsProgrammingLatency = prometheus.NewHistogramVec(prometheus.HistogramOpts{
		Namespace: plugin.Namespace,
		Subsystem: pluginName,
		Name:      "dns_programming_duration_seconds",

		Buckets: prometheus.ExponentialBuckets(0.001, 2, 20),
		Help:    "Histogram of the time (in seconds) it took to program a dns instance.",
	}, []string{"service_kind"})
)

Types

type Kubernetes

type Kubernetes struct {
	Next          plugin.Handler
	Zones         []string
	Upstream      *upstream.Upstream
	APIServerList []string
	APICertAuth   string
	APIClientCert string
	APIClientKey  string
	ClientConfig  clientcmd.ClientConfig
	APIConn       dnsController
	Namespaces    map[string]struct{}

	Fall fall.F

	TransferTo []string
	// contains filtered or unexported fields
}

Kubernetes implements a plugin that connects to a Kubernetes cluster.

func New

func New(zones []string) *Kubernetes

New returns a initialized Kubernetes. It default interfaceAddrFunc to return 127.0.0.1. All other values default to their zero value, primaryZoneIndex will thus point to the first zone.

func ParseStanza

func ParseStanza(c *caddy.Controller) (*Kubernetes, error)

ParseStanza parses a kubernetes stanza

func (*Kubernetes) AutoPath

func (k *Kubernetes) AutoPath(state request.Request) []string

AutoPath implements the AutoPathFunc call from the autopath plugin. It returns a per-query search path or nil indicating no searchpathing should happen.

func (*Kubernetes) External

func (k *Kubernetes) External(state request.Request) ([]msg.Service, int)

External implements the ExternalFunc call from the external plugin. It returns any services matching in the services' ExternalIPs.

func (*Kubernetes) ExternalAddress

func (k *Kubernetes) ExternalAddress(state request.Request) []dns.RR

ExternalAddress returns the external service address(es) for the CoreDNS service.

func (*Kubernetes) InitKubeCache

func (k *Kubernetes) InitKubeCache(ctx context.Context) (err error)

InitKubeCache initializes a new Kubernetes cache.

func (*Kubernetes) IsNameError

func (k *Kubernetes) IsNameError(err error) bool

IsNameError implements the ServiceBackend interface.

func (*Kubernetes) Lookup

func (k *Kubernetes) Lookup(ctx context.Context, state request.Request, name string, typ uint16) (*dns.Msg, error)

Lookup implements the ServiceBackend interface.

func (*Kubernetes) Metadata

func (k *Kubernetes) Metadata(ctx context.Context, state request.Request) context.Context

Metadata implements the metadata.Provider interface.

func (*Kubernetes) MinTTL

func (k *Kubernetes) MinTTL(state request.Request) uint32

MinTTL implements the Transferer interface.

func (Kubernetes) Name

func (k Kubernetes) Name() string

Name implements the Handler interface.

func (*Kubernetes) Ready

func (k *Kubernetes) Ready() bool

Ready implements the ready.Readiness interface.

func (*Kubernetes) Records

func (k *Kubernetes) Records(ctx context.Context, state request.Request, exact bool) ([]msg.Service, error)

Records looks up services in kubernetes.

func (*Kubernetes) RegisterKubeCache

func (k *Kubernetes) RegisterKubeCache(c *caddy.Controller)

RegisterKubeCache registers KubeCache start and stop functions with Caddy

func (*Kubernetes) Reverse

func (k *Kubernetes) Reverse(ctx context.Context, state request.Request, exact bool, opt plugin.Options) ([]msg.Service, error)

Reverse implements the ServiceBackend interface.

func (*Kubernetes) Serial

func (k *Kubernetes) Serial(state request.Request) uint32

Serial implements the Transferer interface.

func (Kubernetes) ServeDNS

func (k Kubernetes) ServeDNS(ctx context.Context, w dns.ResponseWriter, r *dns.Msg) (int, error)

ServeDNS implements the plugin.Handler interface.

func (*Kubernetes) Services

func (k *Kubernetes) Services(ctx context.Context, state request.Request, exact bool, opt plugin.Options) (svcs []msg.Service, err error)

Services implements the ServiceBackend interface.

func (*Kubernetes) Transfer

func (k *Kubernetes) Transfer(ctx context.Context, state request.Request) (int, error)

Transfer implements the Transferer interface.