fqdn

Documentation

Overview

Package fqdn handles DNS based policy enforcment. This is expressed via ToFQDN rules and implements a DNS polling scheme with DNS lookups originating from the Cilium agent.

Note: We add a ToFQDN-UUID label to rules when we process a ToFQDN section. This has the source cilium-generated and should not be modified outside pkg/fqdn

The poller will update imported policy rules that contain ToFQDN sections with matching ToCIDRSet sections (in the same egress rule, thus inheriting the same L4/L7 policy). Each CIDR is a fully qualified IP (i.e. a /32 or /128) and each IP returned in the DNS lookup creates a corresponding CIDR. The package relies on the internal policy logic to return early/trigger no regenerations if the policy is not actually different (e.g. a more broad/permissive rule already applies to an endpoint so any IP changes are irrelevant).

Index

• Constants
• func ConfigFromResolvConf() error
• func DNSLookupDefaultResolver(dnsNames []string) (DNSIPs map[string]*DNSIPRecords, DNSErrors map[string]error)
• func KeepUniqueNames(names []string) []string
• func SetDNSConfig(conf *dns.ClientConfig)
• func StartDNSPoller(poller *DNSPoller)
• type Config
• type DNSCache
  • func NewDNSCache(minTTL int) *DNSCache
  • func NewDNSCacheWithLimit(minTTL int, limit int) *DNSCache
  • func (c *DNSCache) Dump() (lookups []*cacheEntry)
  • func (c *DNSCache) ForceExpire(expireLookupsBefore time.Time, nameMatch *regexp.Regexp) (namesAffected []string)
  • func (c *DNSCache) ForceExpireByNames(expireLookupsBefore time.Time, names []string) (namesAffected []string)
  • func (c *DNSCache) GC() []string
  • func (c *DNSCache) Lookup(name string) (ips []net.IP)
  • func (c *DNSCache) LookupByRegexp(re *regexp.Regexp) (matches map[string][]net.IP)
  • func (c *DNSCache) LookupIP(ip net.IP) (names []string)
  • func (c *DNSCache) MarshalJSON() ([]byte, error)
  • func (c *DNSCache) UnmarshalJSON(raw []byte) error
  • func (c *DNSCache) Update(lookupTime time.Time, name string, ips []net.IP, ttl int) bool
  • func (c *DNSCache) UpdateFromCache(update *DNSCache, namesToUpdate []string)
• type DNSIPRecords
• type DNSPoller
  • func NewDNSPoller(config Config, ruleManager *NameManager) *DNSPoller
  • func (poller *DNSPoller) LookupUpdateDNS(ctx context.Context) error
• type NameManager
  • func NewNameManager(config Config) *NameManager
  • func (n *NameManager) CompleteBootstrap()
  • func (n *NameManager) ForceGenerateDNS(ctx context.Context, namesToRegen []string) (wg *sync.WaitGroup, err error)
  • func (n *NameManager) GenerateSelectorUpdates(fqdnSelectors map[api.FQDNSelector]struct{}) (namesMissingIPs []api.FQDNSelector, ...)
  • func (n *NameManager) GetDNSCache() *DNSCache
  • func (n *NameManager) GetDNSNames() (dnsNames []string)
  • func (n *NameManager) RegisterForIdentityUpdates(selector api.FQDNSelector) []identity.NumericIdentity
  • func (n *NameManager) UnregisterForIdentityUpdates(selector api.FQDNSelector)
  • func (n *NameManager) UpdateDNSIPs(lookupTime time.Time, updatedDNSIPs map[string]*DNSIPRecords) (affectedSelectors map[api.FQDNSelector]struct{}, ...)
  • func (n *NameManager) UpdateGenerateDNS(ctx context.Context, lookupTime time.Time, ...) (wg *sync.WaitGroup, err error)

Constants

const DNSPollerInterval = 5 * time.Second

DNSPollerInterval is the time between 2 complete DNS lookup runs of the DNSPoller controller Note: This cannot be less than 1*time.Second, as it is used as a default for MinTTL in DNSPollerConfig

Functions

func ConfigFromResolvConf

func ConfigFromResolvConf() error

ConfigFromResolvConf parses the configuration in /etc/resolv.conf and sets the configuration for pkg/fqdn. nameservers and opt timeout are supported. search and ndots are NOT supported. This call is not thread safe.

func DNSLookupDefaultResolver

func DNSLookupDefaultResolver(dnsNames []string) (DNSIPs map[string]*DNSIPRecords, DNSErrors map[string]error)

DNSLookupDefaultResolver sequentially and synchronously runs a DNS lookup for every name in dnsNames. It does not use net.DefaultResolver, but dnsConfig from this package (populated from resolv.conf). The configured servers are always queried in the same order, only moving to the next on errors (such as timeouts). The names are queried in random order (as the map iteration is random) but, for each name, A records and then AAAA records are requested in that order. It will return: DNSIPs: a map of DNS names to their IPs and associated smallest TTL (only contains successful lookups). CNAME records in the response are collapsed into the IPs later in the response data. The CNAME TTL is considered when finding the smallest TTL. DNSErrors: a map of DNS names to lookup errors.

DNSLookupDefaultResolver is used by DNSPoller when no alternative LookupDNSNames is provided.

func KeepUniqueNames

func KeepUniqueNames(names []string) []string

KeepUniqueNames it gets a array of strings and return a new array of strings with the unique names.

func SetDNSConfig

func SetDNSConfig(conf *dns.ClientConfig)

SetDNSConfig store conf in pkg/fqdn as the global config. It also creates the global UDP and TCP clients used for DNS lookups in DNSLookupDefaultResolver. Only .Servers and .Timeout are utilized from conf. This call is not thread safe.

func StartDNSPoller

func StartDNSPoller(poller *DNSPoller)

StartDNSPoller spawns a singleton DNS polling controller. The controller will, periodically, run a DNS lookup for each ToFQDN target DNS name inserted with StartPollForDNSName. Note: Repeated calls will replace earlier instances of the controller.

Types

type Config

type Config struct {
	// MinTTL is the time used by the poller to cache information.
	// When set to 0, 2*DNSPollerInterval is used.
	MinTTL int

	// OverLimit is the number of max entries that a host can have in the DNS cache.
	OverLimit int

	// Cache is where the poller stores DNS data used to generate rules.
	// When set to nil, it uses fqdn.DefaultDNSCache, a global cache instance.
	Cache *DNSCache

	// DNSConfig includes the Resolver IPs, port, timeout and retry count. It is
	// expected to be  generated from /etc/resolv.conf.
	DNSConfig *dns.ClientConfig

	// LookupDNSNames is a callback to run the provided DNS lookups.
	// When set to nil, fqdn.DNSLookupDefaultResolver is used.
	LookupDNSNames func(dnsNames []string) (DNSIPs map[string]*DNSIPRecords, errorDNSNames map[string]error)

	// UpdateSelectors is a callback to update the mapping of FQDNSelector to
	// sets of IPs.
	UpdateSelectors func(ctx context.Context, selectorsWithIPs map[api.FQDNSelector][]net.IP, selectorsWithoutIPs []api.FQDNSelector) (*sync.WaitGroup, error)

	// PollerResponseNotify is used when the poller receives DNS data in response
	// to a successful poll.
	// Note: This function doesn't do much, as the poller is still wired to
	// NameManager directly right now.
	PollerResponseNotify func(lookupTime time.Time, qname string, response *DNSIPRecords)
}

Config is a simple configuration structure to set how pkg/fqdn subcomponents behave. DNSPoller relies on LookupDNSNames to control how DNS lookups are done, and UpdateSelectors to control how generated policy rules are emitted.

type DNSCache

type DNSCache struct {
	lock.RWMutex
	// contains filtered or unexported fields
}

DNSCache manages DNS data that will expire after a certain TTL. Information is tracked per-IP address, retaining the latest-expiring DNS data for each address. For most real-world DNS data, the entry per name remains small because newer lookups replace older ones. Large TTLs may cause entries to grow if many unique IPs are returned in separate lookups. Redundant or expired entries are removed on insert. Lookups check for expired entries.

func NewDNSCache

func NewDNSCache(minTTL int) *DNSCache

NewDNSCache returns an initialized DNSCache

func NewDNSCacheWithLimit

func NewDNSCacheWithLimit(minTTL int, limit int) *DNSCache

NewDNSCache returns an initialized DNSCache and set the max host limit to the given argument

func (*DNSCache) Dump

func (c *DNSCache) Dump() (lookups []*cacheEntry)

Dump returns unexpired cache entries in the cache. They are deduplicated, but not usefully sorted. These objects should not be modified.

func (*DNSCache) ForceExpire

func (c *DNSCache) ForceExpire(expireLookupsBefore time.Time, nameMatch *regexp.Regexp) (namesAffected []string)

ForceExpire is used to clear entries from the cache before their TTL is over. This operation does not keep previous guarantees that, for each IP, the most recent lookup to provide that IP is used. Note that all parameters must match, if provided. `time.Time{}` is considered not-provided for time parameters. expireLookupsBefore requires a lookup to have a LookupTime before it in order to remove it. nameMatch will remove any DNS names that match.

func (*DNSCache) ForceExpireByNames

func (c *DNSCache) ForceExpireByNames(expireLookupsBefore time.Time, names []string) (namesAffected []string)

ForceExpireByNames is the same function as ForceExpire but uses the exact names to delete the entries.

func (*DNSCache) GC

func (c *DNSCache) GC() []string

GC garbage collector function that clean expired entries and return the entries that are deleted.

func (*DNSCache) Lookup

func (c *DNSCache) Lookup(name string) (ips []net.IP)

Lookup returns a set of unique IPs that are currently unexpired for name, if any exist. An empty list indicates no valid records exist. The IPs are returned sorted.

func (*DNSCache) LookupByRegexp

func (c *DNSCache) LookupByRegexp(re *regexp.Regexp) (matches map[string][]net.IP)

LookupByRegexp returns all non-expired cache entries that match re as a map of name -> IPs

func (*DNSCache) LookupIP

func (c *DNSCache) LookupIP(ip net.IP) (names []string)

LookupIP returns all DNS names in entries that include that IP. The cache maintains the latest-expiring entry per-name per-IP. This means that multiple names referrring to the same IP will expire from the cache at different times, and only 1 entry for each name-IP pair is internally retained.

func (*DNSCache) MarshalJSON

func (c *DNSCache) MarshalJSON() ([]byte, error)

MarshalJSON serialises the set of DNS lookup cacheEntries needed to reconstruct this cache instance. Note: Expiration times are honored and the reconstructed cache instance is expected to return the same values as the original at that point in time.

func (*DNSCache) UnmarshalJSON

func (c *DNSCache) UnmarshalJSON(raw []byte) error

UnmarshalJSON rebuilds a DNSCache from serialized JSON. Note: This is destructive to any currect data. Use UpdateFromCache for bulk updates.

func (*DNSCache) Update

func (c *DNSCache) Update(lookupTime time.Time, name string, ips []net.IP, ttl int) bool

Update inserts a new entry into the cache. After insertion cache entries for name are expired and redundant entries evicted. This is O(number of new IPs) for eviction, and O(number of IPs for name) for expiration. lookupTime is the time the DNS information began being valid. It should be in the past. name is used as is and may be an unqualified name (e.g. myservice.namespace). ips may be an IPv4 or IPv6 IP. Duplicates will be removed. ttl is the DNS TTL for ips and is a seconds value.

func (*DNSCache) UpdateFromCache

func (c *DNSCache) UpdateFromCache(update *DNSCache, namesToUpdate []string)

UpdateFromCache is a utility function that allows updating a DNSCache instance with all the internal entries of another. Latest-Expiration still applies, thus the merged outcome is consistent with adding the entries individually. When namesToUpdate has non-zero length only those names are updated from update, otherwise all DNS names in update are used.

type DNSIPRecords

type DNSIPRecords struct {
	// TTL is the time, in seconds, that these IPs are valid for
	TTL int

	// IPs are the IPs associated with a DNS Name
	IPs []net.IP
}

DNSIPRecords mimics the RR data from an A or AAAA response. My kingdom for a DNS IP RR type that isn't hidden in the stdlib or has a million layers of type indirection.

type DNSPoller

type DNSPoller struct {
	lock.Mutex // this guards both maps and their contents

	// DNSHistory is the collection of still-valid DNS responses intercepted
	// for the poller.
	// This is not protected by the mutex due to internally is using a mutex.
	DNSHistory *DNSCache
	// contains filtered or unexported fields
}

DNSPoller periodically runs lookups for registered DNS names. It will emit regenerated policy rules when the IPs change. CNAMEs (and DNAMEs) are not handled directly, but will depend on the resolver's behavior. fqdn.Config can be opitonally used to set how the DNS lookups are executed (via LookupDNSNames) and how generated policy rules are handled (via UpdateSelectors).

func NewDNSPoller

func NewDNSPoller(config Config, ruleManager *NameManager) *DNSPoller

NewDNSPoller creates an initialized DNSPoller. It does not start the controller (use .Start)

func (*DNSPoller) LookupUpdateDNS

func (poller *DNSPoller) LookupUpdateDNS(ctx context.Context) error

LookupUpdateDNS runs a DNS lookup for each stored DNS name, storing updates into ruleManager, which may emit regenerated policy rules. The general steps are: 1- take a snapshot of DNS names to lookup from .ruleManager, into dnsNamesToPoll 2- Do a DNS lookup for each DNS name (map key) in poller via LookupDNSNames 3- Update IPs for each dnsName in .ruleManager. If the IPs have changed for the name, it will generate and emit them.

type NameManager

type NameManager struct {
	lock.Mutex
	// contains filtered or unexported fields
}

NameManager maintains state DNS names, via FQDNSelector or exact match for polling, need to be tracked. It is the main structure which relates the FQDN subsystem to the policy subsystem for plumbing the relation between a DNS name and the corresponding IPs which have been returned via DNS lookups. When DNS updates are given to a NameManager it update cached selectors as required via UpdateSelectors. DNS information is cached, respecting TTL.

func NewNameManager

func NewNameManager(config Config) *NameManager

NewNameManager creates an initialized NameManager. When config.Cache is nil, the global fqdn.DefaultDNSCache is used.

func (*NameManager) CompleteBootstrap

func (n *NameManager) CompleteBootstrap()

func (*NameManager) ForceGenerateDNS

func (n *NameManager) ForceGenerateDNS(ctx context.Context, namesToRegen []string) (wg *sync.WaitGroup, err error)

ForceGenerateDNS unconditionally regenerates all rules that refer to DNS names in namesToRegen. These names are FQDNs and toFQDNs.matchPatterns or matchNames that match them will cause these rules to regenerate.

func (*NameManager) GenerateSelectorUpdates

func (n *NameManager) GenerateSelectorUpdates(fqdnSelectors map[api.FQDNSelector]struct{}) (namesMissingIPs []api.FQDNSelector, selectorIPMapping map[api.FQDNSelector][]net.IP)

GenerateSelectorUpdates iterates over all names in the DNS cache managed by gen and figures out to which FQDNSelectors managed by the cache these names map. Returns the set of FQDNSelectors which map to no IPs, and a mapping of FQDNSelectors to IPs.

func (*NameManager) GetDNSCache

func (n *NameManager) GetDNSCache() *DNSCache

GetDNSCache returns the DNSCache used by the NameManager

func (*NameManager) GetDNSNames

func (n *NameManager) GetDNSNames() (dnsNames []string)

GetDNSNames returns a snapshot of the DNS names managed by this NameManager

func (*NameManager) RegisterForIdentityUpdates

func (n *NameManager) RegisterForIdentityUpdates(selector api.FQDNSelector) []identity.NumericIdentity

RegisterForIdentityUpdates exposes this FQDNSelector so that identities for IPs contained in a DNS response that matches said selector can be propagated back to the SelectorCache via `UpdateFQDNSelector`. All DNS names contained within the NameManager's cache are iterated over to see if they match the FQDNSelector. All IPs which correspond to the DNS names which match this Selector will be returned as CIDR identities, as other DNS Names which have already been resolved may match this FQDNSelector.

func (*NameManager) UnregisterForIdentityUpdates

func (n *NameManager) UnregisterForIdentityUpdates(selector api.FQDNSelector)

UnregisterForIdentityUpdates removes this FQDNSelector from the set of FQDNSelectors which are being tracked by the NameManager. No more updates for IPs which correspond to said selector are propagated.

func (*NameManager) UpdateDNSIPs

func (n *NameManager) UpdateDNSIPs(lookupTime time.Time, updatedDNSIPs map[string]*DNSIPRecords) (affectedSelectors map[api.FQDNSelector]struct{}, updatedNames map[string][]net.IP)

UpdateDNSIPs updates the IPs for each DNS name in updatedDNSIPs. It returns: affectedSelectors: a set of all FQDNSelectors which match DNS Names whose corresponding set of IPs has changed. updatedNames: a map of DNS names to all the valid IPs we store for each.

func (*NameManager) UpdateGenerateDNS

func (n *NameManager) UpdateGenerateDNS(ctx context.Context, lookupTime time.Time, updatedDNSIPs map[string]*DNSIPRecords) (wg *sync.WaitGroup, err error)

UpdateGenerateDNS inserts the new DNS information into the cache. If the IPs have changed for a name, store which rules must be updated in rulesToUpdate, regenerate them, and emit via UpdateSelectors.