app

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 27, 2021 License: Apache-2.0 Imports: 38 Imported by: 0

README

The controllers are:

  • M4DApplication controller - creates/updates a blueprint specification from the information in the M4DApplication and from the policy compiler
  • Blueprint controller - reconciles the desired cluster state by orchestrating a blueprint It also configures lower level things like isolation policies.

Documentation

Index

Constants

View Source 
const BlueprintNamespace = "m4d-blueprints"

BlueprintNamespace defines a namespace where blueprints and associated resources will be allocated

Variables

This section is empty.

Functions

func AllocateBucket 

func AllocateBucket(c client.Client, log logr.Logger, owner types.NamespacedName, id string, geo string) (*storage.ProvisionedBucket, error)

AllocateBucket allocates a bucket in the relevant geo The buckets are created as temporary, i.e. to be removed after the owner Dataset is deleted After a successful copy and registering a dataset, the bucket will become persistent

func AnalyzeError 

func AnalyzeError(app *app.M4DApplication, log logr.Logger, assetID string, err error) error

AnalyzeError analyzes whether the given error is fatal, or a retrial attempt can be made. Reasons for retrial can be either communication problems with external services, or kubernetes problems to perform some action on a resource. A retrial is achieved by returning an error to the reconcile method

func ConstructApplicationContext 

func ConstructApplicationContext(datasetID string, input *app.M4DApplication, operation *pb.AccessOperation) *pb.ApplicationContext

ConstructApplicationContext constructs ApplicationContext structure to send to Policy Compiler

func CopyMap 

func CopyMap(m map[string]interface{}) map[string]interface{}

CopyMap copies a map

func GetConnectionDetails 

func GetConnectionDetails(req *modules.DataInfo, input *app.M4DApplication) error

GetConnectionDetails calls the data catalog service

func GetCredentials 

func GetCredentials(req *modules.DataInfo, input *app.M4DApplication) error

GetCredentials calls the credentials manager service TODO: Choose appropriate catalog connector based on the datacatalog service indicated as part of datasetID

func GetSupportedReadSources 

func GetSupportedReadSources(module *app.M4DModule) []*app.InterfaceDetails

GetSupportedReadSources returns a list of supported READ interfaces of a module

func LookupPolicyDecisions 

func LookupPolicyDecisions(datasetID string, policyCompiler pc.IPolicyCompiler, input *app.M4DApplication, op *pb.AccessOperation) ([]*pb.EnforcementAction, error)

LookupPolicyDecisions provides a list of governance actions for the given dataset and the given operation

func SecretToCredentialMap 

func SecretToCredentialMap(cl client.Client, secretRef types.NamespacedName) (map[string]interface{}, error)

SecretToCredentialMap fetches a secret and converts into a map matching credentials proto

func SecretToCredentials 

func SecretToCredentials(cl client.Client, secretRef types.NamespacedName) (*dc.Credentials, error)

SecretToCredentials fetches a secret and constructs Credentials structure

func SetMapField 

func SetMapField(obj map[string]interface{}, k string, v interface{}) bool

SetMapField updates a map

Types

type BlueprintReconciler 

type BlueprintReconciler struct {
	client.Client
	Name   string
	Log    logr.Logger
	Scheme *runtime.Scheme
	Helmer helm.Interface
}

BlueprintReconciler reconciles a Blueprint object

func NewBlueprintReconciler 

func NewBlueprintReconciler(mgr ctrl.Manager, name string, helmer helm.Interface) *BlueprintReconciler

NewBlueprintReconciler creates a new reconciler for Blueprint resources

func (*BlueprintReconciler) Reconcile 

func (r *BlueprintReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error)

Reconcile receives a Blueprint CRD

func (*BlueprintReconciler) SetupWithManager 

func (r *BlueprintReconciler) SetupWithManager(mgr ctrl.Manager) error

SetupWithManager registers Blueprint controller

type ContextInterface 

type ContextInterface interface {
	ResourceExists(ref *app.ResourceReference) bool
	CreateOrUpdateResource(owner *app.ResourceReference, ref *app.ResourceReference, blueprintPerClusterMap map[string]app.BlueprintSpec) error
	DeleteResource(ref *app.ResourceReference) error
	GetResourceStatus(ref *app.ResourceReference) (app.ObservedState, error)
	CreateResourceReference(appName string, appNamespace string) *app.ResourceReference
	GetManagedObject() runtime.Object
}

ContextInterface is an interface for communication with a generated resource (e.g. Blueprint)

type M4DApplicationReconciler 

type M4DApplicationReconciler struct {
	client.Client
	Name              string
	Log               logr.Logger
	Scheme            *runtime.Scheme
	VaultConnection   vault.Interface
	PolicyCompiler    pc.IPolicyCompiler
	ResourceInterface ContextInterface
	ClusterManager    multicluster.ClusterLister
	Provision         storage.ProvisionInterface
}

M4DApplicationReconciler reconciles a M4DApplication object

func NewM4DApplicationReconciler 

func NewM4DApplicationReconciler(mgr ctrl.Manager, name string, vaultConnection vault.Interface,
	policyCompiler pc.IPolicyCompiler, cm multicluster.ClusterLister, provision storage.ProvisionInterface) *M4DApplicationReconciler

NewM4DApplicationReconciler creates a new reconciler for M4DApplications

func (*M4DApplicationReconciler) GenerateBlueprint 

func (r *M4DApplicationReconciler) GenerateBlueprint(instances []modules.ModuleInstanceSpec, appContext *app.M4DApplication) app.BlueprintSpec

GenerateBlueprint creates the Blueprint spec based on the datasets and the governance actions required, which dictate the modules that must run in the m4d Credentials for accessing data set are stored in a credential management system (such as vault) and the paths for accessing them are included in the blueprint. The credentials themselves are not included in the blueprint.

func (*M4DApplicationReconciler) GenerateBlueprints 

func (r *M4DApplicationReconciler) GenerateBlueprints(instances []modules.ModuleInstanceSpec, appContext *app.M4DApplication) map[string]app.BlueprintSpec

GenerateBlueprints creates Blueprint specs (one per cluster)

func (*M4DApplicationReconciler) GetAllModules 

func (r *M4DApplicationReconciler) GetAllModules() (map[string]*app.M4DModule, error)

GetAllModules returns all CRDs of the kind M4DModule mapped by their name

func (*M4DApplicationReconciler) Reconcile 

func (r *M4DApplicationReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error)

Reconcile reconciles M4DApplication CRD It receives M4DApplication CRD and selects the appropriate modules that will run The outcome is either a single Blueprint running on the same cluster or a Plotter containing multiple Blueprints that may run on different clusters

func (*M4DApplicationReconciler) RefineInstances 

func (r *M4DApplicationReconciler) RefineInstances(instances []modules.ModuleInstanceSpec) []modules.ModuleInstanceSpec

RefineInstances collects all instances of the same read/write module and creates a new instance instead, with accumulated arguments. Copy modules are left unchanged.

func (*M4DApplicationReconciler) RegisterAsset 

func (r *M4DApplicationReconciler) RegisterAsset(catalogID string, info *app.DatasetDetails, input *app.M4DApplication) (string, error)

RegisterAsset registers a new asset in the specified catalog Input arguments: - catalogID: the destination catalog identifier - info: connection and credential details Returns: - an error if happened - the new asset identifier

func (*M4DApplicationReconciler) SetupWithManager 

func (r *M4DApplicationReconciler) SetupWithManager(mgr ctrl.Manager) error

SetupWithManager registers M4DApplication controller

type ModuleManager 

type ModuleManager struct {
	Client             client.Client
	Log                logr.Logger
	Modules            map[string]*app.M4DModule
	Clusters           []multicluster.Cluster
	Owner              types.NamespacedName
	PolicyCompiler     pc.IPolicyCompiler
	WorkloadGeography  string
	Provision          storage.ProvisionInterface
	VaultConnection    vault.Interface
	ProvisionedStorage map[string]NewAssetInfo
}

ModuleManager builds a set of modules based on the requirements (governance actions, data location) and the existing set of M4DModules

func (*ModuleManager) GetCopyDestination 

func (m *ModuleManager) GetCopyDestination(item modules.DataInfo, destinationInterface *app.InterfaceDetails, geo string) (*app.DataStore, error)

GetCopyDestination creates a Dataset for bucket allocation by implicit copies or ingest.

func (*ModuleManager) GetProcessingGeography 

func (m *ModuleManager) GetProcessingGeography(applicationContext *app.M4DApplication) (string, error)

GetProcessingGeography determines the geography of the workload cluster. If no cluster has been specified for a workload, a local cluster is assumed.

func (*ModuleManager) SelectModuleInstances 

func (m *ModuleManager) SelectModuleInstances(item modules.DataInfo, appContext *app.M4DApplication) ([]modules.ModuleInstanceSpec, error)

SelectModuleInstances selects the necessary read/copy/write modules for the blueprint for a given data set Write path is not yet implemented

type NewAssetInfo 

type NewAssetInfo struct {
	Storage *storage.ProvisionedBucket
	Details *pb.DatasetDetails
}

NewAssetInfo points to the provisoned storage and hold information about the new asset

type PlotterInterface 

type PlotterInterface struct {
	Client client.Client
}

PlotterInterface context implementation for communication with a single Plotter resource

func NewPlotterInterface 

func NewPlotterInterface(cl client.Client) *PlotterInterface

NewPlotterInterface creates a new plotter interface for M4DApplication controller

func (*PlotterInterface) CreateOrUpdateResource 

func (c *PlotterInterface) CreateOrUpdateResource(owner *app.ResourceReference, ref *app.ResourceReference, blueprintPerClusterMap map[string]app.BlueprintSpec) error

CreateOrUpdateResource creates a new Plotter resource or updates an existing one

func (*PlotterInterface) CreateResourceReference 

func (c *PlotterInterface) CreateResourceReference(appName string, appNamespace string) *app.ResourceReference

CreateResourceReference returns an identifier (name and namespace) of the generated resource.

func (*PlotterInterface) DeleteResource 

func (c *PlotterInterface) DeleteResource(ref *app.ResourceReference) error

DeleteResource deletes the generated Plotter resource

func (*PlotterInterface) GetManagedObject 

func (c *PlotterInterface) GetManagedObject() runtime.Object

GetManagedObject returns the type of the managed runtime object

func (*PlotterInterface) GetResourceSignature 

func (c *PlotterInterface) GetResourceSignature(ref *app.ResourceReference) *app.Plotter

GetResourceSignature returns the namespaced information of the generated Plotter resource

func (*PlotterInterface) GetResourceStatus 

func (c *PlotterInterface) GetResourceStatus(ref *app.ResourceReference) (app.ObservedState, error)

GetResourceStatus returns the generated Plotter status

func (*PlotterInterface) ResourceExists 

func (c *PlotterInterface) ResourceExists(ref *app.ResourceReference) bool

ResourceExists checks whether the Plotter resource generated by M4DApplication controller is active

type PlotterReconciler 

type PlotterReconciler struct {
	client.Client
	Name           string
	Log            logr.Logger
	Scheme         *runtime.Scheme
	ClusterManager multicluster.ClusterManager
}

PlotterReconciler reconciles a Plotter object

func NewPlotterReconciler 

func NewPlotterReconciler(mgr ctrl.Manager, name string, manager multicluster.ClusterManager) *PlotterReconciler

NewPlotterReconciler creates a new reconciler for Plotter resources

func (*PlotterReconciler) Reconcile 

func (r *PlotterReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error)

Reconcile receives a Plotter CRD

func (*PlotterReconciler) SetupWithManager 

func (r *PlotterReconciler) SetupWithManager(mgr ctrl.Manager) error

SetupWithManager registers Plotter controller

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.