client

package
v0.20.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 26, 2022 License: MPL-2.0 Imports: 51 Imported by: 0

Documentation

Index

Constants

View Source 
const DevelopmentVersion = "development"
View Source 
const (

	// UpdateCheckPeriod specifies how much time in seconds must pass between
	// subsequent checks for CloudQuery core update availability.
	UpdateCheckPeriod = int64(23 * time.Hour / time.Second)
)

Variables

View Source 
var (
	ErrMigrationsNotSupported = errors.New("provider doesn't support migrations")
)
View Source 
var (
	// Version variable is injected in build time
	Version = DevelopmentVersion
)

Functions

func MaybeCheckForUpdate added in v0.17.1 

func MaybeCheckForUpdate(ctx context.Context, fs afero.Afero, nowUnix, period int64) (*version.Version, error)

MaybeCheckForUpdate checks if an update to CloudQuery core is available and returns its (new) version. To avoid making those network requests on each CLI invocation it stores last time and version seen on Github in a so called "last update check" file. If there is an error then returned version is nil. To be specific, error is returned if:

* client.Version is not a valid semantic version * if "last update check" file does not exist and we fail to write to it * if "last update check" file has a single word "disable" in it * Github reports an error * last release version on Github is not a valid semantic version

Otherwise, new available version is returned that is obtained from either:

* "last update check" content and that version is newer than current * Github latest release version and it's newer than current

Types

type Client 

type Client struct {
	// Required: List of providers that are required, these providers will be downloaded if DownloadProviders is called.
	Providers []*config.RequiredProvider
	// Optional: Registry url to verify plugins from, defaults to CloudQuery hub
	RegistryURL string
	// Optional: Where to save downloaded providers, by default current working directory, defaults to ./cq/providers
	PluginDirectory string
	// Optional: Where to save downloaded policies, by default current working directory, defaults to ./cq/policy
	PolicyDirectory string
	// Optional: If true cloudquery just runs policy files without using git tag to select a version
	SkipVersioning bool
	// Optional: if this flag is true, plugins downloaded from URL won't be verified when downloaded
	NoVerify bool
	// Optional: DSN connection information for database client will connect to
	DSN string
	// Optional: Skips Building tables on fetch execution
	SkipBuildTables bool
	// Optional: HubProgressUpdater allows the client creator to get called back on download progress and completion.
	HubProgressUpdater ui.Progress
	// Optional: Logger framework can use to log.
	// default: global logger provided.
	Logger hclog.Logger
	// Optional: Hub client to use to download plugins, the Hub is used to download and pluginManager providers binaries
	// if not specified, default cloudquery registry is used.
	Hub registry.Hub
	// manager manages all plugins lifecycle
	Manager *plugin.Manager
	// ModuleManager manages all modules lifecycle
	ModuleManager module.Manager
	// ModuleManager manages all modules lifecycle
	PolicyManager policy.Manager
	// TableCreator defines how tables are created in the database, only for plugin protocol < 4
	TableCreator TableCreator
	// HistoryConfig defines configuration for CloudQuery history mode
	HistoryCfg *history.Config
	// contains filtered or unexported fields
}

Client is the client for executing providers, fetching data and running queries and polices

func New 

func New(ctx context.Context, options ...Option) (*Client, error)

func (*Client) BuildProviderTables added in v0.14.0 

func (c *Client) BuildProviderTables(ctx context.Context, providerName string) (retErr error)

func (*Client) CheckForProviderUpdates added in v0.17.1 

func (c *Client) CheckForProviderUpdates(ctx context.Context) ([]ProviderUpdateSummary, error)

CheckForProviderUpdates checks for provider updates

func (*Client) Close 

func (c *Client) Close()

func (*Client) DowngradeProvider added in v0.14.5 

func (c *Client) DowngradeProvider(ctx context.Context, providerName string) (retErr error)

func (*Client) DownloadProviders added in v0.13.6 

func (c *Client) DownloadProviders(ctx context.Context) (retErr error)

DownloadProviders downloads all provider binaries

func (*Client) DropProvider added in v0.14.5 

func (c *Client) DropProvider(ctx context.Context, providerName string) (retErr error)

func (*Client) ExecuteModule added in v0.15.10 

func (c *Client) ExecuteModule(ctx context.Context, req ModuleRunRequest) (res *module.ExecutionResult, retErr error)

func (*Client) Fetch 

func (c *Client) Fetch(ctx context.Context, request FetchRequest) (res *FetchResponse, retErr error)

func (*Client) GetProviderConfiguration 

func (c *Client) GetProviderConfiguration(ctx context.Context, providerName string) (*cqproto.GetProviderConfigResponse, error)

func (*Client) GetProviderSchema 

func (c *Client) GetProviderSchema(ctx context.Context, providerName string) (*ProviderSchema, error)

func (*Client) LoadPolicy added in v0.19.0 

func (c *Client) LoadPolicy(ctx context.Context, name, source string) (pol *policy.Policy, retErr error)

func (*Client) MigrateCore added in v0.19.2 

func (c *Client) MigrateCore(ctx context.Context, de database.DialectExecutor) error

func (*Client) NormalizeResources added in v0.15.5 

func (c *Client) NormalizeResources(ctx context.Context, providers []*config.Provider) error

NormalizeResources walks over all given providers and in place normalizes their resources list:

* wildcard expansion * no unknown resources * no duplicate resources

func (*Client) RunPolicies added in v0.16.0 

func (c *Client) RunPolicies(ctx context.Context, req *PoliciesRunRequest) ([]*policy.ExecutionResult, error)

func (*Client) SaveFetchSummary added in v0.20.0 

func (c *Client) SaveFetchSummary(ctx context.Context, fs *FetchSummary) error

SaveFetchSummary saves fetch summary into fetches database

func (*Client) SetProviderVersion added in v0.15.1 

func (c *Client) SetProviderVersion(ctx context.Context, providerName, version string) error

func (*Client) TestProvider added in v0.14.6 

func (c *Client) TestProvider(ctx context.Context, providerCfg *config.Provider) error

func (*Client) UpgradeProvider added in v0.14.5 

func (c *Client) UpgradeProvider(ctx context.Context, providerName string) (retErr error)

type FetchDoneResult 

type FetchDoneResult struct {
	// Map of providers and resources that have finished fetching
	DoneResources map[string]map[string]bool
	// Amount of resources collected so far
	ResourceCount string
}

type FetchRequest 

type FetchRequest struct {
	// UpdateCallback allows gets called when the client receives updates on fetch.
	UpdateCallback FetchUpdateCallback
	// Providers list of providers to call for fetching
	Providers []*config.Provider
	// Optional: Adds extra fields to the provider, this is used for history mode and testing purposes.
	ExtraFields map[string]interface{}
}

FetchRequest is provided to the Client to execute a fetch on one or more providers

type FetchResponse added in v0.15.0 

type FetchResponse struct {
	ProviderFetchSummary map[string]ProviderFetchSummary
}

FetchResponse is returned after a successful fetch execution, it holds a fetch summary for each provider that was executed.

type FetchSummary added in v0.19.2 

type FetchSummary struct {
	CqId uuid.UUID `db:"id"`
	//  Unique Id of fetch session
	FetchId            uuid.UUID              `db:"fetch_id"`
	CreatedAt          *time.Time             `db:"created_at"`
	Start              *time.Time             `db:"start"`
	Finish             *time.Time             `db:"finish"`
	IsSuccess          bool                   `db:"is_success"`
	TotalResourceCount uint64                 `db:"total_resource_count"`
	TotalErrorsCount   uint64                 `db:"total_errors_count"`
	ProviderName       string                 `db:"provider_name"`
	ProviderAlias      string                 `db:"provider_alias"`
	ProviderVersion    string                 `db:"provider_version"`
	CoreVersion        string                 `db:"core_version"`
	Resources          ResourceFetchSummaries `db:"results"`
}

FetchSummary includes a summarized report of fetch, such as fetch id, fetch start and finish, resources fetch results

type FetchUpdate 

type FetchUpdate struct {
	Provider string
	Version  string
	// Map of resources that have finished fetching
	FinishedResources map[string]bool
	// Amount of resources collected so far
	ResourceCount uint64
	// Error if any returned by the provider
	Error string
	// PartialFetchResults contains the partial fetch results for this update
	PartialFetchResults []*cqproto.FailedResourceFetch
}

func (FetchUpdate) AllDone 

func (f FetchUpdate) AllDone() bool

func (FetchUpdate) DoneCount 

func (f FetchUpdate) DoneCount() int

type FetchUpdateCallback 

type FetchUpdateCallback func(update FetchUpdate)

type ModuleRunRequest added in v0.15.10 

type ModuleRunRequest struct {
	// Name of the module
	Name string

	// Params are the invocation parameters specific to the module
	Params interface{}

	// Providers is the list of providers to process
	Providers []*cqproto.GetProviderSchemaResponse

	// Config is the config profile provided by the user
	Config hcl.Body
}

ModuleRunRequest is the request used to run a module.

type Option 

type Option func(options *Client)

type PoliciesRunRequest added in v0.16.0 

type PoliciesRunRequest struct {
	// Policies to run
	Policies policy.Policies

	// OutputDir is the output dir for policy execution output.
	OutputDir string

	// RunCallBack is the callback method that is called after every policy execution.
	RunCallback policy.UpdateCallback
}

PoliciesRunRequest is the request used to run a policy.

type ProviderFetchSummary added in v0.15.0 

type ProviderFetchSummary struct {
	ProviderName          string
	ProviderAlias         string
	Version               string
	PartialFetchErrors    []*cqproto.FailedResourceFetch
	FetchErrors           []error
	TotalResourcesFetched uint64
	FetchResources        map[string]cqproto.ResourceFetchSummary
	Status                string
}

ProviderFetchSummary represents a request for the FetchFinishCallback

func (ProviderFetchSummary) Diagnostics added in v0.15.8 

func (p ProviderFetchSummary) Diagnostics() diag.Diagnostics

func (ProviderFetchSummary) HasErrors added in v0.15.1 

func (p ProviderFetchSummary) HasErrors() bool

func (ProviderFetchSummary) Metrics added in v0.17.0 

func (p ProviderFetchSummary) Metrics() map[string]int64

type ProviderSchema added in v0.20.0 

type ProviderSchema struct {
	*cqproto.GetProviderSchemaResponse

	ProtocolVersion int
}

type ProviderUpdateSummary added in v0.17.1 

type ProviderUpdateSummary struct {
	Name          string
	Version       string
	LatestVersion string
}

type ResourceFetchSummaries added in v0.19.2 

type ResourceFetchSummaries []ResourceFetchSummary

func (ResourceFetchSummaries) Value added in v0.19.2 

func (r ResourceFetchSummaries) Value() (driver.Value, error)

Value implements Valuer interface required by goqu

type ResourceFetchSummary added in v0.19.2 

type ResourceFetchSummary struct {
	ResourceName string `json:"resource_name"`
	// map of resources that have finished fetching
	FinishedResources map[string]bool `json:"finished_resources"`
	// Amount of resources collected so far
	// Error value if any, if returned the stream will be canceled
	Error string `json:"error"`
	// list of resources where the fetching failed
	PartialFetchFailedResources []*cqproto.FailedResourceFetch `json:"partial_fetch_failed_resources"`
	// Execution status of resource
	Status string `json:"status"`
	// Total Amount of resources collected by this resource
	ResourceCount uint64 `json:"resource_count"`
	// Diagnostics of failed resource fetch, the diagnostic provides insights such as severity, summary and
	// details on how to solve this issue
	Diagnostics diag.Diagnostics `json:"diagnostics"`
}

ResourceFetchSummary includes a data about fetching specific resource

type TableCreator added in v0.14.0 

type TableCreator interface {
	CreateTable(context.Context, schema.QueryExecer, *schema.Table, *schema.Table) error
}

TableCreator creates tables based on schema received from providers

type UUID added in v0.17.1 

type UUID struct {
	pgtype.UUID
}

func (UUID) Get added in v0.17.1 

func (dst UUID) Get() interface{}

func (UUID) String added in v0.17.1 

func (u UUID) String() string

Source Files

View all Source files

Directories

Path Synopsis
postgres
timescale

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.