prjcfg

package
v0.0.0-...-51f9457 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 9, 2021 License: Apache-2.0 Imports: 25 Imported by: 0

Documentation

Overview

Package prjcfg handles project-scoped CV config.

Configs are ingested and kept up to date using `ProjectConfigRefresher`, which is supposed to be called frequently, typically by a cron job.

Every time config is changed, corresponding Project Manager is notified. Additionally, Project managers is "poked" probabilistically for reliability even if there were no changes so long as corresponding LUCI project's config remains active.

TODO(crbug/1221908): implement pruning of old configs.

Index

Constants

View Source 
const ConfigFileName = "commit-queue.cfg"

Variables

View Source 
var File_go_chromium_org_luci_cv_internal_configs_prjcfg_tasks_proto protoreflect.FileDescriptor

Functions

func DisableProject 

func DisableProject(ctx context.Context, project string, notify NotifyCallback) error

DisableProject disables the given LUCI Project if it is currently enabled.

func GerritHost 

func GerritHost(cfg *pb.ConfigGroup_Gerrit) string

GerritHost returns Gerrit host.

Panics if config is not valid.

func GetAllProjectIDs 

func GetAllProjectIDs(ctx context.Context, enabledOnly bool) ([]string, error)

GetAllProjectIDs returns the names of all projects available in datastore.

func ProjectsWithConfig 

func ProjectsWithConfig(ctx context.Context) ([]string, error)

ProjectsWithConfig returns all LUCI projects which have CV config.

func UpdateProject 

func UpdateProject(ctx context.Context, project string, notify NotifyCallback) error

UpdateProject imports the latest CV Config for a given LUCI Project from LUCI Config if the config in CV is outdated.

Types

type ConfigGroup 

type ConfigGroup struct {
	Project *datastore.Key `gae:"$parent"`
	ID      ConfigGroupID  `gae:"$id"`
	// DrainingStartTime represents `draining_start_time` field in the CV config.
	//
	// Note that this is a project-level field. Therefore, all ConfigGroups in a
	// single version of config should have the same value.
	DrainingStartTime string `gae:",noindex"`
	// SubmitOptions represents `submit_options` field in the CV config.
	//
	// Note that this is currently a project-level field. Therefore, all
	// ConfigGroups in a single version of Config should have the same value.
	SubmitOptions *pb.SubmitOptions
	// Content represents a `pb.ConfigGroup` proto message defined in the CV
	// config
	Content *pb.ConfigGroup
	// contains filtered or unexported fields
}

ConfigGroup is an entity that represents a ConfigGroup defined in CV config.

func GetConfigGroup 

func GetConfigGroup(ctx context.Context, project string, id ConfigGroupID) (*ConfigGroup, error)

GetConfigGroup loads ConfigGroup from datastore if exists.

To handle non-existing ConfigGroup, use datastore.IsErrNoSuchEntity. Doesn't check whether validity or existence of the given LUCI project.

func (*ConfigGroup) ProjectString 

func (c *ConfigGroup) ProjectString() string

ProjectString returns LUCI Project as a string.

type ConfigGroupID 

type ConfigGroupID string

ConfigGroupID is the ID for ConfigGroup Entity.

It is in the format of "hash/name" where

  • `hash` is the `Hash` field in the containing `ProjectConfig`.
  • `name` is the value of `ConfigGroup.Name` if specified. If `name is not provided (Name in ConfigGroup is optional as of Sep. 2020. See: crbug/1063508), use "index#i" as name instead where `i` is the index (0-based) of this ConfigGroup in the config.

func MakeConfigGroupID 

func MakeConfigGroupID(hash, name string) ConfigGroupID

func (ConfigGroupID) Hash 

func (c ConfigGroupID) Hash() string

Returns Hash of the corresponding project config.

func (ConfigGroupID) Name 

func (c ConfigGroupID) Name() string

Returns name component only.

type ConfigHashInfo 

type ConfigHashInfo struct {

	// Hash is the `Hash` of a `ProjectConfig` that CV has imported.
	Hash    string         `gae:"$id"`
	Project *datastore.Key `gae:"$parent"`
	// GitRevision is the git revision (commit hash) of the imported config.
	GitRevision string `gae:",noindex"`
	// ProjectEVersion is largest version of ProjectConfig that this `Hash`
	// maps to.
	//
	// It is possible for a ConfigHash maps to multiple EVersions (e.g. a CV
	// Config change is landed then reverted which results in two new EVersions
	// but only one new Hash). Only the largest EVersion matters when cleanup
	// job runs (i.e. CV will keep the last 5 EVersions).
	ProjectEVersion int64 `gae:",noindex"`
	// UpdateTime is the timestamp when this ConfigHashInfo was last updated.
	UpdateTime time.Time `gae:",noindex"`
	// ConfigGroupNames are the names of all ConfigGroups with this `Hash`.
	ConfigGroupNames []string `gae:",noindex"`
	// contains filtered or unexported fields
}

ConfigHashInfo stores high-level info about a ProjectConfig `Hash`.

It is primarily used for cleanup purpose to decide which `Hash` and its corresponding `ConfigGroup`s can be safely deleted.

type Meta 

type Meta struct {
	// Project is LUCI Project ID.
	Project string
	// Status is status of the LUCI project config.
	Status Status
	// EVersion allows to compare progression of Project's config.
	//
	// Larger values means later config.
	// If StatusNotExists, the value is 0.
	EVersion int64
	// ConfigGroupNames are the names part of all ConfigGroups in this version.
	//
	// If project doesn't exist, empty.
	// Otherwise, contains at least one group.
	ConfigGroupNames []string
	// ConfigGroupIDs are the standalone IDs of all ConfigGroups in this version.
	//
	// If project doesn't exist, empty.
	// Otherwise, contains at least one group.
	ConfigGroupIDs []ConfigGroupID
	// contains filtered or unexported fields
}

Meta describes LUCI project's config version.

func GetHashMeta 

func GetHashMeta(ctx context.Context, project, hash string) (Meta, error)

GetHashMeta returns metadata for a project for a given config hash.

Doesn't check whether a project currently exists. Returns error if specific (project, hash) combo doesn't exist in Datastore.

func GetHashMetas 

func GetHashMetas(ctx context.Context, project string, hashes ...string) ([]Meta, error)

GetHashMetas returns a metadata for each given config hash.

Doesn't check whether a project currently exists. Returns error if any (project, hash) combo doesn't exist in Datastore.

func GetLatestMeta 

func GetLatestMeta(ctx context.Context, project string) (Meta, error)

GetLatestMeta returns latest metadata for a project.

func (*Meta) Exists 

func (m *Meta) Exists() bool

Exists returns whether project config exists.

func (Meta) GetConfigGroups 

func (m Meta) GetConfigGroups(ctx context.Context) ([]*ConfigGroup, error)

GetConfigGroups loads all ConfigGroups from datastore for this meta.

Meta must correspond to an existing project.

func (*Meta) Hash 

func (m *Meta) Hash() string

Hash returns unique identifier of contents of the imported Project config.

Panics if project config doesn't exist.

type NotifyCallback 

type NotifyCallback func(context.Context) error

NotifyCallback is called in a transaction context from UpdateProject and DisableProject. Used by configcron package.

type PM 

type PM interface {
	Poke(ctx context.Context, luciProject string) error
	UpdateConfig(ctx context.Context, luciProject string) error
}

PM encapsulates Project Manager notified by the ConfigRefresher.

In production, this will be prjmanager.Notifier.

type ProjectConfig 

type ProjectConfig struct {

	// Project is the name of this LUCI Project.
	Project string `gae:"$id"`
	// Enabled indicates whether CV is enabled for this LUCI Project.
	//
	// Project is disabled if it is de-registered in LUCI Config or it no longer
	// has CV config file.
	Enabled bool
	// UpdateTime is the timestamp when this ProjectConfig was last updated.
	UpdateTime time.Time `gae:",noindex"`
	// EVersion is the latest version number of this ProjectConfig.
	//
	// It increments by 1 every time a new config change is imported to CV for
	// this LUCI Project.
	EVersion int64 `gae:",noindex"`
	// Hash is a string computed from the content of latest imported CV Config
	// using `computeHash()`.
	Hash string `gae:",noindex"`
	// ExternalHash is the hash string of this CV config in the external source
	// of truth (currently, LUCI Config). Used to quickly decided whether the
	// Config has been updated without fetching the full content.
	ExternalHash string `gae:",noindex"`
	// ConfigGroupNames are the names of all ConfigGroups in the current version
	// of CV Config.
	ConfigGroupNames []string `gae:",noindex"`
	// contains filtered or unexported fields
}

ProjectConfig is the root entity that keeps track of the latest version info of the CV config for a LUCI Project. It only contains high-level metadata about the config. The actual content of config is stored in the `ConfigGroup` entities which can be looked up by constructing IDs using `ConfigGroupNames` field.

type RefreshProjectConfigTask 

type RefreshProjectConfigTask struct {
	Project string `protobuf:"bytes,1,opt,name=project,proto3" json:"project,omitempty"`
	Disable bool   `protobuf:"varint,2,opt,name=disable,proto3" json:"disable,omitempty"`
	// contains filtered or unexported fields
}

RefreshProjectConfigTask is used to import latest CV config for a LUCI Project from LUCI Config or disable a LUCI Project if `disable` is true.

Queue: "refresh-project-config".

func (*RefreshProjectConfigTask) Descriptor deprecated 

func (*RefreshProjectConfigTask) Descriptor() ([]byte, []int)

Deprecated: Use RefreshProjectConfigTask.ProtoReflect.Descriptor instead.

func (*RefreshProjectConfigTask) GetDisable 

func (x *RefreshProjectConfigTask) GetDisable() bool

func (*RefreshProjectConfigTask) GetProject 

func (x *RefreshProjectConfigTask) GetProject() string

func (*RefreshProjectConfigTask) ProtoMessage 

func (*RefreshProjectConfigTask) ProtoMessage()

func (*RefreshProjectConfigTask) ProtoReflect 

func (x *RefreshProjectConfigTask) ProtoReflect() protoreflect.Message

func (*RefreshProjectConfigTask) Reset 

func (x *RefreshProjectConfigTask) Reset()

func (*RefreshProjectConfigTask) String 

func (x *RefreshProjectConfigTask) String() string

type Refresher 

type Refresher struct {
	// contains filtered or unexported fields
}

func NewRefresher 

func NewRefresher(tqd *tq.Dispatcher, pm PM) *Refresher

NewRefresher creates a new project config Refresher and registers its TQ tasks.

func (*Refresher) SubmitRefreshTasks 

func (r *Refresher) SubmitRefreshTasks(ctx context.Context) error

SubmitRefreshTasks submits tasks that update config for LUCI projects or disable projects that do not have CV config in LUCI Config.

It's expected to be called by a cron.

type Status 

type Status int

Status of LUCI Project Config from CV perspective. 

const (
	// StatusNotExists means CV doesn't have config for a LUCI prject.
	//
	// The LUCI project itself may exist in LUCI-Config service.
	StatusNotExists Status = iota
	// StatusDisabled means CV has LUCI project's config, but either CV was
	// disabled for the project or the project was deactivated LUCI-wide.
	StatusDisabled
	// StatusEnabled means CV has LUCI project's config and CV is active for the
	// project.
	StatusEnabled
)

Source Files

View all Source files

Directories

Path Synopsis
Package prjcfgtest eases controlling of project configs in test.
 Package prjcfgtest eases controlling of project configs in test.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.