daisyutils

package
v0.0.0-...-fa603ae Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 18, 2024 License: Apache-2.0 Imports: 20 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// BuildIDOSEnvVarName is the os env var name to get build id
	BuildIDOSEnvVarName = "BUILD_ID"
)

Variables

This section is empty.

Functions

func CheckEnvironment 

func CheckEnvironment(worker DaisyWorker, check func(env EnvironmentSettings))

CheckEnvironment allows a test to check the fields on the EnvironmentSettings associated with a DaisyWorker.

func CheckResourceLabeler 

func CheckResourceLabeler(worker DaisyWorker, check func(rl *ResourceLabeler))

CheckResourceLabeler allows a test to check the fields on the resource labeler associated with a DaisyWorker.

func CheckWorkflow 

func CheckWorkflow(worker DaisyWorker, check func(wf *daisy.Workflow, workflowCreationError error))

CheckWorkflow allows a test to check the fields on the daisy workflow associated with a DaisyWorker.

func GenerateValidDisksImagesName 

func GenerateValidDisksImagesName(name string) string

GenerateValidDisksImagesName generates a valid name for disks/images , It ensures that the generated name length is not greater than 63 by removing characters from the end of the name if the name has ID number at the end (e.g "diskName-id") remove chars before the id

func GetDeviceURI 

func GetDeviceURI(project, zone, name string) string

GetDeviceURI gets a URI for a device based on its attributes. A device is a disk attached to a instance.

func GetDiskURI 

func GetDiskURI(project, zone, name string) string

GetDiskURI gets a URI for a disk based on its attributes. Introduction to a disk resource: https://cloud.google.com/compute/docs/reference/rest/v1/disks

func GetInstanceURI 

func GetInstanceURI(project, zone, name string) string

GetInstanceURI gets a URI for a instance based on its attributes. Introduction to a instance resource: https://cloud.google.com/compute/docs/reference/rest/v1/instances

func GetResourceID 

func GetResourceID(resourceURI string) string

GetResourceID gets resource id from its URI. Definition of resource ID: https://cloud.google.com/apis/design/resource_names#resource_id

func GetSortedOSIDs 

func GetSortedOSIDs() []string

GetSortedOSIDs returns the supported OS identifiers, sorted.

func NewStep 

func NewStep(w *daisy.Workflow, name string, dependencies ...*daisy.Step) (*daisy.Step, error)

NewStep creates a new step for the workflow along with dependencies.

func ParseWorkflow 

func ParseWorkflow(path string, varMap map[string]string, project, zone, gcsPath, oauth, dTimeout string, disableGCSLogs, disableCloudLogs, disableStdoutLogs bool) (*daisy.Workflow, error)

ParseWorkflow parses Daisy workflow file and returns Daisy workflow object or error in case of failure

func PostProcessDErrorForNetworkFlag 

func PostProcessDErrorForNetworkFlag(action string, err error, network string, w *daisy.Workflow)

PostProcessDErrorForNetworkFlag determines whether to show more hints for network flag

func RemovePrivacyLogInfo 

func RemovePrivacyLogInfo(message string) string

RemovePrivacyLogInfo removes privacy log information.

func RemovePrivacyLogTag 

func RemovePrivacyLogTag(message string) string

RemovePrivacyLogTag removes privacy log tag.

func RunWorkflowWithCancelSignal 

func RunWorkflowWithCancelSignal(w *daisy.Workflow, cancel <-chan string) error

RunWorkflowWithCancelSignal runs a Daisy workflow, and allows for cancellation from two sources:

  1. The user types Ctrl-C on their keyboard.
  2. The caller sends a cancellation reason on the cancel channel (or closes it).

func UpdateAllInstanceNoExternalIP 

func UpdateAllInstanceNoExternalIP(workflow *daisy.Workflow, noExternalIP bool)

UpdateAllInstanceNoExternalIP updates all Create Instance steps in the workflow to operate when no external IP access is allowed by the VPC Daisy workflow is running in.

func UpdateToUEFICompatible 

func UpdateToUEFICompatible(workflow *daisy.Workflow)

UpdateToUEFICompatible marks workflow resources (disks and images) to be UEFI compatible by adding "UEFI_COMPATIBLE" to GuestOSFeatures. Debian workers are excluded until UEFI becomes the default boot method.

func ValidateOS 

func ValidateOS(osID string) error

ValidateOS validates that osID is supported by Daisy image import

Types

type ApplyAndValidateVars 

type ApplyAndValidateVars struct {
	// contains filtered or unexported fields
}

ApplyAndValidateVars is a WorkflowHook that applies vars to a daisy workflow. To ensure consistency across worker instances, if vars omits network, subnet, or the compute service account, the modifier will automatically apply these values.

func (*ApplyAndValidateVars) PreRunHook 

func (t *ApplyAndValidateVars) PreRunHook(wf *daisy.Workflow) error

PreRunHook applies daisy vars to a workflow

type ApplyEnvToWorkflow 

type ApplyEnvToWorkflow struct {
	// contains filtered or unexported fields
}

ApplyEnvToWorkflow is a WorkflowHook that applies user-customizable values to the top-level parent workflow.

func (*ApplyEnvToWorkflow) PreRunHook 

func (t *ApplyEnvToWorkflow) PreRunHook(wf *daisy.Workflow) error

PreRunHook updates properties on wf that correspond to user-specified values such as project, zone, and scratch bucket path.

type ByteCountingReader 

type ByteCountingReader struct {
	BytesRead int64
	// contains filtered or unexported fields
}

ByteCountingReader forwards calls to a delegate reader, keeping track of the number bytes that have been read in `BytesRead`. Errors are propagated unchanged.

func NewByteCountingReader 

func NewByteCountingReader(r io.Reader) *ByteCountingReader

NewByteCountingReader is a contructor for ByteCountingReader.

func (*ByteCountingReader) Read 

func (l *ByteCountingReader) Read(p []byte) (n int, err error)

type ConfigureDaisyLogging 

type ConfigureDaisyLogging struct {
	// contains filtered or unexported fields
}

ConfigureDaisyLogging is a WorkflowHook that configures Daisy's logging settings using the user's logging preferences, specified in EnvironmentSettings.

func (*ConfigureDaisyLogging) PreRunHook 

func (t *ConfigureDaisyLogging) PreRunHook(wf *daisy.Workflow) error

PreRunHook applies the user's logging preferences to a daisy workflow.

type DaisyWorker 

type DaisyWorker interface {
	Run(vars map[string]string) error
	RunAndReadSerialValue(key string, vars map[string]string) (string, error)
	RunAndReadSerialValues(vars map[string]string, keys ...string) (map[string]string, error)
	Cancel(reason string) bool
}

DaisyWorker is a facade over daisy.Workflow to facilitate mocking.

func NewDaisyWorker 

func NewDaisyWorker(wf WorkflowProvider, env EnvironmentSettings,
	logger logging.Logger, hooks ...interface{}) DaisyWorker

NewDaisyWorker returns an implementation of DaisyWorker. The returned value is designed to be run once and discarded. In other words, don't run the same instance twice.

hooks contains additional WorkflowPreHook or WorkflowPostHook instances. If hooks doesn't include a resource labeler, one will be created.

type DiskLabelKeyRetrieverFunc 

type DiskLabelKeyRetrieverFunc func(disk *daisy.Disk) string

DiskLabelKeyRetrieverFunc returns GCE label key to be added to given disk

type EnableNestedVirtualizationHook 

type EnableNestedVirtualizationHook struct{}

EnableNestedVirtualizationHook is a WorkflowHook that updates CreateInstances in a daisy workflow such that they will be created with nested virtualization enabled.

For more info on nested virtualization see: 

https://cloud.google.com/compute/docs/instances/nested-virtualization/overview

func (*EnableNestedVirtualizationHook) PreRunHook 

func (t *EnableNestedVirtualizationHook) PreRunHook(wf *daisy.Workflow) error

PreRunHook updates the CreateInstances steps so that they will be created with nested virtualization enabled.

type EndpointsOverride 

type EndpointsOverride struct {
	Compute      string `json:",omitempty"`
	Storage      string `json:",omitempty"`
	CloudLogging string `json:",omitempty"`
}

EndpointsOverride is a type for google cloud APIs that may be overridden

type EnvironmentSettings 

type EnvironmentSettings struct {
	// Location of workflows
	WorkflowDirectory string

	// Fields from daisy.Workflow
	Project, Zone, GCSPath, OAuth, Timeout              string
	DisableGCSLogs, DisableCloudLogs, DisableStdoutLogs bool

	EndpointsOverride EndpointsOverride

	// An optional prefix to include in the bracketed portion of daisy's stdout logs.
	// Gcloud does a prefix match to determine whether to show a log line to a user.
	//
	// With a prefix of `disk-1`, for example, the workflow in `importer.NewDaisyInflater`
	// emits log messages starting with `[disk-1-inflate]`.
	DaisyLogLinePrefix string

	// Worker instance customizations
	Network, Subnet             string
	ComputeServiceAccount       string
	NoExternalIP                bool
	Labels                      map[string]string
	ExecutionID                 string
	StorageLocation             string
	Tool                        Tool
	NestedVirtualizationEnabled bool
	WorkerMachineSeries         []string
}

EnvironmentSettings controls the resources that are used during tool execution.

func (EnvironmentSettings) ApplyToWorkflow 

func (env EnvironmentSettings) ApplyToWorkflow(w *daisy.Workflow)

ApplyToWorkflow sets fields on daisy.Workflow from the environment settings.

type FallbackToPDStandard 

type FallbackToPDStandard struct {
	// contains filtered or unexported fields
}

FallbackToPDStandard detects if a workflow fails due to insufficient SSD quota. If so, it re-runs the workflow and modifies the disks to use PD Standard.

func (*FallbackToPDStandard) PostRunHook 

func (f *FallbackToPDStandard) PostRunHook(err error) (wantRetry bool, wrapped error)

PostRunHook inspects the workflow error to see if it's related to insufficient SSD quota. If so, it requests a retry that will re-run the workflow using standard disks.

func (*FallbackToPDStandard) PreRunHook 

func (f *FallbackToPDStandard) PreRunHook(wf *daisy.Workflow) error

PreRunHook modifies the workflow to use standard disks if `shouldFallback` is true.

type ImageLabelKeyRetrieverFunc 

type ImageLabelKeyRetrieverFunc func(imageName string) string

ImageLabelKeyRetrieverFunc returns GCE label key to be added to given image

type InstanceLabelKeyRetrieverFunc 

type InstanceLabelKeyRetrieverFunc func(instanceName string) string

InstanceLabelKeyRetrieverFunc returns GCE label key to be added to given instance

type RemoveExternalIPHook 

type RemoveExternalIPHook struct{}

RemoveExternalIPHook is a WorkflowHook that updates CreateInstances in a daisy workflow such that they won't be created with an external IP address.

For more info on external IPs, see the `--no-address` flag: 

https://cloud.google.com/sdk/gcloud/reference/compute/instances/create

func (*RemoveExternalIPHook) PreRunHook 

func (t *RemoveExternalIPHook) PreRunHook(wf *daisy.Workflow) error

PreRunHook updates the CreateInstances steps so that they won't have an external IP.

type ResourceLabeler 

type ResourceLabeler struct {
	BuildID                   string
	UserLabels                map[string]string
	BuildIDLabelKey           string
	ImageLocation             string
	InstanceLabelKeyRetriever InstanceLabelKeyRetrieverFunc
	DiskLabelKeyRetriever     DiskLabelKeyRetrieverFunc
	ImageLabelKeyRetriever    ImageLabelKeyRetrieverFunc
}

ResourceLabeler is responsible for labelling GCE resources (instances, disks and images) with labels used to track resource creation by import processes.

func NewResourceLabeler 

func NewResourceLabeler(tool string, buildID string, labels map[string]string, location string) *ResourceLabeler

NewResourceLabeler creates a ResourceLabeler that labels based on the tool's name.

func (*ResourceLabeler) LabelResources 

func (rl *ResourceLabeler) LabelResources(workflow *daisy.Workflow)

LabelResources labels workflow resources temporary and permanent resources with appropriate labels

func (*ResourceLabeler) PreRunHook 

func (rl *ResourceLabeler) PreRunHook(wf *daisy.Workflow) error

PreRunHook is a facade over LabelResources to implement the WorkflowHook interface; no errors will be returned.

type Tool 

type Tool struct {

	// HumanReadableName is used for error messages, for example: "image import".
	HumanReadableName string
	// ResourceLabelName is used when labeling temporary resources, for example: "image-import"
	ResourceLabelName string
}

Tool is used to communicate the tool's name ot the user.

type TranslationSettings 

type TranslationSettings struct {
	// GcloudOsFlag is the user-facing string corresponding to this OS, version, and licensing mode.
	// It is passed as a value of the `--os` flag.
	GcloudOsFlag string

	// LicenseURI is the GCP Compute license corresponding to this OS, version, and licensing mode:
	//  https://cloud.google.com/compute/docs/reference/rest/v1/licenses
	LicenseURI string

	// WorkflowPath is the path to a Daisy json workflow, relative to the
	// `daisy_workflows/image_import` directory.
	WorkflowPath string
}

TranslationSettings includes information that needs to be added to a disk or image after it is imported, for a particular OS and version.

func GetTranslationSettings 

func GetTranslationSettings(osID string) (spec TranslationSettings, err error)

GetTranslationSettings returns parameters required for translating a particular OS, version, and licensing mode to run on GCE.

An error is returned if the OS, version, and licensing mode is not supported for import.

type UpdateMachineTypesHook 

type UpdateMachineTypesHook struct {
	// contains filtered or unexported fields
}

UpdateMachineTypesHook updates the workflow to use the machine series specified as primary. If the workflow fails due to the usage quota then it falls back to secondary machine series. See cli_tools/common/utils/param/machine_series_detector.go for details.

func (*UpdateMachineTypesHook) PostRunHook 

func (f *UpdateMachineTypesHook) PostRunHook(err error) (wantRetry bool, wrapped error)

PostRunHook inspects the workflow error to see if it's related to an insufficient primary machine series CPUs quota. If so, it requests a retry that will re-run the workflow using the secondary machine series.

func (*UpdateMachineTypesHook) PreRunHook 

func (f *UpdateMachineTypesHook) PreRunHook(wf *daisy.Workflow) error

PreRunHook modifies the workflow to use machine series specified as primary or falls back to secondary machine series when needed.

type WorkflowPostHook 

type WorkflowPostHook interface {
	// PostRunHook allows a WorkflowHook to inspect the workflow's run error, and optionally
	// decide whether to retry the workflow, or to wrap the error to expose a more useful
	// error message.
	PostRunHook(err error) (wantRetry bool, wrapped error)
}

WorkflowPostHook executes after the workflow runs.

type WorkflowPreHook 

type WorkflowPreHook interface {
	// PreRunHook allows a WorkflowHook to modify a workflow prior to running.
	PreRunHook(wf *daisy.Workflow) error
}

WorkflowPreHook executes before the workflow runs.

type WorkflowProvider 

type WorkflowProvider func() (*daisy.Workflow, error)

WorkflowProvider returns a new instance of a Daisy workflow.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.