units

package
v0.0.0-...-f52f6c5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 26, 2024 License: Apache-2.0 Imports: 64 Imported by: 16

Documentation

Overview

Package units contains amboy.Job definitions.

Loading the units package registers all jobs in the amboy Job Registry.

By convention the implementations of these jobs are: private with public constructors that return the amboy.Job type, they use amboy/job.Base for core implementation, implementing only Run and whatever additional methods or functions are required. The units package prefers a one-to-one mapping of files to job implementations. Additionally all jobs must be capable of marshalling to JSON/BSON (depending on the amoby.JobType format.)

Index

Constants

View Source 
const (
	// GitHub intent processing errors
	ProjectDisabled  = "project was disabled"
	PatchingDisabled = "patching was disabled"

	PatchTaskSyncDisabled = "task sync was disabled for patches"

	NoTasksOrVariants = "no tasks/variants were configured"

	NoSyncTasksOrVariants  = "no tasks/variants were configured for sync"
	GitHubInternalError    = "GitHub returned an error"
	InvalidConfig          = "config file was invalid"
	EmptyConfig            = "config file was empty"
	ProjectFailsValidation = "Project fails validation"
	OtherErrors            = "Evergreen error"
	MergeBaseTooOld        = "merge base is too old"
)
View Source 
const (
	HostTerminationJobName = "host-termination-job"
)
View Source 
const (
	TSFormat = "2006-01-02.15-04-05"
)

Variables

This section is empty.

Functions

func AddMergeTaskAndVariant 

func AddMergeTaskAndVariant(ctx context.Context, patchDoc *patch.Patch, project *model.Project, projectRef *model.ProjectRef, source string) (*model.ParserProject, error)

AddMergeTaskAndVariant adds the merge task, the merge task's dependencies, and the merge build variant to the project and returns the modified parser project.

func CreateAndEnqueueGenerateTasks 

func CreateAndEnqueueGenerateTasks(ctx context.Context, env evergreen.Environment, tasks []task.Task, ts string) error

CreateAndEnqueueGenerateTasks enqueues a generate.tasks job for each task. Jobs are segregated by task version into separate queues.

func DisableAndNotifyPoisonedHost 

func DisableAndNotifyPoisonedHost(ctx context.Context, env evergreen.Environment, h *host.Host, reason string) error

func EnqueueHostCreateJobs 

func EnqueueHostCreateJobs(ctx context.Context, env evergreen.Environment, hostIntents []host.Host) error

func EnqueueHostReprovisioningJob 

func EnqueueHostReprovisioningJob(ctx context.Context, env evergreen.Environment, h *host.Host) error

EnqueueHostReprovisioningJob enqueues a job to reprovision a host. For hosts that do not need to reprovision, this is a no-op.

func EnqueueSpawnHostModificationJob 

func EnqueueSpawnHostModificationJob(ctx context.Context, env evergreen.Environment, j amboy.Job) error

EnqueueSpawnHostModificationJob enqueues a job to modify a spawn host.

func HandlePoisonedHost 

func HandlePoisonedHost(ctx context.Context, env evergreen.Environment, h *host.Host, reason string) error

func NewAgentDeployJob 

func NewAgentDeployJob(env evergreen.Environment, h host.Host, id string) amboy.Job

func NewAgentMonitorDeployJob 

func NewAgentMonitorDeployJob(env evergreen.Environment, h host.Host, id string) amboy.Job

NewAgentMonitorDeployJob creates a job that deploys the agent monitor to the host.

func NewBuildingContainerImageJob 

func NewBuildingContainerImageJob(env evergreen.Environment, h *host.Host, dockerOptions host.DockerOptions, providerName string) amboy.Job

func NewCacheHistoricalTaskDataJob 

func NewCacheHistoricalTaskDataJob(id, projectID string) amboy.Job

func NewCheckBlockedTasksJob 

func NewCheckBlockedTasksJob(distroId string, ts time.Time) amboy.Job

NewCheckBlockedTasksJob creates a job to audit the dependency state for tasks in the task queues. If it finds any mismatches in dependency state, it fixes them.

func NewCloudCleanupJob 

func NewCloudCleanupJob(env evergreen.Environment, ts, provider, region string) amboy.Job

NewCloudCleanupJob returns a job to call the cloud manager's Cleanup method for the given Provider and Region.

func NewCloudHostReadyJob 

func NewCloudHostReadyJob(env evergreen.Environment, id string) amboy.Job

NewCloudHostReadyJob checks the cloud instance status for all hosts created by cloud providers when the instance is not yet ready to be used (e.g. the instance is still booting up). Once the cloud instance status is resolved, the job can either transition the host into the next step in the host lifecycle or be appropriately handled if it is in an unrecoverable state.

func NewCloudUpdateSSHKeysJob 

func NewCloudUpdateSSHKeysJob(provider, region, id string) amboy.Job

NewCloudUpdateSSHKeysJob updates the SSH keys available for a single region in the given cloud provider.

func NewCommitQueueJob 

func NewCommitQueueJob(env evergreen.Environment, queueID string, id string) amboy.Job

func NewContainerSecretCleanupJob 

func NewContainerSecretCleanupJob(id string) amboy.Job

NewContainerSecretCleanupJob creates a job that cleans up stranded container secrets that are not tracked by Evergreen.

func NewConvertHostToLegacyProvisioningJob 

func NewConvertHostToLegacyProvisioningJob(env evergreen.Environment, h host.Host, id string, attempt int) amboy.Job

NewConvertHostToLegacyProvisioningJob converts a host from a legacy provisioned host to a non-legacy provisioned host.

func NewConvertHostToNewProvisioningJob 

func NewConvertHostToNewProvisioningJob(env evergreen.Environment, h host.Host, id string, attempt int) amboy.Job

NewConvertHostToNewProvisioningJob converts a host from a legacy provisioned host to a non-legacy provisioned host.

func NewCronRemoteFifteenMinuteJob 

func NewCronRemoteFifteenMinuteJob() amboy.Job

func NewCronRemoteFifteenSecondJob 

func NewCronRemoteFifteenSecondJob() amboy.Job

func NewCronRemoteFiveMinuteJob 

func NewCronRemoteFiveMinuteJob() amboy.Job

func NewCronRemoteHourJob 

func NewCronRemoteHourJob() amboy.Job

func NewCronRemoteMinuteJob 

func NewCronRemoteMinuteJob() amboy.Job

func NewDecoHostNotifyJob 

func NewDecoHostNotifyJob(env evergreen.Environment, h *host.Host, err error, message string) amboy.Job

func NewDistroAliasSchedulerJob 

func NewDistroAliasSchedulerJob(distroID string, id string) amboy.Job

func NewDistroSchedulerJob 

func NewDistroSchedulerJob(distroID string, id string) amboy.Job

func NewDuplicateTaskCheckJob 

func NewDuplicateTaskCheckJob(id string) amboy.Job

NewDuplicateTaskCheckJob checks for any tasks that appear in multiple primary task queues.

func NewEventNotifierJob 

func NewEventNotifierJob(env evergreen.Environment, eventID, ts string) amboy.Job

func NewEventSendJob 

func NewEventSendJob(id, ts string) amboy.Job

func NewGenerateTasksJob 

func NewGenerateTasksJob(versionID, taskID string, ts string) amboy.Job

NewGenerateTasksJob returns a job that dynamically updates the project configuration based on the given task's generate.tasks configuration.

func NewGithubStatusRefreshJob 

func NewGithubStatusRefreshJob(p *patch.Patch) amboy.Job

NewGithubStatusRefreshJob is a job that re-sends github statuses to the PR associated with the given patch.

func NewGithubStatusUpdateJobForDeleteFromCommitQueue 

func NewGithubStatusUpdateJobForDeleteFromCommitQueue(owner, repo, ref string, prNumber int) amboy.Job

func NewGithubStatusUpdateJobForExternalPatch 

func NewGithubStatusUpdateJobForExternalPatch(patchID string) amboy.Job

NewGithubStatusUpdateJobForExternalPatch prompts on Github for a user to manually authorize this patch

func NewGithubStatusUpdateJobForNewPatch 

func NewGithubStatusUpdateJobForNewPatch(patchID string) amboy.Job

NewGithubStatusUpdateJobForNewPatch creates a job to update github's API for a newly created patch, reporting it as pending, with description "preparing to run tasks"

func NewGithubStatusUpdateJobForProcessingError 

func NewGithubStatusUpdateJobForProcessingError(githubContext, owner, repo, ref, description string) amboy.Job

NewGithubStatusUpdateJobForProcessingError marks a ref as failed because the evergreen encountered an error creating a patch

func NewGithubStatusUpdateJobForPushToCommitQueue 

func NewGithubStatusUpdateJobForPushToCommitQueue(owner, repo, ref string, prNumber int, patchId string) amboy.Job

func NewGithubStatusUpdateJobWithSuccessMessage 

func NewGithubStatusUpdateJobWithSuccessMessage(githubContext, owner, repo, ref, description string) amboy.Job

NewGithubStatusUpdateJobWithSuccessMessage creates a job to send a passing status to Github with a message.

func NewHostAllocatorJob 

func NewHostAllocatorJob(env evergreen.Environment, distroID string, timestamp time.Time) amboy.Job

func NewHostCreateJob 

func NewHostCreateJob(env evergreen.Environment, h host.Host, id string, currentAttempt int, buildImageStarted bool) amboy.Job

func NewHostDrawdownJob 

func NewHostDrawdownJob(env evergreen.Environment, drawdownInfo DrawdownInfo, ts string) amboy.Job

func NewHostExecuteJob 

func NewHostExecuteJob(env evergreen.Environment, h host.Host, script string, sudo bool, sudoUser string, id string) amboy.Job

NewHostExecuteJob creates a job that executes a script on the host.

func NewHostMonitorContainerStateJob 

func NewHostMonitorContainerStateJob(env evergreen.Environment, h *host.Host, providerName, id string) amboy.Job

func NewHostMonitorExternalStateJob 

func NewHostMonitorExternalStateJob(env evergreen.Environment, h *host.Host, id string) amboy.Job

func NewHostSetupScriptJob 

func NewHostSetupScriptJob(env evergreen.Environment, h *host.Host) amboy.Job

NewHostSetupScriptJob creates a job that executes the spawn host's setup script after task data is loaded onto the host.

func NewHostStatsCollector 

func NewHostStatsCollector(id string) amboy.Job

NewHostStatsCollector logs statistics about host utilization per distro to the default grip logger.

func NewHostStatsJob 

func NewHostStatsJob(ts string) amboy.Job

func NewHostTerminationJob 

func NewHostTerminationJob(env evergreen.Environment, h *host.Host, opts HostTerminationOptions) amboy.Job

func NewIdleHostTerminationJob 

func NewIdleHostTerminationJob(env evergreen.Environment, id string) amboy.Job

func NewJasperManagerCleanup 

func NewJasperManagerCleanup(id string, env evergreen.Environment) amboy.Job

NewJasperManagerCleanup reports basic system information and a report of the go runtime information, as provided by grip.

func NewLastContainerFinishTimeJob 

func NewLastContainerFinishTimeJob(id string) amboy.Job

func NewLocalAmboyStatsCollector 

func NewLocalAmboyStatsCollector(env evergreen.Environment, id string) amboy.Job

NewLocalAmboyStatsCollector reports the status of only the local queue registered in the evergreen service Environment.

func NewLocalUpdateSSHKeysJob 

func NewLocalUpdateSSHKeysJob(id string) amboy.Job

NewLocalUpdateSSHKeysJob updates the SSH key files locally.

func NewNotificationStatsCollector 

func NewNotificationStatsCollector(id string) amboy.Job

func NewOldestImageRemovalJob 

func NewOldestImageRemovalJob(h *host.Host, providerName, id string) amboy.Job

func NewParentDecommissionJob 

func NewParentDecommissionJob(id, d string, maxContainers int) amboy.Job

func NewPatchIntentProcessor 

func NewPatchIntentProcessor(env evergreen.Environment, patchID mgobson.ObjectId, intent patch.Intent) amboy.Job

NewPatchIntentProcessor creates an amboy job to create a patch from the given patch intent. The patch ID is the new ID for the patch to be created, not the patch intent.

func NewPeriodicBuildJob 

func NewPeriodicBuildJob(projectID, definitionID string) amboy.Job

func NewPodAllocatorJob 

func NewPodAllocatorJob(taskID, ts string) amboy.Job

NewPodAllocatorJob returns a job to allocate a pod for the given task ID.

func NewPodCreationJob 

func NewPodCreationJob(podID, id string) amboy.Job

NewPodCreationJob creates a job that starts the pod in the container service.

func NewPodDefinitionCleanupJob 

func NewPodDefinitionCleanupJob(id string) amboy.Job

NewPodDefinitionCleanupJob creates a job that cleans up pod definitions that have not been used recently.

func NewPodDefinitionCreationJob 

func NewPodDefinitionCreationJob(ecsConf evergreen.ECSConfig, opts pod.TaskContainerCreationOptions, id string) amboy.Job

NewPodDefinitionCreationJob creates a job that creates a pod definition in preparation for running a pod.

func NewPodHealthCheckJob 

func NewPodHealthCheckJob(podID string, ts time.Time) amboy.Job

NewPodHealthCheckJob creates a job to check the health of the pod according to the cloud provider.

func NewPodStatsCollector 

func NewPodStatsCollector(id string) amboy.Job

NewPodStatsCollector logs statistics about current pod usage.

func NewPodTerminationJob 

func NewPodTerminationJob(podID, reason string, ts time.Time) amboy.Job

NewPodTerminationJob creates a job to terminate the given pod with the given termination reason. Callers should populate the reason with as much context as possible for why the pod is being terminated.

func NewQueueStatsCollector 

func NewQueueStatsCollector(id string) amboy.Job

func NewReauthorizeUserJob 

func NewReauthorizeUserJob(env evergreen.Environment, u *user.DBUser, id string) amboy.Job

NewReauthorizeUserJob returns a job that attempts to reauthorize the given user.

func NewRemoteAmboyStatsCollector 

func NewRemoteAmboyStatsCollector(env evergreen.Environment, id string) amboy.Job

NewRemoteAmboyStatsCollector reports the status of only the remote queue registered in the evergreen service Environment.

func NewRepotrackerJob 

func NewRepotrackerJob(msgID, projectID string) amboy.Job

NewRepotrackerJob creates a job to run repotracker against a repository. The code creating this job is responsible for verifying that the project should track push events

func NewRestartJasperJob 

func NewRestartJasperJob(env evergreen.Environment, h host.Host, ts string) amboy.Job

NewRestartJasperJob creates a job that restarts an existing Jasper service with new credentials.

func NewSetupHostJob 

func NewSetupHostJob(env evergreen.Environment, h *host.Host, id string) amboy.Job

NewSetupHostJob creates a job that performs any additional provisioning for a host to prepare it to run.

func NewSleepSchedulerJob 

func NewSleepSchedulerJob(env evergreen.Environment, ts string) amboy.Job

NewSleepSchedulerJob creates a job to manage unexpirable host sleep schedules.

func NewSpawnHostTerminationJob 

func NewSpawnHostTerminationJob(h *host.Host, user, ts string) amboy.Job

NewSpawnHostTerminationJob returns a job to terminate a spawn host.

func NewSpawnhostExpirationCheckJob 

func NewSpawnhostExpirationCheckJob(ts string, h *host.Host) amboy.Job

func NewSpawnhostExpirationWarningsJob 

func NewSpawnhostExpirationWarningsJob(id string) amboy.Job

func NewSpawnhostModifyJob 

func NewSpawnhostModifyJob(h *host.Host, changes host.HostModifyOptions, ts string) amboy.Job

func NewSpawnhostStartJob 

func NewSpawnhostStartJob(h *host.Host, source evergreen.ModifySpawnHostSource, user, ts string) amboy.Job

NewSpawnhostStartJob returns a job to start a stopped spawn host.

func NewSpawnhostStopJob 

func NewSpawnhostStopJob(h *host.Host, shouldKeepOff bool, source evergreen.ModifySpawnHostSource, user, ts string) amboy.Job

NewSpawnhostStopJob returns a job to stop a running spawn host.

func NewStaticUpdateSSHKeysJob 

func NewStaticUpdateSSHKeysJob(h host.Host, id string) amboy.Job

NewStaticUpdateSSHKeysJob updates the SSH keys for a static host.

func NewStrandedTaskCleanupJob 

func NewStrandedTaskCleanupJob(id string) amboy.Job

NewStrandedTaskCleanupJob returns a job to detect and clean up tasks that: - Have been stranded on hosts that are already terminated. - Have stuck dispatching for too long.

func NewSysInfoStatsCollector 

func NewSysInfoStatsCollector(id string) amboy.Job

NewSysInfoStatsCollector reports basic system information and a report of the go runtime information, as provided by grip.

func NewTaskExecutionMonitorJob 

func NewTaskExecutionMonitorJob(taskID string, ts string) amboy.Job

NewTaskExecutionMonitorJob returns a job to check if a running task has failed to send a heartbeat recently. If it has timed out, it is cleaned up.

func NewTaskExecutionMonitorPopulateJob 

func NewTaskExecutionMonitorPopulateJob(id string) amboy.Job

NewTaskExecutionMonitorPopulateJob returns a job to populate the queue with jobs to check for stale tasks.

func NewTaskStatsCollector 

func NewTaskStatsCollector(id string) amboy.Job

NewTaskStatsCollector captures a single report of the status of tasks that have completed in the last minute.

func NewTasksRestartJob 

func NewTasksRestartJob(opts model.RestartOptions) amboy.Job

NewTasksRestartJob creates a job to restart failed tasks in a time range

func NewUnexpirableSpawnHostStatsJob 

func NewUnexpirableSpawnHostStatsJob(ts string) amboy.Job

NewUnexpirableSpawnHostStatsJob returns a job to collect estimated statistics on unexpirable spawn host usage.

func NewUserDataDoneJob 

func NewUserDataDoneJob(env evergreen.Environment, hostID string, ts time.Time) amboy.Job

NewUserDataDoneJob creates a job that checks if the host is done provisioning with user data (if bootstrapped with user data). This check only applies to spawn hosts, since hosts running agents check into the server to verify their liveliness.

func NewVersionActivationJob 

func NewVersionActivationJob(ts string) amboy.Job

func NewVolumeDeletionJob 

func NewVolumeDeletionJob(ts string, v *host.Volume) amboy.Job

func NewVolumeExpirationCheckJob 

func NewVolumeExpirationCheckJob(ts string, v *host.Volume, provider string) amboy.Job

func NewVolumeExpirationWarningsJob 

func NewVolumeExpirationWarningsJob(id string) amboy.Job

func NewVolumeMigrationJob 

func NewVolumeMigrationJob(env evergreen.Environment, volumeID string, modifyOptions cloud.SpawnOptions, ts string) amboy.Job

func NewVolumeUnstickJob 

func NewVolumeUnstickJob(ts string, v *host.Volume) amboy.Job

func PopulateActivationJobs 

func PopulateActivationJobs(part int) amboy.QueueOperation

func PopulateCacheHistoricalTaskDataJob 

func PopulateCacheHistoricalTaskDataJob(part int) amboy.QueueOperation

func PopulateCheckUnmarkedBlockedTasks 

func PopulateCheckUnmarkedBlockedTasks() amboy.QueueOperation

func PopulateCloudCleanupJob 

func PopulateCloudCleanupJob(env evergreen.Environment) amboy.QueueOperation

PopulateCloudCleanupJob returns a QueueOperation to enqueue a CloudCleanup job for Fleet in the default EC2 region.

func PopulateDuplicateTaskCheckJobs 

func PopulateDuplicateTaskCheckJobs() amboy.QueueOperation

func PopulateHostProvisioningConversionJobs 

func PopulateHostProvisioningConversionJobs(env evergreen.Environment) amboy.QueueOperation

PopulateHostProvisioningConversionJobs enqueues the jobs to convert the host provisioning type.

func PopulateHostRestartJasperJobs 

func PopulateHostRestartJasperJobs(env evergreen.Environment) amboy.QueueOperation

PopulateHostRestartJasperJobs enqueues the jobs to restart the Jasper service on the host.

func PopulateHostStatJobs 

func PopulateHostStatJobs(parts int) amboy.QueueOperation

PopulateHostStatJobs adds host stats jobs.

func PopulateLocalQueueJobs 

func PopulateLocalQueueJobs(env evergreen.Environment) amboy.QueueOperation

func PopulatePeriodicBuilds 

func PopulatePeriodicBuilds() amboy.QueueOperation

func PopulatePodHealthCheckJobs 

func PopulatePodHealthCheckJobs() amboy.QueueOperation

PopulatePodHealthCheckJobs enqueues the jobs to check pods that have not checked in recently to determine if they are still healthy.

func PopulatePodResourceCleanupJobs 

func PopulatePodResourceCleanupJobs() amboy.QueueOperation

PopulatePodResourceCleanupJobs populates the jobs to clean up pod resources.

func PopulateReauthorizeUserJobs 

func PopulateReauthorizeUserJobs(env evergreen.Environment) amboy.QueueOperation

func PopulateSSHKeyUpdates 

func PopulateSSHKeyUpdates(env evergreen.Environment) amboy.QueueOperation

PopulateSSHKeyUpdates updates the remote SSH keys in the cloud providers and static hosts.

func PopulateSpawnhostExpirationCheckJob 

func PopulateSpawnhostExpirationCheckJob() amboy.QueueOperation

func PopulateTaskMonitoring 

func PopulateTaskMonitoring(mins int) amboy.QueueOperation

func PopulateUnexpirableSpawnHostStatsJob 

func PopulateUnexpirableSpawnHostStatsJob() amboy.QueueOperation

PopulateUnexpirableSpawnHostStatsJob populates jobs to collect statistics on unexpirable spawn host usage.

func PopulateUnstickVolumesJob 

func PopulateUnstickVolumesJob() amboy.QueueOperation

PopulateUnstickVolumesJob looks for volumes that are marked as attached to terminated hosts in our DB, and enqueues jobs to mark them unattached.

func PopulateVolumeExpirationCheckJob 

func PopulateVolumeExpirationCheckJob() amboy.QueueOperation

func PopulateVolumeExpirationJob 

func PopulateVolumeExpirationJob() amboy.QueueOperation

func ProcessTriggerAliases 

func ProcessTriggerAliases(ctx context.Context, p *patch.Patch, projectRef *model.ProjectRef, env evergreen.Environment, aliasNames []string) error

func SchedulePatch 

func SchedulePatch(ctx context.Context, env evergreen.Environment, patchId string, version *model.Version, patchUpdateReq model.PatchUpdate) (int, error)

SchedulePatch schedules a patch. It returns an error and an HTTP status code.

Types

type CloudHostModification 

type CloudHostModification struct {
	HostID string                          `bson:"host_id" json:"host_id" yaml:"host_id"`
	UserID string                          `bson:"user_id" json:"user_id" yaml:"user_id"`
	Source evergreen.ModifySpawnHostSource `bson:"source" json:"source" yaml:"source"`
	// contains filtered or unexported fields
}

CloudHostModification is a helper to perform cloud manager operations on a single host.

type DrawdownInfo 

type DrawdownInfo struct {
	DistroID     string `bson:"distro_id" json:"distro_id" yaml:"distro_id"`
	NewCapTarget int    `bson:"new_cap_target" json:"new_cap_target" yaml:"new_cap_target"`
}

type HostTerminationOptions 

type HostTerminationOptions struct {
	// TerminateIfBusy, if set, will terminate a host even if it's currently
	// running a task. Otherwise, if it's running a task, termination will
	// either refuse to terminate the host or will reset the task.
	TerminateIfBusy bool `bson:"terminate_if_busy,omitempty" json:"terminate_if_busy,omitempty"`
	// SkipCloudHostTermination, if set, will skip terminating the host in the
	// cloud. The host will still be marked terminated in the DB.
	SkipCloudHostTermination bool `bson:"skip_cloud_host_termination,omitempty" json:"skip_cloud_host_termination,omitempty"`
	// TerminationReason is the reason that the host was terminated.
	TerminationReason string `bson:"termination_reason,omitempty" json:"termination_reason,omitempty"`
}

HostTerminationOptions represent options to control how a host is terminated.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.