Documentation

Overview

Package tasks contains definition of task queue tasks used by the mapper.

Index

Constants

This section is empty.

Variables

View Source 
var File_go_chromium_org_luci_appengine_mapper_internal_tasks_tasks_proto protoreflect.FileDescriptor

Functions

This section is empty.

Types

type FanOutShards 

type FanOutShards struct {
	JobId int64 `protobuf:"varint,1,opt,name=job_id,json=jobId,proto3" json:"job_id,omitempty"`
	// contains filtered or unexported fields
}

FanOutShards enqueues a bunch of ProcessShard named tasks (one per shard).

Enqueued transactionally by SplitAndLaunch after it has constructed shards.

func (*FanOutShards) Descriptor 

func (*FanOutShards) Descriptor() ([]byte, []int)

Deprecated: Use FanOutShards.ProtoReflect.Descriptor instead.

func (*FanOutShards) GetJobId 

func (x *FanOutShards) GetJobId() int64

func (*FanOutShards) ProtoMessage 

func (*FanOutShards) ProtoMessage()

func (*FanOutShards) ProtoReflect 

func (x *FanOutShards) ProtoReflect() protoreflect.Message

func (*FanOutShards) Reset 

func (x *FanOutShards) Reset()

func (*FanOutShards) String 

func (x *FanOutShards) String() string

type ProcessShard 

type ProcessShard struct {
	JobId   int64 `protobuf:"varint,1,opt,name=job_id,json=jobId,proto3" json:"job_id,omitempty"`
	ShardId int64 `protobuf:"varint,2,opt,name=shard_id,json=shardId,proto3" json:"shard_id,omitempty"`
	TaskNum int64 `protobuf:"varint,3,opt,name=task_num,json=taskNum,proto3" json:"task_num,omitempty"`
	// contains filtered or unexported fields
}

ProcessShard sequentially reads the entities belonging to a key range assigned to a shard and applies the mapper to them.

Upon reaching 1 min mark, relaunches itself, increasing task_num. Thus ProcessShard is actually a chain of tasks that runs as long as needed to completely process the shard.

func (*ProcessShard) Descriptor 

func (*ProcessShard) Descriptor() ([]byte, []int)

Deprecated: Use ProcessShard.ProtoReflect.Descriptor instead.

func (*ProcessShard) GetJobId 

func (x *ProcessShard) GetJobId() int64

func (*ProcessShard) GetShardId 

func (x *ProcessShard) GetShardId() int64

func (*ProcessShard) GetTaskNum 

func (x *ProcessShard) GetTaskNum() int64

func (*ProcessShard) ProtoMessage 

func (*ProcessShard) ProtoMessage()

func (*ProcessShard) ProtoReflect 

func (x *ProcessShard) ProtoReflect() protoreflect.Message

func (*ProcessShard) Reset 

func (x *ProcessShard) Reset()

func (*ProcessShard) String 

func (x *ProcessShard) String() string

type RequestJobStateUpdate 

type RequestJobStateUpdate struct {
	JobId   int64 `protobuf:"varint,1,opt,name=job_id,json=jobId,proto3" json:"job_id,omitempty"`
	ShardId int64 `protobuf:"varint,2,opt,name=shard_id,json=shardId,proto3" json:"shard_id,omitempty"` // mostly FYI
	// contains filtered or unexported fields
}

RequestJobStateUpdate is transactionally emitted by ProcessShard when shard's state changes.

It eventually (with some throttling) causes UpdateJobState to be emitted, which updates the job state based on states of the shards.

func (*RequestJobStateUpdate) Descriptor 

func (*RequestJobStateUpdate) Descriptor() ([]byte, []int)

Deprecated: Use RequestJobStateUpdate.ProtoReflect.Descriptor instead.

func (*RequestJobStateUpdate) GetJobId 

func (x *RequestJobStateUpdate) GetJobId() int64

func (*RequestJobStateUpdate) GetShardId 

func (x *RequestJobStateUpdate) GetShardId() int64

func (*RequestJobStateUpdate) ProtoMessage 

func (*RequestJobStateUpdate) ProtoMessage()

func (*RequestJobStateUpdate) ProtoReflect 

func (x *RequestJobStateUpdate) ProtoReflect() protoreflect.Message

func (*RequestJobStateUpdate) Reset 

func (x *RequestJobStateUpdate) Reset()

func (*RequestJobStateUpdate) String 

func (x *RequestJobStateUpdate) String() string

type SplitAndLaunch 

type SplitAndLaunch struct {
	JobId int64 `protobuf:"varint,1,opt,name=job_id,json=jobId,proto3" json:"job_id,omitempty"`
	// contains filtered or unexported fields
}

SplitAndLaunch task splits the key range into shards and kicks off processing of each individual shard.

Enqueued transactionally when creating a new mapping job.

func (*SplitAndLaunch) Descriptor 

func (*SplitAndLaunch) Descriptor() ([]byte, []int)

Deprecated: Use SplitAndLaunch.ProtoReflect.Descriptor instead.

func (*SplitAndLaunch) GetJobId 

func (x *SplitAndLaunch) GetJobId() int64

func (*SplitAndLaunch) ProtoMessage 

func (*SplitAndLaunch) ProtoMessage()

func (*SplitAndLaunch) ProtoReflect 

func (x *SplitAndLaunch) ProtoReflect() protoreflect.Message

func (*SplitAndLaunch) Reset 

func (x *SplitAndLaunch) Reset()

func (*SplitAndLaunch) String 

func (x *SplitAndLaunch) String() string

type UpdateJobState 

type UpdateJobState struct {
	JobId int64 `protobuf:"varint,1,opt,name=job_id,json=jobId,proto3" json:"job_id,omitempty"`
	// contains filtered or unexported fields
}

UpdateJobState is emitted after one or more shards have changed their state.

It recalculates the job's state based on state of all its shards. Throttled to 0.5 QPS.

func (*UpdateJobState) Descriptor 

func (*UpdateJobState) Descriptor() ([]byte, []int)

Deprecated: Use UpdateJobState.ProtoReflect.Descriptor instead.

func (*UpdateJobState) GetJobId 

func (x *UpdateJobState) GetJobId() int64

func (*UpdateJobState) ProtoMessage 

func (*UpdateJobState) ProtoMessage()

func (*UpdateJobState) ProtoReflect 

func (x *UpdateJobState) ProtoReflect() protoreflect.Message

func (*UpdateJobState) Reset 

func (x *UpdateJobState) Reset()

func (*UpdateJobState) String 

func (x *UpdateJobState) String() string

Source Files

View all