transport

package
v0.0.0-...-0d862b9 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 9, 2024 License: Apache-2.0, MPL-2.0 Imports: 27 Imported by: 0

Documentation

Index

Constants

View Source 
const DefaultBatchSendIntervalSec = 3
View Source 
const METADATA_FINGERPRINT_RETRIES = 10
View Source 
const TestFakeCredentialsPath = "../test-fixtures/fake_account.json"

Variables

View Source 
var DefaultRequestTimeout = 5 * time.Minute
View Source 
var FINGERPRINT_FAIL_ERRORS = []string{"Invalid fingerprint.", "Supplied fingerprint does not match current metadata fingerprint."}

We've encountered a few common fingerprint-related strings; if this is one of them, we're confident this is an error due to fingerprints.

View Source 
var MutexStore = NewMutexKV()

Global MutexKV

View Source 
var TimeoutErr = &TimeoutError{timeout: true}

Functions

func AddQueryParams 

func AddQueryParams(rawurl string, params map[string]string) (string, error)

func ClientWithAdditionalRetries 

func ClientWithAdditionalRetries(baseClient *http.Client, predicates ...RetryErrorPredicateFunc) *http.Client

Helper method to create a shallow copy of an HTTP client with a shallow-copied retryTransport s.t. the base HTTP transport is the same (i.e. client connection pools are shared, retryPredicates are different)

func DatastoreIndex409Contention 

func DatastoreIndex409Contention(err error) (bool, string)

func ExternalIpServiceNotActive 

func ExternalIpServiceNotActive(err error) (bool, string)

Retry the creation of `google_vmwareengine_external_address` resource if the network policy's External IP field is not active yet.

func FirestoreField409RetryUnderlyingDataChanged 

func FirestoreField409RetryUnderlyingDataChanged(err error) (bool, string)

relevant for firestore in datastore mode

func FirestoreIndex409Retry 

func FirestoreIndex409Retry(err error) (bool, string)

relevant for firestore in datastore mode

func HandleDataSourceNotFoundError 

func HandleDataSourceNotFoundError(err error, d *schema.ResourceData, resource, url string) error

func HandleNotFoundError 

func HandleNotFoundError(err error, d *schema.ResourceData, resource string) error

func HealthcareDatasetNotInitialized 

func HealthcareDatasetNotInitialized(err error) (bool, string)

func IamMemberMissing 

func IamMemberMissing(err error) (bool, string)

If a permission necessary to provision a resource is created in the same config as the resource itself, the permission may not have propagated by the time terraform attempts to create the resource. This allows those errors to be retried until the timeout expires

func IamServiceAccountNotFound 

func IamServiceAccountNotFound(err error) (bool, string)

If a service account is deleted in the middle of updating an IAM policy it can cause the API to return an error. In fine-grained IAM resources we read the policy, modify it, then send it back to the API. Retrying is useful particularly in high-traffic projects. We don't want to retry _every_ time we see this error because the user-provided SA could trigger this too. At the callsite, we should check if the current etag matches the old etag and short-circuit if they do as that indicates the new config is the likely problem.

func IapClient409Operation 

func IapClient409Operation(err error) (bool, string)

func Is429QuotaError 

func Is429QuotaError(err error) (bool, string)

Do not retry if operation returns a 429

func IsApiNotEnabledError 

func IsApiNotEnabledError(err error) bool

func IsApigeeRetryableError 

func IsApigeeRetryableError(err error) (bool, string)

Concurrent Apigee operations can fail with a 400 error

func IsAppEngineRetryableError 

func IsAppEngineRetryableError(err error) (bool, string)

Retry if App Engine operation returns a 409 with a specific message for concurrent operations, or a 404 indicating p4sa has not yet propagated.

func IsBigTableRetryableError 

func IsBigTableRetryableError(err error) (bool, string)

Bigtable uses gRPC and thus does not return errors of type *googleapi.Error. Instead the errors returned are *status.Error. See the types of codes returned here (https://pkg.go.dev/google.golang.org/grpc/codes#Code).

func IsBigqueryIAMQuotaError 

func IsBigqueryIAMQuotaError(err error) (bool, string)

Retry if Bigquery operation returns a 403 with a specific message for concurrent operations (which are implemented in terms of 'edit quota').

func IsCloudRunCreationConflict 

func IsCloudRunCreationConflict(err error) (bool, string)

Cloud Run APIs may return a 409 on create to indicate that a resource has been deleted in the foreground (eg GET and LIST) but not the backing apiserver. When we encounter a 409, we can retry it. Note that due to limitations in MMv1's error_retry_predicates this is currently applied to all requests. We only expect to receive it on create, though.

func IsCryptoKeyVersionsPendingGeneration 

func IsCryptoKeyVersionsPendingGeneration(err error) (bool, string)

Retry if KMS CryptoKeyVersions returns a 400 for PENDING_GENERATION

func IsDataflowJobUpdateRetryableError 

func IsDataflowJobUpdateRetryableError(err error) (bool, string)

func IsFingerprintError 

func IsFingerprintError(err error) (bool, string)

Retry the operation if it looks like a fingerprint mismatch.

func IsGoogleApiErrorWithCode 

func IsGoogleApiErrorWithCode(err error, errCode int) bool

func IsMonitoringConcurrentEditError 

func IsMonitoringConcurrentEditError(err error) (bool, string)

Retry if Monitoring operation returns a 409 with a specific message for concurrent operations.

func IsMonitoringPermissionError 

func IsMonitoringPermissionError(err error) (bool, string)

Retry if Monitoring operation returns a 403

func IsPeeringOperationInProgress 

func IsPeeringOperationInProgress(err error) (bool, string)

func IsRetryableError 

func IsRetryableError(topErr error, retryPredicates, abortPredicates []RetryErrorPredicateFunc) bool

func IsSqlOperationInProgressError 

func IsSqlOperationInProgressError(err error) (bool, string)

Retry if Cloud SQL operation returns a 429 with a specific message for concurrent operations.

func IsSwgAutogenRouterRetryable 

func IsSwgAutogenRouterRetryable(err error) (bool, string)

Gateway of type 'SECURE_WEB_GATEWAY' automatically creates a router but does not delete it. This router might be re-used by other gateways located in the same network. When multiple gateways are being deleted at the same time, multiple attempts to delete the same router will be triggered and the api throws an error saying the "The resource <router> is not ready".

func LockedCall 

func LockedCall(lockKey string, f func() error) error

func MetadataRetryWrapper 

func MetadataRetryWrapper(update func() error) error

Since the google compute API uses optimistic locking, there is a chance we need to resubmit our updated metadata. To do this, you need to provide an update function that attempts to submit your metadata

func NewTransportWithDefaultRetries 

func NewTransportWithDefaultRetries(t http.RoundTripper) *retryTransport

NewTransportWithDefaultRetries constructs a default retryTransport that will retry common temporary errors

func NewTransportWithHeaders 

func NewTransportWithHeaders(baseTransit http.RoundTripper) headerTransportLayer

func PollingWaitTime 

func PollingWaitTime(pollF PollReadFunc, checkResponse PollCheckResponseFunc, activity string,
	timeout time.Duration, targetOccurrences int) error

func PubsubTopicProjectNotReady 

func PubsubTopicProjectNotReady(err error) (bool, string)

Cloud PubSub returns a 400 error if a topic's parent project was recently created and an organization policy has not propagated. See https://github.com/hashicorp/terraform-provider-google/issues/4349

func Retry 

func Retry(opt RetryOptions) error

func RetryWithTargetOccurrences 

func RetryWithTargetOccurrences(timeout time.Duration, targetOccurrences int,
	f resource.RetryFunc) error

RetryWithTargetOccurrences is a basic wrapper around StateChangeConf that will retry a function until it returns the specified amount of target occurrences continuously. Adapted from the Retry function in the go SDK.

func SendRequest 

func SendRequest(opt SendRequestOptions) (map[string]interface{}, error)

func ServiceUsageInternalError160009 

func ServiceUsageInternalError160009(err error) (bool, string)

See https://github.com/hashicorp/terraform-provider-google/issues/14691 for details on the error message this handles This is a post-operation error so it uses tpgresource.CommonOpError instead of googleapi.Error

func ServiceUsageServiceBeingActivated 

func ServiceUsageServiceBeingActivated(err error) (bool, string)

Retry if service usage decides you're activating the same service multiple times. This can happen if a service and a dependent service aren't batched together- eg container.googleapis.com in one request followed by compute.g.c in the next (container relies on compute and implicitly activates it)

Types

type BatchRequest 

type BatchRequest struct {
	// ResourceName represents the underlying resource for which
	// a request is made. Its format is determined by what SendF expects, but
	// typically should be the name of the parent GCP resource being changed.
	ResourceName string

	// Body is this request's data to be passed to SendF, and may be combined
	// with other bodies using CombineF.
	Body interface{}

	// CombineF function determines how to combine bodies from two batches.
	CombineF BatcherCombineFunc

	// SendF function determines how to actually send a batched request to a
	// third party service. The arguments given to this function are
	// (ResourceName, Body) where Body may have been combined with other request
	// Bodies.
	SendF BatcherSendFunc

	// ID for debugging request. This should be specific to a single request
	// (i.e. per Terraform resource)
	DebugId string
}

BatchRequest represents a single request to a global batcher.

type BatcherCombineFunc 

type BatcherCombineFunc func(body interface{}, toAdd interface{}) (interface{}, error)

BatcherCombineFunc is a function type for combine existing batches and additional batch data

type BatcherSendFunc 

type BatcherSendFunc func(resourceName string, body interface{}) (interface{}, error)

BatcherSendFunc is a function type for sending a batch request

type BatchingConfig 

type BatchingConfig struct {
	SendAfter      time.Duration
	EnableBatching bool
}

BatchingConfig contains user configuration for controlling batch requests.

type BigtableClientFactory 

type BigtableClientFactory struct {
	UserAgent           string
	TokenSource         oauth2.TokenSource
	BillingProject      string
	UserProjectOverride bool
	// contains filtered or unexported fields
}

func (BigtableClientFactory) NewAdminClient 

func (s BigtableClientFactory) NewAdminClient(project, instance string) (*bigtable.AdminClient, error)

func (BigtableClientFactory) NewClient 

func (s BigtableClientFactory) NewClient(project, instance string) (*bigtable.Client, error)

func (BigtableClientFactory) NewInstanceAdminClient 

func (s BigtableClientFactory) NewInstanceAdminClient(project string) (*bigtable.InstanceAdminClient, error)

type MutexKV 

type MutexKV struct {
	// contains filtered or unexported fields
}

MutexKV is a simple key/value store for arbitrary mutexes. It can be used to serialize changes across arbitrary collaborators that share knowledge of the keys they must serialize on.

The initial use case is to let aws_security_group_rule resources serialize their access to individual security groups based on SG ID.

func NewMutexKV 

func NewMutexKV() *MutexKV

Returns a properly initialized MutexKV

func (*MutexKV) Lock 

func (m *MutexKV) Lock(key string)

Locks the mutex for the given key. Caller is responsible for calling Unlock for the same key

func (*MutexKV) RLock 

func (m *MutexKV) RLock(key string)

Acquires a read-lock on the mutex for the given key. Caller is responsible for calling RUnlock for the same key

func (*MutexKV) RUnlock 

func (m *MutexKV) RUnlock(key string)

Releases a read-lock on the mutex for the given key. Caller must have called RLock for the same key first

func (*MutexKV) Unlock 

func (m *MutexKV) Unlock(key string)

Unlock the mutex for the given key. Caller must have called Lock for the same key first

type PollCheckResponseFunc 

type PollCheckResponseFunc func(resp map[string]interface{}, respErr error) PollResult

Function to check the response from polling once

type PollReadFunc 

type PollReadFunc func() (resp map[string]interface{}, respErr error)

Function handling for polling for a resource

type PollResult 

type PollResult *resource.RetryError

func ErrorPollResult 

func ErrorPollResult(err error) PollResult

Helper functions to construct result of single pollRead as return result for a PollCheckResponseFunc

func PendingStatusPollResult 

func PendingStatusPollResult(status string) PollResult

func PollCheckForAbsence 

func PollCheckForAbsence(_ map[string]interface{}, respErr error) PollResult

PollCheckForAbsence waits for a 404 response, continues polling on a successful response, and returns any other error.

func PollCheckForAbsenceWith403 

func PollCheckForAbsenceWith403(_ map[string]interface{}, respErr error) PollResult

PollCheckForAbsence waits for a 404/403 response, continues polling on a successful response, and returns any other error.

func PollCheckForExistence 

func PollCheckForExistence(_ map[string]interface{}, respErr error) PollResult

PollCheckForExistence waits for a successful response, continues polling on 404, and returns any other error.

func PollCheckForExistenceWith403 

func PollCheckForExistenceWith403(_ map[string]interface{}, respErr error) PollResult

PollCheckForExistenceWith403 waits for a successful response, continues polling on 404 or 403, and returns any other error.

func SuccessPollResult 

func SuccessPollResult() PollResult

type RequestBatcher 

type RequestBatcher struct {
	sync.Mutex

	*BatchingConfig
	// contains filtered or unexported fields
}

RequestBatcher keeps track of batched requests globally. It should be created at a provider level. In general, one should be created per service that requires batching to:

  • prevent blocking batching for one service due to another,
  • minimize the possibility of overlap in batchKey formats (see SendRequestWithTimeout)

func NewRequestBatcher 

func NewRequestBatcher(debugId string, ctx context.Context, config *BatchingConfig) *RequestBatcher

Initializes a new batcher.

func (*RequestBatcher) SendRequestWithTimeout 

func (b *RequestBatcher) SendRequestWithTimeout(batchKey string, request *BatchRequest, timeout time.Duration) (interface{}, error)

SendRequestWithTimeout is a blocking call for making a single request, run alone or as part of a batch. It manages registering the single request with the batcher and waiting on the result.

Params: batchKey: A string to group batchable requests. It should be unique to the API request being sent, similar to the HTTP request URL with GCP resource ID included in the URL (the caller may choose to use a key with method if needed to diff GET/read and POST/create)

As an example, for google_project_service, the batcher is called to batch services.batchEnable() calls for a project $PROJECT. The calling code uses the template "serviceusage:projects/$PROJECT/services:batchEnable", which mirrors the HTTP request: POST https://serviceusage.googleapis.com/v1/projects/$PROJECT/services:batchEnable

type RetryErrorPredicateFunc 

type RetryErrorPredicateFunc func(error) (bool, string)

func IsForbiddenIamServiceAccountRetryableError 

func IsForbiddenIamServiceAccountRetryableError(opType string) RetryErrorPredicateFunc

Retry if getting a resource/operation returns a 403 for specific IAM Admin API Service Account operations. opType should describe the operation for which it can be retryable. IAM API is eventually consistent and returns 403 Forbidden (instead of 404 Not found) for some operations when a newly created resource is attempted to be read right after the creation and not found.

func IsNotFoundRetryableError 

func IsNotFoundRetryableError(opType string) RetryErrorPredicateFunc

Retry if getting a resource/operation returns a 404 for specific operations. opType should describe the operation for which 404 can be retryable.

type RetryOptions 

type RetryOptions struct {
	RetryFunc            func() error
	Timeout              time.Duration
	PollInterval         time.Duration
	ErrorRetryPredicates []RetryErrorPredicateFunc
	ErrorAbortPredicates []RetryErrorPredicateFunc
}

type SendRequestOptions 

type SendRequestOptions struct {
	Config               *Config
	Method               string
	Project              string
	RawURL               string
	UserAgent            string
	Body                 map[string]any
	Timeout              time.Duration
	ErrorRetryPredicates []RetryErrorPredicateFunc
	ErrorAbortPredicates []RetryErrorPredicateFunc
}

type TimeoutError 

type TimeoutError struct {
	// contains filtered or unexported fields
}

func (*TimeoutError) Error 

func (e *TimeoutError) Error() string

func (*TimeoutError) Timeout 

func (e *TimeoutError) Timeout() bool

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.