service

Documentation

Overview

File: https://github.com/data-engineering-helpers/dppctl/blob/main/service/aws.go

Index

• func AWSAirflowCLI(webServerHostname string, cliToken string, command string) (string, error)
• func AWSAirflowCreateLoginToken(environment string) (string, string, middleware.Metadata, error)
• func AWSCodeArtifactDescribePackageVersion(domainName string, domainOwner string, repoName string, ...) (string, error)
• func AWSCodeArtifactFormatFromString(format string) (awscatypes.PackageFormat, error)
• func AWSCodeArtifactListDomains() ([]string, error)
• func AWSCodeArtifactListPackageVersions(domainName string, domainOwner string, repoName string, ...) ([]string, error)
• func AWSECRDescribeImages(repoName string) ([]string, error)
• func AWSECRListImages(repoName string) ([]string, error)
• func AWSECRListRepositories() ([]string, error)
• func AWSGetCallerIdentity() (string, error)
• func AWSS3List(bucketName string, prefix string) ([]string, error)
• type MWAAResponse

Functions

func AWSAirflowCLI

func AWSAirflowCLI(webServerHostname string, cliToken string,
	command string) (string, error)

*

• AWS Managed Workflows for Apache Airflow (MWAA) - Execute a given
• MWAA CLI command *
• As of 2023, it does not seem possible to target/use the Airflow API
• directly on the AWS managed service (MWAA). One has to use
• the API backend of the MWAA CLI. That is why the code for
• that Go function is not straightforward.
• Note that the use of the MWAA CLI API (through `curl`) is itself
• convoluted. See also
• https://github.com/data-engineering-helpers/dppctl/blob/main/README.md *
• References:
• + Stack Overflow - Is it possible to access the Airflow API in AWS MWAA?
• https://stackoverflow.com/questions/67884770/is-it-possible-to-access-the-airflow-api-in-aws-mwaa
• + Apache Airflow - Airflow API reference guide: https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html
• + AWS - Amazon Managed Workflows for Apache Airflow (MWAA) User Guide:
• https://docs.aws.amazon.com/mwaa/index.html
• + AWS - Accessing the Apache Airflow UI:
• https://docs.aws.amazon.com/mwaa/latest/userguide/access-airflow-ui.html
• + AWS - Apache Airflow CLI command reference:
• https://docs.aws.amazon.com/mwaa/latest/userguide/airflow-cli-command-reference.html)
• + GitHub - AWS - Sample code for MWAA:
• https://github.com/aws-samples/amazon-mwaa-examples
• + GitHub - AWS - Sample code for MWAA - Bash operator script:
• https://github.com/aws-samples/amazon-mwaa-examples/tree/main/dags/bash_operator_script

func AWSAirflowCreateLoginToken

func AWSAirflowCreateLoginToken(environment string) (string, string,
	middleware.Metadata, error)

*

• AWS Managed Workflows for Apache Airflow (MWAA) - Create a CLI token
• References:
• + https://github.com/aws/aws-sdk-go-v2/blob/main/service/mwaa/api_op_CreateCliToken.go
• + https://github.com/aws/aws-sdk-go-v2/blob/main/service/mwaa/api_op_CreateWebLoginToken.go
• + https://github.com/aws/smithy-go/blob/main/middleware/metadata.go *

func AWSCodeArtifactDescribePackageVersion

func AWSCodeArtifactDescribePackageVersion(domainName string,
	domainOwner string, repoName string, repoFormat awscatypes.PackageFormat,
	packageName string, packageVersion string) (string, error)

*

• AWS CodeArticat (CA) - Details for a given combination of package and version
• References:
• + https://github.com/aws/aws-sdk-go-v2/blob/main/service/codeartifact/types/types.go
• + https://github.com/aws/aws-sdk-go-v2/blob/main/service/codeartifact/api_op_DescribePackageVersion.go
• + https://docs.aws.amazon.com/codeartifact/latest/APIReference/API_PackageVersionDescription.html *

func AWSCodeArtifactFormatFromString

func AWSCodeArtifactFormatFromString(format string) (awscatypes.PackageFormat,
	error)

*

• AWS CodeArticat (CA) - Get the repository format from a string *
• Not sure that this function is needed in Go. It is way possible that
• Go can do the same thing in a much safer and automated way thanks to
• the AWS SDK for go. Contributions are welcome (https://github.com/data-engineering-helpers/dppctl/pulls)
• if you find out.

func AWSCodeArtifactListDomains

func AWSCodeArtifactListDomains() ([]string, error)

*

• AWS CodeArticat (CA) - List of domains
• References:
• + https://github.com/aws/aws-sdk-go-v2/blob/main/service/codeartifact/types/types.go
• + https://github.com/aws/aws-sdk-go-v2/blob/main/service/codeartifact/api_op_ListDomains.go
• + https://docs.aws.amazon.com/codeartifact/latest/APIReference/API_DomainSummary.html

func AWSCodeArtifactListPackageVersions

func AWSCodeArtifactListPackageVersions(domainName string,
	domainOwner string, repoName string, repoFormat awscatypes.PackageFormat,
	packageName string) ([]string, error)

*

• AWS CodeArticat (CA) - List of versions for a given package
• References:
• + https://github.com/aws/aws-sdk-go-v2/blob/main/service/codeartifact/types/types.go
• + https://github.com/aws/aws-sdk-go-v2/blob/main/service/codeartifact/api_op_ListPackageVersions.go
• + https://docs.aws.amazon.com/codeartifact/latest/APIReference/API_PackageVersionSummary.html *

func AWSECRDescribeImages

func AWSECRDescribeImages(repoName string) ([]string, error)

*

• AWS Elastic Container Registry (ECR) - List of images
• References:
• + https://github.com/aws/aws-sdk-go-v2/blob/main/service/ecr/types/types.go
• + https://github.com/aws/aws-sdk-go-v2/blob/main/service/ecr/api_op_DescribeImages.go
• + https://docs.aws.amazon.com/AmazonECR/latest/APIReference/API_ImageDetail.html

func AWSECRListImages

func AWSECRListImages(repoName string) ([]string, error)

*

• AWS Elastic Container Registry (ECR) - List of images
• References:
• + https://github.com/aws/aws-sdk-go-v2/blob/main/service/ecr/types/types.go
• + https://github.com/aws/aws-sdk-go-v2/blob/main/service/ecr/api_op_ListImages.go
• + https://docs.aws.amazon.com/AmazonECR/latest/APIReference/API_ImageIdentifier.html

func AWSECRListRepositories

func AWSECRListRepositories() ([]string, error)

*

• AWS Elastic Container Registry (ECR) - List of repositories
• References:
• + https://github.com/aws/aws-sdk-go-v2/blob/main/service/ecr/types/types.go
• + https://github.com/aws/aws-sdk-go-v2/blob/main/service/ecr/api_op_DescribeRepositories.go
• + https://docs.aws.amazon.com/AmazonECR/latest/APIReference/API_Repository.html

func AWSGetCallerIdentity

func AWSGetCallerIdentity() (string, error)

*

• AWS STS - Get caller identity

func AWSS3List

func AWSS3List(bucketName string, prefix string) ([]string, error)

*

• AWS S3 - List of objects within a specific folder (prefix)

Types

type MWAAResponse

type MWAAResponse struct {
	StdErr string `json:"stderr"`
	StdOut string `json:"stdout"`
}

A Response struct to map the MWAA CLI API response