awclip

README

awclip

Local caching of aws cli commands

• Store command results in .awsclip as md5hash of parameters
• Store metadata as hash-md.json

Use it as aws cli alias:

If awclip binary is in /usr/local/bin:

alias aws='/usr/local/bin/awclip'

Status

Base functionality is working, no guarantee that it works on all services.

Example

Time with aws cli:

time aws iam list-roles >/dev/null
aws iam list-roles > /dev/null  
0,36s user 0,17s system 39% cpu 1,324 total

Time varies from 1..2 seconds

• copy awclip executable to your local filesystem

  • see https://github.com/megaproaktiv/awclip/tags
  • e.g. to /usr/local/bin/awclip

• create an alias

  alias aws=`/usr/local/bin/awclip`
  

• create a local .awclip directory

  mkdir .awclip
  

• or clean existing directory

  rm .awclip/*
  

1st time with awclip:

time aws iam list-roles >/dev/null
/Users/silberkopf/letsbuild/awclip/dist/awclip iam list-roles > /dev/null  
0,36s user 0,18s system 32% cpu 1,671 total

2nd time with awclip:

 time aws iam list-roles >/dev/null
/Users/silberkopf/letsbuild/awclip/dist/awclip iam list-roles > /dev/null 
 0,00s user 0,00s system 40% cpu 0,014 total

run certain api calls in awclip

The following aws cli command are used by prowler and are beeing performed by awclip.

• aws ec2 describe-instances --query "Reservations[*].Instances[*].[InstanceId]" --output text
• aws ec2 describe-regions --query "Regions[].RegionName" --output text
• aws sts get-caller-identity

If a call is recognized as a supported call, the metadata says: "Provider":"go". The aws cli calls have "Provider":"python"

Working with prowler

Change include/awscli_detector

Change

if [ ! -z $(which aws) ]; then
  AWSCLI=$(which aws)
elif [ ! -z $(type -p aws) ]; then
  AWSCLI=$(type -p aws)
else
  echo -e "\n$RED ERROR!$NORMAL AWS-CLI (aws command) not found. Make sure it is installed correctly and in your \$PATH\n"
  EXITCODE=1
  exit $EXITCODE
fi

to

AWSCLI=./awclip

if you copy awclip into the same directory.

Todo

• create .awcli automatically
• implement ttl (time to live), currently you have to clean .awclip yourself
• create storable configuration
• speed up region prefetch

Version

v0.1.6

• implement iam list-user-policies with additional parameters

v0.1.4

• implement api calls with specific query in program

v0.1.0

• reads command line
• calls aws with (python) aws cli
• writes metadata
• does not cache "generate-credential-report"

Documentation

Index

• func ArrangeParameters(args []string) []string
• func CacheHit(id *string) bool
• func CacheMiss(id *string) bool
• func CommandLine(args []string) *string
• func DiscriminatedCommand(service *string, action *string) bool
• func GetLocationData(contentId *string) *string
• func GetLocationMetaData(contentId *string) *string
• func HashValue(commandLine *string) *string
• func ReadContent(id *string) (*string, error)
• func ReadContentUpdate(id *string) (*string, error)
• func SpaceStringsBuilder(str string) string
• func UpdateMetaData(md *CacheEntry) error
• func WriteContent(id *string, content *string) error
• func WriteMetadata(md *CacheEntry) error
• type ApiCallProvider
• type ApiCallProviderName
• type CacheEntry
  • func ReadMetaData(id *string) (*CacheEntry, error)
  • func (item *CacheEntry) ArgumentsToCachedEntry(args []string)
• type Parameters
  • func (a *Parameters) AlmostEqual(b *Parameters) bool
  • func (a *Parameters) AlmostEqualWithParameters(b *Parameters) bool

Functions

func ArrangeParameters

func ArrangeParameters(args []string) []string

ArrangeParameters if args contains "--profile" and "profilename", put them at the end so optimizer can regognize it

func CacheHit

func CacheHit(id *string) bool

func CacheMiss

func CacheMiss(id *string) bool

func CommandLine

func CommandLine(args []string) *string

func DiscriminatedCommand

func DiscriminatedCommand(service *string, action *string) bool

DiscriminatedCommand check whether a command is cachable

func GetLocationData

func GetLocationData(contentId *string) *string

func GetLocationMetaData

func GetLocationMetaData(contentId *string) *string

func HashValue

func HashValue(commandLine *string) *string

HashVaule calculates the id of the cahce entries

func ReadContent

func ReadContent(id *string) (*string, error)

func ReadContentUpdate

func ReadContentUpdate(id *string) (*string, error)

func SpaceStringsBuilder

func SpaceStringsBuilder(str string) string

func UpdateMetaData

func UpdateMetaData(md *CacheEntry) error

func WriteContent

func WriteContent(id *string, content *string) error

func WriteMetadata

func WriteMetadata(md *CacheEntry) error

Types

type ApiCallProvider

type ApiCallProvider struct {
	// Open for extensions like provided calls
	Name ApiCallProviderName
}

type ApiCallProviderName

type ApiCallProviderName string

const (
	ApiCallProviderNameAws ApiCallProviderName = "aws cli python"
	ApiCallProviderNameGo  ApiCallProviderName = "go sdk v2"
)

type CacheEntry

type CacheEntry struct {
	Id            *string
	Cmd           *string
	Created       time.Time
	LastAccessed  time.Time
	AccessCounter int
	Parameters    Parameters
	Provider      string
}

CacheEntry Cmd executable command line e.g. aws ec2 describe instances

func ReadMetaData

func ReadMetaData(id *string) (*CacheEntry, error)

func (*CacheEntry) ArgumentsToCachedEntry

func (item *CacheEntry) ArgumentsToCachedEntry(args []string)

type Parameters

type Parameters struct {
	Service    *string
	Action     *string
	Output     *string
	Region     *string
	Profile    *string
	Parameters map[string]*string
	Query      *string
}

func (*Parameters) AlmostEqual

func (a *Parameters) AlmostEqual(b *Parameters) bool

func (*Parameters) AlmostEqualWithParameters

func (a *Parameters) AlmostEqualWithParameters(b *Parameters) bool