aws-sqs-filter-redrive
aws-sqs-filter-redrive
is a tool to selectively redrive Amazon Simple Queue Service (SQS) messages based on their content. With this tool, you can get and filter messages from a queue based on their attributes or JSON body fields, and then move or delete them to a new queue.
Installation
Download the latest binaries and run
Or install via go install
go install github.com/natenho/aws-sqs-filter-redrive@latest
Usage
aws-sqs-filter-redrive [OPTIONS]
Here are the available options:
-source
: The URL of the source queue.
-target
: The URL of the target queue.
-count
: The number of messages to delete or move.
-delete
: Delete the messages instead of moving them.
-move
: Move the messages to the target queue.
-dry
: Perform a dry run to show which messages would be processed.
-workers
: The number of workers to use (default: 8).
-polling-duration
: The duration for polling messages from the source queue (default: 30s).
-rate-limit
: The maximum number of messages to process per second (default: 10).
-before
: Get only messages sent before a certain date in format 2006-01-02T15:04:05Z07:00.
-after
: Get only messages sent after a certain date in format 2006-01-02T15:04:05Z07:00.
-attribute-filter
: Filter messages by a certain attribute using JQ expression.
-body-filter
: Filter messages by JSON body field content using JQ expression.
-message-attribute-filter
: Filter messages by message attribute using JQ expression.
Examples
Move messages that were sent after a date
aws-sqs-filter-redrive \
-source https://sqs.us-east-1.amazonaws.com/123456789012/source-queue \
-target https://sqs.us-east-1.amazonaws.com/123456789012/target-queue \
-move \
-after 2022-01-01T00:00:00Z
Delete messages sent before a date
aws-sqs-filter-redrive \
-source https://sqs.us-east-1.amazonaws.com/123456789012/source-queue \
-delete \
-before 2022-01-01T00:00:00Z
Move messages from one queue to another based on a JSON body field
aws-sqs-filter-redrive \
-source https://sqs.us-east-1.amazonaws.com/123456789012/source-queue \
-target https://sqs.us-east-1.amazonaws.com/123456789012/target-queue \
-move \
-body-filter '.field == "value"'
Delete messages from a queue based on an attribute
aws-sqs-filter-redrive \
-source https://sqs.us-east-1.amazonaws.com/123456789012/source-queue \
-delete \
-attribute-filter '.AttributeName == "AttributeValue"'
Delete messages from a queue based on a message attribute. Message attributes have different value types, so you need to specify the type of the value you are filtering.
aws-sqs-filter-redrive \
-source https://sqs.us-east-1.amazonaws.com/123456789012/source-queue \
-delete \
-message-attribute-filter '.AttributeName.StringValue == "AttributeValue"' \
-count 10000 -polling-duration 60s
Contributing
If you find a bug or have a feature request, please open an issue or submit a pull request. Contributions are always welcome!
License