README
¶
s3Compat: An S3 Compatibility Library for Go
Overview
s3Compat is a Go package that simplifies interactions with Amazon Simple Storage Service (S3) or S3-compatible storage services. The package provides an easy-to-use and straightforward API for various S3 operations such as uploading, downloading, and deleting objects.
Features
- Upload objects to S3 using byte arrays, streams, or seekable streams.
- Download objects from S3 as byte arrays or streams.
- Delete objects from S3.
- List objects in a bucket.
- Support for custom S3-compatible services by specifying the endpoint.
Installation
To install s3Compat, run the following command:
go get -u github.com/8ff/s3Compat
Usage
Importing
First, import the package into your Go application:
import "github.com/8ff/s3Compat"
Initialize
Create a new S3 Params struct:
client, err := s3Compat.New(s3Compat.Params{
REGION: "us-west-2",
BUCKET: "my-bucket",
ACCESS_KEY: "your-access-key",
SECRET_KEY: "your-secret-key",
ENDPOINT: "https://s3.amazonaws.com",
})
Uploading an Object
Upload a byte array to S3:
err := client.PutObject("my-object-key", []byte("Hello, world!"))
Downloading an Object
Download an object from S3:
data, err := client.GetObject("my-object-key")
Deleting an Object
Delete an object from S3:
err := client.DeleteObject("my-object-key")
Listing Objects in a Bucket
List objects in the bucket:
objectNames, err := client.ListObjects()
Example
Here's a simple example that uploads and then downloads a text file:
package main
import (
"github.com/8ff/s3Compat"
)
func main() {
client, _ := s3Compat.New(s3Compat.Params{
REGION: "us-west-2",
BUCKET: "my-bucket",
ACCESS_KEY: "your-access-key",
SECRET_KEY: "your-secret-key",
ENDPOINT: "https://s3.amazonaws.com",
})
// Upload
_ = client.PutObject("test-file", []byte("Hello, world!"))
// Download
data, _ := client.GetObject("test-file")
println(string(data))
}
Examples with Stream and Pipe Operations
The following examples demonstrate how to use s3Compat for stream and pipe operations.
Example: Upload an Object from Stdin (pipeObject)
This example reads from the standard input and uploads the data to S3.
func pipeObject(access_key string, secret string, endpoint string, args []string) {
// (Omitted: Error checks and argument validation)
api, err := s3Compat.New(s3Compat.Params{BUCKET: bucket, ACCESS_KEY: access_key, SECRET_KEY: secret, ENDPOINT: endpoint})
if err != nil {
// Handle error
}
err = api.PutObjectStream(file, os.Stdin)
if err != nil {
// Handle error
}
}
Example: Download an Object to Stdout (catObject)
This example downloads an object from S3 and writes it to the standard output.
func catObject(access_key string, secret string, endpoint string, args []string) {
// (Omitted: Error checks and argument validation)
api, err := s3Compat.New(s3Compat.Params{BUCKET: bucket, ACCESS_KEY: access_key, SECRET_KEY: secret, ENDPOINT: endpoint})
if err != nil {
// Handle error
}
readCloser, err := api.GetObjectStream(file)
if err != nil {
// Handle error
}
defer readCloser.Close()
_, err = io.Copy(os.Stdout, readCloser)
if err != nil {
// Handle error
}
}
Example: Delete an Object (rmObject)
This example deletes an object from S3.
func rmObject(access_key string, secret string, endpoint string, args []string) {
// (Omitted: Error checks and argument validation)
api, err := s3Compat.New(s3Compat.Params{BUCKET: bucket, ACCESS_KEY: access_key, SECRET_KEY: secret, ENDPOINT: endpoint})
if err != nil {
// Handle error
}
err = api.DeleteObject(file)
if err != nil {
// Handle error
}
}
Additionally s3Compat_test.go contains more examples.
Contributing
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
License
This project is licensed under the AGPL 3.0 License. See the LICENSE file for details.
Documentation
¶
Index ¶
- type Params
- func (p *Params) DeleteObject(key string) error
- func (p *Params) GetMetadata(key string) (map[string]string, error)
- func (p *Params) GetObject(key string) ([]byte, error)
- func (p *Params) GetObjectStream(key string) (io.ReadCloser, error)
- func (p *Params) GetObjectStreamSimple(key string) (io.ReadCloser, error)
- func (p *Params) ListObjects() ([]string, error)
- func (p *Params) PutObject(key string, data []byte) error
- func (p *Params) PutObjectSeekableStream(key string, data io.Reader) error
- func (p *Params) PutObjectStream(key string, data io.Reader) error
- func (p *Params) SetMetadata(key string, metadata map[string]string) error
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Params ¶
type Params struct {
REGION string
BUCKET string
ACCESS_KEY string
SECRET_KEY string
ENDPOINT string
Service *s3.Client
}
func (*Params) DeleteObject ¶
Function that deletes object from S3
func (*Params) GetMetadata ¶
Function that takes key and returns metadata for that key
func (*Params) GetObjectStream ¶
func (p *Params) GetObjectStream(key string) (io.ReadCloser, error)
Parallel
func (*Params) GetObjectStreamSimple ¶
func (p *Params) GetObjectStreamSimple(key string) (io.ReadCloser, error)
Function to get object stream in a simple way
func (*Params) ListObjects ¶
Function that lists all objects in a bucket
func (*Params) PutObjectSeekableStream ¶
Function to put object with a seekable stream
func (*Params) PutObjectStream ¶
Take stream data non-seekable stream like StdIn and upload it to S3
Source Files