README
¶
s3fs
A S3 filesystem implementation of io.Fs.
Supports handling directories and files transparently, while being memory efficient, which allows handles large files without being limited by the available memory.
Install
go get -u github.com/jacoelho/s3fs
Example
package main
import (
"context"
"fmt"
"io/fs"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/s3"
"github.com/jacoelho/s3fs"
)
func main() {
cfg, err := config.LoadDefaultConfig(context.Background())
if err != nil { panic(err) }
client := s3.NewFromConfig(cfg, func(opt *s3.Options) {
opt.UsePathStyle = true
})
filesystem := s3fs.New(client, "test")
data, err := fs.ReadFile(filesystem, "a-file") // not recommend when handling large files
if err != nil { panic(err) }
fmt.Println(string(data))
}
Policies
| policy | resource |
|---|---|
| s3:ListBucket | arn:aws:s3:::YOUR_BUCKET |
| s3:GetObject | arn:aws:s3:::YOUR_BUCKET/YOUR_PREFIX/* |
| s3:PutObject | arn:aws:s3:::YOUR_BUCKET/YOUR_PREFIX/* |
| s3:DeleteObject | arn:aws:s3:::YOUR_BUCKET/YOUR_PREFIX/* |
| s3:ListMultipartUploadParts | arn:aws:s3:::YOUR_BUCKET/YOUR_PREFIX/* |
| s3:AbortMultipartUpload | arn:aws:s3:::YOUR_BUCKET/YOUR_PREFIX/* |
License
MIT License
See LICENSE to see the full text.
Documentation
¶
Index ¶
- type Directory
- type File
- func (f *File) Close() error
- func (f *File) Info() (fs.FileInfo, error)
- func (f *File) IsDir() bool
- func (f *File) Name() string
- func (f *File) Read(b []byte) (int, error)
- func (f *File) ReadAt(b []byte, offset int64) (int, error)
- func (f *File) Seek(offset int64, whence int) (int64, error)
- func (f *File) Stat() (fs.FileInfo, error)
- func (f *File) Type() fs.FileMode
- func (f *File) Write(p []byte) (n int, err error)
- func (f *File) WriteAt(p []byte, off int64) (n int, err error)
- type FileInfo
- type Fs
- func (f *Fs) Create(name string) (*File, error)
- func (f *Fs) CreateDir(name string) (fs.DirEntry, error)
- func (f *Fs) CreateDirWithContext(ctx context.Context, name string) (fs.DirEntry, error)
- func (f *Fs) CreateWithContext(ctx context.Context, name string) (*File, error)
- func (f *Fs) Open(name string) (fs.File, error)
- func (f *Fs) OpenWithContext(ctx context.Context, name string) (fs.File, error)
- func (f *Fs) ReadDir(dirName string) ([]fs.DirEntry, error)
- func (f *Fs) ReadDirWithContext(ctx context.Context, dirName string) ([]fs.DirEntry, error)
- func (f *Fs) Remove(filename string) error
- func (f *Fs) RemoveDir(name string) error
- func (f *Fs) RemoveDirWithContext(ctx context.Context, name string) error
- func (f *Fs) RemoveWithContext(ctx context.Context, fileName string) error
- func (f *Fs) Rename(oldpath, newpath string) error
- func (f *Fs) RenameWithContext(ctx context.Context, oldpath, newpath string) error
- func (f *Fs) Stat(name string) (FileInfo, error)
- func (f *Fs) StatWithContext(ctx context.Context, name string) (FileInfo, error)
- type Option
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type File ¶
type File struct {
// contains filtered or unexported fields
}
type Fs ¶
type Fs struct {
// contains filtered or unexported fields
}
Fs is fs.FS S3 filesystem abstraction.
func (*Fs) CreateDir ¶
CreateDir creates a name directory Since S3 doesn't have the concept of directories, an empty file .keep is created.
func (*Fs) CreateDirWithContext ¶
CreateDirWithContext creates a name directory Since S3 doesn't have the concept of directories, an empty file .keep is created.
func (*Fs) CreateWithContext ¶
CreateWithContext opens a named file for writing.
func (*Fs) OpenWithContext ¶
OpenWithContext opens the named file or directory for reading.
func (*Fs) ReadDir ¶
ReadDir reads the named directory and returns a list of directory entries sorted by filename.
func (*Fs) ReadDirWithContext ¶
ReadDirWithContext reads the named directory and returns a list of directory entries sorted by filename.
func (*Fs) RemoveDirWithContext ¶
RemoveDirWithContext removes an empty directory.
func (*Fs) RemoveWithContext ¶
RemoveWithContext removes the named file.
func (*Fs) Rename ¶
Rename renames (moves) oldpath to newpath. If newpath already exists and is not a directory, Rename replaces it.
func (*Fs) RenameWithContext ¶
RenameWithContext renames (moves) oldpath to newpath. If newpath already exists and is not a directory, Rename replaces it.
type Option ¶
type Option func(*Fs)
Option is a Fs configuration.
func WithDirectoryFile ¶
WithDirectoryFile sets the file created when CreateDir is used.
func WithPartSize ¶
WithPartSize sets the part size used on multipart download or upload.
func WithPrefix ¶
WithPrefix defines a common prefix inside a bucket.
func WithTemporaryDirectory ¶
WithTemporaryDirectory sets the temporary directory where the unlinked temporary files will be created.
func WithTimeout ¶
WithTimeout sets the timeout when interacting with S3.
Source Files