s3fs

README

s3fs

Package s3fs provides a S3 implementation for Go1.16 filesystem interface.

Since S3 is a flat structure, s3fs simulates directories by using prefixes and "/" delim. ModTime on directories is always zero value.

The implementation wraps aws sdk go s3 client.

const bucket = "my-bucket"

s, err := session.NewSession()
if err != nil {
    log.Fatal(err)
}

s3fs := s3fs.New(s3.New(s), bucket)

// print out all files in s3 bucket.
_ = fs.WalkDir(s3fs, ".", func(path string, d fs.DirEntry, err error) error {
    if err != nil {
        return err
    }

    if d.IsDir() {
        fmt.Println("dir:", path)
        return nil
    }
    fmt.Println("file:", path)
    return nil
})

Installation

go get github.com/jszwec/s3fs

Requirements

• Go1.16+

Documentation

Overview

Package s3fs provides a S3 implementation for Go1.16 filesystem interface.

Index

• func WithReadSeeker(fsys *S3FS)
• type Option
• type S3Client
• type S3FS
  • func New(cl S3Client, bucket string, opts ...Option) *S3FS
  • func (f *S3FS) Open(name string) (fs.File, error)
  • func (f *S3FS) ReadDir(name string) ([]fs.DirEntry, error)
  • func (f *S3FS) Stat(name string) (fs.FileInfo, error)
• Bugs

Functions

func WithReadSeeker

func WithReadSeeker(fsys *S3FS)

WithReadSeeker enables Seek functionality on files opened with this fs.

BUG(WilliamFrei): Seeking on S3 requires reopening the file at the specified position. This can cause problems if the file changed between opening and calling Seek. In that case, fs.ErrNotExist error is returned, which has to be handled by the caller.

Types

type Option

type Option func(*S3FS)

Option is a function that provides optional features to S3FS.

type S3Client

type S3Client interface {
	manager.ListObjectsV2APIClient
	manager.DeleteObjectsAPIClient
	manager.DownloadAPIClient
	manager.HeadBucketAPIClient
	manager.UploadAPIClient
	HeadObject(ctx context.Context, params *s3.HeadObjectInput, optFns ...func(*s3.Options)) (*s3.HeadObjectOutput, error)
}

type S3FS

type S3FS struct {
	// contains filtered or unexported fields
}

S3FS is a S3 filesystem implementation.

S3 has a flat structure instead of a hierarchy. S3FS simulates directories by using prefixes and delims ("/"). Because directories are simulated, ModTime is always a default Time value (IsZero returns true).

func New

func New(cl S3Client, bucket string, opts ...Option) *S3FS

New returns a new filesystem that works on the specified bucket.

func (*S3FS) Open

func (f *S3FS) Open(name string) (fs.File, error)

Open implements fs.FS.

func (*S3FS) ReadDir

func (f *S3FS) ReadDir(name string) ([]fs.DirEntry, error)

ReadDir implements fs.ReadDirFS.

func (*S3FS) Stat

func (f *S3FS) Stat(name string) (fs.FileInfo, error)

Stat implements fs.StatFS.

Notes

Bugs

• Seeking on S3 requires reopening the file at the specified position. This can cause problems if the file changed between opening and calling Seek. In that case, fs.ErrNotExist error is returned, which has to be handled by the caller.