goofys

README

Goofys is a Filey-System interface to Amazon S3

Overview

Goofys allows you to mount an S3 bucket as a filey system.

It's a Filey System instead of a File System because goofys strives for performance first and POSIX second. Particularly things that are difficult to support on S3 or would translate into more than one round-trip would either fail (random writes) or faked (no per-file permission). Goofys does not have a on disk data cache, and consistency model is close-to-open.

Usage

Pre-built binaries are available here. You may also need to install fuse-utils first.

$ export GOPATH=$HOME/work
$ go get github.com/kahing/goofys
$ go install github.com/kahing/goofys
$ cat > ~/.aws/credentials
[default]
aws_access_key_id = AKID1234567890
aws_secret_access_key = MY-SECRET-KEY
$ $GOPATH/bin/goofys <bucket> <mountpoint>
$ $GOPATH/bin/goofys <bucket:prefix> <mountpoint> # if you only want to mount objects under a prefix

Users can also configure credentials via the AWS CLI or the AWS_ACCESS_KEY and AWS_SECRET_KEY environment variables.

To mount an S3 bucket on startup, make sure the credential is configured for root, and can add this to /etc/fstab:

goofys#bucket   /mnt/mountpoint        fuse     _netdev,allow_other,--file-mode=0666    0       0

Got more questions? Check out questions other people asked

Benchmark

Using --stat-cache-ttl 0 --type-cache-ttl 0 for goofys -ostat_cache_expire=1 for s3fs to simulate cold runs. Detail for the benchmark can be found in bench.sh. Raw data is available as well. Test was run on an EC2 hi1.4xlarge in us-west-2a connecting to a bucket in us-west-2. Units are seconds.

(†) riofs does not wait for HTTP response before returning from release(), so the create files benchmarks do not measure the right thing for it

License

Copyright (C) 2015 Ka-Hing Cheung

Licensed under the Apache License, Version 2.0

Current Status

goofys has only been tested under Linux.

List of non-POSIX behaviors/limitations:

• only sequential writes supported
• does not store file mode/owner/group
  • use --(dir|file)-mode or --(uid|gid) options
• does not support symlink or hardlink
• ctime, atime is always the same as mtime
• cannot rename non-empty directories
• unlink returns success even if file is not present

In addition to the items above, the following supportable but not yet implemented:

• appending to a file
• creating files larger than 1TB

References

• Data is stored on Amazon S3
• Amazon SDK for Go
• Other related fuse filesystems
  • s3fs: another popular filesystem for S3
  • gcsfuse: filesystem for Google Cloud Storage. Goofys borrowed some skeleton code from this project.
• S3Proxy is used for go test
• fuse binding, also used by gcsfuse