gofile

package module
v0.5.0-1648137530401987 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 24, 2022 License: MIT Imports: 15 Imported by: 2

README

gorepotemplate

Tricky and fun utilities for Go programs.


GitHub Workflow Status Codecov

Contributor Covenant

Twitter Follow GitHub followers


Getting Started

Prerequisites

Developed with $( go version; ). Go is extremely backwards compatible and semver stable. Nearly any v1.x should work fine.


Installation

To use this repo as a template for your own project:

gh repo create -y --public --template "https://github.com/skeptycal/gorepotemplate"

Clone this repo to test and contribute:

# add repo to $GOPATH (xxxxxx is your computer login username)
go get github.com/xxxxxx/gorepotemplate

cd ${GOPATH}/src/github.com/xxxxxx/gorepotemplate

# test results and coverage info
./go.test.sh

# install as a utility package
go install

Use the Issues and PR templates on the GitHub repo page to contribute.


Basic Usage

This is a copy of the example script available in the cmd/example/gorepotemplate folder:

package main

import "github.com/skeptycal/gorepotemplate"

func main() {
    gorepotemplate.Example()
}

To try it out:

# change to the sample folder
cd cmd/example/gorepotemplate

# run the main.go program
go run ./main.go

# to compile as an executable
go build

Code of Conduct and Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us. Please read the Code of Conduct for details before submitting anything.


Versioning

We use SemVer for versioning. For the versions available, see the tags on this repository.


Contributors and Inspiration

  • Michael Treanor (GitHub / Twitter) - Initial work, updates, maintainer
  • Francesc Campoy - Inspiration and great YouTube videos!

See also the list of contributors who participated in this project.


License

Licensed under the MIT https://opensource.org/licenses/MIT - see the LICENSE file for details.

Documentation

Overview

Package gofile provides support for file operations.

Package fs defines basic interfaces to a file system. A file system can be provided by the host operating system but also by other packages.

Index

Constants

View Source
const (
	NormalMode        os.FileMode = 0644
	DirMode           os.FileMode = 0755
	MinBufferSize                 = 16
	SmallBufferSize               = 64
	Chunk                         = 512
	DefaultBufferSize             = 1024
	DefaultBufSize                = 4096
	MaxInt                        = int(^uint(0) >> 1)
	MinRead                       = bytes.MinRead
)
View Source
const (
	PathSep = os.PathSeparator
	ListSep = os.PathListSeparator
	NewLine = '\n'
)

Variables

View Source
var (
	ErrNoAlloc        = errors.New("failed to allocate memory for file")
	ErrNotImplemented = errors.New("not implemented")

	ErrFsInvalid        = fs.ErrInvalid
	ErrPermission       = fs.ErrPermission
	ErrExist            = fs.ErrExist
	ErrNotExist         = fs.ErrNotExist
	ErrClosed           = fs.ErrClosed
	ErrInvalid          = os.ErrInvalid
	ErrNoDeadline       = os.ErrNoDeadline
	ErrDeadlineExceeded = os.ErrDeadlineExceeded
	ErrProcessDone      = os.ErrProcessDone
	ErrClosedPipe       = io.ErrClosedPipe
	ErrNoProgress       = io.ErrNoProgress
	ErrShortBuffer      = io.ErrShortBuffer
	ErrShortWrite       = io.ErrShortWrite
	ErrUnexpectedEOF    = io.ErrUnexpectedEOF
	ErrBadPattern       = filepath.ErrBadPattern
)

Portable analogs of some common errors.

Errors returned from this package may be tested against these errors with errors.Is.

View Source
var Err = log.Err

Err is the logging function for the global ErrorLogger.

Err logs an error to the provided logger, if it is enabled, and returns the error unchanged.

View Source
var NewSyscallError = os.NewSyscallError

NewSyscallError returns, as an error, a new SyscallError with the given system call name and error details. As a convenience, if err is nil, NewSyscallError returns nil.

View Source
var SameFile = os.SameFile

SameFile reports whether fi1 and fi2 describe the same file. For example, on Unix this means that the device and inode fields of the two underlying structures are identical; on other systems the decision may be based on the path names. SameFile only applies to results returned by this package gofile It returns false in other cases.

Functions

func Copy

func Copy(src, dest string) (int64, error)

func CopyBuffer

func CopyBuffer(src, dst string, buffersize int) (written int64, err error)

func CopyUtil

func CopyUtil(src, dst string) (written int64, err error)

func Create

func Create(name string) (io.ReadWriteCloser, error)

Create creates or truncates the named file and returns an opened file as io.ReadWriteCloser.

If the file already exists, it is truncated. If the file does not exist, it is created with mode 0644 (before umask). If successful, methods on the returned File can be used for I/O; the associated file descriptor has mode O_RDWR.

If there is an error, it will be of type *PathError.

func CreateSafe

func CreateSafe(name string) (io.ReadWriteCloser, error)

CreateSafe creates the named file and returns an opened file as io.ReadWriteCloser.

If the file already exists, an error is returned. If the file does not exist, it is created with mode 0644 (before umask). If successful, methods on the returned File can be used for I/O; the associated file descriptor has mode O_RDWR.

If there is an error, it will be of type *PathError.

func Exists

func Exists(filename string) bool

func InitialCapacity

func InitialCapacity(capacity int) int

InitialCapacity returns the multiple of 'chunk' one more than needed to accomodate the given capacity.

func IsDir

func IsDir(name string) bool

IsDir reports whether a file is a directory. That is, it tests for the ModeDir bit being set in m.

func IsRegular

func IsRegular(name string) bool

IsRegular reports whether a file describes a regular file. That is, it tests that no mode type bits are set.

func Mode

func Mode(file string) os.FileMode

Mode returns the filemode of file.

func NotExists

func NotExists(filename string) bool

func PWD

func PWD() string

PWD returns a rooted path name corresponding to the current directory. If the current directory can be reached via multiple paths (due to symbolic links), PWD may return any one of them.

If an error occurs, the empty string is returned and the error is logged.

func Stat

func Stat(filename string) (os.FileInfo, error)

Stat returns the os.FileInfo for file if it exists.

It is a convenience wrapper for os.Stat that traps and processes errors that may occur using the the ErrorLogger package.

If the file does not exist, nil is returned. Errors are logged if Err is active.

func StatCheck

func StatCheck(filename string) (os.FileInfo, error)

StatCheck returns file information (after symlink evaluation) using os.Stat(). If the file does not exist, is not a regular file, or if the user lacks adequate permissions, an error is returned. StatCheck returns file information (after symlink evaluation and path cleaning) using os.Stat().

If the file does not exist, is not a regular file, or if the user lacks adequate permissions, an error is returned.

It is a convenience wrapper for os.Stat that traps and processes errors that may occur using the the ErrorLogger package.

If the file does not exist, nil is returned. Errors are logged if Err is active.

Types

type BasicFile

type BasicFile interface {
	// Handle returns the file handle, *os.File.
	// The minimum interface that is implemented
	// by a File is:
	GoFile
	//  Stat()
	Handle() *os.File

	// Stat returns the FileInfo portion of the
	// BasicFile interface.
	Stat() (fs.FileInfo, error)

	FileModer
	FileInfo

	// Additional basic methods:
	Abs() string // absolute path of the file

}

A BasicFile provides access to a single file as an in memory buffer.

The BasicFile interface is the minimum implementation required of the file and may be extended to specific file types. (e.g. CSV, JSON, Esri Shapefile, config files, etc.)

It may also be implemented as an abstract "file" interface that provides access to a single file that is too large to fit in memory at once.

An implementation for large files should include a way to cache one section at a time, perhaps using a maxAlloc value or a mutex of file sections.

Caching write requests will likely be the bottleneck and collecting multiple write requests and then writing the results of the most recent or most active areas of the file may be effective. However, performance profiling and some research into whether a database is more efficient is warranted.

It could also be implemented as a way to access a database, API, buffer, or other storage.

A file may implement additional interfaces, such as ReadDirFile, ReaderAt, or Seeker, to provide additional or optimized functionality.

type FileModer interface {
	String() string
	IsDir() bool
	IsRegular() bool
	Perm() FileMode
	Type() FileMode
}

A FileInfo describes a file and is returned by Stat.

type FileInfo interface {
    Name() string       // base name of the file
    Size() int64        // length in bytes for regular files; system-dependent for others
    Mode() FileMode     // file mode bits
    ModTime() time.Time // modification time
    IsDir() bool        // abbreviation for Mode().IsDir()
    Sys() interface{}   // underlying data source (can return nil)
}

Reference: standard library fs.go using File, FileInfo, and FileModer interfaces

Minimum required to implement fs.File interface:

type File interface {
   Stat() (fs.FileInfo, error)
   Read([]byte) (int, error)
   Close() error
}

Implements fs.FileInfo interface:

	// A FileInfo describes a file and is returned by Stat.
 type FileInfo interface {
 	Name() string       // base name of the file
	Size() int64        // length in bytes for regular files; sy stem-dependent for others
 	Mode() FileMode     // file mode bits
 	ModTime() time.Time // modification time
 	IsDir() bool        // abbreviation for Mode().IsDir()
 	Sys() interface{}   // underlying data source (can return nil)
 }

func NewBasicFile

func NewBasicFile(filename string) (BasicFile, error)

func NewFile

func NewFile(providedName string) BasicFile

NewFile returns a new BasicFile, but no error, as is the custom in the standard library os package. Most often, if a file cannot be opened or created, we do not care why. It is often beyond the scope of the application to correct these types of errors. It simply means that we cannot proceed.

This allows for convenient inline usage with the standard pattern of Go nil checking. If errorlogger is active, any error is still recoreded in the log. This offloads error logging duties to errorlogger, or whichever standard library compatible logger function is assigned to the global logger function:

func Err(err error) error

TLDR: Check for nil if you only want to know whether *any* error occurred. If you care about a *specific* error, use

NewFileWithErr() (f *os.File, err error)

for a more os.Open()-ish way.

func NewFileWithErr

func NewFileWithErr(providedName string) (BasicFile, error)

func Open

func Open(name string) (BasicFile, error)

Open opens the named file for reading as an in memory object. If successful, methods on the returned file can be used for reading; the associated file descriptor has mode O_RDONLY. If there is an error, it will be of type *os.PathError.

type Closer

type Closer interface {
	Close() error
}

Reference types copied from io package

type DIR

type DIR interface {
	Len() int
	Path() string
	List() ([]BasicFile, error)
	SetOpts(opts dirOpts)

	// Chdir changes the current working directory to the file, which must be a directory. If there is an error, it will be of type *PathError.
	Chdir() error
}

type DataFile

type DataFile = BasicFile // todo - finish DataFile implementation

type DirEntry

type DirEntry = fs.DirEntry

A DirEntry is an entry read from a directory (using the ReadDir function or a ReadDirFile's ReadDir method).

type DirEntry interface {
	// Name returns the name of the file (or subdirectory) described by the entry.
	// This name is only the final element of the path (the base name), not the entire path.
	// For example, Name would return "hello.go" not "/home/gopher/hello.go".
	Name() string

 	// IsDir reports whether the entry describes a directory.
	IsDir() bool

 	// Type returns the type bits for the entry.
	// The type bits are a subset of the usual FileMode bits, those returned by the FileMode.Type method.
	Type() FileMode

 	// Info returns the FileInfo for the file or subdirectory described by the entry.
	// The returned FileInfo may be from the time of the original directory read
	// or from the time of the call to Info. If the file has been removed or renamed
	// since the directory read, Info may return an error satisfying errors.Is(err, ErrNotExist).
	// If the entry denotes a symbolic link, Info reports the information about the link itself,
	// not the link's target.
	Info() (FileInfo, error)
}

Reference: standard library fs.go

type Errer

type Errer interface {
	Error() string
	// Wrap() error
	Unwrap() error
	Timeout() bool
}

Errer implements the common error methods associated with file and system errors.

func NewPathError

func NewPathError(op, path string, err error) Errer

NewPathError returns, as an error, a new PathError with the given operation and file path that caused it. The error will be of type *os.PathError As a convenience, if err is nil, NewPathError returns nil.

type FS

type FS = fs.FS

An FS provides access to a hierarchical file system.

The FS interface is the minimum implementation required of the file system. A file system may implement additional interfaces, such as ReadFileFS, to provide additional or optimized functionality.

type FS interface {
	// Open opens the named file.
	//
	// When Open returns an error, it should be of type *PathError
	// with the Op field set to "open", the Path field set to name,
	// and the Err field describing the problem.
	//
	// Open should reject attempts to open names that do not satisfy
	// ValidPath(name), returning a *PathError with Err set to
	// ErrInvalid or ErrNotExist.
	Open(name string) (File, error)
}

Reference: standard library fs.go

type File

type File = fs.File

A File provides access to a single file. The File interface is the minimum implementation required of the file. A file may implement additional interfaces, such as ReadDirFile, ReaderAt, or Seeker, to provide additional or optimized functionality.

type File interface {
	Stat() (fs.FileInfo, error)
	Read([]byte) (int, error)
	Close() error
}

Reference: standard library fs.go

type FileInfo

type FileInfo = fs.FileInfo

A FileInfo describes a file and is returned by Stat.

Implements fs.FileInfo interface:

type FileInfo interface {
	Name() string       // base name of the file
	Size() int64        // length in bytes for regular files; system-dependent for others
	Mode() FileMode     // file mode bits
	ModTime() time.Time // modification time
	IsDir() bool        // abbreviation for Mode().IsDir()
	Sys() interface{}   // underlying data source (can return nil)
}

Reference: standard library fs.go

type FileMode

type FileMode = fs.FileMode

A FileMode represents a file's mode and permission bits. The bits have the same definition on all systems, so that information about files can be moved from one system to another portably. Not all bits apply to all systems. The only required bit is ModeDir for directories.

type FileMode uint32

Reference: standard library fs.go

type FileModer

type FileModer interface {

	// human-readable representation of the file
	String() string

	// IsDir reports whether m describes a directory.
	// That is, it tests for the ModeDir bit being set in m.
	IsDir() bool

	// IsRegular reports whether m describes a regular file.
	// That is, it tests that no mode type bits are set.
	IsRegular() bool

	// Perm returns the Unix permission bits in m (m & ModePerm).
	Perm() FileMode

	// Type returns type bits in m (m & ModeType).
	Type() FileMode
}

FileModer implements fs.FileMode methods

A FileMode represents a file's mode and permission bits. The bits have the same definition on all systems, so that information about files can be moved from one system to another portably. Not all bits apply to all systems. The only required bit is ModeDir for directories.

type FileOps

type FileOps interface {
	Chmod(mode os.FileMode) error
	Move(path string) error
	Abs() (string, error)
	Split(path string) (dir, file string)
	Base(path string) string
	Dir(path string) string
	Ext(path string) string
}

Reference types copied from io package

type GoDir

type GoDir interface {
	ReadDirFile
	GoFile
	Chdir() error

	Readdirnames(dir string) (n int, err error)
}

Reference types copied from io package

type GoFile

type GoFile interface {
	File
	Writer
	StringWriter
	ToFrom
	ReadWriterAt
	Seeker
	fmt.Stringer

	Open(name string) (http.File, error)

	// FileInfo methods
	Name() string       // base name of the file
	Size() int64        // length in bytes for regular files; system-dependent for others
	Mode() FileMode     // file mode bits
	ModTime() time.Time // modification time
	// IsDir() bool        // abbreviation for Mode().IsDir()
	Sys() any

	// FileMode methods
	String() string // human-readable representation of the file
	IsDir() bool    // abbreviation for Mode().IsDir()
	IsRegular() bool
	Perm() FileMode
	Type() FileMode

	// Unix File Operations
	Truncate(size int64) error
	Remove() error
	Link(newname string) error
	Symlink(newname string) error
	Readlink() (string, error)
	Fd() uintptr

	Chmod(mode FileMode) error
	Chown(uid int, gid int) error

	SetDeadline(t time.Time) error
	SetReadDeadline(t time.Time) error
	SetWriteDeadline(t time.Time) error

	Sync() error

	SyscallConn() (syscall.RawConn, error)
}

Reference types copied from io package

type GoFileError

type GoFileError struct {
	Op   string
	Path string
	Err  error
}

GoFileError records an error from a specific GoFile function call. If there is a relevant path, the path will be set and the error will be of type *os.PathError

func NewGoFileError

func NewGoFileError(op, path string, err error) *GoFileError

NewGoFileError returns, as an error, a new GoFileError which implements the gofile.Errer interface. If there is a relevant path, the path will be set and the error will wrap *os.PathError. As a convenience, if err is nil, NewGoFileError returns nil.

func (*GoFileError) As

func (e *GoFileError) As(any)

As finds the first error in err's chain that matches target, and if one is found, sets target to that error value and returns true. Otherwise, it returns false.

The chain consists of err itself followed by the sequence of errors obtained by repeatedly calling Unwrap.

An error matches target if the error's concrete value is assignable to the value pointed to by target, or if the error has a method As(interface{}) bool such that As(target) returns true. In the latter case, the As method is responsible for setting target.

An error type might provide an As method so it can be treated as if it were a different error type.

As panics if target is not a non-nil pointer to either a type that implements error, or to any interface type.

func (*GoFileError) Error

func (e *GoFileError) Error() string

func (*GoFileError) Timeout

func (e *GoFileError) Timeout() bool

Timeout reports whether this error represents a timeout.

func (*GoFileError) Unwrap

func (e *GoFileError) Unwrap() error

Unwrap returns the result of calling the Unwrap method on err, if err's type contains an Unwrap method. Otherwise, Unwrap returns nil.

func (*GoFileError) WithMessage

func (e *GoFileError) WithMessage(msg string) *GoFileError

func (*GoFileError) Wrap

func (e *GoFileError) Wrap(message string) *GoFileError

Wrap replaces the underlying error (err) with a wrapper annotating it with a stack trace at the point Wrap is called, and the supplied message.

A pointer to the new, wrapped error is returned. If err is nil, Wrap returns nil and performs no other operations.

func (*GoFileError) Wrapf

func (e *GoFileError) Wrapf(format string, args ...interface{}) *GoFileError

type PathError

type PathError = os.PathError

PathError records an error and the operation and file path that caused it.

type ReadCloser

type ReadCloser interface {
	Reader
	Closer
}

Reference types copied from io package

type ReadDirFile

type ReadDirFile = fs.ReadDirFile

A ReadDirFile is a directory file whose entries can be read with the ReadDir method. Every directory file should implement this interface. (It is permissible for any file to implement this interface, but if so ReadDir should return an error for non-directories.)

  type ReadDirFile interface {
  	File
		// ReadDir reads the contents of the directory and returns
		// a slice of up to n DirEntry values in directory order.
		// Subsequent calls on the same file will yield further DirEntry values.
		//
		// If n > 0, ReadDir returns at most n DirEntry structures.
		// In this case, if ReadDir returns an empty slice, it will return
		// a non-nil error explaining why.
		// At the end of a directory, the error is io.EOF.
		//
		// If n <= 0, ReadDir returns all the DirEntry values from the directory
		// in a single slice. In this case, if ReadDir succeeds (reads all the way
		// to the end of the directory), it returns the slice and a nil error.
		// If it encounters an error before the end of the directory,
		// ReadDir returns the DirEntry list read until that point and a non-nil error.
  	ReadDir(n int) ([]DirEntry, error)
  }

Reference: standard library fs.go

type ReadWriteCloser

type ReadWriteCloser interface {
	ReadCloser
	Writer
}

Reference types copied from io package

type ReadWriterAt

type ReadWriterAt interface {
	// ReaderAt is the interface that wraps the basic ReadAt method.
	//
	// ReadAt reads len(p) bytes into p starting at offset off in the
	// underlying input source. It returns the number of bytes
	// read (0 <= n <= len(p)) and any error encountered.
	//
	// When ReadAt returns n < len(p), it returns a non-nil error
	// explaining why more bytes were not returned. In this respect,
	// ReadAt is stricter than Read.
	//
	// Even if ReadAt returns n < len(p), it may use all of p as scratch
	// space during the call. If some data is available but not len(p) bytes,
	// ReadAt blocks until either all the data is available or an error occurs.
	// In this respect ReadAt is different from Read.
	//
	// If the n = len(p) bytes returned by ReadAt are at the end of the
	// input source, ReadAt may return either err == EOF or err == nil.
	//
	// If ReadAt is reading from an input source with a seek offset,
	// ReadAt should not affect nor be affected by the underlying
	// seek offset.
	//
	// Clients of ReadAt can execute parallel ReadAt calls on the
	// same input source.
	//
	// Implementations must not retain p.
	ReadAt(p []byte, off int64) (n int, err error)

	// WriterAt is the interface that wraps the basic WriteAt method.
	//
	// WriteAt writes len(p) bytes from p to the underlying data stream
	// at offset off. It returns the number of bytes written from p (0 <= n <= len(p))
	// and any error encountered that caused the write to stop early.
	// WriteAt must return a non-nil error if it returns n < len(p).
	//
	// If WriteAt is writing to a destination with a seek offset,
	// WriteAt should not affect nor be affected by the underlying
	// seek offset.
	//
	// Clients of WriteAt can execute parallel WriteAt calls on the same
	// destination if the ranges do not overlap.
	//
	// Implementations must not retain p.
	WriteAt(p []byte, off int64) (n int, err error)
}

ReadWriteAt implements io.ReaderAt and io.WriterAt for concurrent, non-overlapping reads and writes.

type Reader

type Reader interface {
	Read(p []byte) (n int, err error)
}

Reference types copied from io package

type Seeker

type Seeker interface {
	Seek(offset int64, whence int) (int64, error)
}

Seeker is the interface that wraps the basic Seek method.

Seek sets the offset for the next Read or Write to offset, interpreted according to whence: SeekStart means relative to the start of the file, SeekCurrent means relative to the current offset, and SeekEnd means relative to the end. Seek returns the new offset relative to the start of the file or an error, if any.

Seeking to an offset before the start of the file is an error. Seeking to any positive offset may be allowed, but if the new offset exceeds the size of the underlying object the behavior of subsequent I/O operations is implementation-dependent.

Reference: io.Seeker

type SortType

type SortType int

SortType is a list of constants representing sort methods for directory listings.

const (
	Alpha SortType = iota + 1
	Size
	Version
	Extension
	Atime
	Ctime
)

func (SortType) String

func (s SortType) String() string

type StringWriter

type StringWriter interface {
	WriteString(s string) (n int, err error)
}

StringWriter is the interface that wraps the WriteString method.

type SyscallError

type SyscallError = os.SyscallError

SyscallError records an error from a specific system call.

type TimeZone

type TimeZone int
const (
	EST TimeZone = -(5 + iota)
	CST
	MST
	PST
)

American Time Zones

func (TimeZone) String

func (tz TimeZone) String() string

type ToFrom

type ToFrom interface {

	// ReaderFrom is the interface that wraps the ReadFrom method.
	//
	// ReadFrom reads data from r until EOF or error. The return value n is the number of bytes read. Any error except EOF encountered during the read is also returned.
	//
	// The Copy function uses ReaderFrom if available.
	//
	// Reference: io.ReaderFrom
	ReadFrom(r Reader) (n int64, err error)

	// WriterTo is the interface that wraps the WriteTo method.
	//
	// WriteTo writes data to w until there's no more data to write or when an error occurs. The return value n is the number of bytes written. Any error encountered during the write is also returned.
	//
	// The Copy function uses WriterTo if available.
	//
	// Reference: io.WriterTo
	WriteTo(w Writer) (n int64, err error)
}

Reference types copied from io package

type WriteCloser

type WriteCloser interface {
	Writer
	Closer
}

Reference types copied from io package

type Writer

type Writer interface {
	Write(p []byte) (n int, err error)
}

Reference types copied from io package

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL