gofile

package module
v0.5.0-1630924068056288 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 6, 2021 License: MIT Imports: 12 Imported by: 2

README

gorepotemplate

Tricky and fun utilities for Go programs.


GitHub Workflow Status Codecov

Contributor Covenant

Twitter Follow GitHub followers


Getting Started

Prerequisites

Developed with $( go version; ). Go is extremely backwards compatible and semver stable. Nearly any v1.x should work fine.


Installation

To use this repo as a template for your own project:

gh repo create -y --public --template "https://github.com/skeptycal/gorepotemplate"

Clone this repo to test and contribute:

# add repo to $GOPATH (xxxxxx is your computer login username)
go get github.com/xxxxxx/gorepotemplate

cd ${GOPATH}/src/github.com/xxxxxx/gorepotemplate

# test results and coverage info
./go.test.sh

# install as a utility package
go install

Use the Issues and PR templates on the GitHub repo page to contribute.


Basic Usage

This is a copy of the example script available in the cmd/example/gorepotemplate folder:

package main

import "github.com/skeptycal/gorepotemplate"

func main() {
    gorepotemplate.Example()
}

To try it out:

# change to the sample folder
cd cmd/example/gorepotemplate

# run the main.go program
go run ./main.go

# to compile as an executable
go build

Code of Conduct and Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us. Please read the Code of Conduct for details before submitting anything.


Versioning

We use SemVer for versioning. For the versions available, see the tags on this repository.


Contributors and Inspiration

  • Michael Treanor (GitHub / Twitter) - Initial work, updates, maintainer
  • Francesc Campoy - Inspiration and great YouTube videos!

See also the list of contributors who participated in this project.


License

Licensed under the MIT https://opensource.org/licenses/MIT - see the LICENSE file for details.

Documentation

Overview

Package gofile provides support for file operations.

Package fs defines basic interfaces to a file system. A file system can be provided by the host operating system but also by other packages.

Index

Constants

View Source
const (
	NormalMode        os.FileMode = 0644
	DirMode           os.FileMode = 0755
	MinBufferSize                 = 16
	SmallBufferSize               = 64
	Chunk                         = 512
	DefaultBufferSize             = 1024
	DefaultBufSize                = 4096
	MaxInt                        = int(^uint(0) >> 1)
	MinRead                       = bytes.MinRead
)
View Source
const (
	PathSep = os.PathSeparator
	ListSep = os.PathListSeparator
	NewLine = '\n'
)

Variables

View Source
var (
	ErrBadCount   = errors.New("datafile: bad read count")
	ErrNotRegular = errors.New("datafile: not regular file")
)
View Source
var (
	// Log is the default global ErrorLogger. It implements the ErrorLogger interface as well as the logrus.Logger interface, which is compatible with the standard library "log" package gofile
	Log = el.Log

	// Err is the logging function for the global ErrorLogger.
	Err = el.Err
)

errorlogger implements error logging to a logrus log by default. It is completely compatible with the standard library 'log' logger API. It provides an efficient and convenient way to log errors with minimal overhead and to temporarily disable or enable logging.

A global Log and Err with default behaviors are supplied that may be aliased if you wish:

Log = errorlogger.Log // implements the ErrorLogger interface
Err = errorlogger.Err // trigger function

ErrorLogger is:

 type ErrorLogger interface {
	 Disable()
	 Enable()
	 Err(err error) error
	 SetLoggerFunc(fn func(args ...interface{}))

	// SetErrorType allows ErrorLogger to wrap errors in a
	// specified custom type. For example, if you want all errors
 // returned to be of type *os.PathError
	 SetErrorType(errType error)
}
View Source
var (
	FSErr         = errorlogger.Err
	ErrNoAlloc    = errors.New("failed to allocate memory for file")
	ErrClosed     = fs.ErrClosed
	ErrInvalid    = fs.ErrInvalid
	ErrPermission = fs.ErrPermission
	ErrExist      = NewGoFileError("gofile", "", fs.ErrExist)
	ErrNotExist   = NewGoFileError("gofile", "", fs.ErrNotExist)
)
View Source
var SameFile = os.SameFile

SameFile reports whether fi1 and fi2 describe the same file. For example, on Unix this means that the device and inode fields of the two underlying structures are identical; on other systems the decision may be based on the path names. SameFile only applies to results returned by this package gofile It returns false in other cases.

Functions

func Copy

func Copy(src, dest string) (int64, error)

func CopyBuffer

func CopyBuffer(src, dst string, buffersize int) (written int64, err error)

func CopyUtil

func CopyUtil(src, dst string) (written int64, err error)

func Create

func Create(name string) (io.ReadWriteCloser, error)

Create creates or truncates the named file and returns an opened file as io.ReadWriteCloser.

If the file already exists, it is truncated. If the file does not exist, it is created with mode 0644 (before umask). If successful, methods on the returned File can be used for I/O; the associated file descriptor has mode O_RDWR.

If there is an error, it will be of type *PathError.

func CreateSafe

func CreateSafe(name string) (io.ReadWriteCloser, error)

CreateSafe creates the named file and returns an opened file as io.ReadWriteCloser.

If the file already exists, an error is returned. If the file does not exist, it is created with mode 0644 (before umask). If successful, methods on the returned File can be used for I/O; the associated file descriptor has mode O_RDWR.

If there is an error, it will be of type *PathError.

func InitialCapacity

func InitialCapacity(capacity int64) int

InitialCapacity returns the multiple of 'chunk' one more than needed to accomodate the given capacity.

func IsDir

func IsDir(name string) bool

func IsRegular

func IsRegular(name string) bool

func Mode

func Mode(file string) os.FileMode

Mode returns the filemode of file.

func PWD

func PWD() string

PWD returns a rooted path name corresponding to the current directory. If the current directory can be reached via multiple paths (due to symbolic links), PWD may return any one of them.

If an error occurs, the empty string is returned and the error is logged.

func Stat

func Stat(file string) (os.FileInfo, error)

Stat returns the os.FileInfo for file if it exists. If the file does not exist, nil is returned. Errors are logged if Err is active.

func StatCheck

func StatCheck(filename string) (os.FileInfo, error)

StatCheck returns file information (after symlink evaluation) using os.Stat(). If the file does not exist, is not a regular file, or if the user lacks adequate permissions, an error is returned.

Types

type BasicFile

type BasicFile interface {
	File
	FileInfo
}

A BasicFile provides access to a single file as an in memory buffer.

The BasicFile interface is the minimum implementation required of the file and may be extended to specific file types. (e.g. CSV, JSON, Esri Shapefile, config files, etc.)

It may also be implemented as an abstract "file" interface that provides access to a single file that is too large to fit in memory at once.

An implementation for large files should include a way to cache one section at a time, perhaps using a maxAlloc value or a mutex of file sections.

Caching write requests will likely be the bottleneck and collecting multiple write requests and then writing the results of the most recent or most active areas of the file may be effective. However, performance profiling and some research into whether a database is more efficient is warranted.

It could also be implemented as a way to access a database, API, buffer, or other storage.

A file may implement additional interfaces, such as ReadDirFile, ReaderAt, or Seeker, to provide additional or optimized functionality.

Reference: standard library fs.go

func Open

func Open(name string) (BasicFile, error)

Open opens the named file for reading as an in memory object. If successful, methods on the returned file can be used for reading; the associated file descriptor has mode O_RDONLY. If there is an error, it will be of type *os.PathError.

type Basicfile

type Basicfile struct {
	ProvidedName string // file name as provided by various constructor methods

	os.FileInfo   // os.FileInfo interface
	*bytes.Buffer // buffered io.ReadWriter
	// contains filtered or unexported fields
}

Basicfile provides implements BasicFile by providing access to information and data from a single local file. It is designed to cache file information and contents in memory.

func NewBasicFile

func NewBasicFile(filename string) (*Basicfile, error)

NewBasicFile creates a new basic file with the provided name.

The underlying file will be created if it does not exist. Create creates or truncates the named file. If the file already exists, it is truncated. If the file does not exist, it is created with mode 0666 (before umask). If successful, methods on the returned File can be used for I/O; the associated file descriptor has mode O_RDWR. If there is an error, it will be of type *PathError.

func (*Basicfile) Abs

func (d *Basicfile) Abs() string

Abs returns an absolute representation of the file's path. If the path is not absolute it will be joined with the current working directory to turn it into an absolute path. The absolute path name for a given file is not guaranteed to be unique. Abs calls Clean on the result.

The path is cached in the basicfile object. If the file is moved to another location, a new basicfile object should be created to track it.

If an error occurs, it is logged and the empty string is returned.

func (*Basicfile) Close

func (d *Basicfile) Close() (err error)

Close resets the in memory buffer that is used to cache the file contents. The actual file was closed after reading the data into memory.

Reset resets the buffer to be empty, but it retains the underlying storage for use by future writes. This will cause problems for long running programs that access many files. Use Purge() to release the buffer.

func (*Basicfile) Data

func (d *Basicfile) Data() ([]byte, error)

Data returns the contents of the data source buffer in the format expected in common use cases.

For typical files, this is []byte, but it may be implemented as a CSV string, Shapefile object, serialized JSON, or other format.

func (*Basicfile) File

func (d *Basicfile) File() (f *os.File, err error)

File returns a file pointer to the underlying file.

func (*Basicfile) IsRegular

func (d *Basicfile) IsRegular() bool

IsRegular reports whether the file is a regular file. That is, it tests that no mode type bits are set.

func (*Basicfile) Move

func (d *Basicfile) Move(dst string) error

Move moves the file to another location and updates the basicfile object with the result. If an error occurs, it is logged and returned.

If the dst string provided does not contain an absolute path, the relative path of the source BasicFile is used. This means that the file is actually renamed to dst in the same directory.

If the destination file already exists, an error is returned. To replace the destination file with the source file, use Rename().

func (*Basicfile) Purge

func (d *Basicfile) Purge() error

Purge clears the in memory buffer and resets the capacity to zero.

func (*Basicfile) ReadFile

func (d *Basicfile) ReadFile() (n int, err error)

ReadFile reads the file using os.ReadFile and writes the data into the buffer.

The buffer is *reset* before calling os.ReadFile. All data in the buffer is *truncated* at this time. If the desired outcome is to APPEND the data to the current buffer, use ReadFrom() instead.

A successful call returns err == nil, not err == EOF. Because ReadFile reads the whole file, it does not treat an EOF from Read as an error to be reported.

The buffer contents are not guaranteed to be written to the underlying file until FSync() is called. If WriteCache is false, FSync() is called immediately.

func (*Basicfile) ReadFrom

func (d *Basicfile) ReadFrom(r io.Reader) (n int64, err error)

ReadFrom reads data from the underlying file until EOF and appends it to the buffer, growing the buffer as needed. The return value n is the number of bytes read. Any error except io.EOF encountered during the read is also returned. If the buffer becomes too large, ReadFrom will panic with ErrTooLarge.

The buffer contents are not guaranteed to be written to the underlying file until FSync() is called.

The buffer is not reset prior to calling ReadFrom. If this is desired, use ReadFile instead.

func (*Basicfile) Rename

func (d *Basicfile) Rename(dst string) error

Rename renames (moves) the underlying file to dst. If dst already exists and is not a directory, Rename replaces it. OS-specific restrictions may apply when they are in different directories. If there is an error, it will be of type *LinkError.

func (*Basicfile) SetData

func (d *Basicfile) SetData(p []byte) (n int, err error)

SetData truncates the buffer and reads p into it.

func (*Basicfile) Stat

func (d *Basicfile) Stat() (FileInfo, error)

Stat returns the cached file information for the underlying BasicFile. It is intended to be used to cache permanent information, such as file name, Mode(), IsDir(), etc.

While this works fine with permanent values such as the file name, it will not give useful results for data that are often changed. (e.g. Last Access, Size) In those cases, use os.Stat() directly instead.

func (*Basicfile) String

func (d *Basicfile) String() string

type DIR

type DIR interface {
	Len() int
	Path() string
	List() ([]DataFile, error)
	SetOpts(opts dirOpts)
}

func NewDIR

func NewDIR(name string) (DIR, error)

type DataFile

type DataFile interface {
	BasicFile
	Data() ([]byte, error)
}

func NewDataFile

func NewDataFile(filename string) (DataFile, error)

type Datafile

type Datafile struct {
	Basicfile
	// contains filtered or unexported fields
}

datafile is a file type that is specialized for binary data

type DirEntry

type DirEntry = fs.DirEntry

A DirEntry is an entry read from a directory (using the ReadDir function or a ReadDirFile's ReadDir method).

type DirEntry interface {
	// Name returns the name of the file (or subdirectory) described by the entry.
	// This name is only the final element of the path (the base name), not the entire path.
	// For example, Name would return "hello.go" not "/home/gopher/hello.go".
	Name() string

 	// IsDir reports whether the entry describes a directory.
	IsDir() bool

 	// Type returns the type bits for the entry.
	// The type bits are a subset of the usual FileMode bits, those returned by the FileMode.Type method.
	Type() FileMode

 	// Info returns the FileInfo for the file or subdirectory described by the entry.
	// The returned FileInfo may be from the time of the original directory read
	// or from the time of the call to Info. If the file has been removed or renamed
	// since the directory read, Info may return an error satisfying errors.Is(err, ErrNotExist).
	// If the entry denotes a symbolic link, Info reports the information about the link itself,
	// not the link's target.
	Info() (FileInfo, error)
}

Reference: standard library fs.go

type FS

type FS = fs.FS

An FS provides access to a hierarchical file system.

The FS interface is the minimum implementation required of the file system. A file system may implement additional interfaces, such as ReadFileFS, to provide additional or optimized functionality.

type FS interface {
	// Open opens the named file.
	//
	// When Open returns an error, it should be of type *PathError
	// with the Op field set to "open", the Path field set to name,
	// and the Err field describing the problem.
	//
	// Open should reject attempts to open names that do not satisfy
	// ValidPath(name), returning a *PathError with Err set to
	// ErrInvalid or ErrNotExist.
	Open(name string) (File, error)
}

Reference: standard library fs.go

type File

type File = fs.File

A File provides access to a single file. The File interface is the minimum implementation required of the file. A file may implement additional interfaces, such as ReadDirFile, ReaderAt, or Seeker, to provide additional or optimized functionality.

type File interface {
	Stat() (fs.FileInfo, error)
	Read([]byte) (int, error)
	Close() error
}

Reference: standard library fs.go

type FileInfo

type FileInfo = fs.FileInfo

A FileInfo describes a file and is returned by Stat.

type FileInfo interface {
	Name() string       // base name of the file
	Size() int64        // length in bytes for regular files; system-dependent for others
	Mode() FileMode     // file mode bits
	ModTime() time.Time // modification time
	IsDir() bool        // abbreviation for Mode().IsDir()
	Sys() interface{}   // underlying data source (can return nil)
}

Reference: standard library fs.go

type FileMode

type FileMode = fs.FileMode

A FileMode represents a file's mode and permission bits. The bits have the same definition on all systems, so that information about files can be moved from one system to another portably. Not all bits apply to all systems. The only required bit is ModeDir for directories.

type FileMode uint32

Reference: standard library fs.go

type FileModer

type FileModer interface {
	String() string
	IsDir() bool
	IsRegular() bool
	Perm() FileMode
	Type() FileMode
}

FileModer implements fs.FileMode functionality ... yes, I hate the name, too

type GoFile

type GoFile interface {
	Name() string
	Read(b []byte) (n int, err error)
	ReadAt(b []byte, off int64) (n int, err error)
	ReadFrom(r io.Reader) (n int64, err error)
	Write(b []byte) (n int, err error)
	WriteAt(b []byte, off int64) (n int, err error)
	Seek(offset int64, whence int) (ret int64, err error)
	WriteString(s string) (n int, err error)
	Chmod(mode FileMode) error
	SetDeadline(t time.Time) error
	SetReadDeadline(t time.Time) error
	SetWriteDeadline(t time.Time) error
	SyscallConn() (syscall.RawConn, error)
}

type GoFileError

type GoFileError struct {
	Op   string
	Path string
	Err  error
}

func NewGoFileError

func NewGoFileError(op, path string, err error) *GoFileError

NewGoFileError returns a new GoFileError which is also an os.PathError

func (*GoFileError) Error

func (e *GoFileError) Error() string

func (*GoFileError) Timeout

func (e *GoFileError) Timeout() bool

Timeout reports whether this error represents a timeout.

func (*GoFileError) Unwrap

func (e *GoFileError) Unwrap() error

type PathError

type PathError = os.PathError

PathError records an error and the operation and file path that caused it.

 type PathError struct {
 	Op   string
 	Path string
 	Err  error
 }

 interface {
		Error() string
		Unwrap() error
		Timeout() bool
	}

func NewPathError

func NewPathError(op, path string, err error) *PathError

NewPathError records an error and the operation and file path that caused it.

type PathError struct {
	Op   string
	Path string
	Err  error
}

type PathErrorWrapper

type PathErrorWrapper interface {
	Error() string
	Unwrap() error
	Timeout() bool
}

PathErrorWrapper impliments an interface to fs.PathError which records an error and the operation and file path that caused it.

Any error passed in is wrapped with *fs.PathError as well as GoFileError and may be tested with errors.Is().

Use NewPathError to create a new PathErrorWrapper.

type ReadDirFile

type ReadDirFile = fs.ReadDirFile

A ReadDirFile is a directory file whose entries can be read with the ReadDir method. Every directory file should implement this interface. (It is permissible for any file to implement this interface, but if so ReadDir should return an error for non-directories.)

  type ReadDirFile interface {
  	File
		// ReadDir reads the contents of the directory and returns
		// a slice of up to n DirEntry values in directory order.
		// Subsequent calls on the same file will yield further DirEntry values.
		//
		// If n > 0, ReadDir returns at most n DirEntry structures.
		// In this case, if ReadDir returns an empty slice, it will return
		// a non-nil error explaining why.
		// At the end of a directory, the error is io.EOF.
		//
		// If n <= 0, ReadDir returns all the DirEntry values from the directory
		// in a single slice. In this case, if ReadDir succeeds (reads all the way
		// to the end of the directory), it returns the slice and a nil error.
		// If it encounters an error before the end of the directory,
		// ReadDir returns the DirEntry list read until that point and a non-nil error.
  	ReadDir(n int) ([]DirEntry, error)
  }

Reference: standard library fs.go

type SortType

type SortType int

SortType is a list of constants representing sort methods for directory listings.

const (
	Alpha SortType = iota + 1
	Size
	Version
	Extension
	Atime
	Ctime
)

func (SortType) String

func (s SortType) String() string

type TextFile

type TextFile interface {
	BasicFile
	Text() string
	Lines() (retval []string, err error)
	Sep(c byte)
}

type TimeZone

type TimeZone int
const (
	EST TimeZone = -(5 + iota)
	CST
	MST
	PST
)

American Time Zones

func (TimeZone) String

func (tz TimeZone) String() string

Directories

Path Synopsis
cmd
dir command
redlog command
Package redlogger implements logging to an io.Writer (default stderr) with text wrapped in ANSI escape codes
Package redlogger implements logging to an io.Writer (default stderr) with text wrapped in ANSI escape codes

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL