fsx

package module
v1.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 27, 2026 License: Apache-2.0 Imports: 19 Imported by: 0

README

fsx

Go Version License Release Build Status PRs Welcome

A Go filesystem extension package that provides convenient helpers for common file and directory operations: absolute path resolution, blank-line cleanup, content trimming by markers, line/character counting, embedded filesystem helpers, ZIP archiving, HTTP file download/upload, and more.

Installation

go get -u github.com/fishfinal/fsx

Features

  • Absolute path resolution
  • Remove blank lines from files or []byte data
  • Remove content before/after a specified marker string
  • Count lines and characters in files or []byte data
  • File and directory existence checks (including embed.FS)
  • Create, truncate, touch, copy, and read directories
  • Filename and extension utilities
  • Read file content as []byte or line slices
  • Extract lines matching a regular expression
  • Remove lines matching a regular expression
  • ZIP archive creation from a file or directory
  • HTTP file download helpers (with RFC 5987 filename encoding)
  • HTTP upload MIME type validation
  • Response header helpers (Content-Disposition filename parsing)
  • Go version parsing (VersionGithub)

Quick Start

Absolute path
path, err := fsx.AbsPath("./data", true) // create the directory if it does not exist
if err != nil {
    log.Fatal(err)
}
fmt.Println(path)
Clean blank lines
// From bytes
cleaned := fsx.CleanBlankLines([]byte("hello\n\nworld\n"))

// From a file
cleaned, err := fsx.CleanBlankLinesForFile("input.txt")
Trim content by marker
data, _ := os.ReadFile("article.txt")

// Keep content from the marker onward (marker included by default)
prefix := fsx.MarkContentPrefixClear(data, "START")

// Keep content up to the marker (marker excluded by default)
suffix := fsx.MarkContentSuffixClear(data, "END")

// Control whether the marker string itself is kept
prefix = fsx.MarkContentPrefixClear(data, "START",
    fsx.WithSpecificContainsMarkString(false))
Count lines and characters
lines := fsx.Lines([]byte("a\nb\nc"))           // 3
chars := fsx.Characters([]byte("a\nb\nc"))      // 3 (blank lines ignored)

lines, err := fsx.LinesForFile("input.txt")
chars, err = fsx.CharactersForFile("input.txt")
File existence
exists := fsx.Exists("config.yaml")

// Works with embed.FS as well
//go:embed static/*
var staticFS embed.FS

exists = fsx.EmbedsFileExists(staticFS, "static/css/style.css")
exists = fsx.EmbedsExists(staticFS, "static/images/")
Create a file
path, err := fsx.CreateFile("report", []byte("hello"),
    fsx.WithSpecificBaseDir("output"),
    fsx.WithSpecificFileSystemPath("docs"),
    fsx.WithSpecificFileExtension(fsx.FileExtensionWithDocumentOfMarkdown),
)
// output/docs/static/report.md
Touch / truncate / copy
_ = fsx.Touch("logs/app.log")            // creates parent dirs if needed
_ = fsx.Truncate("logs/app.log", 0)      // truncate to zero bytes

files, _ := fsx.ReadDir("src", nil)
_ = fsx.WriteDir("dst", files, fsx.WithOriginalFileNameTrimPrefix("src"))
Read file content as line slices
lines, err := fsx.GetFileContentStringSliceWithFilename("input.txt",
    fsx.WithIgnoreBlankLine(true))

lines, err = fsx.GetFileContentStringSliceWithBuffer(buf)

lines, err = fsx.GetFileContentStringSliceWithEmbedFS(staticFS, "static/readme.md")
Regular expression helpers
// Extract matching lines
matches, err := fsx.GetContentWithRegularExpression(content, `\d+`)

// Remove matching lines
cleaned := fsx.CleanSpecificContentLines(data, `^#`)
ZIP archive
// Compress a directory or a single file. The archive is placed next to the source.
zipPath, err := fsx.CreateZIPFileWithFilename("./test")
// -> ./test.zip

zipPath, err = fsx.CreateZIPFileWithFilename("./main.go")
// -> ./main.zip
HTTP download
func handler(w http.ResponseWriter, r *http.Request) {
    if err := fsx.DownloadFile(w, "files/report.pdf"); err != nil {
        log.Println(err)
    }
}

// Or stream from an io.Reader (pass -1 to auto-detect size for io.ReadSeeker)
func streamHandler(w http.ResponseWriter, r *http.Request) {
    file, _ := os.Open("files/report.pdf")
    defer file.Close()

    _ = fsx.DownloadFileByIOReader(w, file, "报告.pdf", -1)
}

Filenames are encoded per RFC 5987 (filename*=UTF-8''...), so non-ASCII names are transmitted safely.

HTTP upload MIME check
allowed, err := fsx.UploadIsAllowMIMEWithFormFile(r, "avatar",
    []string{"image/jpeg", "image/png"})
Response header helpers
// Extract the filename from a Content-Disposition header
name := fsx.GetResponseHeaderFunc(w, "Content-Disposition",
    fsx.ContentDispositionFilename())
Go version parsing
v, err := fsx.GetVersion(runtime.Version()) // e.g. "go1.21.0"
fmt.Println(v.GetFullVersion())  // 1.21.0
fmt.Println(v.GetMinorVersion()) // 1.21

API Overview

File Description
abs.go AbsPath – resolve and optionally create absolute paths
blank_line.go CleanBlankLines, CleanBlankLinesForFile
content.go MarkContentPrefixClear, MarkContentSuffixClear
create.go CreateFile
directory.go Exists, EmbedsExists, MkdirAll, ReadDir, WriteDir, Copy, IsDir, DirNotExists, LastDirName, …
download.go DownloadFile, DownloadFileByIOReader
extension.go FileExtension type and predefined constants
file.go CreateFilename, GetFileExtension, IsFile, Touch, Truncate, GetFileContent*, GetContentWithRegularExpression, …
line.go Lines, LinesForFile, Characters, CharactersForFile
option.go Functional options (WithSpecific*, WithIgnoreBlankLine, …)
path.go FileSystemPath type and predefined paths
remove_line.go CleanSpecificContentLines, CleanSpecificContentLinesForFile
response.go GetResponseHeader, ContentDispositionFilename
slice.go InSlice
upload.go UploadIsAllowMIMEWithFormFile
version.go VersionGithub, GetVersion
zip.go CreateZIPFileWithFilename

Testing

go test ./...

Test fixtures live under test/data/. The suite uses github.com/stretchr/testify for assertions and suite-based organization. Generated artifacts are cleaned up in TearDownTest.

License

Licensed under the Apache License, Version 2.0. See LICENSE for details.


Copyright 2021 fishfinal <contact.fishfinal@gmail.com>

Documentation

Index

Constants

View Source
const (
	FileSystemOfGOPATH = "GOPATH"
)
View Source
const (
	// GlobalSeparatorWithFileName File name separator.
	GlobalSeparatorWithFileName = "."
)
View Source
const (
	ZIPExtension = "zip"
)

Variables

This section is empty.

Functions

func AbsPath

func AbsPath(path string, create bool) (string, error)

AbsPath Checks whether a directory is an absolute path and can be created by creating a path that does not exist

func Characters

func Characters(data []byte) int

Characters Count the number of data characters

func CharactersForFile

func CharactersForFile(filename string) (int, error)

CharactersForFile Count the number of data characters

func CleanBlankLines

func CleanBlankLines(data []byte) []byte

CleanBlankLines clean blank lines with file.

func CleanBlankLinesForFile

func CleanBlankLinesForFile(filename string) ([]byte, error)

CleanBlankLinesForFile clean blank lines with file.

func CleanSpecificContentLines

func CleanSpecificContentLines(data []byte, findStr string) []byte

CleanSpecificContentLines clean specific content lines.

func CleanSpecificContentLinesForFile

func CleanSpecificContentLinesForFile(filename string, findStr string) ([]byte, error)

CleanSpecificContentLinesForFile clean specific content lines for file.

func ContentDispositionFilename

func ContentDispositionFilename() func(val string) string

ContentDispositionFilename Gets filename for response header Content-Disposition. Header such as Content-Disposition: attachment; filename="helloshaohua.txt"

func Copy

func Copy(src string, dst string, ops ...Option) error

Copy file to destination directory.

func CreateFile

func CreateFile(filename string, data []byte, ops ...Option) (string, error)

CreateFile Create new file, your can specific multiple options.

func CreateFilename

func CreateFilename(filename, extension string, ops ...Option) string

CreateFilename Create the filename.

func CreateZIPFileWithFilename

func CreateZIPFileWithFilename(source string) (string, error)

CreateZIPFileWithFilename CreateZIPFile Create ZIP file, return the storage path and error. source parameter specifies a file or directory.

func DirNotExists

func DirNotExists(path string) bool

DirNotExists Check whether the specified file path directory does not exists.

func DownloadFile

func DownloadFile(resp http.ResponseWriter, path string) error

DownloadFile will write download file into the HTTP response body.

func DownloadFileByIOReader

func DownloadFileByIOReader(resp http.ResponseWriter, reader io.Reader, filename string, filesize int64) error

DownloadFileByIOReader streams file content from an io.Reader to the HTTP response as a downloadable file. filesize: File size in bytes. Pass -1 to let the function attempt to determine the size automatically (only works when reader implements io.ReadSeeker).

func EmbedsExists

func EmbedsExists(embeds embed.FS, path string) bool

EmbedsExists checks whether the specified file or directory exists in the embedded file system. This is a more general function that can handle both files and directories.

Parameters:

  • embeds: The embedded file system instance to search in
  • path: The relative path of the file or directory to check

Returns:

  • true if the file or directory exists
  • false if it does not exist

Example:

//go:embed static/*
var staticFS embed.FS

if EmbedsExists(staticFS, "static/css/style.css") {
    // File exists
}

if EmbedsExists(staticFS, "static/images/") {
    // Directory exists (note trailing slash for directories)
}

func EmbedsFileExists

func EmbedsFileExists(embeds embed.FS, path string) bool

EmbedsFileExists checks whether the specified file exists in the embedded file system. It attempts to read the file from the embed.FS to determine its existence.

Parameters:

  • embeds: The embedded file system instance to search in
  • path: The relative path of the file to check (relative to the embed.FS root)

Returns:

  • true if the file exists and can be read (or opened)
  • false if the file does not exist or cannot be accessed

Note: This function only checks for the existence of files, not directories. Use EmbedsExists() if you need to check for both files and directories.

func EmbedsNotExists

func EmbedsNotExists(embeds embed.FS, path string) bool

EmbedsNotExists checks whether the specified file or directory does NOT exist in the embedded file system. This is the inverse of EmbedsExists() for semantic clarity in certain contexts.

Parameters:

  • embeds: The embedded file system instance to search in
  • path: The relative path to check

Returns:

  • true if the file or directory does NOT exist
  • false if it exists

Usage note: Consider using !EmbedsExists() for better readability in most cases. This function is provided for consistency with Go's os.IsNotExist pattern.

func Exists

func Exists(path string) bool

Exists checks whether the target file or directory exists. Return TRUE if the file exists, FALSE otherwise.

func FilenameTrimPrefix

func FilenameTrimPrefix(filename, prefix string) string

FilenameTrimPrefix Trim the filename prefix.

func GetContentWithRegularExpression

func GetContentWithRegularExpression(str, pattern string, ops ...Option) ([]string, error)

GetContentWithRegularExpression Get the string content into slice's string, using a regular expression.

func GetFileContentBytesWithEmbedFS

func GetFileContentBytesWithEmbedFS(fs embed.FS, filename string) ([]byte, error)

GetFileContentBytesWithEmbedFS Get file content bytes slice with embed filesystem.

func GetFileContentStringSliceWithBuffer

func GetFileContentStringSliceWithBuffer(buffer *bytes.Buffer, ops ...Option) ([]string, error)

GetFileContentStringSliceWithBuffer Get file content string slice with bytes buffer

func GetFileContentStringSliceWithEmbedFS

func GetFileContentStringSliceWithEmbedFS(fs embed.FS, filename string, ops ...Option) ([]string, error)

GetFileContentStringSliceWithEmbedFS Get file content string slice with embed filesystem.

func GetFileContentStringSliceWithFilename

func GetFileContentStringSliceWithFilename(filename string, ops ...Option) ([]string, error)

GetFileContentStringSliceWithFilename Get the file contents as a string for slice.

func GetFileExtension

func GetFileExtension(filename string, ops ...Option) string

GetFileExtension Gets the file extension.

func GetFileOrDirectoryName

func GetFileOrDirectoryName(path string) (string, bool, error)

GetFileOrDirectoryName Gets the file or directory name.

func GetFilePathWithFileSystemPath

func GetFilePathWithFileSystemPath(baseDir string, ops ...Option) (string, error)

GetFilePathWithFileSystemPath Get a file path with filesystem path.

func GetGOPATH

func GetGOPATH() string

GetGOPATH Gets the current operating system GOPATH from environment.

func GetGOPATHProjectSource

func GetGOPATHProjectSource() string

GetGOPATHProjectSource Gets the current operating system GOPATH/src from environment and filesystem.

func GetResponseHeader

func GetResponseHeader(resp http.ResponseWriter, field string) string

GetResponseHeader Gets response header field value.

func GetResponseHeaderFunc

func GetResponseHeaderFunc(resp http.ResponseWriter, field string, process func(val string) string) string

GetResponseHeaderFunc GetResponseHeader Gets response header field value, processing of response field values by attaching custom functions.

func InSlice

func InSlice(source []interface{}, find interface{}) bool

func IsDir

func IsDir(path string) (bool, error)

IsDir Checks whether the specified file name is a directory.

func IsFile

func IsFile(filename string) (bool, error)

IsFile Checks whether the specified file name is a file.

func IsNotExists deprecated

func IsNotExists(path string) bool

IsNotExists check whether the target file or directory exists. Return FALSE if the file exists, TRUE otherwise.

Deprecated: use Exists instead.

func IsNotExistsWithEmbedFS deprecated

func IsNotExistsWithEmbedFS(embeds embed.FS, path string) bool

IsNotExistsWithEmbedFS check whether the target file or directory exists with embed.FS. Return FALSE if the file exists, TRUE otherwise.

Deprecated: use EmbedsFileExists instead.

func LastDirName

func LastDirName(filename string) string

LastDirName Gets filename the last directory name.

func Lines

func Lines(data []byte) int

Lines Count the number of data rows

func LinesForFile

func LinesForFile(filename string) (int, error)

LinesForFile Count the number of data rows from file.

func MarkContentPrefixClear

func MarkContentPrefixClear(data []byte, mark string, ops ...Option) []byte

MarkContentPrefixClear Mark content prefix clear.

func MarkContentSuffixClear

func MarkContentSuffixClear(data []byte, mark string, ops ...Option) []byte

MarkContentSuffixClear Mark content suffix clear.

func MkdirAll

func MkdirAll(path string) error

MkdirAll creates a directory named path.

func MustGetFilePathWithFileSystemPath

func MustGetFilePathWithFileSystemPath(baseDir string, ops ...Option) string

func NotExistsMkdir

func NotExistsMkdir(path string) error

NotExistsMkdir detects whether the target folder exists. and if it does not, create the folder.

func NotExistsMkdirAll

func NotExistsMkdirAll(path string) error

NotExistsMkdirAll Check whether the directory exists and create it if it does not

func ReadDir

func ReadDir(path string, result []string) ([]string, error)

ReadDir Get the directory file name by recursive reading.

func RuntimeMasterDir

func RuntimeMasterDir() string

RuntimeMasterDir returns the root path corresponding to the current directory.

func Touch

func Touch(filename string) error

Touch Create file, if parent directory not exists, then create.

func Truncate

func Truncate(filename string, size int64) error

Truncate changes the size of the file. It does not change the I/O offset. If there is an error, it will be of type *PathError.

func UploadFileForMultipartFields

func UploadFileForMultipartFields(values map[string]interface{}) (contentType string, buffer bytes.Buffer, err error)

UploadFileForMultipartFields Mock form upload file.

func UploadIsAllowMIMEWithFormFile

func UploadIsAllowMIMEWithFormFile(request *http.Request, key string, allows []string) (bool, error)

func WriteDir

func WriteDir(dst string, files []string, ops ...Option) error

WriteDir Write file to dst directory.

Types

type FileExtension

type FileExtension string
const (
	FileExtensionWithImageOfJPEG FileExtension = "jpeg"
	FileExtensionWithImageOfPNG  FileExtension = "png"
	FileExtensionWithImageOfGIF  FileExtension = "gif"
	FileExtensionWithImageOfBMP  FileExtension = "bmp"
)
const (
	FileExtensionWithOfficeOfExcel FileExtension = "xlsx"
	FileExtensionWithOfficeOfWord  FileExtension = "docx"
	FileExtensionWithOfficeOfCSV   FileExtension = "csv"
)
const (
	FileExtensionWithDocumentOfMarkdown FileExtension = "md"
	FileExtensionWithDocumentOfFileText FileExtension = "txt"
)

func (FileExtension) String

func (file FileExtension) String() string

type FileSystemPath

type FileSystemPath string
const (
	AssetsSavePathForDatabase FileSystemPath = "static/assets/database"
	AssetsSavePathForExcel    FileSystemPath = "static/assets/excel"
	AssetsSavePathForCSV      FileSystemPath = "static/assets/csv"
)

Assets files save path definition.

const GlobalDefaultDir FileSystemPath = "static"

func (FileSystemPath) String

func (p FileSystemPath) String() string

type Option

type Option interface {
	// contains filtered or unexported methods
}

func WithIgnoreBlankLine

func WithIgnoreBlankLine(ignored bool) Option

WithIgnoreBlankLine Specify whether blank lines are ignored.

func WithOriginalFileNameTrimPrefix

func WithOriginalFileNameTrimPrefix(str string) Option

WithOriginalFileNameTrimPrefix WithSpecificContainsMarkString Override the original value by trimPrefixStr string.

func WithSpecificBaseDir

func WithSpecificBaseDir(baseDir string) Option

WithSpecificBaseDir Override the original value by specifying base directory.

func WithSpecificContainsMarkString

func WithSpecificContainsMarkString(contains bool) Option

WithSpecificContainsMarkString Override the original value by containsMarkString mark string.

func WithSpecificFileExtension

func WithSpecificFileExtension(extension FileExtension) Option

WithSpecificFileExtension Override the original value by specifying file extension.

func WithSpecificFileExtensionContainsDot

func WithSpecificFileExtensionContainsDot(containsdot bool) Option

WithSpecificFileExtensionContainsDot Override the original value by specifying dot.

func WithSpecificFileSystemPath

func WithSpecificFileSystemPath(path FileSystemPath) Option

WithSpecificFileSystemPath Override the original value by specifying filepath.

func WithSpecificTrimOriginalFileExtension

func WithSpecificTrimOriginalFileExtension(trim bool) Option

WithSpecificTrimOriginalFileExtension Override the original value by specifying trim.

type VersionGithub

type VersionGithub struct {
	Major string `json:"major"`
	Minor string `json:"minor"`
	Patch string `json:"patch"`
}

VersionGithub Github version number format.

func GetVersion

func GetVersion(version string) (*VersionGithub, error)

GetVersion Get Github version information in version number format.

func (*VersionGithub) GetFullVersion

func (v *VersionGithub) GetFullVersion() string

GetFullVersion Get a full version string. Such as 1.16.6, will return 1.16.6

func (*VersionGithub) GetMinorVersion

func (v *VersionGithub) GetMinorVersion() string

GetMinorVersion Get a minor version string. Such as 1.16.6, will return 1.16

func (*VersionGithub) Join

func (v *VersionGithub) Join(element ...string) string

Join version section, default use dot.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL