Documentation
¶
Overview ¶
Package memfs provide a library for mapping file system into memory and/or to embed it inside go source file.
Usage ¶
The first step is to create new instance of memfs using `New()`. The following example embed all files inside directory named "include" or any files with extension ".css", ".html", and ".js"; but exclude any files inside directory named "exclude".
opts := &Options{
Root: "./mydir",
Includes: []string{
`.*/include`,
`.*\.(css|html|js)$`,
},
Excludes: []string{
`.*/exclude`,
},
}
mfs, err := memfs.New(opts)
By default only file with size less or equal to 5 MB will be included in memory. To increase the default size set the MaxFileSize (in bytes). For example, to set maximum file size to 10 MB,
opts.MaxFileSize = 1024 * 1024 * 10
Later, if we want to get the file from memory, call Get() which will return the node object with content can be accessed from field "Content". Remember that if file size is larger that maximum, the content will need to be read manually,
node, err := mfs.Get("/")
if err != nil {
// Handle file not exist
}
if node.mode.IsDir() {
// Handle directory.
}
if node.Content == nil {
// Handle large file.
node.V, err = ioutil.ReadFile(child.SysPath)
}
// Do something with content of file system.
Go embed ¶
The memfs package also support embedding the files into Go generated source file. After we create memfs instance, we call GoEmbed() to dump all directory and files as Go source code.
First, define global variable as container for all files later in the same package as generated code,
package mypackage var myFS *memfs.MemFS
Second, create new instance of MemFS with Options.Embed is set, and write the content of memfs instance as Go source code file,
opts := &Options{
Root: "./mydir",
Includes: []string{
`.*/include`,
`.*\.(css|html|js)$`,
},
Excludes: []string{
`.*/exclude`,
},
Embed: EmbedOptions{
PackageName: "mypackage",
VarName: "myFS",
GoFileName: "mypackage/embed.go",
ContentEncoding: memfs.EncodingGzip,
},
}
mfs, _ := memfs.New(opts)
mfs.GoEmbed()
The Go generated file will be defined with package named "mypackage" using global variable "myFS" as container stored in file "mypackage/file.go" with each content encoded (compressed) using gzip.
Thats it!
Index ¶
- Constants
- type EmbedOptions
- type MemFS
- func (mfs *MemFS) AddChild(parent *Node, fi os.FileInfo) (child *Node, err error)
- func (mfs *MemFS) AddFile(internalPath, externalPath string) (node *Node, err error)
- func (mfs *MemFS) ContentEncode(encoding string) (err error)
- func (mfs *MemFS) Get(path string) (node *Node, err error)
- func (mfs *MemFS) GoEmbed() (err error)
- func (mfs *MemFS) ListNames() (paths []string)
- func (mfs *MemFS) MarshalJSON() ([]byte, error)
- func (mfs *MemFS) MustGet(path string) (node *Node)
- func (mfs *MemFS) Open(path string) (http.File, error)
- func (mfs *MemFS) RemoveChild(parent *Node, child *Node) (removed *Node)
- func (mfs *MemFS) Search(words []string, snippetLen int) (results []SearchResult)
- func (mfs *MemFS) Update(node *Node, newInfo os.FileInfo)
- type Node
- func (node *Node) AddChild(child *Node)
- func (node *Node) Close() error
- func (node *Node) Decode() ([]byte, error)
- func (node *Node) Encode(content []byte) (err error)
- func (node *Node) IsDir() bool
- func (node *Node) MarshalJSON() ([]byte, error)
- func (node *Node) ModTime() time.Time
- func (node *Node) Mode() os.FileMode
- func (node *Node) Name() string
- func (node *Node) Read(p []byte) (n int, err error)
- func (node *Node) Readdir(count int) (fis []os.FileInfo, err error)
- func (node *Node) Save(content []byte) (err error)
- func (node *Node) Seek(offset int64, whence int) (int64, error)
- func (node *Node) SetModTime(modTime time.Time)
- func (node *Node) SetModTimeUnix(seconds, nanoSeconds int64)
- func (node *Node) SetMode(mode os.FileMode)
- func (node *Node) SetName(name string)
- func (node *Node) SetSize(size int64)
- func (node *Node) Size() int64
- func (node *Node) Stat() (os.FileInfo, error)
- func (node *Node) Sys() interface{}
- func (node *Node) Update(newInfo os.FileInfo, maxFileSize int64) (err error)
- type Options
- type PathNode
- type SearchResult
Examples ¶
Constants ¶
const ( DefaultEmbedPackageName = "main" // Default package name for GoEmbed(). DefaultEmbedVarName = "memFS" // Default variable name for GoEmbed(). DefaultEmbedGoFileName = "memfs_generate.go" // Default file output for GoEmbed(). )
const (
EncodingGzip = "gzip"
)
List of valid content encoding for ContentEncode().
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type EmbedOptions ¶ added in v0.31.0
type EmbedOptions struct {
// The generated package name for GoEmbed().
// If its not defined it will be default to "main".
PackageName string
// VarName is the global variable name with type *memfs.MemFS which
// will be initialized by generated Go source code on init().
// If its empty it will default to "memFS".
VarName string
// GoFileName the path to Go generated file, where the file
// system will be embedded.
// If its not defined it will be default to "memfs_generate.go"
// in current directory from where its called.
GoFileName string
// ContentEncoding if this value is not empty, it will encode the
// content of node and set the node ContentEncoding.
//
// List of available encoding is "gzip".
//
// For example, if the value is "gzip" it will compress the content of
// file using gzip and set Node.ContentEncoding to "gzip".
ContentEncoding string
// WithoutModTime if its true, the modification time for all
// files and directories are not stored inside generated code, instead
// all files will use the current time when the program is running.
WithoutModTime bool
}
EmbedOptions define an options for GoEmbed.
type MemFS ¶
type MemFS struct {
http.FileSystem
PathNodes *PathNode
Root *Node
Opts *Options
// contains filtered or unexported fields
}
MemFS contains directory tree of file system in memory.
func Merge ¶ added in v0.27.0
Merge one or more instances of MemFS into single hierarchy.
If there are two instance of Node that have the same path, the last instance will be ignored.
func New ¶
New create and initialize new memory file system from directory Root using list of regular expresssion for Including or Excluding files.
Example ¶
/**
Let say we have the "testdata" directory,
testdata/
├── direct
│ └── add
│ ├── file
│ └── file2
├── exclude
│ ├── dir
│ ├── index-link.css -> ../index.css
│ ├── index-link.html -> ../index.html
│ └── index-link.js -> ../index.js
├── include
│ ├── dir
│ ├── index.css -> ../index.css
│ ├── index.html -> ../index.html
│ └── index.js -> ../index.js
├── index.css
├── index.html
├── index.js
└── plain
Assume that we want to embed all files with extension .css, .html,
and .js only; but not from directory "exclude".
We can create the Options like below,
*/ Options like below,
*/
opts := &Options{
Root: "./testdata",
Includes: []string{`.*/include`, `.*\.(css|html|js)$`},
Excludes: []string{`.*/exclude`},
}
mfs, err := New(opts)
if err != nil {
log.Fatal(err)
}
node, err := mfs.Get("/index.html")
if err != nil {
log.Fatal(err)
}
fmt.Printf("Content of /index.html: %s", node.Content)
fmt.Printf("List of embedded files: %+v\n", mfs.ListNames())
_, err = mfs.Get("/exclude/index.html")
if err != nil {
fmt.Printf("Error on Get /exclude/index.html: %s\n", err)
}
Output: Content of /index.html: <html></html> List of embedded files: [/ /include /include/index.css /include/index.html /include/index.js /index.css /index.html /index.js] Error on Get /exclude/index.html: file does not exist
func (*MemFS) AddFile ¶ added in v0.10.1
AddFile add the external file directly as internal file. If the internal file is already exist it will be replaced. Any directories in the internal path will be generated automatically if its not exist.
func (*MemFS) ContentEncode ¶ added in v0.10.1
ContentEncode encode each node's content into specific encoding, in other words this method can be used to compress the content of file in memory or before being served or written.
Only file with size greater than 0 will be encoded.
List of known encoding is "gzip".
func (*MemFS) Get ¶
Get the node representation of file in memory. If path is not exist it will return os.ErrNotExist.
func (*MemFS) MarshalJSON ¶ added in v0.29.1
MarshalJSON encode the MemFS object into JSON format.
The field that being encoded is the Root node.
func (*MemFS) MustGet ¶ added in v0.27.0
MustGet return the Node representation of file in memory by its path if its exist or nil the path is not exist.
func (*MemFS) Open ¶ added in v0.11.0
Open the named file for reading. This is an alias to Get() method, to make it implement http.FileSystem.
func (*MemFS) RemoveChild ¶ added in v0.6.0
RemoveChild remove a child on parent, including its map on PathNode. If child is not part if node's childrens it will return nil.
func (*MemFS) Search ¶ added in v0.10.1
func (mfs *MemFS) Search(words []string, snippetLen int) (results []SearchResult)
Search one or more strings in each content of files.
Example ¶
opts := &Options{
Root: "./testdata",
}
mfs, err := New(opts)
if err != nil {
log.Fatal(err)
}
q := []string{"body"}
results := mfs.Search(q, 0)
for _, result := range results {
fmt.Printf("Path: %s\n", result.Path)
fmt.Printf("Snippets: %q\n", result.Snippets)
}
Output: Path: /include/index.css Snippets: ["body {\n}\n"] Path: /exclude/index-link.css Snippets: ["body {\n}\n"] Path: /index.css Snippets: ["body {\n}\n"]
type Node ¶
type Node struct {
os.FileInfo
http.File
SysPath string // The original file path in system.
Path string // Absolute file path in memory.
Content []byte // Content of file.
ContentType string // File type per MIME, for example "application/json".
ContentEncoding string // File type encoding, for example "gzip".
Parent *Node // Pointer to parent directory.
Childs []*Node // List of files in directory.
GenFuncName string // The function name for embedded Go code.
// contains filtered or unexported fields
}
Node represent a single file.
func NewNode ¶ added in v0.6.0
NewNode create a new node based on file information "fi".
The parent parameter is required to allow valid system path generated for new node.
If maxFileSize is greater than zero, the file content and its type will be saved in node as Content and ContentType.
func (*Node) Decode ¶ added in v0.10.1
Decode the contents of node (for example, uncompress with gzip) and return it.
func (*Node) MarshalJSON ¶ added in v0.29.1
MarshalJSON encode the node into JSON format. If the node is a file it will return the content of file; otherwise it will return the node with list of childs, but not including childs of childs.
func (*Node) Readdir ¶ added in v0.11.0
Readdir reads the contents of the directory associated with file and returns a slice of up to n FileInfo values, as would be returned by Lstat, in directory order. Subsequent calls on the same file will yield further FileInfos.
func (*Node) Save ¶ added in v0.29.2
Save the content to file system and update the content of Node.
func (*Node) Seek ¶ added in v0.11.0
Seek sets the offset for the next Read offset, interpreted according to whence: SeekStart means relative to the start of the file, SeekCurrent means relative to the current offset, and SeekEnd means relative to the end. Seek returns the new offset relative to the start of the file and an error, if any.
func (*Node) SetModTime ¶ added in v0.11.0
SetModTime set the file modification time.
func (*Node) SetModTimeUnix ¶ added in v0.31.0
SetModTimeUnix set the file modification time using seconds and nanoseconds since January 1, 1970 UTC.
func (*Node) Sys ¶ added in v0.11.0
func (node *Node) Sys() interface{}
Sys return the underlying data source (can return nil).
func (*Node) Update ¶ added in v0.30.0
Update the node metadata or content based on new file information.
The newInfo parameter is optional, if its nil, it will read the file information based on node's SysPath.
The maxFileSize parameter is also optional. If its negative, the node content will not be updated. If its zero, it will default to 5 MB.
There are two possible changes that will happen: its either change on mode or change on content (size and modtime). Change on mode will not affect the content of node.
type Options ¶ added in v0.23.0
type Options struct {
// Root contains path to directory where its contents will be mapped
// to memory.
Root string
// The includes and excludes pattern applied to path of file in file
// system.
Includes []string
Excludes []string
// MaxFileSize define maximum file size that can be stored on memory.
// The default value is 5 MB.
// If its value is negative, the content of file will not be mapped to
// memory, the MemFS will behave as directory tree.
MaxFileSize int64
// EmbedOptions for GoEmbed.
Embed EmbedOptions
// Development define a flag to bypass file in memory.
// If its true, any call to Get will result in direct read to file
// system.
Development bool
}
type PathNode ¶ added in v0.6.0
type PathNode struct {
// contains filtered or unexported fields
}
PathNode contains a mapping between path and Node.
func NewPathNode ¶ added in v0.6.0
func NewPathNode() *PathNode
NewPathNode create and initialize new PathNode.
func (*PathNode) MarshalJSON ¶ added in v0.29.1
type SearchResult ¶ added in v0.10.1
SearchResult contains the result of searching where the Path will be filled with absolute path of file system in memory and the Snippet will filled with part of the text before and after the search string.
Source Files