README
¶
stank: analyzers for determining whether files smell like rotten POSIX shell scripts, or faintly rosy like Ruby and Python scripts
ABOUT
stank is a library and collection of command line utilities for sniffing files to identify shell scripts like bash, sh, zsh, ksh and so on, those funky farmfresh gobs of garbaggio; versus other more palatable files like rb, py, pl. Believe it or not, shell scripts are notoriously difficult to write well, so it behooves a developer to either write shell scripts in safer languages, or else wargame your scripts with an armada of linters. Trouble is, in large projects one can never be too sure which files are honest to dog POSIX compliant shell scripts, and which are pretenders. csh, tcsh, fish, ion, rc, and most other nonderivatives of bash tend to be NOT POSIX compatible. If you're geeky enough to have followed thus far, let's get crackalackin with some fruity examples dammit!
EXAMPLES
The stank system includes the stank Go library as well as three command line utilities for convenience. rosy recursively searches directory trees for POSIX shell scripts. By default, rosy recommends that results be rewritten in safer general purpose languages like Ruby, Python, Node.js, etc. Rosy also offers a -kame flag for recommending faster shells like sh, ksh, dash, and posh; or an -usagi flag for recommending more robust shells that support set -euo pipefail like ksh, bash, and zsh.
$ rosy examples
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/blank.bash
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/derp.zsh
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/globs.bash
Clarify interpreter with a shebang line: examples/goodbye.sh
Clarify interpreter with a shebang line: examples/greetings.bash
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/hello
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/hello-crlf.sh
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/hello-ksh88.ksh
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/hello-mksh.ksh
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/hello.bosh
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/hello.ksh
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/hello.ksh88
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/hello.oksh
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/hello.pdksh
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/hello.sh
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/hello.yash
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/hooks/post-update
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/hooks/pre-applypatch
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/hooks/pre-commit
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/hooks/pre-push
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/hooks/pre-rebase
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/hooks/update
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/howdy
Clarify interpreter with a shebang line: examples/howdy.zsh
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/i-should-have-an-extension
Clarify interpreter with a shebang line: examples/just-eol.bash
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/just-shebang.bash
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/pipefail
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/salutations.bash
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/salutations.sh
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/salutations4.bash
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/sample.envrc
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/wednesday
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/wednesday-bom
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/welcome
Rewrite POSIX script in Ruby or other safer general purpose scripting language: examples/welcome.sh
$ echo "$?"
1
$ rosy examples/just-python
$ echo "$?"
0
$ rosy -kame examples
Rewrite script in sh, ksh, posh, dash, etc. for performance boost: examples/blank.bash
Rewrite script in sh, ksh, posh, dash, etc. for performance boost: examples/derp.zsh
Rewrite script in sh, ksh, posh, dash, etc. for performance boost: examples/globs.bash
Clarify interpreter with a shebang line: examples/goodbye.sh
Clarify interpreter with a shebang line: examples/greetings.bash
Rewrite script in sh, ksh, posh, dash, etc. for performance boost: examples/hello.bosh
Rewrite script in sh, ksh, posh, dash, etc. for performance boost: examples/hello.yash
Rewrite script in sh, ksh, posh, dash, etc. for performance boost: examples/howdy
Clarify interpreter with a shebang line: examples/howdy.zsh
Clarify interpreter with a shebang line: examples/just-eol.bash
Rewrite script in sh, ksh, posh, dash, etc. for performance boost: examples/just-shebang.bash
Rewrite script in sh, ksh, posh, dash, etc. for performance boost: examples/pipefail
Rewrite script in sh, ksh, posh, dash, etc. for performance boost: examples/salutations.bash
Rewrite script in sh, ksh, posh, dash, etc. for performance boost: examples/salutations.sh
Rewrite script in sh, ksh, posh, dash, etc. for performance boost: examples/salutations4.bash
Rewrite script in sh, ksh, posh, dash, etc. for performance boost: examples/wednesday
Rewrite script in sh, ksh, posh, dash, etc. for performance boost: examples/wednesday-bom
Rewrite script in sh, ksh, posh, dash, etc. for performance boost: examples/welcome
Rewrite script in sh, ksh, posh, dash, etc. for performance boost: examples/welcome.sh
$ rosy -usagi examples
Rewrite script in ksh, bash, zsh, etc., and enable debugging flags for robustness: examples/blank.bash
Clarify interpreter with a shebang line: examples/goodbye.sh
Clarify interpreter with a shebang line: examples/greetings.bash
Rewrite script in ksh, bash, zsh, etc., and enable debugging flags for robustness: examples/hello
Rewrite script in ksh, bash, zsh, etc., and enable debugging flags for robustness: examples/hello-crlf.sh
Rewrite script in ksh, bash, zsh, etc., and enable debugging flags for robustness: examples/hello.bosh
Rewrite script in ksh, bash, zsh, etc., and enable debugging flags for robustness: examples/hello.oksh
Rewrite script in ksh, bash, zsh, etc., and enable debugging flags for robustness: examples/hello.sh
Rewrite script in ksh, bash, zsh, etc., and enable debugging flags for robustness: examples/hello.yash
Rewrite script in ksh, bash, zsh, etc., and enable debugging flags for robustness: examples/hooks/post-update
Rewrite script in ksh, bash, zsh, etc., and enable debugging flags for robustness: examples/hooks/pre-applypatch
Rewrite script in ksh, bash, zsh, etc., and enable debugging flags for robustness: examples/hooks/pre-commit
Rewrite script in ksh, bash, zsh, etc., and enable debugging flags for robustness: examples/hooks/pre-push
Rewrite script in ksh, bash, zsh, etc., and enable debugging flags for robustness: examples/hooks/pre-rebase
Rewrite script in ksh, bash, zsh, etc., and enable debugging flags for robustness: examples/hooks/update
Clarify interpreter with a shebang line: examples/howdy.zsh
Rewrite script in ksh, bash, zsh, etc., and enable debugging flags for robustness: examples/i-should-have-an-extension
Clarify interpreter with a shebang line: examples/just-eol.bash
Rewrite script in ksh, bash, zsh, etc., and enable debugging flags for robustness: examples/salutations.sh
Rewrite script in ksh, bash, zsh, etc., and enable debugging flags for robustness: examples/salutations4.bash
Rewrite script in ksh, bash, zsh, etc., and enable debugging flags for robustness: examples/sample.envrc
$ rosy -help
-help
Show usage information
-kame
Recommend faster shells
-usagi
Recommend more robust shells
-version
Show version information
Note that rosy ignores scripts housed in a bin directory, as these tend to connote shims for running JAR's as command line applications. These shims are ideally written in pure POSIX sh for maximum portability.
The stank application prints paths to POSIX shell scripts, designed for use in combination with xargs to help per-file shell static analysis applications lint large projects.
$ stank examples
examples/.profile
examples/.zshrc
examples/badconfigs/.bash_profile
examples/badconfigs/zprofile
examples/blank.bash
examples/derp.zsh
examples/globs.bash
examples/goodbye.sh
examples/greetings.bash
examples/hello
examples/hello.sh
examples/hooks/post-update
examples/hooks/pre-applypatch
examples/hooks/pre-commit
examples/hooks/pre-push
examples/hooks/pre-rebase
examples/hooks/update
examples/howdy
examples/howdy.zsh
examples/i-should-have-an-extension
examples/just-eol.bash
examples/just-shebang.bash
examples/pipefail
examples/salutations.bash
examples/salutations.sh
examples/salutations4.bash
examples/sample.envrc
examples/wednesday
examples/welcome
examples/welcome.sh
$ stank examples/hooks | xargs checkbashisms
error: examples/hooks/pre-rebase: Unterminated quoted string found, EOF reached. Wanted: <'>, opened in line 133
$ stank -sh examples
examples/.profile
examples/goodbye.sh
examples/greetings.bash
examples/hello
examples/hello.sh
examples/hooks/post-update
examples/hooks/pre-applypatch
examples/hooks/pre-commit
examples/hooks/pre-push
examples/hooks/pre-rebase
examples/hooks/update
examples/howdy.zsh
examples/i-should-have-an-extension
examples/sample.envrc
$ stank -help
-help
Show usage information
-sh
Limit results to specifically bare POSIX sh scripts
-version
Show version information
The funk linter reports strange odors emanating from scripts, such as missing shebangs.
Note that funk cannot reliably warn for missing shebangs if the extension is also missing; typically, script authors use one or the other to mark files as shell scripts. In any case, know that the shebang is requisite for ensuring your scripts are properly interpreted.
Note that funk may fail to present permissions warnings if the scripts are housed on non*nix file systems such as NTFS, where executable bits are often missing from the file metadata altogether. When storing shell scripts, be sure to set the appropriate file permissions, and transfer files as a bundle in a tarball or similar to safeguard against dropped permissions.
Note that funk may warn of interpreter mismatches for scripts with extraneous dots in the filename. Rather than .envrc.sample, name the file sample.envrc. Rather than wget-google.com, name the file wget-google-com. Appending .sh is also an option, so update.es.cluster renames to update.es.cluster.sh.
$ funk examples
Configuration features shebang: examples/badconfigs/.bash_profile
Configuration features executable permissions: examples/badconfigs/zprofile
Missing final end of line sequence: examples/blank.bash
Missing shebang: examples/blank.bash
Interpreter mismatch between shebang and extension: examples/derp.zsh
Missing shebang: examples/greetings.bash
Missing final end of line sequence: examples/hello-crlf.sh
NonPOSIX CR/CRLF line ending detected: examples/hello-crlf.sh
Missing shebang: examples/howdy.zsh
Missing shebang: examples/just-eol.bash
Leading BOM reduces portability: examples/wednesday-bom
$ funk -modulino examples
Configuration features shebang: examples/badconfigs/.bash_profile
Configuration features executable permissions: examples/badconfigs/zprofile
Missing final end of line sequence: examples/blank.bash
Missing shebang: examples/blank.bash
Interpreter mismatch between shebang and extension: examples/derp.zsh
Missing shebang: examples/greetings.bash
Missing final end of line sequence: examples/hello-crlf.sh
NonPOSIX CR/CRLF line ending detected: examples/hello-crlf.sh
Modulino ambiguity. Either have owner executable permissions with no extension, or else remove executable bits and use an extension like .lib.sh: examples/hello-crlf.sh
Modulino ambiguity. Either have owner executable permissions with no extension, or else remove executable bits and use an extension like .lib.sh: examples/howdy
Missing shebang: examples/howdy.zsh
Missing shebang: examples/just-eol.bash
Modulino ambiguity. Either have owner executable permissions with no extension, or else remove executable bits and use an extension like .lib.sh: examples/pipefail
Modulino ambiguity. Either have owner executable permissions with no extension, or else remove executable bits and use an extension like .lib.sh: examples/shout.sh
Modulino ambiguity. Either have owner executable permissions with no extension, or else remove executable bits and use an extension like .lib.sh: examples/wednesday
Modulino ambiguity. Either have owner executable permissions with no extension, or else remove executable bits and use an extension like .lib.sh: examples/wednesday-bom
Leading BOM reduces portability: examples/wednesday-bom
Modulino ambiguity. Either have owner executable permissions with no extension, or else remove executable bits and use an extension like .lib.sh: examples/welcome
$ funk -help
-cr
Report presence/absence of final end of line sequence (default true)
-eol
Report presence/absence of final end of line sequence (default true)
-help
Show usage information
-modulino
Enforce strict separation of application scripts vs. library scripts
-version
Show version information
The optional -modulino flag to funk enables strict separation of script duties, into distinct application scripts vs. library scripts. Application scripts are generally executed by invoking the path, such as ./hello or ~/bin/hello or simply hello when $PATH is appropriately modified. Application scripts feature owner executable permissions, and perhaps group and other as well depending on system configuration needs. In contrast, library scripts are intended to be imported with dot (.) or source into user shells or other scripts, and should feature a file extension like .lib.sh, .sh, .bash, etc. By using separate naming conventions, we more quickly communicate to downstream users how to interact with a shell script. In particular, by dropping file extensions for shell script applications, we encourage authors to choose more meaningful script names. Instead of the generic build.sh, choose build-docker. Instead of kafka.sh, choose start-kafka, kafka-entrypoint, etc.
Finally, stink prints a record of each file's POSIXyness, including any interesting fields it identified along the way. Note that some fields may be zero valued if the stench of POSIX or rosy waft of nonPOSIX is overwhelming, short-circuiting analysis. This short-circuiting feature dramatically speeds up how stank and rosy search large projects.
Note that permissions are relayed as decimals, due to constraints on JSON integer formatting (we didn't want to use a custom octal string field). Use echo 'obase=8;<some integer> | bc to display these values in octal.
Note that legacy systems, packages, and shell scripts referencing "sh" may refer to a plethora of pre-POSIX shells. Modern systems rename "sh" to "lksh", "tsh", "etsh", etc. to avoid confusion. In general, the stank suite will assume that the majority of scripts being scanned are targeting post-1971 technology, so use your human intuition and context to note any legacy Thompson UNIX v6 "sh", etc. scripts. Most modern linters will neither be able to parse such scripts of any complexity, nor will they recognize them for the legacy scripts that they are, unless the scripts' shebangs are rendered with the modern retro interpreters "lksh", "tsh", "etsh", etc. for deployment on modern *nix systems. One could almost use the fs stats for modification/change to try to identify these legacy outliers, but this is a practically unrealistic assumption except for the most obsessive archaeologist, diligently ensuring their legacy scripts continue to present 1970's metadata even after experimental content modifications. So the stank system will simply punt and assume sh -> POSIX sh, ksh -> ksh88 / ksh93 for the sake of modernity and balance.
Similarly, the old Bourne shell AKA "sh" AKA "bsh" presents language identification difficulties. Old Bourne shell scripts are most likely to present themselves with "sh" shebangs, which is okay as Bourne sh and ksh88/pdksh/ksh served as the bases for the POSIX sh standard. Some modern systems may present a Bourne shell as a "sh" or "bsh" binary. The former presents few problems for stank identification, though "bsh" is tricky, as the majority of its uses today are not associated with the Bourne shell but with the Java BeanShell. So stank may default to treating bsh scripts as non-POSIXy, and any such Bourne shell scripts are advised to feature either bash or sh shebangs, and perhaps .sh or .bash extensions, in order to self-identify as modern, POSIX compliant scripts.
$ stink examples/hello
{"Path":"examples/hello","Filename":"hello","Basename":"hello","Extension":"","Shebang":"#!/bin/sh","Interpreter":"sh","LineEnding":"\n","FinalEOL":false,"ContainsCR":false
,"Permissions":509,"Directory":false,"OwnerExecutable":true,"BOM":false,"POSIXy":true}
$ stink -pp examples/hello
{
"Path": "examples/hello",
"Filename": "hello",
"Basename": "hello",
"Extension": "",
"Shebang": "#!/bin/sh",
"Interpreter": "sh",
"LineEnding": "\n",
"FinalEOL": false,
"ContainsCR": false,
"Permissions": 509,
"Directory": false,
"OwnerExecutable": true,
"BOM": false,
"POSIXy": true
}
$ stink -pp examples/hello.py
{
"Path": "examples/hello.py",
"Filename": "hello.py",
"Basename": "hello.py",
"Extension": ".py",
"Shebang": "#!/usr/bin/env python",
"Interpreter": "python",
"LineEnding": "\n",
"FinalEOL": false,
"ContainsCR": false,
"Permissions": 420,
"Directory": false,
"OwnerExecutable": false,
"BOM": false,
"POSIXy": false
}
$ stink -help
-cr
Report presence/absence of any CR/CRLF's
-eol
Report presence/absence of final end of line sequence
-help
Show usage information
-pp
Prettyprint smell records
-version
Show version information
The included examples/ directory demonstrates many edge cases, such as empty scripts, shebang-less scripts, extensioned and extensionless scripts, and various Hello World applications in across many programming languages. Some files, such as examples/goodbye may contain 100% valid POSIX shell script content, but fail to self-identify with either shebangs or relevant file extensions. In a large project, such files may be mistakenly treated as whoknowswhat format, or simply plain text. Perhaps statistical methods could help identify POSIX grammars, but even an empty file is technically POSIX, which is unhelpful from a reliable classification standpoint. In any case, examples/ hopefully covers the more common edge cases.
DOWNLOADS
https://github.com/mcandre/stank/releases
DOCUMENTATION
http://godoc.org/github.com/mcandre/stank
REQUIREMENTS
Each of the applications in the stank suite are standalone, with no requirements other than deploying to a suitable operating system.
Development
- Go 1.7+
- gox
- zipc
- coreutils
- findutils
- make
- golint
- goimports
- errcheck
- opennota/check (e.g.
go get github.com/opennota/check/cmd/...) - megacheck (e.g.
go get github.com/dominikh/go-tools/cmd/megacheck) - flcl
- editorconfig-cli
INSTALL FROM REMOTE GIT REPOSITORY
$ go get github.com/mcandre/stank/...
INSTALL FROM LOCAL GIT REPOSITORY
$ mkdir -p $GOPATH/src/github.com/mcandre
$ git clone git@github.com:mcandre/stank.git $GOPATH/src/github.com/stank
$ sh -c "cd $GOPATH/src/github.com/mcandre/stank/cmd/stink && go install"
$ sh -c "cd $GOPATH/src/github.com/mcandre/stank/cmd/stank && go install"
$ sh -c "cd $GOPATH/src/github.com/mcandre/stank/cmd/rosy && go install"
WARNING ON FALSE POSITIVES
Some rather obscure files, such as Common Lisp source code with multiline, polyglot shebangs and no file extension, may falsely trigger the stank library, and the rosy, stink, and stank applications, which short-circuit on the first line of the hacky shebang. Such files may be falsely identified as "POSIX" code, which is actually the intended behavior! This is because the polyglot shebang is a hack to work around limitations in the Common Lisp language, which ordinarily does not accept POSIX shebang comments, in order to get Common Lisp scripts to be dot-slashable in bash. For this situation, it is best to supply a proper file extension to such files.
$ head examples/i-should-have-an-extension
#!/usr/bin/env sh
#|
exec clisp -q -q $0 $0 ${1+"$@"}
|#
(defun hello-main (args)
(format t "Hello from main!~%"))
;;; With help from Francois-Rene Rideau
;;; http://tinyurl.com/cli-args
$ stink -pp examples/i-should-have-an-extension
{
"Path": "examples/i-should-have-an-extension",
"Filename": "i-should-have-an-extension",
"Basename": "i-should-have-an-extension",
"Extension": "",
"BOM": false,
"Shebang": "#!/usr/bin/env sh",
"Interpreter": "sh",
"LineEnding": "\n",
"POSIXy": true
}
Perhaps append a .lisp extension to such files. Or separate the modulino into clear library vs. command line modules. Or extract the shell interaction into a dedicated script. Or convince the language maintainers to treat shebangs as comments. Write your congressman. However you resolve this, know that the current situation is far outside the norm, and likely to break in a suitably arcane and dramatic fashion. With wyverns and flaming seas and portents of all ill manner.
LINT
$ make lint
PORT
$ make port
Shell script linters
These bad bois help to shore up ur shell scripts. Though they're designed to work on individual files, so be sure to stank-ify larger projects and pipe the results to xargs checkbashisms, yo!
Honorable mentions
ack offers --shell [-f] flags that act similarly to stank, with the caveat that ack includes nonPOSIX shells like csh, tcsh, and fish in these results; but as of this writing fails to include POSIX shells like ash, dash, posh, pdksh, ksh93, and mksh. ack also depends on Perl, making it more heavyweight for Docker microservices and other constrained platforms.
linguist, GitHub's extraordinary effort to identify which language each of its millions of repositories are written in. While this stanky Go project does not employ linguist in automated analysis, it's worth mentioning for forensic purposes, if you ever come across a strange, unidentified (or misidentified!) source code file.
Documentation
¶
Index ¶
Constants ¶
const Version = "0.0.10"
Version is semver.
Variables ¶
var BOMS = map[string]bool{ "\uFFBBBF": true, "\uFEFF": true, "\uFFFE": true, "\u0000FEFF": true, "\uFFFE0000": true, "\u2B2F7638": true, "\u2B2F7639": true, "\u2B2F762B": true, "\u2B2F762F": true, }
BOMS acts as a registry set of known Byte Order mark sequences. See https://en.wikipedia.org/wiki/Byte_order_mark for more information.
var INTERPRETERS2POSIXyNESS = map[string]bool{ "sh": true, "tsh": false, "etsh": false, "bash": true, "bash4": true, "bosh": true, "osh": false, "yash": true, "zsh": true, "lksh": false, "ksh": true, "ksh88": true, "pdksh": true, "ksh93": true, "mksh": true, "oksh": true, "dash": true, "posh": true, "ash": true, "csh": false, "tcsh": false, "fish": false, "rc": false, "python": false, "jython": false, "perl": false, "perl6": false, "ruby": false, "jruby": false, "lua": false, "node": false, "awk": false, "sed": false, "swift": false, "tclsh": false, "ion": false, }
INTERPRETERS2POSIXyNESS is a fairly exhaustivemap of interpreters to whether or not the interpreter is a POSIX compatible shell. Newly minted interpreters can be added by stank contributors.
var LOWEREXTENSIONS2CONFIG = map[string]bool{ ".shrc": true, ".shinit": true, ".profile": true, ".bash_profile": true, ".bashrc": true, ".bash_login": true, ".bash_logout": true, ".ashrc": true, ".dashrc": true, ".kshrc": true, ".zshenv": true, ".zprofile": true, ".zshrc": true, ".zlogin": true, ".zlogout": true, ".csh": true, ".cshrc": true, ".tcsh": true, ".tcshrc": true, ".fish": true, ".rc": true, ".ionrc": true, }
LOWEREXTENSIONS2CONFIG is a fairly exhaustive map of lowercase file extensions to whether or not they represent shell script configurations. Newly minted extensions can be added by stank contributors.
var LOWEREXTENSIONS2INTERPRETER = map[string]string{
".shrc": "sh",
".shinit": "sh",
".bashrc": "bash",
".zshrc": "zsh",
".lkshrc": "lksh",
".kshrc": "ksh",
".pdkshrc": "pdksh",
".ksh93rc": "ksh93",
".mkshrc": "mksh",
".dashrc": "dash",
".poshrc": "posh",
".ashrc": "ash",
".zshenv": "zsh",
".zprofile": "zsh",
".zlogin": "zsh",
".zlogout": "zsh",
".cshrc": "csh",
".tcshrc": "tcsh",
".fishrc": "fish",
".rcrc": "rc",
".ionrc": "ion",
".profile": "sh",
".bash_profile": "bash",
".bash_login": "bash",
".bash_logout": "bash",
".zshprofile": "zsh",
}
LOWEREXTENSIONS2INTERPRETER is a fairly exhaustive map of lowercase file extensions to their corresponding interpreters. Newly minted config extensions can be added by stank contributors.
var LOWEREXTENSIONS2POSIXyNESS = map[string]bool{ ".sh": true, ".tsh": false, ".etsh": false, ".bash": true, ".bash4": true, ".bosh": true, ".osh": false, ".yash": true, ".zsh": true, ".lksh": false, ".ksh": true, ".ksh88": true, ".pdksh": true, ".ksh93": true, ".mksh": true, ".oksh": true, ".dash": true, ".posh": true, ".ash": true, ".shrc": true, ".shinit": true, ".bash_profile": true, ".bashrc": true, ".bash_login": true, ".bash_logout": true, ".kshrc": true, ".zshenv": true, ".zprofile": true, ".zshrc": true, ".zlogin": true, ".zlogout": true, ".csh": false, ".cshrc": false, ".tcsh": false, ".tcshrc": false, ".fish": false, ".rc": false, ".ionrc": false, ".py": false, ".pyw": false, ".pl": false, ".rb": false, ".lua": false, ".js": false, ".lisp": false, ".mf": false, ".exe": false, ".bin": false, ".cmd": false, ".bat": false, ".psh": false, ".vbs": false, ".ada": false, ".c": false, ".cl": false, ".e": false, ".erl": false, ".escript": false, ".fth": false, ".groovy": false, ".j": false, ".pike": false, ".rkt": false, ".scala": false, ".sf": false, ".txr": false, ".zkl": false, ".txt": false, ".md": false, ".markdown": false, ".doc": false, ".docx": false, ".pdf": false, ".log": false, ".gitignore": false, ".gitmodules": false, ".gitkeep": false, ".xml": false, ".json": false, ".yml": false, ".yaml": false, ".conf": false, ".properties": false, ".svg": false, ".gif": false, ".jpg": false, ".jpeg": false, ".png": false, ".bmp": false, ".tiff": false, ".mp3": false, ".wav": false, ".mp4": false, ".mov": false, ".flv": false, ".swp": false, ".ds_store": false, }
LOWEREXTENSIONS2POSIXyNESS is a fairly exhaustive map of lowercase file extensions to whether or not they represent POSIX shell scripts. Newly minted extensions can be added by stank contributors.
var LOWERFILENAMES2CONFIG = map[string]bool{ "shrc": true, "shinit": true, "profile": true, "login": true, "logout": true, "bash_login": true, "bash_logout": true, "zshenv": true, "zprofile": true, "zshrc": true, "zlogin": true, "zlogout": true, "csh.login": true, "csh.logout": true, "tcsh.login": true, "tcsh.logout": true, "rcrc": true, }
LOWERFILENAMES2CONFIG is a fairly exhaustive map of lowercase filenames to whether or not they represent shell script configurations. Newly minted config filenames can be added by stank contributors.
var LOWERFILENAMES2INTERPRETER = map[string]string{
".shrc": "sh",
".shinit": "sh",
".bashrc": "bash",
".zshrc": "zsh",
".lkshrc": "lksh",
".kshrc": "ksh",
".pdkshrc": "pdksh",
".ksh93rc": "ksh93",
".mkshrc": "mksh",
".dashrc": "dash",
".poshrc": "posh",
".ashrc": "ash",
".zshenv": "zsh",
".zprofile": "zsh",
".zlogin": "zsh",
".zlogout": "zsh",
".cshrc": "csh",
".tcshrc": "tcsh",
".fishrc": "fish",
".rcrc": "rc",
".ionrc": "ion",
"profile": "sh",
".login": "sh",
".logout": "sh",
"zshenv": "zsh",
"zprofile": "zsh",
"zshrc": "zsh",
"zlogin": "zsh",
"zlogout": "zsh",
"csh.login": "csh",
"csh.logout": "csh",
"tcsh.login": "tcsh",
"tcsh.logout": "tcsh",
}
LOWERFILENAMES2INTERPRETER is a fairly exhaustive map of lowercase filenames to their corresponding interpreters. Newly minted config filenames can be added by stank contributors.
var LOWERFILENAMES2POSIXyNESS = map[string]bool{ "shrc": true, "shinit": true, ".profile": true, "profile": true, "login": true, "logout": true, "bash_login": true, "bash_logout": true, "zshenv": true, "zprofile": true, "zshrc": true, "zlogin": true, "zlogout": true, "csh.login": false, "csh.logout": false, "tcsh.login": false, "tcsh.logout": false, "rcrc": false, "makefile": false, "readme": false, "changelog": false, "applypatch-msg.sample": false, "commit-msg.sample": false, "post-update.sample": false, "pre-applypatch.sample": false, "pre-commit.sample": false, "pre-push.sample": false, "pre-rebase.sample": false, "prepare-commit-msg.sample": false, "update.sample": false, "thumbs.db": false, }
LOWERFILENAMES2POSIXyNESS is a fairly exhaustive map of lowercase filenames to whether or not they represent POSIX shell scripts. Newly minted config filenames can be added by stank contributors.
Functions ¶
This section is empty.
Types ¶
type Smell ¶
type Smell struct {
Path string
Filename string
Basename string
Extension string
Shebang string
Interpreter string
LineEnding string
FinalEOL bool
ContainsCR bool
Permissions os.FileMode
Directory bool
OwnerExecutable bool
BOM bool
POSIXy bool
}
Smell describes the overall impression of a file's POSIXyness, using several factors to determine with a reasonably high accuracy whether or not the file is a POSIX compatible shell script.
An idiomatic shebang preferably leads the file, such as #!/bin/bash, #!/bin/zsh, #!/bin/sh, etc. This represents good form when writing shell scripts, in particular ensuring that the script will be evaluated by the right interpreter, even if the extension is omitted or a generic ".sh". Shell scripts, whether executable applications or source'able libraries, should include a shebang. One attribute not analyzed by this library is unix file permission bits. Application shell scripts should set the executable bit(s) to 1, while shell scripts intended to be sourced or imported should not set these bits. Either way, the bits have hardly any correlation with the POSIXyness of a file, as the false positives and false negatives are too frequent.
Common filenames for POSIX compatible scripts include .profile, .login, .bashrc, .bash_profile, .zshrc, .kshrc, .envrc*, and names for git hooks. The stank library will catalog some of these standard names, though application-specific filenames are various and sundry. Ultimately, all files containing POSIX compatible shell content should include a shebang, to help interpreters, editors, and linters identify POSIX shell content.
File extension is another way to estimate a script's POSIXyness. For example, ".bash", ".ksh", ".posh", ".sh", etc. would each indicate a POSIX compatible shell script, whereas ".py", ".pl", ".rb", ".csh", ".rc", and so on would indicate nonPOSIX script. File extensions are often omitted or set to a generic ".sh" for command line applications, in which case the extension is insufficient for establishing the POSIX vs. nonPOSIX nature of the script. This is why shebangs are so important; while file extensions can be helpful, shell scripts really rely moreso on the shebang for self identification, and extensions aren't always desirable, as unix CLI applications prefer to omit the extension from the filename for brevity. Note that some filenames such as ".profile" may be logically considered to have basename "" (blank) and extension ".profile", or basename ".profile" with extension ".profile", or else basename ".profile" and extension "" (blank). In practice, Go treats both the basename and extension for these kinds of files as containing ".profile", and Smell will behave accordingly.
File encoding also sensitive for shell scripts. Generally, ASCII subset is recommended for maximum portability. If your terminal supports it, the LANG environment variable can be altered to accept UTF-8 and other encodings, enabling raw UTF-8 data to be used in script contents. However, this restricts your scripts to running only on systems explicitly configured to match the encoding/locale of your script; and tends to furter limit the platforms for your script to specifically GNU libc Linux distributions, so using nonASCII content in your scripts is inadvisable. Shell scripts conforming to POSIX should really use pure ASCII characters. NonUTF-8 encodings such as UTF-16, UTF-32, and even nonUnicode encodings like EBCDIC, Latin1, and KOI8-R usually indicate a nonPOSIX shell script, even a localization file or other nonscript. These encodings are encountered less often than ASCII and UTF-8, and are generally considered legacy formats. For performance reasons, the stank library will not attempt to discern the exact encoding of a file, but merely report whether the file leads with a byte order marker such as 0xEFBBBF (UTF-8) or 0xFEFF (UTF-16, UTF-32). If BOM, then the file is Unicode, which may lead to a stank warning, as POSIX shell scripts are best written in pure ASCII, for maximum cross-platform compatibliity. BOMs are outside of the 127 max integer range for ASCII values, so a file with a BOM is likely not a POSIX shell script, while a file without a BOM may be a POSIX shell script.
Line endings for POSIX shell scripts should LF="\n" in C-style notation. Alternative line endings such as CRLF="\r\n", ancient Macintosh CR="\r", and bizarre forms like vertical tab (ASCII code 0x0B) or form feed (ASCII code 0x0C) are possible in a fuzzing sense, but may lead to undefined behavior depending on the particular shell interpreter. For the purposes of identifying POSIX vs nonPOSIX scripts, a Smell will look for LF, CRLF, and CR; and ignore the presence or absence of these other exotic whitespace separators. NonPOSIX scripts written in Windows, such as Python and Ruby scripts, are ideally written with LF line endings, though it is common to observe CRLF endings, as Windows users more frequently invoke these as "python script.py", "ruby script.rb", rather than the bare "script" or dot slash "./script" forms typically used by unix administrators. For performance, the stank library will not report possible multiple line ending styles, such as poorly formatted text files featuring both CRLF and LF line endings. The library will simply report the first confirmed line ending style.
Moreover, POSIX line ending LF is expected at the end of a text file, so a final end of line character "\n" is good form. Common unix utilities such as cat expect this final EOL, and will misrender the successive shell prompt when processing files that omit the final EOL. Make expects a final EOL, and gcc may produce malformed .c code if the .h header files neglect to include a final EOL. For performance reasons, the stank library will not attempt to read the entire file to report on the presence/absence of a final EOL. Shell script authors should nonetheless configure their text editors to consistently include a final EOL in the vast majority of text file formats.
A POSIXy flag indicates that, to the best of the stank library's ability, a file is identified as either very likely a POSIX shell script, or something else. Something else encompasses nonPOSIX shell scripts such as Csh, Tcsh, Python, Ruby, Lua scripts; also encompasses nonscript files such as multimedia images, audio, rich text documents, machine code, and other nonUTF-8, nonASCII content.
func Sniff ¶
func Sniff(pth string, config SniffConfig) (Smell, error)
Sniff analyzes the holistic smell of a given file path, returning a Smell record of key indicators tending towards either POSIX compliance or noncompliance, including a flag for the final "POSIXy" trace scent of the file.
For performance, if the scent of one or more attributes obviously indicates POSIX or nonPOSIX, Sniff() may short-circuit, setting the POSIXy flag and returning a record with some attributes set to zero value.
Polyglot and multiline shebangs are technically possible in languages that do not support native POSIX-style shebang comments ( see https://rosettacode.org/wiki/Multiline_shebang ). However, Sniff() can reliably identify only ^#!.+$ POSIX-style shebangs, and will populate the Shebang field accordingly.
If an I/O problem occurs during analysis, an error value will be set. Otherwise, the error value will be nil.
type SniffConfig ¶ added in v0.0.9
SniffConfig bundles together the various options when sniffing files for POSIXyNESS.
Source Files
Directories