simple

README

Simple package

Simple is a lightweight Go package designed for utilities tasks. Avoid boilerplate and insecure code and make it simple!

The package is currently in its early v1. The purpose of this package is to get simple functions with strong features like context management, easy configuration, algorithm efficiency or default security. While I am developing, I am improving the package day by day. Finding new ideas and ways to make the features even more simpler. Don't mind sharing your thoughts and advice.

Features

• Database management :

  • Choose your preferred driver: MySQL, PostgreSQL, or SQLite
  • Simple methods: OpenDatabase(), CloseDatabase(), and standard SQL queries.
  • Thread-safe operations with internal mutexes.
  • Use the provided Database struct or implement your own via the DatabaseManager interface.

• Web client management: :

  • Make web requests more simple
  • Choose the client of your choice : http, Tor sock, your own proxy
  • Fetch raw web content with GetContent().
  • Parse HTML documents with GetParsedContent().
  • Download files with DownloadDocument(), and with automatic sha-256 hash computation via DownloadDocumentReturnHash().

• Hash functions :

  • Supported algorithms:
    • sha-224, sha-256, sha-384, sha-512
    • sha3 family (224, 256, 384, 512)
    • shake-128, shake-256
    • blake2b (256, 384, 512)
    • blake2s (128, 256)
    • md5 (legacy)
    • sha1 (legacy)
  • File hashing with optional buffer sizes for efficiency:
    • Constants: buf_32_kb, buf_64_kb, buf_1_mb, buf_5_mb, buf_10_mb
  • Compare files or hash arbitrary data.

• Environnement variables :

  • Use the gotenv package to pick up your env variables easily

• Deserialize and parse files :

  • open your files and parse the content into your struct
  • supported file format : json, yaml, toml and xml

Development

v0 to v1 :

• 🚩 v0.1.0 : first commit
• 🪜 v0.2.0 : more feature for database and major rework of requests with a clean client management
• 🪜 v0.2.1 : add Hash feature
• 🪜 v0.2.2 : add Env feature + add Test and Mock + small improvements
• 🪜 v0.2.3 : rework of Database feature + small fixes and improvements
• ➡️ v0.2.4 : add new feature Deserializer for opening files like json or yaml and parse it into your struct
• 👷‍♂️ v0.2.5 : add Context as a new feature for easy context creation
• 🏁 v1.0.0 : first release, full revision of the code + comments

TODO :

• fix leak issue with Query()
• create new feature Context
• make every feature support Context
• improvement for Requests feature
• add more encoding to deserialize
• add suport for legacy hash registry

v1 and further :

• Better error model : no panic + set of sentinel errors ...
• Security by default policy (i.e : for database, always setup the sslmode or support certificate for requets)
• Support ORM for database
• Integration of workflows
• Logging Hooks

Dependencies

• Database features are based on the Go’s standard database/sql package
• Requests features are based on Go’s standard net/http package
• Hash features are based on Go’s standard crypto package
• Env features are based on gotenv package
• Deserialize features are based on
  • Go’s standard encoding package
  • go-toml package
  • yaml.v3 package
  • validator package

Documentation

Index

• Constants
• Variables
• func CompareFiles(hash string, fileA string, fileB string) (bool, error)
• func CompareFilesKey(hash string, keyA []byte, fileA string, keyB []byte, fileB string) (bool, error)
• func DownloadDocument(url string, filePath string, client *http.Client) error
• func DownloadDocumentReturnHash(url string, filePath string, client *http.Client) (string, error)
• func GetContent(url string, client *http.Client) ([]byte, error)
• func GetMysql(user string, password string, host string, port string, dbName string) (string, string)
• func GetParsedContent(url string, client *http.Client) (*html.Node, error)
• func GetPostgres(host string, user string, password string, dbName string, port int, ...) (string, string)
• func GetSqlite(filePath string) (string, string)
• func HashData(hash string, data []byte) (string, error)
• func HashFile(hash string, filePath string) (string, error)
• func HashFileBuffer(hash string, filePath string, bufSize int) (string, error)
• func HashFileKey(hash string, key []byte, filePath string) (string, error)
• func HttpClient() *http.Client
• func LoadFileMultiple[S any](fileFormat string, file string, validation bool) ([]S, error)
• func LoadFileSingle[S any](fileFormat string, file string, validation bool) (S, error)
• func OnionClient() (*http.Client, error)
• func OpenEnv(tags ...string) []string
• func OpenEnvFilenames(filenames []string, tags ...string) []string
• type Database
  • func OpenDatabase(driver string, dsn string) (*Database, error)
  • func (db *Database) Close() error
  • func (db *Database) Open(driver string, dsn string) error
  • func (db *Database) Query(query string, args ...any) (*sql.Rows, error)
• type DatabaseManager
• type FileFormatFactory
• type HashKeyFactory

Constants

const (
	KB = 1024
	MB = KB * 1024
)

Variables

var FileFormatRegistry = map[string]FileFormatFactory{
	"json": json.Unmarshal,
	"yaml": yaml.Unmarshal,
	"toml": toml.Unmarshal,
	"xml":  xml.Unmarshal,
}

var Registry = map[string]func() hash.Hash{
	"sha256":   sha256.New,
	"sha384":   sha512.New384,
	"sha512":   sha512.New,
	"sha3-224": sha3.New224,
	"sha3-256": sha3.New256,
	"sha3-384": sha3.New384,
	"sha3-512": sha3.New512,
	"shake-128": func() hash.Hash {
		return &shakeAdapter{shake: sha3.NewShake128(), length: 32}
	},
	"shake-256": func() hash.Hash {
		return &shakeAdapter{shake: sha3.NewShake256(), length: 64}
	},
}

var RegistryKey = map[string]HashKeyFactory{
	"blake2b-256": func(key []byte) (hash.Hash, error) {
		return blake2b.New256(key)
	},
	"blake2b-384": func(key []byte) (hash.Hash, error) {
		return blake2b.New384(key)
	},
	"blake2b-512": func(key []byte) (hash.Hash, error) {
		return blake2b.New512(key)
	},
	"blake2s-128": func(key []byte) (hash.Hash, error) {
		return blake2s.New128(key)
	},
	"blake2s-256": func(key []byte) (hash.Hash, error) {
		return blake2s.New256(key)
	},
}

var RegistryLegacy = map[string]func() hash.Hash{
	"md5":  md5.New,
	"sha1": sha1.New,
}

Legacy hash not supported yet.

Functions

func CompareFiles

func CompareFiles(hash string, fileA string, fileB string) (bool, error)

func CompareFilesKey

func CompareFilesKey(hash string, keyA []byte, fileA string, keyB []byte, fileB string) (bool, error)

func DownloadDocument

func DownloadDocument(url string, filePath string, client *http.Client) error

download a document give the url, the filepath where you want to store the document and the client

func DownloadDocumentReturnHash

func DownloadDocumentReturnHash(url string, filePath string, client *http.Client) (string, error)

download a document and return its hash value

func GetContent

func GetContent(url string, client *http.Client) ([]byte, error)

Get the body of a webpage. Give the url and it will return the html body

func GetMysql

func GetMysql(user string, password string, host string, port string, dbName string) (string, string)

func GetParsedContent

func GetParsedContent(url string, client *http.Client) (*html.Node, error)

Get the body and parse it. The package net/html allow to parse a html body into nodes to easily retrieve every html tags

func GetPostgres

func GetPostgres(host string, user string, password string, dbName string, port int, sslMode string, timezone string) (string, string)

func GetSqlite

func GetSqlite(filePath string) (string, string)

func HashData

func HashData(hash string, data []byte) (string, error)

func HashFile

func HashFile(hash string, filePath string) (string, error)

func HashFileBuffer

func HashFileBuffer(hash string, filePath string, bufSize int) (string, error)

func HashFileKey

func HashFileKey(hash string, key []byte, filePath string) (string, error)

func HttpClient

func HttpClient() *http.Client

http client Default config

func LoadFileMultiple

func LoadFileMultiple[S any](fileFormat string, file string, validation bool) ([]S, error)

The file content must represent an array/list of objects in the specified format

func LoadFileSingle

func LoadFileSingle[S any](fileFormat string, file string, validation bool) (S, error)

The file content must represent a single object in the specified format

func OnionClient

func OnionClient() (*http.Client, error)

onion client Default config

func OpenEnv

func OpenEnv(tags ...string) []string

Open a .env file Return results in a slice of string

func OpenEnvFilenames

func OpenEnvFilenames(filenames []string, tags ...string) []string

Merge several .env files and Return results in a slice of string

Types

type Database

type Database struct {
	Dsn         string
	Conn        *sql.DB
	Driver      string
	PingSuccess bool
	// contains filtered or unexported fields
}

Default database manager

func OpenDatabase

func OpenDatabase(driver string, dsn string) (*Database, error)

func (*Database) Close

func (db *Database) Close() error

func (*Database) Open

func (db *Database) Open(driver string, dsn string) error

func (*Database) Query

func (db *Database) Query(query string, args ...any) (*sql.Rows, error)

Issue : leaking connection with rows.

type DatabaseManager

type DatabaseManager interface {
	Open(dbName string, dsn string) error
	Close() error
	Query(query string, args ...string) (*sql.Rows, error)
}

type FileFormatFactory

type FileFormatFactory func(data []byte, v any) error

type HashKeyFactory

type HashKeyFactory func(key []byte) (hash.Hash, error)