portfoliodb

package module
v0.0.0-...-6f6c2d3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 8, 2021 License: MIT Imports: 28 Imported by: 0

README

portfoliodb

I'm following RDD (readme-driven development) for this project, so, until v0.1.0 is released, this document describes what the program will look like

A readable, easy and enjoyable way to manage portfolio databases using directories and text files.

Installation

Pre-compiled binaries are available through GitHub Releases:

$ wget https://github.com/ewen-lbh/portfoliodb/releases/latest/portfoliodb
# Put the command in a directory that is in your PATH, so that you can use portfoliodb from anywhere, e.g.:
$ mv portfoliodb /usr/bin/portfoliodb

See Compiling for instructions on how to compile this yourself

Usage

Usage:
  portfoliodb [options] <database> build <to-filepath> [--config=FILEPATH] [-msS] [--]
  portfoliodb [options] replicate <from-filepath> <to-directory> [--config=FILEPATH]
  portfoliodb [options] <database> add <fullname> [<metadata-item>...]
  portfoliodb [options] <database> validate <database>

Options:
  -C --config=<filepath>      Use the configuration path at <filepath>. [default: .portfoliodb.yml]
  -m --minified               Output a minifed JSON file
  -s --silent                 Do not write to stdout
  -S --scattered              Operate in scattered mode. See Scattered Mode section for more information.

Examples:
  portfoliodb database build database.json
  portfoliodb database add schoolsyst/presentation -#web -#site --color 268CCE
  portfoliodb replicate database.json replicated-database --config=.portfoliodb.yml

Commands:
  build <from-directory> <to-filepath>
    Scan in <from-directory> for folders with description.md files
    (and potential media files)
    and compile the whole database into a JSON file at <to-filepath>

  replicate <from-filepath> <to-directory>
    The reverse operation of 'build'.
    Note that <to-directory> must be an empty directory

  add <name> [<metadata-item>...]
    Creates a new description.md in the appropriate folder.
    <name> is the work's name.
    You can provide additional metadata items in the form --ITEM_NAME=VALUE,
    eg. 'add phelng --tag=cli --tag=program' will generate ./phelng/description.md,
    with the following contents:
    ---
    collection: null
    ---
    # phelng
    program, cli

  validate <database>
    Make sure that everything is OK in the database:
    Each one of these checks are configurable and deactivable in .portfoliodb.yml:validate.checks,
    the step name is the one in [square brackets] at the beginning of these lines.
    1. [schema compliance] validate compliance to schema for .portfoliodb.yml and .portfoliodb-metadata.yml
    2. [work folder names] check work folder names for url-unsafe characters or case-insensitively non-unique folder names
    3. for each work directory:
        a. [yaml header] check YAML header for unknown keys using .portfoliodb-metadata.yml
        b. [title presence] check presence of work title
        c. [title uniqueness] check uniqueness (case-insensitive) of work title
        d. [tags presence] check if at least one tag is present
        e. [tags knowledge] check absence of unknown tags (using .portfoliodb-metadata.yml)
        f. [working media files] check all local paths for links (audio/video files, image files, other files)
        g. [working urls] check that no http url gives errors

Scattered mode:
  With this mode activated, when building, portfoliodb will go through each folder (non-recursively) of <from-directory>, and, if it finds a .portfoliodb file in the folder, consider the files in that .portfoliodb folder.

  Consider the following directory tree:

  <from-directory>
    project1
	  index.html
	  src
	  dist
	  .portfoliodb
	    file1.png
		description.md
	project2
	  .portfoliodb
	    file-2.png
		description.md
	otherfolder
	  stuff

  Running portfoliodb build --scattered on this tree is equivalent to builing without --scattered on the following tree:

  <from-directory>
    project1
	  file.png
	  description.md
	project2
	  file-2.png
	  description.md

  Concretely, it allows you to store your portfoliodb descriptions and supporting files directly in your projects, assuming that your store all of your projects under the same directory.
`

How it works

Your database is a folder, which has one folder per work in it. In each folder, you'll have a markdown file describing your work, and other files relevant to the work (images, PDFs, audio files, videos, etc.).

Here's an example tree:

database/
├── ideaseed
│   ├── logo.png
│   └── description.md
├── phelng
│   └── description.md
├── portfolio
│   └── description.md
└── portfoliodb
    └── description.md

"Building" your database is just translating that easy-to-maintain and natural directory tree to a single JSON file, easily consummable by your frontend website. This way, you can add new projects to your portfolio without having to write a single line of code: just create a new folder, describe your project, build the database, upload it, and done!

description.md files

Description files are separated in "blocks": blocks are separated by an empty line. There are:

You can also translate your description.md file into multiple languages by using language markers

Start your file with a top-level header # Like this to give your work a title (it can differ from the folder's name, since the folder name is used as the work's identifier, and is guaranteed to be unique).

Paragraphs

These blocks allow you to write some text using an extended markdown syntax, adding support for abbreviations and footnotes.

Paragraphs will be accessible in the JSON file in the paragraphs object. Each paragraph has two properties: content, which contains the paragraph content, and an id, which can be specified manually:

other stuff...

{#my-paragraph-id}
The start of the paragraph.
Specify the paragraph's ID by starting your paragraph with a {#your-identifier} on a single line.

other stuff...

and will be empty otherwise. This id can be useful to link to a specific paragraph of your page by using it as, for example, a <p> tag's id: you can then link to that specific paragraph with https://example.com/...#my-paragraph-id

Media

A "media" block allows you to declare files embedded in your page: YouTube videos, local files, etc.

With native markdown, you can only declare embeds for images. We abuse the syntax to extend it to any file you want.

![alt text "title"](./demo.mp4)

source can be a relative path, an absolute one or a URL.

When building, the compiler will look for these files and analyze them to determine their content type, dimensions, aspect ratio, duration and file size, and will then be accessible in the JSON file as an array of media objects having the following structure:

{
  "dimensions": {
    "height": 1080, // 0 if the file has no dimensions (eg. an audio file)
    "width": 1920, // 0 for the same reasons
    "aspect_ratio": 1.777777778 // 0 if either of the dimensions are zero. aspect_ratio is width / height.
  },
  "source": "./demo.mp4",
  "alt": "alt text",
  "title": "title",
  "duration": 68, // In seconds. 0 if the file has no duration (eg. an image)
  "size": 1854210, // In bytes
  "content_type": "video/mp4", // MIME types
}

Of course, you can use links inside of a paragraphs, but you can also declare isolated links that don't need context to be meaningful. Here are some use cases:

  • For a website, you can link to the source code repository and the website itself,
  • For a t-shirt, you can link to a marketplace so that people viewing that work can buy it
  • and plenty of other use cases

Configuration

Put this in .portfoliodb.yml in the root of your database:

build steps:
  - step: extract colors
    default file names: [logo.png]

  - step: make gifs
    # <filetitle> refers to the filename without its extension.
    file name template: <filetitle>.gif

  - step: make thumbnails
    widths: [20, 100, 500]
    # Paths are always relative to the work's database folder
    file name template: ../../static/thumbs/<id>/<width>.png


validate:
  checks:
    # can be `off` (not checked for)
    # can be `on` (uses the default level)
    # can be a level:
    # - `fatal`: also checked when building, triggers end of build if fails
    # - `error`: prints an error message (red), makes validate command exit with 1
    # - `warn` : prints a warning message (orange), does not make validate exit with 1
    # - `info` : regular message, informative
    # these are the default values
    schema compliance: fatal
    work folder uniqueness: fatal
    work folder safeness: error
    yaml header: error
    title presence: error
    title uniqueness: error
    tags presence: warn
    tags knowledge: error
    working media: warn
    working urls: off

PRO TIP: You can use the provided .portfoliodb.yml.schema.json to validate your YAML file with this JSONSchema

Extra markdown features

Except for the >[text](video/audio URL/filepath) feature, the markdown also supports a number of non-standard features:

  • all of what GFM supports (except autolinking of issues and commit hashes, ofc)
  • Abbreviations: *[YAML]: Yet Another Markup Language
  • Definition lists: - key: value or the more standard, PHP-markdown-extra-style
  • Admonitions: !!! type "Optional title", see this documentation
  • Footnotes: footnote reference[^1] and then [^1]: footnote content
  • Markdown in HTML: See documentation here
  • (off by default) New-line-to-line-break: Transforms line breaks in markdown into <br>s, see the documentation
  • Smarty pants: typographic replacements (not replaced inside code):
    • -- to –
    • --- to —
    • -> to →
    • <- to ←
    • ... to …
    • << to «
    • >> to »
  • (off by default) Anchored headings: Each headings is assigned an id to reference in the URL with example.com#heading
Configuring markdown

The extra features discussed just above are all available or disable, using the module name:

.portfoliodb.yml

markdown:
  abbreviations: on
  definition lists: on
  admonitions: off
  footnotes: on
  markdown in html: on
  new-line-to-line-break: on
  smarty pants: off
  anchored headings:
  # you can also use an object form to pass in config options
    enabled: yes
    format: <content> # default value
  custom syntaxes:
    # this is just an example, not an actual implementation of the video/audio embed feature
    - from: '>\[(?P<fallback>[^\]]+)\]\((?P<source>.+)\)'
      to: <video src="${source}">${fallback}</video>

Compiling

  1. Clone the repository: git clone https://github.com/ewen-lbh/portfoliodb
  2. cd into it: cd portfoliodb
  3. make the binary: make
  4. Install it (this just copies the file to /usr/bin/): make install

Documentation

Index

Constants

View Source 
const CLIUsage = `` /* 3558-byte string literal not displayed */

CLIUsage is the entire usage string for the CLI

View Source 
const ConfigurationJSONSchema = `` /* 3187-byte string literal not displayed */

ConfigurationJSONSchema is the entire json string from .portfoliodb.yml.json.schema

View Source 
const DatabaseJSONSchema = `` /* 3750-byte string literal not displayed */

DatabaseJSONSchema is the entire json string from database.json.schema

Variables

This section is empty.

Functions

func AnalyzeAllMediae 

func AnalyzeAllMediae(ctx RunContext, embedDeclarations map[string][]MediaEmbedDeclaration, currentDirectory string) (map[string][]Media, error)

AnalyzeAllMediae analyzes all the mediae from ParsedDescription's MediaEmbedDeclarations and returns analyzed mediae, ready for use as Work.Media

func DisplayValidationErrors 

func DisplayValidationErrors(errors []gojsonschema.ResultError, name string)

DisplayValidationErrors takes in a slice of json schema validation errors and displays them nicely to in the terminal

func FileExists 

func FileExists(filepath string) bool

FileExists checks if the file at “filepath“ exists, and returns “true“ if it exists or “false“ otherwise

func FilepathBaseNoExt 

func FilepathBaseNoExt(pth string) string

FilepathBaseNoExt returns the basename of pth with the extension removed

func FilterSlice 

func FilterSlice(s []string, cond func(string) bool) []string

FilterSlice returns a slice of strings containing only the elements that return true when called with cond.

func GetAudioDuration 

func GetAudioDuration(file *os.File) uint

GetAudioDuration takes in an os.File and returns the duration of the audio file in seconds. If any error occurs the duration will be 0.

func IsValidURL 

func IsValidURL(URL string) bool

IsValidURL tests a string to determine if it is a well-structured url or not.

func LoadConfiguration 

func LoadConfiguration(filepath string, loadInto *Configuration) error

LoadConfiguration loads the .portfoliodb.yml file in “databaseFolderPath“ and puts it contents into “loadInto“.

func LowerCaseWithUnderscores 

func LowerCaseWithUnderscores(name string) string

LowerCaseWithUnderscores one strategy to SetNamingStrategy for. It will change HelloWorld to hello_world.

func MapKeys 

func MapKeys(m map[string]string) []string

MapKeys returns a slice of strings containing the map's keys

func ParseYAMLHeader 

func ParseYAMLHeader(descriptionRaw string) (map[string]interface{}, string)

ParseYAMLHeader parses the YAML header of a description markdown file and returns the rest of the content (all except the YAML header)

func ReadDescriptionFile 

func ReadDescriptionFile(directory string) (string, error)

ReadDescriptionFile reads the description.md file in directory. Returns an empty string if the file is a directory or does not exist.

func ReadFile 

func ReadFile(filepath string) (string, error)

ReadFile reads the content of “filepath“ and returns the contents as a string

func ReadFileBytes 

func ReadFileBytes(filepath string) ([]byte, error)

ReadFileBytes reads the content of “filepath“ and returns the contents as a byte array

func RegexpGroups 

func RegexpGroups(regex string, s string) []string

RegexpGroups returns all the capture groups' contents from the first match of regex regex in s. The first element [0] is the entire match. [1] is the first capture group's content, et cætera.

func RegexpMatches 

func RegexpMatches(regex string, s string) bool

RegexpMatches checks if s matches the regex regex at least once

func ReplicateAll 

func ReplicateAll(ctx RunContext, targetDatabase string, works []Work) error

ReplicateAll recreates a database inside targetDatabase containing all the works in `works`

func ReplicateDescription 

func ReplicateDescription(work Work) (string, error)

ReplicateDescription reconstructs the contents of a description.md file from a Work struct

func ReplicateOne 

func ReplicateOne(targetDatabase string, work Work) error

ReplicateOne creates a description.md file in targetDatabase in the correct folder in order to replicate Work

func ResolveConfigurationPath 

func ResolveConfigurationPath(databaseDirectory string, explicitlySpecifiedConfigurationFilepath string) string

ResolveConfigurationPath determines the path of the configuration file to use

func RunCommandAdd 

func RunCommandAdd(args docopt.Opts) error

RunCommandAdd runs the command 'add' given parsed CLI args from docopt

func RunCommandBuild 

func RunCommandBuild(args docopt.Opts) error

RunCommandBuild runs the command 'build' given parsed CLI args from docopt

func RunCommandReplicate 

func RunCommandReplicate(args docopt.Opts) error

RunCommandReplicate runs the command 'replicate' given parsed CLI args from docopt

func RunCommandValidate 

func RunCommandValidate(args docopt.Opts) error

RunCommandValidate runs the command 'validate' given parsed CLI args from docopt

func SetJSONNamingStrategy 

func SetJSONNamingStrategy(translate func(string) string)

SetJSONNamingStrategy rename struct fields uniformly

func StepExtractColors 

func StepExtractColors(metadata map[string]interface{}, project ProjectTreeElement, databaseDirectory string, config Configuration) map[string]interface{}

StepExtractColors executes the step "extract colors" and returns a metadata object with the `colors` entry modified accordingly.

func StringInSlice 

func StringInSlice(haystack []string, needle string) bool

StringInSlice checks if needle is in haystack

func ValidateConfiguration 

func ValidateConfiguration(configFilepath string) (bool, []gojsonschema.ResultError, error)

ValidateConfiguration uses the JSON configuration schema ConfigurationJSONSchema to validate the configuration file at configFilepath

func ValidateWithJSONSchema 

func ValidateWithJSONSchema(document string, schema string) (bool, []gojsonschema.ResultError, error)

ValidateWithJSONSchema checks if the JSON document “document“ conforms to the JSON schema “schema“

func WriteFile 

func WriteFile(filename string, content []byte) error

WriteFile writes content to file filepath

Types

type Abbreviation 

type Abbreviation struct {
	Name       string
	Definition string
}

Abbreviation represents an abbreviation declaration in a description.md file

type Configuration 

type Configuration struct {
	Checks              checks               `yaml:"checks"`
	ReplaceMediaSources []replaceMediaSource `yaml:"replace media sources"`
	// contains filtered or unexported fields
}

Configuration represents what the .portfoliodb.yml configuration file describes

func GetConfiguration 

func GetConfiguration(filepath string) (Configuration, error)

GetConfiguration reads from the .portfoliodb.yml file in “databaseFolderPath“ and returns a “Configuration“ struct

func GetConfigurationFromCLIArgs 

func GetConfigurationFromCLIArgs(args docopt.Opts) (Configuration, []gojsonschema.ResultError, error)

GetConfigurationFromCLIArgs gets the configuration by using the CLI arguments

type ExtractedColors 

type ExtractedColors struct {
	Primary   string
	Secondary string
	Tertiary  string
}

ExtractedColors reprensents the object in a Work's metadata.colors

type Footnote 

type Footnote struct {
	Name    string
	Content string
}

Footnote represents a footnote declaration in a description.md file

type ImageDimensions 

type ImageDimensions struct {
	Width       int
	Height      int
	AspectRatio float32
}

ImageDimensions represents metadata about a media as it's extracted from its file

func GetImageDimensions 

func GetImageDimensions(file *os.File) (ImageDimensions, error)

GetImageDimensions returns an “ImageDimensions“ object, given a pointer to a file

func GetVideoDimensionsDurationHasSound 

func GetVideoDimensionsDurationHasSound(filename string) (dimensions ImageDimensions, duration uint, hasSound bool, err error)

GetVideoDimensionsDurationHasSound returns an ImageDimensions struct with the video's height, width and aspect ratio and a duration in seconds. 

type Link struct {
	ID    string
	Name  string
	Title string
	URL   string
}

Link represents an (isolated) link declaration in a description.md file

type Media 

type Media struct {
	ID          string
	Alt         string
	Title       string
	Source      string
	ContentType string
	Size        uint64 // In bytes
	Dimensions  ImageDimensions
	Duration    uint // In seconds
	Online      bool // Whether the media is hosted online (referred to by an URL)
	Attributes  MediaAttributes
	HasSound    bool // The media is either an audio file or a video file that contains an audio stream
}

Media represents a media object inserted in the work object's “media“ array.

func AnalyzeMediaFile 

func AnalyzeMediaFile(filename string, embedDeclaration MediaEmbedDeclaration, config Configuration) (Media, error)

AnalyzeMediaFile analyzes the file at filename and returns a Media struct, merging the analysis' results with information from the matching MediaEmbedDeclaration

type MediaAttributes 

type MediaAttributes struct {
	Looped      bool // Controlled with attribute character ~ (adds)
	Autoplay    bool // Controlled with attribute character > (adds)
	Muted       bool // Controlled with attribute character > (adds)
	Playsinline bool // Controlled with attribute character = (adds)
	Controls    bool // Controlled with attribute character = (removes)
}

MediaAttributes stores which HTML attributes should be added to the media

type MediaEmbedDeclaration 

type MediaEmbedDeclaration struct {
	Alt        string
	Title      string
	Source     string
	Attributes MediaAttributes
}

MediaEmbedDeclaration represents media embeds. (abusing the ![]() syntax to extend it to any file) Only stores the info extracted from the syntax, no filesystem interactions.

type Paragraph 

type Paragraph struct {
	ID      string
	Content string
}

Paragraph represents a paragraph declaration in a description.md file

type ParsedDescription 

type ParsedDescription struct {
	Metadata               map[string]interface{}
	Title                  map[string]string
	Paragraphs             map[string][]Paragraph
	MediaEmbedDeclarations map[string][]MediaEmbedDeclaration
	Links                  map[string][]Link
	Footnotes              map[string][]Footnote
}

ParsedDescription represents a work, but without analyzed media. All it contains is information from the description.md file

func ParseDescription 

func ParseDescription(ctx RunContext, markdownRaw string) ParsedDescription

ParseDescription parses the markdown string from a description.md file and returns a ParsedDescription

type ProjectTreeElement 

type ProjectTreeElement struct {
	ID             string
	DescriptionRaw string
	MediaFilepaths []string
	ScatteredMode  bool // Whether the build was run with --scattered
}

ProjectTreeElement represents a project

func BuildProjectsTree 

func BuildProjectsTree(databaseDirectory string) ([]ProjectTreeElement, error)

BuildProjectsTree scans databaseDirectory to return a slice of ProjectTreeElement's, gathering media files and other various information

func BuildProjectsTreeScatteredMode 

func BuildProjectsTreeScatteredMode(projectsDirectory string) ([]ProjectTreeElement, error)

func (*ProjectTreeElement) GetProjectPath 

func (p *ProjectTreeElement) GetProjectPath(databaseDirectory string) string

GetProjectPath returns the project's folder path with regard to databaseDirectory

func (*ProjectTreeElement) MediaAbsoluteFilepaths 

func (p *ProjectTreeElement) MediaAbsoluteFilepaths(databaseDirectory string) []string

MediaAbsoluteFilepaths is like MediaFilepaths but returns absolute paths with regard to databaseDirectory

type RunContext 

type RunContext struct {
	// contains filtered or unexported fields
}

RunContext holds several "global" references used throughout all the functions of a command

func (*RunContext) Status 

func (ctx *RunContext) Status(text string)

Status prints the current compilation progress

type Thumbnail 

type Thumbnail struct {
	Type        string
	ContentType string
	Format      string
	Source      string
	// contains filtered or unexported fields
}

Thumbnail represents a thumbnail

type Work 

type Work struct {
	ID         string
	Metadata   map[string]interface{}
	Title      map[string]string
	Paragraphs map[string][]Paragraph
	Media      map[string][]Media
	Links      map[string][]Link
	Footnotes  map[string][]Footnote
}

Work represents a complete work, with analyzed mediae

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.