teflon

package module
v0.0.0-...-bc02ef2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 18, 2020 License: Apache-2.0 Imports: 20 Imported by: 0

README

Teflon

Teflon is a post-production management framework for VFX heavy cinematic shows, like feature films, and TV series. Teflon is under early development, the code and the documentation is only published here to aid discussion about the design. Please don't use it in production in any circumstances, but feel free to experiment with the tools provided.

Teflon is rather different from the pipeline frameworks we know about. Teflon is file-system based, there is no database backend behind the logic. All metadata is stored in the file-system together with the data they refer to. The teflon tool creates a view in RAM about the file-system, makes modification to the objects then write the results back to the file-system.

There are some useful information scattered throughout the Teflon Wiki, but a proper white paper is also in the works. Also feel free to come and chat with us at the Teflon Gitter community.

GoDoc

Documentation

Overview

The teflon package implements all the system functionalities of Teflon.

Unfortunately this document is polluted with struct fields and methods generated by `protoc` during compilation. Method names starting with `XXX` and `Get` belong to this category, please ignore them, as we don't use them and we are not able to document them by `godoc`. Real Teflon getters' names are identical to the field they are getting, without the 'Get' prefix, so look for them for information. Also note that the source is not polluted.

Please see the glossary (https://github.com/gradient-images/teflon/wiki/Glossary) for the exact definition of terms used throughout the source code.

Index

Constants

View Source 
const (
	ShowProtoDirName = "show_proto"
)

Variables

View Source 
var Done chan bool = make(chan bool)
View Source 
var Events chan Event = make(chan Event)
View Source 
var Objects = map[string]*TeflonObject{}

Objects associates teflon object pointers to absolute file-system paths.

View Source 
var Shows = []*TeflonObject{}

Shows are a list of all the show obejcts that the process knows about.

View Source 
var TeflonConf string

Full path to the configuration directory, which stores configuration and show prototypes for the system. The teflon command gets its value from the $TEFLONCONF environment variable.

Functions

func Exist 

func Exist(fspath string) bool

Returns true if a file-system object exists at a give path.

func IsDir 

func IsDir(fspath string) bool

Tells if a path is a dir or not.

func Isl 

func Isl(v interface{}) []interface{}

Convert interface{} to []interface{}

func Parse 

func Parse(filename string, b []byte, opts ...Option) (interface{}, error)

Parse parses the data from b using filename as information in the error messages.

func ParseFile 

func ParseFile(filename string, opts ...Option) (i interface{}, err error)

ParseFile parses the file identified by filename.

func ParseReader 

func ParseReader(filename string, r io.Reader, opts ...Option) (interface{}, error)

ParseReader parses the data from r using filename as information in the error messages.

func Path 

func Path(target string) (string, error)

Converts a target to a file-sytem absolute path.

func ShowAbs 

func ShowAbs(fspath string) string

Converts file-system absolute path to show-absolute notation. If the conversion can not be made, the function reurns the original string.

Types

type AbsPath 

type AbsPath struct {
	// contains filtered or unexported fields
}

func (*AbsPath) GenerateAll 

func (node *AbsPath) GenerateAll(fspSl []string) (res []string)

func (*AbsPath) NextMatch 

func (apn *AbsPath) NextMatch(o *TeflonObject) (res *TeflonObject)

func (*AbsPath) SetNext 

func (apn *AbsPath) SetNext(node *ONode)

type AddNode 

type AddNode struct {
	// contains filtered or unexported fields
}

Adds numbers numberically and concatenate strings

func (*AddNode) Eval 

func (a *AddNode) Eval(c *Context) (interface{}, error)

type AllMetaNode 

type AllMetaNode struct{}

AllMetaNode returns all metadata of an object

func (*AllMetaNode) Eval 

func (amn *AllMetaNode) Eval(c *Context) (interface{}, error)

type Cloner 

type Cloner interface {
	Clone() interface{}
}

Cloner is implemented by any value that has a Clone method, which returns a copy of the value. This is mainly used for types which are not passed by value (e.g map, slice, chan) or structs that contain such types.

This is used in conjunction with the global state feature to create proper copies of the state to allow the parser to properly restore the state in the case of backtracking.

type Context 

type Context struct {
	IMap map[string]interface{}
	Dir  *TeflonObject
}

String addressable version of the meta hierarchy.

type DivNode 

type DivNode struct {
	// contains filtered or unexported fields
}

Adds numbers numberically and concatenate strings

func (*DivNode) Eval 

func (a *DivNode) Eval(c *Context) (interface{}, error)

type ENode 

type ENode interface {
	Eval(*Context) (interface{}, error)
}

ENode is the building block of the AST. The meta selector and the object selector both implemeted as a chain of ENodes, only the evaluation is different. The meta selector part is evaluated bottom up as traditional C-like expressions do, while the object selector part is evaluated top down as traditional globbing does.

type Event 

type Event struct {
	Object *TeflonObject
	Type   EventType
	Result chan *TeflonObject
}

type EventType 

type EventType int

EventType is an enum describing the type of a Teflon event. 

const (
	PreNew EventType = iota
	PostNew
)

func (EventType) String 

func (i EventType) String() string

type ExactName 

type ExactName struct {
	// contains filtered or unexported fields
}

func (*ExactName) GenerateAll 

func (node *ExactName) GenerateAll(fspSl []string) (res []string)

func (*ExactName) NextMatch 

func (enn *ExactName) NextMatch(o *TeflonObject) (res *TeflonObject)

func (*ExactName) SetNext 

func (enn *ExactName) SetNext(node *ONode)

type Expr 

type Expr struct {
	MetaSelector   ENode
	ObjectSelector ONode
	// contains filtered or unexported fields
}

Expr is an object representing a Teflon expression.

func NewExpr 

func NewExpr(text string) (*Expr, error)

Creates a new expression object from a string

func (*Expr) Eval 

func (ex *Expr) Eval(c *Context) (res interface{}, err error)

Evaluation starts with the object selector, since it provides the context for the meta selection.

func (*Expr) Generate 

func (ex *Expr) Generate(c *Context) (res []string, err error)

Generation is the process of generating a []string from an object selector.

func (*Expr) String 

func (ex *Expr) String() string

type MetaNode 

type MetaNode struct {
	NameList []string
}

MetaNode represents a metadata identifier

func (*MetaNode) Eval 

func (M *MetaNode) Eval(c *Context) (interface{}, error)

type MulNode 

type MulNode struct {
	// contains filtered or unexported fields
}

Adds numbers numberically and concatenate strings

func (*MulNode) Eval 

func (a *MulNode) Eval(c *Context) (interface{}, error)

type MultiName 

type MultiName struct {
	// contains filtered or unexported fields
}

func (*MultiName) GenerateAll 

func (node *MultiName) GenerateAll(fspSl []string) (res []string)

func (*MultiName) NextMatch 

func (mnn *MultiName) NextMatch(o *TeflonObject) (res *TeflonObject)

func (*MultiName) SetNext 

func (mnn *MultiName) SetNext(node *ONode)

type NumberNode 

type NumberNode struct {
	Value float64
}

NumberNode represents a number literal

func (*NumberNode) Eval 

func (N *NumberNode) Eval(c *Context) (interface{}, error)

func (NumberNode) String 

func (N NumberNode) String() string

NumberNode needs to be Stringer for string concatenation

type ONode 

type ONode interface {
	// Match(string) bool
	NextMatch(*TeflonObject) *TeflonObject
	GenerateAll([]string) []string
	SetNext(*ONode)
}

ONode must be implemented by object selector nodes.

type Option 

type Option func(*parser) Option

Option is a function that can set an option on the parser. It returns the previous setting as an Option.

func AllowInvalidUTF8 

func AllowInvalidUTF8(b bool) Option

AllowInvalidUTF8 creates an Option to allow invalid UTF-8 bytes. Every invalid UTF-8 byte is treated as a utf8.RuneError (U+FFFD) by character class matchers and is matched by the any matcher. The returned matched value, c.text and c.offset are NOT affected.

The default is false.

func Debug 

func Debug(b bool) Option

Debug creates an Option to set the debug flag to b. When set to true, debugging information is printed to stdout while parsing.

The default is false.

func Entrypoint 

func Entrypoint(ruleName string) Option

Entrypoint creates an Option to set the rule name to use as entrypoint. The rule name must have been specified in the -alternate-entrypoints if generating the parser with the -optimize-grammar flag, otherwise it may have been optimized out. Passing an empty string sets the entrypoint to the first rule in the grammar.

The default is to start parsing at the first rule in the grammar.

func GlobalStore 

func GlobalStore(key string, value interface{}) Option

GlobalStore creates an Option to set a key to a certain value in the globalStore.

func InitState 

func InitState(key string, value interface{}) Option

InitState creates an Option to set a key to a certain value in the global "state" store.

func MaxExpressions 

func MaxExpressions(maxExprCnt uint64) Option

MaxExpressions creates an Option to stop parsing after the provided number of expressions have been parsed, if the value is 0 then the parser will parse for as many steps as needed (possibly an infinite number).

The default for maxExprCnt is 0.

func Memoize 

func Memoize(b bool) Option

Memoize creates an Option to set the memoize flag to b. When set to true, the parser will cache all results so each expression is evaluated only once. This guarantees linear parsing time even for pathological cases, at the expense of more memory and slower times for typical cases.

The default is false.

func Recover 

func Recover(b bool) Option

Recover creates an Option to set the recover flag to b. When set to true, this causes the parser to recover from panics and convert it to an error. Setting it to false can be useful while debugging to access the full stack trace.

The default is true.

func Statistics 

func Statistics(stats *Stats, choiceNoMatch string) Option

Statistics adds a user provided Stats struct to the parser to allow the user to process the results after the parsing has finished. Also the key for the "no match" counter is set.

Example usage: 

input := "input"
stats := Stats{}
_, err := Parse("input-file", []byte(input), Statistics(&stats, "no match"))
if err != nil {
    log.Panicln(err)
}
b, err := json.MarshalIndent(stats.ChoiceAltCnt, "", "  ")
if err != nil {
    log.Panicln(err)
}
fmt.Println(string(b))

type RelPath 

type RelPath struct {
	// contains filtered or unexported fields
}

func (*RelPath) GenerateAll 

func (node *RelPath) GenerateAll(fspSl []string) (res []string)

func (*RelPath) NextMatch 

func (rpn *RelPath) NextMatch(o *TeflonObject) (res *TeflonObject)

func (*RelPath) SetNext 

func (rpn *RelPath) SetNext(node *ONode)

type Stats 

type Stats struct {
	// ExprCnt counts the number of expressions processed during parsing
	// This value is compared to the maximum number of expressions allowed
	// (set by the MaxExpressions option).
	ExprCnt uint64

	// ChoiceAltCnt is used to count for each ordered choice expression,
	// which alternative is used how may times.
	// These numbers allow to optimize the order of the ordered choice expression
	// to increase the performance of the parser
	//
	// The outer key of ChoiceAltCnt is composed of the name of the rule as well
	// as the line and the column of the ordered choice.
	// The inner key of ChoiceAltCnt is the number (one-based) of the matching alternative.
	// For each alternative the number of matches are counted. If an ordered choice does not
	// match, a special counter is incremented. The name of this counter is set with
	// the parser option Statistics.
	// For an alternative to be included in ChoiceAltCnt, it has to match at least once.
	ChoiceAltCnt map[string]map[string]int
}

Stats stores some statistics, gathered during parsing

type StringNode 

type StringNode struct {
	Value string
}

StringNode represents a string literal

func (*StringNode) Eval 

func (S *StringNode) Eval(c *Context) (interface{}, error)

type SubNode 

type SubNode struct {
	// contains filtered or unexported fields
}

Adds numbers numberically and concatenate strings

func (*SubNode) Eval 

func (a *SubNode) Eval(c *Context) (interface{}, error)

type TeflonObject 

type TeflonObject struct {
	Path     string
	Show     *TeflonObject
	FileInfo meta.FileInfo
	Parent   *TeflonObject
	meta.PersistentMeta
}

TeflonObject is the main type of teflon. All Teflon objects are represented by this struct in RAM.

func NewTeflonObject 

func NewTeflonObject(target string) (*TeflonObject, error)

NewTeflonObject creates a new initialized Teflon object in memory, that represents a file-system object.

Initialization is always complete. It is not allowed to have half-baked objects in memory. Since it has to set the Show field to its correct value it first has to find the show root of the target. This is done recursively by creating all the parent objects until and including the show root. This means that not only the created object is fully initialized but there will be a complete chain of objects leading from the target to the show root.

If the target is show-absolute then the system first has to find the show root from the current directory to get the file-system path of the target. This means that the end result is two initialized chains to the same show root.

func (*TeflonObject) Children 

func (o *TeflonObject) Children() (ch []*TeflonObject)

Children() returns the child objects of the object.

func (*TeflonObject) ChildrenNames 

func (o *TeflonObject) ChildrenNames() (ch []string)

ChildrenNames() returns a slice of strings with the names of the children of the object.

func (*TeflonObject) CreateObject 

func (o *TeflonObject) CreateObject(exs string, file bool) (oSl []*TeflonObject, err error)

CreateObject() creates a new FS object and triggers a new event.

func (*TeflonObject) CreateShow 

func (o *TeflonObject) CreateShow(exs string, protoName string) (oSl []*TeflonObject, err error)

CreateShow() creates new Teflon show.

func (*TeflonObject) DelMeta 

func (o *TeflonObject) DelMeta(key string)

Deletes an entry from the user section of the metadata.

func (*TeflonObject) Get 

func (o *TeflonObject) Get(exs string) (res interface{}, err error)

Get() evaluates a Teflon expression and returns the result.

func (*TeflonObject) GetPath 

func (o *TeflonObject) GetPath() string

Helper function to get empty string for nil objects during JSON marshaling.

func (*TeflonObject) IMap 

func (o *TeflonObject) IMap() map[string]interface{}

GetContext returns the Context of the object. Currently it is implemented as a marshal and unmarshal sequence to and from JSON. It is rather primitive and uneffective (slow), but it has the benefits of being extremely simple to implement and it forces us to be compliant with the JSON standard.

func (TeflonObject) MarshalJSON 

func (o TeflonObject) MarshalJSON() ([]byte, error)

Marshaling JSON manually to avoid recursion. There is probably a more elegant way of doing this.

func (*TeflonObject) MetaFile 

func (o *TeflonObject) MetaFile() string

MetaFile returns the file path to the TeflonObject's meta file. In the case of a file it is: 

$DIR/.teflon/$FILE._

In the case of a directory it is: 

$DIR/.teflon/_

func (*TeflonObject) SetContractPattern 

func (o *TeflonObject) SetContractPattern(exs string, pat string) (oSl []*TeflonObject, err error)

func (*TeflonObject) SetMeta 

func (o *TeflonObject) SetMeta(key, value string)

Sets an entry in the user section of the metadata.

func (*TeflonObject) SyncMeta 

func (o *TeflonObject) SyncMeta() error

SincMeta() writes metadata to disk.

Source Files

View all Source files

Directories

Path Synopsis
internal
meta
The meta package defines the metadata format.
 The meta package defines the metadata format.
commands

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.