Documentation
¶
Overview ¶
Package ogdl is used to process OGDL, the Ordered Graph Data Language.
OGDL is a textual format to write trees or graphs of text, where indentation and spaces define the structure. Here is an example:
network ip 192.168.1.100 gw 192.168.1.9
The languange is simple, either in its textual representation or its number of productions (the specification rules), allowing for compact implementations.
OGDL character streams are normally formed by Unicode characters, and encoded as UTF-8 strings, but any encoding that is ASCII transparent is compatible with the specification and the implementations.
See the full spec at http://ogdl.org.
Installation ¶
To install this package just do:
go get github.com/rveen/ogdl
An example ¶
If we have a text file 'config.g' containing:
eth0
ip
192.168.1.1
gateway
192.168.1.10
mask
255.255.255.0
timeout
20
then,
g := ogdl.ParseFile("config.g")
ip,_ := g.GetString("eth0.ip")
to,_ := g.GetInt64("eth0.timeout")
println("ip:",ip,", timeout:",to)
will print
ip: 192.168.1.1, timeout: 20
The configuration file can be written in a conciser way:
eth0 ip 192.168.1.1 gateway 192.168.1.10 mask 255.255.255.0 timeout 20
A template example ¶
The package includes a template processor. It takes an arbitrary input stream with some variables in it, and produces an output stream with the variables resolved out of a Graph object which acts as context.
For example (given the previous config file):
g := ogdl.ParseFile("config.g")
t := ogdl.NewTemplate("The gateway's IP is $eth0.gateway")
b := t.Process(g)
string(b) is then:
The gateway's IP is 192.168.1.10
Index ¶
- Constants
- func FunctionAdd(s string, f func(*Graph, *Graph, int) []byte)
- func FunctionAddConstructor(s string, f func() interface{})
- func IsBreakChar(c int) bool
- func IsDigit(c int) bool
- func IsEndChar(c int) bool
- func IsInteger(s string) bool
- func IsLetter(c int) bool
- func IsOperatorChar(c int) bool
- func IsSpaceChar(c int) bool
- func IsTemplateTextChar(c int) bool
- func IsTextChar(c int) bool
- func IsTokenChar(c int) bool
- type BinParser
- type EventHandler
- func (e *EventHandler) Add(s string) bool
- func (e *EventHandler) AddAt(s string, l int)
- func (e *EventHandler) AddBytes(b []byte) bool
- func (e *EventHandler) AddBytesAt(b []byte, l int)
- func (e *EventHandler) Dec()
- func (e *EventHandler) Delete()
- func (e *EventHandler) Graph() *Graph
- func (e *EventHandler) GraphTop(s string) *Graph
- func (e *EventHandler) Inc()
- func (e *EventHandler) Level() int
- func (e *EventHandler) SetLevel(l int)
- type Graph
- func (g *Graph) Add(n interface{}) *Graph
- func (g *Graph) AddNodes(g2 *Graph) *Graph
- func (g *Graph) Ast()
- func (g *Graph) Binary() []byte
- func (g *Graph) Bool() (bool, bool)
- func (g *Graph) Bytes() []byte
- func (schema *Graph) Check(g *Graph) (bool, string)
- func (g *Graph) Copy(c *Graph)
- func (g *Graph) Delete(n interface{})
- func (g *Graph) DeleteAt(i int)
- func (g *Graph) Depth() int
- func (g *Graph) Equal(c *Graph) bool
- func (g *Graph) Eval(e *Graph) interface{}
- func (g *Graph) EvalBool(e *Graph) bool
- func (g *Graph) EvalExpression(p *Graph) interface{}
- func (g *Graph) EvalPath(p *Graph) interface{}
- func (g *Graph) Float64() (float64, bool)
- func (g *Graph) Function(p *Graph, ix int, context *Graph) (interface{}, error)
- func (g *Graph) Function2(p *Graph, ix int, context *Graph) (interface{}, error)
- func (g *Graph) Get(s string) *Graph
- func (g *Graph) GetAt(i int) *Graph
- func (g *Graph) GetBool(path string) (bool, error)
- func (g *Graph) GetBytes(path string) ([]byte, error)
- func (g *Graph) GetFloat64(path string) (float64, error)
- func (g *Graph) GetInt64(path string) (int64, error)
- func (g *Graph) GetSimilar(re string) (*Graph, error)
- func (g *Graph) GetString(path string) (string, error)
- func (g *Graph) Int64() (int64, bool)
- func (g *Graph) IsNil() bool
- func (g *Graph) Len() int
- func (g *Graph) Node(s string) *Graph
- func (g *Graph) Number() interface{}
- func (t *Graph) Process(c *Graph) []byte
- func (g *Graph) Scalar() interface{}
- func (g *Graph) Send(cfg *Graph) (*Graph, error)
- func (g *Graph) Set(s string, val interface{}) *Graph
- func (g *Graph) String() string
- func (g *Graph) Substitute(s string, v interface{})
- func (g *Graph) Text() string
- func (g *Graph) Type() string
- func (g *Graph) Value() reflect.Value
- type Log
- type Parser
- func (p *Parser) ArgList() bool
- func (p *Parser) Args() (bool, error)
- func (p *Parser) Block() (string, bool)
- func (p *Parser) Break() bool
- func (p *Parser) Comment() bool
- func (p *Parser) Dec()
- func (p *Parser) Emit(s string)
- func (p *Parser) EmitBytes(b []byte)
- func (p *Parser) End() bool
- func (p *Parser) Expression() bool
- func (p *Parser) Graph() *Graph
- func (p *Parser) GraphTop(s string) *Graph
- func (p *Parser) Group() (bool, error)
- func (p *Parser) Inc()
- func (p *Parser) Index() bool
- func (p *Parser) Line() (bool, error)
- func (p *Parser) Newline() bool
- func (p *Parser) NextByteIs(c int) bool
- func (p *Parser) Number() (string, bool)
- func (p *Parser) Ogdl() error
- func (p *Parser) Operator() (string, bool)
- func (p *Parser) Path() bool
- func (p *Parser) Quoted() (string, bool)
- func (p *Parser) Read() int
- func (p *Parser) Scalar() (string, bool)
- func (p *Parser) Selector() bool
- func (p *Parser) Sequence() (bool, bool, error)
- func (p *Parser) Space() (bool, int)
- func (p *Parser) String() (string, bool)
- func (p *Parser) Template()
- func (p *Parser) Text() bool
- func (p *Parser) Token() (string, bool)
- func (p *Parser) TokenList()
- func (p *Parser) UnaryExpression() bool
- func (p *Parser) Unread()
- func (p *Parser) Variable() bool
- func (p *Parser) WhiteSpace() bool
- type RFunction
Examples ¶
Constants ¶
const ( TypeExpression = "!e" TypePath = "!p" TypeVariable = "!v" TypeSelector = "!s" TypeIndex = "!i" TypeGroup = "!g" TypeTemplate = "!t" TypeIf = "!if" TypeEnd = "!end" TypeElse = "!else" TypeFor = "!for" TypeBreak = "!break" )
Nodes containing these strings are special
Variables ¶
This section is empty.
Functions ¶
func FunctionAdd ¶
FunctionAdd adds a function to the context.
func FunctionAddConstructor ¶
func FunctionAddConstructor(s string, f func() interface{})
FunctionAddConstructor adds a factory kind of function to the context.
func IsBreakChar ¶
IsBreakChar returns true for 10 and 13 (newline and carriage return)
func IsEndChar ¶
IsEndChar returns true for all integers < 32 that are not newline, carriage return or tab.
func IsInteger ¶
IsInteger returns true for strings containing exclusively digits, with an optional minus sign at the beginning. Starting and trailing spaces are allowed.
func IsOperatorChar ¶
IsOperatorChar returns true for all operator characters used in OGDL expressions (those parsed by NewExpression).
func IsTemplateTextChar ¶
IsTemplateTextChar returns true for all not END chars and not $
func IsTextChar ¶
IsTextChar returns true for all integers > 32 and are not OGDL separators (parenthesis and comma)
func IsTokenChar ¶
IsTokenChar returns true for letters, digits and _ (as per Unicode).
Types ¶
type BinParser ¶
type BinParser struct {
// contains filtered or unexported fields
}
BinParser and its methods implement a parser for binary OGDL, as defined in the OGDL Binary specification, v1.0, available at ogdl.org, reproduced below.
ogdl-binary ::= header ( level node )* 0x00 header ::= 0x01 'G' 0x00 level ::= varInt node ::= text-node | binary-node text-node ::= text 0x00 binary-node ::= 0x01 ( length data )* 0x00 length ::= multibyte-integer data :: byte[length]
func NewBinParser ¶
NewBinParser creates a parser that can convert a binary OGDL stream into an ogdl.Graph object. To actually parse the stream, the method Parse() has to be invoked.
func NewBytesBinParser ¶
NewBytesBinParser creates a parser that can convert a binary OGDL byte stream into an ogdl.Graph object. To actually parse the stream, the method Parse() has to be invoked.
func NewFileBinParser ¶
NewFileBinParser creates a parser that can convert a binary OGDL file into an ogdl.Graph object. To actually parse the stream, the method Parse() has to be invoked.
type EventHandler ¶
type EventHandler struct {
// contains filtered or unexported fields
}
EventHandler receives events and produces a Graph.
func NewEventHandler ¶
func NewEventHandler() EventHandler
NewEventHandler creates an event handler that produces a Graph object from the events received.
func (*EventHandler) Add ¶
func (e *EventHandler) Add(s string) bool
Add creates a node at the current level.
Only one error is possible: an empty graph where we should be writing the event. It that case, false is returned.
func (*EventHandler) AddAt ¶
func (e *EventHandler) AddAt(s string, l int)
AddAt creates a node at the specified level
func (*EventHandler) AddBytes ¶
func (e *EventHandler) AddBytes(b []byte) bool
AddBytes creates a node at the current level, with the given byte array as content.
func (*EventHandler) AddBytesAt ¶
func (e *EventHandler) AddBytesAt(b []byte, l int)
AddBytesAt creates a node at the specified level, with the byte slice as content.
func (*EventHandler) Graph ¶
func (e *EventHandler) Graph() *Graph
Graph returns the Graph object built from the events sent to this event handler.
func (*EventHandler) GraphTop ¶
func (e *EventHandler) GraphTop(s string) *Graph
GraphTop returns the Graph object built from the events sent to this event handler, and sets the root node to the string given.
func (*EventHandler) SetLevel ¶
func (e *EventHandler) SetLevel(l int)
SetLevel sets the current level
type Graph ¶
type Graph struct {
This interface{}
Out []*Graph
}
Graph is a node with outgoing pointers to other Graph objects. It is implemented as a named list.
func NewExpression ¶
NewExpression parses an expression in text format (given in the string) to a Graph, in the form of a suitable syntax tree.
expression := expr1 (op2 expr1)*
expr1 := path | constant | op1 path | op1 constant | '(' expr ')' | op1 '(' expr ')'
constant ::= quoted | number
Example ¶
e := NewExpression("1-2+3")
g := NilGraph()
i := g.Eval(e)
fmt.Println(i)
Output: 2
func NewGraph ¶
func NewGraph(n interface{}) *Graph
NewGraph creates a Graph instance with the given name. At this stage it is a single node without outgoing edges.
func NewPath ¶
NewPath takes an Unicode string representing an OGDL path, parses it and returns it as a Graph object.
It also parses extended paths, as those used in templates, which may have argument lists.
func NewTemplate ¶
NewTemplate parses a text template given as a string and converts it to a Graph. Templates have fixed and variable parts. Variables all begin with '$'.
A template is a text file in any format: plain text, HTML, XML, OGDL or whatever. The dolar sign acts as an escape character that switches from the text to the variable plane. Parsed templates are converted back to text by evaluating the variable parts against a Graph object, by means of the Process() method.
Template grammar
template ::= ( text | variable )*
variable ::= ('$' path) | ('$' '(' expression ')') | ('$' '{' expression '}')
path ::= as defined in path.go
expression ::= as defined in expression.go
Some variables act as directives: $if, $else, $end, $for, $break.
$if(expression) $else $end $for(destPath,sourcepath) $break $end
Example ¶
p := NewTemplate("Hello, $user")
g := NilGraph()
g.Add("user").Add("Jenny")
fmt.Println(string(p.Process(g)))
Output: Hello, Jenny
func NilGraph ¶
func NilGraph() *Graph
NilGraph returns a pointer to a 'null' Graph, also called transparent node.
func ParseString ¶
ParseString parses OGDL text from the given string. It returns a *Graph
func (*Graph) Add ¶
Add adds a subnode to the current node.
An eventual nil root will not be bypassed.
func (*Graph) Ast ¶
func (g *Graph) Ast()
Ast reorganizes the expression graph in the form of an abstract syntax tree.
func (*Graph) Check ¶
Check returns true if the Graph given as a parameter conforms to the schema represented by the receiver Graph.
Example ¶
schema := ParseString("a !int, b !string, c !float, d !bool")
g := ParseString("a 1, b s, c 1.0, d true")
b, message := schema.Check(g)
fmt.Println(b, message)
Output: true
func (*Graph) Copy ¶
Copy adds a copy of the graph given to the current graph.
Warning (from the Go faq): Copying an interface value makes a copy of the thing stored in the interface value. If the interface value holds a struct, copying the interface value makes a copy of the struct. If the interface value holds a pointer, copying the interface value makes a copy of the pointer, but not the data it points to.
func (*Graph) Delete ¶
func (g *Graph) Delete(n interface{})
Delete removes all subnodes with the given value or content
func (*Graph) Depth ¶
Depth returns the depth of the graph if it is a tree, or -1 if it has cycles.
TODO: Cycles are inferred if level>100, but nodes traversed are not remembered (they should if cycles need to be detected).
func (*Graph) Eval ¶
Eval takes a parsed expression and evaluates it in the context of the current graph.
Example ¶
g := NilGraph()
g.Add("a").Add(4)
g.Add("b").Add("4")
e := NewExpression("a+3")
e2 := NewExpression("b+3")
fmt.Println(g.Eval(e))
fmt.Println(g.Eval(e2))
Output: 7 43
func (*Graph) EvalBool ¶
EvalBool takes a parsed expression and evaluates it in the context of the current graph, and converts the result to a boolean.
func (*Graph) EvalExpression ¶
EvalExpression evaluates expressions (!e) g can have native types (other things than strings), but p only []byte or string
func (*Graph) EvalPath ¶
EvalPath traverses g following a path p. The path needs to be previously converted to a Graph with NewPath().
This function is similar to ogdl.Get, but for complexer paths. Code could be shared.
func (*Graph) Function ¶
Function enables calling Go functions from templates. Path in templates are translated into Go functions if !type definitions are present.
Functions and type methods are handled here, based on the two maps, factory[] and functions[].
Also remote functions are called from here. A remote function is a call to a TCP/IP server, in which both the request and the response are binary encoded OGDL objects.
(This code can be much improved)
func (*Graph) Get ¶
Get recurses a Graph following the given path and returns the result.
This function returns consequently a *Graph. It may be a pointer within the recursed Graph (the receiver), or a newly created one. We leave the handling of specific types to the functions defined in get_types.go.
OGDL Path: elements are separated by '.' or [] or {} index := [N] selector := {N} tokens can be quoted
Future: .*., .**. ./regex/.
Nil receiver behavior: return nil.
Example ¶
g := ParseString("a (b 1, c 2, b 3)")
fmt.Println(g.Get("a.b{0}").Text())
fmt.Println(g.Get("a.b{1}").Text())
fmt.Println("---")
fmt.Println(g.Get("a.b{}").Text())
fmt.Println("---")
fmt.Println(g.Get("a[0]").Text())
Output: 1 3 --- 1 3 --- b 1
func (*Graph) GetBool ¶
GetBool returns the result of applying a path to the given Graph. The result is returned as a bool. If the path result cannot be converted to a boolean, then an error is returned.
func (*Graph) GetBytes ¶
GetBytes returns the result of applying a path to the given Graph. The result is returned as a byte slice.
func (*Graph) GetFloat64 ¶
GetFloat64 returns the result of applying a path to the given Graph. The result is returned as a float64. If the path result cannot be converted to a float, then an error is returned.
func (*Graph) GetInt64 ¶
GetInt64 returns the result of applying a path to the given Graph. The result is returned as an int64. If the path result cannot be converted to an integer, then an error is returned.
func (*Graph) GetSimilar ¶
GetSimilar returns a Graph with all subnodes found that match the regular expression given. It only walks through the subnodes of the current Graph. If the regex doesn't compile, an error will be returned. If the result set is empty, both return values are nil (no error is signaled).
func (*Graph) GetString ¶
GetString returns the result of applying a path to the given Graph. The result is returned as a string.
func (*Graph) Node ¶
Node returns the first subnode whose string value is equal to the given string. It returns nil if not found.
func (*Graph) Number ¶
func (g *Graph) Number() interface{}
Number returns either a float64, int64 or nil
func (*Graph) Process ¶
Process processes the parsed template, returning the resulting text in a byte array. The variable parts are resolved out of the Graph given.
func (*Graph) Scalar ¶
func (g *Graph) Scalar() interface{}
Scalar returns the current node content, reducing the number of types following these rules:
uint* -> int64 int* -> int64 float* -> float64 byte -> int64 rune -> int64 bool -> bool string, []byte: if it represents an int or float or bool, convert to int64, float64 or bool
Any other type is returned as is.
func (*Graph) Send ¶
Send opens a connection to a remote server and makes a remote call. It sends the given Graph in binary format to the server and returns the response Graph. The connection is closed before leaving this method.
func (*Graph) Set ¶
Set sets the first occurrence of the given path to the value given.
TODO: Support indexes
Example ¶
g := ParseString("a b c")
g.Set("a.b", "d")
fmt.Println(g.Text())
Output: a b d
Example (A) ¶
g := NilGraph()
g.Add("R").Add("b")
r := g.Node("R")
r.Set("id", "1")
fmt.Println(g.Text())
Output: R b id 1
func (*Graph) Substitute ¶
Substitute traverses the graph substituting all nodes with content equal to s by v.
func (*Graph) Text ¶
Text is the OGDL text emitter. It converts a Graph into OGDL text.
Strings are quoted if they contain spaces, newlines or special characters. Null elements are not printed, and act as transparent nodes.
BUG():Handle comments correctly.
type Log ¶
type Log struct {
// contains filtered or unexported fields
}
Log is a log store for binary OGDL objects.
All objects are appended to a file, and a position is returned.
func (*Log) Add ¶
Add adds an OGDL object to the log. The starting position into the log is returned.
func (*Log) AddBinary ¶
AddBinary adds an OGDL binary object to the log. The starting position into the log is returned.
func (*Log) Get ¶
Get returns the OGDL object at the position given and the position of the next object, or an error.
type Parser ¶
type Parser struct {
// contains filtered or unexported fields
}
Parser is used to parse textual OGDL streams, paths, empressions and templates into Graph objects.
Simple productions return a scalar (normally a string), more complex ones write to and event handler.
BUG(): Level 2 (graphs) not implemented.
func NewBytesParser ¶
NewBytesParser creates an OGDL parser from a []byte source
func NewFileParser ¶
NewFileParser creates an OGDL parser that reads from a file
func NewStringParser ¶
NewStringParser creates an OGDL parser from a string
func (*Parser) ArgList ¶
ArgList ::= space? expression space? [, space? expression]* space?
arglist < stream > events
arglist can be empty, then returning false (this fact is not represented in the BNF definition).
func (*Parser) Comment ¶
Comment consumes anything from # up to the end of the line.
BUG(): Special cases: #?, #{
func (*Parser) Graph ¶
Graph returns the *Graph object associated with this parser (where root where the OGDL tree is build on).
func (*Parser) GraphTop ¶
GraphTop returns the *Graph object associated with this parser (where root where the OGDL tree is build on). Additionally, the name of the root node is set to the given string.
func (*Parser) Line ¶
Line processes an OGDL line or a multiline scalar.
- A Line is composed of scalars and groups. - A Scalar is a Quoted or a String. - A Group is a sequence of Scalars enclosed in parenthesis - Scalars can be separated by commas or by space - The last element of a line can be a Comment, or a Block
The indentation of the line and the Scalar sequences and Groups on it define the tree structure characteristic of OGDL level 1.
Line ::= Space(n) Sequence? ((Comment? Break)|Block)?
Anything other than one Scalar before a Block should be an syntax error. Anything after a closing ')' that is not a comment is a syntax error, thus only one Group per line is allowed. That is because it would be difficult to define the origin of the edges pointing to what comes after a Group.
Indentation rules:
a -> level 0
b -> level 1
c -> level 1
d -> level 2
e -> level 2
f -> level 1
func (*Parser) NextByteIs ¶
NextByteIs tests if the next character in the stream is the one given as parameter, in which case it is consumed.
func (*Parser) Number ¶
Number returns true if it finds a number at the current parser position It returns also the number found.
func (*Parser) Ogdl ¶
Ogdl is the main function for parsing OGDL text.
An OGDL stream is a sequence of lines (a block of text or a quoted string can span multiple lines but is still parsed by Line())
Graph ::= Line* End
func (*Parser) Operator ¶
Operator returns true if it finds an operator at the current parser position It returns also the operator found.
func (*Parser) Path ¶
Path parses an OGDL path, or an extended path as used in templates.
path ::= element ('.' element)*
element ::= token | integer | quoted | group | index | selector
(Dot optional before Group, Index, Selector)
group := '(' Expression [[,] Expression]* ')'
index := '[' Expression ']'
selector := '{' Expression '}'
The OGDL parser doesn't need to know about Unicode. The character classification relies on values < 127, thus in the ASCII range, which is also part of Unicode.
Note: On the other hand it would be stupid not to recognize for example Unicode quotation marks if we know that we have UTF-8. But when do we know for sure?
func (*Parser) Sequence ¶
Sequence ::= (Scalar|Group) (Space? (Comma? Space?) (Scalar|Group))*
[!] with the requirement that after a group a comma is required if there are more elements. Examples: a b c a b,c a(b,c) (a b,c) (b,c),(d,e) <-- This can be handled a (b c) d <-- This is an error This method returns two booleans: if there has been a sequence, and if the last element was a Group
func (*Parser) Space ¶
Space is (0x20|0x09)+. It returns a boolean indicating if space has been found, and an integer indicating how many spaces, iff uniform (either all 0x20 or 0x09)
func (*Parser) String ¶
String is a concatenation of characters that are > 0x20 and are not '(', ')', ',', and do not begin with '#'.
NOTE: '#' is allowed inside a string. For '#' to start a comment it must be preceeded by break or space, or come after a closing ')'.
TOTHINK: Many productions return a string and not []byte, which could be more efficient, but has no type information: []byte can be a raw binary or a string.
func (*Parser) Token ¶
Token reads from the Parser input stream and returns a token or nil. A token is defined as a sequence of letters and/or numbers and/or _.
Examples of tokens:
_a 1 143lasd034
func (*Parser) UnaryExpression ¶
UnaryExpression := cpath | constant | op1 cpath | op1 constant | '(' expr ')' | op1 '(' expr ')'
func (*Parser) Unread ¶
func (p *Parser) Unread()
Unread puts the last readed character back into the stream. Up to two consecutive Unread()'s can be issued.
BUG: line-- if newline
func (*Parser) WhiteSpace ¶
WhiteSpace is equivalent to Space | Break. It consumes all white space, whether spaces, tabs or newlines
type RFunction ¶
type RFunction struct {
// contains filtered or unexported fields
}
RFunction represents a remote function (also known as a remote procedure call).
func NewRFunction ¶
NewRFunction opens a connection to a TCP/IP server specified in the Graph supplied. It also makes an initialization call, if the Graph has an 'init' section.
func (*RFunction) Call ¶
Call makes a remote call. It sends the given Graph in binary format to the server and returns the response Graph.
func (*RFunction) CallBinary ¶
CallBinary makes a remote call. It sends the given Graph in binary format to the server and returns the response Graph.
TODO: Return []byte