schematree

README

Schematree Module

The Schematree Module contains the main datastructure and the standart recommendation algorithm.

SchemaTree.go, SchemaNode.go and DataTypes.go form the tree data structure. SubjectReader.go reads property / type informations from a rdf file. Recommendation.go performs the recommendations.

Interface

Create(filename string, firstNsubjects uint64, typed bool, minSup uint32) creates a new Schematree from a rdf file Load(filePath string) loads a schematree from a encoded file

Recommend(properties []string, types []string) recommends a list of property candidates

Documentation

Index

• func PrintMemUsage()
• func Serve(schema *SchemaTree, port int)
• func SubjectSummaryReader(fileName string, pMap propMap, handler func(s *SubjectSummary), firstN uint64, ...) (subjectCount uint64)
• func UniversalReader(fileName string) (reader io.ReadCloser, err error)
• type IItem
  • func (p *IItem) IsProp() bool
  • func (p *IItem) IsType() bool
  • func (p IItem) String() string
• type IList
  • func (l IList) Sort()
  • func (l IList) String() string
• type PropertyRecommendations
  • func (ps PropertyRecommendations) String() string
  • func (ps PropertyRecommendations) Top10AvgProbibility() float32
• type RankedPropertyCandidate
• type SchemaNode
• type SchemaTree
  • func Create(filename string, firstNsubjects uint64, typed bool, minSup uint32) (*SchemaTree, error)
  • func Load(filePath string) (*SchemaTree, error)
  • func New(typed bool, minSup uint32) (tree *SchemaTree)
  • func (tree *SchemaTree) BuildPropertyList(properties []string, types []string) IList
  • func (tree *SchemaTree) Insert(e *SubjectSummary)
  • func (tree *SchemaTree) Recommend(properties []string, types []string) PropertyRecommendations
  • func (tree *SchemaTree) RecommendPropertiesAndTypes(properties IList) (ranked PropertyRecommendations)
  • func (tree *SchemaTree) RecommendProperty(properties IList) (ranked PropertyRecommendations)
  • func (tree *SchemaTree) Save(filePath string) error
  • func (tree SchemaTree) String() string
  • func (tree *SchemaTree) Support(properties IList) uint32
  • func (tree *SchemaTree) TwoPass(fileName string, firstN uint64)
  • func (tree SchemaTree) WritePropFreqs(file string)
  • func (tree SchemaTree) WriteTypeFreqs(file string)
• type SubjectSummary
  • func (subj *SubjectSummary) String() string

Functions

func PrintMemUsage

func PrintMemUsage()

PrintMemUsage outputs the current, total and OS memory being used. As well as the number of garage collection cycles completed.

func Serve

func Serve(schema *SchemaTree, port int)

Serve provides a REST API for the given schematree on the given port

func SubjectSummaryReader

func SubjectSummaryReader(
	fileName string,
	pMap propMap,
	handler func(s *SubjectSummary),
	firstN uint64,
	willConvertTypes bool,
) (subjectCount uint64)

SubjectSummaryReader reads a RDF Dataset from disk (in N-Triples format) which is expected to be grouped by subjects. For each subject group, the method will build a SubjectSummary structure and send it to a handler function. It will always detect types, but may choose to ignore them.

todo: The parsing is done a subset of N-Triple format files. If subjects, predicates or objects contain

any spaces, even if inside quotes, it will break.

func UniversalReader

func UniversalReader(fileName string) (reader io.ReadCloser, err error)

UniversalReader opens a file for read mode. It is able to automatically decompress files that use either GZ, BGZ or BZ2. It will also display a loading bar on stdout.

Types

type IItem

type IItem struct {
	Str        *string
	TotalCount uint64
	SortOrder  uint32
	// contains filtered or unexported fields
}

A struct capturing - a string IRI (str) and - its support, i.e. its total number of occurrences (totalCount) - an integer indicating sort order

func (*IItem) IsProp

func (p *IItem) IsProp() bool

func (*IItem) IsType

func (p *IItem) IsType() bool

func (IItem) String

func (p IItem) String() string

type IList

type IList []*IItem

An array of pointers to IRI structs

func (IList) Sort

func (l IList) Sort()

Sort the list according to the current iList Sort order

func (IList) String

func (l IList) String() string

type PropertyRecommendations

type PropertyRecommendations []RankedPropertyCandidate

PropertyRecommendations is a list of RankedPropertyCandidates

func (PropertyRecommendations) String

func (ps PropertyRecommendations) String() string

String returns the string representation of property candidates

func (PropertyRecommendations) Top10AvgProbibility

func (ps PropertyRecommendations) Top10AvgProbibility() float32

Top10AvgProbibility calculates average of probability of the top ten recommendations if less than 10 recommendations, then missing values have probability 0

type RankedPropertyCandidate

type RankedPropertyCandidate struct {
	Property    *IItem
	Probability float64
}

RankedPropertyCandidate is a struct to rank suggestions

type SchemaNode

type SchemaNode struct {
	ID *IItem

	Children []*SchemaNode

	Support uint32 // total frequency of the node in the path
	// contains filtered or unexported fields
}

SchemaNode is a nodes of the Schema FP-Tree

type SchemaTree

type SchemaTree struct {
	PropMap propMap    // PropMap maps the string representations of properties to the corresponding IItem
	Root    SchemaNode // Root is the root node of the schematree. All further nodes are descendants of this node.
	MinSup  uint32     // TODO (not used)
	Typed   bool       // Typed indicates if this schematree includes type information as properties
}

TypedSchemaTree is a schematree that includes type information as property nodes

func Create

func Create(filename string, firstNsubjects uint64, typed bool, minSup uint32) (*SchemaTree, error)

Create creates a new schema tree from given dataset with given first n subjects, typed and minSup

func Load

func Load(filePath string) (*SchemaTree, error)

Load loads a binarized SchemaTree from disk

func New

func New(typed bool, minSup uint32) (tree *SchemaTree)

New returns a newly allocated and initialized schema tree

func (*SchemaTree) BuildPropertyList

func (tree *SchemaTree) BuildPropertyList(properties []string, types []string) IList

BuildPropertyList receives prop and type strings, and builds a list of IItem from it that can later be used to execute the recommender.

func (*SchemaTree) Insert

func (tree *SchemaTree) Insert(e *SubjectSummary)

Insert inserts all properties of a new subject into the schematree The subject is given by thread-safe

func (*SchemaTree) Recommend

func (tree *SchemaTree) Recommend(properties []string, types []string) PropertyRecommendations

Recommend recommends a ranked list of property candidates by given strings Note: This method should be used in the future where assessments have no access to IItem.

func (*SchemaTree) RecommendPropertiesAndTypes

func (tree *SchemaTree) RecommendPropertiesAndTypes(properties IList) (ranked PropertyRecommendations)

RecommendPropertiesAndTypes recommends a ranked list of property and type candidates by given IItems

func (*SchemaTree) RecommendProperty

func (tree *SchemaTree) RecommendProperty(properties IList) (ranked PropertyRecommendations)

RecommendProperty recommends a ranked list of property candidates by given IItems

func (*SchemaTree) Save

func (tree *SchemaTree) Save(filePath string) error

Save stores a binarized version of the schematree to the given filepath

func (SchemaTree) String

func (tree SchemaTree) String() string

String returns the string represantation of the schema tree

func (*SchemaTree) Support

func (tree *SchemaTree) Support(properties IList) uint32

Support returns the total cooccurrence-frequency of the given property list

func (*SchemaTree) TwoPass

func (tree *SchemaTree) TwoPass(fileName string, firstN uint64)

TwoPass constructs a SchemaTree from the firstN subjects of the given NTriples file using a two-pass approach

func (SchemaTree) WritePropFreqs

func (tree SchemaTree) WritePropFreqs(file string)

WritePropFreqs writes all Properties together with their Support to the given File as CSV

func (SchemaTree) WriteTypeFreqs

func (tree SchemaTree) WriteTypeFreqs(file string)

WriteTypeFreqs writes all Types together with their Support to the given File as CSV

type SubjectSummary

type SubjectSummary struct {
	Properties        map[*IItem]uint32
	Str               string // @TODO: Temporarily added the subject names for easier evaluation debugging
	NumPredicates     int
	NumTypePredicates int
}

All type annotations (types) and properties (properties) for a fixed subject equivalent of a transaction in frequent pattern mining

func (*SubjectSummary) String

func (subj *SubjectSummary) String() string