native

Documentation

Overview

native package is used to parse AST using an external binary.

Index

• Variables
• type Client
  • func ExecClient(path string, args ...string) (*Client, error)
  • func (c *Client) Close() error
  • func (c *Client) ParseNative(req *ParseNativeRequest) (*ParseNativeResponse, error)
• type FillType
• type ObjectToNoder
  • func (c *ObjectToNoder) ToNode(v interface{}) (*uast.Node, error)
• type ParseNativeRequest
• type ParseNativeResponse
• type Parser
  • func ExecParser(toNoder ToNoder, path string, args ...string) (*Parser, error)
  • func (p *Parser) Close() error
  • func (p *Parser) Parse(req *protocol.ParseRequest) *protocol.ParseResponse
• type ToNoder

Variables

var (
	ErrEmptyAST             = errors.NewKind("input AST was empty")
	ErrTwoTokensSameNode    = errors.NewKind("token was already set (%s != %s)")
	ErrTwoTypesSameNode     = errors.NewKind("internal type was already set (%s != %s)")
	ErrUnexpectedObject     = errors.NewKind("expected object of type %s, got: %#v")
	ErrUnexpectedObjectSize = errors.NewKind("expected object of size %d, got %d")
	ErrUnsupported          = errors.NewKind("unsupported: %s")
)

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is a wrapper of the native command.

func ExecClient

func ExecClient(path string, args ...string) (*Client, error)

ExecNative executes the given command and returns a *Client for it.

func (*Client) Close

func (c *Client) Close() error

Close closes the native client.

func (*Client) ParseNative

func (c *Client) ParseNative(req *ParseNativeRequest) (*ParseNativeResponse, error)

ParseNative sends a request to the native client and returns its response.

type FillType

type FillType int

const (
	None FillType = iota
	OffsetFromLineCol
	LineColFromOffset
)

type ObjectToNoder

type ObjectToNoder struct {
	// InternalTypeKey is the name of the key that the native AST uses
	// to differentiate the type of the AST nodes. This internal key will then be
	// checkable in the AnnotationRules with the `HasInternalType` predicate. This
	// field is mandatory.
	InternalTypeKey string
	// OffsetKey is the key used in the native AST to indicate the absolute offset,
	// from the file start position, where the code mapped to the AST node starts.
	OffsetKey string
	// EndOffsetKey is the key used in the native AST to indicate the absolute offset,
	// from the file start position, where the code mapped to the AST node ends.
	EndOffsetKey string
	// LineKey is the key used in the native AST to indicate
	// the line number where the code mapped to the AST node starts.
	LineKey string
	// EndLineKey is the key used in the native AST to indicate
	// the line number where the code mapped to the AST node ends.
	EndLineKey string
	// ColumnKey is a key that indicates the column inside the line
	ColumnKey string
	// EndColumnKey is a key that indicates the column inside the line where the node ends.
	EndColumnKey string
	// TokenKeys establishes what properties (as in JSON
	// keys) in the native AST nodes can be mapped to Tokens in the UAST. If the
	// InternalTypeKey is the "type" of a node, the Token could be tough of as the
	// "value" representation; this could be a specific value for string/numeric
	// literals or the symbol name for others.  E.g.: if a native AST represents a
	// numeric literal as: `{"ast_type": NumLiteral, "value": 2}` then you should have
	// to add `"value": true` to the TokenKeys map.  Some native ASTs will use several
	// different fields as tokens depending on the node type; in that case, all should
	// be added to this map to ensure a correct UAST generation.
	TokenKeys map[string]bool
	// SyntheticTokens is a map of InternalType to string used to add
	// synthetic tokens to nodes depending on its InternalType; sometimes native ASTs just use an
	// InternalTypeKey for some node but we need to add a Token to the UAST node to
	// improve the representation. In this case we can add both the InternalKey and
	// what token it should generate. E.g.: an InternalTypeKey called "NullLiteral" in
	// Java should be mapped using this map to "null" adding “`"NullLiteral":
	// "null"“` to this map.
	SyntheticTokens map[string]string
	// PromotedPropertyLists allows to convert some properties in the native AST with a list value
	// to its own node with the list elements as children. 	By default the UAST
	// generation will set as children of a node any object that hangs from any of the
	// original native AST node properties. In this process, object key serving as
	// the parent is lost and its name is added as the "internalRole" key of the children.
	// This is usually fine since the InternalTypeKey of the parent AST node will
	// usually provide enough context and the node won't any other children. This map
	// allows you to change this default behavior for specific nodes so the properties
	// are "promoted" to a new node (with an InternalTypeKey named "Parent.KeyName")
	// and the objects in its list will be shown in the UAST as children. E.g.: if you
	// have a native AST where an "If" node has the JSON keys "body", "else" and
	// "condition" each with its own list of children, you could add an entry to
	// PromotedPropertyLists like
	//
	// "If": {"body": true, "orelse": true, "condition": true},
	//
	// In this case, the new nodes will have the InternalTypeKey "If.body", "If.orelse"
	// and "If.condition" and with these names you should be able to write specific
	// matching rules in the annotation.go file.
	PromotedPropertyLists map[string]map[string]bool
	// If this option is set, all properties mapped to a list will be promoted to its own node. Setting
	// this option to true will ignore the PromotedPropertyLists settings.
	PromoteAllPropertyLists bool
	// PromotedPropertyStrings allows to convert some properties which value is a string
	// in the native AST as a full node with the string value as Token like:
	//
	// "SomeKey": "SomeValue"
	//
	// that would be converted to a child node like:
	//
	// {"internalType": "SomeKey", "Token": "SomeValue"}
	PromotedPropertyStrings map[string]map[string]bool
	// TopLevelIsRootNode tells ToNode where to find the root node of
	// the AST.  If true, the root will be its input argument. If false,
	// the root will be the value of the only key present in its input
	// argument.
	TopLevelIsRootNode bool
	// PositionFill specifies if the noder has to fill missing positions (col, line, offset)
	// from ones that the native AST fills. The possible values are "None" (don't fill
	// anything), "OffsetFromLineCol" (fill the offset from the line and column values) and
	// "LineColFromOffset" (fill line and col from the offset).
	PositionFill FillType
}

ObjectToNoder is a ToNoder for trees that are represented as nested JSON objects. That is, an interface{} containing maps, slices, strings and integers. It then converts from that structure to *Node.

func (*ObjectToNoder) ToNode

func (c *ObjectToNoder) ToNode(v interface{}) (*uast.Node, error)

type ParseNativeRequest

type ParseNativeRequest struct {
	Content  string            `json:"content"`
	Encoding protocol.Encoding `json:"encoding"`
}

ParseNativeRequest to use with the native parser. This is for internal use.

type ParseNativeResponse

type ParseNativeResponse struct {
	Status protocol.Status `json:"status"`
	Errors []string        `json:"errors"`
	AST    interface{}     `json:"ast"`
}

ParseNativeResponse is the reply to ParseNativeRequest by the native parser.

type Parser

type Parser struct {
	Client  *Client
	ToNoder ToNoder
}

Parser uses a *Client to parse source code and a ToNoder to convert it to a *uast.Node.

func ExecParser

func ExecParser(toNoder ToNoder, path string, args ...string) (*Parser, error)

ExecParser constructs a uast.Parser based on a native parser binary.

func (*Parser) Close

func (p *Parser) Close() error

func (*Parser) Parse

func (p *Parser) Parse(req *protocol.ParseRequest) *protocol.ParseResponse

type ToNoder

type ToNoder interface {
	// ToNode transforms an arbitrary value into a *Node, or emits an error.
	ToNode(interface{}) (*uast.Node, error)
}

ToNoder transforms a decoded JSON into a *uast.Node. This decoded JSON can be any type based on maps, slices, int, float64 and strings.