py

README

py supports Python module and instance using PyObject.

Set up to link python

Go codes in py package use cgo to call PyObject, cgo code is here:

"cgolinks.go"

/*
#cgo pkg-config: python-2.7
*/

other *.go, for example "pydatetime.go"

/*
#include "Python.h"
#include "datetime.h"
...
*/

Currently py package library use pkg-config to link "Python.h". User needs to set up pkg-config and "python-2.7.pc".

Example to set up pkg-config (with pyenv)

If user uses pyenv, "python-2.7.pc" would be installed at ~/.pyenv/versions/<version>/lib/pkgconfig, and user can use this file.

Default pkg-config path (PKG_CONFIG_PATH) would be /usr/local/lib/pkgconfig/, then

ln -s ~/.pyenv/versions/<version>/lib/pkgconfig/python-2.7.pc /usr/local/lib/pkgconfig/

When occur link object error

When python is installed as static object (*.so), error will be occurred on building Sensorbee. Please try to re-install Python with enabled shared.

env PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install -v <version>

Default UDS/UDF

pystate

"lib/sample_module.py"

class SampleClass(object):
    @staticmethod
    def create(arg1, arg2="arg2", arg3="arg3", **arg4):
        self = SampleClass()
        # initialize
        return self
        # blow BQL sample will set like:
        # arg1 = 'arg1'
        # arg2 = 'arg2' # default value
        # arg3 = 'arg3a' # overwritten
        # arg4 = {'new_arg1':1, 'new_arg2':'2'} # variable named arguments

    def sample_method(self, v1, v2, v3):
        # do something

    def write_method(self, value):
        # do something

    @staticmethod
    def load(filepath, arg1):
        # load from filepath and set params
        # return SampleClass constructor

    def save(self, filepath, params):
        # save params and output on filepath

    def terminate(self):
        # called when the state is terminated

The above python class can be created as SharedState, BQL is written like:

CREATE STATE sample_module TYPE pystate
    WITH module_path = "lib", -- optional, default ""
         module_name = "sample_module", -- required
         class_name = "SampleClass",  -- required
         write_method = "write_method", -- optional
         -- rest parameters are used for initializing constructor arguments.
         arg1 = "arg1",
         arg3 = "arg3a",
         new_arg1 = 1,
         new_arg2 = "2"
;

pystate_func

UDF query is written like:

EVAL pystate_func("sample_module", "sample_method", arg1, arg2, arg3)
;

User must make correspond with python sample_method arguments with UDF arguments.

python code

Those UDS creation query and UDF are same as following python code.

import sample_module

# same as CREATE STATE
sm = sample_module.SampleClass(arg1='arg1', arg3='arg3a', new_arg1=1, new_arg2='2')

# same as EVAL
sm.sample_method(arg1, arg2, arg3)

insert into sink

When a pystate is set "write_method" value, then the state is writable, and if not set "write_method" then SensorBee will return an error.

See SensorBee document: Writing Tuples to a UDS

save & load

When SAVE BQL command is executed, then the pystate's save function is called. User can implement saving module code to write on filepath which is pass from SensorBee. SensorBee will save the module as state.

SAVE STATE sample_module;

When LOAD BQL command is executed, then the pystate's load function is called. User need to return python constructor and SensorBee loads the python object as state.

LOAD STATE sample_module TYPE pystate SET arg1="arg1";

More detail, see Saving and Loading a UDS

pystate terminate

When a class provides terminate method, it'll be called in finalization so that the class can release resources it has allocated. More precisely, it'll be called when the state is dropped from SensorBee's topology. The terminate method is optional and will not be called if the class doesn't implement it.

Default Register

py/pystate supports default registration.

UDS

User can add original type name pystate. In an application, write like:

import (
    "gopkg.in/sensorbee/py.v0/pystate"
)

func init() {
    pystate.MustRegisterPyUDSCreator("my_uds", "lib", "sample_module",
        "SampleClass", "write_method")
}

Then user can use TYPE my_uds UDS, like:

CREATE STATE sample_module TYPE org_uds
    WITH v1 = -1,
         v2 = "string"
;

User can also use "pystate_func" same as "pystate"

EVAL pystate_func("sample_module", "sample_method", arg1, arg2, arg3)
;

MustRegisterPyUDSCreator is just an alias.

UDF

User can register a python module method directly.

"lib/sample_module.py"

# module method
def sample_module_method(arg1, arg2):
    # do something

In an application, write like:

import (
    "gopkg.in/sensorbee/py.v0/pystate"
)

func init() {
    pystate.MustRegisterPyUDF("my_udf", "lib", "sample_module",
        "sample_module_method")
}

User can my_udf.

EVAL my_udf(arg1, arg2)
;

This sample query is same as following Python code.

import sample_module

sample_module.sample_module_method(arg1, arg2)

Attention

• sensorbee/py supports only python2.x, not support 3.x
• on windows OS, user need to customize cgo code to link between go and python.
• To reload updated modules, need to re-run SensorBee. Python modules are imported when SensorBee setup to run, and cannot reload modifies python modules in SensorBee running.

Documentation

Overview

Package py provides tools to interact with Python. This package is mainly for SensorBee but can be used for other purposes.

py package depends on py/mainthread package. mainthread requires that it initializes the Python interpreter so that it can keep the main thread under its control. As a result, py package may not work with other packages which initializes Python.

Index

• type Object
  • func (o *Object) Release()
• type ObjectFunc
• type ObjectInstance
  • func (ins *ObjectInstance) Call(name string, args ...data.Value) (data.Value, error)
  • func (ins *ObjectInstance) CallDirect(name string, args []data.Value, kwdArg data.Map) (Object, error)
  • func (ins *ObjectInstance) CheckFunc(name string) bool
• type ObjectModule
  • func LoadModule(name string) (ObjectModule, error)
  • func (m *ObjectModule) Call(name string, args ...data.Value) (data.Value, error)
  • func (m *ObjectModule) GetClass(name string) (ObjectInstance, error)
  • func (m *ObjectModule) NewInstance(name string, args []data.Value, kwdArgs data.Map) (ObjectInstance, error)

Types

type Object

type Object struct {
	// contains filtered or unexported fields
}

Object is a bind of `*C.PyObject`

func (*Object) Release

func (o *Object) Release()

Release decreases reference counter of `C.PyObject` and released the object. This function is public for API users and it acquires GIL of Python interpreter. A user can safely call this method even when its target object is null.

type ObjectFunc

type ObjectFunc struct {
	Object
	// contains filtered or unexported fields
}

ObjectFunc is a bind of `PyObject` used as `PyFunc`

type ObjectInstance

type ObjectInstance struct {
	Object
}

ObjectInstance is a bind of Python instance, used as `PyInstance`.

func (*ObjectInstance) Call

func (ins *ObjectInstance) Call(name string, args ...data.Value) (data.Value,
	error)

Call calls `name` function. [TODO] this function is not supported named arguments

func (*ObjectInstance) CallDirect

func (ins *ObjectInstance) CallDirect(name string, args []data.Value,
	kwdArg data.Map) (Object, error)

CallDirect calls `name` function and return `PyObject` directly. This method is suitable for getting the instance object that called method returned.

func (*ObjectInstance) CheckFunc

func (ins *ObjectInstance) CheckFunc(name string) bool

CheckFunc checks if function having the name exists. It returns true when the function is found.

type ObjectModule

type ObjectModule struct {
	Object
}

ObjectModule is a bind of `PyObject`, used as `PyModule`

func LoadModule

func LoadModule(name string) (ObjectModule, error)

LoadModule loads `name` module. The module needs to be placed at `sys.path`. User can set optional `sys.path` using `mainthread.AppendSysPath`

func (*ObjectModule) Call

func (m *ObjectModule) Call(name string, args ...data.Value) (data.Value, error)

Call calls `name` function. This function is supported for module method of python.

func (*ObjectModule) GetClass

func (m *ObjectModule) GetClass(name string) (ObjectInstance, error)

GetClass returns `name` class instance. User needs to call DecRef when finished using instance.

func (*ObjectModule) NewInstance

func (m *ObjectModule) NewInstance(name string, args []data.Value, kwdArgs data.Map) (
	ObjectInstance, error)

NewInstance returns 'name' constructor with named arguments.

class Sample(object):
	def __init__(self, a, b=5, **c):
		# initializing

To get that "Sample" python instance, callers use a map object as named arguments, and set `kwdArgs`, like:

arg1 := data.Map{
	"a":     data.Value, // ex) data.String("v1")
	"b":     data.Int,   // ex) data.Int(5)
	"hoge1": data.Value, // ex) data.Float(100.0)
	"hoge2": data.Value, // ex) data.True
}

`arg1` is same as `Sample(a-'v1', b=5, hoge1=100.0, hoge2=True)`.

arg2 := data.Map{
	"a":     data.Value, // ex) data.String("v1")
	"hoge1": data.Value, // ex) data.Float(100.0)
	"hoge2": data.Value, // ex) data.True
}

`arg2` is same as `Sample(a='v1', hoge1=100.0, hoge2=True)`, and `self.b` will be set default value (=5).

arg3 := data.Map{
	"a": data.Value, // ex) data.String("v1")
}

`arg3` is same as `Sample(a='v1')`, `self.b` will be set default value (=5), and `self.c` will be set `{}`