python

package module
v0.0.0-...-8ea5f8a Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 27, 2024 License: Apache-2.0 Imports: 9 Imported by: 0

README

py4go

License Go Report Card

Call Python 3 functions and methods from within your Go program while exposing Go functions and methods to Python.

This is not an implementation of Python in Go. Rather, py4go works by embedding the CPython runtime into your Go program using cgo functionality.

The expected use cases are not low-latency integration, but rather tight bidirectional integration. You can combine the full Go ecosystem with the full Python ecosystem.

Though you can achieve some integration by using Go's exec package to run python, with py4go you get fine-grained access to individual functions, objects, methods, and variables.

To get started try running scripts/example. Note that you need the Python development libraries installed. E.g. in Fedora:

sudo dnf install python3-devel

Example Usage

package main

import (
    "fmt"
    "github.com/scoursen/py4go"
)

func main() {
    // Initialize Python
    python.Initialize()
    defer python.Finalize()

    // Import Python code (foo.py)
    foo, _ := python.Import("foo")
    defer foo.Release()

    // Get access to a Python function
    hello, _ := foo.GetAttr("hello")
    defer hello.Release()

    // Call the function with arguments
    r, _ := hello.Call("myargument")
    defer r.Release()
    fmt.Printf("Returned: %s\n", r.String())

    // Expose a Go function to Python via a C wrapper
    // (Just use "import api" from Python)
    api, _ := python.CreateModule("api")
    defer api.Release()
    api.AddModuleCFunctionNoArgs("my_function", C.api_my_function)
    api.EnableModule()
}

Calling Python code from Go is relatively straightforward because Python is a dynamic language and CPython is an interpreted runtime. Exposing Go code to Python is more involved as it requires writing wrapper functions in C, which we omitted in the example above. See the examples directory for more detail.

In the future we are hoping to make this easier, perhaps via a C code generator based on static analysis of function signatures and types.

Caveats

There are several issues to be aware of:

  • It's more difficult to distribute your Go program because you must have the CPython library available on the target operating system with a specific name. Because different operating systems have their own conventions for naming this library, to create a truly portable distribution it may be best to distribute your program as a packaged container, e.g. using Flatpak, Docker, or Snap.
  • It is similarly more difficult to build your Go program. We are using pkg-config: python3-embed to locate the CPython SDK, which works on Fedora-based operating systems. But, because where you build will determine the requirements for where you will run, it may be best to build on Fedora, either directly or in a virtual machine or container. Unfortunately cgo does not let us parameterize that pkg-config directive, thus you will have to modify our source files in order to build on/for other operating systems.
  • Calling functions and passing data between these two high-level language's runtime environments obviously incurs some overhead. Notably strings are sometimes copied multiple times internally, and may be encoded and decoded (Go normally uses UTF-8, Python defaults to UCS4). If you are frequently calling back and forth be aware of possible performance degradation. As always, if you experience a problem measure first and identify the bottleneck before prematurely optimizing!
  • Similarly, be aware that you are simultaneously running two memory management runtimes, each with its own heap allocation and garbage collection threads, and that Go is unaware of Python's. Your Go code will thus need to explicitly call Release on all Python references to ensure that they are garbage collected. Luckily, the defer keyword makes this easy enough in many circumstances.
  • Concurrency is a bit tricky in Python due to its infamous Global Interpreter Lock (GIL). If you are calling Python code from a Goroutine make sure to call python.SaveThreadState and python.EnsureGilState as appropriate. See the examples for more detail.

References

  • go-python is a similar and more mature project for Python 2.
  • goPy is a much older project for Python 2.
  • gopy generates Python wrappers for Go functions.
  • setuptools-golang allows you to include Go libraries in Python packages.

Documentation

Index

Constants

View Source 
const PYTHONPATH = "PYTHONPATH"

Variables

View Source 
var BoolType = NewType(&C.PyBool_Type)
View Source 
var ByteArrayType = NewType(&C.PyByteArray_Type)
View Source 
var BytesType = NewType(&C.PyBytes_Type)
View Source 
var DictType = NewType(&C.PyDict_Type)
View Source 
var False = NewReference(C.Py_False)
View Source 
var FloatType = NewType(&C.PyFloat_Type)
View Source 
var FrozenSetType = NewType(&C.PyFrozenSet_Type)
View Source 
var ListType = NewType(&C.PyList_Type)
View Source 
var LongType = NewType(&C.PyLong_Type)
View Source 
var ModuleType = NewType(&C.PyModule_Type)
View Source 
var None = NewReference(C.Py_None)
View Source 
var SetType = NewType(&C.PySet_Type)
View Source 
var True = NewReference(C.Py_True)
View Source 
var TupleType = NewType(&C.PyTuple_Type)
View Source 
var UnicodeType = NewType(&C.PyUnicode_Type)

Functions

func AppendPythonPath 

func AppendPythonPath(path ...string)

func Finalize 

func Finalize() error

func GetError 

func GetError() error

func HasException 

func HasException() bool

func Initialize 

func Initialize()

func PrependPythonPath 

func PrependPythonPath(path ...string)

func SetPythonPath 

func SetPythonPath(path ...string)

func Version 

func Version() string

Types

type CPyObject 

type CPyObject = C.PyObject

type Exception 

type Exception struct {
	Type      *Reference
	Value     *Reference
	Traceback *Reference
}

func FetchException 

func FetchException() *Exception

func NewExceptionRaw 

func NewExceptionRaw(type_ *Reference, value *Reference, traceback *Reference) *Exception

func (*Exception) Error 

func (self *Exception) Error() string

error signature

type GilState 

type GilState struct {
	State C.PyGILState_STATE
}

func EnsureGilState 

func EnsureGilState() *GilState

func (*GilState) Release 

func (self *GilState) Release()

type Reference 

type Reference struct {
	Object *CPyObject
}

func CreateModule 

func CreateModule(name string) (*Reference, error)

func Import 

func Import(name string) (*Reference, error)

func NewByteArray 

func NewByteArray(value []byte) (*Reference, error)

func NewBytes 

func NewBytes(value []byte) (*Reference, error)

func NewCapsule 

func NewCapsule(v interface{}, name *string, dtor unsafe.Pointer) (*Reference, error)

func NewDict 

func NewDict() (*Reference, error)

func NewFloat 

func NewFloat(value float64) (*Reference, error)

func NewFrozenSet 

func NewFrozenSet(iterable *Reference) (*Reference, error)

func NewList 

func NewList(items ...interface{}) (*Reference, error)

func NewListRaw 

func NewListRaw(size int) (*Reference, error)

func NewLong 

func NewLong(value int64) (*Reference, error)

func NewModuleRaw 

func NewModuleRaw(name string) (*Reference, error)

func NewPrimitiveReference 

func NewPrimitiveReference(value interface{}) (*Reference, error)

func NewReference 

func NewReference(pyObject *CPyObject) *Reference

func NewSet 

func NewSet(iterable *Reference) (*Reference, error)

func NewTuple 

func NewTuple(items ...interface{}) (*Reference, error)

func NewTupleRaw 

func NewTupleRaw(size int) (*Reference, error)

func NewUnicode 

func NewUnicode(value string) (*Reference, error)

func (*Reference) AccessByteArray 

func (self *Reference) AccessByteArray() (unsafe.Pointer, int, error)

func (*Reference) AccessBytes 

func (self *Reference) AccessBytes() (unsafe.Pointer, int, error)

func (*Reference) Acquire 

func (self *Reference) Acquire()

func (*Reference) AddModuleCFunctionArgs 

func (self *Reference) AddModuleCFunctionArgs(name string, function unsafe.Pointer) error

PyCFunction signature, second argument is tuple of Python arguments

func (*Reference) AddModuleCFunctionArgsAndKeywords 

func (self *Reference) AddModuleCFunctionArgsAndKeywords(name string, function unsafe.Pointer) error

PyCFunctionWithKeywords signature

func (*Reference) AddModuleCFunctionFastArgs 

func (self *Reference) AddModuleCFunctionFastArgs(name string, function unsafe.Pointer) error

_PyCFunctionFast signature

func (*Reference) AddModuleCFunctionFastArgsAndKeywords 

func (self *Reference) AddModuleCFunctionFastArgsAndKeywords(name string, function unsafe.Pointer) error

_PyCFunctionFastWithKeywords signature

func (*Reference) AddModuleCFunctionNoArgs 

func (self *Reference) AddModuleCFunctionNoArgs(name string, function unsafe.Pointer) error

PyCFunction signature, second argument unused

func (*Reference) AddModuleCFunctionOneArg 

func (self *Reference) AddModuleCFunctionOneArg(name string, function unsafe.Pointer) error

PyCFunction signature, second argument is the Python argument

func (*Reference) ByteArrayToBytes 

func (self *Reference) ByteArrayToBytes() ([]byte, error)

func (*Reference) Call 

func (self *Reference) Call(args ...interface{}) (*Reference, error)

func (*Reference) CallRaw 

func (self *Reference) CallRaw(args *Reference, kw *Reference) (*Reference, error)

func (*Reference) EnableModule 

func (self *Reference) EnableModule() error

func (*Reference) GetAttr 

func (self *Reference) GetAttr(name string) (*Reference, error)

func (*Reference) GetModuleName 

func (self *Reference) GetModuleName() (string, error)

func (*Reference) GetPointer 

func (self *Reference) GetPointer(name *string) unsafe.Pointer

func (*Reference) IsBool 

func (self *Reference) IsBool() bool

func (*Reference) IsByteArray 

func (self *Reference) IsByteArray() bool

func (*Reference) IsBytes 

func (self *Reference) IsBytes() bool

func (*Reference) IsCapsule 

func (self *Reference) IsCapsule(name *string) bool

func (*Reference) IsDict 

func (self *Reference) IsDict() bool

func (*Reference) IsFloat 

func (self *Reference) IsFloat() bool

func (*Reference) IsFrozenSet 

func (self *Reference) IsFrozenSet() bool

func (*Reference) IsList 

func (self *Reference) IsList() bool

func (*Reference) IsLong 

func (self *Reference) IsLong() bool

func (*Reference) IsSet 

func (self *Reference) IsSet() bool

func (*Reference) IsTuple 

func (self *Reference) IsTuple() bool

func (*Reference) IsUnicode 

func (self *Reference) IsUnicode() bool

func (*Reference) Release 

func (self *Reference) Release()

func (*Reference) SetAttr 

func (self *Reference) SetAttr(name string, reference *Reference) error

func (*Reference) SetDictItem 

func (self *Reference) SetDictItem(key *Reference, value *Reference) error

func (*Reference) SetListItem 

func (self *Reference) SetListItem(index int, item *Reference) error

func (*Reference) SetTupleItem 

func (self *Reference) SetTupleItem(index int, item *Reference) error

func (*Reference) Str 

func (self *Reference) Str() (*Reference, error)

func (*Reference) String 

func (self *Reference) String() string

fmt.Stringer interface

func (*Reference) ToBool 

func (self *Reference) ToBool() bool

func (*Reference) ToBytes 

func (self *Reference) ToBytes() ([]byte, error)

func (*Reference) ToFloat64 

func (self *Reference) ToFloat64() (float64, error)

func (*Reference) ToInt64 

func (self *Reference) ToInt64() (int64, error)

func (*Reference) ToString 

func (self *Reference) ToString() (string, error)

func (*Reference) Type 

func (self *Reference) Type() *Type

func (*Reference) Unref 

func (self *Reference) Unref(name *string)

type SubInterpreter 

type SubInterpreter struct {
	State *C.PyThreadState
}

func NewSubInterpreter 

func NewSubInterpreter() (*SubInterpreter, error)

func (*SubInterpreter) End 

func (self *SubInterpreter) End()

type ThreadState 

type ThreadState struct {
	State *C.PyThreadState
}

func SaveThreadState 

func SaveThreadState() *ThreadState

func (*ThreadState) Restore 

func (self *ThreadState) Restore()

type Type 

type Type struct {
	Object *C.PyTypeObject
}

func NewType 

func NewType(pyTypeObject *C.PyTypeObject) *Type

func (*Type) HasFlag 

func (self *Type) HasFlag(flag C.ulong) bool

func (*Type) IsSubtype 

func (self *Type) IsSubtype(type_ *Type) bool

Source Files

View all Source files

Directories

Path Synopsis
examples
hello-world command
hello-world/api

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.