expansionx

Documentation

Overview

Package expansionx contains utilities for starting expansion services for cross-language transforms.

Index

• func GetBeamJar(gradleTarget, version string) (string, error)
• func GetDefaultRepositoryURL() string
• func GetPythonVersion() (string, error)
• func MakeJar(mainJar string, classpath string) (string, error)
• func SetDefaultRepositoryURL(repoURL string) error
• func SetUpPythonEnvironment(extraPackage string) (string, error)
• type ExpansionServiceRunner
  • func NewExpansionServiceRunner(jarPath, servicePort string) (*ExpansionServiceRunner, error)
  • func NewPyExpansionServiceRunner(pythonExec, module, servicePort string) (*ExpansionServiceRunner, error)
  • func (e *ExpansionServiceRunner) Endpoint() string
  • func (e *ExpansionServiceRunner) StartService() error
  • func (e *ExpansionServiceRunner) StopService() error
  • func (e *ExpansionServiceRunner) String() string

Functions

func GetBeamJar

func GetBeamJar(gradleTarget, version string) (string, error)

GetBeamJar checks a temporary directory for the desired Beam JAR, downloads the appropriate JAR from Maven if not present, then returns the file path to the JAR.

func GetDefaultRepositoryURL

func GetDefaultRepositoryURL() string

GetDefaultRepositoryURL returns the current target URL for the defaultJarGetter, indicating what repository will be connected to when getting a Beam JAR.

func GetPythonVersion

func GetPythonVersion() (string, error)

GetPythonVersion returns the Python version to use. It checks for env variable PYTHON_PATH and returns that it if set. If no PYTHON_PATH is defined then it checks for python or python3 and returns that. Otherwise it returns an error.

func MakeJar

func MakeJar(mainJar string, classpath string) (string, error)

MakeJar fetches additional classpath JARs and adds it to the classpath of main JAR file and compiles a fresh JAR.

func SetDefaultRepositoryURL

func SetDefaultRepositoryURL(repoURL string) error

SetDefaultRepositoryURL updates the target URL for the defaultJarGetter, changing which Maven repository will be connected to when getting a Beam JAR. Also validates that it has been passed a URL and returns an error if not.

When changing the target repository, make sure that the value is the prefix up to "org/apache/beam" and that the organization of the repository matches that of the default from that point on to ensure that the conversion of the Gradle target to the JAR name is correct.

func SetUpPythonEnvironment

func SetUpPythonEnvironment(extraPackage string) (string, error)

SetUpPythonEnvironment sets up the virtual ennvironment required for the Apache Beam Python SDK to run an expansion service module.

Types

type ExpansionServiceRunner

type ExpansionServiceRunner struct {
	// contains filtered or unexported fields
}

ExpansionServiceRunner is a type that holds information required to start up a Beam Expansion Service JAR and maintain a handle on the process running the service to enable shutdown as well.

func NewExpansionServiceRunner

func NewExpansionServiceRunner(jarPath, servicePort string) (*ExpansionServiceRunner, error)

NewExpansionServiceRunner builds an ExpansionServiceRunner struct for a given gradle target and Beam version and returns a pointer to it. Passing an empty string as servicePort will request an open port to be assigned to the service.

func NewPyExpansionServiceRunner

func NewPyExpansionServiceRunner(pythonExec, module, servicePort string) (*ExpansionServiceRunner, error)

NewPyExpansionServiceRunner builds an ExpansionServiceRunner struct for a given python module and Beam version and returns a pointer to it. Passing an empty string as servicePort will request an open port to be assigned to the service.

func (*ExpansionServiceRunner) Endpoint

func (e *ExpansionServiceRunner) Endpoint() string

Endpoint returns the formatted endpoint the ExpansionServiceRunner is set to start the expansion service on.

func (*ExpansionServiceRunner) StartService

func (e *ExpansionServiceRunner) StartService() error

StartService starts the expansion service for a given ExpansionServiceRunner. If this is called and does not return an error, the expansion service will be running in the background until StopService is called. This will leak resources if not addressed.

func (*ExpansionServiceRunner) StopService

func (e *ExpansionServiceRunner) StopService() error

StopService stops the expansion service for a given ExpansionServiceRunner. Returns an error if the command hasn't been run or if the process has already exited.

func (*ExpansionServiceRunner) String

func (e *ExpansionServiceRunner) String() string