Documentation

Overview

    Package harness implements the SDK side of the Beam FnAPI.

    Index

    Constants

    This section is empty.

    Variables

    This section is empty.

    Functions

    func EnableCaptureHook

    func EnableCaptureHook(name string, opts []string)

      EnableCaptureHook is called to request the use of a hook in a pipeline. It updates the supplied pipelines to capture this request.

      func Main

      func Main(ctx context.Context, loggingEndpoint, controlEndpoint string) error

        Main is the main entrypoint for the Go harness. It runs at "runtime" -- not "pipeline-construction time" -- on each worker. It is a FnAPI client and ultimately responsible for correctly executing user code.

        func RegisterCaptureHook

        func RegisterCaptureHook(name string, c CaptureHookFactory)

          RegisterCaptureHook registers a CaptureHookFactory for the supplied identifier.

          Types

          type CaptureHook

          type CaptureHook io.WriteCloser

            CaptureHook writes the messaging content consumed and produced by the worker, allowing the data to be used as an input for the session runner. Since workers can exist in a variety of environments, this allows the runner to tailor the behavior best for its particular needs.

            type CaptureHookFactory

            type CaptureHookFactory func([]string) CaptureHook

              CaptureHookFactory produces a CaptureHook from the supplied options.

              type DataChannel

              type DataChannel struct {
              	// contains filtered or unexported fields
              }

                DataChannel manages a single gRPC stream over the Data API. Data from multiple bundles can be multiplexed over this stream. Data is pushed over the channel, so data for a reader may arrive before the reader connects. Thread-safe.

                func (*DataChannel) OpenRead

                func (c *DataChannel) OpenRead(ctx context.Context, ptransformID string, instID instructionID) io.ReadCloser

                  OpenRead returns an io.ReadCloser of the data elements for the given instruction and ptransform.

                  func (*DataChannel) OpenWrite

                  func (c *DataChannel) OpenWrite(ctx context.Context, ptransformID string, instID instructionID) io.WriteCloser

                    OpenWrite returns an io.WriteCloser of the data elements for the given instruction and ptransform.

                    type DataChannelManager

                    type DataChannelManager struct {
                    	// contains filtered or unexported fields
                    }

                      DataChannelManager manages data channels over the Data API. A fixed number of channels are generally used, each managing multiple logical byte streams. Thread-safe.

                      func (*DataChannelManager) Open

                      func (m *DataChannelManager) Open(ctx context.Context, port exec.Port) (*DataChannel, error)

                        Open opens a R/W DataChannel over the given port.

                        type ScopedDataManager

                        type ScopedDataManager struct {
                        	// contains filtered or unexported fields
                        }

                          ScopedDataManager scopes the global gRPC data manager to a single instruction. The indirection makes it easier to control access.

                          func NewScopedDataManager

                          func NewScopedDataManager(mgr *DataChannelManager, instID instructionID) *ScopedDataManager

                            NewScopedDataManager returns a ScopedDataManager for the given instruction.

                            func (*ScopedDataManager) Close

                            func (s *ScopedDataManager) Close() error

                              Close prevents new IO for this instruction.

                              func (*ScopedDataManager) OpenRead

                              func (s *ScopedDataManager) OpenRead(ctx context.Context, id exec.StreamID) (io.ReadCloser, error)

                                OpenRead opens an io.ReadCloser on the given stream.

                                func (*ScopedDataManager) OpenWrite

                                func (s *ScopedDataManager) OpenWrite(ctx context.Context, id exec.StreamID) (io.WriteCloser, error)

                                  OpenWrite opens an io.WriteCloser on the given stream.

                                  type ScopedStateReader

                                  type ScopedStateReader struct {
                                  	// contains filtered or unexported fields
                                  }

                                    ScopedStateReader scopes the global gRPC state manager to a single instruction for side input use. The indirection makes it easier to control access.

                                    func NewScopedStateReader

                                    func NewScopedStateReader(mgr *StateChannelManager, instID instructionID) *ScopedStateReader

                                      NewScopedStateReader returns a ScopedStateReader for the given instruction.

                                      func (*ScopedStateReader) Close

                                      func (s *ScopedStateReader) Close() error

                                        Close closes all open readers.

                                        func (*ScopedStateReader) OpenIterable

                                        func (s *ScopedStateReader) OpenIterable(ctx context.Context, id exec.StreamID, key []byte) (io.ReadCloser, error)

                                          OpenIterable opens a byte stream for reading unwindowed iterables from the runner.

                                          func (*ScopedStateReader) OpenSideInput

                                          func (s *ScopedStateReader) OpenSideInput(ctx context.Context, id exec.StreamID, sideInputID string, key, w []byte) (io.ReadCloser, error)

                                            OpenSideInput opens a byte stream for reading iterable side input.

                                            type StateChannel

                                            type StateChannel struct {
                                            	DoneCh <-chan struct{}
                                            	// contains filtered or unexported fields
                                            }

                                              StateChannel manages state transactions over a single gRPC connection. It does not need to track readers and writers as carefully as the DataChannel, because the state protocol is request-based.

                                              func (*StateChannel) Send

                                                Send sends a state request and returns the response.

                                                type StateChannelManager

                                                type StateChannelManager struct {
                                                	// contains filtered or unexported fields
                                                }

                                                  StateChannelManager manages data channels over the State API. A fixed number of channels are generally used, each managing multiple logical byte streams. Thread-safe.

                                                  func (*StateChannelManager) Open

                                                    Open opens a R/W StateChannel over the given port.

                                                    Directories

                                                    Path Synopsis
                                                    Package init contains the harness initialization code defined by the FnAPI.
                                                    Package init contains the harness initialization code defined by the FnAPI.
                                                    Package session is a generated protocol buffer package.
                                                    Package session is a generated protocol buffer package.