mindmeld

README

mindmeld

...is a toy project for port-forwarding between machines on the internet. It works by passing traffic through an intermediary server which is reachable by all parties.

It uses gRPC streaming to pass control messages and proxy traffic, which means that the intermediary server can run in Cloud Run.

Motivation

In pre-COVID times we would often port-forward between machines to quickly test new servers as they were being developed. When everyone was in the same room this was easy! Working remotely has made this a lot more complex...

Although there are a lot of existing solutions out there for this, seemed like an interesting problem to tackle - especially from the point of view of trying to make it as simple as possible, and practical to run (i.e. not too expensive).

Design

There are two main pieces which make the system work:

• router: the intermediary process (see cmd/crrouter, named as it is the Cloud Run variant of the original mmrouter).
• client the CLI tool run by users to create/use forwards (see cmd/mmclient).

Creating a service

Users share a local forward on their machine by registering a service.

1. Client process calls CreateService which creates the service in router, and waits for forward requests to come through.
2. When a forward request arrives on the router it responds to the client via the response stream waiting on the CreateService request, signalling it to create a new proxying connection to handle the traffic for the new forward. The connection is given a token which identifies it when it is received by the router.
3. When the proxying request arrives at the router, it matches it to the forward (using the token) and bridges the two connections.

Users access a service by:

1. Client exposes a local port, which has a TCP listener. Connections made to this listener will be forwarded to the service.
2. For each new connection, the client process calls ForwardToService which checks the service still exists, and returns a token to identify the proxying connection.
3. Client creates a proxying connection, using the provided token, and begins to copy data between the local connection and the proxying connection.

Emulating net.Conn with gRPC

The code was initially designed so that a separate TCP server would run on the router and host the proxy connections. Though easier to debug, this meant it couldn't be used in Cloud Run.

Since Cloud Run was updated to support gRPC bi-directional streaming, it was suddenly a possibility!

The proxying connections are now handled via internal/protoproxy which creates implementations of net.Conn and net.Listener to translate calls to Write into stream sends, calls to Read into stream receives, and Accept into new connections.

The tricky bit is correctly handling when a connection closes, and so there are likely some lingering bugs here.

Documentation

Index

• func RegisterServer(gs *grpc.Server, s *Server)
• type ForwardClient
  • func NewForwardClient(cc *grpc.ClientConn, service, localAddr string) *ForwardClient
  • func (fc *ForwardClient) Close() error
  • func (fc *ForwardClient) Forward() error
• type Server
  • func NewServer(proxyDial string) *Server
  • func (s *Server) Close() error
  • func (s *Server) CreateService(r *pb.CreateServiceRequest, css pb.ControlService_CreateServiceServer) error
  • func (s *Server) ForwardToService(ctx context.Context, r *pb.ForwardToServiceRequest) (*pb.ForwardToServiceResponse, error)
  • func (s *Server) ListServices(ctx context.Context, _ *pb.ListServicesRequest) (*pb.ListServicesResponse, error)
  • func (s *Server) ProxyListen(l net.Listener) error
• type ServiceClient
  • func NewServiceClient(cc *grpc.ClientConn, name, target string) *ServiceClient
  • func (sc *ServiceClient) Close() error
  • func (sc *ServiceClient) Register(ctx context.Context) error
• type TokenSource
  • func NewTokenSource() *TokenSource
  • func (t *TokenSource) Token() string

Functions

func RegisterServer

func RegisterServer(gs *grpc.Server, s *Server)

Types

type ForwardClient

type ForwardClient struct {
	// contains filtered or unexported fields
}

ForwardClient sets up a local TCP listener which takes incoming connections and forwards them to the service.

func NewForwardClient

func NewForwardClient(cc *grpc.ClientConn, service, localAddr string) *ForwardClient

NewForwardClient creates a new forward for service, that will forward connections from localAddr.

func (*ForwardClient) Close

func (fc *ForwardClient) Close() error

Close shuts down all running connections.

func (*ForwardClient) Forward

func (fc *ForwardClient) Forward() error

Forward sets up a local listener and forward the connections to the service.

type Server

type Server struct {
	pb.UnimplementedControlServiceServer
	// contains filtered or unexported fields
}

Server.

func NewServer

func NewServer(proxyDial string) *Server

NewServer creates a new Server.

func (*Server) Close

func (s *Server) Close() error

Close shutsdown any running forwards.

func (*Server) CreateService

func (s *Server) CreateService(r *pb.CreateServiceRequest, css pb.ControlService_CreateServiceServer) error

Create a service.

func (*Server) ForwardToService

func (s *Server) ForwardToService(ctx context.Context, r *pb.ForwardToServiceRequest) (*pb.ForwardToServiceResponse, error)

Forward to remote service.

func (*Server) ListServices

func (s *Server) ListServices(ctx context.Context, _ *pb.ListServicesRequest) (*pb.ListServicesResponse, error)

func (*Server) ProxyListen

func (s *Server) ProxyListen(l net.Listener) error

ProxyListen starts the net.Listener l and handles the incoming connections as proxy connections.

type ServiceClient

type ServiceClient struct {
	// contains filtered or unexported fields
}

ServiceClient registers services and constructs connections to pass traffic through when forwards arrive for them.

func NewServiceClient

func NewServiceClient(cc *grpc.ClientConn, name, target string) *ServiceClient

NewServiceClient creates a new ServiceClient.

func (*ServiceClient) Close

func (sc *ServiceClient) Close() error

Close shuts down all running connections.

func (*ServiceClient) Register

func (sc *ServiceClient) Register(ctx context.Context) error

Register the service and run it. Running connections will continue to operate after Register returns (even with non-nil error). Call Close to shutdown all running connections.

type TokenSource

type TokenSource struct {
	// contains filtered or unexported fields
}

TokenSource is a simple token generator.

func NewTokenSource

func NewTokenSource() *TokenSource

NewTokenSource creates a new TokenSource.

func (*TokenSource) Token

func (t *TokenSource) Token() string

Token returns a new token.