cortical

README

Cortical

import "github.com/owulveryck/cortical"

What is Cortical?

Cortical is a go framework middleware piece of code that acts as a message dispatcher. The messages are transmitted in full duplex over a websocket. Cortical is therefore a very convenient way to distribute messages to "processing units" (other go functions) and to get the responses back in a concurrent and asynchronous way.

The "processing units" are called Cortexes and do not need to be aware of any web mechanism.

Cortical? Cortex? is it related to ML?

Actually I have developed this code as a support for my tests with tensorflow, Google Cloud Plafeform and AWS ML services. I needed a way to capture images from my webcam. I have used my chrome browser for this purpose. Every Cortex is a specific ML implementation (eg a Memory Cortex that captures all the images and send them to a cloud storage is needed for training models).

See my blog post "Chrome, the eye of the cloud - Computer vision with deep learning and only 2Gb of RAM" for more explanation.

Cortexes

A cortex is any go code that provides two functions:

• A "send" function that returns a channel of []byte. The content of the channel is sent to the websocket once available (cf GetInfoFromCortexFunc)
• A "receive" method that take a pointer of []byte. This function is called each time a message is received (cf SendInfoToCortex)

A cortex object must therefore be compatible with the cortical.Cortex interface:

ex:

// echo is a dummy type that reads a message, and send back an "ack"
type echo struct{}

func new() *echo {
	return &echo{}
}

// NewCortex is filling the cortical.Cortex interface
func (e *echo) NewCortex(ctx context.Context) (cortical.GetInfoFromCortexFunc, cortical.SendInfoToCortex) {
	c := make(chan []byte)
	return func(ctx context.Context) chan []byte {
			return c
		}, func(ctx context.Context, b *[]byte) {
			c <- *b
		}
}

Cortical take care of extracting and sending the []byte to the websocket and dispatches them through all the cortexes.

Registering the cortexes and creating a http.HandlerFunc

By now, the registration of cortexes is done at the creation of the Cortical object.

WARNING This will probably change in the future

brain := &cortical.Cortical{
     Upgrader: websocket.Upgrader{},
     Cortexes:  []cortical.Cortex{
          &echo{}, // See example in the previous paragraph
     }, 
}
http.HandleFunc("/ws", brain.ServeWS)
log.Fatal(http.ListenAndServe(":8080", nil))

Examples of cortex

See the example in godoc

Why is it coded in go?

1. Because I love go
2. Because I only know how to code in go
3. The real reason: concurrency (take a look at Rob Pike - 'Concurrency Is Not Parallelism' on youtube to truly understand why)

Caution

The API may change a lot; use it at your own risks. PR are welcome.

TODO

• Tests
• Doc
• Benchmarks
• Demo
• More functional tests
• Using the context cancel mechanism instead of my done chan
• Evaluating the opportunity of changing the interface for an io.ReadWriteCloser
• More and better doc

Documentation

Overview

Package cortical is a utility that handles websocket connexions and dispatch all the []byte received to the consumers called "cortex". It also get all the informations of the producers and sends them back to the websocket

The context passed to the Cortex holds a uuid encoded into a string that is specific for each connection The UUID can be retrieved by calling

uuid := ctx.Value(ContextKeyType(ContextKey))

Example

package main

import (
	"context"
	"fmt"
	"github.com/gorilla/websocket"
	"github.com/owulveryck/cortical"
	"log"
	"net/http"
)

// echo is a dummy type that reads a message, and send back an "ack"
type echo struct{}

func new() *echo {
	return &echo{}
}

// NewCortex is filling the  ...
func (e *echo) NewCortex(ctx context.Context) (cortical.GetInfoFromCortexFunc, cortical.SendInfoToCortex) {
	c := make(chan []byte)
	return func(ctx context.Context) chan []byte {
			return c
		}, func(ctx context.Context, b *[]byte) {
			c <- *b
		}
}

func main() {
	brain := &cortical.Cortical{
		Upgrader: websocket.Upgrader{},
		Cortexes: []cortical.Cortex{new()},
	}
	http.HandleFunc("/ws", brain.ServeWS)

	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprintf(w, index)
	})

	log.Fatal(http.ListenAndServe(":8080", nil))

}

var index = `
<html>
    <head>
	<title>demo</title>
	<script type="text/javascript">
	window.onload = function () {
	    var conn;
	    var log = document.getElementById("log");
	    function appendLog(item) {
		var doScroll = log.scrollTop > log.scrollHeight - log.clientHeight - 1;
		log.appendChild(item);
		if (doScroll) {
		    log.scrollTop = log.scrollHeight - log.clientHeight;
		}
	    }
	    setInterval(function () {
		if (!conn) {
		    return false;
		}
		conn.send("ping");
		return false;
	    },1000);
	    if (window["WebSocket"]) {
		conn = new WebSocket("ws://" + document.location.host + "/ws");
		conn.onclose = function (evt) {
		    var item = document.createElement("div");
		    item.innerHTML = "<b>Connection closed.</b>";
		    appendLog(item);
		};
		conn.onmessage = function (evt) {
		    var messages = evt.data.split('\n');
		    for (var i = 0; i < messages.length; i++) {
			var item = document.createElement("div");
			item.innerText = messages[i];
			appendLog(item);
		    }
		};
	    } else {
		var item = document.createElement("div");
		item.innerHTML = "<b>Your browser does not support WebSockets.</b>";
		appendLog(item);
	    }
	};
	</script>
    </head>
    <body>
	<p id="log"> </p>
	<script>

	</script>
    </body>
</html>
`

Index

• Constants
• type ContextKeyType
• type Cortex
• type Cortical
  • func (wsd *Cortical) ServeWS(w http.ResponseWriter, r *http.Request)
• type GetInfoFromCortexFunc
• type SendInfoToCortex

Examples

• Package

Constants

const ContextKey = "uuid"

ContextKey is the key name where the session is stored

Types

type ContextKeyType

type ContextKeyType string

ContextKeyType is the type of the key of the context

type Cortex

type Cortex interface {
	NewCortex(context.Context) (GetInfoFromCortexFunc, SendInfoToCortex)
}

A Cortex shall implement an interface that returns two functions for getting and sending a byte array

type Cortical

type Cortical struct {
	Upgrader websocket.Upgrader
	Cortexes []Cortex
}

Cortical specifies how to upgrade an HTTP connection to a Websocket connection as well as the action to be performed on receive a []byte

func (*Cortical) ServeWS

func (wsd *Cortical) ServeWS(w http.ResponseWriter, r *http.Request)

ServeWS is the dispacher function

type GetInfoFromCortexFunc

type GetInfoFromCortexFunc func(ctx context.Context) chan []byte

GetInfoFromCortexFunc is the method implenented by a chatter to send objects

type SendInfoToCortex

type SendInfoToCortex func(context.Context, *[]byte)

SendInfoToCortex is the method implemented by a chatter to receive objects