astibob

package module
v1.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 7, 2020 License: MIT Imports: 9 Imported by: 4

README

GoReportCard GoDoc

Golang framework to build an AI that can understand and speak back to you, and everything else you want.

WARNING: the code below doesn't handle errors for readability purposes, however you SHOULD!

Demos

Here's a list of AIs built with astibob (if you're using astibob and want your project to be listed here, please submit a PR):

How it works

Overview

Overview

  • humans operate the AI through the Web UI
  • the Web UI interacts with the AI through the Index
  • the Index keeps an updated list of all Workers and forwards Web UI messages to Workers and vice versa
  • Workers have one or more Abilities and are usually located on different machines
  • Abilities run simple tasks such as reading an audio input (e.g. a microphone), executing speech-to-text analyses or doing speech-synthesis
  • Abilities can communicate directly between each other even if on different Workers
  • all communication is done via JSON messages exchanged through HTTP or Websocket

FAQ

  • Why split abilities between several workers?

    Because abilities may need to run on different machines located in different part of the world. The simplest example is wanting to read microphones inputs located in several rooms of your house. Each microphone is an ability whereas each room of your house is a worker.

Install the project

Run the following command:

$ go get -u github.com/asticode/go-astibob/...

I want to see some code

Index

// Create index
i, _ := index.New(index.Options{
    Server: astibob.ServerOptions{
        Addr:     "127.0.0.1:4000",
        Password: "admin",
        Username: "admin",
    },
})

// Make sure to properly close the index
defer i.Close()

// Handle signals
i.HandleSignals()

// Serve
i.Serve()

// Blocking pattern
i.Wait()

Worker

// Create worker
w := worker.New("Worker #1", worker.Options{
    Index: astibob.ServerOptions{
        Addr:     "127.0.0.1:4000",
        Password: "admin",
        Username: "admin",
    },
    Server: astibob.ServerOptions{Addr: "127.0.0.1:4001"},
})

// Make sure to properly close the worker
defer w.Close()

// Create runnables
r1 := pkg1.NewRunnable("Runnable #1")
r2 := pkg2.NewRunnable("Runnable #2")

// Register runnables
w.RegisterRunnables(
	worker.Runnable{
        AutoStart: true,
        Runnable:  r1,
    },
	worker.Runnable{
        Runnable:  r2,
    },
)

// Create listenables
l1 := pkg3.NewListenable(pkg3.ListenableOptions{
	OnEvent1: func(arg1 string) { log.Println(arg1) },
})
l2 := pkg4.NewListenable(pkg4.ListenableOptions{
	OnEvent2: func(arg2 string) { log.Println(arg2) },
})

// Register listenables
w.RegisterListenables(
	worker.Listenable{
        Listenable: l1,
        Runnable:   "Runnable #1",
        Worker:     "Worker #1",
    },
	worker.Listenable{
        Listenable: l2,
        Runnable:   "Runnable #3",
        Worker:     "Worker #2",
    },
)

// Handle an event and send a message to one of the runnables
w.On(astibob.DispatchConditions{
    From: astibob.NewRunnableIdentifier("Runnable #1", "Worker #1"),
    Name: astikit.StrPtr("Event #1"),
}, func(m *astibob.Message) (err error) {
    // Send message
    if err = w.SendMessages("Worker #1", "Runnable #1", pkg2.NewMessage1("Hello world")); err != nil {
        err = errors.Wrap(err, "main: sending message failed")
        return
    }
    return
})

// Handle signals
w.HandleSignals()

// Serve
w.Serve()

// Register to index
w.RegisterToIndex()

// Blocking pattern
w.Wait()

Abilities

The framework comes with a few abilities located in the abilities folder:

Audio input

This ability allows you reading from an audio stream e.g. a microphone.

Dependencies

It's strongly recommended to use PortAudio and its astibob wrapper.

To know which devices are available on the machine run:

$ go run abilities/audio_input/portaudio/cmd/main.go
Runnable and operatable
// Create portaudio
p := portaudio.New()

// Initialize portaudio
p.Initialize()

// Make sure to close portaudio
defer p.Close()

// Create default stream
s, _ := p.NewDefaultStream(portaudio.StreamOptions{
    BitDepth:             32,
    BufferLength:         5000,
    MaxSilenceLevel:      5 * 1e6,
    NumInputChannels:     2,
    SampleRate:           44100,
})

// Create runnable
r := audio_input.NewRunnable("Audio input", s)

// Register runnables
w.RegisterRunnables(worker.Runnable{
    AutoStart: true,
    Runnable:  r,
})

// Register listenables
// This is mandatory for the Web UI to work properly
w.RegisterListenables(worker.Listenable{
    Listenable: r,
    Runnable:   "Audio input",
    Worker:     "Worker #1",
})
Listenable
// Register listenables
w.RegisterListenables(
    worker.Listenable{
        Listenable: audio_input.NewListenable(audio_input.ListenableOptions{
            OnSamples: func(from astibob.Identifier, samples []int, bitDepth, numChannels, sampleRate int, maxSilenceLevel float64) (err error) {
                // TODO Do something with the samples
                return
            },
        }),
        Runnable: "Audio input",
        Worker:   "Worker #1",
    },
)

Speech to Text

This ability allows you to execute speech-to-text analyses.

Dependencies

It's strongly recommended to install DeepSpeech and its astibob wrapper.

I don't want to train a new model

  • create a working directory (for simplicity purposes, we'll assume its absolute path is /path/to/deepspeech)

  • download a client native_client.<your system>.tar.xz" matching your system at the bottom of this page

  • create the /path/to/deepspeech/lib directory and extract the client content inside it

  • create the /path/to/deepspeech/include directory and download deepspeech.h inside it

  • create the /path/to/deepspeech/model/en directory, and download and extract the english model inside it

  • whenever you run a worker that needs deepspeech, make sure to have the following environment variables:

      CGO_CXXFLAGS="-I/path/to/deepspeech/include"
  LIBRARY_PATH=/path/to/deepspeech/lib:$LIBRARY_PATH
  LD_LIBRARY_PATH=/path/to/deepspeech/lib:$LD_LIBRARY_PATH
I want to train a new model

In addition to the steps above:

  • create the /path/to/deepspeech/model/custom directory
  • run git clone https://github.com/mozilla/DeepSpeech inside /path/to/deepspeech
  • install the dependencies
Runnable and Operatable
// Create deepspeech
mp := "/path/to/deepspeech/model/en"
d := deepspeech.New(deepspeech.Options{
    AlphabetPath:   mp + "/alphabet.txt",
    BeamWidth:      1024,
    ClientPath:     "/path/to/deepspeech/DeepSpeech/DeepSpeech.py",
    LMPath:         mp + "/lm.binary",
    LMWeight:       0.75,
    ModelPath:      mp + "/output_graph.pb",
    PrepareDirPath: "/path/to/deepspeech/prepare",
    TrainingArgs: map[string]string{
        "checkpoint_dir":   "/path/to/deepspeech/model/custom/checkpoints",
        "dev_batch_size":   "4",
        "export_dir":       "/path/to/deepspeech/model/custom",
        "noearly_stop":     "",
        "test_batch_size":  "4",
        "train_batch_size": "20",

        // Mozilla values
        "learning_rate": "0.0001",
        "dropout_rate":  "0.15",
        "lm_alpha":      "0.75",
        "lm_beta":       "1.85",
    },
    TriePath:             mp + "/trie",
    ValidWordCountWeight: 1.85,
})

// Make sure to close deepspeech
defer d.Close()

// Initialize deepspeech
d.Init()

// Create runnable
r := speech_to_text.NewRunnable("Speech to Text", d, speech_to_text.RunnableOptions{
    SpeechesDirPath: "/path/to/speech_to_text/speeches",
})

// Initialize runnable
r.Init()

// Make sure to close the runnable
defer r.Close()

// Register runnables
w.RegisterRunnables(worker.Runnable{
    AutoStart: true,
    Runnable:  r,
})

// Send samples
w.SendMessage(worker.MessageOptions{
    Message:  speech_to_text.NewSamplesMessage(
        from,
        samples,
        bitDepth,
        numChannels,
        sampleRate,
        maxSilenceLevel,
    ),
    Runnable: "Speech to Text",
    Worker:   "Worker #3",
})
Listenable
// Register listenables
w.RegisterListenables(
    worker.Listenable{
        Listenable: speech_to_text.NewListenable(speech_to_text.ListenableOptions{
            OnText: func(from astibob.Identifier, text string) (err error) {
                // TODO Do something with the text
                return
            },
        }),
        Runnable: "Speech to Text",
        Worker:   "Worker #3",
    },
)

Text to Speech

This ability allows you to run speech synthesis.

Dependencies

It's strongly recommended to use astibob wrapper.

If you're using Linux it's strongly recommended to use ESpeak.

Runnable
// Create speaker
s := speak.New(speak.Options{})

// Initialize speaker
s.Initialize()

// Make sure to close speaker
defer s.Close()

// Register runnables
w.RegisterRunnables(worker.Runnable{
    AutoStart: true,
    Runnable:  text_to_speech.NewRunnable("Text to Speech", s),
})

// Say something
w.SendMessage(worker.MessageOptions{
    Message:  text_to_speech.NewSayMessage("Hello world"),
    Runnable: "Text to Speech",
    Worker:   "Worker #1",
})

Create your own ability

Creating your own ability is pretty straight-forward: you need to create an object that implements the astibob.Runnable interface. Optionally it can implement the astibob.Operatable interface as well.

If you want other abilities to be able to interact with it you'll need to create another object that implements the astibob.Listenable interface.

I strongly recommend checking out how provided abilities are built and trying to copy them first.

Runnable

The quickest way to implement the astibob.Runnable interface is to add an embedded astibob.BaseRunnable attribute to your object.

You can then use astibob.NewBaseRunnable to initialize it which allows you providing the proper options.

Operatable

The quickest way to implement the astibob.Operatable interface is to add an embedded astibob.BaseOperatable attribute to your object.

You can then use the cmd/operatable command to generate an operatable.go file binding your resources folder containing your static and template files. You can finally add custom routes manually to the astibob.BaseOperatable using the AddRoute method.

Listenable

No shortcut here, you need to create an object that implements the astibob.Listenable interface yourself.

Contribute

If you've created an awesome Ability and you feel it could be of interest to the community, create a PR here.

Documentation

Index

Constants

View Source 
const (
	IndexIdentifierType    = "index"
	RunnableIdentifierType = "runnable"
	UIIdentifierType       = "ui"
	WorkerIdentifierType   = "worker"
)

Identifier types

View Source 
const (
	ListenablesRegisterMessage  = "listenables.register"
	RunnableCrashedMessage      = "runnable.crashed"
	RunnableDoneMessage         = "runnable.done"
	RunnableStartMessage        = "runnable.start"
	RunnableStartedMessage      = "runnable.started"
	RunnableStopMessage         = "runnable.stop"
	RunnableStoppedMessage      = "runnable.stopped"
	UIDisconnectedMessage       = "ui.disconnected"
	UIMessageNamesAddMessage    = "ui.message.names.add"
	UIMessageNamesDeleteMessage = "ui.message.names.delete"
	UIPingMessage               = "ui.ping"
	UIRegisterMessage           = "ui.register"
	UIWelcomeMessage            = "ui.welcome"
	WorkerDisconnectedMessage   = "worker.disconnected"
	WorkerRegisterMessage       = "worker.register"
	WorkerRegisteredMessage     = "worker.registered"
	WorkerWelcomeMessage        = "worker.welcome"
)

Message names

View Source 
const (
	RunningStatus = "running"
	StoppedStatus = "stopped"
)

Statuses

Variables

This section is empty.

Functions

func ContentHandle 

func ContentHandle(path string, c []byte, l astikit.SeverityLogger) httprouter.Handle

func DirHandle 

func DirHandle(path string) httprouter.Handle

func ParseRunnableStartPayload 

func ParseRunnableStartPayload(m *Message) (name string, err error)

func ParseRunnableStopPayload 

func ParseRunnableStopPayload(m *Message) (name string, err error)

func ParseUIDisconnectedPayload 

func ParseUIDisconnectedPayload(m *Message) (name string, err error)

func ParseUIMessageNamesAddPayload 

func ParseUIMessageNamesAddPayload(m *Message) (names []string, err error)

func ParseUIMessageNamesDeletePayload 

func ParseUIMessageNamesDeletePayload(m *Message) (names []string, err error)

func ParseWorkerDisconnectedPayload 

func ParseWorkerDisconnectedPayload(m *Message) (worker string, err error)

func WriteHTTPData 

func WriteHTTPData(l astikit.SeverityLogger, rw http.ResponseWriter, data interface{})

func WriteHTTPError 

func WriteHTTPError(l astikit.SeverityLogger, rw http.ResponseWriter, code int, err error)

Types

type BaseOperatable 

type BaseOperatable struct {
	// contains filtered or unexported fields
}

func NewBaseOperatable 

func NewBaseOperatable() *BaseOperatable

func (*BaseOperatable) AddRoute 

func (o *BaseOperatable) AddRoute(path, method string, h httprouter.Handle)

func (*BaseOperatable) AddTemplate 

func (o *BaseOperatable) AddTemplate(n string, c []byte)

func (*BaseOperatable) Routes 

func (o *BaseOperatable) Routes() map[string]map[string]httprouter.Handle

func (*BaseOperatable) Templates 

func (o *BaseOperatable) Templates() map[string][]byte

type BaseRunnable 

type BaseRunnable struct {
	// contains filtered or unexported fields
}

func NewBaseRunnable 

func NewBaseRunnable(o BaseRunnableOptions) *BaseRunnable

func (*BaseRunnable) Dispatch 

func (r *BaseRunnable) Dispatch(m *Message)

func (*BaseRunnable) Metadata 

func (r *BaseRunnable) Metadata() Metadata

func (*BaseRunnable) NewTask 

func (r *BaseRunnable) NewTask() *astikit.Task

func (*BaseRunnable) OnMessage 

func (r *BaseRunnable) OnMessage(m *Message) (err error)

func (*BaseRunnable) RootCtx 

func (r *BaseRunnable) RootCtx() context.Context

func (*BaseRunnable) SetDispatchFunc 

func (r *BaseRunnable) SetDispatchFunc(f DispatchFunc)

func (*BaseRunnable) SetRootCtx 

func (r *BaseRunnable) SetRootCtx(ctx context.Context)

func (*BaseRunnable) SetTaskFunc 

func (r *BaseRunnable) SetTaskFunc(f astikit.TaskFunc)

func (*BaseRunnable) Start 

func (r *BaseRunnable) Start(ctx context.Context) (err error)

func (*BaseRunnable) Status 

func (r *BaseRunnable) Status() string

func (*BaseRunnable) Stop 

func (r *BaseRunnable) Stop()

type BaseRunnableOptions 

type BaseRunnableOptions struct {
	Logger    astikit.StdLogger
	Metadata  Metadata
	OnMessage func(m *Message) error
	OnStart   func(ctx context.Context) error
}

type DispatchConditions 

type DispatchConditions struct {
	From  *Identifier
	Name  *string
	Names map[string]bool
	To    *Identifier
}

type DispatchFunc 

type DispatchFunc func(m *Message)

type Dispatcher 

type Dispatcher struct {
	// contains filtered or unexported fields
}

func NewDispatcher 

func NewDispatcher(ctx context.Context, t astikit.TaskFunc, l astikit.SeverityLogger) *Dispatcher

func (*Dispatcher) Close 

func (d *Dispatcher) Close()

func (*Dispatcher) Dispatch 

func (d *Dispatcher) Dispatch(m *Message)

func (*Dispatcher) On 

func (d *Dispatcher) On(c DispatchConditions, h MessageHandler)

type Error 

type Error struct {
	Message string `json:"message"`
}

type Identifier 

type Identifier struct {
	Name   *string         `json:"name,omitempty"`
	Type   string          `json:"type,omitempty"`
	Types  map[string]bool `json:"types,omitempty"`
	Worker *string         `json:"worker,omitempty"`
}

func NewIndexIdentifier 

func NewIndexIdentifier() *Identifier

func NewRunnableIdentifier 

func NewRunnableIdentifier(runnable, worker string) *Identifier

func NewUIIdentifier 

func NewUIIdentifier(name string) *Identifier

func NewWorkerIdentifier 

func NewWorkerIdentifier(name string) *Identifier

func (Identifier) Clone 

func (i Identifier) Clone() (o *Identifier)

func (Identifier) WorkerName 

func (i Identifier) WorkerName() string

type Listenable 

type Listenable interface {
	MessageNames() []string
	OnMessage(m *Message) error
}

type Listenables 

type Listenables struct {
	Names    []string `json:"names"`
	Runnable string   `json:"runnable"`
}

func ParseListenablesRegisterPayload 

func ParseListenablesRegisterPayload(m *Message) (l Listenables, err error)

type Message 

type Message struct {
	From    Identifier      `json:"from"`
	ID      int             `json:"id,omitempty"`
	Name    string          `json:"name"`
	Payload json.RawMessage `json:"payload,omitempty"`
	To      *Identifier     `json:"to,omitempty"`
}

func NewListenablesRegisterMessage 

func NewListenablesRegisterMessage(from Identifier, to *Identifier, l Listenables) (m *Message, err error)

func NewMessage 

func NewMessage() *Message

func NewRunnableCrashedMessage 

func NewRunnableCrashedMessage(from Identifier, to *Identifier) *Message

func NewRunnableDoneMessage 

func NewRunnableDoneMessage(to *Identifier, d RunnableDone) (m *Message, err error)

func NewRunnableStartedMessage 

func NewRunnableStartedMessage(from Identifier, to *Identifier) *Message

func NewRunnableStoppedMessage 

func NewRunnableStoppedMessage(from Identifier, to *Identifier) *Message

func NewUIDisconnectedMessage 

func NewUIDisconnectedMessage(from Identifier, to *Identifier, name string) (m *Message, err error)

func NewUIMessageNamesAddMessage 

func NewUIMessageNamesAddMessage(from Identifier, to *Identifier, names []string) (m *Message, err error)

func NewUIMessageNamesDeleteMessage 

func NewUIMessageNamesDeleteMessage(from Identifier, to *Identifier, names []string) (m *Message, err error)

func NewUIWelcomeMessage 

func NewUIWelcomeMessage(from Identifier, to *Identifier, w WelcomeUI) (m *Message, err error)

func NewWorkerDisconnectedMessage 

func NewWorkerDisconnectedMessage(from Identifier, to *Identifier, worker string) (m *Message, err error)

func NewWorkerRegisterMessage 

func NewWorkerRegisterMessage(from Identifier, to *Identifier, w Worker) (m *Message, err error)

func NewWorkerRegisteredMessage 

func NewWorkerRegisteredMessage(from Identifier, to *Identifier, w Worker) (m *Message, err error)

func NewWorkerWelcomeMessage 

func NewWorkerWelcomeMessage(from Identifier, to *Identifier, w WelcomeWorker) (m *Message, err error)

func (*Message) Clone 

func (m *Message) Clone() (o *Message)

type MessageHandler 

type MessageHandler func(m *Message) error

type Metadata 

type Metadata struct {
	Description string `json:"description"`
	Name        string `json:"name"`
}

type Operatable 

type Operatable interface {
	Routes() map[string]map[string]httprouter.Handle // Indexed by path --> method
	Templates() map[string][]byte                    // Indexed by name
}

type Runnable 

type Runnable interface {
	Metadata() Metadata
	OnMessage(m *Message) error
	SetDispatchFunc(f DispatchFunc)
	SetRootCtx(ctx context.Context)
	SetTaskFunc(f astikit.TaskFunc)
	Start(ctx context.Context) error
	Status() string
	Stop()
}

type RunnableDone 

type RunnableDone struct {
	ID      int  `json:"id"`
	Success bool `json:"success"`
}

func ParseRunnableDonePayload 

func ParseRunnableDonePayload(m *Message) (d RunnableDone, err error)

type RunnableMessage 

type RunnableMessage struct {
	Metadata
	Status      string `json:"status"`
	WebHomepage string `json:"web_homepage,omitempty"`
}

type ServerOptions 

type ServerOptions struct {
	Addr     string `toml:"addr"`
	Password string `toml:"password"`
	Username string `toml:"username"`
}

type UI 

type UI struct {
	MessageNames []string `json:"message_names"`
	Name         string   `json:"name"`
}

func ParseUIRegisterPayload 

func ParseUIRegisterPayload(m *Message) (u UI, err error)

type WelcomeUI 

type WelcomeUI struct {
	Name    string   `json:"name"`
	Workers []Worker `json:"workers,omitempty"`
}

type WelcomeWorker 

type WelcomeWorker struct {
	UIMessageNames []string `json:"ui_message_names,omitempty"`
	Workers        []Worker `json:"workers,omitempty"`
}

func ParseWorkerWelcomePayload 

func ParseWorkerWelcomePayload(m *Message) (w WelcomeWorker, err error)

type Worker 

type Worker struct {
	Addr      string            `json:"addr,omitempty"`
	Name      string            `json:"name"`
	Runnables []RunnableMessage `json:"runnables,omitempty"`
}

func ParseWorkerRegisterPayload 

func ParseWorkerRegisterPayload(m *Message) (w Worker, err error)

func ParseWorkerRegisteredPayload 

func ParseWorkerRegisteredPayload(m *Message) (w Worker, err error)

Source Files

View all Source files

Directories

Path Synopsis
abilities
audio_input
Generated by cmd/operatable DO NOT EDIT
 Generated by cmd/operatable DO NOT EDIT
audio_input/portaudio
audio_input/portaudio/cmd
speech_to_text
Generated by cmd/operatable DO NOT EDIT
 Generated by cmd/operatable DO NOT EDIT
speech_to_text/deepspeech
text_to_speech
text_to_speech/speak
cmd
index
operatable

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.