e4

README

• Introduction
• Using our client application
• Creating a client
  • Symmetric-key client
  • Public-key client
  • From a saved state
  • Client storage
• Integration instructions
  • Receiving a message
  • Transmitting a message
  • Handling errors
  • Key generation
  • Bindings
    • Android
• Contributing
• Security
• Support
• Intellectual property

Introduction

This repository provides the e4 Go package, the client library for Teserakt's E4, and end-to-end encryption and key management framework for MQTT and other publish-subscribe protocols.

The e4 package defines a Client object that has a minimal interface, making its integration straightforward via the following methods:

• ProtectMessage(payload []byte, topic string) takes a cleartext payload to protect and the associated topic, and returns a []byte that is the payload encrypted and authenticated with the topic's key.

• Unprotect(protected []byte, topic string) takes a protected payload and attempts to decrypt and verify it. If topic is the special topic reserved for control messages, then the control message is processed and the client's state updated accordingly.

Note that we talk of message protection instead of just encryption because the protection operation includes also authentication and replay defense. The unprotection operation thus involves decryption and additional checks, and includes the processing of control messages sent by the server.

E4's server (C2) is necessary to send control messages and manage a fleet of clients through GUIs, APIs, and automation components. The server can for example deploy key rotation policies, grant and revoke rights, and enable forward secrecy.

Please contact us to request access to a private instance of the server, or test the limited public version. Without the C2 server, the E4 client library can be used to protect messages using static keys, manually managed.

Using our client application

To try E4 without writing your own application, we created a simple interactive client application that you can use in combination with our public demo server interface. You can directly download the client's binary for your platform or build it yourself, and then follow the instructions in the client's README.

Creating a client

The following instructions assume that your program imports e4 as follows:

    import e4 "github.com/teserakt-io/e4go"

The E4 protocol supports both symmetric key and public-key mode. Depending on the mode, different functions should be used to instantiate a client:

Symmetric-key client

A symmetric-key client can be created from a 16-byte identifier (type []byte), a 32-byte key (type []byte), and an e4.ReadWriteSeeker implementation, used to persist the client's state (see client storage section for details):

    client, err := e4.NewClient(&e4.SymIDAndKey{ID: id, Key: key}, store)

A symmetric-key client can also be created from a name (string of arbitrary length) and a password (string of a least 16 characters), as follows:

    client, err := e4.NewClient(&e4.SymNameAndPassword{Name: name, Password: password}, store)

The latter is a wrapper over NewSymKeyClient() that creates the ID by hashing name with SHA-3-256, and deriving a key using Argon2.

Public-key client

A public-key client can be created from a 16-byte identifier (type []byte), an Ed25519 private key (type ed25519.PrivateKey), an e4.ReadWriteSeeker implementation, which will be used to store the client's state (see client storage section for details), and a Curve25519 public key (32-byte []byte):

client, err := e4.NewClient(&e4.PubIDAndKey{ID:id, Key: key, C2PubKey: c2PubKey}, store)

Compared to the symmetric-key mode, and additional argument is c2PubKey, the public key of the C2 server that sends control messages.

A public-key client can also be created from a name (string of arbitrary length) and a password (string of a least 16 characters), as follows:

client, err := e4.NewClient(&e4.PubNameAndPassword{Name:name, Password: password, C2PubKey: c2PubKey}, store)

The Ed25519 private key is then created from a seed that is derived from the password using Argon2. The Ed25519 public key can also be retrieved:

config := &e4.PubNameAndPassword{Name:name, Password: password, C2PubKey: c2PubKey}
pubKey, err := config.PubKey()

From a saved state

A client instance can be recovered using the LoadClient() helper given an e4.ReadWriteSeeker implementation::

    client, err := e4.LoadClient(store)

Note that a client's state is automatically saved to the provided store when the client is created, and every time its state changes, and therefore does not need be manually saved.

Client storage

E4 client offer a way to persist its internal state, allowing to shut it down and reload without having to retransmit all the keys, by providing an e4.ReadWriteSeeker implementation to the client. This interface is compatible with any io.ReadWriteSeeker, such as the os.File type, which should be the most common option. But it also allows custom implementations for platforms where filesystem isn't available, see the e4.NewInMemoryStore([]byte) we provide as an example of custom storage implementation.

Integration instructions

To integrate E4 into your application, the protect/unprotect logic needs be added between the network layer and the application layer when transmitting/receiving a message.

This section provides further instructions related to error handling and to the special case of control messages received from the C2 server.

Note that E4 is essentially an application security layer, therefore it processes the payload of a message (such as an MQTT payload), excluding header fields. References to "messages" below therefore refer to payload data (or application message),as opposed to the network-level message.

Receiving a message

Assume that you receive messages over MQTT or Kafka, and have topics and payload defined as

    var topic string
    var message []byte

Having instantiated a client, you can then unprotect the message as follows:

    plaintext, err := client.Unprotect(message, topic)
    if err != nil {
        // your error reporting here
    }

If you receive no error, plaintext may still be nil. This happens when E4 has processed a control message, that is, a message sent by the C2 server, for example to provision or delete a topic key. In this case, you do not need to act on the message, since E4 has already processed it. If you want to detect this case you can test for

    if len(plainText) == 0 { ... }

or alternatively

    if client.IsReceivingTopic(topic)

which indicates a message on E4's control channel. You should not have to parse E4's messages yourself. Control messages are thus deliberately not returned to users.

If plaintext is not nil and err is nil, your application can proceed with the unprotected, plaintext message.

Transmitting a message

To protect a message to be transmitted, suppose say that you have the topic and payload defined as:

    var topic string
    var message []byte

You can then use the Protect method from the client instance as follows:

    protected, err := client.Protect(message, topic)
    if err != nil {
        // your error reporting here
    }

Handling errors

All errors should be reported, and the plaintext and protected values discarded upon an error, except potentially in one case: if you receive an ErrTopicKeyNotFound error from ProtectMessage() or Unprotect(), it is because the client does not have the key for this topic. Therefore,

• When transmitting a message, your application can either discard the message to be sent, or choose to transmit it in clear.

• When receiving a message, your application can either discard the message (for example if all messages are assumed to be encrypted in your network), or forward the message to the application (if you call Unprotect() for all messages yet tolerate the receiving of unencrypted messages over certain topics, which thus don't have a topic key).

In order to have the key associated to a certain topic, you must instruct the C2 to deliver said topic key to the client.

Key generation

To ease key creation, we provide a key generation application that you can use to generate symmetric, Ed25519 or Curve25519 keys needed for E4 operations. You can download the binary for your platform or build it yourself, and then follow the instructions in the keygen README.

Our key generator relies on Go's crypto/rand package, which guarantees cryptographically secure randomness across various platforms.

Bindings

Android

Latest bindings for Android can be downloaded from the release page. On an environment having an Android SDK and NDK available, an Android AAR package can be generated invoking the following script:

./scripts/android_bindings.sh

This will generate:

• dist/bindings/android/e4.aar: the Android package, containing compiled Java class and native libraries for most common architectures
• dist/bindings/android/e4-sources.jar: the Java source files

After importing the AAR in your project, E4 client can be created and invoked in a similar way than the Go version, for example using Kotlin:

import java.io.RandomAccessFile

import io.teserakt.e4.E4
import io.teserakt.e4.SymNameAndPassword
import io.teserakt.crypto.Crypto

val cfg = SymNameAndPassword()
cfg.name = "deviceXYZ"
cfg.password = "secretForDeviceXYZ"

val store = FileStore(filesDir.absolutePath + "/" + cfg.name + ".json")
val client = E4.newClient(cfg, store)

// From here, messages can be protected / unprotected :
val topic = "/deviceXYZ/data";
val protectedMessage = client.protectMessage("Hello".toByteArray(Charsets.UTF_8), topic)
val unprotectedMessage = client.unprotect(protectedMessage, topic)

Here We are using a custom file storage implemented as such:

import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.RandomAccessFile;

import io.teserakt.e4.Store;

public class FileStore implements Store {
    private static final int SEEK_START = 0;
    private static final int SEEK_CURRENT = 1;
    private static final int SEEK_END = 2;

    private RandomAccessFile file;

    public FileStore(String filepath) throws FileNotFoundException {
        this.file = new RandomAccessFile(filepath, "rw");
    }

    public long read(byte[] buf) throws IOException {
        return this.file.read(buf);
    }

    public long write(byte[] buf) throws IOException {
        this.file.write(buf);
        return buf.length;
    }

    public long seek(long offset, long whence) throws Exception {
        long abs;
        switch((int)whence) {
            case SEEK_START:
                abs = offset;
                break;
            case SEEK_CURRENT:
                abs = this.file.getChannel().position() + offset;
                break;
            case SEEK_END:
                abs = this.file.length() + offset;
                break;
            default:
                throw new Exception("invalid whence");
        }
        if (abs < 0) {
            throw new Exception("negative position");
        }

        this.file.getChannel().position(abs);
        return abs;
    }
}

Contributing

Before contributing, please read our CONTRIBUTING guide.

Security

To report a security vulnerability (or potential vulnerability where private discussion is preferred) see SECURITY.

Support

To request support, please contact team@teserakt.io.

Intellectual property

e4go is copyright (c) Teserakt AG 2018-2020, and released under Apache 2.0 License (see LICENCE).

Documentation

Overview

Package e4 provides a e4 client implementation and libraries.

It aims to be quick and easy to integrate in IoT devices applications enabling to secure their communications, as well as exposing a way to manage the various keys required.

Protecting and unprotecting messages

Once created, a client provide methods to protect messages before sending them to the broker:

protectedMessage, err := client.ProtectMessage([]byte("secret message"), topicKey)

or unprotecting the messages it receives.

originalMessage, err := client.Unprotect([]byte(protectedMessage, topicKey))

ReceivingTopic and client commands

A special topic (called ReceivingTopic) is reserved to communicate protected commands to the client. Such commands are used to update the client state, like setting a new key for a topic, or renewing its private key. There is nothing particular to be done when receiving a command, just passing its protected form to the Unprotect() method and the client will automatically unprotect and process it (thus returning no unprotected message). See commands.go for the list of available commands and their respective parameters.

Index

• Constants
• Variables
• func CmdRemovePubKey(name string) ([]byte, error)
• func CmdRemoveTopic(topic string) ([]byte, error)
• func CmdResetPubKeys() ([]byte, error)
• func CmdResetTopics() ([]byte, error)
• func CmdSetC2Key(c2PubKey e4crypto.Curve25519PublicKey) ([]byte, error)
• func CmdSetIDKey(key []byte) ([]byte, error)
• func CmdSetPubKey(pubKey e4crypto.Ed25519PublicKey, name string) ([]byte, error)
• func CmdSetTopicKey(topicKey []byte, topic string) ([]byte, error)
• func TopicForID(id []byte) string
• type Client
  • func LoadClient(store ReadWriteSeeker) (Client, error)
  • func NewClient(config ClientConfig, store ReadWriteSeeker) (Client, error)
• type ClientConfig
• type PubIDAndKey
• type PubNameAndPassword
  • func (np *PubNameAndPassword) PubKey() (e4crypto.Ed25519PublicKey, error)
• type ReadWriteSeeker
  • func NewInMemoryStore(buf []byte) ReadWriteSeeker
• type SymIDAndKey
• type SymNameAndPassword

Examples

• NewClient (FileStorage)
• NewClient (PubIDAndKey)
• NewClient (PubNameAndPassword)
• NewClient (SymIDAndKey)
• NewClient (SymNameAndPassword)

Constants

const (
	// RemoveTopic command allows to remove a topic key from the client.
	// It expects a topic hash as argument
	RemoveTopic byte = iota
	// ResetTopics allows to clear out all the topics on a client.
	// It doesn't have any argument
	ResetTopics
	// SetIDKey allows to set the private key of a client.
	// It expects a key as argument
	SetIDKey
	// SetTopicKey allows to add a topic key on the client.
	// It takes a key, followed by a topic hash as arguments.
	SetTopicKey
	// RemovePubKey allows to remove a public key from the client.
	// It takes the ID to be removed as argument
	RemovePubKey
	// ResetPubKeys removes all public keys stored on the client.
	// It expects no argument
	ResetPubKeys
	// SetPubKey allows to set a public key on the client.
	// It takes a public key, followed by an ID as arguments.
	SetPubKey
	// SetC2PubKey replaces the current C2 public key with the newly transmitted one.
	SetC2Key

	// UnknownCommand must stay the last element. It's used to
	// know if a Command is out of range
	UnknownCommand = 0xFF
)

List of supported commands

Variables

var (
	// ErrTopicKeyNotFound occurs when a topic key is missing when encryption/decrypting
	ErrTopicKeyNotFound = errors.New("topic key not found")
	// ErrUnsupportedOperation occurs when trying to manipulate client public keys with a ClientKey not supporting it
	ErrUnsupportedOperation = errors.New("this operation is not supported")
)

var (
	// ErrInvalidCommand is returned when trying to process an unsupported command
	ErrInvalidCommand = errors.New("invalid command")
)

Functions

func CmdRemovePubKey

func CmdRemovePubKey(name string) ([]byte, error)

CmdRemovePubKey creates a command to remove the public key identified by given name from the client

func CmdRemoveTopic

func CmdRemoveTopic(topic string) ([]byte, error)

CmdRemoveTopic creates a command to remove the key associated with the topic, from the client

func CmdResetPubKeys

func CmdResetPubKeys() ([]byte, error)

CmdResetPubKeys creates a command to removes all public keys from the client

func CmdResetTopics

func CmdResetTopics() ([]byte, error)

CmdResetTopics creates a command to remove all topic keys stored on the client

func CmdSetC2Key

func CmdSetC2Key(c2PubKey e4crypto.Curve25519PublicKey) ([]byte, error)

CmdSetC2Key creates a command to replace the c2 public key by the given one.

func CmdSetIDKey

func CmdSetIDKey(key []byte) ([]byte, error)

CmdSetIDKey creates a command to set the client private key to the given key

func CmdSetPubKey

func CmdSetPubKey(pubKey e4crypto.Ed25519PublicKey, name string) ([]byte, error)

CmdSetPubKey creates a command to set a given public key, identified by given name on the client

func CmdSetTopicKey

func CmdSetTopicKey(topicKey []byte, topic string) ([]byte, error)

CmdSetTopicKey creates a command to set the given topic key and its corresponding topic, on the client

func TopicForID

func TopicForID(id []byte) string

TopicForID generate the receiving topic that a client should subscribe to in order to receive commands

Types

type Client

type Client interface {
	// ProtectMessage will encrypt the given payload using the key associated to topic.
	// When the client doesn't have a key for this topic, ErrTopicKeyNotFound will be returned.
	// When no errors, the protected cipher bytes are returned
	ProtectMessage(payload []byte, topic string) ([]byte, error)
	// Unprotect attempts to decrypt the given cipher using the topic key.
	// When the client doesn't have a key for this topic, ErrTopicKeyNotFound will be returned.
	// When no errors, the clear payload bytes are returned, unless the protected message was a client command.
	// Message are client commands when received on the client receiving topic. The command will be processed
	// when unprotecting it, making a nil,nil response indicating a success
	Unprotect(protected []byte, topic string) ([]byte, error)
	// IsReceivingTopic returns true when the given topic is the client receiving topics.
	// Message received from this topics will be protected commands, meant to update the client state
	IsReceivingTopic(topic string) bool
	// GetReceivingTopic returns the receiving topic for this client, which will be used to transmit commands
	// allowing to update the client state, like setting a new private key or adding a new topic key.
	GetReceivingTopic() string
	// contains filtered or unexported methods
}

Client defines interface for protecting and unprotecting E4 messages and commands

func LoadClient

func LoadClient(store ReadWriteSeeker) (Client, error)

LoadClient loads a client state from the file system

func NewClient

func NewClient(config ClientConfig, store ReadWriteSeeker) (Client, error)

NewClient creates a new E4 client, working either in symmetric key mode, or public key mode depending the given ClientConfig

config is a ClientConfig, either SymIDAndKey, SymNameAndPassword, PubIDAndKey or PubNameAndPassword store is an e4.ReadWriteSeeker implementation

Example (FileStorage)

package main

import (
	"fmt"
	"os"

	e4 "github.com/teserakt-io/e4go"
	e4crypto "github.com/teserakt-io/e4go/crypto"
)

func main() {
	f, err := os.OpenFile("/storage/clientID.json", os.O_CREATE|os.O_RDWR, 0600)
	if err != nil {
		panic(err)
	}
	defer f.Close()

	client, err := e4.NewClient(&e4.SymIDAndKey{
		ID:  []byte("clientID"),
		Key: e4crypto.RandomKey(),
	}, f)
	if err != nil {
		panic(err)
	}

	protectedMessage, err := client.ProtectMessage([]byte("very secret message"), "topic/name")
	if err != nil {
		panic(err)
	}
	fmt.Printf("Protected message: %v", protectedMessage)
}

Example (PubIDAndKey)

package main

import (
	"fmt"

	e4 "github.com/teserakt-io/e4go"
	e4crypto "github.com/teserakt-io/e4go/crypto"
	"golang.org/x/crypto/curve25519"
)

func main() {
	privateKey, err := e4crypto.Ed25519PrivateKeyFromPassword("verySecretPassword")
	if err != nil {
		panic(err)
	}

	c2PubKey, err := curve25519.X25519(e4crypto.RandomKey(), curve25519.Basepoint)
	if err != nil {
		panic(err)
	}

	client, err := e4.NewClient(&e4.PubIDAndKey{
		ID:       []byte("clientID"),
		Key:      privateKey,
		C2PubKey: c2PubKey,
	}, e4.NewInMemoryStore(nil))

	if err != nil {
		panic(err)
	}

	protectedMessage, err := client.ProtectMessage([]byte("very secret message"), "topic/name")
	if err != nil {
		panic(err)
	}
	fmt.Printf("Protected message: %v", protectedMessage)
}

Example (PubNameAndPassword)

package main

import (
	"fmt"

	e4 "github.com/teserakt-io/e4go"
	e4crypto "github.com/teserakt-io/e4go/crypto"
	"golang.org/x/crypto/curve25519"
)

func main() {
	c2PubKey, err := curve25519.X25519(e4crypto.RandomKey(), curve25519.Basepoint)
	if err != nil {
		panic(err)
	}

	config := &e4.PubNameAndPassword{
		Name:     "clientName",
		Password: "verySecretPassword",
		C2PubKey: c2PubKey,
	}
	client, err := e4.NewClient(config, e4.NewInMemoryStore(nil))
	if err != nil {
		panic(err)
	}

	// We may need to get the public key derived from the password:
	pubKey, err := config.PubKey()
	if err != nil {
		panic(err)
	}
	fmt.Printf("Client public key: %x", pubKey)

	protectedMessage, err := client.ProtectMessage([]byte("very secret message"), "topic/name")
	if err != nil {
		panic(err)
	}
	fmt.Printf("Protected message: %v", protectedMessage)
}

Example (SymIDAndKey)

package main

import (
	"fmt"

	e4 "github.com/teserakt-io/e4go"
	e4crypto "github.com/teserakt-io/e4go/crypto"
)

func main() {
	client, err := e4.NewClient(&e4.SymIDAndKey{
		ID:  []byte("clientID"),
		Key: e4crypto.RandomKey(),
	}, e4.NewInMemoryStore(nil))
	if err != nil {
		panic(err)
	}

	protectedMessage, err := client.ProtectMessage([]byte("very secret message"), "topic/name")
	if err != nil {
		panic(err)
	}
	fmt.Printf("Protected message: %v", protectedMessage)
}

Example (SymNameAndPassword)

package main

import (
	"fmt"

	e4 "github.com/teserakt-io/e4go"
)

func main() {
	client, err := e4.NewClient(&e4.SymNameAndPassword{
		Name:     "clientName",
		Password: "verySecretPassword",
	}, e4.NewInMemoryStore(nil))
	if err != nil {
		panic(err)
	}

	protectedMessage, err := client.ProtectMessage([]byte("very secret message"), "topic/name")
	if err != nil {
		panic(err)
	}
	fmt.Printf("Protected message: %v", protectedMessage)
}

type ClientConfig

type ClientConfig interface {
	// contains filtered or unexported methods
}

ClientConfig defines an interface for client configuration

type PubIDAndKey

type PubIDAndKey struct {
	ID       []byte
	Key      e4crypto.Ed25519PrivateKey
	C2PubKey e4crypto.Curve25519PublicKey
}

PubIDAndKey defines a configuration to create an E4 client in public key mode from an ID, an ed25519 private key, and a curve25519 public key.

type PubNameAndPassword

type PubNameAndPassword struct {
	Name     string
	Password string
	C2PubKey e4crypto.Curve25519PublicKey
}

PubNameAndPassword defines a configuration to create an E4 client in public key mode from a name, a password and a curve25519 public key. The password must contains at least 16 characters.

func (*PubNameAndPassword) PubKey

func (np *PubNameAndPassword) PubKey() (e4crypto.Ed25519PublicKey, error)

PubKey returns the ed25519.PublicKey derived from the password

type ReadWriteSeeker

type ReadWriteSeeker interface {
	io.ReadWriteSeeker
}

ReadWriteSeeker is a redefinition of io.ReadWriteSeeker to ensure that gomobile bindings still get generated without incompatible type removals

func NewInMemoryStore

func NewInMemoryStore(buf []byte) ReadWriteSeeker

NewInMemoryStore creates a new ReadWriteSeeker in memory

type SymIDAndKey

type SymIDAndKey struct {
	ID  []byte
	Key []byte
}

SymIDAndKey defines a configuration to create an E4 client in symmetric key mode from an ID and a symmetric key

type SymNameAndPassword

type SymNameAndPassword struct {
	Name     string
	Password string
}

SymNameAndPassword defines a configuration to create an E4 client in symmetric key mode from a name and a password. The password must contains at least 16 characters.