beaconpi

README

Beaconpi

A system for monitoring Beacons using Raspberry Pis. Created as a part of a Percom 2018 Demo Paper submission. Linked here. The intention is to build on this system to make location predictions more accurate.

Components

Beaconpi requires many physical and software components. We give a high level overview of what each component is in this section.

• Beacons Currently supported beacons include only the Apple iBeacon of any brand.
• Edges Edges receive the announcements from the beacons and relay them onto the beacon server. This is accomplished using Raspberry Pis.
• Beacon Client The edges use the beacon client, which communicates with the beacon
  server to get control commands and relay the beacon sightings
• Beacon Server Manages the data from the edge units and the database.
• Database SQL database the efficiently indexs and stores all beacon data.
• Metrics Server Serves data from the database
• Web Client Serves data from the Metrics Server into visual format

Requirements

Requirements are software/hardware requirements for the entire system. The software should be installed before proceeding.

• Hardware Requirements
  • Raspberry Pis, with bluetooth and some network connectivity, Raspberry Pi 3 works best as it has built in Bluetooth and WiFi.
  • Centralized server (application server) with network open to Raspberry Pis, cloud is acceptable.
• Software Requirements
  • Go compiler (1.7+) suggested on both the application server and clients (Raspberry Pi), we can cross compile but updates are easier if the client has the compiler too.
  • Nodejs & npm used for the web interface
  • Raspbian OS for the Raspberry Pis, you should use the minimal image (lite)
  • bluez Install on Raspberry Pis to provide hcitool
  • bluez-hcidump
    Install on Raspberry Pis to provide hcidump which is depreciated in bluez; this requirement will be removed in future versions
  • Application Server can run any Unix-like OS that supports Go
  • Postgres Sql 9.5.10+ for data storage and persistant configuration. Install to application server only.
  • Python 3.5+ for metricsserver components tracking components

Configuration Requirements

• Raspberry Pis must be altered to allow non root users to use hcitool the following command

  sudo setcap 'cap_net_raw,cap_net_admin+eip' $(which hcitool)
  

  satisifies this requirement.

Build Requirements

• GNU Make (recommended install requirement)
• git (recommended install requirement)
• OpenSSL
• Node and NPM (optional for webinterface)

Install

The instructions below are suggestions. Make any directories that don't already exist. These instructions do not walk you through basic Linux commands so intermediate understanding of Unix-like systems is required. Some helper programs assume $GOPATH is set, you should set it if not already set. It should match your Go path. By default Go uses $HOME/go. The database schema(tables) must be created before the application is run.

1. Postgres must be running, check with your OS vendor how to do this. In some cases sudo systemctl start postgres is sufficient but will vary across vendors.
2. CREATE DATABASE command or createdb program can be used to make the database. You can use any authentication that lib/pq supports but my suggestion is to use trust authentication for simplicity and should be as secure as your user on the server. Use createdb beacons or any other name you would like.
3. Once your database is created you need to apply the schema. Use the script etc/devtools/setup-db.sh to apply migrations, it uses the environment.cfg file in order to connect to the database
4. Compile the Go programs. Copy this repo to your $GOPATH/src/github.com/co60ca/beaconpi directory. Copying this repo can also be done with go get github.com/co60ca/beaconpi. Then run make all from the beaconpi directory. Make will build the programs to build/ relative to the beaconpi directory. It will not beaconclient program to arm64 which is the target platform for Raspberry Pi 3. You will need to do this on your target edge nodes.
5. Copy beaconclient to your cluster of Raspberry Pis, you can put this anywhere you like as the application relies on no relative pathes. A possible suggestion is to put it in the $GOPATH/bin directory as some helper scripts are designed to expect that.
6. Copy beaconserver to the application server. Copy etc/devtools/start-beacon-server.sh and etc/devtools/environment.cfg to the application server as well. The configuration for the database is located in environment.cfg and defaults to password based authentication. Check lib/pq for more options.
7. Copy metricsserver to the application server. Suggested location is the same. Additionally copy the etc/devtools/start-web.sh and etc/devtools/environment.cfg
8. Generate security keys. Change directory to etc/x509. You may generate the client and server keys by using the generate-keys.sh helper script however you should only use the ./generate-keys.sh newserver mode of the script. Client keys are made in the next step automatically. Generating a new server key will invalidate all prior keys.
9. Generate client keys. Change directory to etc/client-maker. Use setup-client.sh from the same machine you generated the new server keys. Generate new clients by running ./setup-client.sh <name of client> the name of the client can be anything. For simplicity you should just use numbers. To generate many at once you can use a bash for loop.

   # Creates 20 clients
   for i in {0..19} ; do ./setup-client.sh ${i} ; done
   

   The helper script creates folders with the required configure files for the client start helper script start-client.sh.
10. Copy the client files to each of the edge units. Ensure the files in each folder and the start-client.sh are in the same directory on each of the edge units.

Web Install

The whole system can be operated only using the sql database for reporting however the JavaScript/Web interface is build using Node/npm. Before running the below command(s) change the target directory to the host/port of your application server system. Use https:// if available. Open etc/beaconpi-react/src/config.js and change

const home = '<url>'

to

const home = 'http(s)://<fully qualified domain name>:<port>'

to the root of where your web server will be hosted. Then change

const app = 'http(s)://<fully qualified domain name>:<port>'

to the path where your app server will be hosted.

Finally install required 3rd party libraries and build the bundle

# Get required packages and build bundle
npm install && npm run build

then serve this bundle in build/ with any webserver (even Github Pages!)

Required Configuration

Version 2 has introduced beaconpi-react which is a very easy to use web interface, it should allow you to add beacons, edges, and users to the system without using SQL. For the Lateration tab SQL is still required as no system admin page has been made for the MapConfigs yet.

All users in the current version are admins and have full access to the system, the first user must be made in SQL unfortunatly. To do so:

insert into webauth_users 
  (displayname, email, password, active) values
  ('<your displayname>', '<your email>', <password from next step>, 1)

the password must be hashed in advance, since we use github.com/co60ca/webauth we can use the password entry tool from that application. So compile go install src/github.com/co60ca/webauth/passgen then run passgen from $GOPATH/bin which will ask you for your password then give you the exact text to put in the blank in the step prior.

Once you have the user created you can login to the webinterface once started. The system can operate with no data in the database.

1. Beacons - Simply add the iBeacon settings under the Admin -> Beacon tab
2. Edges - Simply add the Edge settings under the Admin -> Edge tab, the UUID for each edge is given in the /client-options.cfg for each edge

Starting Everything

The clients will fail and retry to connect to the server so as long as the database is up any order is permitted. However this the supported startup sequence.

1. Start the database. Check with your OS vendor on how to do this. You probably want this to start with the application server OS.
2. Start the beaconserver. Use ./start-beacon-server.sh with the correct configuration set in this file or use beaconserver directly with the correct command line arguments. See beaconserver --help for the arguments.
3. Start each of the clients. You probably want to configure this to start with each of the clients. Use ./start-client.sh to do so.
4. Start the metricserver. Simply run ./start-metrics-server.sh.
5. Start your webserver for the client facing code.

Documentation

Index

• Constants
• func IBeaconListener(validbeacons []BeaconData, brs chan BeaconRecord)
• func LoadFileToCert(file string) *x509.CertPool
• func StartClient()
• func StartServer(x509cert, x509key, drivername, dsn string, end chan struct{})
• type BeaconData
  • func (b *BeaconData) MarshalBinary() ([]byte, error)
  • func (b *BeaconData) String() string
  • func (b *BeaconData) UnmarshalBinary(data []byte) error
• type BeaconLog
  • func (b *BeaconLog) MarshalBinary() ([]byte, error)
  • func (b *BeaconLog) UnmarshalBinary(data []byte) error
• type BeaconLogPacket
  • func (b *BeaconLogPacket) MarshalBinary() ([]byte, error)
  • func (b *BeaconLogPacket) UnmarshalBinary(data []byte) error
• type BeaconRecord
• type BeaconResponsePacket
  • func (b *BeaconResponsePacket) MarshalBinary() ([]byte, error)
  • func (b *BeaconResponsePacket) UnmarshalBinary(d []byte) error
• type ServerConfig
  • func GetFlags() (out ServerConfig)
• type Uuid
  • func UuidFromString(s string) (Uuid, error)
  • func (u Uuid) String() string

Constants

const (
	// Milliseconds
	TIMEOUT_BEACON_REFRESH = 60000
	TIMEOUT_BEACON         = 50
	BACKOFF_MAX            = 30 * time.Second
	BACKOFF_MIN            = 50 * time.Millisecond
	BACKOFF_MULTIPLIER     = 2
)

const (
	ERROR_TRACE = 0
	ERROR_DEBUG = 1
	ERROR_INFO  = 2
	ERROR_WARN  = 3
	ERROR_ERROR = 4
	ERROR_FATAL = 5
)

const (
	ERROR_NULL = iota
	ERROR_DESYNC
)

const (
	DEFAULT_PORT = "32969"
	MAX_BEACONS  = 256
	MAX_LOGS     = 256
	MAX_CTRL     = 65535
	// Based on the size of the encompassing types this is the max size
	// of a packet, all others should be dropped
	// 16 is for UUID, 1 is for Flags
	MAX_SIZE        = MAX_CTRL + MAX_LOGS*12 + MAX_BEACONS*20 + 16 + 1
	CURRENT_VERSION = 1
)

const (
	// binary mask that leaves the version of the packet protocol as is
	VERSION_MASK = 0x0F
	// any protocol failures
	RESPONSE_INVALID = 0x10
	// request is ok and will complete
	RESPONSE_OK = 0x20
	// should be returned if the server is rate limiting the client
	RESPONSE_TOOMANY = 0x40
	// the client should restart
	RESPONSE_RESTART = 0x80
	// the client should shutdown
	RESPONSE_SHUTDOWN = 0x100
	// the client should attempt to update its code and restart
	RESPONSE_UPDATE = 0x200
	// the client should accept the beacon list updates sent by the server
	RESPONSE_BEACON_UPDATES = 0x400
	// the server is notifying the client there is a problem on its side that
	// it cannot recover from
	RESPONSE_INTERNAL_FAILURE = 0x800
	// the client should run the command in its shell
	RESPONSE_SYSTEM = 0x8000

	// the client is requesting beacon updates from the server
	REQUEST_BEACON_UPDATES = 0x10
	// the client is ending a control log, i.e. the stdout from the system
	// response
	REQUEST_CONTROL_LOG = 0x20
	// the client is signalling that it has completed the control and the
	// server can stop sending it
	REQUEST_CONTROL_COMPLETE = 0x40
)

Functions

func IBeaconListener

func IBeaconListener(validbeacons []BeaconData, brs chan BeaconRecord)

IBeaconListener is public provider for BeaconRecords

func LoadFileToCert

func LoadFileToCert(file string) *x509.CertPool

LoadFileToCert is a helper that opens a file by string and returns a x509 cert pool giving a fatal error on failure

func StartClient

func StartClient()

Main entry point for the client app that is run on the edge devices

func StartServer

func StartServer(x509cert, x509key, drivername, dsn string, end chan struct{})

StartServer is the main interface for the BeaconServer

Types

type BeaconData

type BeaconData struct {
	Uuid  Uuid `json:"string"`
	Major uint16
	Minor uint16
}

20 Bytes corresponding to the iBeacon profile

func (*BeaconData) MarshalBinary

func (b *BeaconData) MarshalBinary() ([]byte, error)

MarshalBinary implements the encoding.BinaryMarshaler interface. Output is 20 bytes

func (*BeaconData) String

func (b *BeaconData) String() string

func (*BeaconData) UnmarshalBinary

func (b *BeaconData) UnmarshalBinary(data []byte) error

UnmarshalBinary implements the encoding.BinaryMarshaler interface.

type BeaconLog

type BeaconLog struct {
	Datetime time.Time
	Rssi     int16
	// Index of value within a packet
	BeaconIndex uint16
}

12 Bytes which represents one time series value

func (*BeaconLog) MarshalBinary

func (b *BeaconLog) MarshalBinary() ([]byte, error)

MarshalBinary implements the encoding.BinaryMarshaler interface. Output is 12 bytes

func (*BeaconLog) UnmarshalBinary

func (b *BeaconLog) UnmarshalBinary(data []byte) error

UnmarshalBinary implements the encoding.BinaryMarshaler interface.

type BeaconLogPacket

type BeaconLogPacket struct {
	// Request flags
	Flags uint8
	// Sender uuid
	// Edge UUID
	Uuid    Uuid
	Logs    []BeaconLog
	Beacons []BeaconData
	// Extra unstructed data
	ControlData string
}

BeaconLogPacket should be sent by clients to the server

func (*BeaconLogPacket) MarshalBinary

func (b *BeaconLogPacket) MarshalBinary() ([]byte, error)

MarshalBinary implements the encoding.BinaryMarshaler interface.

func (*BeaconLogPacket) UnmarshalBinary

func (b *BeaconLogPacket) UnmarshalBinary(data []byte) error

type BeaconRecord

type BeaconRecord struct {
	BeaconData
	Datetime time.Time
	Rssi     int16
}

BeaconRecord represents a complete Beacon record including ID data and time & power

type BeaconResponsePacket

type BeaconResponsePacket struct {
	// Response flags
	Flags uint16
	//LengthData uint32
	// Unstructered data
	Data string
}

BeaconResponsePacket is the response to the client from the server

func (*BeaconResponsePacket) MarshalBinary

func (b *BeaconResponsePacket) MarshalBinary() ([]byte, error)

MarshalBinary implements the encoding.BinaryMarshaler interface.

func (*BeaconResponsePacket) UnmarshalBinary

func (b *BeaconResponsePacket) UnmarshalBinary(d []byte) error

UnmarshalBinary implements the encoding.BinaryMarshaler interface.

type ServerConfig

type ServerConfig struct {
	X509cert string
	X509key  string
	// Database drivername, must be psql
	Drivername string
	// Database data source name
	DSN string
}

Required BeaconServer config

func GetFlags

func GetFlags() (out ServerConfig)

type Uuid

type Uuid [16]byte

func UuidFromString

func UuidFromString(s string) (Uuid, error)

func (Uuid) String

func (u Uuid) String() string