ezstack

README

ezstack = "Easy Zigbee Stack"

Goals:

• Provide the easiest to understand, high-quality & pragmatic Zigbee stack.
• Replace Zigbee2MQTT: bridge devices to be usable in Home Assistant via MQTT (& Hautomo, uses the same path).

ezstack is part of Hautomo, but is usable without Hautomo (it works as a library).

Why

I've been using Zigbee2MQTT for many years, but I've had too many issues so I decided I want a Go-based solution.

Other Golang Zigbee stack projects were too complex. Multi-repo projects that could've been packages to accomplish Zigbee comms. I found them hard to understand. Or it might've just been that things seemed complex when I didn't know about Zigbee enough. Well anyway, here we are now. Shimmering Bee didn't have MQTT integration (I wanted to just replace Zigbee2MQTT in my setup).

I also wanted to take full control of my network by understanding the low-level tooling so when I encounter problems I know where to look.

Current status

Supported devices: tested most Xiaomi Aqara sensors & IKEA Trådfri lights.

Works for my 30+ -node Zigbee network. I actually have two Zigbee networks to overcome the ~20-nodes-per-CC2531 limit & other issues. So multi-radio Zigbee support (it's mainly just MQTT topic partitioning and some udev rules for COM port naming robustness, though) can be considered to have first-class support.

I don't expect this project to be mature enough for stable use to other people very soon, because my goal is not support as many devices as Zigbee2MQTT (it supports absolutely massive amount of devices), but instead to support the things I need as cleanly as possible. I'll expect to keep refactoring without having to care if I break things for someone else.

This code is public to reciprocate for the help I've received. In the rare case that you're brave enough to run this code I'll gladly accept contributions, but I will not work very hard to add features/devices you'll need unless it benefits me as well.

Architecture

Basically: the CC2531 USB sticks runs as a standalone, autonomous coordinator - it is not just a radio API with RX/TX frames. Our Go-based coordinator package can therefore be thought of as the API for asking coordinator to do things / get data from it.

We want to get data from sensors and send commands to light bulbs etc. Therefore our ezstack does its thing by using the coordinator package, which in turn uses ZNP protocol to talk to the USB stick. The ZNP protocol builds on top of UNP framing. We use ZNP to ask the Texas Instruments' radio to usually send ZCL to the network.

Logical component interaction:

ezstack
└── Coordinator
    ├── ZCL
    └── ZNP
        └── UNP

UNP = Unified Network Processor (Interface): Texas Instruments' protocol for communicating with their Zigbee/Bluetooth/etc radios

ZNP = Zigbee Network Processor: low-level commands on top of UNP to send system or Zigbee commands to the Texas Instruments' Zigbee radio

ZCL = Zigbee Cluster Library: standardized message formats for features ("clusters") to turn on/off power, control lamp brightness etc.

Coordinator = Handles network node management (device asks to join the network), passes app-level messages to consumer (usually ezstack)

ezstack = Opens connection to the radio, Zigbee-level messaging, handles node registration, keeps a database of nodes and forwards app-level messages to consumer (usually ezhub).

ezhub = ("Easy Zigbee hub", separate README) is a higher-level component meant to offer abstractions for sensor devices, light bulbs etc. Receives Zigbee app-level messages and provides vendor-specific parsers to cleaner abstractions so your app can receive "new temperature measurement from sensor XYZ". It also implements Home Assistant integration.

ezhub logical component interaction:

ezhub
├── deviceadapters
├── ezstack
│   └── ...
└── homeassistantmqtt

ezstack works without ezhub, but ezhub needs ezstack.

A word on ZCL

Application-level things like sensor data and other end device commands (e.g. "set light bulb brighness") are communicated using ZCL which is a standardized framing structure/data format that Zigbee devices communicate with. ZCL tries to standardize things like attribute IDs and values for temperature readings and for controlling lights. Unfortunately ZCL fails to be a very good standard, and there are lots of manufacturer-specific quirks and therefore we need abstractions to hide the warts from the user.

A good example of these stupid differences:

• Xiaomi button sends:
  • single-click as generic power on/off but
  • double/triple/etc. click as manufacturer-specific attribute (specific examples of this in code walkthrough!).
• Xiaomi (same vendor) double-button sends single/double/triple/etc. clicks as genMultistateInput.
• Window/Door contact sensors sends generic power on/off, water leak detector acts as an alarm..
• IKEA remote doesn't send button clicks, but sends direct specific commands to control brightness and change scenes. So some remotes are intended to control specific device, and if you want to use them as generic remotes (to control something else), you've to translate "brightness+" to "up button". Even the IKEA remote's "scene change" uses a mystery command that isn't specified in ZCL.

If you want, you can read the ZCL spec.

Code walkthrough

Major pieces of functionality:

• Zigbee message comes in
• MQTT client (like Home Assistant) wants to send a command to Zigbee network
• ezhub advertises its device registry to Home Assistant
• Adapter definition for a button that needs abstractions to paper over its non-standard use of ZCL
  • The button sends single-click as off -> on events
  • The button sends >= 2 clicks as manufacturer-specific attribute
  • The same manufacturer's 2-button sensor sends single/double/triple/etc. clicks as genMultistateInput cluster (each button represented with different Zigbee endpoints), so whatever the right way to send button clicks is, at least one of them is wrong as there isn't consensus even with same manufacturer's same type of sensor.
  • I can't blame the manufacturer much, since honestly ZCL spec is not very good and I don't even know which cluster would be the most semantic one to generically represent single/double/triple/etc. clicks, let alone over sensor that has multiple buttons.
• Incoming Zigbee message and its MQTT transformation "end-to-end" tested (for same adapter as above)

Acknowledgements

Standing on the shoulders of giants, i.e. this project wouldn't be possible without these people.

ezstack is a fork (albeit a really-major re-write) of dyrkin's zigbee-steward (the project is on hiatus). See differences to zigbee-steward for rationale of fork.

For things unclear from zigbee-steward I also learned a lot from Shimmering Bee's zstack implementation.

Michael Stapelberg sent me his own Zigbee code to get inspiration from.

Roadmap

• Decouple ezstack more from Hautomo. It's mostly dependency-free, but not totally.
• Simplify dependency relationships between ezstack/... sub-packages. There is something hairy somewhere.
• ZCL data structures library needs more refactoring. Ideally the data structures would be generated from a specification (other than the ZCL spec .pdf..) instead of being manually written..
• Improve the ZCL binary serializer/deserializer. Maybe switch to restruct.

Really long-term goals:

• Instead of using CC2531 to be an autonomous coordinator, I would like to use it as RX/TX radio for Zigbee raw packets:
  • The radio firmware would be much simpler.
  • We wouldn't have to change CC2531 firmware to use different Zigbee protocol versions.
  • We'd get much more low-level control.
  • There are really silly node count limitations that I think are the result of the CC2531 having to keep too much state and do too much work.
  • It'd be much cleaner having the whole Zigbee stack in Go, instead of separate codebases in different languages running in different devices trying to accomplish one thing as a whole.

Alternative software

Other Go-based Zigbee projects.

For Texas Instruments' CC2531 family of radios:

• https://godoc.org/github.com/shimmeringbee/zstack (very active project)
• https://github.com/dyrkin/zigbee-steward (on hiatus)

Other:

• https://github.com/pauleyj/gobee (for XBee radio)

Additional reading

• Tasmota's Zigbee docs seem very understandable for explaining many Zigbee concepts!

Documentation

Overview

Easy Zigbee Stack - the goal is to be easiest-to-understand Zigbee codebase available.

Index

• Constants
• type Channels
  • func (c *Channels) OnDeviceBecameAvailable() chan *Device
  • func (c *Channels) OnDeviceIncomingMessage() chan *DeviceIncomingMessage
  • func (c *Channels) OnDeviceRegistered() chan *Device
  • func (c *Channels) OnDeviceUnregistered() chan *Device
• type Device
• type DeviceAndEndpoint
• type DeviceIncomingMessage
• type Endpoint
• type Model
• type NodeDatabase
• type PowerSource
  • func (ps PowerSource) String() string
• type Stack
  • func New(configuration coordinator.Configuration, db NodeDatabase) *Stack
  • func (s *Stack) BindToCoordinator(sourceAddress zigbee.IEEEAddress, sourceEndpoint zigbee.EndpointId, ...) error
  • func (s *Stack) Channels() *Channels
  • func (s *Stack) ConfigureReporting(nwkAddress string, clusterId cluster.ClusterId, ...) error
  • func (f *Stack) LocalCommand(dev DeviceAndEndpoint, command cluster.LocalCommand) error
  • func (s *Stack) ReadAttributes(nwkAddress string, clusterId cluster.ClusterId, ...) (*cluster.ReadAttributesResponse, error)
  • func (s *Stack) Run(ctx context.Context, joinEnable bool, packetCaptureFilename string, ...) error
  • func (s *Stack) WriteAttributes(nwkAddress string, clusterId cluster.ClusterId, ...) (*cluster.WriteAttributesResponse, error)

Constants

const (
	DefaultSingleEndpointId = 1 // for simple single-endpoint devices, its endpoint ID usually is 1
)

Types

type Channels

type Channels struct {
	// contains filtered or unexported fields
}

func (*Channels) OnDeviceBecameAvailable

func (c *Channels) OnDeviceBecameAvailable() chan *Device

TODO: document what's the difference between available and registered seems to be signalled only when device's network address changes

func (*Channels) OnDeviceIncomingMessage

func (c *Channels) OnDeviceIncomingMessage() chan *DeviceIncomingMessage

"application-level" message, i.e. sensor sending data TODO: rename to reduce confusion between device registration (name sounds like device is incoming to the cluster..)

func (*Channels) OnDeviceRegistered

func (c *Channels) OnDeviceRegistered() chan *Device

func (*Channels) OnDeviceUnregistered

func (c *Channels) OnDeviceUnregistered() chan *Device

type Device

type Device struct {
	IEEEAddress    zigbee.IEEEAddress // "MAC address" that never changes. longer form of *NetworkAddress*, but curiously not present in incoming messages, so what is it used for?
	NetworkAddress string             // shorter address that is used in Zigbee frames. this changes each time the device leaves and then enters the network
	Manufacturer   string
	ManufacturerId uint16
	Model          Model
	LogicalType    zigbee.LogicalType // coordinator | router | end device
	MainPowered    bool
	PowerSource    PowerSource
	Endpoints      []*Endpoint
}

type DeviceAndEndpoint

type DeviceAndEndpoint struct {
	NetworkAddress string
	EndpointId     zigbee.EndpointId
}

combines device and endpoint, since they both are needed when communicating with a device

type DeviceIncomingMessage

type DeviceIncomingMessage struct {
	Device          *Device
	IncomingMessage *zcl.ZclIncomingMessage
}

type Endpoint

type Endpoint struct {
	Id            zigbee.EndpointId
	ProfileId     uint16
	DeviceId      uint16
	DeviceVersion uint8

	InClusterList  []cluster.ClusterId // input, i.e. what endpoint attributes the device can receive from us
	OutClusterList []cluster.ClusterId // output, i.e. what endpoint attributes the device can send to us
}

I think the value proposition of "endpoint" is because you simply cannot set OnOff cluster's "on" attribute for *the entire Zigbee device* - what if it's a Zigbee power strip? E.g. with endpoints the Zigbee device (= power strip) can present its multiple sockets separately

type Model

type Model string

type NodeDatabase

type NodeDatabase interface {
	InsertDevice(*Device) error
	GetDeviceByNetworkAddress(nwkAddress string) (*Device, bool)
	GetDevice(address zigbee.IEEEAddress) (*Device, bool)
	RemoveDevice(address zigbee.IEEEAddress) error
}

type PowerSource

type PowerSource uint8

const (
	Unknown PowerSource = iota
	MainsSinglePhase
	Mains2Phase
	Battery
	DCSource
	EmergencyMainsConstantlyPowered
	EmergencyMainsAndTransfer
)

func (PowerSource) String

func (ps PowerSource) String() string

type Stack

type Stack struct {
	// contains filtered or unexported fields
}

func New

func New(configuration coordinator.Configuration, db NodeDatabase) *Stack

func (*Stack) BindToCoordinator

func (s *Stack) BindToCoordinator(
	sourceAddress zigbee.IEEEAddress,
	sourceEndpoint zigbee.EndpointId,
	clusterId cluster.ClusterId,
	coordinatorEndpoint zigbee.EndpointId,
) error

func (*Stack) Channels

func (s *Stack) Channels() *Channels

func (*Stack) ConfigureReporting

func (s *Stack) ConfigureReporting(nwkAddress string, clusterId cluster.ClusterId, configs ...*cluster.AttributeReportingConfigurationRecord) error

ZCL spec section 2.5.7

func (*Stack) LocalCommand

func (f *Stack) LocalCommand(dev DeviceAndEndpoint, command cluster.LocalCommand) error

func (*Stack) ReadAttributes

func (s *Stack) ReadAttributes(
	nwkAddress string,
	clusterId cluster.ClusterId,
	attributeIds []cluster.AttributeId,
) (*cluster.ReadAttributesResponse, error)

ZCL spec section 2.5.1

func (*Stack) Run

func (s *Stack) Run(ctx context.Context, joinEnable bool, packetCaptureFilename string, settingsFlash bool) error

if *packetCaptureFile* non-empty, specifies a file to log inbound UNP frames

func (*Stack) WriteAttributes

func (s *Stack) WriteAttributes(nwkAddress string, clusterId cluster.ClusterId, writeAttributeRecords []*cluster.WriteAttributeRecord) (*cluster.WriteAttributesResponse, error)

ZCL spec section 2.5.3