toepid

command module

v0.0.0-...-27e1478 Latest Latest Go to latest Published: Aug 25, 2026 License: AGPL-3.0 Imports: 29 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

codeberg.org/bronger/toepid

Links

Open Source Insights

README ¶

Toepid – the totally over-engineered PID controller
===================================================

Toepid is a program which usually runs 24/7 on a home server, e.g. a Raspi. It
reads an MQTT topic to get the desired temperature for a certain room (set
point) and sets the valves of all TRVs in that room accordingly.

Logging is done to stderr.

Input/output
------------

This program has two inputs:

1. The current room temperature reported by a sensor and read from the
configuration setting ``TChan``.
2. The current set point read from the configuration setting ``topicSetPoint``.

It has two outputs: It sets the valve opening of each TRV, and it reports
informational data about its status to any other program insterested.

All communication is realised using an MQTT broker.

Configuration
-------------

The YAML configuration file is searched for at the path given in the
environment variable ``TOEPID_CONF_PATH``.

A valid configuration file may look like this::

name: living-room
mqttBroker: tcp://192.168.178.30:1883
TChan:
topic: zwave/living_room/sofa_lamp/49/2/Air_temperature
pattern: '$JSON:{"value": "temperature"}'
topicSetPoint: raspi/HeatingScheduler/living/setPoint/set
heaters:
south:
calibDataPath: /etc/toepid/south.dat
valveOffset: 5
radiatorExp: 1.1
valveChans:
position:
topic: shellies/south/thermostat/0/command/valve_pos
pattern: '{{.valve}}'
Δt: 60
window: 900
resetTime: 1800
topicPreload: raspi/preload/office/preload
controlParams:
20:
Kp: 10
Ti: 3600
Td: 1000

Ti, Td, Δt, window, and resetTime are given in seconds. Set Ti or Td to 0 to
erase that term from the PID control. Set Kp to 0 to activate bang–bang
control. Δt is the time between two control updates. window is the timescale
on which temperature changes happen in the room. It is used for the decay rate
of the derivative term, and for the switching times in PWM mode. If the set
point has been set to 0 for at least resetTime, the whole PID control is reset,
in particular, the integral term is zeroed.

The optional environment variable ``TOEPID_DATA_PATH`` points to a directory
which stored state data of the heater. So far, this is only the file
``controller.gob`` with the PID controller state. This way, its state can
survive program restarts.

Reference of the configuration file
-----------------------------------

All MQTT topic are subscribed with QoS 0.

``name``
The name of the controller, typically the name of the room that it is
responsible for. It must be unique at your MQTT broker.

``mqttBroker``
The URL to the MQTT broker, e.g. ``tcp://wilson:1883``.

``TChan``
The MQTT channel for the room temperature that is to be controlled.

``TOffsChan``
(optional) The channel of an offset temperature to be substracted from the
above. This was introduced for the secondary part of a cascaded controller,
which controls the over-temperature of the heater. Then, you set ``TChan``
to the sensor on the heater and ``TOffsChan`` to the room temperature.

``topicSetPoint``
The MQTT topic on which the new set points are received.

``topicPreload``
(optional) The MQTT topic on which preload updates are received. They are in
JSON and look like this::

{
"time": 1732937173490,
"T_out": 4.3,
"T_r_0": 15.8,
"preload": -0.48,
"preloadUncertainty": 0.15
}

``preload`` and ``preloadUncertainty`` are given in units of maximal heating
power. ``T_out`` is the effective external temperature (i.e.,
amplitude-damped and phase-corrected). ``T_r_0`` is the equilibrium room
temperature (which the room would have if no heater was active for a very
long time).

The default preload is 0 with an uncertainty of 0.2.

Note that a value of 0 for ``preloadUncertainty`` might pin the integral term
in PID control to 0.

``type``
The method of control. It might be ``basic``, ``saturated``, or ``pwm``. A
basic controller does not make much sense, as every heater has a lower and
upper power limit, which the saturated controller sets to 0 and 1,
respectively. A PWM (pulse width modulated) controller makes sense if the
valve cannot be calibrated sensefully, so only fully closed and fully open
are reproducible.

``Δt``
The time in seconds between two updates of the control variable (i.e. valve
position or heating power). Updates might happen more frequently due to room
temperature updates, but this sets a minimal frequency.

``window``
Time in seconds that defines the typical time for expecting temperature
changes. Concretely, this is the time constant of the exponential decay of
the derivative term, and the mean switching time in PWM mode.

``controlParams``
The PID parameters Kₚ, Tᵢ, and T_d for different effective external
temperatures Tₒᵤₜ. The program interpolates linearly between them, and
extrapolates constantly. So, this setting maps temperatures to parameter
sets. It may look like this::

controlParams:
-5:
Kp: 2.3
Ti: 4500
Td: 700
10:
Kp: 2.0
Ti: 6000
Td: 200

``heaters``
The heaters for this controller, see below.

``resetTime``
(optional) If the PID controller receives a set point after a period of
having been switched off, this time in seconds determined whether the
controller resets all internal state, in particular the time series data and
the integral term, because they are probably outdated. Set it to a time
which is slightly longer than your typical airing duration, e.g. 1800.

Heater configuration
....................

The following explains the fields in the ``heaters`` section of the
configuration file. The heaters section is a mapping of heater names to their
configuration. The names make the heaters identifiable in the log. This is an
example::

heaters:
south:
calibDataPath: /etc/toepid/north_and_south.dat
valveOffset: 0
radiatorExp: 1.1
valveChans:
position:
topic: shellies/south/thermostat/0/command/valve_pos
pattern: '{{.valve}}'
north:
calibDataPath: /etc/toepid/north_and_south.dat
valveOffset: -1
radiatorExp: 1.1
valveChans:
position:
topic: shellies/north/thermostat/0/command/valve_pos
pattern: '{{.valve}}'

``calibDataPath``
Path to the file with the calibration file. Its format is described in the
following section.

``valveOffset``
(optional) A float value that is added to all valve values in the
configuration file.

``radiatorExp``
The radiator exponent of the heater. It is typically a value a little bit
greater than 1.

``valveChans``
The MQTT channels that set the valve position of the heater. Possible
entries (keys) are ``position`` for setting the position of the valve, ``on``
for fully open, and ``off`` for fully closed. The “basic” and “saturated”
controller types need ``position``, while “pwm” may use ``position`` and/or
``on`` and ``off``. (``position`` is allowed for PWM to be able to switch
easily between PWM and non-PWM modes for experimenting.)

``weight``
(optional, defaults to 1) The relative weight of the radiator in the
calculation of the total heating power. This is necessary if a room has
differently sized heaters.

``topicLimits``
(optional) MQTT topic for updates for that heater according to the limits
API. See the heat-curve program for further information.

``maximalLimitsError``
(optional, defaults to 0.04) Maximal tolarable relative error in messages
coming through ``topicLimits``. Updates with an error greater than this are
discarded.

Channels
........

A channel is an MQTT topic plus a pattern of the syntax of messages in this
topic. For outgoing channels, i.e. the PID controller sends something through
them, you need to define two keys with string values, namely ``topic`` for the
topic’s name and ``pattern`` for the template of what to send. In the
template, you may use ``{{.valve}}`` where the (rounded) value of the valve
position should be placed. For example,

::

position:
topic: zigbee2mqtt/living_room_trv/set
pattern: '{"trv_mode": 1, "valve_position": {{.valve}}}'

will send a JSON string with the keys ``trv_mode`` (always 1) and
``valve_position`` to the TRV. You can also use ``{{.time}}``::

pattern: '{"last_seen": "{{rfc3339 .time}}", …

This also shows the possibility to modify the value with a prefix. “rfc3339”
transforms the time into an RFC3339 timestamp. You may also use “timestamp”
for the UNIX timestamp, and “unixMilli” for the UNIX timestamp in milliseconds.
As for the valve position, you may prepend it with “exact” to get the
non-rounded floating-point valve position::

position:
topic: zigbee2mqtt/living_room_trv/set
pattern: '{"valve_position": {{exact .valve}}}'

For incoming topics, i.e. the PID controller receives something through them,
the ``pattern`` defines how to parse them. There are two possibilities: JSON
and regular expressions.

For incoming JSON messages, you need to declare a mapping from key names in the
message to the names that the PID header expects. For example::

TChan:
topic: zwave/living/ceiling_lamp/49/0/Air_temperature
pattern: '$JSON:{"value": "temperature"}'

However, you need only to include names that are different::

TChan:
topic: zigbee2mqtt/heater sensor office
pattern: '$JSON:{}'

It is important to note that only JSON objects are allowed. By contrast, if
the JSON message consists only of a string or a number, you have to use a
regular expression.

Regular expressions are very versatile but also not so simple to use. For the
above Z-Wave example, it would look like this::

TChan:
topic: zwave/living/ceiling_lamp/49/0/Air_temperature
pattern: '"value": (?P<temperature>[0-9.])'

Here, “temperature” is the key name that the PID reader expects for ``TChan``.
Thus, it is the name of its group in the regular expression. However, the
above pattern only works if the sensor sticks to this simple scheme. For
instance, there must not be whitespace before the colon, exactly one space
character after it, and the temperature must never be negative or need
scientific notation. All of this can be addressed if necessary, but use
``$JSON:`` patterns wherever possible.

Calibration files
.................

Calibration files are text files with two space-separated columns. The first
column is the valve position, while the second column is the resulting
over-temperature.

Set point messages
------------------

Set point messages over the MQTT topic ``topicSetPoint`` are in JSON format and
look like this::

{
"mode": "on"
"setPoint": 19.5,
"reset": "derivative"
}

It contains the following fields:

``mode``
(default is “on”) Can be ``on`` (normal heating to the set point), ``off``
(no heating), ``antifreeze`` (pure proportional control to 5 ℃), or ``set
power``. The latter means that ``setPoint`` is interpreted as a power in
units of maximal heating power, which is then set constantly. (This is
mainly for experimental purposes.)

``setPoint``
(optional and ignored for the modes “off” and “antifreeze”) The desired
temperature. If mode is “set power”, the desired power.

``reset``
(default is “auto”) Which state of the PID controller to be reset. Possible
values are ``true`` (always reset), ``false`` (never reset), ``auto`` (reset
if switching from any non-“on” mode to “on”, and ``resetTime`` has been
exceeded after the latest switch to non-“on”). The rationale for the rather
convoluted rule for ``auto`` is that the controller state (in particular, the
integral term, time series and the lower/upper saturation limits) probably
have become outdated.

Finally, ``reset`` may be ``derivative``, which means the same as “auto” but
*always* zeroes the derivative term. This should be used after airing has
been finished, and heating should start again, in order to avoid a bogus high
negative value for the differential term.

Outgoing topic: control-var
---------------------------

On the MQTT topic ``toepid/<name>/control-var``, Toepid publishes informative
(i.e., not used for actual control) update of the form

::

{
"time": 1744292730140,
"power": 0.8,
"realPower": 0.45,
"processVar": 19.4
}

It contains the following fields:

``time``
The point in time the data refers to, as a UNIX milliseconds timestamp.

``power``
The power requested by the PID algorithm, in units of maximal heating power.
In particular, this might be greater or smaller than the power that the
heater can provide. It might be positive or negative, and it can take the
values ±∞, serialised to JSON as the strings ``"Infinity"`` and
``"-Infinity"``.

``realPower``
The power that the heaters in the room actually provide, in the same unit as
``power``.

``processVar``
The process variable (typically the room temperature). For the PWM
controller, this is the unsmoothed, current value. For the base and
saturated controller, it is more or less smoothed, according to “window”.

Outgoing topic: nominal-power
-----------------------------

On the MQTT topic ``toepid/<name>/nominal-power/<heater>``, Toepid publishes
the current nominal power in units of maximal heating power (maximal for all
times, not just due to e.g. the current value of a heat curve). “Nominal” in
this context means the power that is concretely sent to the heater. It might
be mapped to valve position and such, but the heater is supposed to deliver
this power.

This differs from the “real power” above by effects that reduce the flow
temperature, which is addressed by the concept of “relative power”. The
heater has a reduced maximal power, so in order to get the desired power, the
nominal power must be artificially increased (because the heater itself is not
aware of the reduction). Moreover, non-continuous valve position might create
a difference between nominal and real power.

A message on this topic has the following form::

{
"time": 1744292490731,
"nominalPower": 0
}

It contains the following fields:

``time``
The point in time the data refers to, as a UNIX milliseconds timestamp.

``nominalPower``
The nominal power as explained above.

Uncertainty of preload
----------------------

The ``Uncertainty`` field of the ``preload`` parameter of the ``ControlVar``
methods is an essential part of the anti-windup measures.

The “uncertainty” of the preload is a band around the set point within which
the I term is modified by integration. Outside this band, the I term is used
for determining the controller output, but it is not modified. This
*conditional integration* implements an anti-windup scheme for the I term.

Because the preload means that the long-term value for I is zero, a rather
strict conditional integration is necessary in order to keep I close to zero.
In particular, binding integration to the non-saturated regime (which is a
common method) is not enough. In other words, the so-called proportional band
is too wide.

The name “uncertainty” suggests that this value is an estimate for the expected
deviation of the given preload from the actual long-term value for I.

Note that because the uncertainty is of the same physical unit as the preload
itself, its dimension is that of a controller output (like I) rather than a
temperature. In terms of temperature, the band lies around the set point like

::

preload uncertainty
Tₛₚ ± ———————————————————
Kₚ

.. warning::
It is important to see that with an uncertainty of zero, one does not have
PID control but only PD control because the integral terms stays at zero.

Running the program
-------------------

You can set the log level in the environment variable ``TOEPID_LOGLEVEL``.
Possible values are “debug” and “info” (default).

If you set the environment variable ``TOEPID_COLORED_LOG=true``, *escaped* ANSI
escape sequences will be included into the log output. Thus, you have to
replace the sequence ``\x1b`` with the actual escape character, e.g. using sed,
to see the colour effect.

Cascaded PID control
--------------------

The primary controller controls the room temperature and sets an
over-temperature at the radiator, while the secondary controls that
over-temperature and sets a valve position.

The primary uses the reciprocal value of the radiator exponent for the radiator
exponent. Moreover, it should set the *exact* valve position with
``{{exact .valve}}`` instead of just ``{{.valve}}``. It has the following
calibration file::

0 0
⋮ ⋮
Tₘₐₓᵒᵛᵉʳ 1

The intermediate values are linearly interpolated. The program will later
replace them with the ones corrected b the radiator exponent, and then fit a
spline through them. Three intermediate values are mostly sufficient. If the
sensor on the radiator measures temperatures too high by an offset, add this
offset to the first column.

Note that although only two values are given, a saturated controller is
possible (and not only in bang–bang mode) because all interpolated values are
correct, too.

The secondary uses 1 for the radiator exponent. Its calibration file is also
trivial::

0 0
valveₘₐₓ Tₘₐₓᵒᵛᵉʳ

Its type can only be “saturated” in bang–bang mode, or “PWM”, because only
the extreme values of the calibration are valid.

The secondary needs the additional setting “TOffsChan” in its configuration to
control the over-temperature of the radiator instead of the real radiator
temperature.

Of course, the “position” channel of the single heater of the primary is the
“topicSetPoint” of the secondary. Note that the pattern of the first must be
``{"setPoint": {{exact .valve}}}``.

One may have multiple secondaries if the room has multiple radiators.

Hints for special TRV models
----------------------------

Settings at the Shelly TRV
..........................

The Shelly TRV must be connected to the MQTT broker. “Auto temperature
control” must be off.

In the configuration file of this program, the ``topic`` must be
``shellies/<ID>/thermostat/0/command/valve_pos``, and the ``pattern`` must be
``{{.valve}}``.

Settings for the ZigBee Eurotronic TRV
......................................

Using zigbee2mqtt, the ``topic`` must be ``zigbee2mqtt/<name>/set``, and the
``pattern`` must be ``{"trv_mode": 1, "valve_position": {{.valve}}}``.

Literature
----------

A good read is Åström, Advanced PID Control, chapter 3 “PID Control”. See
<https://folk.ntnu.no/skoge/puublications_others/books/%C3%85strom-2006_Advanced%20PID%20Control/>.

Todo
----

- In the Buderus model in heat-curve, assume a 90% (configurable) loss of the
overtemperature in the pipes from the central heating system to the heaters.
See <https://www.ikz.de/uploads/media/35--100.pdf>. The overtemperature
should be calculated against a weighted mean of average room temperature and
smoothed external temperature. Sensible defaults are important, because
nobody wants to measure that.
- Add a correction for PWM mode that takes deviations from the perfect power
curve into account.
- Bug: Bang–bang control does not work properly in PWM due to loss of
information: It incorrectly assumes that +∞ means “upper limit” and changes
the integrals accordingly. Instead, it should switch on immediately.
- Distinguish between SaturatedLow and a (newly to be introduced) SaturatedOff.
Otherwise, PWM for cooling is not implementable. Then, “Saturated…” might
need to be renamed, and we need an additional channel for SaturatedLow, which
defaults to the channel for SaturatedOff if lower is zero.
- Double the values for ``window`` in the configuration, and use half of these
value for the switching times in PWM mode, and for the mollifier kernel
width, so that effectively, nothing changes.
- I think the base controller is only useful if you can switch off heater
control altogether and using only the control-var topic.
- Write the controller type to the beginning of the ``controller.dat``
file. When reading, check if the type matches, and if not, ignore the file.
As long as this is not fixed, you have to remove the ``controller.dat`` file
when switching types.
- Write a version number to the beginning of the ``controller.dat`` and reject
a wrong version number. (Or, switch to JSON?)

Soft limit
..........

In order to implement an overlay heat curve, the following steps are necessary:

1. Add a parameter to ControlVar called “uActual”, which is the mean power
injected into the room since the last call.
2. Ignore that parameter in BaseController and SaturatedController, and use it
in PWMController for integActual.
3. For the time being, the surrounding code passes the latest returned u for
uActual.
4. heat-curve returns an additional key “softUpperLimit”, which is identical to
“upperLimit” for the time being.
5. The surrounding code uses softUpperLimit wherever upperLimit was used, and
upperLimit to calculate uActual in case of upper saturation.

Overlay heat curve
..................

The overlay heat curve is defined by three parameters:

- Flow temperature at 20℃ external temperature
- Flow temperature at −10℃ external temperature
- heater exponent

The overlay heat curve sets softUpperLimit to a new values and leaves
relativePower and upperLimit untouched.