Skip to content

feat: Mirror system and air quality events to Home Assistant via MQTT - #50

Closed
netscout2001 wants to merge 11 commits into
21cncstudio:mainfrom
netscout2001:feature/mqtt-events
Closed

netscout2001 wants to merge 11 commits into
21cncstudio:mainfrom
netscout2001:feature/mqtt-events

Conversation

@netscout2001

Copy link
Copy Markdown

Summary

This PR adds a new sensor entity to the existing MQTT + Home Assistant integration that mirrors all significant runtime events to HA in real time — matching what the web dashboard Events tab already shows locally.

No new dependencies, no new configuration options, no breaking changes to existing entities or topics.


Why a sensor entity instead of event?

The HA event platform (introduced in HA 2023.8) is designed for discrete, fire-and-forget signals. While it correctly models the intent, it has one significant UX limitation: the HA logbook and history graph display every state change as the generic string "Hat ein Ereignis erkannt" ("An event was detected"), regardless of the payload content. The actual message, category, and event_type attributes are only visible when drilling into the entity's detail page.

Using the sensor platform with "value_template": "{{ value_json.message }}" makes the human-readable message text — e.g. "CO2 worsened to bad: 1480 ppm" or "NTP sync completed, local time=2026-03-19 09:41:28" — the sensor's state value. This means:

  • The full event text appears directly in the HA logbook without any additional configuration.
  • The full event text appears in the HA history graph label.
  • Automations can still access event_type, category, value, and unit via trigger.to_state.attributes.
  • No template sensors, helper entities, or logbook.log automations are needed in HA.

The payload format is identical regardless of platform type, so switching back to event in a future firmware version would only require changing the discovery component string and adding an event_types array.


New MQTT topic

Topic Direction Retained Description
<base_topic>/events Device → HA No JSON event payload

Air quality event payload:

{
  "event_type": "co2_bad",
  "message": "CO2 worsened to bad: 1480 ppm",
  "value": 1480.00,
  "unit": "ppm"
}

System event payload:

{
  "event_type": "system_warning",
  "message": "SEN66 not found (1/3)",
  "category": "SENSORS"
}

event_type is either system_info, system_warning, or a <metric>_<band> string (e.g. co2_bad, pm25_critical).


Changed files

src/modules/MqttManager.h

  • Added public method publishEvent(metric_key, band, value, unit, message) — publishes an air quality band-change event to <base_topic>/events.
  • Added public method publishSystemEvent(category, severity, message) — publishes a system log event to <base_topic>/events.
  • Updated private method publishDiscoveryEvent() declaration — now registers a sensor entity instead of an event entity.
  • Added free function MqttPublishEvent(...) — callable from any module without creating a circular header dependency with MqttManager.h.
  • Added free function MqttPublishSystemEvent(...) — same rationale.

src/modules/MqttManager.cpp

publishDiscoveryEvent() (rewritten)

  • Changed MQTT Discovery component type from "event" to "sensor".
  • Removed event_types array (not applicable to sensor platform).
  • Added "value_template": "{{ value_json.message }}" so the human-readable message is used as the sensor state.
  • Added "icon": "mdi:bell-alert".
  • Reduced payload reserve from 1024 to 512 bytes.

publishDiscovery() (extended)

  • Added empty retained publish for legacy event/<id>_air_events/config to remove the intermediate event-type entity from HA device registry on first connect after firmware update.
  • Added empty retained publish for legacy event/<id>_events/config for the same reason.

publishEvent() (new)

  • Publishes a JSON payload with event_type, message, value, and unit to <base_topic>/events with retain=false.
  • Uses append_json_escaped() for the message field to handle any special characters that could appear in log strings.

publishSystemEvent() (new)

  • Publishes a JSON payload with event_type, message, and category to <base_topic>/events with retain=false.
  • Maps severity string "warning" → event_type "system_warning", anything else → "system_info".

MqttPublishEvent() / MqttPublishSystemEvent() (new free functions)

  • Module-level free functions accessing MqttManager through the existing g_mqtt singleton set in MqttManager::begin().
  • No-op if MQTT has not been initialised or the broker is disconnected.
  • Both SensorManager and MqttManager run on the same Arduino loop task (Core 1), so no mutex is required.

src/modules/SensorManager.cpp

Include

  • Added #include "modules/MqttManager.h" guarded with #ifdef ARDUINO to prevent compilation failure in the native host test environment (which has no WiFi.h).

log_air_metric_transition() (extended)

  • Added mqtt_key parameter (e.g. "co2", "pm25") used to build the event_type string together with the band name.
  • Added message buffer that assembles the same human-readable string already written to the serial log, e.g. "CO2 worsened to bad: 1480 ppm".
  • Replaced separate LOGW/LOGI calls with a single snprintf into message followed by the log call, so the serial log and the MQTT payload always contain identical text.
  • Added MqttPublishEvent() call guarded with #ifdef ARDUINO at each band-transition point (initial non-good reading, worsening, improving, and recovery to Good).

log_soft_warnings() (extended)

  • All 10 log_air_metric_transition() calls updated to pass the new mqtt_key argument: "co2", "co", "pm05", "pm1", "pm25", "pm4", "pm10", "hcho", "voc", "nox".

SensorManager::begin() (extended)

  • Added MqttPublishSystemEvent() guarded with #ifdef ARDUINO at:
    • "Pressure sensor not found" → system_warning
    • "SFA30 init failed" → system_warning
    • "SFA30 OK" → system_info

SensorManager::poll() (extended)

  • Added MqttPublishSystemEvent() guarded with #ifdef ARDUINO at:
    • "SEN66 not found (x/y)" → system_warning (uses same snprintf buffer as the existing LOGW call, ensuring identical text)
    • "SEN66 start attempts exhausted, stop probing until reboot" → system_warning
    • "SEN66 OK" (recovery after fault) → system_info

src/modules/StorageManager.cpp

Include

  • Added #include "modules/MqttManager.h" inside #ifndef UNIT_TEST block so the native unit test build is unaffected.

StorageManager::begin() (extended)

  • Added MqttPublishSystemEvent() inside #ifndef UNIT_TEST at:
    • "LittleFS mount failed" → system_warning
    • "restored last good config" → system_warning
    • "last good config missing, factory reset" → system_warning
    • "factory reset requested" → system_warning

StorageManager::poll() (extended)

  • Added MqttPublishSystemEvent() inside #ifndef UNIT_TEST at:
    • "config committed as last known good" → system_info
    • "last known good commit failed" → system_warning

src/modules/TimeManager.cpp

Include

  • Added #include "modules/MqttManager.h" inside #ifndef UNIT_TEST block. TimeManager is compiled in the native test environment; MqttManager is not available there.

TimeManager::initRtc() (extended)

  • Added MqttPublishSystemEvent() inside #ifndef UNIT_TEST at:
    • "<RTC label> init failed" → system_warning

TimeManager::ntpPoll() (extended)

  • Added MqttPublishSystemEvent() inside #ifndef UNIT_TEST at:
    • "NTP sync completed, local time=<timestamp>" → system_info (uses same buf as the existing LOGI call)
    • "NTP sync timeout after <N> ms" → system_warning

TimeManager::noteRtcReadSuccess() (extended)

  • Added MqttPublishSystemEvent() inside #ifndef UNIT_TEST at:
    • "<RTC label> communication restored" → system_info

TimeManager::noteRtcReadFailure() (extended)

  • Added MqttPublishSystemEvent() inside #ifndef UNIT_TEST at:
    • "<RTC label> read failed repeatedly" → system_warning

TimeManager::applyRtcBatteryLowState() (extended)

  • Added MqttPublishSystemEvent() inside #ifndef UNIT_TEST at:
    • "RTC battery low" → system_warning
    • "RTC battery status OK" → system_info

src/modules/FanControl.cpp

Include

  • Added #include "modules/MqttManager.h". FanControl has no native test build, so no guard is needed.

FanControl::begin() (extended)

  • Added MqttPublishSystemEvent() at:
    • "DAC ready" → system_info

FanControl::poll() (extended)

  • Added MqttPublishSystemEvent() at:
    • "DAC recovered" → system_info

FanControl::handleDacFault() (extended)

  • Added MqttPublishSystemEvent() at:
    • "DAC error: <reason>" → system_warning (uses same snprintf buffer to match the existing LOGW text exactly)

Native test build

All MqttManager.h includes and MqttPublishEvent() / MqttPublishSystemEvent() call sites that exist in files compiled by the native test runner (SensorManager.cpp, StorageManager.cpp, TimeManager.cpp) are guarded with either #ifdef ARDUINO or #ifndef UNIT_TEST. The native test suite passes unchanged.


Backward compatibility

  • All existing MQTT topics, payloads, entity IDs, and discovery configs are unchanged.
  • The new <base_topic>/events topic is only published when a relevant event occurs; it is never retained, so subscribers that were not connected at the time of an event will not receive it retroactively.
  • If MQTT is disabled or the broker is unreachable, all MqttPublishEvent() / MqttPublishSystemEvent() calls are silent no-ops.

Home Assistant result

After flashing, a new entity sensor.<device_name>_events appears automatically via MQTT Discovery. No manual YAML configuration is required.

The HA logbook shows entries like:

Aura Events  →  CO2 worsened to bad: 1480 ppm
Aura Events  →  NTP sync completed, local time=2026-03-19 09:41:28
Aura Events  →  SEN66 not found (1/3)
Aura Events  →  config committed as last known good
Aura Events  →  DAC recovered
Aura Events  →  RTC battery low

Example HA automation trigger using the full attribute payload:

trigger:
  - platform: state
    entity_id: sensor.aura_events
condition:
  - condition: template
    value_template: >
      {{ trigger.to_state.attributes.get('event_type') in
         ['co2_bad', 'co2_critical', 'pm25_bad', 'pm25_critical'] }}
action:
  - service: notify.mobile_app
    data:
      message: >
        Aura: {{ trigger.to_state.state }}

Checklist

  • I have read CONTRIBUTING.md and agree to the CLA.
  • I tested my changes or explained why tests are not applicable.
  • I updated documentation if needed.

@netscout2001 netscout2001 mentioned this pull request Mar 19, 2026
@netscout2001

Copy link
Copy Markdown
Author

Added PressureHistory events (3h/24h delta ready, gap fills, stale resets) and boot screens released event from UiBootFlow to the same MQTT events sensor.

@netscout2001

Copy link
Copy Markdown
Author

Additional event sources added

Extended the initial implementation with two more modules so the HA Events sensor
now covers all significant runtime events that appear in the web dashboard.

src/modules/ChartsHistory.cpp

Added MqttPublishSystemEvent() for all 5 warning conditions that indicate chart
history data loss or corruption:

Message Trigger
Charts history: invalid header, reset Persisted blob has wrong magic/version
Charts history: invalid index/count, reset Stored index or count out of range
Charts history: stale, reset Stored history too old to be useful (load)
Charts history: stale, reset History became stale during runtime
Charts history: epoch moved backwards, reset System clock jumped backwards

src/modules/NetworkManager.cpp

Added MqttPublishSystemEvent() for the 3 network-level warnings that are
actionable from HA (low-level internal retries intentionally excluded to avoid spam):

Message Trigger
WiFi connect failed All connection retries exhausted, entering error state
WiFi AP failed to start softAP() returned false during AP mode setup
mDNS start failed MDNS.begin() returned false — device unreachable via .local

All new call sites follow the same pattern as the initial implementation —
no new methods, no new topics, no Discovery changes required.

@netscout2001

Copy link
Copy Markdown
Author

Additional events added — boot sequence and sensor init now fully covered

Extended the implementation with 9 more info-level events across 6 files so the
HA Events sensor now mirrors the complete boot sequence visible in the web
dashboard Events tab.

src/modules/NetworkManager.cpp

Message Trigger
WiFi connected, IP: <ip> STA connection established
mDNS responder started: <hostname>.local mDNS started successfully

src/modules/MqttManager.cpp

Message Trigger
MQTT connected Every successful broker connect / reconnect

src/modules/TimeManager.cpp

Message Trigger
NTP sync start (tz=<timezone>) NTP sync initiated after WiFi connect

src/modules/SensorManager.cpp

Message Trigger
BMP580/581 OK / BMP3xx OK / DPS310 OK Whichever pressure sensor is detected
SEN0466 CO OK at 0x<addr> CO sensor detected and started

src/modules/PressureHistory.cpp

Message Trigger
Pressure history restored Stored pressure history successfully loaded from flash

src/modules/ChartsHistory.cpp

Message Trigger
Charts history restored Stored chart history successfully loaded from flash

No new methods, topics, or Discovery changes. All call sites in files compiled
by the native test runner remain guarded with #ifdef ARDUINO or
#ifndef UNIT_TEST.

@netscout2001

Copy link
Copy Markdown
Author

Resolved merge conflicts after upstream refactored the MQTT client from PubSubClient to the native ESP-IDF esp_mqtt_client library.

Changes required to adapt our implementation:

  • client_.connected() → mqtt_connected_ (new member variable)
  • client_.publish(topic, payload, retain) → publishMessage(topic, payload, retain) (new wrapper method)
  • connectClient(const SensorData&, bool, bool, bool) → connectClient() (new async signature)

All our event publishing methods (publishEvent, publishSystemEvent, publishDiscoveryEvent) and the free functions (MqttPublishEvent, MqttPublishSystemEvent) are fully adapted and compile cleanly against the new API. Tested on device — firmware boots and runs correctly.

@netscout2001

Copy link
Copy Markdown
Author

Fixed native test build failure in ChartsHistory.cpp and PressureHistory.cpp — the #include "modules/MqttManager.h" and all MqttPublishSystemEvent() call sites were missing #ifndef UNIT_TEST guards, causing WiFi.h: No such file or directory errors in the native test environment. All calls are now consistently guarded, matching the pattern already used in StorageManager.cpp and TimeManager.cpp.

========================================================================================================= SUMMARY =========================================================================================================
Environment Test Status Duration


native_test test_bmp3xx_probe PASSED 00:00:02.307
native_test test_bmp58x_driver PASSED 00:00:01.811
native_test test_boot_diag_policy PASSED 00:00:01.634
native_test test_boot_policy PASSED 00:00:01.636
native_test test_charts_history PASSED 00:00:01.626
native_test test_init_config PASSED 00:00:01.675
native_test test_logger_recent PASSED 00:00:01.648
native_test test_mqtt_payload PASSED 00:00:01.633
native_test test_ota_deferred_restart PASSED 00:00:01.600
native_test test_pressure_history PASSED 00:00:01.688
native_test test_pressure_shared_fallback PASSED 00:00:01.657
native_test test_rtc_drivers PASSED 00:00:01.679
native_test test_sen66_driver PASSED 00:00:01.762
native_test test_sen66_status PASSED 00:00:01.639
native_test test_sensor_manager PASSED 00:00:01.723
native_test test_sfa3x_driver PASSED 00:00:01.716
native_test test_storage_manager_save PASSED 00:00:01.732
native_test test_time_manager PASSED 00:00:01.700
native_test test_web_api_utils PASSED 00:00:01.639
native_test test_web_charts_api_utils PASSED 00:00:01.681
native_test test_web_charts_utils PASSED 00:00:01.611
native_test test_web_color_utils PASSED 00:00:01.651
native_test test_web_dac_api_utils PASSED 00:00:01.662
native_test test_web_dac_utils PASSED 00:00:01.648
native_test test_web_dashboard_page PASSED 00:00:01.667
native_test test_web_deferred_actions_state PASSED 00:00:01.691
native_test test_web_diag_api_utils PASSED 00:00:01.633
native_test test_web_events_api_utils PASSED 00:00:01.690
native_test test_web_events_utils PASSED 00:00:01.672
native_test test_web_input_validation PASSED 00:00:01.637
native_test test_web_mqtt_page PASSED 00:00:01.654
native_test test_web_mqtt_save_utils PASSED 00:00:01.605
native_test test_web_multipart PASSED 00:00:01.641
native_test test_web_network_utils PASSED 00:00:01.638
native_test test_web_ota_api_utils PASSED 00:00:01.626
native_test test_web_ota_state PASSED 00:00:01.659
native_test test_web_query_string PASSED 00:00:01.625
native_test test_web_response_utils PASSED 00:00:01.633
native_test test_web_settings_api_utils PASSED 00:00:01.625
native_test test_web_settings_utils PASSED 00:00:01.712
native_test test_web_state_api_utils PASSED 00:00:01.660
native_test test_web_stream_policy PASSED 00:00:01.628
native_test test_web_stream_state PASSED 00:00:01.687
native_test test_web_stream_writer PASSED 00:00:01.749
native_test test_web_text_utils PASSED 00:00:01.627
native_test test_web_theme_api_utils PASSED 00:00:01.687
native_test test_web_theme_page PASSED 00:00:01.625
native_test test_web_ui_bridge_adapters PASSED 00:00:01.671
native_test test_web_wifi_page PASSED 00:00:01.680
native_test test_web_wifi_save_utils PASSED 00:00:01.647
native_test test_web_wifi_scan_utils PASSED 00:00:01.653
====================================================================================== 216 test cases: 216 succeeded in 00:01:25.480 ======================================================================================

@netscout2001

Copy link
Copy Markdown
Author

@21cncstudio you had a chance to review the PR?

@netscout2001 netscout2001 left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR ready for implementation.

@21cncstudio

Copy link
Copy Markdown
Owner

Thanks for the PR. I like the overall direction, and mirroring runtime events into Home Assistant definitely makes sense for this project.

I had a first look and there are a few things I want to review more carefully before we decide how to move forward, mainly around early boot events being lost before MQTT is connected, and some mismatch between the events sensor discovery/docs and the actual HA behavior. The implementation also touches quite a few modules, so I want to go through it properly instead of giving rushed feedback.

I’ll take a closer, more concrete look on Monday and then come back with specific notes on what should be changed before merge. For now, please leave the PR as it is and don’t update anything yet.

@21cncstudio

Copy link
Copy Markdown
Owner

Thanks a lot for the idea and for putting time into this PR.

I ended up taking a slightly different approach on my side so it fits better with the current MQTT/Home Assistant setup and the direction I want for the project, so I’m not going to merge this one as-is. But your PR was definitely helpful and pushed this feature in the right direction.

Really appreciate the contribution and the effort here.

@netscout2001
netscout2001 deleted the feature/mqtt-events branch March 23, 2026 19:14

Copy link
Copy Markdown
Owner

Hi @netscout2001,

Thank you again for the work you put into this PR!

I ended up implementing the Home Assistant MQTT event stream with a different internal design so it would fit the current Aura architecture, but your proposal and implementation were genuinely helpful and pushed the feature in the right direction. The resulting feature was added in 4e14540.

I’m now improving how Project Aura records community contributions. The project has an Individual Contributor License Agreement, and I’d appreciate it if you could read the current CLA. If you’re happy with it, please post the following exact text as a separate comment containing only this sentence:

I have read and agree to the Project Aura Individual Contributor License Agreement v2.0, and I hereby sign it.

Separately from the CLA, I’d like to record this PR in the contributor history and clearly acknowledge your role in bringing the Home Assistant event-stream idea to Aura.

Thanks again for the idea and all the detailed implementation work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants