Support MQTT Discovery (AD) with Domoticz

[proddy]

There's been a few comments in various issues around correct support for Domoticz, so thought best to create it's own GitHub Issue for tracking progress.

@bbqkees pointed me to https://github.com/domoticz/domoticz/blob/master/hardware/MQTTAutoDiscover.cpp so now we can see what is and what isn't supported.

This is related to #2138 and #2156

[Domoberry]

Just got an (immediate :-) ) reply on the same from Domoticz in this post https://www.domoticz.com/forum/viewtopic.php?p=321086#p321086. It refers to the development branch of the same: https://github.com/domoticz/domoticz/blob/development/hardware/MQTTAutoDiscover.cpp.

[proddy]

Thanks. I'll take a look. Interesting that this file is 5200 lines without a single comment and there isn't any documentation either. I guess most people have moved over to Home Assistant

[proddy]

Looks like Domoticz does support select, switch and climate so we can add that back in. Except for 'text'.

And we need to make sure every entity has a uom ("unit_of_meas") set

[MichaelDvP]

Remember #1360 (comment)
Quote:

However, I would leave the Domoticz choice in there and mark it as specific for pre 2024.1 Domoticz versions. Reason is that the 2024.1 Domoticz lost support for ArmV6 Raspberry's, so a lot off people will be on the older Domoticz for some time.

I think this is still valid.

[proddy]

Remember #1360 (comment) Quote:

However, I would leave the Domoticz choice in there and mark it as specific for pre 2024.1 Domoticz versions. Reason is that the 2024.1 Domoticz lost support for ArmV6 Raspberry's, so a lot off people will be on the older Domoticz for some time.

I think this is still valid.

yes, we just replace 'text' and also fix the analog & temp sensors (I made TODOs). I'll do the final changes after your PR is merged.

[proddy]

made some changes in dev-4 that need testing (when the PR is merged). I'm not sure how to handle the uom for sensors without a uom (like a number). It's not clear from the Domoitcz c++ code how they handle this.

[Domoberry]

with the risk of posting 'old news'...
Had V3.7.1.-test.3 running for a while. These are some notes/suggestions:

• [this is your point above] there are a number of Domoticz devices which shows up in the UI with "??" as uom, These entities are dimension-less, so indeed do not have a uom. I played around with config messages and carefully concluded this is the result of not adding a oum in the config message. I have posted a related question in the Domoticz forum.
• no errors in the Domoticz log (you were right, the previous two were caused by retained config messages, clear Mosquitto dB first :-) )
• there are some 'number' devices appearing in the Domoticz UI without a UOM and/or scaling [same issue?]
  o The heatpump (energy price gas) shows up as “13” for example, meaning 1.3 € in the Easycontrol App (note the scaling)
  o The heatpump (energy price electric) shows up as “25” for example, meaning 0.25€ in the App
  o The heatpump (energy feed in PV) price shows up as “24” for example, meaning 0.24€ in the App
  o The heatpump (low noise starttime) is a number (afaik) in minutes
  o The heatpump (low noise stoptime) is a number (afaik) in minutes
• There is a 'number' device "mixer (hc1 Setpoint flow temperature)". This is a value in degrees C. Additionally, from my point of view I would like this to appear as a 'sensor' device in Domoticz to allow incorporation in the Domoticz UI. I understood 'number' devices are typically entities which you only want 'set' very infrequently. In my case, I would never 'set' this, I believe this is done by the thermostat. I would use the value for example in the UI.
• similarly, by making the price entities a 'number', they cannot (simply?) be set the Domoticz UI or a local script. I can imagine that with a dynamic energy contract, this would be a parameter you would like to regularly update.
  Please, if (part of the) above is 'mosterd na de maaltijd', let me know, I'll remove it here.
  I will try v3.7.1-dev.4 soon once I can download it.

[Domoberry]

I have posted a related question in the Domoticz forum.
Feedback so far: "I got a tip that it should be a sensor with unit text. That would create a text device."
I'll update if more suggestions come in.

imho making it into a text device would effective remove the need to have an UOM obviously. However, it would make it hard to use its 'value' if it arrives in Domoticz as text. In the current situation (this is from 3.6.5) for example this entity becomes a numerical device 'within unknwon uom' in Domoticz, its value can be used as per normal:

[proddy]

It needs to be a number, but looking through the Domoticz code, I can't find anything that would be suitable for a numerical value. They have a condition here that says if there is no uom then it will default to a number, otherwise a string. So in theory, it should work.

Can you try dev-4 at https://github.com/proddy/EMS-ESP32/releases/tag/latest ?

[Brugje]

I have quick tested rev-4. Still the UMO is ?? in domoticz.
The devices at mqtt number (the editable vallues) are still not recognized by domoticz.
Tomorrow I will have a better look at rev-4. This is my first impression

[proddy]

ok, thanks for trying. For the ?? (like Tx Reads) I don't know what to do, or to add. For the number ones, the same. Perhaps you can adjust the topics in MQTTExplorer directly until it starts working in Domoticz

[MichaelDvP]

For the ?? maybe send a empty uom, here

EMS-ESP32/src/mqtt.cpp

Lines 1142 to 1144 in 77330c9

	} else if (uom != DeviceValueUOM::NONE) {
	doc[uom_ha] = EMSdevice::uom_to_string(uom); // default
	}

we send no uom for NONE, but uom_to_string(uom) would give back a space.
If HA does not like it, you can add for domotics.

        } else if (discovery_type() != discoveryType::HOMEASSISTANT) {
            doc[uom_ha] = "";

@Brugje : The number values are only generated for type DOMOTCZ-Latest, selecting the DOMOTICZ as type should publish as sensor.

[proddy]

good spot Michael. @Brugje wen you can test the dev-4 version from this link: https://github.com/proddy/EMS-ESP32/releases/tag/latest

[proddy]

@Domoberry re: your previous comment about wanting to use Sensors instead of numbers. You can either set the Entity Format to DOMOTICZ (not the latest) and it will not use numbers. But the correct method is to use the EMS-ESP Customizations Module and mark those writable entities 'read-only' by checking the "disable write action' icon. The change should be immediate, no need to restart anything.

I'll create a new issue for the other corrections (price, noise times etc.). Thanks for testing.

[Brugje]

Today I have updated domoticz to the latests stable version Domoticz_2024_7.16322 and installed the firmware dev-4 from the link above. But no changes to our issues. For the ?? i found that it is easy to change in domoticz. Replace ?? by a single space and we get only the number.

Also tried to set mqtt to domoticz instead of latest. Then the sensor are in domoticz but are not editable like in ems-esp. I would like to manege my mixer flowsettemp in domoticz. When the sun is shining i would like to lower this temperature.
(I have no room thermostat on my heating pump. It is just weather dependent)

And they just the 9 under mqtt number.

I hope ems-esp and domoticz come to a solution.
Thanks for all your quick support.

[proddy]

dat is jammer. i thought we had fixed this. Keep using the Domoitcz (latest) so you have the numbers, you can always make them sensors by making them read-only/not-writable via the EMS-ESP Customizations Web module.

Would you mind send the topic for one of those numbers in MQTT Explorer, say the boiler_selflowtemp. We can see what the unit_of_measure is set to. It should be a space " "

[Brugje]

The uom for this is not " " but degrees C.
Also show in screenshot above. 6 out of 9 have the right uom.
Here is the topic:
{"uniq_id":"boiler_selflowtemp","obj_id":"boiler_selflowtemp","cmd_t":"ems-esp-jan/boiler/selflowtemp","mode":"box","step":1,"min":0,"max":90,"name":"Ingestelde aanvoertemperatuur","stat_t":"ems-esp-jan/boiler_data","val_tpl":"{{value_json['selflowtemp']}}","unit_of_meas":"°C","stat_cla":"measurement","dev_cla":"temperature","ic":"mdi:coolant-temperature","dev":{"ids":["ems-esp-jan-boiler"],"name":"ems-esp-jan Boiler"}}

[proddy]

ok so Tx Reads works now, or still showing ?? I need an example of an MQTT topic where it is not showing what you expect

[Domoberry]

Below is all based on using 3.7.1-dev.4 which I temporary installed on the gateway for testing (I cleared the Mosquitto database)

On another entity that ends up in Domoticz as a device with uom "??":
Entity: boiler_service_code_number

This is the related mqtt config message:
Topic: test210/sensor/emstest/boiler_servicecode/config
Payload:
{
"uniq_id" : "boiler_service_code",
"obj_id" : "boiler_service_code",
"name" : "Service code",
"stat_t" : "emstest/boiler_data",
"val_tpl" : "{{value_json['servicecode']}}",
"unit_of_meas" : " ",
"dev" : {
"ids" : [ "emstest-boiler" ],
"name" : "emstest Boiler"
}
}

Some numbers after installing and waiting a while:

]] With my system the following devices are found:

]] On Domoticz side

• 35 entities ended up as 'configurable', e.g. not a Domoticz useable device
• 121 devices
• 11 devices having uom "??":
  

]] On the tip to use customization to 'disable write action' (thanks!):
Tried this with “Mixer (hc1 Setpoint flow temperature)”, which currently shows up as one of the configurable. I could not change the write action as the option is greyd out:

Tried with another entity which had the option to 'disable write action': "selected flow temperature":

Which did the job as expected, the entityh disappeared from 'Confugurable' devices and showed up as normal Domoticz device:

[proddy]

good testing! Two things more:

1. Could you manually change the uom in topic test210/sensor/emstest/boiler_servicecode/config to just "" using MQTT Explorer and see what happens? So the line "unit_of_meas" : "", . And then, after that test, just remove the uom line completely and test again. Trial and error!

2. the mixer flow temperature (TC1) is not a command, so not writeable. Which is greyed out. It should be shown as a sensor in Domoticz with a uom of 'C'/degrees. If this is not the case please post back the contents of that topic. Look for flowTempHc using MQTT Explorer.

[Domoberry]

First a correction to the above: I talked about "Entity: boiler_service_code_number" (value: 203), yet unfortunetely pasted the config for "Entity: boiler_service_code" (value: 0H)

On (1)

I hence assumed this should be about “test210/sensor/emstest/boiler_servicecodenumber/config” (instead of “test210/sensor/emstest/boiler_servicecode/config”)

The current config message:
Current payload for “test210/sensor/emstest/boiler_servicecodenumber/config”
{
"uniq_id" : "boiler_service_code_number",
"obj_id" : "boiler_service_code_number",
"name" : "Service code number",
"stat_t" : "emstest/boiler_data",
"val_tpl" : "{{value_json['servicecodenumber']}}",
"unit_of_meas" : " ",
"ic" : "mdi:counter",
"stat_cla" : "measurement",
"dev" : {
"ids" : [ "emstest-boiler" ],
"name" : "emstest Boiler"
}
}

Replaced "unit_of_meas" : " ", with "unit_of_meas" : "",
and published this to “test210/sensor/emstest/boiler_servicecodenumber/config”

This was picked up by the Domoticz log:
2024-11-08 17:58:58.359 Status: TST: Discovered: emstest Boiler (Service code number)/emstest Boiler (Service code number) (unique_id: boiler_service_code_number)_
But, no change on the device:

Next, I removed the entire line and published this config message:
2024-11-08 18:03:54.156 Status: TST: Discovered: emstest Boiler (Service code number)/emstest Boiler (Service code number) (unique_id: boiler_service_code_number)
Also no change in the way the device appears:

On 2

On Mixer (hc1 Setpoint flow temperature)
This entity is shown as ‘configuration’ in Domoticz:

e.g. it is not seen as a sensor in Domoticz
Checking for topics containing “flowTempHc” (only one found)
Topic: test210/sensor/emstest/mixer_hc1_flowtemphc/config
Payload:
{
"uniq_id" : "mixer_hc1_flow_temperature_(tc1)",
"obj_id" : "mixer_hc1_flow_temperature_(tc1)",
"name" : "hc1 Flow temperature (TC1)",
"stat_t" : "emstest/mixer_data",
"val_tpl" : "{{value_json['hc1']['flowtemphc']}}",
"unit_of_meas" : "°C",
"stat_cla" : "measurement",
"dev_cla" : "temperature",
"ic" : "mdi:coolant-temperature",
"dev" : {
"ids" : [ "emstest-mixer" ],
"name" : "emstest Mixer",
"mf" : "",
"mdl" : "MH210/MH210",
"via_device" : "emstest"
}
}

[Brugje]

I found at domoticz that there is a (thermostat) setpoint device. Is this something we can work with?

It is from this page https://www.domoticz.com/wiki/Dummy_for_virtual_Switches

[Domoberry]

A general setpoint device would be great for setting generic parameters indeed.
But, perhaps not fully understanding how mqtt-ad works, wouldn't this require some change in 'domoticz mqtt ad code' since that piece of code decides how to handle a mqtt-ad device request of type "abc"?. Do pitch in if I got it wrong here.

[Brugje]

I have asked this at the domoticz github page and their answer is here
https://github.com/domoticz/domoticz/issues/6204

[Domoberry]

The link did not work in my case, the link-text is correct though
domoticz/domoticz#6204
I shows an example on how a Domoticz 'setpoint device' can be created using a mqtt config message. Thanks Brugje!
@proddy: in case there is a test version using this, I would be pleased to check!

[proddy]

@proddy: in case there is a test version using this, I would be pleased to check!

We need to get the basic working working again. It hasn't been touched in years and and suddenly things are broken. Maybe something changed on the Domoticz side that is not in line with the MQTT autodiscovery standard which Home Assistant, I believe, invented. As I mentioned before it is very hard for me to work on this as I'm stretched for time and have no Domoticz environment to test against. I'm relying on someone from the 10K+ EMS-ESP users to help us. It's really just a matter of understanding the MQTT config topics and manually editing them using MQTT Explorer until it works as it should. Same with the setpoint thermostat thing.

How about if a give you a https://www.home-assistant.io/green? :-)