MQTT on new Eisy - something missing?

[maxnorth]

I was a long-time user of MQTT for Sonoff devices on PG2.  Recently upgraded to Eisy and PG3.  In the old days, running PG2 on my Pi, we had to configure a MQTT broker on that device, and I used port 1883.  With Eisy migration, I did nothing with a MQTT broker.  I just installed PG3 and then installed the MQTT node server.  It is not working for me as expected. The installation instructions for the MQTT poly suggest that nothing more is needed, but that does not feel right.

Devices turn on, but they do not report back their status. The devlist is identical to what I used with PG2.

I am guessing that there is a piece to configuring the MQTT broker that I am missing, and I cannot find any documentation about it.  In May, @Michel Kohanim said "In our next build, we'll have two mosquitto instances running." That is interesting, but I have no idea how to configure either one of them, and can find no documentation on that.

Question: port? user/pw? is the user/pw integrated with the eisy? is there a script to set this up properly?

My current config is using mqtt_port=1844 and mqtt_password is using the password I have set up for each sonoff device (and had set up with MQTT on PG2).  I have also tried admin|admin and the user/pw I use for Eisy. It seems to make no difference what I use, as I get the same number (14) of sonoff devices being added to the PG3 instance in any events and no change in the communications.

Nothing meaningful shows in the PG3 MQTT live log. When I download a log, though, there are some entries, attached. I feel as though the broker is not working as it should to process return stats messages.

This poly by @xKingis great when it works, and I'd be happy to help create some actual installation instructions for others, if I could figure out what I was doing wrong.

[maxnorth]

An update to my post.  See this post prior to Michel saying "our next build will have two instances': 

 It concludes with: "Update on what I learned from UDI support. MQTT is installed, however PG3 and PG3x make use of a private MQTT broker for internal communication between the various components, and that package is specifically configured to support PG3x.  They advised I should just use another mqtt broker on another device on my network, so I used a broker on my Raspberry Pi that is for a different project, but is now serving two purposes."

So, my question is whether this is still the recommendation with a new eisy and PG3.  Is it necessary to install yet another MQTT broker for this node server?

I now have the MQTT node server configured to point at the eisy port 1884 for the broker.  And I actually get a success message in the log "Poly MQTT Connected, subscribing."  And I can connect to the MQTT broker (no user or pw required) using another MQTT client.  And the Sonoff device also shows a connection.  But commands are not going through, so I suspect the broker is still "private" whatever that means.

 

[Michel Kohanim]

@maxnorth,

If you have the latest version of isy/udx, then the generic mqtt already comes with it and gets started automatically. It's called

mosquitto.gen (as in generic)

You can do whatever you want to configure it. You can also start/stop it using:

sudo service mosquitto.gen start/stop/restart

With kind regards,
Michel

[maxnorth]

Thanks.  That command is not working for me.  Is mosquito.gen documented anywhere?

In the meantime, I've installed an eclipse broker using docker on my synology NAS and it seems to connect fine.  But the status of the devices is still not updating in the Admin Console.

[Michel Kohanim]

@maxnorth,

Apologies, it's 

gen.mosquitto.ud

This is a plain mqtt broker. There are no documentations above and beyond those you can get from:

man mosquitto

With kind regards,
Michel

[maxnorth]

Thank you, @Michel Kohanim.  I can see the service at usr/local/etc/rc.d and have reviewed the config at /usr/local/etc/udx.d/static/gen.broker.conf. This confirms it is a broker running on 1884 and allows anonymous connections.

Interesting, because the polyglot MQTT server (and the sonoff device config) requires that a user name and password be configured, although it may be ok for those values to be null.

What I have not yet tried is adding a user and password to gen.mosquitto, then adding that to the node server and the sonoff devices to see if perhaps that will make a difference.  

The eisy still does not display the status of the sonoff devices.

[Michel Kohanim]

@maxnorth,

Have you contacted the author of the MQTT plugin? I will post your question on Discord.

With kind regards,
Michel

[TriLife]

Greetings,

By default, on PG3/x you do not need user/PW for things to run. You do, however need to use port 1884.

Hope that helps.

Please use @TriLife to ping me. I'm out of town for a few days and not monitoring the forum.

Cheers.

[Goose66]

You don’t have to change the configuration of the generic MQTT broker, and you probably shouldn’t because there are other node servers using it as configured. What errors are logged in the node server log (not the pg3 log) when the node server is started?

[Goose66]

Also, depending on what you are looking for, there is a Tasmota node server available as well. Are your Sonoff devices using Tasmota firmware?

[maxnorth]

Thanks, everyone, for the input.  I am happy to report that this is resolved.  I must have taken an extra dose of stupid pills at some point in the last couple of years, because the resolution was to pay attention to the font case of the topic specification in my devlist.  At some point in the past, all of my devices had their "status_topic" changed to use lowercase "power," for example: "status_topic": "stat/sonoffbr2c/power." The correct usage is "status_topic": "stat/sonoffbr2c/POWER."

So the Sonoff devices, although they would respond to a cmnd, they would not properly publish their stat topic; hence eisy was not updating with their current status.

I discovered this thanks to some testing using mosquitto_sub (which is built in to the eisy Mosquitto implementation).

To close out this thread, here is a summary for new eisy users:

1. Eisy contains an extra instance of Mosquitto available for use by the MQTT node server.  You do NOT need to set up a separate instance as is specified in the original installation instructions for the node server. See above for reference to the gen.mosquitto.ud. This uses port 1884 and does not require authentication (no user or password required). While you could try to change the authentication config, I did not do so.

2. No change to the configuration of the gen instance is needed to run the node server, but use the IP of your eisy and port 1884 in the config for the nodeserver.

3. Since the node server does "require" a user name and password, go ahead and add the mqtt_user and mqtt_password Custom Parameters in the nodeserver config.  You can fill in anything in these fields, or leave them blank, but the keys are needed for it to start correctly.

4. Note that I had a problem where the Configuration Help was not displaying on the node server configuration page  -- UNTIL I added a Custom Parameter, then it showed up. This makes it exceeding difficult for a first-time user. I have attached a screenshot of the help text for new users. 

4. On the sonoff device MQTT config page, which you access using the device's IP address, same thing, leave the user name and password blank, or fill them in with anything. Use the IP of the eisy as the host and port 1884 on the MQTT Configuration page.

5. Make sure the "Topic" you enter on the sonoff device MQTT config page matches the topic specified in your devlist.

6. Make sure the "Client" name on the MQTT Config page of the device is different than the Topic name.

7. **Most importantly**, pay attention to case in your devlist when setting topic names   The below works for me (you would insert your own topic in the middle):

- "status_topic": "stat/sonoffas6_a/POWER"

- "cmd_topic": "cmnd/sonoffas6_a/POWER"

8. If you want to view the MQTT activity in the MQTT Nodeserver log, set the logging level to "debug," not "info."

 

 

[sjenkins]

On 10/30/2023 at 6:13 PM, maxnorth said:

Interesting, because the polyglot MQTT server (and the sonoff device config) requires that a user name and password be configured, although it may be ok for those values to be null.

To fill out this question as well.  If you "do nothing" the defaults will result in the internal generic mqtt server on the EISY/Polisy.  Some people still have mqtt running on an external device ; hence the settings are there to be used.

I will add a pull request to continuously improve the docs in the plugin.