Hello all!
I have a project where I am trying to use the balena blocks connector and dashboard to display some sensor data I have via a local MQTT broker.
You can see my whole project and docker compose file here
It seems like everything is working, but my grafana dashboard only shows the system temp and not my actual sensor data. Here’s a link to the public dashboard for now.
Anyone have any idea what I’m doing wrong?
Thanks!
Hello @keenanjohnson thanks for posting this issue on the forums!
Do you have more logs of the mqtt service printing data? or showing errors/warnings? or even can you access the influxDB service from the balenaCloud and query if the data is stored properly?
Let us know if you find more information!
Hey Keenan,
Are the other fields strings, by any chance?
Phil
Hey again,
There was an open issue for MQTT string fields - so I’ve added an extra environment variable to allow them:
Enable json_strings envar on MQTT input plugin · Issue #24 · balenablocks/connector (github.com)
If I’m correct and your missing data is strings, you will need to add their field names (comma-separated) to an enVar called MQTT_INPUT_STRINGS_FIELDS like this:
If the issue is not due to string fields, let me know and I’ll take a look at the device with you.
Phil
Hey @phil-d-wilson !
My data is not in string format. It looks like this when I print it before sending it to the MQTT broker:
15.06.21 07:58:28 (-0700) co2 {"CO2": 406.90521240234375, "Temperature": 23.33104705810547, "Relative_Humidity": 50.64849853515625}
Looking at the InfluxDB, it seems that the data isn’t making it into the InfluxDB, as the Influx CLI only shows the same data as Grafana:
> SHOW SERIES
key
---
temp,host=d14c957,sensor=cpu_thermal_input
>
Here is the log from the connector container:
15.06.21 08:21:40 (-0700) connector balenaBlocks connector version: 0.0.5
15.06.21 08:21:40 (-0700) connector Changing hostname to d14c957
15.06.21 08:21:40 (-0700) connector Generating config
15.06.21 08:21:43 (-0700) connector Settings file not found or not in proper format. Rewriting default settings to: /root/.balena/balena.cfg
15.06.21 08:21:43 (-0700) connector balenablocks/connector
15.06.21 08:21:43 (-0700) connector ----------------------
15.06.21 08:21:43 (-0700) connector Intelligently connecting data sources with data sinks
15.06.21 08:21:45 (-0700) connector Loading base config plugin
15.06.21 08:21:45 (-0700) connector Loading influxdb plugin
15.06.21 08:21:45 (-0700) connector Loading mqtt plugin
15.06.21 08:21:45 (-0700) connector 2021-06-15T15:21:45Z I! Starting Telegraf 1.18.2
15.06.21 08:21:45 (-0700) connector 2021-06-15T15:21:45Z I! Loaded inputs: mqtt_consumer
15.06.21 08:21:45 (-0700) connector 2021-06-15T15:21:45Z I! Loaded aggregators:
15.06.21 08:21:45 (-0700) connector 2021-06-15T15:21:45Z I! Loaded processors:
15.06.21 08:21:45 (-0700) connector 2021-06-15T15:21:45Z I! Loaded outputs: influxdb
15.06.21 08:21:45 (-0700) connector 2021-06-15T15:21:45Z I! Tags enabled: host=d14c957
15.06.21 08:21:45 (-0700) connector 2021-06-15T15:21:45Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"d14c957", Flush Interval:10s
15.06.21 08:21:45 (-0700) connector 2021-06-15T15:21:45Z W! [inputs.mqtt_consumer] Server "mqtt:1883" should be updated to use `scheme://host:port` format
I’ve granted support access to the device fyi. Feel free to go in and poke around since this device isn’t doing anything critical 
https://dashboard.balena-cloud.com/devices/d14c9570bc4c341a76d11599c4aef438/
I’ve confirmed that the MQTT broker is recieving the data, so it seems like the issue is just that the Connector block isn’t pushing the data into InfluxDB
/ # mosquitto_sub -d -t sensors
Client (null) sending CONNECT
Client (null) received CONNACK (0)
Client (null) sending SUBSCRIBE (Mid: 1, Topic: sensors, QoS: 0, Options: 0x00)
Client (null) received SUBACK
Subscribed (mid: 1): 0
Client (null) received PUBLISH (d0, q0, r0, m0, 'sensors', ... (100 bytes))
{"CO2": 444.6472473144531, "Temperature": 33.86015319824219, "Relative_Humidity": 28.49273681640625}
Client (null) received PUBLISH (d0, q0, r0, m0, 'sensors', ... (100 bytes))
{"CO2": 444.7749328613281, "Temperature": 33.81475830078125, "Relative_Humidity": 28.50494384765625}
@keenanjohnson I just pinged Phil, but it’s the middle of the night there, so I think it might be tomorrow before we hear back from him, ha. 
No worries. This isn’t urgent :), just trying to provide enough context for him to eliminate back and forth
Hello Keenan, as a test can you try changing your mqtt to build from this Dockerfile: Mosquitto 2 Broker container · GitHub instead of using the official image? (so build: ./mqtt instead of image: arm32v6/eclipse-mosquitto in your docker-compose file, assuming you put the Dockerfile in a folder named mqtt) I’m wondering if some of the default security settings in Mosquitto are affecting the connector.
Ahhhhh interesting! That did seem to fix things! Not sure why.
I’ve pushed my latest working code here: GitHub - keenanjohnson/ghg-gas-cloud: A large network of open source, low cost, CO2 Gas Detection Sensors.
My public URL is here: Grafana
Aha! Interesting Keenan - I wonder why that was affecting anything. 
Good old @alanb128 ! 
Glad you’re up and running at last @keenanjohnson - do let me know if you find any other issues.
Phil
I think Mosquitto v2 default settings do not allow allow remote access or anonymous connections, which that Dockerfile modifies via mosquitto.conf. You might be able to get the same effect with image: eclipse-mosquitto:1.6.15 if you want to eliminate the extra Dockerfile/folder.