MING project (MQTT, InfluxDB, NodeRED and Grafana)

hayeskg · November 16, 2022, 4:24pm

Hi @mpous, thanks for the quick reply.

I’ve already set up a device for a fleet and running a balenaSense instance successfully. I wanted to extend on this and start connecting other MQTT devices and came across your project.

Cloned the repo
balena login
balena push <my-fleet>

At this point on the dash I observe the project starting to build on the local device, and the CLI tells me it successfully pulled other services listed in the docker-compose file (wifi-connect, influxdb, etc.). These are the logs:

[Info]          Starting build for pilot, user XXX
[Info]          Dashboard link: https://dashboard.balena-cloud.com/apps/.../devices
[Info]          Building on arm05
[Info]          Pulling previous images for caching purposes...
[Success]       Successfully pulled cache images
[Info]          Uploading images
[Success]       Successfully uploaded images
[Error]         Some services failed to build:
[Error]           Service: node-red
[Error]             Error: (HTTP code 404) unexpected - pull access denied for bh.cr/balenablocks/balena-node-red, repository does not exist or may require 'docker login': denied: requested access to the resource is denied 
[Info]          Built on arm05
[Error]         Not deploying release.
Remote build failed

alexcorvis84 · November 16, 2022, 7:14pm

Hi @hayeskg,

It seems Node-RED resource from Balena blocks is missing or has been removed… not sure why.

Let’s wait for @mpous feedback

EDIT: After some research, I can confirm it has been moved to a new balena organization (balenalabs) and that’s the reason why it fails to fetch it. So, if you change the docker-compose.yml file image to use with the new one bh.cr/balenalabs/balena-node-red I guess it would work !!! Please confirm if you test it.

Commit of the change that confirms the new organization.

I’ve opened a PR with the new image changes.

mpous · November 17, 2022, 6:45am

Thanks! Let me check internally!

El dc., 16 de nov. 2022, 21:59, Alex Corvis via balenaForums <notifications@balena.discoursemail.com> va escriure:

hayeskg · November 17, 2022, 10:46am

Hi @alexcorvis84 thanks so much for your quick help. It makes sense, I guess I’m new to the platform, but nothing like learning through doing, eh?

I can confirm your suggested fix works and I’m now running all services - inc. Node-RED.

Screenshot 2022-11-17 at 10.46.10

Thanks again!

alexcorvis84 · November 17, 2022, 3:59pm

Glad to know!!

Yeah, I’m also new to the platform and still learning by doing, so happy to know I helped you fixing the issue

mpous · November 18, 2022, 12:43pm

Hey @alexcorvis84 @hayeskg thanks for your comments and PR!

We did some changes internally, as we are renaming some repos and that affected the link of the images from the containers registry! Sorry for that! We are going to work so this is not going to happen again.

diverjoe · November 24, 2022, 2:20am

What about nginx? I have been using a docker build for several years now based off of

GitHub - janvda/balena-node-red-mqtt-nginx-TIG-stack: Composite docker application with "8" containers (2x Node-RED, 2x MQTT broker, Telegraf, InfluxDb, Grafana, Nginx) for deployment on Raspberry Pi through Balena. with some tweaks.

I do not want to hijack this thread so I will look elsewhere in the forum to be apart of this effort as well. I will end my comment here. thanks for your work!

mpous · November 24, 2022, 11:59am

Thanks for sharing this @diverjoe (and welcome to the balena community)!

Why don’t you add your application to the balenaHub? or add a deploy with balena button on your github repo?

Let me know if this make sense

diverjoe · November 24, 2022, 2:53pm

I am ok with supplying my next build as it goes. There are several pieces to the puzzle that I have not been able to get quite right along the lines of just the standard workflow. IOW, I have an app deployed across the several containers and I am constantly developing new and the standard fare of break/fix. Currently that is being done on my production instance that runs my fish tank. I am having the most difficulty with upgrades and configuration changes within the containers. I am thinking that the individual containers need to be blocks maybe. I have been using Balena and telling everyone I talk to about it. I REALLY believe it solves many of the problems I here within the IoT space. I am a Software Architect (hands-on) and been in the craft for over 30yrs. I have been using this app to work through practicality issues while automating EVERY task I can. It is a 180g, with automated water changes, Reverse Osmosis (RODI). I run several hundred gallons of fresh through the system a week. But the point is I build working apps for a living, and I know this technology (Balena) is on target. I know this post is a crazy ramble of the thousand things I have going on. But I am just putting this out there so folks can ask questions and I can tell what works and what doesn’t and maybe get some answers and give some as well. I guess I am attempting to create what is called a Reference Architecture. An app that showcases it in practice and how someone can replicate the process of taking their tech and putting it on this platform (Balena / Docker / Node-red, Python, Go, etc.) Nuf for now…

mpous · November 28, 2022, 1:13pm

@diverjoe this is an amazing use case using the MING stack! Good job it’s beautiful

I would love to see this Application for fish tanks on the balenaHub. Let me know if you need any help!

BTW look at this @ajlennon

ajlennon · November 29, 2022, 10:33am

Who that’s fantastic @diverjoe !!! Great work !!! From Diver Alex

alexcorvis84 · January 28, 2023, 6:33pm

Hi @marc, let’s see if you can help me…

I’ve added a new service (Thigsboard) to the project. My docker-compose file is as below:

version: '2'

volumes:
  node-red-data:
  grafana-data:
  influxdb-data:
  influxdb-etc:
  mytb-data:
  mytb-logs:

services:

  # ------------------------------------------------------------------------------------
  # BalenaLabs WiFi-Connect
  # ------------------------------------------------------------------------------------
  # wifi-connect:
  #   image: bh.cr/balenalabs/wifi-connect-rpi
  #   container_name: wifi-connect
  #   network_mode: "host"
  #   labels:
  #       io.balena.features.dbus: '1'
  #       io.balena.features.firmware: '1'
  #   cap_add:
  #       - NET_ADMIN
  #   environment:
  #       DBUS_SYSTEM_BUS_ADDRESS: "unix:path=/host/run/dbus/system_bus_socket"

  # ------------------------------------------------------------------------------------
  # Node-RED
  # ------------------------------------------------------------------------------------
  node-red:
    image: bh.cr/balenalabs/balena-node-red
    container_name: node-red
    privileged: true
    restart: unless-stopped
    volumes:
        - 'node-red-data:/data'
    labels:
        io.balena.features.dbus: '1'
        io.balena.features.sysfs: '1'
        io.balena.features.supervisor-api: '1'
        io.balena.features.balena-api: '1'
    ports:
        - 80:80
    environment:
        PORT: 80
        USERNAME: "balena"
        PASSWORD: "balena"
        ENCRIPTION_KEY: "balena"        
    cap_add:
        - SYS_RAWIO
    devices:
        - "/dev/mem:/dev/mem"
        - "/dev/gpiomem:/dev/gpiomem"
        - "/dev/i2c-1:/dev/i2c-1"

  # ------------------------------------------------------------------------------------
  # Influxdb
  # ------------------------------------------------------------------------------------
  influxdb:
    image: influxdb:1.8.10
    container_name: influxdb
    restart: unless-stopped
    volumes:
        - influxdb-data:/var/lib/influxdb
        - influxdb-etc:/etc/influxdb

  # ------------------------------------------------------------------------------------
  # Grafana
  # ------------------------------------------------------------------------------------
  grafana:
    image: bh.cr/balenalabs/dashboard
    container_name: grafana
    restart: unless-stopped
    ports:
        - "8080:80"
    volumes:
        - "grafana-data:/data"

  # ------------------------------------------------------------------------------------
  # MQTT
  # ------------------------------------------------------------------------------------
  mqtt:
    image: eclipse-mosquitto
    network_mode: host
    ports:
      - "1883:1883"
    restart: always

  # ------------------------------------------------------------------------------------
  # Thingsboard
  # ------------------------------------------------------------------------------------
  thingsboard:
    image: thingsboard/tb-postgres:latest
    container_name: thingsboard
    restart: unless-stopped
    ports:
      - "9090:9090"
      - "1884:1883"
      - "7070:7070"
      - "5683-5688:5683-5688/udp"
    environment:
      TB_QUEUE_TYPE: in-memory
    volumes:
      - mytb-data:/data
      - mytb-logs:/var/log/thingsboard

I’ve setup my device as Local Mode. Then I pushed the changes through Balena CLI with balena push Device_IP_address.

The first time the services built and I could see the logs, but due some config missing (I guess) the Thingsboard service did not brought up.

Now, issuing the same command (with docker-compose.yml attached), the build works, but I can´t see the Thingsboard service.

I’m missing something? Some configuration is wrong? Thanks in advance!

mpous · February 8, 2023, 10:15am

This is awesome @alexcorvis84

BTW how are you trying to access to thingsboard? local ip address to which port?

mpous · February 8, 2023, 10:59am

@alexcorvis84 just tested on my side and it worked using the port 9090 with my local ip address!

Could you please confirm if this works on your side?

In case that you might need the Public Device URL, my recommendation is to redirect the port 9090 to 8080 or 443 as they are the only ports available (plus the 80 which is being used by NodeRED now).

alexcorvis84 · February 8, 2023, 11:27am

Hi @mpous ,

I was not able to test it again as I had issues as per above. The first time I did the balena push command I checked that the Thingsboard service tried to built but failed. Then the following attempts I tried to push the code with balena push it did not work and I couldn’t see any logs pointing to the service.

Q: Working in local mode, what is the best way to ‘clean’ the services to re-install the project? Or is it automatically cleaned (removed containers and images) with every new push? This part is not clear to me.

I might need to delete the device and load again de imageOS into SD, due not sure what’s happening…

Anyway, will give you feedback when I’ll be at home and try it again.

Thank you in advance!

mpous · February 8, 2023, 11:41am

mmmm that’s weird! could you please share more logs next time @alexcorvis84 ?

alexcorvis84 · March 28, 2023, 3:25pm

Finally I deleted the device and started again, burnt the SD, pushed the release…and it’s working now.

Just to note !!! , I had issues to connect the RPI via WiFi to balena due:

I had to change the WiFi channel below 12 (even working with a RPi 4!)
I had to change manually the WiFi password on SD, as it included a ‘#’ character and it wasn’t saved correctly

alexcorvis84 · April 25, 2023, 8:57pm

Hi @mpous

I’m working in a modified version of this project.

Here is my docker-compose.yml

Services starts ok and everything works fine BUT after some days, I can’t access to the URLs provided by the different services (yet I can see in the Logs window they’re running).

Device is connected to a 2.4GHz WiFi and I checked that inside Host Os it has IP on wlan0.

What could be happening? What can I check?

Thank you in advance

mpous · April 26, 2023, 8:06am

Thanks for sharing! I will test this today @alexcorvis84

@alanb128 did you have this behaviour before?

alexcorvis84 · May 16, 2023, 8:57am

I think we should the fabulous Blog Tutorial @mpous has published as reference!

Use the MING stack to accelerate your IoT application development

Thanks a lot for your work! Let’s continue promoting the MING stack!