MING project (MQTT, InfluxDB, NodeRED and Grafana)

Hi,

I’ve deployed MING stack project in a RaspberryPi 4.

It’s a really good project that automates the deploying of this typical open-Source LAMP-like stack for the Internet of Things I’ve been using (and deploying each component manually :sweat_smile:) so far.

After I’ve set it up successfully, I’m thinking about next steps or questions:

  • Node-RED version deployed belongs to a custom Balena image version, as I’ve checked on docker-compose.yml file. What if I want to use and deploy the offficial image version of Node-RED (wich is more up-to-date)? What changes should I need?

  • Node-RED is deployed on 80 host port by default. What if I have another service already listening at that port and want to use another one? What changes should I need? Reference Issue

These are my first draft inquires I’ve think about and would like to resolve somehow (and sureley will help more users using it).

Thanks a lot for sharing the project & kudos for Marc Pous! :wink:

1 Like

Hola Alex! welcome to the balena community!

This is a great question @alexcorvis84

Actually if you check the docker-compose of the MING project i took the advantage of the balenaBlocks here https://github.com/mpous/ming/blob/master/docker-compose.yml

That means that NodeRED is built from this block → balenaHub - IoT embedded device apps, edge computing resources, and developer tools from this repo → GitHub - balena-labs-projects/balena-node-red: a node-red application with balena-supervisor support, can be managed remotely via balena publicURL i think we were not using the docker image because it was not working perfectly with some devices.

Actually i’m thinking on updating this and showing to the developers how to add more flows and more libraries from the Dockerfile template :slight_smile:

  • Node-RED is deployed on 80 host port by default. What if I have another service already listening at that port and want to use another one? What changes should I need? Reference Issue

Regarding with the port 80, the balenaCloud public device URL only supports por 80, 443 and 8080, this is why I used 80 for NodeRED and 8080 for Grafana, so I can access remotely to my MING device :slight_smile: My main goal was to take the advantage of the balenaCloud Public Device URL.

There are different possible solutions!
(1) Feel free to change it the docker-compose
(2) Add a new container and use a proxy that depending on the URL goes to one container or another (e.g. Traefik or others)

What service would you like to use the port 80?

Let me know if that works for you !

1 Like

That means that NodeRED is built from this block → balenaHub: an easier way to find and publish fleets, apps, and blocks for edge devices from this repo → GitHub - balenablocks/balena-node-red) i think we were not using the docker image because it was not working perfectly with some devices.

Even so, isn’t there a possibility to try to deploy the official Node-RED image? If possible, would it only be necessary to change the image in the docker-compose.yml?

  • I´ve been able to install node-red-contrib-influxdb nodes from Balena Node-RED (through Manage palette-> Install) in order to save data received from an MQTT endpoint on Influxdb databases. I’ve faced that connection fails. Can you confirm if the url http://influxdb:8086 is the one exposed? Checking at the docker-compose.yml file, there’s no ports: option listed for influxdb service…maybe is it missing?)

  • I’ve checked that enabling PUBLIC DEVICE URL I can access Node-RED. Is the only service exposed publicly? I was (expecting) trying to access Grafana from https://mypublic_id.balena-devices.com:8080 but no success. So, Grafana can only be accessed locally? :thinking:

What service would you like to use the port 80?

I’m thinking of running Eclipse Kura

Many thanks for your inputs so far @mpous :wink: !

1 Like

Hey @alexcorvis84

did you try it? i don’t have any reason why the official nodeRED image shouldn’t work. We use that one because it has balena flows deployed on it. Feel free to test and let us know if that works. Are you testing on a Raspberry Pi or on a x86 device?

to access to the port 8080 try http instead of https. That might work!

Why do you need Kura and NodeRED at the same device? What is the use case where you need both?

Could you please share more details of the error you get? it works. Not sure the influxdb:8086 uses http. Feel free to send me more details of the issue :slight_smile:

Looking forward to learn more from your side!

1 Like

did you try it? i don’t have any reason why the official nodeRED image shouldn’t work. We use that one because it has balena flows deployed on it. Feel free to test and let us know if that works. Are you testing on a Raspberry Pi or on a x86 device?

Not yet, but I’d like to test it as I would like to have an up-to-date version of Node-RED. I´ll let you know in case!

to access to the port 8080 try http instead of https. That might work!

YES! It’s working now! :rocket: :metal:

Why do you need Kura and NodeRED at the same device? What is the use case where you need both?

Oh, yeah, no need or reason to have both :sweat_smile:! Personally I use Node-RED in my DIY projects, but we use a fleet of devices with Kura running where I’m workin now. So that’s the reason why I’m thinking in next steps if I’d like to customize somehow the project!

Could you please share more details of the error you get? it works. Not sure the influxdb:8086 uses http . Feel free to send me more details of the issue :slight_smile:

Forget about it. I watched today your Node-RED Con 22 presentation & then I realized that you used [1.x] influxdb node configuration, instead the (wrong) one I was using [1.8-flux]! So I can confirm it’s working now as well.

Many thanks again for your quick feedback @mpous !

I’ll keep you updated as soon as I’ll continue doing tests!

1 Like

Hey @alexcorvis84

I will do some experiments here soon! So happy to learn if you try anything here :slight_smile:

oh nice! happy to see Eclipse Kura running on balena! keep me updated on this :slight_smile: if you need any help, let me know!

Looking forward to know more about your tests and happy to help :raised_hands:

1 Like

Hi @mpous,

I want to inform you about a the following fact I faced today:

I checked that my device was offline in Balena Cloud after several hours up&working ok, so I thought it was probably caused by some Wi-Fi connection issues. So great moment to check that the WiFi connect service was running and creating the captive portal.

I checked & it was active, so I tried to connect over exposed WiFi AP and expected to see the captive portal… yet the Balena Node-RED showed on local IP port 80. No way to access captive portal :frowning:

So, did you checked this feature? Is it working as expected in your end? As you can check, I was not able to access the service and I had to unplug/plug again the RaspberryPi to access & see it online on Balena Cloud.

Any feedback is apppreciated! :wink:

Looks like we will need to add the proxy traefik solution or change nodeRED to another port different than 80!

What did you try?

El dj., 10 de nov. 2022, 20:32, Alex Corvis via balenaForums <notifications@balena.discoursemail.com> va escriure:

Hi @mpous , I’ve not tried any option yet… Regarding Traefik solution I think it’ll be probably the best, but it’s out of my actual knowledge base, so I’d try to change the host port of Node-RED on the docker-compose.yml project file and check if it fixes the issue.

Please keep me updated! :wink:

1 Like

Hi,
new to Balena and the forum so apologies if the question is basic or my use of the threads incorrect.

I’m trying to deploy the MING stack project using balena push <my-fleet>, but I’m getting the following error:

[Error]         Some services failed to build:
[Error]           Service: node-red
[Error]             Error: (HTTP code 404) unexpected - pull access denied for bh.cr/balenablocks/balena-node-red, repository does not exist or may require 'docker login': denied: requested access to the resource is denied 

I’m logged in on my docker CLI and on inspection of the link, I get a 404 in my browser also, so I suspect it’s the wrong link. Can anyone else confirm the issue or suggest where I may be going wrong?

Thanks, looking forward to building more on balena.

Hello @hayeskg welcome to the balena community!

Could you please paste the instructions that you are doing? you cloned the repo on your computer, typed balena login and then balena push <your-fleet>?

The more information the better :slight_smile:

Hi @mpous, thanks for the quick reply.

I’ve already set up a device for a fleet and running a balenaSense instance successfully. I wanted to extend on this and start connecting other MQTT devices and came across your project.

  1. Cloned the repo
  2. balena login
  3. balena push <my-fleet>

At this point on the dash I observe the project starting to build on the local device, and the CLI tells me it successfully pulled other services listed in the docker-compose file (wifi-connect, influxdb, etc.). These are the logs:

[Info]          Starting build for pilot, user XXX
[Info]          Dashboard link: https://dashboard.balena-cloud.com/apps/.../devices
[Info]          Building on arm05
[Info]          Pulling previous images for caching purposes...
[Success]       Successfully pulled cache images
[Info]          Uploading images
[Success]       Successfully uploaded images
[Error]         Some services failed to build:
[Error]           Service: node-red
[Error]             Error: (HTTP code 404) unexpected - pull access denied for bh.cr/balenablocks/balena-node-red, repository does not exist or may require 'docker login': denied: requested access to the resource is denied 
[Info]          Built on arm05
[Error]         Not deploying release.
Remote build failed

Hi @hayeskg,

It seems Node-RED resource from Balena blocks is missing or has been removed… not sure why.

Let’s wait for @mpous feedback

EDIT: After some research, I can confirm it has been moved to a new balena organization (balenalabs) and that’s the reason why it fails to fetch it. So, if you change the docker-compose.yml file image to use with the new one bh.cr/balenalabs/balena-node-red I guess it would work :wink: !!! Please confirm if you test it.

Commit of the change that confirms the new organization.

I’ve opened a PR with the new image changes.

Thanks! Let me check internally!

El dc., 16 de nov. 2022, 21:59, Alex Corvis via balenaForums <notifications@balena.discoursemail.com> va escriure:

1 Like

Hi @alexcorvis84 thanks so much for your quick help. It makes sense, I guess I’m new to the platform, but nothing like learning through doing, eh?

I can confirm your suggested fix works and I’m now running all services - inc. Node-RED.

Screenshot 2022-11-17 at 10.46.10

Thanks again!

1 Like

Glad to know!!

Yeah, I’m also new to the platform and still learning by doing, so happy to know I helped you fixing the issue :wink:

1 Like

Hey @alexcorvis84 @hayeskg thanks for your comments and PR!

We did some changes internally, as we are renaming some repos and that affected the link of the images from the containers registry! Sorry for that! We are going to work so this is not going to happen again.

2 Likes

What about nginx? I have been using a docker build for several years now based off of

GitHub - janvda/balena-node-red-mqtt-nginx-TIG-stack: Composite docker application with "8" containers (2x Node-RED, 2x MQTT broker, Telegraf, InfluxDb, Grafana, Nginx) for deployment on Raspberry Pi through Balena. with some tweaks.

I do not want to hijack this thread so I will look elsewhere in the forum to be apart of this effort as well. I will end my comment here. thanks for your work!

1 Like

Thanks for sharing this @diverjoe (and welcome to the balena community)!

Why don’t you add your application to the balenaHub? or add a deploy with balena button on your github repo?

Let me know if this make sense :slight_smile:

1 Like

I am ok with supplying my next build as it goes. There are several pieces to the puzzle that I have not been able to get quite right along the lines of just the standard workflow. IOW, I have an app deployed across the several containers and I am constantly developing new and the standard fare of break/fix. Currently that is being done on my production instance that runs my fish tank. I am having the most difficulty with upgrades and configuration changes within the containers. I am thinking that the individual containers need to be blocks maybe. I have been using Balena and telling everyone I talk to about it. I REALLY believe it solves many of the problems I here within the IoT space. I am a Software Architect (hands-on) and been in the craft for over 30yrs. I have been using this app to work through practicality issues while automating EVERY task I can. It is a 180g, with automated water changes, Reverse Osmosis (RODI). I run several hundred gallons of fresh through the system a week. But the point is I build working apps for a living, and I know this technology (Balena) is on target. I know this post is a crazy ramble of the thousand things I have going on. But I am just putting this out there so folks can ask questions and I can tell what works and what doesn’t and maybe get some answers and give some as well. I guess I am attempting to create what is called a Reference Architecture. An app that showcases it in practice and how someone can replicate the process of taking their tech and putting it on this platform (Balena / Docker / Node-red, Python, Go, etc.) Nuf for now…

3 Likes