Balena Database: My adventures in adding a Distributed Database for collecting sensor data

[wisehackermonkey]

Update:

Project is abandoned, I’ll not be working on this project anymore.

if anyone ends up creating it please ping me. I’d love to see how they end up building it.

Background:

I’ve been following the developments of IPFS for the past 2-3ish years. and wanted to add it to one my future projects. Collecting sensor data from a bunch of raspberry pi’s, in a offline way. seemed like a good fit.

Why this project would be helpful?

Imaging your setting up a new fleet of 10x RPI4’s with temperature sensors, and you need a quick way to collect all the data from all the sensors streaming data in from the sensors. And you don’t feel like setting up a full scale database, or the complexities of a cloud database.

Ideally this would be as easy as adding something that looks like this to your docker file

  dashboard:

    image: balenablocks/database:raspberrypi4

    restart: always

    volumes:

        - 'database-data:/data'

A example use case, imagine the sensors are for temperature reading.

Using balena blocks you can get a dashboard up and running very quickly, by connecting to a local or internet connected graphana dashboard. see web application

Created a Orbis Definition

Orbis: Product Discovery Process™️ is a process developed by Malcolm Knapp, highly recommend for helping define medium to small projects.

link to notion doc

Notion – The all-in-one workspace for your notes, tasks, wikis, and databases.

Project status updates

Got my first orbit db sample code to work.

the issue i was facing was “ipfs” the npm package is not the same thing as the ipfs daemon! which needed to be installed seperatily ARG! not super clear…

Opening it up in brave

Made some small progress with getting id info

Cool links!
Amazing thorough book on orbit-db orbitdb/field-manual: The Offical User’s Guide to OrbitDB

More updates to follow…

[bversluijs]

Will be following this!
We want to accomplish something like this in the future, where Raspberry Pi’s or Arduino’s are collecting telemetry from sensors / GPIO’s and sending it to a “hub” device, which collects all this data in a secure way. But not via the internet, but something like LoRaWAN, because not every device is capable of getting a decent WiFi / Ethernet connection. Is this something that you’re trying to do or is this via a local WiFi / Ethernet network provided by the Pi?

[wisehackermonkey]

Good point!

@bversluijs

But not via the internet

This project targets devices attached to hub devices vs multiple devices attached to one hub.

Within the network exists

temp sensor <-wires-> Arduino uno ← serial cable-> rpi4 + balena blocks
^
| connection over database software like orbis db.
V
temp sensor <-wires->Arduino uno <- serial cable-> rpi4 + balena blocks

connection topology

bellow is how you could achieve this.

temp sensor <-wires-> Arduino uno + LoRaWAN <-LoRaWAN radio> LoRaWAN device <-i2c-> rpi4 + balena blocks

What your talking about i think looks like this

pictured above has all the sensor connected over a radio EX: LoRaWan, zibee etc… to one hub device. this project is designed to make colating multiple pi’s sensors easy.

The goal of the project is to be more like this

where the sensors are directly attached to the pi’s

Related topic talk:

I absolutely think there is a space for way to connect embedded internet connected devices directly to Balena. I have heard of a project that attempts to do this by using a hub device as a gateway. from what I heard that project turned out to be really complicated!

I’d love to see a way to push code as easy as balena but using something like the ESP32 or something in that class of boards.

@bversluijs does that answer your question?

[bversluijs]

This answers my question indeed, thanks!

[mpous]

Hey @wisehackermonkey did you try this IPFS project on the Hub.

Looking forward to get your feedback on this, and maybe @luandro can help here to tell us more insights

[luandro]

We’re trailing similar paths @wisehackermonkey. We’ve been testing out IPFS for creating ditributed data repositories.

We’re currently stuck at having different IPFS instances discover each other on the local network. I’ve found this example of using avahi with Balena, but we haven’t gotten it working yet. Here’s how I’m trying to do it, but with SSB (Secure Scuttlebutt):

GitHub - coolabnet/hotspot at ssb-avahi

Maybe @mpous has some tips on doing this the right way.

[luandro]

Turns out nothing special was needed for mDNS to work. We have a working example at the p2p hotspot repository.

[mpous]

that’s super interesting @luandro it would be amazing to see an ipfs step-by-step tutorial!