Running crate database on balena

Show and Tell
1

On this message I would like to share that with Marija from Crate we built a Crate Database instance running on balena.

CrateDB is an open-source database that makes storage and analysis of massive amounts of data simple and efficient. It can handle various data types, from time-series, geospatial, structured, and semi-structured data. Combining SQL and NoSQL features, CrateDB offers a high degree of scalability and availability, SQL compatible interface, and easy integration. Furthermore, CrateDB supports dynamic schemas, queryable objects, and real-time full-text search over millions of documents in just a few seconds. These capabilities make CrateDB a perfect choice for many data-intensive analytics applications.

DISCLAIMER: This is a test to demonstrate that CrateDB can run on balena. We are looking for developers from the community who would like to run applications on balena + crateDB.

Getting started

Hardware

  • Raspberry Pi 4 (only tested on this device)
  • SD card
  • Power supply and Ethernet cable (if you don’t use WiFi)

Software

Deploy CrateDB on your fleet

The easiest way to deploy a CrateDB instance on a balena device is clicking on the button below :point_down::point_down::point_down:

deploy-with-balena

Follow instructions, click Add a Device and flash an SD card with that OS image dowloaded from balenaCloud. Enjoy the magic :star2:Over-The-Air​:star2:!

Find here the github repo with the source code.

On balenaCloud click Add Device and follow the steps to download an image. Flash a SD card to flash the balenaOS image to your device and see how the crateDB service gets deployed on your device’s fleet.

Run crateDB instance on the Raspberry Pi

Change the max_map_count

Once the CrateDB service is deployed on your device, you might see an error on the Logs and the service restarting. Open the HostOS ssh Terminal and type:

sysctl -w vm.max_map_count=212144

This command increments max_map_count which is the maximum number of memory map areas a process may have of your container. It’s by default 65536 as most of the applications need less than a thousand maps.

Once the instraction is submitted, the CrateDB service might start properly.

Access to the CrateDB UI

Copy your local IP address and paste it into your browser with the port 4200 and that might open the CrateDB UI.

Install crash CLI

If you need a CLI to interact with the Crate database instance you will need to install crash. To install crash open the cratedb Terminal and type:

curl -o crash https://cdn.crate.io/downloads/releases/crash_standalone_latest

chmod +x crash

./crash

Questions? Comments?

Feel free to add your comments here. And if you build a new project using CrateDB and balena let us know :slight_smile:

2

Hi.
What about make sysctl -w vm.max_map_count=212144 call persistent to reboots?
Andrea

3

@netliteIT welcome to the balena community!

Let me ask to the balenaOS team how to do the vm.max_map_count persistent to reboots.

Let me know if you have done any experiment and the results :slight_smile:

4

Yes I did tests using every documented container config voice with no result.
Other sysctls are easily made writable, network for example.

5

@netliteIT did you try to uncomment these 2 lines? crate-balena/docker-compose.yml at 3e1bf2ca0ad9e425ddd93d9f8df45c83f656f4c5 · mpous/crate-balena · GitHub reading this documentation from docker, it might work → Compose file version 2 reference | Docker Documentation

6

Yes tried.
Doesn’t work.

7

Hello,
any updates here?
How to make te value change persistent to reboots?

8

Hey @specter456

You could replace the docker-compose to reference a dockerfile and in there bring in the crate image, run the sysctl command and then call crate itself.

1 Like
9

I was only referring to the problem mentioned with the sysctl -w vm.max_map_count=212144 command. For me, the error occurs with the docker image of sonarqube. What exactly do I have to do here to be able to solve the problem sustainably/persistently?