ble-lighthouse: balenaLabs Residency project - build logs

As a part of my balenaLabs residency, I am building a project that will allow me to deploy a fleet of centrally managed (from balenaCloud) bluetooth low energy beacons. I am calling this project :
ble-lighthouse as beacons works similar to lighthouse, putting out signal without waiting for anyone to respond/connect/use etc.

I am planning to create beacons that can broadcast

  • Fixed data (numbers/strings/hex)
  • URLs
  • Sensor Data (temp, humidity etc)
1 Like

I will explain my motivation and intent to do this project as a blog post later and here I’ll just post the build logs and updates as I make progress

First thing first, let’s see how other companies are using ble beacons. A great way I find to do this is download a beacon scanner app and see what’s nearby. I recommend nRF Connect app by Nordic Semiconductor available for android and ios. It’s like a Swiss Army Knife app for ble scanning. A quick scan locally to my surprise shows me a few beacons in my vicinity which i was totally not expecting. It shows Manufacturer is Microsoft and it’s broadcasting some sort of data which I am not aware of and also I cannot figure out this device, it probably belong to someone else around me. But this gave me some idea of how beacon looks like:

2 Likes

**Let’s get to the fun part **

I am choosing debian as my base image and on Linux, bluez is the de-facto utility when it comes to managing bluetooth radios. I’ll prototype on a raspberry pi 4 board because that’s what I have at the moment but I want this project to work on Rpi zero w / w 2 , so we can just throw away some beacons here and there without thinking much about the added cost.

With bluez comes various tools like hcitool, gattool which can be used to create bt peripherals and centrals with GATT profiles, I’ve worked on few before so i know certain bits and pieces but consider reading bluez docs and trying some examples to get a hang of it.

When it comes to creating ble peripherals hcitool is your go-to friend. After reading around I figured that the main characteristics of beacons is they donot connect to ble centrals, hence to achieve that we need to first activate the radio and then put it in non connection mode.

Raspberry pi comes with a ble radio (hci0) which we gonna use, this should work with any other board with balenaOS support ble radio (internal or external)

  • bring the radio up : hcitool hci0 up
  • put the radio in no-connect state hciconfig hci0 leadv 3
  • we also don’t need to scan for other devices so we can turn off the scan feature using hciconfig hci0 noscan

The next thing is mostly to figure out packet structure, it looks like bunch of hex bytes, some of them are functioning flags that some of them contains data type and then actual data.

for ex: bluetooth SIG (special interest group) provides manufacturer ID to it’s partner like Microsoft and ST Microelectronics has their unique ID, using which the scanners can figure out who made these devices/radios. All this can be defined and broadcasted using same hcitool

A typical command might look something like:

hcitool -i hci0 cmd 0x08 0x0008 1E 02 01 1A 1A FF 30 00 02 15 63 6F 3F 8F 64 91 4B EE 95 F7 D8 CC 64 A8 63 B5 00 00 00 00 C8

Fortunately, instead of figuring out the hex byte codes for each sort of data, I found a neat little library which does the job. https://github.com/forksociety/PyBeacon , unfortunately it’s not in active development and is quite bloated with features I don’t need for this project, like a beacon scanner, hence I decided to fork and library and modify it accordingly. https://github.com/iayanpahwa/PyBeacon

The library definitely helped with URL and UID broadcast but I still need to figure out way to broadcast RAW data values using which I will send sensor data values of say temperature and humidity sensor .

the hcitool utility from bluez is quite powerful and allows you to define what sort of functionality you want from the radio, for my sensor broadcast part, I’ll be using it directly and maybe once when everything starts working I’ll add that part to my fork of PyBeacon library.

the command for broadcasting as beacon is sudo hcitool -i hci0 cmd 0x08 0x0008 followed by certain structures called AD structures, will require 2 for this purpose, 1 for configuration which shall include manufacturer ID (from SIG), the other AD structure includes UID, the data following it and Tx power.

Hi, how can i create a multi-container/services device using the ble as one of the containers?

Hi @nl-limburg , you might want to check balena bluetooth block which essentially takes away all the hard bluetooth stuff and make it easy to establish connection with bluetooth peripherals if that’s your use case. Check here and let me know if it works for you :slight_smile:

@iayanpahwa,
Thank you for the link, I tested it, but it is not working for my use case.
My device has several containers running, and I would like to add an extra a container that makes the device also act like a bleutooth beacon. No other bleutooth functionality is needed.

Hey there @nl-limburg ,

You can clone the repo from @iayanpahwa 's project here: iayanpahwa/ble-lighthouse: create ble beacons on single board linux computer using balena.io (github.com)

Place it in a folder alongside your other containers, then you can add that to your docker-compose file, and it will be a service that creates a BLE beacon. There’s some guidance on this type of thing here:
Two projects, one device: turn your Raspberry Pi into a multitool! (balena.io)

Hope that helps,
Phil

Hi Phil,
Thx for the tip, i already tried that in the first place. But that gave some errors when starting the container. Bleutooth apperently needs some specific setting for it to be used in a container.
After some googling I changed the docker-compose.yml to:

version: '2'

services:

# Name of service in Dashboard  

  clean_set:
    build: ./clean_set
    
    network_mode: host  # allows us to handle network adapters natively
    cap_add:
      - NET_ADMIN  # allows it to “Perform various network-related operations.”
    labels:
      io.balena.features.dbus: 1
    # This will ensure that the host propagates udev events into the container, enabling us to manipulate the device from within it.
    privileged: true
    environment:
      # Set Timezone
      - TZ=Europe/Amsterdam

And the dockertemplate to:

FROM balenalib/%%BALENA_MACHINE_NAME%%-debian-python:3.8-buster-run as builder

#This will ensure that the host propagates udev events into the container, enabling us to manipulate the device from within it.
ENV UDEV=on

# Set our working directory
WORKDIR /usr/src/app

# Install required baseline packages and cleanup
RUN apt-get clean && apt-get update && apt-get install -y --no-install-recommends \
    rfkill \
	&& apt-get autoremove


# Installing the app
# ==================
RUN install_packages git bluetooth bluez bluez-firmware bluez-hcidump libbluetooth-dev libudev-dev  

RUN python3 -m pip install --upgrade pip

RUN python3 -m pip install enum34

RUN git clone https://github.com/iayanpahwa/PyBeacon.git
RUN cd PyBeacon && python3 setup.py install

# env variables:
# ADVERTISE : TRUE / FALSE
# BEACON_TYPE : URL / UID (16 bytes) / RAW (4 bytes)

# Set default when creating service
ENV ADVERTISE='TRUE'
ENV BEACON_TYPE='URL'
ENV URL='https://hub.balena.io'
ENV UID='01234567890123456789012345678901'

COPY ./code/main.sh main.sh
CMD [ "/bin/bash", "main.sh" ]

I left main.sh unchanged.
Now the first error is gone and the service starts, however i get an other one:
(clean_set is the BLE service)

04.03.22 14:29:20 (+0100)  clean_set  TRUE
04.03.22 14:29:20 (+0100)  clean_set  URL
04.03.22 14:29:20 (+0100)  clean_set  https://hub.balena.io
04.03.22 14:29:20 (+0100)  clean_set  01234567890123456789012345678901
04.03.22 14:29:20 (+0100)  clean_set  
04.03.22 14:29:20 (+0100)  clean_set  ======================
04.03.22 14:29:20 (+0100)  clean_set  Can't set advertise mode on hci0: Network is down (100)
04.03.22 14:29:20 (+0100)  clean_set  Can't set scan mode on hci0: Network is down (100)
04.03.22 14:29:22 (+0100)  clean_set  Advertising: url : https://hub.balena.io

When i check from terminal:

root@7e91c1b:/usr/src/app# rfkill list
0: hci0: Bluetooth
        Soft blocked: no
        Hard blocked: no
1: phy0: Wireless LAN
        Soft blocked: no
        Hard blocked: no

So bluetooth should be working. However:

root@7e91c1b:/usr/src/app# hcitool dev
Devices

So apperentely there are no bluetooth devices.
And now i’m lost.

Any thoughts?

Hmmm @iayanpahwa - any ideas?

Tried it first on Raspberry Pi Zero 2 W with the result as stated above.
Did some additional tests; it works when running on raspberry pi 4 model B.

Interesting, Everything is looking fine w.r.t to the docker-compose file, privilege mode is also set, and network mode is host but the container can’t access the BT interface. I’m not sure what should be causing it but I might try it at my end today. Meanwhile this stackoverflow thread has some pointers which you can try such as making sure host-os is not using the bluetooth and dbus mode with bluez is enabled on the container . I’m not sure how could it be specific to raspberry pi zero 2 w while it works on Rpi 4 :thinking: