# Combining Balena and Azure IoT Edge

**URL:** <https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985>\
**Category:** Project help\
**Created:** [January 14, 2019, 9:46pm UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985 "2019-01-14T21:46:41Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![stenbergd](https://avatars.discourse-cdn.com/v4/letter/s/258eb7/32.png) [@stenbergd](https://forums.balena.io/u/stenbergd)\
**Post date:** [January 14, 2019, 9:46pm UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985/1 "2019-01-14T21:46:41Z")

</div>

Hi!

I’m new to the forums and quite new to Balena as well. I’ve been experimenting with Balena on a Raspberry Pi 2 and so far I’m a fan!

I’m looking in to basing my IoT infrastructure on Balena and at the same time tying in to Azure services using their Azure IoT Edge runtime. Azure IoT Edge is a container-based service building on Azure IoT Hub enabling managed deployment of Azure services as well as custom code to fleets of connected devices. It provides some neat features through their IoT SDK such as agnostic configuration of communication between services and the upstream IoT Hub in the cloud. You can read about it here: [https://docs.microsoft.com/en-us/azure/iot-edge/](https://docs.microsoft.com/en-us/azure/iot-edge/)

As nice as that may seem, it is nowhere near the Balena in terms of providing a complete solution to device management (OS updates, connectivity set up, VPN/SSH). Microsoft instructions on updating the IoT Edge runtime itself basically involves running apt commands, which would require one to access the command line through e.g. SSH to actually execute this on every device to be updated - not the most convenient approach. They use systemctl (systemd) to run the IoT Edge runtime itself.

With this in mind I thought an ideal solution could be to:

- Use Balena to access great functionality for device management, including the ability to update the OS
- Run Azure IoT Edge runtime in a container deployed and updated through Balena
- Run and manage Azure-based applications within the Azure IoT Edge runtime container

Has anyone here considered such a set up? Does this seem like a reasonable approach to using Azure IoT Edge with Balena, or am I over-complicating things?

I started out toying around with a Debian stretch base image to run IoT Edge in, installing it as recommended in the Azure documents within a Dockerfile. I made some headway and managed to run the IoT Edge daemon. However, it complains about unix:///var/run/iotedge/mgmt.sock and exits. Anyone more experienced with Docker and such that could point me in a good direction?

Best regards  
Daniel

---

<div class="post-metadata">

**Author:** ![CameronDiver](https://avatars.discourse-cdn.com/v4/letter/c/2acd7d/32.png) [@CameronDiver](https://forums.balena.io/u/CameronDiver)\
**Post date:** [January 15, 2019, 4:25pm UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985/2 "2019-01-15T16:25:50Z")

</div>

Hey @stenbergd

Glad to hear you’re enjoying balena!

We have some documentation for exactly this, located here: [https://www.balena.io/docs/learn/develop/integrations/azure/](https://www.balena.io/docs/learn/develop/integrations/azure/)

Note that a github redirect is not working on that page for some reason, and the link should be [https://github.com/balena-io-projects/balena-azure-iot-remote-monitoring](https://github.com/balena-io-projects/balena-azure-iot-remote-monitoring) for the example code. I’ve notified our docs maintainer and this should be fixed soon.

---

<div class="post-metadata">

**Author:** ![stenbergd](https://avatars.discourse-cdn.com/v4/letter/s/258eb7/32.png) [@stenbergd](https://forums.balena.io/u/stenbergd)\
**Post date:** [January 17, 2019, 9:11pm UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985/3 "2019-01-17T21:11:59Z")

</div>

Thank you for your reply Cameron.

I’m not sure we’re talking about the same thing here. What you’re referring to is integration with Azure IoT Suite (I think its being referred to as Azure IoT Accelerators nowadays), which is a preconfigured Azure IoT solution. Azure IoT Edge is quite a different concept.

I found this old Resin blog post on using Azure IoT Edge **V1** , which has since been superseeded by V2 and became generally available last year. Since V2 is quite different from V1, the blog post is not of any real use now.  
Link: [https://www.balena.io/blog/move-to-the-edge-with-azure-iot-and-resin-io/](https://www.balena.io/blog/move-to-the-edge-with-azure-iot-and-resin-io/)

It would be great with some similar guide on using Azure IoT Edge **V2** with Balena.

---

<div class="post-metadata">

**Author:** ![pcarranzav](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/pcarranzav/32/94_2.png) [@pcarranzav](https://forums.balena.io/u/pcarranzav)\
**Post date:** [January 18, 2019, 11:33pm UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985/4 "2019-01-18T23:33:55Z")

</div>

Hi @stenbergd - we weren’t aware of that IoT Edge V2 release. It looks interesting, and I think your proposed approach of using balena as a way to deploy the runtime and do device management makes a lot of sense.

We’re gonna start looking into building a demo project that does this, but it might take a while. In the meantime, maybe we can help debug your implementation? Could you post the full logs when you get that error message about `mgmt.sock`? It would also help to see your Dockerfile, and docker-compose.yml if you’re doing multicontainer, if you want to share it.

I suppose you’re either starting another container engine (docker?) inside the container, or accessing the host’s balenaEngine socket?

Cheers!

---

<div class="post-metadata">

**Author:** ![stenbergd](https://avatars.discourse-cdn.com/v4/letter/s/258eb7/32.png) [@stenbergd](https://forums.balena.io/u/stenbergd)\
**Post date:** [January 20, 2019, 5:02pm UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985/5 "2019-01-20T17:02:31Z")

</div>

Hi! No worries, I find it impossible to keep up with all the latest developments on the IoT scene myself! IoT Edge V2 became generally available last summer.

That sounds very promising! I’m no Docker wizz myself, but got some rudimentary understanding and experience of using it. I’m only doing single application from a Balena perspective, but it is as you say: it’s IoT Edge Docker/Moby engine inside Balena Docker/Moby engine 🙂

I think I got it a bit mixed up with my description earlier, it is not an mgmt.sock error (think I got this from some initial research on the issue). Here’s the printout when attempting to start the iotedged daemon.

```
root@5b1f0a4:/# iotedged -c /etc/iotedge/config.yaml
<6>2019-01-20T16:45:03Z [INFO] - Starting Azure IoT Edge Security Daemon
<6>2019-01-20T16:45:03Z [INFO] - Version - 1.0.5 (d76e0316c6f324345d77c48a83ce836d09392699)
<6>2019-01-20T16:45:03Z [INFO] - Using config file: /etc/iotedge/config.yaml
<6>2019-01-20T16:45:04Z [INFO] - Using runtime network id azure-iot-edge
<3>2019-01-20T16:45:04Z [ERR!] - The daemon could not start up successfully: Could not initialize module runtime
<3>2019-01-20T16:45:04Z [ERR!] - caused by: Could not initialize module runtime
<3>2019-01-20T16:45:04Z [ERR!] - caused by: Invalid URL "unix:///var/run/docker.sock": Socket file could not be found

```

I defined a Dockerfile.template as:

```
# Dockerfile to run Azure IoT Edge on Balena OS devices

FROM balenalib/%%BALENA_MACHINE_NAME%%-debian:stretch

RUN apt-get update && apt-get install -y \
  iptables \
 && rm -rf /var/lib/apt/lists/*

# Download and install the moby-engine
RUN curl -L https://aka.ms/moby-engine-armhf-latest -o moby_engine.deb && sudo dpkg -i ./moby_engine.deb

# Download and install the moby-cli
RUN curl -L https://aka.ms/moby-cli-armhf-latest -o moby_cli.deb && sudo dpkg -i ./moby_cli.deb

# Run apt-get fix
RUN sudo apt-get install -f

# To avoid error message
RUN echo exit 0 > /usr/sbin/policy-rc.d

# Download and install the standard libiothsm implementation
RUN curl -L https://aka.ms/libiothsm-std-linux-armhf-latest -o libiothsm-std.deb && sudo dpkg -i ./libiothsm-std.deb

# Download and install the IoT Edge Security Daemon
RUN curl -L https://aka.ms/iotedged-linux-armhf-latest -o iotedge.deb && sudo dpkg -i ./iotedge.deb

# Run apt-get fix
RUN sudo apt-get install -f

ENV INITSYSTEM on

CMD sleep infinity

```

As I’m using a Raspberry Pi 2, I’ve created the Dockerfile.template based on the installation instructions for Linux ARM32 found here: [https://docs.microsoft.com/en-us/azure/iot-edge/how-to-install-iot-edge-linux-arm](https://docs.microsoft.com/en-us/azure/iot-edge/how-to-install-iot-edge-linux-arm)

I inserted the sleep infinity at the end so that I could SSH in to the running container and play around with the iotedge CLI and daemon manually.

Within the IoT Edge Dev Tool repository, I actually found some files that may help me in my use case. I haven’t had the time to try it out yet. I also found some issues where other developers have asked about running IoT Edge in a Docker container, but it seems Microsoft are stressing that it should not be used in production scenarios (don’t know the exact reason for this…)

Here’s the files I was referring to, will come back with more info when I’ve tried it out myself: [https://github.com/Azure/iotedgedev/tree/master/docker/runtime/arm](https://github.com/Azure/iotedgedev/tree/master/docker/runtime/arm)

I also found a Yocto layer for IoT Edge which could (technically) be an option. Not sure about Balena support for adding custom Yocto layers to a base OS and I’m a complete Yocto novice.

> **[Azure/meta-iotedge](https://github.com/Azure/meta-iotedge)**
>
> Yocto layer for Azure IoT Edge. Contribute to Azure/meta-iotedge development by creating an account on GitHub.

---

<div class="post-metadata">

**Author:** ![stenbergd](https://avatars.discourse-cdn.com/v4/letter/s/258eb7/32.png) [@stenbergd](https://forums.balena.io/u/stenbergd)\
**Post date:** [February 3, 2019, 8:00am UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985/6 "2019-02-03T08:00:16Z")

</div>

I’ve put some more effort in to this trying to adapt the Microsoft instructions to Balena. However, I’m still getting the docker.sock error: `Invalid URL "unix:///var/run/docker.sock": Socket file could not be found`

Looking at the suggested launch script for the Docker container, it seems I need to volume mount the \*.sock file (from host /var/run/balena.sock ?) to the running container, forward a couple of ports, run as a network bridge, set env variables etc.

Is there an equivalent to achieving this in Balena, exposing the same functionality as docker run. Environment variables seem easy from the Balena dashboard, but what about the rest?

This is what I’m looking to do with Balena instead of Docker:

```
docker run \
    -i \
    -t \
    --rm \
    -v //var//run//docker.sock://var//run//docker.sock \
    -p 15580:15580 \
    -p 15581:15581 \
    --network bridge \
    --name iotedgec \
    -e IOT_DEVICE_CONNSTR="$IOT_DEVICE_CONNSTR" \
    iot-edge-c
```

---

<div class="post-metadata">

**Author:** ![echteler](https://avatars.discourse-cdn.com/v4/letter/e/9de053/32.png) [@echteler](https://forums.balena.io/u/echteler)\
**Post date:** [February 4, 2019, 7:56pm UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985/7 "2019-02-04T19:56:50Z")

</div>

Also Interested in this Solution, since we have a Similar request at work

---

<div class="post-metadata">

**Author:** ![lpierrat](https://avatars.discourse-cdn.com/v4/letter/l/edb3f5/32.png) [@lpierrat](https://forums.balena.io/u/lpierrat)\
**Post date:** [June 26, 2019, 11:05am UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985/8 "2019-06-26T11:05:28Z")

</div>

@stenbergd @pcarranzav do you have any news about IoT Edge v2 ? Thanks

---

<div class="post-metadata">

**Author:** ![CameronDiver](https://avatars.discourse-cdn.com/v4/letter/c/2acd7d/32.png) [@CameronDiver](https://forums.balena.io/u/CameronDiver)\
**Post date:** [June 26, 2019, 12:05pm UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985/10 "2019-06-26T12:05:54Z")

</div>

@stenbergd we have a docker compose label that you can add, that will cause the supervisor to mount the balena-engine socker into the container, and provide an address with the `DOCKER_HOST` environment variable.

@lpierrat I’m not aware on any progress on this, but I’ll reach out to the team to see if I can get anymore information for you.

---

<div class="post-metadata">

**Author:** ![stenbergd](https://avatars.discourse-cdn.com/v4/letter/s/258eb7/32.png) [@stenbergd](https://forums.balena.io/u/stenbergd)\
**Post date:** [June 28, 2019, 3:48pm UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985/14 "2019-06-28T15:48:29Z")

</div>

Unfortunately, I haven’t been able to pursue this further. My next step would probably be to to experiment with docker compose as @CameronDiver suggests. However, I don’t feel there’s a clear way forward. It seems Microsoft assumes that one should run their IoT Edge solution directly in the host OS and currently only have “production level” support for Raspbian OS for ARM32v7 platforms such as Raspberry Pi ([https://docs.microsoft.com/en-us/azure/iot-edge/support](https://docs.microsoft.com/en-us/azure/iot-edge/support))

I’m glad more people seem interested in this and I hope this fact can move things along.

---

<div class="post-metadata">

**Author:** ![banto\_78](https://avatars.discourse-cdn.com/v4/letter/b/7c8e57/32.png) [@banto\_78](https://forums.balena.io/u/banto_78)\
**Post date:** [July 1, 2019, 11:46am UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985/17 "2019-07-01T11:46:38Z")

</div>

Azure integration is a reason why i would go for Balena. I like OS management. But for me is key the integration with Azure Edge V2. Is there a doc I can follow to try it? From that tutorial i do not understand where the IoT Edge connection string to IoT Hub is configured- Please advice.

---

<div class="post-metadata">

**Author:** ![banto\_78](https://avatars.discourse-cdn.com/v4/letter/b/7c8e57/32.png) [@banto\_78](https://forums.balena.io/u/banto_78)\
**Post date:** [July 1, 2019, 2:54pm UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985/18 "2019-07-01T14:54:26Z")

</div>

Between I am trying to follow the tutorial. When I push for building i get this error:

…  
W: Failed to fetch [http://deb.debian.org/debian/dists/jessie-updates/InRelease](http://deb.debian.org/debian/dists/jessie-updates/InRelease) Unable to find expected entry ‘main/binary-armhf/Packages’ in Release file (Wrong sources.list entry or malformed file)  
[main] E: Some index files failed to download. They have been ignored, or old ones used instead.  
[main]  
[main] Removing intermediate container 4d1f13d07938  
[Info] Uploading images  
[main] The command ‘/bin/sh -c apt-get update’ returned a non-zero code: 100  
[Success] Successfully uploaded images  
[Error] Some services failed to build:  
[Error] Service: main  
[Error] Error: The command ‘/bin/sh -c apt-get update’ returned a non-zero code: 100  
[Error] Not deploying release.  
Remote build failed

Any hints?

---

<div class="post-metadata">

**Author:** ![CameronDiver](https://avatars.discourse-cdn.com/v4/letter/c/2acd7d/32.png) [@CameronDiver](https://forums.balena.io/u/CameronDiver)\
**Post date:** [July 1, 2019, 3:02pm UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985/20 "2019-07-01T15:02:44Z")

</div>

@banto_78 it looks like the problem that you’re having there is because of debian dropping the `jessie-updates` repository: [https://github.com/balena-io-library/base-images/issues/532](https://github.com/balena-io-library/base-images/issues/532)

You can fix it by updating to a newer base image, or alternatively running `RUN sed -i '/jessie-updates/{s/^/#/}' /etc/apt/sources.list` before performing an `apt update` - but I’d certainly recommend updating instead.

There currently isn’t an Azure Edge v2 integration document or tutorial at the moment.

---

<div class="post-metadata">

**Author:** ![banto\_78](https://avatars.discourse-cdn.com/v4/letter/b/7c8e57/32.png) [@banto\_78](https://forums.balena.io/u/banto_78)\
**Post date:** [July 1, 2019, 4:10pm UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985/22 "2019-07-01T16:10:53Z")

</div>

Many thanks. I think I’ve anyway resolved by updating to:

FROM balenalib/raspberrypi3-python:3.6-jessie-build

I have built and deployed. However I do not see any message uploaded in Azure IoT Hub…and by the way where can i find the `image.png’ used in the script? This is in the log view:

01.07.19 18:05:10 (+0200) main convert: improper image header `image.png' @ error/png.c/ReadPNGImage/3943. 01.07.19 18:05:10 (+0200) main convert: no images defined`rotated.png’ @ error/convert.c/ConvertImageCommand/3210.  
01.07.19 18:05:11 (+0200) main using “DejaVu Sans Mono-16”, pixelsize=16.67 file=/usr/share/fonts/truetype/dejavu/DejaVuSansMono.ttf

---

<div class="post-metadata">

**Author:** ![CameronDiver](https://avatars.discourse-cdn.com/v4/letter/c/2acd7d/32.png) [@CameronDiver](https://forums.balena.io/u/CameronDiver)\
**Post date:** [July 1, 2019, 4:41pm UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985/31 "2019-07-01T16:41:30Z")

</div>

@banto_78 That is just any image, and is used as a small example, I’d just add something simple yourself.

This also may be of interest, but I’ve not used it, so YMMV [https://github.com/maxtheaviator/resin-azure-iotedge-multicontainer](https://github.com/maxtheaviator/resin-azure-iotedge-multicontainer)

---

<div class="post-metadata">

**Author:** ![banto\_78](https://avatars.discourse-cdn.com/v4/letter/b/7c8e57/32.png) [@banto\_78](https://forums.balena.io/u/banto_78)\
**Post date:** [July 1, 2019, 6:59pm UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985/33 "2019-07-01T18:59:44Z")

</div>

ok. The script start.sh has different bugs. For example " replaced.json" it is produced as empty file. Any idea?

---

<div class="post-metadata">

**Author:** ![CameronDiver](https://avatars.discourse-cdn.com/v4/letter/c/2acd7d/32.png) [@CameronDiver](https://forums.balena.io/u/CameronDiver)\
**Post date:** [July 1, 2019, 7:03pm UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985/34 "2019-07-01T19:03:02Z")

</div>

My assumption would be that the file it reads this from `cat /usr/src/app/iot-edge/samples/simulated_device_cloud_upload/src/simulated_device_cloud_upload_lin.json` is empty, can you confirm?

---

<div class="post-metadata">

**Author:** ![banto\_78](https://avatars.discourse-cdn.com/v4/letter/b/7c8e57/32.png) [@banto\_78](https://forums.balena.io/u/banto_78)\
**Post date:** [July 1, 2019, 7:16pm UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985/35 "2019-07-01T19:16:26Z")

</div>

it is not empty that file. I can prove that the result of this:

```auto
echo "$J_FILE" | \  
  jq '.modules[0].args.IoTHubName="<IoTHubName>"' | \
  jq '.modules[0].args.IoTHubSuffix="azure-devices.net"' | \
  jq '.modules[0].args.Transport="AMQP"' | \
  jq '.modules[1].args[0].deviceId="'$DEVICE1'"' | \
  jq '.modules[1].args[0].deviceKey="'$DEVICE1KEY'"' | \
  jq '.modules[1].args[1].deviceId="'$DEVICE2'"' | \
  jq '.modules[1].args[1].deviceKey="'$DEVICE2KEY'"' \
  > replaced.json

```

is an empty file!

---

<div class="post-metadata">

**Author:** ![dt-rush](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/dt-rush/32/1405_2.png) [@dt-rush](https://forums.balena.io/u/dt-rush)\
**Post date:** [July 2, 2019, 2:54am UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985/37 "2019-07-02T02:54:17Z")

</div>

Hey, so it turns out that the repo structure for the [iot-edge](https://github.com/Azure/iot-edge-v1) example app from Azure has changed since the article’s script was written. This is why the files show up as non-existent. You should replace instances of the `iot-edge` directory with `iot-edge/v1` in the `Dockerfile` and `start.sh`.

---

<div class="post-metadata">

**Author:** ![banto\_78](https://avatars.discourse-cdn.com/v4/letter/b/7c8e57/32.png) [@banto\_78](https://forums.balena.io/u/banto_78)\
**Post date:** [July 2, 2019, 7:26am UTC](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985/41 "2019-07-02T07:26:09Z")

</div>

Yes, I already found out the issue with “V1” in the path. Still, that script produces a wrong (not replacing the values) or empty file. I am wondering because the structure of the JSON ([https://github.com/Azure/iot-edge-v1/blob/master/v1/samples/simulated\_device\_cloud\_upload/src/simulated\_device\_cloud\_upload\_lin.json](https://github.com/Azure/iot-edge-v1/blob/master/v1/samples/simulated_device_cloud_upload/src/simulated_device_cloud_upload_lin.json) ) should be fine… Any thoughts?

[Next page](https://forums.balena.io/t/combining-balena-and-azure-iot-edge/4985.md?page=2)