balenaos-in-container not connecting to s3 registry on 1st boot

openBalena
1

I have an issue with balenaos-in-container connecting the to the s3 registry on out openBalena server.

I have perfomed the following stesp:

  • cloned the balenaos-in-container repo down onto a Ubuntu 20.04 EC2 instance

  • created a config.json file for my app using the balena cli

  • used docker-compose -p up -d to create the devices.

This pulls down the 2.68.1_rev1.dev-intel-nuc image from resinos dockerhub and builds a new local image for the virtual device…this all works as expected.

When the device starts for the 1st time there are errors in the balena-supervisor logs where it can’t download the containers required for the application. It is showing an x509: certificate signed by unknown authority error.

I have checked in the main container and also in the supervisor container and the balenaRootCA.crt is there and and all env variables are set.

If I restart the device by doing a docker-compose down and then docker-compose -p up -d again the device boots and the downloads the containers without issue…every time.

I can’t find the reason why it wont do it the 1st time. I originally though it might be something to do with docker on the host so I added the ca cert for openbalena there as well but it hasn’t helped.

See attached logs for balena supervisor on 1st boot and 2nd boots and also the docker-compose file.

Has anyone seen this before or can give me any points why it can;t download from the registry on the 1st boot. We have physical devices using the same config.json and they don’t have any issues.

Thanks,
Gerard.

virt_dev_302_2nd_boot.txt (5.1 KB)
virt_dev_302_1st_boot.txt (9.5 KB)
docker-compose.txt (848 Bytes)

2

Some additional info - I just created 20 virtual devices on a new EC2 instance using a script. 17 of them have the x509 cert issue and 3 have successfully download the containers from the registry :confused:

3

Hello @g.corrigan sorry for the late reply! Could you please send us a last update? Did you find why only 3 devices were successfully downloading the containers from the registry?

Let’s stay connected

4

HI @mpous, no problem. I have also been busy on other things so haven’t had time to do too much more on it.

I did do one test where I trusted the cert in the balenaos-in-container docker image and that seemed to work. I think it is a docker in docker issue but need to do some further work on it.

I’ll get to it again at the weekend and post an update.

1 Like
5

Thank you for your update @g.corrigan

Keep us updated if you succeed and if you find issues let us know as well!